Learn how to automate data pipelines using Python and GitHub Actions in this comprehensive video tutorial. Explore two methods for automation: using orchestration tools and combining Python with triggers. Dive into a practical example of automating an ETL (Extract, Transform, Load) pipeline, covering the entire process from creating the Python script to setting up a GitHub repository and configuring GitHub Actions. Discover how to create workflow YAML files, add repository secrets, and commit changes. Gain insights into building a full-stack data science project, with additional resources provided for further learning and implementation.
Overview
Syllabus
Intro -
Motivation -
2 Ways to Automate -
Way 1: Orchestration Tool -
Way 2: Python + Triggers -
GitHub Actions -
Example Code: Automating ETL Pipeline -
1 Create ETL Python Script -
2 Create GitHub Repo -
3 Create Workflow .yml File -
4 Add Repo Secrets -
5 Commit and Push -
Final ML App -
Taught by
Shaw Talebi
