Explore a comprehensive series of tutorials and hands-on videos covering various aspects of Apache Airflow. Dive into topics such as the ExternalTaskSensor, defining variables through environment variables, fixing dependencies errors, and scaling with Kubernetes Executors. Learn about password authentication, filtering by owner, variables, macros, and templating. Discover how to implement metrics for monitoring, use timezones, and work with the DockerOperator. Build and run your first Airflow Docker image, use the Sequential Executor, and interact with PostgreSQL using Hooks. Master DAG creation, understand SubDAGs, and explore different types of Operators. Gain insights into storing logs in AWS S3, running Airflow with KubernetesExecutor on multi-node clusters, and integrating with AWS RDS. Delve into advanced concepts like XCom, BranchPythonOperator, TaskGroups, and TriggerDagRunOperator. Set up Airflow 2.0 with Docker, configure LDAP, and get started with the Airflow Helm Chart and Sensors.
Overview
Syllabus
Apache Airflow: The ExternalTaskSensor demystified.
Apache Airflow: How to define variables through environment variables.
Apache Airflow: How to fix dependencies errors.
Scaling Apache Airflow with the Kubernetes Executors.
Password authentication and filtering by owner in Apache Airflow.
Variables, Macros and Templating in Apache Airflow.
Introduction to metrics for monitoring Apache Airflow.
How to use timezones in Apache Airflow.
Apache Airflow | How to Use the DockerOperator with Airflow.
Apache Airflow | Building And Running Your First Airflow Docker Image.
Apache Airflow | Running Airflow Using Sequential Executor With Docker.
Apache Airflow | Getting Results From PostgreSQL Using Hooks.
Apache Airflow | How to Create an Operator.
Apache Airflow | Coding Your First Dag.
Apache Airflow | What Is a DAG?.
Apache Airflow | Minimising DAGs With SubDAGs.
Apache Airflow | How to Use Operators.
Apache Airflow | What Is An Operator.
Apache Airflow: Storing logs in AWS S3.
Apache Airflow: Storing logs in AWS S3.
Airflow 2.0: What to expect?.
Running Apache Airflow with the KubernetesExecutor on a multi-node Kubernetes cluster locally.
Running Apache Airflow with the KubernetesExecutor on a multi-node Kubernetes cluster locally.
Apache Airflow with AWS RDS.
Airflow XCom for Beginners - All you have to know in 10 mins.
The Airflow BranchPythonOperator for Beginners in 10 mins.
Apache Airflow: Adios SubDAGs! Welcome TaskGroups!.
The TriggerDagRunOperator in Airflow! Create DAG dependencies at ease.
Running Airflow 2.0 with Docker in 5 mins.
Airflow DAG: Coding your first DAG for Beginners.
Big data in Airflow? Discover the XCom Backends with AWS S3!.
Configure Airflow 2.0 with LDAP in 15 mins.
Airflow Helm Chart : Quick Start For Beginners in 10mins.
Airflow Sensors : Get started in 10 mins.
Taught by
Marc Lamberti
