Discover four powerful techniques to enhance your Airflow pipelines in this DevOps conference talk. Learn how to create typical DAGs using CSV files and sharable configs, autogenerate DAGs based on Airflow entities, and transform your pipelines into event-driven systems. Gain practical insights from Aliaksandr Sheliutsin's presentation at Conf42 DevOps 2024, covering topics such as CSV-based DAG creation, config-driven DAG generation, entity-based DAG autogeneration, and implementing event-driven pipelines in Airflow.
Overview
Syllabus
intro
preamble
about aliaksandr
1: you can create typical dags based on a csv file
2: you can create typical dags based on a sharable config
3: autogenerate dags based on the airflow entities
4: make your airflow pipeline event-driven
have questions?
Taught by
Conf42
