Data Lineage Courses and Certifications

Automating Airflow Backfills with Marquez

Learn to automate Airflow backfills using Marquez, enhancing data pipeline efficiency and reliability in complex workflows.

• YouTube
• 35 minutes
• On-Demand
• Free Video

CNCF [Cloud Native Computing Foundation]

MetaOps - Metadata Operations for End-to-End Data and Machine Learning Platforms

Explore metadata management challenges in large-scale ML systems, best practices, and solutions for lineage, auditability, and reproducibility in heterogeneous end-to-end platforms.

• YouTube
• 37 minutes
• On-Demand
• Free Video

PASS Data Community Summit

Unified Data Governance with Azure Purview

Explore Azure Purview's innovative approach to unified data governance, including automated discovery, lineage extraction, and centralized access control for cloud-scale data estates across various systems.

• YouTube
• 43 minutes
• On-Demand
• Conference Talk

Drive a Data Culture for Your Business on the Secure Foundation of Azure & Power BI

Empower data consumers with unified governance, automated discovery, and end-to-end lineage. Learn to create a holistic map of your data landscape while ensuring security and compliance.

• YouTube
• 26 minutes
• On-Demand
• Conference Talk

Databricks

Data Discovery at Databricks with Amundsen - Improving Productivity and Trust

Explore how Databricks leverages Amundsen for efficient data discovery, improving productivity by surfacing relevant datasets, SQL dashboards, and metadata programmatically.

• YouTube
• 28 minutes
• On-Demand
• Free Video

Databricks

Data Lakehouse Symposium - Evolution of Data Architecture

Explore data lakehouse architecture with industry experts, covering its evolution, comparison to data warehouses, and impact on business value. Gain insights into handling diverse data types and real-world applications.

• YouTube
• 2 hours 15 minutes
• On-Demand
• Free Video

PASS Data Community Summit

Building Modern Data Pipelines with Spark on Azure HDInsight

Explore common patterns for building end-to-end data analytics pipelines using Apache Spark on Azure HDInsight, covering architecture, integration points, and various pipeline options.

• YouTube
• 1 hour 18 minutes
• On-Demand
• Conference Talk

CNCF [Cloud Native Computing Foundation]

The Data Pipelines Behind Forest Carbon Credits - Why Pachama Uses Flyte to Orchestrate Workflows

Explore how Pachama uses Flyte to manage complex data and ML pipelines for transparent forest carbon credit computation, including workload management, configuration, and lessons learned.

• YouTube
• 34 minutes
• On-Demand
• Free Video

Confluent

Data Democratization: Importance, Tools, and Real-World Applications

Explore data democratization's importance in modern systems. Learn how making data accessible to all can drive better business decisions, foster collaboration, and provide valuable insights across organizations.

• YouTube
• 47 minutes
• On-Demand
• Free Video

Data Council

Data Contracts: The Next Frontier in Data Management

Explore data contracts with experts, gaining practical advice and key strategies for success. Learn to navigate this emerging field and enhance your data management skills.

• YouTube
• 59 minutes
• On-Demand
• Free Video

Data Council

Ten Years of Building Open Source Standards: From Parquet to Arrow to OpenLineage

Explore the journey of building open-source data standards, from Apache Parquet to Arrow to OpenLineage. Gain insights into successful project ideation, growth, and ecosystem impact in data management.

• YouTube
• 35 minutes
• On-Demand
• Free Video

Data Council

Feed the Alligators With the Lights On - How Data Engineers Can See Who Really Uses Data

Explore data catalog creation, metadata utilization, and self-serve data culture. Learn to navigate large datasets, distinguish trustworthy data, and leverage query logs for effective data management.

• YouTube
• 24 minutes
• On-Demand
• Free Video

The ASF

Modern Data Orchestration: Best Practices and Real-World Use Cases

Elevate your data pipeline game with real-world tricks and best practices. Explore patterns, avoid dependencies, and integrate fundamental concepts for efficient data orchestration.

• YouTube
• 26 minutes
• On-Demand
• Free Video

Data Council

Bridging the Gap Between Batch and Real-Time with Mixed-Latency Pipelines

Explore mixed-latency data platforms, manage data freshness, and tackle pipeline bottlenecks. Learn to merge batch and low-latency processing while maintaining explicit lineage for reliable data products.

• YouTube
• 13 minutes
• On-Demand
• Free Video

Data Council

Modern Data Management - Setting Up Your Data Team for Success

Discover best practices for implementing data discovery, enabling searchable knowledge for all. Learn to set up a data discovery portal as a single source of truth for mature data management.

• YouTube
• 16 minutes
• On-Demand
• Free Video