Data Pipelines Courses and Certifications

Establishing Data Infrastructure

Learn to build scalable data infrastructure, identify data producers/consumers, design effective pipelines, and create a comprehensive data strategy for your business needs.

• Udacity
• 3 weeks, 5-6 hours a week
• On-Demand
• Paid Course

ETL and ELT in Python

25 ratings at DataCamp

Learn to build effective, performant, and reliable data pipelines using Extract, Transform, and Load principles.

• DataCamp
• 4 hours
• On-Demand
• Free Trial Available

Data Science on Google Cloud Platform: Building Data Pipelines

126 ratings at LinkedIn Learning

Learn how to design and build big data pipelines on Google Cloud Platform.

• LinkedIn Learning
• 1 hour 8 minutes
• On-Demand
• Free Trial Available

Databricks

Generative Hyperloop Design - Managing Massively Scaled Simulations for Demand Modeling

Explore large-scale simulation framework for Hyperloop design, utilizing cloud computing, data pipelines, and machine learning to address key technical and business questions for safe and efficient mass transportation.

• YouTube
• 26 minutes
• On-Demand
• Free Video

Smart Data Pipelines to Azure - Ingesting and Migrating Data the DataOps Way

Explore StreamSets' DataOps platform for optimizing data ingestion and migration to Azure. Learn about smart data pipelines, integration with Azure services, and how to speed up cloud adoption using modern data engineering practices.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Open Data Science

Reliable Pipelines and High Quality Data Without the Toil - Kyle Kirwan

Explore data observability's power in enhancing pipeline quality, reducing maintenance toil, and optimizing performance. Learn key components: metrics, data, lineage, and alerts for effective data management.

• YouTube
• 26 minutes
• On-Demand
• Free Video

The Future of Data Pipelines with Atomic Wasm Transformations - Evolving Role of Data Engineers

Explore atomic Wasm transformations in data pipelines and their impact on data engineering roles, focusing on innovative approaches to data processing and workflow optimization.

• YouTube
• 44 minutes
• On-Demand
• Free Video

Building an Open Source Streaming Analytics Stack with Kafka and Druid

Learn to build a robust streaming analytics stack using Kafka and Druid for real-time data processing, flexible querying, and maintaining data integrity in high-volume environments.

• YouTube
• 41 minutes
• On-Demand
• Free Video

Deploying Fast Data Pipelines on Mesos and DC/OS

Overview of Fast Data systems, their deployment on Mesos and DC/OS, and their impact on Big Data processing. Learn about the evolving landscape of data pipelines and efficient operational strategies.

• YouTube
• 39 minutes
• On-Demand
• Free Video

Open Data Science

Orchestrating Data Assets Instead of Tasks, With Dagster - Sandy Ryza

Explore data orchestration with Dagster, focusing on managing data assets over tasks. Learn about pipeline development, testing, deployment, and monitoring in data engineering and machine learning.

• YouTube
• 31 minutes
• On-Demand
• Free Video

Databricks

Observability for Data Pipelines with OpenLineage

Explore data pipeline observability using OpenLineage, enhancing reliability and auditability. Learn how metadata collection enables understanding of data flow and dependencies across teams and technologies.

• YouTube
• 24 minutes
• On-Demand
• Free Video

Databricks

Dumb-Proofing Data Pipelines: Techniques for Configurable and Maintainable ETL - Databricks

Techniques to create configurable, maintainable data pipelines. Learn to externalize configurations, validate inputs, and leverage Scala features for robust, easily deployable ETL processes.

• YouTube
• 22 minutes
• On-Demand
• Free Video

Databricks

Code Once, Use Often - Declarative Data Pipelines

Learn to build efficient, reusable data pipelines using declarative approaches. Explore techniques for reducing food waste through data-driven solutions and automation in this developer-focused session.

• YouTube
• 28 minutes
• On-Demand
• Free Video

PyCon US

Testing Data Pipelines - Techniques for Ensuring Data Flow Integrity

Explore effective techniques for testing data pipelines, ensuring smooth data flow and quick problem resolution. Learn about unit, integration, and end-to-end tests, plus unique methods like data snapshot testing.

• YouTube
• 25 minutes
• On-Demand
• Free Video

Database Native Support for CDC Streaming Pipeline

Explore a database-native solution for Cassandra's CDC logs, addressing common challenges and proposing an improved architecture based on industry insights and managed CDC solutions.

• YouTube
• 24 minutes
• On-Demand
• Free Video