Building Robust Data Pipelines for Modern Data Engineering - End-to-End Project

Building Robust Data Pipelines for Modern Data Engineering - End-to-End Project

CodeWithYu via YouTube Direct link

Introduction

1 of 18

1 of 18

Introduction

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Building Robust Data Pipelines for Modern Data Engineering - End-to-End Project

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Introduction
  2. 2 System Architecture
  3. 3 Creating resource groups on Azure
  4. 4 Setting up the medallion architecture storage account
  5. 5 Setting up Azure Data Factory
  6. 6 Azure Key Vault setup for secrets
  7. 7 Azure database with automatic data population
  8. 8 Azure Data Factory pipeline orchestration
  9. 9 Setting up Databricks
  10. 10 Azure Databricks Secret Scope and Key Vault
  11. 11 Verifying Databricks - Key Vault - Secret Scope Integration
  12. 12 Azure Data Factory - Databricks Integration
  13. 13 DBT Setup
  14. 14 DBT Configuration with Azure Databricks
  15. 15 DBT Snapshots with Azure Databricks and ADLS Gen2
  16. 16 DBT Data Marts with Azure Databricks and ADLS Gen2
  17. 17 DBT Documentation
  18. 18 Outro

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.