Completed
How to make your data pipelines dumb-proof?
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Dumb-Proofing Data Pipelines: Techniques for Configurable and Maintainable ETL - Databricks
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Why make your data pipelines dumb-proof?
- 3 How to make your data pipelines dumb-proof?
- 4 Fixing Hard coded Data Pipelines
- 5 Parameters & Input Validation
- 6 Externalizing Configuration
- 7 Configuration in JSON Format
- 8 Optimized Configuration in HOCON format
- 9 Readable and maintainable Configuration
- 10 Configuration Library
- 11 Refactor Code - Loading and Parsing Configuration
- 12 Boilerplate free configuration code
- 13 Sample Code
- 14 Summary
