Learn how Delta Live Tables (DLT) simplifies building and managing reliable batch and streaming data pipelines for high-quality data delivery on the Databricks Lakehouse Platform in this 37-minute conference talk from DataGalaxy Tech Summit 2023. Discover the advantages of declarative pipeline development, automatic data testing, and comprehensive monitoring capabilities for data engineering teams. Explore the integration with Databricks SQL's serverless data warehouse for executing SQL and BI applications at scale, while following a practical demonstration covering topics like data ingestion, declarative APIs, change data capture, data quality assurance, automated operations, orchestration, autoscaling, and interactive query functionality.
Next Generation Data Products Build on Databricks Lakehouse - Delta Live Tables and SQL Warehousing
DataGalaxy via YouTube
Overview
Syllabus
Intro
Databricks Lakehouse
Data Engineering Ecosystem
How it can help
Ingest Data
Declarative API
Change Data Capture
Ensure Data Quality
Delta Live Table
Automated Operations
Orchestration
Autoscaling
Shell
Demo
Data Warehouse
Rest API
Interactive Query
Serverless SQL
SQL Compute
SQL Demo
Taught by
DataGalaxy
Related Courses
-
Data Engineering with Databricks
-
Delta Live Tables: Building Reliable Data Pipelines with Databricks
-
Declarative ETL Pipelines with Delta Live Tables - Modern Software Engineering for Data Analysts and Engineers
-
Delta Live Tables: Modern Software Engineering for ETL Pipelines
-
Data Engineering with Databricks - Leveraging the Lakehouse Platform for ETL Pipelines
-
Introduction to Databricks Delta Live Tables
