The Data Lakehouse for Streaming Data - A Talk for Everyone Who Loves Data

Explore the technical aspects of Data Lakehouses and their impact on streaming data in this comprehensive talk. Delve into the fusion of data lakes and data warehouses, examining how open-source projects like Delta Lake enhance data management with ACID transactions, schema enforcement, and efficient metadata handling. Discover the capabilities of open-source solutions for streaming data and gain insights into future improvements. Investigate streaming data analysis, machine learning on the lakehouse, and Project Lightspeed's potential for low-latency Apache Spark Structured Streaming. Witness a live demonstration of Twitter stream ingestion using a declarative, auto-scaling data pipeline for sentiment analysis with Hugging Face. Ideal for data architects, engineers, and practitioners interested in open-source and cloud services, this presentation offers a deep dive into the Databricks Lakehouse and its practical applications.