Efficiently Streaming Data into Medallion Architecture with Apache Hudi

Discover how to efficiently stream data into your Medallion Architecture using Apache Hudi in this 42-minute conference talk from Confluent. Learn about the challenges of building a low-latency medallion architecture and how Apache Hudi, a transactional data lake platform, can address these issues. Explore the power of Hudi's new record-level index for faster upserts, and its database-style change data capture feature for improved incremental processing. Gain insights into efficiently graduating raw data through bronze, silver, and gold tables while avoiding computationally expensive operations. Walk away with knowledge on overcoming current technology limitations in lakehouses, understanding Hudi's record index and incremental updates, and leveraging new features to unlock efficient data processing on the lake. Perfect for data engineers and architects looking to optimize their data streaming and processing workflows in a medallion architecture.