Shift Left: Rethinking Data Management with Streams and Tables

Learn about the Shift Left approach to data management in this 16-minute video from Confluent. Explore how DataStreams, Change Data Capture, FlinkSQL, and Tableflow can address challenges in multi-hop and medallion architectures using batch pipelines. Discover how shifting data preparation, cleaning, and schemas to the point of data creation enables building fresh, trustworthy datasets as streams for operational use cases or Apache Iceberg tables for analytical use cases. Gain insights into building data products closer to the source, unlocking both near-real-time and batch-based use cases. Follow the video's progression from an introduction to multi-hop and medallion architectures, through the problems with multi-hop, to implementing Shift Left with streams and tables. Conclude with discussions on plugging in data, data evolution, and consistency.