Productionizing Big Data - Stories from the Trenches

Explore real-world challenges and solutions in big data production environments through this 40-minute conference talk from Scala Days Conferences. Uncover the mysteries behind data pipeline anomalies, learn effective approaches for troubleshooting issues, and discover strategies to prevent future problems. Gain valuable insights into handling production-specific challenges such as data evolution, anomalies, error correction in historical data, and real-time data processing. Drawing from experiences in the AdTech industry, delve into practical scenarios that showcase the complexities of managing big data systems beyond textbook knowledge. Equip yourself with essential skills for navigating the unpredictable landscape of production data pipelines and learn how to bridge the gap between ideal system expectations and real-world operational demands.