Scaling Machine Learning with Spark

Explore a comprehensive conference talk on scaling machine learning solutions with Apache Spark. Dive into the practical insights shared by Adi Polak and Holden Karau as they discuss their book "Scaling Machine Learning with Spark." Learn about the Apache Spark ecosystem, MLlib, MLflow, TensorFlow, and PyTorch for building end-to-end distributed ML workflows. Discover how to manage the ML lifecycle, perform data preprocessing, explore feature engineering, and train models using MLlib. Gain valuable knowledge on combining Spark with deep learning, working with distributed TensorFlow, and scaling machine learning with PyTorch. Understand the challenges and trade-offs in distributed ML, and explore the intersection of ML and data engineering.