Scaling Up Machine Learning Without Tears - Programming Languages' Role

Explore the challenges and solutions in scaling machine learning workloads across thousands of accelerators in this OOPSLA 2023 conference talk by Dimitrios Vytiniotis. Delve into the importance of partitioning techniques in ML systems and the programming efforts required to achieve high hardware utilization. Discover how key concepts from programming languages, such as domain-specific abstractions, types, program transformations, and abstract interpretation, address these challenges. Learn about the unique aspects of ML programs and accelerators that both simplify certain problems and introduce new complexities compared to general-purpose languages. Gain insights into the future of ML systems design and the role of programming languages in facilitating efficient distribution and performance prediction of ML workloads.