Explore the evolution of efficient and robust AutoML systems in this 32-minute conference talk. Delve into the technical methods behind recent advancements, starting with a brief recap of early systems like Auto-WEKA and Auto-sklearn. Examine the next generation of AutoML, including Auto-learn 2.0 and Auto-PyTorch, focusing on components that significantly improve efficiency. Learn about practical considerations, multi-fidelity optimization, portfolio construction, and automated policy selection. Discover how these approaches outperform Auto-sklearn 1.0 and gain insights into regularization techniques for state-of-the-art deep learning on tabular data.
Evolution of Efficient and Robust AutoML Systems
Overview
Syllabus
Intro
Traditional Machine Learning and AutoML Traditional Ml practice
Focus of This Talk: Progression of Our Ready-To-Use AutoML Systems
Decades of Work On Predecessors of AutoML European projects
Auto-WEKA
Practical Improvement 1: Intermittent Results Retrieval
Practical Improvement 2: Reduced Search Space
Multi-Fidelity Optimization: General Overview
Multi-Fidelity Optimization: Methods
Multi-Fidelity Optimization: Results
Portfolio Construction: Why Without Meta Features?
Portfolio Construction: Problem Definition
Portfolio Construction: Method
Portfolio Construction: Theory
Portfolio Construction: Empirical Results
Automated policy selection problem Problem. There is no single
Automated policy selection method
Automated policy selection: results
Putting it all together: Auto-sklearn 2.0
Auto-PyTorch: Multi-Fidelity Optimization
Auto-PyTorch: Portfolio Construction w/o Meta-Features
Auto-PyTorch: Comparison Against Previous AutoML Frameworks
Auto-PyTorch: Evaluation on Image Data
Regularization Cocktails for SOTA Deep Learning on Tabular Data Combination of 13 different regularizers
Regularization Cocktails: Evaluation on 40 datasets
Taught by
Open Data Science
