Advanced Predictive Modeling: Mastering Ensembles and Metamodeling

Introduction

• The most accurate machine learning models
• What you should know

1. Key Modeling Concepts

• Ensemble wins Netflix Prize
• What is an ensemble?
• Types of models and modeling algorithms
• Types of ensembles

2. Understanding Model Error

• Measuring model accuracy: Value estimation
• Understanding model error: Classification

3. Simple Heterogeneous Ensembles

• Stacking
• Voting for classification

4. The Bias-Variance Tradeoff

• Error decomposition
• Visualizing bias and variance
• Curse of dimensionality
• Is Occam's Razor always true?

5. Ensemble Algorithms Fundamentals

• What is Bootstrap aggregating?
• What is Boosting and how does it work?
• Gradient boosting demo

6. Important Ensemble Algorithms

• Random forest
• Model search by bumping
• AdaBoost, XGBoost, Light GBM, CatBoost
• Super Learner, Subsemble, StackNet
• What are people working on now?

7. Ensemble and Meta-Modeling Case Studies

• Combining supervised and unsupervised
• Routing cases to different models

Conclusion

• Next steps