Explore the evolution of generalization in machine learning through this comprehensive lecture by Misha Belkin from the University of California, San Diego. Delve into recent developments that have challenged traditional theoretical foundations, including empirical findings in neural networks. Examine the limitations of using training loss as a proxy for test loss and understand the implications of phenomena such as interpolation and double descent. Investigate the practice of early stopping and its potential shortcomings in light of emergent phenomena like grokking. Analyze the fundamental challenges these discoveries present to both the theory and practice of machine learning, and gain insights into the current state of understanding in the field.
The Elusive Generalization: Classical Bounds to Double Descent to Grokking
Overview
Syllabus
The elusive generalization: classical bounds to double descent to grokking
Taught by
Simons Institute
Related Courses
-
The Elusive Generalization and Easy Optimization - Part 2
-
The Elusive Generalization and Easy Optimization in Machine Learning - Part 1
-
Grokking - Generalization Beyond Overfitting on Small Algorithmic Datasets
-
From Classical Statistics to Modern Machine Learning
-
From Classical Statistics to Modern ML - The Lessons of Deep Learning - Mikhail Belkin
-
Beyond Empirical Risk Minimization - The Lessons of Deep Learning
