Watch a 48-minute lecture from Harvard CMSA where Florent Krzakala from Ecole Polytechnique Federale de Lausanne explores the role and validity of Gaussian assumptions in machine learning theory. Delve into the theoretical foundations of high-dimensional statistics and physics to understand when Gaussian models are effective for learning and when they fall short. Learn about recent developments in rigorous probabilistic approaches, including the Gaussian equivalence of generative models, universality of Gaussian mixtures, and conditions for high-dimensional estimation. Follow along as Krzakala examines practical applications through examples like ridge regression on MNIST, classification methods, and GAN-generated data, while addressing key theoretical concepts such as random matrix theory and data-agnostic approaches. Gain insights into the strengths and limitations of Gaussian assumptions in theoretical statistical learning, supported by mathematical proofs and real-world examples.
Are Gaussian Data All You Need for Machine Learning Theory - A Statistical Physics Perspective
Harvard CMSA via YouTube
Overview
Syllabus
Intro
The machine learning revolutic
Gaussians data
Physicists & Theoretical Neuroscie
Random Matrix Theory, Statistics
Data agnostic approaches
Fitting real dataset with a Gaussian model
Ridge regression on MNIST
But Gaussian theory does not always work
Ridge, Logistic, Hinge classification vs Gaussian
What is a better model than a single Gaussian?
Theorem: Gaussian Mixture through random
Open problems: beyond proportional regim
Generative Neural Networks as a proxy for r
GAN generated data behaves as Gaussian Mi
GMM stays GMM through random features
Theorem: Asymptotic of the Gaussian Mixture mo
2 Gaussians vs 1 Gaussian for different teacher
Single Gaussian for randon
Remember this plot with random label?
Universality of phase transition for homoskedast
Ridge interpolator & random
Many more questions...
Taught by
Harvard CMSA
