Random Matrices and Dynamics of Optimization in Very High Dimensions - Lecture 4

Explore a comprehensive lecture on random matrices and optimization dynamics in high-dimensional spaces. Delve into the complexities of machine learning and data science algorithms, focusing on the effectiveness of simple tools like Stochastic Gradient Descent in complex, over-parametrized regimes. Learn about the framework for non-experts, covering the structure of typical tasks and neural networks in standard contexts. Examine the classical context of SGD in finite dimensions before surveying recent work on projected "effective dynamics" for summary statistics in smaller dimensions. Discover how these dynamics govern the performance of high-dimensional systems and define complex dynamical systems in finite dimensions. Investigate the process of finding summary statistics through a dynamical spectral transition in Random Matrix Theory, exploring the behavior of Gram and Hessian matrices along optimization paths. Gain insights into the Random Matrix Tools needed for understanding edge spectrum behavior and BBP transition. Apply this knowledge to central machine learning examples, including multilayer neural networks for classification of Gaussian mixtures and XOR examples.