Infinite Limits and Scaling Laws for Deep Neural Networks

Explore a comprehensive mathematics seminar presentation that delves into the scaling laws and infinite parameter limits of deep neural networks. Learn how increasing model size and training horizons have led to significant advances in computer vision and natural language processing, with a focus on understanding how finite parameter models improve as they grow larger. Examine the preservation of representation learning in infinite parameter limits and discover the convergence rates of finite models through dynamical mean field theory methods. Investigate the practical implications by comparing training dynamics of finite networks to idealized limits. Master a theoretical framework explaining how generalization depends on training time, model size, and data quantity, while understanding compute-optimal scaling strategies. Gain insights into how representation learning can enhance neural scaling laws, potentially doubling the training-time exponent compared to static kernel limits for complex tasks.