Explore the intersection of neural networks and nonparametric regression in this insightful lecture by Yu-Xiang Wang from UC San Diego. Delve into the reasons behind the superior performance of overparameterized deep learning models compared to classical methods like kernels and splines. Examine how standard hyperparameter tuning in deep neural networks (DNNs) implicitly uncovers hidden sparsity and low-dimensional structures, leading to improved adaptivity. Gain new perspectives on overparameterization, representation learning, and the generalization capabilities of neural networks through optimization-algorithm induced implicit biases such as Edge-of-Stability and Minima Stability. Analyze theory and examples that illustrate how DNNs achieve adaptive and near-optimal generalization, shedding light on their effectiveness in practical applications.
Neural Networks Meet Nonparametric Regression: Generalization by Weight Decay and Large Learning Rates
Overview
Syllabus
Neural Networks meet Nonparametric Regression: Generalization by Weight Decay and Large...
Taught by
Simons Institute
Related Courses
-
Deep Learning Meets Nonparametric Regression: Are Weight-decayed DNNs Locally Adaptive?
-
Build Decision Trees, SVMs, and Artificial Neural Networks
-
Learning and Generalization in Over-parametrized Neural Networks - Going Beyond Kernels
-
One of Three Theoretical Puzzles - Generalization in Deep Networks
-
Stochastic Learning Dynamics and Generalization in Neural Networks
-
Metric Space Magnitude and Generalization in Neural Networks
