Explore the optimization landscape of two-layer neural networks in this 58-minute seminar on theoretical machine learning presented by Rong Ge from Duke University. Delve into topics such as non-convexity, saddle points, and local-optimizable functions. Examine results for symmetric distributions and gain insights into optimization landscapes with symmetric input distributions. Learn about high-level ideas and interpolation techniques as applied to two-layer neural networks. This comprehensive talk, delivered at the Institute for Advanced Study, offers a deep dive into the mathematical foundations of neural network optimization.
Optimization Landscape and Two-Layer Neural Networks - Rong Ge
Institute for Advanced Study via YouTube
Overview
Syllabus
Introduction
Non convexity
Saddle points
Localoptimizable functions
Results
Symmetric Distribution
Optimization Landscape
symmetric input distribution
TwoLayer Neural Network
HighLevel Idea
First Attempt
Interpolate
Summary
Taught by
Institute for Advanced Study
Related Courses
-
Explaining Landscape Connectivity of Low-Cost Solutions for Multilayer Nets
-
Recent Developments in Over-parametrized Neural Networks, Part I
-
Can Non-Convex Optimization Be Robust?
-
Learning One-Hidden-Layer Neural Networks with Landscape Design
-
Gradient Descent on Infinitely Wide Neural Networks - Global Convergence and
-
Phase Diagram of Stochastic Gradient Descent in High-Dimensional Two-Layer Neural Networks
