Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Insights on Gradient-Based Algorithms in High-Dimensional Learning

Simons Institute via YouTube

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore gradient-based algorithms in high-dimensional learning through this Richard M. Karp Distinguished Lecture. Delve into the analysis of gradient descent algorithms and their noisy variants in nonconvex settings. Examine several high-dimensional statistical learning problems where gradient-based algorithm performance can be analyzed precisely. Discover how statistical physics provides exact closed solutions for algorithm performance in the high-dimensional limit. Cover topics including the spiked mixed matrix-tensor model, perceptron, and phase retrieval. Gain insights into dynamical mean-field theory, Langevin dynamics, and stochastic gradient descent. Investigate phase diagrams, landscape analysis, and the teacher-student perceptron model. Understand the behavior of gradient descent in phase retrieval and explore theories for over-parameterized landscapes.

Syllabus

Intro
WORKHORSE OF MACHINE LEARNIN
IN DEEP LEARNING
STRATEGY
WHY THIS MODEL?
ESTIMATORS
GRADIENT-BASED ALGORITHMS
DYNAMICAL MEAN FIELD THEORY
LANGEVIN STATE EVOLUTION (NUMERICAL SOLUTION)
LANGEVIN PHASE DIAGRAM
GRADIENT-FLOW PHASE DIAGRAM
POPULAR "EXPLANATION"
SPURIOUS MINIMA DO NOT NECESSARILY CAUSE GF TO FAIL
WHAT IS GOING ON?
TRANSITION RECIPE
TRANSITION CONJECTUR
LANDSCAPE ANALYSIS
CONCLUSION ON SPIKED MATRIX-TENSOR MODEL
TEACHER-NEURAL SETTING
TEACHER STUDENT PERCEPTRON
PHASE RETRIEVAL: OPTIMAL SOLUTION
GRADIENT DESCENT FOR PHASE RETRIEVAI
PERFORMANCE OF GRADIENT DESCENT
GRADIENT DESCENT NUMERICALLY
TOWARDS A THEORY
OVER-PARAMETRISED LANDSPACE
STOCHASTIC GRADIENT DESCENT
DYNAMICAL MEAN-FIELD THEOR Mignaco, Urbani, Krzakala, LZ, 2006.06098
DMFT FOLLOWS THE WHOLE TRAJECTORY

Taught by

Simons Institute

Reviews

Start your review of Insights on Gradient-Based Algorithms in High-Dimensional Learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.