Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Loss Landscape and Performance in Deep Learning by Stefano Spigler

International Centre for Theoretical Sciences via YouTube

Overview

Explore the intricacies of deep learning performance and loss landscapes in this 46-minute conference talk. Delve into supervised deep learning concepts, including network architecture and dataset setup. Examine learning dynamics as a descent in the loss landscape, drawing parallels with granular matter jamming. Analyze theoretical phase diagrams and empirical tests using MNIST parity. Investigate landscape curvature, flat directions, and the potential for overfitting. Consider ensemble averages and how fluctuations impact error rates. Discover scaling arguments and explore infinitely-wide networks, including initialization and learning processes. Gain insights into the Neural Tangent Kernel and finite N asymptotics. Enhance your understanding of deep learning theory and its connections to statistical physics.

Syllabus

Loss Landscape and Performance in Deep Learning
Supervised Deep Learning
Set-up: Architecture
Set-up: Dataset
Learning
Learning dynamics = descent in loss landscape
Analogy with granular matter: Jamming
Theoretical results: Phase diagram
Empirical tests: MNIST parity
Landscape curvature
Flat directions
Outline
Overfitting?
Ensemble average
Fluctuations increase error
Scaling argument!
Infinitely-wide networks: Initialization
Infinitely-wide networks: Learning
Neural Tangent Kernel
Finite N asymptotics?
Conclusion

Taught by

International Centre for Theoretical Sciences

Reviews

Start your review of Loss Landscape and Performance in Deep Learning by Stefano Spigler

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.