Loss Landscape and Performance in Deep Learning by Stefano Spigler
International Centre for Theoretical Sciences via YouTube
Overview
Syllabus
Loss Landscape and Performance in Deep Learning
Supervised Deep Learning
Set-up: Architecture
Set-up: Dataset
Learning
Learning dynamics = descent in loss landscape
Analogy with granular matter: Jamming
Theoretical results: Phase diagram
Empirical tests: MNIST parity
Landscape curvature
Flat directions
Outline
Overfitting?
Ensemble average
Fluctuations increase error
Scaling argument!
Infinitely-wide networks: Initialization
Infinitely-wide networks: Learning
Neural Tangent Kernel
Finite N asymptotics?
Conclusion
Taught by
International Centre for Theoretical Sciences