Explore a workshop presentation on tightening information-theoretic generalization bounds with data-dependent estimates, focusing on their application to Stochastic Gradient Langevin Dynamics (SGLD). Delve into the nature of generalization understanding, open problems, and barriers in the field. Examine non-vacuous bounds, stochastic gradient dynamics, and expected generalization error. Learn from Daniel Roy of the University of Toronto as he discusses these advanced concepts in deep learning theory, providing insights into current challenges and potential future directions in the field.
Tightening Information-Theoretic Generalization Bounds with Data-Dependent Estimates with an Application to SGLD - Daniel Roy
Institute for Advanced Study via YouTube
Overview
Syllabus
Intro
The nature of generalization understanding
Open problem
Barriers
Non vacuous bounds
Stochastic gradient dynamics
Expected generalization error
Plot
Conclusion
Taught by
Institute for Advanced Study
Related Courses
-
Information-Theoretic Generalization Bounds for Quantum Learning
-
Studying Generalization in Deep Learning via PAC-Bayes
-
On the Foundations of Deep Learning - SGD, Overparametrization, and Generalization
-
PAC-Bayesian Approaches to Understanding Generalization in Deep Learning - Gintare Dziugaite
-
Insights on Gradient-Based Algorithms in High-Dimensional Learning
