Explore the Information Bottleneck Theory of Deep Neural Networks in this lecture by Naftali Tishby from the Hebrew University of Jerusalem. Delve into statistical learning theory, neural network applications, and information theory. Examine concepts such as soft partitioning, information plan, and stochastic gradient descent. Analyze the average per layer, classical theory, dimensionality, confidence, factorization, cardinality, and the ultimate bound. Gain insights into targeted discovery in brain data and expand your understanding of deep neural networks through this comprehensive presentation from the Simons Institute.
The Information Bottleneck Theory of Deep Neural Networks
Overview
Syllabus
Intro
Statistical Learning Theory
Neural Network Applications
Information Theory
Soft Partitioning
Information Plan
Stochastic Gradient Descent
Average Per Layer
Classical Theory
Dimensionality
Confidence
Factorization
Cardinality
The Ultimate Bound
Taught by
Simons Institute
Related Courses
-
Deep Learning - IIT Ropar
-
Deep Learning Interview Prep Course
-
Information Theory of Deep Learning - Naftali Tishby
-
Implicit and Explicit Regularization in Deep Neural Networks
-
Neural SDEs, Deep Learning and Stochastic Control
-
Phase Diagram of Stochastic Gradient Descent in High-Dimensional Two-Layer Neural Networks
