Generalization in Deep Learning Through the Lens of Implicit Rank Minimization
Hausdorff Center for Mathematics via YouTube
Overview
Syllabus
Intro
Generalization via Bis-Variance Tradeoff
Generalization in Deep Learning
Linear Models: Implicit Norm Minimization Linear Regression
Implicit Norm Minimization In Deep Learning?
Perspective: Implicit Rank Minimization
Outline
Matrix Completion Two-Dimensional Prediction
MF Linear NN
Conjecture: Implicit Nuclear Norm Minimization
Dynamical Analysis of Implicit Regularization in MF
Implicit Regularization in MF Norm Minimization Does the implicit regularization in MF minimize a norm?
Drawbacks of Studying MF
Tensor Completion Multi-Dimensional Prediction
TF Shallow Non-Linear Convolutional NN
Dynamical Analysis of Implicit Regularization in TF
Analogy Between Implicit Regularizations
HTF Deep Non-Linear CNN TF does not account for depth
Dynamical Analysis of Implicit Regularization in HTF
Practical Application: Rank Minimization in NN Layers
Potential Explanation for Generalization on Natural Data
Countering Locality of CNNs via Regularization
Recap
Implicit Rank Minimization in Deep Learning
Taught by
Hausdorff Center for Mathematics