Implications of Implicit Bias in ReLU Networks - Understanding Generalization and Robustness

Explore a one-hour lecture examining the implications of implicit bias in ReLU networks, delivered by postdoctoral researcher Gal Vardi from TTI-Chicago and Hebrew University. Dive into three key findings about neural networks trained with logistic loss: how implicit bias leads to generalization in shallow univariate networks, its role in creating networks vulnerable to adversarial examples, and its potential for reconstructing training data from network parameters - with implications for both representation learning and privacy concerns. Learn from Vardi's extensive research background in theoretical machine learning and deep learning theory, developed through his work at prestigious institutions including the Weizmann Institute and Hebrew University. The presentation draws from collaborative research with leading scholars in the field and offers valuable insights into the theoretical foundations of deep learning.