Understanding Contrastive Learning and Self-training in Neural Networks

Watch a 45-minute lecture from the Joint IFML/MPG Symposium at Simons Institute where Sujay Sanghavi from UT Austin and Amazon explores two key phenomena in neural network training. Dive into the analysis of contrastive representation learning, examining how the InfoNCE loss for linear networks trained on Gaussian Mixtures leads to improved dimensionality reduction compared to spectral methods. Learn about self-training techniques in binary classification with linear classifiers, discovering how filtering and retraining processes can enhance model accuracy when dealing with noisy labels. Understand the implications for recovering accuracy from data that has been artificially corrupted for label privacy through the collaborative research findings of Parikshit Bansal and Rudrajit Das.