Explore neural networks and their susceptibility to spurious correlations in this 40-minute lecture by Aditi Raghunathan from UC Berkeley. Delve into the challenges of distribution shifts and their impact on model performance. Learn about techniques to train neural networks that are robust to known spurious correlations, including counter-intuitive observations on model size and training data effects. Examine approaches to mitigate spurious correlations when complete information is unavailable. Investigate the empirical performance of reweighting and subsampling methods, focusing on tasks like bird type classification with spurious backgrounds. Gain insights into why neural networks pick up on spurious correlations, understand the simplicity bias, and discover strategies to overcome it. Conclude with valuable parting thoughts on improving neural network robustness in the presence of spurious correlations.
Overview
Syllabus
Intro
Typical machine learning pipeline
Taming the neural network beasts
In this talk...
Models latch onto spurious correlations
Improving worst-group error Simple and well-studied approach: reweighting loss based on group proportion
Talk outline
Empirical performance of reweighting Task: bird type Spurious background
When do larger models have high worst-group error?
Can larger models obtain low worst-group error?
Empirical performance of subsampling
Summary (spurious correlations)
Why do networks pick up on spurious correlations?
Simplicity bias of neural networks
How to overcome simplicity bias
Just train twice - results
Parting thoughts
Taught by
Simons Institute
