Supervised Contrastive Learning

Explore a groundbreaking approach to supervised learning in this 30-minute video that introduces the supervised contrastive loss function. Discover how this novel training methodology consistently outperforms the traditional cross-entropy loss across various architectures and data augmentations. Learn about the modification of the batch contrastive loss to leverage label information more effectively, pulling together clusters of same-class points in embedding space while pushing apart different-class clusters. Examine the key ingredients such as large batch sizes and normalized embeddings that contribute to the method's success. Delve into the impressive results, including a new state-of-the-art 78.8% accuracy on ImageNet using AutoAugment data augmentation. Investigate the benefits for robustness to natural corruptions, improved calibration, and increased stability to hyperparameter settings. Gain insights from the research conducted by Prannay Khosla and colleagues, as presented by Yannic Kilcher.