Energy Based Models and Self-Supervised Learning

Overview

Explore energy-based models and self-supervised learning in this comprehensive lecture by Yann LeCun. Delve into the concept of energy-based models as an alternative to feed-forward networks, and learn how latent variables overcome inference challenges. Discover the relationship between EBMs and probabilistic models, and gain insights into self-supervised learning techniques. Examine the training process for Energy-Based Models, including Latent Variable EBMs with a K-means example. Investigate Contrastive Methods, denoising autoencoders, and the BERT model. Conclude with an in-depth look at Contrastive Divergence using topographic maps. Access additional resources through the provided course website and YouTube playlist for a comprehensive understanding of these advanced deep learning concepts.

Syllabus

– Week 7 – Lecture
– Energy-based model concept
– Latent-variable EBM: inference
– EBM vs. probabilistic models
– Self-supervised learning
– Training an Energy-Based Model
– Latent Variable EBM, K-means example, Contrastive Methods