Explore latent variable models in advanced natural language processing through this comprehensive lecture from CMU's CS 11-711 course. Delve into the distinctions between generative and discriminative models, as well as deterministic and random variables. Gain insights into Variational Autoencoders (VAEs) and their applications in NLP, including techniques for handling discrete latent variables. Examine the trade-offs between learning features and learning structure. Cover topics such as evidence lower bound, procedural training, KL divergence annealing, and aggressive inference network learning. Understand sampling methods and reparameterization for discrete latent variables, equipping you with advanced knowledge for implementing sophisticated NLP models.
Overview
Syllabus
Introduction
Discriminative vs generative models
Types of variables
Loss function
Two tasks
Bias and variance
Evidence lower bound
Procedural training
Questions
Learning the VAE
Generating Sentences
Problems
kl divergence annealing
Free bits
Weaken the decoder
Aggressive inference network learning
Standard variational autoencoder
What are discrete latent variables
Method 1 Sampling
Method 2 Sampling
Method 2 Reparameterization
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP - Latent Variable Models
-
CMU Advanced NLP 2022 - Latent Variable Models
-
Neural Nets for NLP - Models with Latent Random Variables
-
Neural Nets for NLP - Models with Latent Random Variables
-
CMU Neural Nets for NLP - Models with Latent Random Variables
-
Neural Nets for NLP - Latent Random Variables
