Explore the evolution of machine learning models in this comprehensive lecture, starting from Latent Variable Energy-Based Models (LV-EBM) and progressing through target propagation to various types of autoencoders. Delve into conditional and unconditional LV-EBMs, understand the training process for warm cases and zero-temperature limits, and discover the crucial missing step in model development. Learn about target propagation and its transition to autoencoders, examining reconstruction costs, loss functionals, and the impact of under and over-complete hidden layers. Investigate specialized autoencoder variants, including denoising and contractive autoencoders, before culminating in an in-depth exploration of variational autoencoders. Compare different autoencoder types, gain insights into the inner workings of variational autoencoders, and grasp the innovative bubble-of-bubble interpretation for a thorough understanding of these powerful machine learning techniques.
Overview
Syllabus
– 2021 edition disclaimer
– Conditional and unconditional LV EBM
– Variables' name: x, y, z, h, ỹ
– LV EBM training recap, warm case
– LV EBM training recap, zero-temperature limit
– Today's plan: the missing step
– Target propagation
– From target prop to autoencoder
– Reconstruction costs
– Loss functional
– Under and over complete hidden layer
 – Denoising autoencoder
– Contractive autoencoder
– Autoencoders recap
– From autoencoder to variational autoencoder
– Comparison between variational autoencoder and denoising autoencoder
– How a variational autoencoder actually works
– The bubble-of-bubble variational autoencoder interpretation
– And that was it :
Taught by
Alfredo Canziani
