Learning and Amortized Inference in Probabilistic Programs

Explore a comprehensive seminar on learning and amortized inference in probabilistic programs presented by Tuan Anh Le, a postdoctoral associate at MIT's Computational Cognitive Science Lab. Delve into two powerful methods for Bayesian inference: Reweighted Wake-Sleep (RWS) and the Thermodynamic Variational Objective (TVO). Discover how RWS simultaneously learns model parameters and amortizes inference, supporting discrete latent variables without suffering from the "tighter bounds" effect. Examine the novel TVO approach, which connects thermodynamic integration with variational inference. Gain insights into the applications of these methods beyond deep generative models, extending to probabilistic programs. Uncover the statistical connections, viewing RWS as adaptive importance sampling and understanding TVO's derivation from thermodynamic integration used in model selection via Bayes factors. This 71-minute seminar offers a deep dive into advanced techniques for enhancing probabilistic programming and Bayesian inference.