Explore nonlinear independent component analysis in this seminar on theoretical machine learning presented by Aapo Hyvärinen from the University of Helsinki. Delve into the fundamental differences between ICA and PCA, examine the success of artificial intelligence and deep learning, and understand the importance of unsupervised learning. Discover how ICA performs blind source separation, with examples in brain source separation and image features. Investigate the unsolved problem of nonlinear ICA, including the Darmois construction and how temporal structure aids in nonlinear ICA. Learn about the algorithmic trick of "self-supervised" learning and the theorem on TCL estimates for nonlinear nonstationary ICA. Examine permutation contrastive learning and its demixing capability, as well as extensions of nonlinear ICA on time series. Explore the general framework of Deep Latent Variable Models and how conditioning makes DLVM identifiable. Gain insights into alternative approaches to the DLVM case in this comprehensive seminar on advanced machine learning concepts.
Overview
Syllabus
Intro
Abstract
Success of Artificial Intelligence
Neural networks
Deep learning
Importance unsupervised learning
ICA as principled unsupervised learning
Fundamental difference between ICA and PCA
Identifiability means ICA does blind source separation
Example of ICA: Brain source separation
Example of ICA: Image features
Nonlinear ICA is an unsolved problem
Darmois construction
Temporal structure helps in nonlinear ICA
Algorithmic trick: "Self-supervised" learning
Theorem: TCL estimates nonlinear nonstationary ICA Assume data follows nonlinear ICA model (t)-f s(tl) with
Permutation contrastive learning (Hyvärinen and Morioka 2017)
Illustration of demixing capability by PCL Non Gaussian AR model for sources
Extensions of nonlinear ICA on time series
General framework: Deep Latent Variable Models
Conditioning makes DLVM identifiable
Alternative approaches to DLVM case
Conclusion
Taught by
Institute for Advanced Study
