Explore a comprehensive lecture on unsupervised and semi-supervised learning of structure in neural networks for natural language processing. Delve into the distinctions between learning features and learning structure, examine various semi-supervised and unsupervised learning methods, and understand key design decisions for unsupervised models. Gain insights into practical examples of unsupervised learning, including cross-lingual applications and the challenges of learning latent structure. Learn about advanced concepts such as normalizing flow, gated convolution, and reinforcement learning approaches in the context of NLP. Discover how to leverage weak supervision and navigate the complexities of soft vs. hard tree structures in neural network architectures.
Neural Nets for NLP 2020 - Unsupervised and Semi-supervised Learning of Structure
Graham Neubig via YouTube
Overview
Syllabus
Supervised, Unsupervised, Semi-supervised
Learning Features vs. Learning Discrete Structure
Unsupervised Feature Learning (Review)
How do we Use Learned Features?
What About Discrete Structure?
What is our Objective?
A Simple First Attempt
Problem: Embeddings May Not be Indicative of Syntax
Normalizing Flow (Rezende and Mohamed 2015)
Cross-lingual Application of Unsupervised Models (He et al. 2019)
Soft vs. Hard Tree Structure
One Other Paradigm: Weak Supervision
Gated Convolution (Cho et al. 2014)
Learning with RL (Yogatama et al. 2016)
Difficulties in Learning Latent Structure
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP 2019 - Unsupervised and Semi-supervised Learning of Structure
-
Neural Nets for NLP 2017 - Unsupervised Learning of Structure
-
CMU Neural Nets for NLP 2020 - Multitask and Multilingual Learning
-
Neural Nets for NLP - Models with Latent Random Variables
-
Neural Nets for NLP - Latent Variable Models
-
Neural Nets for NLP - Latent Random Variables
