Explore adversarial learning in neural networks for natural language processing in this lecture from CMU's CS 11-747 course. Dive into generative adversarial networks (GANs), examining their application to both features and outputs in NLP tasks. Learn about the challenges of using GANs with discrete outputs and discover techniques for overcoming these obstacles. Investigate the use of adversarial training methods for domain adaptation, multi-task learning, and unsupervised style transfer in text. Gain insights into stabilization tricks and innovative applications like GAN-based data cleaning. Enhance your understanding of advanced NLP concepts through practical examples and theoretical explanations provided by Professor Graham Neubig.
Overview
Syllabus
Intro
Generative Models
Adversarial Training
Basic Paradigm
Problems with Generation • Over-emphasis of common outputs, fuzziness Adversarial
Training Method
In Equations
Problems w/ Training
Applications of GAN Objectives to Language
Problem! Can't Backprop through Sampling
Solution: Use Learning Methods for Latent Variables
Discriminators for Sequences
Stabilization Trick
Interesting Application: GAN for Data Cleaning (Yang et al. 2017)
Adversaries over Features vs. Over Outputs
Learning Domain-invariant Representations (Ganin et al. 2016) • Learn features that cannot be distinguished by domain
Adversarial Multi-task Learning (Liu et al. 2017)
Implicit Discourse Connection Classification w/ Adversarial Objective
Professor Forcing (Lamb et al. 2016)
Unsupervised Style Transfer for Text (Shen et al. 2017)
Taught by
Graham Neubig
Related Courses
-
CMU Advanced NLP 2021 - Adversarial Learning
-
Advanced NLP: Adversarial Methods for Text
-
Neural Nets for NLP 2021 - Adversarial Methods
-
Neural Nets for NLP 2017 - Multilingual and Multitask Learning
-
Neural Nets for NLP 2017 - Unsupervised Learning of Structure
-
Neural Nets for NLP 2019: Sentence and Contextualized Word Representations
