Neural Nets for NLP 2019: Sentence and Contextualized Word Representations

Overview

Explore sentence and contextualized word representations in this comprehensive lecture from CMU's Neural Networks for NLP course. Dive into sentence representations, contextual word representations, and their applications in various NLP tasks. Learn about sentence classification, paraphrase identification, and textual entailment. Discover different types of learning, multi-task learning, and pre-trained representations. Examine language model transfer, context prediction transfer, and entailment transfer. Gain insights into bi-directional language modeling objectives and masked word prediction techniques. Enhance your understanding of advanced NLP concepts and their practical implementations in this in-depth presentation by Graham Neubig.

Syllabus

Intro
Goal for Today
Where would we need/use Sentence Representations?
Sentence Classification
Paraphrase Identification (Dolan and Brockett 2005) • Identify whether A and B mean the same thing
Textual Entailment (Dagan et al. 2006, Marelli et al. 2014)
Model for Sentence Pair Processing
Types of Learning
Plethora of Tasks in NLP
Rule of Thumb 2
Standard Multi-task Learning
Thinking about Multi-tasking, and Pre-trained Representations
General Model Overview
Language Model Transfer
End-to-end vs. Pre-training
Context Prediction Transfer (Skip-thought Vectors) (Kiros et al. 2015)
Paraphrase ID Transfer (Wieting et al. 2015)
Large Scale Paraphrase Data (ParaNMT-50MT) (Wieting and Gimpel 2018)
Entailment Transfer (InferSent) (Conneau et al. 2017)
Bi-directional Language Modeling Objective (ELMO)
Masked Word Prediction (BERT)

Taught by

Graham Neubig

Reviews

Start your review of Neural Nets for NLP 2019: Sentence and Contextualized Word Representations

Taught by

Neural Nets for NLP 2019 - Word Vectors

Neural Nets for NLP 2017 - Adversarial Learning

CMU Low Resource NLP Bootcamp 2020 - Neural Representation Learning

CMU Multilingual NLP 2020 - Advanced Text Classification-Labeling

CMU Advanced NLP: Pre-training Methods

CMU Neural Nets for NLP - Distributional Semantics and Word Vectors

Never Stop Learning.