Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Neural Nets for NLP 2019: Sentence and Contextualized Word Representations

Graham Neubig via YouTube

Overview

Explore sentence and contextualized word representations in this comprehensive lecture from CMU's Neural Networks for NLP course. Dive into sentence representations, contextual word representations, and their applications in various NLP tasks. Learn about sentence classification, paraphrase identification, and textual entailment. Discover different types of learning, multi-task learning, and pre-trained representations. Examine language model transfer, context prediction transfer, and entailment transfer. Gain insights into bi-directional language modeling objectives and masked word prediction techniques. Enhance your understanding of advanced NLP concepts and their practical implementations in this in-depth presentation by Graham Neubig.

Syllabus

Intro
Goal for Today
Where would we need/use Sentence Representations?
Sentence Classification
Paraphrase Identification (Dolan and Brockett 2005) • Identify whether A and B mean the same thing
Textual Entailment (Dagan et al. 2006, Marelli et al. 2014)
Model for Sentence Pair Processing
Types of Learning
Plethora of Tasks in NLP
Rule of Thumb 2
Standard Multi-task Learning
Thinking about Multi-tasking, and Pre-trained Representations
General Model Overview
Language Model Transfer
End-to-end vs. Pre-training
Context Prediction Transfer (Skip-thought Vectors) (Kiros et al. 2015)
Paraphrase ID Transfer (Wieting et al. 2015)
Large Scale Paraphrase Data (ParaNMT-50MT) (Wieting and Gimpel 2018)
Entailment Transfer (InferSent) (Conneau et al. 2017)
Bi-directional Language Modeling Objective (ELMO)
Masked Word Prediction (BERT)

Taught by

Graham Neubig

Reviews

Start your review of Neural Nets for NLP 2019: Sentence and Contextualized Word Representations

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.