Explore advanced methods for text classification and sequence labeling in this 50-minute video lecture from CMU's Multilingual Natural Language Processing course. Delve into subword models, unsupervised training, and structured prediction models. Learn about bi-directional RNNs, bag of n-grams, and solutions for unknown words. Discover subword segmentation algorithms and embedding models, including cross-lingual learning techniques. Examine strategies for handling limited labeled data, such as joint multi-task learning and pre-training with masked language modeling. Compare multilingual representations and understand the importance of structured prediction in NLP tasks. Gain insights into local vs. global normalization and potential functions in advanced text classification and labeling techniques.
Overview
Syllabus
Intro
Text Classification
Sequence Labeling Given an input text X, predict an output label sequence of equal length
Reminder: Bi-RNNS - Simple and standard model for sequence labeling for classification
Issues w/ Simple BiRNN
Alternative: Bag of n-grams
Unknown Words
Sub-word Segmentation
Unsupervised Subword Segmentation Algorithms
Sub-word Based Embeddings
Sub-word Based Embedding Models
Embeddings for Cross-lingual Learning: Soft Decoupled Encoding
Labeled/Unlabeled Data Problem: we have very little labeled data for most analysis tasks for most languages
Joint Multi-task Learning
Pre-training
Masked Language Modeling
Thinking about Multi-tasking, and Pre-trained Representations
Other Monolingual BERTS
XTREME: Comparing Multilingual Representations
Why Call it "Structured" Prediction?
Why Model Interactions in Output?
Local Normalization vs. Global Normalization
Potential Functions
Discussion
Taught by
Graham Neubig
Related Courses
-
CMU Multilingual NLP 2020 - Text Classification and Sequence Labeling
-
CMU Multilingual NLP - Unsupervised Translation
-
CMU Neural Nets for NLP 2020 - Multitask and Multilingual Learning
-
Neural Nets for NLP - Structured Prediction Basics
-
CMU Neural Nets for NLP - Structured Prediction Basics
-
Neural Nets for NLP - Multi-task, Multi-lingual Learning
