Explore search-based structured prediction in natural language processing through this comprehensive lecture from CMU's Neural Networks for NLP course. Delve into the Structured Perceptron algorithm, examining its simplicity in training non-probabilistic global models. Contrast perceptron and global normalization approaches, and investigate structured training techniques. Learn about cost-augmented hinge loss and its application to sequence modeling. Gain insights into addressing exposure bias with simple remedies. Understand the intricacies of structured max-margin objectives and their role in NLP tasks. Discover how corrupt training data impacts model performance and explore strategies to mitigate its effects.
Overview
Syllabus
Intro
Types of Prediction
Two Methods for Approximation
Structured Perceptron Loss
The Structured Perceptron Algorithm . An extremely simple way of training (non-probabilistic) global models . Find the one-best, and if it's score is better than the correct answer adjust parameters to fix this
Contrasting Perceptron and Global Normalization • Globally normalized probabilistic model
Structured Training and Pre-training
Cost-augmented Hinge
Costs over Sequences
Corrupt Training Data
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP - Structured Prediction Basics
-
CMU Neural Nets for NLP - Structured Prediction Basics
-
CMU Advanced NLP: Structured Learning Algorithms
-
Neural Nets for NLP 2021 - Structured Prediction with Local Independence Assumptions
-
Neural Nets for NLP 2020 - Structured Prediction with Local Independence Assumptions
-
Neural Nets for NLP - Structured Prediction with Local Independence Assumptions
