Explore recurrent neural networks in this lecture from CMU's Neural Networks for NLP course. Dive into the fundamentals of recurrent networks, addressing challenges like vanishing gradients through LSTMs. Analyze the strengths and weaknesses of recurrence in sentence modeling and discover pre-training techniques for RNNs. Access accompanying slides and code examples to reinforce your understanding of key concepts including parameter tying, language modeling, sentence representation, and handling long sequences with mini-batching methods.
Overview
Syllabus
Intro
NLP and Sequential Data
Long-distance Dependencies in Language
Parameter Tying
What Can RNNs Do?
e.g. Language Modeling
Representing Sentences
Representing Contexts
Recurrent Neural Networks in DyNet
Parameter Initialization
Sentence Initialization
A Solution: Long Short-term Memory (Hochreiter and Schmichuber 1997)
Other Alternatives
Handling Mini-batching
Mini-batching Method
Handling Long Sequences
Example: LM - Sentence Classifier
LSTM Structure
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP 2021 - Recurrent Neural Networks
-
Deep Learning - Recurrent Neural Networks with TensorFlow
-
Neural Nets for NLP: Recurrent Neural Networks
-
Neural Nets for NLP - Recurrent Networks for Sentence or Language Modeling
-
Advanced RNN Concepts and Projects
-
Intermediate Deep Learning with PyTorch
