Explore recurrent neural networks in this advanced natural language processing lecture from Carnegie Mellon University. Delve into the intricacies of long-distance dependencies, the Winigrad Schema Challenge, and various types of predictions. Examine the structure and functionality of recurrent networks, addressing the vanishing gradient problem and introducing Long Short-Term Memory (LSTM) networks. Analyze the strengths and weaknesses of recurrence in sentence modeling, and discover the potential of pre-training techniques for RNNs. Gain insights into efficiency considerations and optimization strategies for these powerful deep learning models.
Overview
Syllabus
Intro
Long Distance Dependencies
Winigrad Schema Challenge
Types of Prediction
Unconditioned vs Condition Prediction
Types of Unconditioned Prediction
Types of Condition Prediction
Recurrent Neural Networks
Vanishing Gradients
LTSM
RNNs
Other examples
Efficiency
Optimization
Taught by
Graham Neubig
Related Courses
-
Recurrent Neural Networks (RNNs): Deep Learning for Sequences and Time Series
-
Recurrent Neural Networks (RNNs) for Language Modeling with Keras
-
Neural Nets for NLP 2021 - Recurrent Neural Networks
-
Intermediate Deep Learning with PyTorch
-
Neural Nets for NLP - Recurrent Networks for Sentence or Language Modeling
-
Neural Nets for NLP 2017 - Recurrent Neural Networks
