Completed
Adam
Class Central Classrooms beta
YouTube playlists curated by Class Central.
Classroom Contents
Neural Nets for NLP 2021 - Language Modeling, Efficiency/Training Tricks
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Language Modeling: Calculating the Probability of a Sentence
- 3 Count-based Language Models
- 4 A Refresher on Evaluation
- 5 Problems and Solutions?
- 6 An Alternative: Featurized Models
- 7 A Computation Graph View
- 8 A Note: "Lookup"
- 9 Training a Model
- 10 Parameter Update
- 11 Unknown Words
- 12 Evaluation and Vocabulary
- 13 Linear Models can't Learn Feature Combinations
- 14 Neural Language Models . (See Bengio et al. 2004)
- 15 Tying Input/Output Embeddings
- 16 Standard SGD
- 17 SGD With Momentum
- 18 Adagrad
- 19 Adam
- 20 Shuffling the Training Data
- 21 Neural nets have lots of parameters, and are prone to overfitting
- 22 Efficiency Tricks: Mini-batching
- 23 Minibatching
- 24 Manual Mini-batching
- 25 Mini-batched Code Example
- 26 Automatic Mini-batching!
- 27 Code-level Optimization . eg. TorchScript provides a restricted representation of a PyTorch module that can be run efficiently in C++
- 28 Regularizing and Optimizing LSTM Language Models (Merity et al. 2017)
- 29 In-class Discussion
