Neural Nets for NLP 2020 - Language Modeling, Efficiency/Training Tricks

Neural Nets for NLP 2020 - Language Modeling, Efficiency/Training Tricks

Graham Neubig via YouTube Direct link

Intro

1 of 21

1 of 21

Intro

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Neural Nets for NLP 2020 - Language Modeling, Efficiency/Training Tricks

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Intro
  2. 2 Language Modeling: Calculating
  3. 3 Count-based Language Models
  4. 4 A Refresher on Evaluation
  5. 5 Problems and Solutions? • Cannot share strength among similar words
  6. 6 Example
  7. 7 Softmax
  8. 8 A Computation Graph View
  9. 9 A Note: "Lookup"
  10. 10 Training a Model
  11. 11 Parameter Update
  12. 12 Unknown Words
  13. 13 Evaluation and Vocabulary
  14. 14 Linear Models can't Learn Feature Combinations
  15. 15 Neural Language Models (See Bengio et al. 2004)
  16. 16 Tying Input/Output Embeddings
  17. 17 Standard SGD
  18. 18 SGD With Momentum
  19. 19 Adagrad
  20. 20 Adam . Most standard optimization option in NLP and beyond . Considers rolling average of gradient, and momentum
  21. 21 Shuffling the Training Data

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.