Completed
problem: we can't model long-term dependencies
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Sequence Modeling with Neural Networks
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 What is a sequence?
- 3 a sequence modeling problem
- 4 idea: use a fixed window
- 5 problem: we can't model long-term dependencies
- 6 idea: use entire sequence, as a set of counts
- 7 idea: use a really big fixed window
- 8 problem: no parameter sharing
- 9 to model sequences, we need
- 10 example network
- 11 RNNS remember their previous state
- 12 "unfolding" the RNN across time
- 13 remember: backpropagation
- 14 let's try it out for W with the chain rule
- 15 backpropagation through time
- 16 problem: vanishing gradient
- 17 activation functions
- 18 initialization
- 19 gated cells
- 20 possible task: music generation
- 21 possible task: machine translation
- 22 problem: a single encoding is limiting
- 23 solution: attend over all encoder states
