Completed
- Are our training objectives wrong?
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Author Interview - Typical Decoding for Natural Language Generation
Automatically move to the next video in the Classroom when playback concludes
- 1 - Intro
- 2 - Sponsor: Introduction to GNNs Course link in description
- 3 - Why does sampling matter?
- 4 - What is a "typical" message?
- 5 - How do humans communicate?
- 6 - Why don't we just sample from the model's distribution?
- 7 - What happens if we condition on the information to transmit?
- 8 - Does typical sampling really represent human outputs?
- 9 - What do the plots mean?
- 10 - Diving into the experimental results
- 11 - Are our training objectives wrong?
- 12 - Comparing typical sampling to top-k and nucleus sampling
- 13 - Explaining arbitrary engineering choices
- 14 - How can people get started with this?
