Completed
- Problem Formulation
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
REALM- Retrieval-Augmented Language Model Pre-Training
Automatically move to the next video in the Classroom when playback concludes
- 1 - Introduction & Overview
- 2 - World Knowledge in Language Models
- 3 - Masked Language Modeling for Latent Document Retrieval
- 4 - Problem Formulation
- 5 - Knowledge Retriever Model using MIPS
- 6 - Question Answering Model
- 7 - Architecture Recap
- 8 - Analysis of the Loss Gradient
- 9 - Initialization using the Inverse Cloze Task
- 10 - Prohibiting Trivial Retrievals
- 11 - Null Document
- 12 - Salient Span Masking
- 13 - My Idea on Salient Span Masking
- 14 - Experimental Results and Ablations
- 15 - Concrete Example from the Model
