REALM- Retrieval-Augmented Language Model Pre-Training

REALM- Retrieval-Augmented Language Model Pre-Training

Yannic Kilcher via YouTube Direct link

- Introduction & Overview

1 of 15

1 of 15

- Introduction & Overview

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

REALM- Retrieval-Augmented Language Model Pre-Training

Automatically move to the next video in the Classroom when playback concludes

  1. 1 - Introduction & Overview
  2. 2 - World Knowledge in Language Models
  3. 3 - Masked Language Modeling for Latent Document Retrieval
  4. 4 - Problem Formulation
  5. 5 - Knowledge Retriever Model using MIPS
  6. 6 - Question Answering Model
  7. 7 - Architecture Recap
  8. 8 - Analysis of the Loss Gradient
  9. 9 - Initialization using the Inverse Cloze Task
  10. 10 - Prohibiting Trivial Retrievals
  11. 11 - Null Document
  12. 12 - Salient Span Masking
  13. 13 - My Idea on Salient Span Masking
  14. 14 - Experimental Results and Ablations
  15. 15 - Concrete Example from the Model

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.