Overview
Explore masked language modeling with Hugging Face in this 48-minute Python coding tutorial. Learn the fundamentals of masked language modeling, then apply a pre-trained language model to the Microsoft Research Sentence Completion Challenge. Follow along using the provided Colab notebook to inspect data, implement the model, and evaluate its performance. Gain practical experience with Hugging Face and Transformer models, focusing on implementation rather than theory. Access additional resources, including the RoBERTa paper and Microsoft Research Sentence Completion Challenge details, to deepen your understanding of the topic.
Syllabus
Intro and outline:
What is masked language modelling?
Colab notebook / Inspect data:
Applying the language model to MRSCC:
Evaluating the model:
Taught by
rupert ai