Overview
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Dive into an in-depth exploration of GPT-3, a groundbreaking language model, in this comprehensive video lecture. Examine how scaling up language models significantly improves task-agnostic, few-shot performance, potentially rivaling state-of-the-art fine-tuning approaches. Learn about the model's architecture, training process, and its impressive capabilities across various NLP tasks, including translation, question-answering, and complex reasoning. Discover the model's strengths in generating human-like text and its performance on challenging tasks such as arithmetic expressions and word unscrambling. Explore the broader implications of GPT-3's capabilities, including potential societal impacts and methodological challenges related to training on large web corpora. Gain insights into the future of natural language processing and the potential of large-scale language models to revolutionize AI applications.
Syllabus
- Intro & Overview
- Language Models
- Language Modeling Datasets
- Model Size
- Transformer Models
- Fine Tuning
- In-Context Learning
- Start of Experimental Results
- Question Answering
- What I think is happening
- Translation
- Winograd Schemes
- Commonsense Reasoning
- Reading Comprehension
- SuperGLUE
- NLI
- Arithmetic Expressions
- Word Unscrambling
- SAT Analogies
- News Article Generation
- Made-up Words
- Training Set Contamination
- Task Examples
Taught by
Yannic Kilcher