Explore a user-friendly approach to working with transformers and large language models for natural language processing.
Overview
Syllabus
Introduction
- Learning about Large Language Models
- What are large language models?
- Transformers in production
- Transformers: History
- Transfer learning
- Transformer: Architecture overview
- Self-attention
- Multi-head attention and Feed Forward Network
- GPT-3
- GPT-3 use cases
- Challenges and shortcomings of GPT-3
- GLaM
- Megatron-Turing NLG Model
- Gopher
- Scaling laws
- Chinchilla
- BIG-bench
- PaLM
- OPT and BLOOM
- Going further with Transformers
Taught by
Jonathan Fernandes