Completed
Learning Rates and Annealing
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Fine-tuning Llama 3 on Wikipedia Datasets for Low-Resource Languages
Automatically move to the next video in the Classroom when playback concludes
- 1 Fine-tuning Llama 3 for a low resource language
- 2 Overview of Wikipedia Dataset and Loss Curves
- 3 Video overview
- 4 HuggingFace Dataset creation with WikiExtractor
- 5 Llama 3 fine-tuning setup, incl. LoRA
- 6 Dataset blending to avoid catastrophic forgetting
- 7 Trainer setup and parameter selection
- 8 Inspection of losses and results
- 9 Learning Rates and Annealing
- 10 Further tips and improvements
