Completed
Tip 7: Start by only training on one GPU
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Top Ten Tips for Fine-tuning Large Language Models
Automatically move to the next video in the Classroom when playback concludes
- 1 Top Ten Fine-tuning Tips
- 2 Tip 1: Start with a Small Model
- 3 Tip 2: Use LoRA or QLoRA
- 4 Tip 3: Create 10 manual questions
- 5 Tip 4: Create datasets manually
- 6 Tip 5: Start training with just 100 rows
- 7 Tip 6: Always create a validation data split
- 8 Tip 7: Start by only training on one GPU
- 9 Tip 8: Use weights and biases for logging
- 10 Scale up rows, tuning type, then model size
- 11 Tip 9: Consider unsupervised fine-tuning if you've lots of data
- 12 Tip 10: Use preference fine-tuning ORPO
- 13 Recap of the ten tips
- 14 Ten tips applied to multi-modal fine-tuning
- 15 Playlists to watch
- 16 Trelis repo overview
- 17 ADVANCED Fine-tuning repo Trelis.com/ADVANCED-fine-tuning
- 18 Training on completions only
- 19 ADVANCED fine-tuning repo CONTINUED
- 20 ADVANCED vision Trelis.com/ADVANCED-vision
- 21 ADVANCED inference trelis.com/enterprise-server-api-and-inference-guide/
- 22 ADVANCED transcription trelis.com/ADVANCED-transcription
- 23 Support + Resources Trelis.com/About
