Completed
Data quality review
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Instruction Tuning of Large Language Models - Lecture
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 ChatGPT/GPT4 are real generalists
- 3 How did models acquire the vast capabilities?
- 4 NLP before 2018: building task-specific models
- 5 Classical multi-task learning
- 6 Generalization to unseen tasks via instructions
- 7 Expert-written instructions for all tasks
- 8 Strict train/test split for cross-task generalization
- 9 Instruction tuning significantly improves LLMs
- 10 What are the most important factors?
- 11 Other models trained on existing NLP datasets
- 12 Data is OpenAl's secret weapon
- 13 Can we construct a similar instruction dataset by crowdsourcing?
- 14 LLMs can be prompted to generate instructions
- 15 LM can be prompted to generate instances
- 16 Instruction data generation pipeline
- 17 Generating 52K instructions with GPT3
- 18 Tasks generated by GPT3
- 19 Data quality review
- 20 Performance on SuperNI
- 21 Expert evaluation on 252 user-oriented instructions
- 22 Effect of data size and data quality (using human eval)
- 23 Takeaways
- 24 Licensing concern about using OpenAl output?
