Black Friday 2024: 25+ Ways to Save on Online Learning
Efficient Large-Scale Language Model Training on GPU Clusters

Efficient Large-Scale Language Model Training on GPU Clusters

Databricks via YouTube Direct link

Pipelining

6 of 15
Previous
Next

6 of 15

Pipelining

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Efficient Large-Scale Language Model Training on GPU Clusters

Automatically move to the next video in the Classroom when playback concludes
  1. 1 Introduction
  2. 2 GPU Cluster
  3. 3 Model Training Graph
  4. 4 Training
  5. 5 Idle Periods
  6. 6 Pipelining
  7. 7 Pipeline Bubble
  8. 8 Tradeoffs
  9. 9 Interleave Schedule
  10. 10 Results
  11. 11 Hyperparameters
  12. 12 DomainSpecific Optimization
  13. 13 GPU throughput
  14. 14 Implementation
  15. 15 Conclusion

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.