Large Model Training and Inference with DeepSpeed

Explore the journey of DeepSpeed and its transformative impact on large model training and inference in this 36-minute conference talk by Samyam Rajbhandari at the LLMs in Prod Conference. Discover how technologies like ZeRO and 3D-Parallelism have become fundamental building blocks for training large language models at scale, powering LLMs such as Bloom-176B and Megatron-Turing 530B. Learn about heterogeneous memory training systems like ZeRO-Offload and ZeRO-Infinity, which have democratized LLMs by making them accessible with limited resources. Gain insights into DeepSpeed-Inference and DeepSpeed-MII, which simplify the application of powerful inference optimizations to accelerate LLMs for deployment. Understand how DeepSpeed has been integrated into platforms like HuggingFace, PyTorch Lightning, and Mosaic ML, and how its technologies are offered in PyTorch, Colossal-AI, and Megatron-LM. Delve into the motivations, insights, and stories behind the development of these groundbreaking technologies that have revolutionized large language model training and inference.