Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

VeScale - A PyTorch Native LLM Training Framework for Automatic Parallelism

CNCF [Cloud Native Computing Foundation] via YouTube

Overview

Explore a groundbreaking PyTorch native framework for large language model (LLM) training in this 24-minute conference talk by Hongyu Zhu from ByteDance. Learn about VeScale, a novel solution that combines PyTorch nativeness with automatic parallelism to address the challenges of distributed training for giant LLMs. Discover how this framework prioritizes ease of use, allowing developers to write single-device PyTorch code while automatically parallelizing it into nD parallelism. Gain insights into the importance of PyTorch ecosystem dominance and the necessity of complex nD parallelism for training massive models. Understand the limitations of existing industry-level frameworks and how VeScale aims to overcome them by offering a user-friendly approach to scaling LLM training.

Syllabus

VeScale: A PyTorch Native LLM Training Framework | veScale:一个PyTorch原生LLM训练框架 - Hongyu Zhu

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of VeScale - A PyTorch Native LLM Training Framework for Automatic Parallelism

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.