Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

Unlocking LLM Performance with eBPF - Optimizing Training and Inference Pipelines

CNCF [Cloud Native Computing Foundation] via YouTube

Overview

Explore how to optimize Large Language Model (LLM) performance using eBPF in this 38-minute conference talk from the Cloud Native Computing Foundation (CNCF). Discover techniques for achieving observability in LLM training and inference processes without disruption, including Memory Profiling for model and training data loading performance, Network Profiling for data exchange performance, and GPU Profiling for analyzing Model FLOPs Utilization (MFU) and performance bottlenecks. Learn about the practical effects of implementing eBPF-based observability in PyTorch LLM applications and the llm.c project to enhance training and inference performance. Gain insights into overcoming the challenges of improving GPU utilization in LLM processes that handle vast amounts of data and consume significant computational resources.

Syllabus

Unlocking LLM Performance with EBPF: Optimizing Training and Inference Pipelines - Yang Xiang

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of Unlocking LLM Performance with eBPF - Optimizing Training and Inference Pipelines

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.