Explore strategies for improving GPU utilization in Kubernetes environments in this 38-minute conference talk by Maulin Patel and Pradeep Venkatachalam from Google. Learn about the limitations of Kubernetes' current GPU allocation system, which requires integer-based resource requests, leading to potential overprovisioning and waste. Discover user-friendly solutions developed to enable multiple containers to share a single GPU, enhancing resource efficiency and reducing costs. Gain insights into the performance results of these solutions through demonstrations and real-world examples, with a focus on optimizing GPU usage for inference workloads processing real-time data samples.
Overview
Syllabus
Improving GPU Utilization using Kubernetes - Maulin Patel & Pradeep Venkatachalam, Google
Taught by
CNCF [Cloud Native Computing Foundation]
Related Courses
-
Improving GPU Utilization and Accelerating Model Training with Kubernetes Scheduling Framework and NRI
-
Kubernetes for the Absolute Beginners with Hands-on Labs
-
Scaling AI Workloads with Kubernetes - Sharing GPU Resources Across Multiple Containers
-
GPU Configuration on the Fly Using Dynamic Resource Allocation - A Tale of Two Drivers
-
Load-Aware GPU Fractioning for LLM Inference on Kubernetes
-
Are You Really Out of GPUs? How to Better Understand Your GPU Utilization
