Explore the fundamentals of vLLM, a fast and user-friendly library for LLM inference and serving, in this 28-minute video tutorial. Learn how to run vLLM locally and deploy it on Kubernetes in a production environment with GPU-attached nodes using a DaemonSet. Follow along with a hands-on demonstration that guides you through the process of implementing vLLM in a production setting. Gain insights from a real-world case study on cost-effective deployment of open-source AI technologies, as detailed in the accompanying blog post. Presented by John McBride, this Kubesimplify tutorial offers practical knowledge for efficiently leveraging vLLM on Kubernetes.
Overview
Syllabus
vLLM on Kubernetes in Production
Taught by
Kubesimplify
Related Courses
-
Databricks' vLLM Optimization for Cost-Effective LLM Inference - Ray Summit 2024
-
vLLM Inference and LLM Server Engine for Machine Learning
-
Building Your Own ChatGPT-style LLM AI Infrastructure with Kubernetes
-
The Evolution of Multi-GPU Inference in vLLM
-
The State of vLLM - Advancements in LLM Inference and Serving
-
Load-Aware GPU Fractioning for LLM Inference on Kubernetes
