Achieving vLLM Platform Portability with Triton Autotuning - Ray Summit 2024

Explore a conference talk from Ray Summit 2024 where Burkhard Ringlein of IBM Research delves into achieving platform portability for vLLM using Triton autotuning. Learn about the challenges of vLLM's reliance on hand-written CUDA kernels and how IBM Research's innovative "dejavu" mechanism for the Triton autotuner eliminates production environment autotuning overhead. Discover early results showing significant kernel speed-ups, improved cross-platform performance, and reduced external dependencies for vLLM. Gain insights into Triton-only vLLM deployments and more portable, future-proof solutions for serving Large Language Models across different hardware platforms.