Scaling Batch, Big Data, and AI Workloads Beyond the Kubernetes Scheduler

Explore the challenges and solutions for scaling batch, big data, and AI workloads on Kubernetes in this informative conference talk. Delve into the limitations of the traditional Kubernetes scheduler when handling resource-intensive, heterogeneous processes. Discover recent innovations in the Kubernetes ecosystem designed to address issues such as resource fragmentation, lack of all-or-nothing semantics, low throughput, and limited priority, quota, and preemption management. Compare and contrast projects like Koordinator, Kueue, MCAD, Volcano, and YuniKorn, examining their design choices and trade-offs. Gain valuable insights to help determine the most suitable solution for optimizing Kubernetes cluster utilization for batch workloads. Learn from Red Hat experts Antonin Stefanutti and Anish Asthana as they provide a comprehensive overview of the evolving landscape of Kubernetes scheduling for demanding workloads.