Explore the challenges and solutions of running Apache Spark on Kubernetes in this conference talk from Conf42 Cloud Native 2024. Discover the advantages of this approach and learn about Chidori, an AI/ML cluster management platform designed to address common issues. Gain insights into the high-level design of Chidori and its role in optimizing AI/ML workloads on Kubernetes. Watch a demonstration of the platform in action and understand how it can improve cluster management for data-intensive applications.
Overview
Syllabus
intro
preamble
agenda
spark on kubernetes
advantages
challenges
chidori
high level design
ai/ml cluster management
demo
thank you!
Taught by
Conf42
Related Courses
-
Amazon EMR Getting Started
-
Introduction to Kubernetes on Azure
-
Faster, Safer, Serverless - Empowering Apache Spark Standalone Cluster on Kubernetes
-
Kubernetes Native Two-Level Resource Management for AI-ML Workloads
-
Advanced CSI-FUSE Filesystem for AI/ML Data Management in Kubernetes
-
Spark on Kubernetes: Building an Elastic Service - Apple's Approach
