Completed
Intro
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
How to Make Apache Spark on Kubernetes Run Reliably on Spot Instances
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Kubernetes is a new cluster manager for Spark
- 3 The Spark on Kubernetes Journey
- 4 Spark on YARN: architecture & pain points
- 5 Spark on Kubernetes: architecture & benefits
- 6 Our background - Ocean for Apache Spark
- 7 Spot instances
- 8 How does Spark cope with spot interruptions?
- 9 Best practice: run driver OD, execs on Spot
- 10 This is how your cluster may look like
- 11 Limitation: Avoid cross-Az data transfer
- 12 We ran an experiment to measure the impact
- 13 Experiment results
- 14 Since Spark 3.1: Graceful Exec Decommissioning
- 15 Spark 3.1 - Graceful Exec Decommissioning
- 16 Graceful Exec Decommissioning - Experiment
- 17 Since Spark 3.2: Executor PVC Reuse
- 18 What's new in Spark 3.3 for Spark-on-kes
- 19 DATA+AI SUMMIT 2022
