Completed
Problems of existing Spark K8s infrastructure Complexity of layers of custom Kås controllers to handle the scale of the
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Cloud-Native Apache Spark Scheduling with YuniKorn on Kubernetes
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Role of Kos in Lyft's Data Landscape
- 3 Multi-step creation for a Spark KBs job
- 4 Problems of existing Spark K8s infrastructure Complexity of layers of custom Kås controllers to handle the scale of the
- 5 Why we need a customized K8s Scheduler
- 6 Flavors of Running Spark on KBS
- 7 Resource Scheduling in K8s
- 8 Spark on K8s: the scheduling challenges
- 9 Apache Yunikorn (Incubating)
- 10 Resource Scheduling in Yunikorn land compare w/default scheduler
- 11 Main difference (Yunikorn v.s Default Scheduler)
- 12 Run Spark with Yunikorn
- 13 Job Ordering
- 14 Resource Quota Management: K8s Namespace ResourceQuota
- 15 Resource Quota Management: Yunikorn Queue Capacity
- 16 Resource Fairness in Yunikorn Queues
- 17 Scheduler Throughput Benchmark
- 18 Fully K8s Compatible
- 19 Yunikorn Management Console
- 20 Compare Yunikorn with other K8s schedulers
- 21 Current Status
- 22 The Community
- 23 Roadmap
- 24 Our Vision - Resource Mgmt for Big Data & ML
