Completed
Next-gen Spark shuffle service
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Tackling Scaling Challenges of Apache Spark at LinkedIn - Infrastructure Optimization and User Productivity
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Challenges of Scaling Spark
- 3 Tackling Scaling Challenges
- 4 Typical Spark User Questions
- 5 Automatic Failure Root Cause Analysis
- 6 Platform Failure Reason Breakdown
- 7 Grid Bench - Performance Analysis
- 8 Tuning Heuistics & Recommendations
- 9 Scaling Spark History Server
- 10 A Low-Latency Solution
- 11 Issues with Spark Shuffle Service
- 12 Next-gen Spark shuffle service
- 13 Push-Merge Shuffle
- 14 Fetch Merged Shuffle Data
- 15 Magnet Shuffle Service Recap
- 16 Takeaways
