Completed
Example: Prune Redundant Filter
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Optimizing Catalyst Optimizer for Complex Spark Plans
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Spark in Workday Prism Analytics
- 3 Example: Data Validation
- 4 About Complex Plans
- 5 Common Subexpression Elimination (CSE)
- 6 CSE Benchmark
- 7 Logging Complex Plans (10s of MBs in Size)
- 8 Problems with Large Case Expressions
- 9 Handling Large Case Expressions in Catalyst
- 10 Large Case Expression Benchmark
- 11 Example: Generate New Filter
- 12 Example: Prune Redundant Filter
- 13 Example: New Filter on Other Side of Join
- 14 Current Constraint Propagation Algorithm
- 15 Current Algorithm Takes High Memory
- 16 Recall: Fix for Large Case Expressions
- 17 Optimized Constraint Propagation (SPARK-33152)
- 18 Constraint Propagation Algorithms Comparison
- 19 Constraint Propagation Benchmark
- 20 Effect on Customer Pipeline
- 21 Tuning Tips
- 22 Future Work
