Deep Dive into New Features of Apache Spark 3.1

Deep Dive into New Features of Apache Spark 3.1

Databricks via YouTube Direct link

Rocks DB State Store

18 of 30

18 of 30

Rocks DB State Store

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Deep Dive into New Features of Apache Spark 3.1

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Intro
  2. 2 ANSI SOL Compliance
  3. 3 Fail Earlier for Invalid Data
  4. 4 Forbid Confusing CAST
  5. 5 ANSI Mode GA in Spark 3.2
  6. 6 Unified CREATE TABLE SOL Syntax
  7. 7 CHAR/VARCHAR Support
  8. 8 More ANSI Features Coming in Spark 3.2!
  9. 9 Node Decommissioning
  10. 10 Summary
  11. 11 SOL Performance
  12. 12 Shuffle Hash Join Improvement
  13. 13 Partition Pruning Improvement
  14. 14 Predicate Pushdown Improvement
  15. 15 Reduce Query Compiling Latency (3.2)
  16. 16 Stream-stream Join
  17. 17 State Store for Structured Streaming
  18. 18 Rocks DB State Store
  19. 19 Add the type hints PEP 484 to PySpark!
  20. 20 Static Error Detection
  21. 21 Python Dependency Management
  22. 22 Visualization and Plotting
  23. 23 Usability Enhancements
  24. 24 New Utility Functions for Unix Time
  25. 25 New Utility Functions for Time Zone
  26. 26 EXPLAIN FORMMATTED
  27. 27 Ignore Hints
  28. 28 Documentation and Environments
  29. 29 New Doc for PySpark
  30. 30 Deprecations and Removals

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.