Explore techniques for optimizing Apache Spark and SQL performance in this 30-minute conference talk from Conf42 ML 2024. Delve into the evolution of Apache Spark and examine whether SQL alone is sufficient for modern data processing needs. Learn through practical case studies that demonstrate optimization strategies, covering topics such as data processing efficiency, query optimization, and performance tuning. Gain insights into improving your data processing workflows and enhancing the overall performance of your Apache Spark and SQL implementations.
Optimizing Apache Spark and SQL for Improved Performance
Overview
Syllabus
intro
preamble
why care?
about me
short history of apache spark
is sql enough now?
case #1
case #2
case #3 & 4
summary
thank you!
Taught by
Conf42
