Dive into advanced Apache Spark 3 concepts and techniques in this comprehensive course designed for experienced Spark users. Learn crucial skills for Databricks Spark certification, job interviews, and real-world applications. Explore Spark cluster architecture, runtime environments, deployment modes, and job execution processes. Master Spark SQL engine intricacies, query planning, and optimization techniques such as Adaptive Query Execution and Dynamic Join Optimization. Delve into memory management, data caching, and performance tuning strategies. Tackle complex topics like handling data skew, dynamic partition pruning, and speculative execution. Gain hands-on experience through practice quizzes and solution videos. Prepare to elevate your Spark expertise, enhance your career prospects, and contribute to the course's ongoing development by sharing your own challenges and questions.
Apache Spark 3 - Beyond Basics and Cracking Job Interviews
Overview
Syllabus
Apache Spark 3 - Beyond Basics and Cracking Job Interviews | Course Introduction.
Spark Cluster and Runtime Architecture.
Spark Submit and Some Important Options.
Deploy Modes - Client and Cluster mode.
Spark Jobs - Stage, Shuffle, Task, Slots.
Spark SQL Engine and Query Planning.
Lets Practice - Quiz 1 Solution Video.
Lets Practice - Quiz 2 Solution Video.
Spark Memory Allocation.
Spark Memory Management.
Spark Adaptive Query Execution.
Spark AQE Dynamic Join Optimization.
Handling Data Skew in Spark Joins.
Spark Dynamic Partition Pruning.
Data Caching in Spark.
Repartition and Coalesce.
Dataframe Hints.
Broadcast Variables.
Accumulators.
Speculative Execution.
Dynamic Resource Allocation.
Spark Schedulers.
Lets Practice Quiz 3 Solution Video.
Lets Practice Quiz 4 Solution Video.
Taught by
Learning Journal
