Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Flare - Scale Up Spark with Native Compilation and Set Your Data on Fire

Scala Days Conferences via YouTube

Overview

Explore a cutting-edge approach to enhancing Spark's performance through native compilation in this conference talk from Scala Days Chicago 2017. Discover Flare, a novel back-end for Spark that significantly accelerates query execution by compiling Catalyst query plans to native code. Learn how Flare leverages NUMA-aware scheduling and optimized data layouts to achieve 'mechanical sympathy' and maximize hardware utilization on modern server-class machines. Understand the potential of scale-up solutions for big memory systems as an alternative to traditional cluster-based scale-out approaches, particularly in terms of data center cost efficiency. Gain insights into Flare's design and its impressive performance improvements, with experimental results showing order of magnitude speedups over Spark 2.1 on standard benchmarks.

Syllabus

Flare: Scale Up Spark with Native Compilation and Set Your Data on Fire by Tiark Rompf

Taught by

Scala Days Conferences

Reviews

Start your review of Flare - Scale Up Spark with Native Compilation and Set Your Data on Fire

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.