Explore the challenges and solutions of data processing in datacenters through a humorous and insightful conference talk. Dive into the world of BPF (Berkeley Packet Filter) and its application in network-attached storage systems. Learn why recompiling the kernel became necessary to perform seemingly simple tasks like averaging numbers. Discover the speaker's journey through various approaches, including moving computations to storage and leveraging eBPF for networking. Gain insights into datacenter requirements, prototype development, and the intricacies of implementing filter-reduce operations in BPF. Understand the complexities of hooking BPF extensions, handling data, and optimizing performance in large-scale data processing scenarios.
BPF Tales: Recompiling the Kernel to Average Numbers in Datacenters
Overview
Syllabus
Intro
The data deluge
So, this is a datacenter
What is happening?
Ok, let's average numbers
Ok, let's compute the average
This doesn't make sense
Let's move it to the storage!
This makes sense!
Datacenter requirements
And my requirement
What are the options?
eBPF for networking
What am I looking for?
Somewhere in the read path...
This is the prototype
Why Filter-Reduce?
First of all, where to hook the bpf extensions
Second, what about data
Third, how much can I
Well at least it computes the average, right?
So, to summarize
Taught by
Linux Foundation
