Explore the concept of a ZooKeeper layer for FoundationDB in this 20-minute conference talk by Paul Hemberger from HubSpot. Delve into the potential of simplifying distributed system operations by replacing ZooKeeper with FoundationDB, examining the implications for system design when clients have access to a highly scalable ZooKeeper ensemble with strict serializability. Investigate the building blocks of a proof-of-concept ZooKeeper layer for FDB, comparing data models, consistency guarantees, and watch semantics between ZooKeeper and FoundationDB. Learn about leveraging the Directory Layer for data modeling, using changefeeds for ZooKeeper watches, and evaluating approaches to essential components like ephemeral nodes and session management. Gain insights into ZooKeeper's typical usage, performance considerations, and secret recipes for configuration and synchronization, while understanding the intricacies of watch semantics, session management, and mini-elections in the context of building a ZooKeeper layer on FoundationDB.
A ZooKeeper Layer for FoundationDB - Paul Hemberger, HubSpot
Overview
Syllabus
Intro
Typical Zookeeper Usage
Zookeeper Performance
FoundationDB Performance
Why a ZK Layer?
Secret ZK Recipe for Configuration
Secret ZK Recipe for Synchronization
ZK Data Model
Filesystems: The Lazy Way
ZK Guarantees: Seq. Consistency
FDB Guarantees: Strict Serializability
Watch Semantics
Watches: Understanding ZK
Building the Watch Event Log
Notifying the Client
Building the Log Completed
What We Need to Build on FDB
Sessions: Heartbeats
Sessions: Detecting Disconnects
Persisting Sessions
Cleaning Up Sessions
Session Cleanup Mini-Elections
Session State Wrap-Up
Mini-Elections: Picking a Leader
Taught by
Linux Foundation
