Dive deep into Discord's engineering journey as they migrate from Cassandra to ScyllaDB to handle trillions of messages. Explore the fundamentals of database engineering, comparing relational and distributed systems. Understand the challenges faced with Cassandra, including hot partitions and garbage collector pauses. Learn about crucial concepts like SnowFlake vs UUID, B+Trees and their interaction with SSDs, and LSM Trees. Discover how Discord rearchitected their backend, implemented data services, and executed the migration. Gain insights into zoned namespaces and walk away with a comprehensive understanding of large-scale message storage solutions in this informative video lecture.
How Discord Stores Trillions of Messages - Deep Dive
Overview
Syllabus
Intro
Relational vs Distributed
The Cassandra Troubles
SnowFlake vs UUID
B+Tree
B+Tree and SSDs
LSM Trees
Hot partitions
Cassandra Garbage Collector Pauses
Changing the Architecture
The Data Services
The Migration
Zoned Named Spaces
Summary
Taught by
Hussein Nasser
