Explore the evolution and future of stream processing in the cloud era through this CMU Database Group seminar. Delve into the values, costs, and challenges of stateful stream processing, tracing its history from single-node systems to big data and cloud environments. Examine innovative approaches to state management, including tiered storage and rethinking design principles. Investigate critical aspects such as failure recovery and elastic scaling. Gain insights into challenging problems facing the field and potential solutions. Learn how RisingWave Labs is addressing these issues and reimagining stream processing for modern cloud architectures.
Overview
Syllabus
Intro
About the Startup
Stream Processing: Values and Costs
Cost in Stream Processing
Stateful Stream Processing
Stream Processing: History
State Management: Single Node Era
State Management: Big Data Era
State Management: Cloud Era
Tiered Storage for State Management
Rethinking State Management Design
Failure Recovery
Elastic Scaling
Challenging Problems: #1
Challenging Problems: #3
Summary
New Idea? Old Idea!
The Reason
Thank you! Questions?
Challenging Problems: #2
Taught by
CMU Database Group
