Overview
Learn to build streaming data pipelines by combining Apache Flink and Pulsar in this 27-minute conference talk from PulsarSummit SF22. Discover how to leverage Flink SQL's abstractions to create language-agnostic pipelines without extensive technical expertise. Follow along with step-by-step demonstrations showing how to construct unified batch and streaming pipelines from scratch using the Flink SQL client with Pulsar's messaging system. Explore stream processing challenges, prerequisites, and practical examples designed for complex systems while remaining accessible to all skill levels. Master essential concepts like datageneration, unified data processing stacks, and Flink workarounds through hands-on demonstrations. Access additional resources including the Flink SQL Cookbook, community support through GitHub and Flink Slack, and comprehensive documentation to continue developing streaming pipeline expertise.
Syllabus
Intro
Stream Processing Challenges
Prereqs
First Demo
Next Level
Datagen
Results
Why Flink with Pulsar
Unified Data Processing Stack
Demo
Flink workaround
Final steps
Flink SQL Cookbook
Community Resources
GitHub
Flink Slack
Show Tell
Thanks
Taught by
StreamNative