Building Real-time Systems with Open Source Technologies

Explore the construction of real-time analytics systems using open-source technologies in this 32-minute conference talk from Strange Loop. Discover how to leverage Kafka, Storm, and Druid to create a robust analytics stack capable of processing vast quantities of data with minimal lag. Learn how combining these technologies with Hadoop can ensure system availability, maintain data integrity, and support fast, flexible queries. Gain insights from real-world experiences in building such a stack for online advertising analytics at Metamarkets. Understand the roles of each component: Kafka as a fast message bus, Storm and Hadoop working together to load data into Druid, and Druid providing highly available, low-latency queries. Presented by Gian Merlino and Fangjin Yang, experienced software engineers with backgrounds in infrastructure and data analytics.