Kafka as a Platform - The Ecosystem from the Ground Up

Explore the Apache Kafka ecosystem from the ground up in this comprehensive conference talk. Delve into the core concepts of Kafka as a messaging system and streaming data platform. Learn about key-value pairs, topics, partitioning, replication, and low-level Producer and Consumer APIs. Discover how to group consumers into scalable, fault-tolerant application clusters and utilize advanced stream processing APIs like Kafka Streams and ksqlDB. Gain insights into schema management for team collaboration around data formats. Follow along as the speaker builds up the Kafka platform, covering events, logs, topics, partitions, pub/sub patterns, brokers, replication, and more. Conclude with a practical demo and summary of Kafka's capabilities as a central component in modern data infrastructure.