Kafka as a Platform - The Ecosystem from the Ground Up

Explore the comprehensive Apache Kafka ecosystem in this 1-hour conference talk. Delve into the core components of Kafka as a messaging system and discover how it has evolved into a powerful streaming data platform. Learn about topics, partitioning, replication, and low-level Producer and Consumer APIs. Understand how to create elastically scalable, fault-tolerant application clusters and utilize advanced stream processing APIs like Kafka Streams and ksqlDB. Discover techniques for schema management to improve team collaboration around data formats. Explore methods for integrating Kafka with legacy systems without custom coding. By the end of this talk, gain a deep understanding of how Kafka has transformed from a simple message queue into an enterprise-grade streaming platform, covering key concepts such as events, pub/sub models, data production and consumption, broker functionality, Kafka Connect, asynchronous coupling, and schemas.