Why Apache Kafka Clusters Are Like Galaxies - Cosmic Kafka Quandaries Explored

Explore the cosmic mysteries of Apache Kafka clusters in this 30-minute conference talk by Paul Brebner, Open Source Technology Evangelist at Instaclustr. Dive into surprising discoveries about Kafka cluster performance, scalability, and resource characteristics, drawing parallels between cluster sizes and distribution with Zipf's Law. Examine horizontal vs. vertical scalability and learn techniques for predicting Kafka performance using metrics, modeling, and regression. Gain valuable insights relevant to both Kafka developers and operators, based on data from hundreds of managed Kafka clusters across various sizes and use cases. Benefit from Brebner's extensive experience in distributed systems R&D and performance engineering as he shares findings from his recent inward-focused exploration of Kafka cluster behavior.