Improving Apache Kafka Scalability and Elasticity with Tiered Storage

Explore the implementation of tiered storage for improving Apache Kafka's scalability and elasticity in this 30-minute podcast episode featuring Rittika Adhikari, Software Engineer at Confluent. Learn about the motivating factors behind tiered storage, including cost, performance, and manageability. Discover how this method decouples compute and storage, allowing for indefinite data retention and more efficient re-replication processes. Gain insights into the evolution of Confluent Tiered Storage, the concept of tiered compaction, and its impact on performance. Understand the benefits for end users and how to get started with Confluent Tiered Storage. Delve into the technical aspects of Kafka's storage foundations and learn about the future developments in supporting compacted topics through tier compaction.