Automatically Detect and Remediate Degraded Storage in Kafka

Dive into a 31-minute conference talk that addresses a critical issue for Kafka administrators: detecting and remediating degraded storage. Learn how Confluent developed an automated system to tackle this problem, which can lead to increased latencies and potential unavailability. Explore the importance of monitoring storage performance and gain insights into the detection algorithm formulation, monitor creation and fine-tuning, and end-to-end pipeline testing. Discover the tools and processes developed for mitigating storage degradation, and understand how this streamlined system improved performance and availability of clusters in Confluent Cloud. Gain valuable knowledge about maintaining optimal Kafka infrastructure and preventing client application issues caused by degraded storage components.