A Kafka Client's Request - There and Back Again

Explore the intricate journey of data through Apache Kafka® in this 40-minute Devoxx conference talk. Delve into the complex process behind writing to and reading from Kafka, following a request from the initial producer.send() or consumer.poll() call to disk storage and back to the client. Examine client and broker configurations affecting request handling, and learn about crucial metrics for monitoring each stage of the request life cycle. Gain in-depth knowledge of read and write request mechanics, enhancing your ability to debug and analyze Kafka performance. Cover topics including schema and topic creation, partitioning configurations, batching settings and metrics, request configurations and general metrics, network threads, request queues, I/O threads, page cache/disk configuration, and the return journey to the producer. Equip yourself with the expertise to navigate Kafka's intricate data flow, making future troubleshooting and optimization tasks more manageable.