Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

EoS in Kafka - Listen Up, I Will Only Say This Once

Strange Loop Conference via YouTube

Overview

Explore the intricacies of exactly-once semantics (EoS) in Apache Kafka in this comprehensive conference talk from Strange Loop. Dive deep into the recent additions to Kafka that enable EoS, including idempotence and transactions in Kafka clients. Gain insights into the specific semantics of Kafka distributed transactions and the underlying mechanics that allow them to scale efficiently. Learn about Kafka's unique approach to standard two-phase commit protocols, the maintenance and replication of transaction state, and how various failure scenarios are handled. Discover the future improvements planned to enhance exactly-once stream processing with Kafka. Led by Jason Gustafson, a top contributor to Kafka and part of the team that designed and implemented EoS support, this talk covers topics such as Kafka producers and consumers, processing models and semantics, idempotent producers, Chandy-Lamport snapshots, transaction log availability, zombie fencing, consumer read isolation, broker and consumer fetch handling, concurrent transactions, performance observations, and end-to-end EoS.

Syllabus

Intro
KAFKA IN A HURRY
KAFKA PRODUCER
PRODUCER PROTOCOL
KAFKA CONSUMER
CONSUMER PROTOCOL
PROCESSING MODEL
PROCESSING SEMANTICS
AT-LEAST-ONCE SEMANTICS
SEMANTIC WEAKNESSES
IDEMPOTENT PRODUCER
CHANDY-LAMPORT SNAPSHOTS
KAFKA TRANSACTIONS
TRANSACTION LOG AVAILABILITY
ZOMBIE FENCING
CONSUMER READ ISOLATION
BROKER FETCH HANDLING
CONSUMER FETCH HANDLING
CONCURRENT TRANSACTIONS
KAFKA EXACTLY ONCE SEMANTICS
PERFORMANCE OBSERVATIONS
END-TO-END EOS
FUTURE IMPROVEMENTS
EOS TEAM
APACHE KAFKA

Taught by

Strange Loop Conference

Reviews

Start your review of EoS in Kafka - Listen Up, I Will Only Say This Once

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.