Explore the innovative concept of incremental snapshots in Debezium, a core feature for Change Data Capture (CDC) connectors. Learn how this advanced technique interleaves snapshot queries with log-based change events to address common challenges in CDC pipelines. Discover solutions for re-snapshotting individual tables, pausing and resuming long-running snapshots, and running snapshots concurrently with log reading. Delve into the inner workings of this scheme and its applications in solving typical tasks when managing CDC pipelines. Additionally, examine advanced topics such as parallelizing snapshots and customizing snapshot contents. This 43-minute talk, presented by Confluent, offers valuable insights for data engineers and developers working with Debezium and CDC implementations.
Overview
Syllabus
Debezium Snapshots Revisited!
Taught by
Confluent