Real-Time Data Integration Practice Based on Flink CDC at Alibaba Cloud

Explore real-time data integration practices using Flink CDC at Alibaba Cloud in this 25-minute conference talk. Delve into the core design and key implementation of Flink CDC technology, with a focus on the new features introduced in version 2.4.0. Learn about the technical advantages of Flink CDC, including full incremental integration, lock-free reading, concurrent reading, and distributed architecture. Discover how Flink CDC supports powerful data processing capabilities, allowing for real-time association, aggregation, and flattening of database data using SQL. Gain insights into Alibaba Cloud's internal Flink CDC solutions for addressing specific business challenges, such as data lake and warehouse integration scenarios and binlog expiration issues. Understand how processed data can be seamlessly written to downstream systems like Kafka, Hudi, Iceberg, and Doris, enabling efficient real-time data lake and warehouse integration.