Explore advanced data import techniques for Citus in this 32-minute conference talk from Citus Con 2022. Dive into the fundamentals of loading data, comparing \COPY in Postgres to single-row inserts, before delving into more complex topics. Learn about microbatching \COPY operations, examine a case study on data ordering, and gain insights into direct worker writes in Postgres. Discover the intricacies of distributed triggers and their impact on data loading. Benefit from the expertise of Colton Shepard, a former Citus and Microsoft Solutions Engineer, as he shares valuable knowledge on optimizing data import processes for hyperscale environments.
Loading at Hyperscale: Advanced Data Import Techniques for Citus - Citus Con 2022
Overview
Syllabus
Introduction.
Single row inserts, upserts, & \COPY .
Microbatched & parallelized \COPY .
Index juggling .
Data ordering with case study .
Distributed triggers .
Direct worker loading .
Taught by
Microsoft Developer
