Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Database Hacking 101 - Building Fast Bulk Import in Java + C

Devoxx via YouTube

Overview

Explore techniques for building fast bulk import functionality in Java and C in this conference talk from Devoxx. Dive into the performance optimization strategies employed by QuestDB, an open-source time-series database, to accelerate CSV file imports. Learn how the engineers leverage io_uring to enhance speed and efficiency. Discover implementation details, benchmark results, and gain insights into the potential performance gains achievable through io_uring adoption. Follow along as Vlad Ilyushchenko, Co-Founder & CTO of QuestDB, shares his expertise in high-performance engineering and explains the intricacies of their CSV import optimization process, including sorting input, copy commands, and hot pass optimization. Gain valuable knowledge about advanced database performance tuning and cutting-edge techniques for handling large-scale data imports in time-series databases.

Syllabus

Introduction
What is QuestDB
Why bulk import
Previous approach
User experience
Sorting the input
Before we begin
Copy Command
The Plan
Benchmarking
Benchmark table
Pseudocode
CSV parser
Hot pass optimization
Flamegraph
Period
FIO
Results
Conclusions

Taught by

Devoxx

Reviews

Start your review of Database Hacking 101 - Building Fast Bulk Import in Java + C

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.