Fast Vector Query Processing for Large Datasets Beyond GPU Memory with Reordered Pipelining

Explore a groundbreaking GPU-accelerated vector query processing system presented at NSDI '24. Dive into RUMMY, a novel solution that achieves high performance for large vector datasets exceeding GPU memory capacity. Learn about the innovative reordered pipelining technique that efficiently manages data transmission between host and GPU memory while optimizing query processing. Discover how cluster-based retrofitting, dynamic kernel padding, and query-aware reordering work together to maximize GPU utilization and overlap computation with data transfer. Understand the tailored GPU memory management strategies that reduce fragmentation and cache misses. Examine the impressive performance gains of RUMMY, outperforming existing solutions by up to 135× and offering significant cost-effectiveness improvements. Gain insights into the future of AI applications powered by efficient vector query processing in this 15-minute conference talk from researchers at Peking University's School of Computer Science.