Explore the world of LanceDB, an open-source serverless vector database built in Rust, in this conference talk from Conf42 Rustlang 2023. Dive into the history and development of LanceDB, understanding its core concepts and applications in text-to-image recommendations. Learn about the challenges faced in building vector indexes and how Rust's capabilities were leveraged to overcome them. Discover the intricacies of implementing k-means clustering and handling I/O operations in a vector database context. Gain insights into the on-disk layout of IVF PQ indexes and the integration of SQL and full-text search capabilities. Explore the unique in-process nature of LanceDB and its cloud offerings, providing a comprehensive overview of this innovative database solution.
LanceDB: An Open-Source Serverless Vector Database in Rust
Overview
Syllabus
Intro
about lancedb
a bit of history of lancedb
we love rust!
what is vector database
applicaiton: text-to-image recommendation
challenges
build vector index in rust: ivf_pq
yet another kmean in rust!
i/o is tricky too! no linear indexing = scan!
ivf pq index on-disk layout
sql and full text search
but, how can i use it
did we mention that lancedb is in-process db?
lancedb is in-process db
lancedb cloud
thank you
Taught by
Conf42
