Vision Language Models and PDFs: What You See Is What You Search - Haystack EU 2024

Explore a groundbreaking approach to information extraction from complex PDF documents in this conference talk from Haystack EU 2024. Discover how Vision Language Models (VLMs) are revolutionizing the traditional multi-step process of text extraction, OCR, layout analysis, chunking, and embedding. Learn about ColPali, a new retrieval model that efficiently embeds entire PDF pages, including text, figures, and charts, resulting in improved retrieval quality and a simplified extraction and indexing process. Gain insights into representing ColPali in Vespa and its superior performance on the Visual Document Retrieval (ViDoRe) Benchmark. Benefit from the expertise of Jo Kristian, Chief Scientist at Vespa.ai, as he shares his two decades of experience in building and deploying search and recommender systems.