Learn how to build a powerful multi-modal hybrid search engine for e-commerce using OpenAI's CLIP, BM25, Pinecone vector database, and Python in this comprehensive 35-minute tutorial. Explore the process of creating a search system that handles both text and image-based queries, offering superior results compared to traditional methods. Discover the differences between sparse and dense vectors, and understand how to construct multi-modal embeddings. Follow along as the instructor demonstrates connecting to Pinecone vector database, creating an index, preparing data, and generating BM25 sparse vectors and dense vectors using sentence transformers. Master the techniques for indexing data in Pinecone, making hybrid queries, and fine-tuning results by adjusting the balance between dense and sparse vectors. Additionally, learn how to incorporate product metadata filtering to enhance search functionality. By the end of this tutorial, gain valuable insights into building an advanced e-commerce search engine that can process diverse user queries and deliver highly relevant results.
Overview
Syllabus
Multi-modal hybrid search
Multi-modal hybrid search in e-commerce
How do we construct multi-modal embeddings
Difference between sparse and dense vectors
E-commerce search in Python
Connect to Pinecone vector db
Creating a Pinecone index
Data preparation
Creating BM25 sparse vectors
Creating dense vectors with sentence transformers
Indexing everything in Pinecone
Making hybrid queries
Mixing dense vs sparse with alpha
Adding product metadata filtering
Final thoughts on search
Taught by
James Briggs
