Dive into a comprehensive 31-minute video tutorial on embeddings in natural language processing. Learn the fundamentals of semantic search, how to effectively use embeddings with language models, and explore key concepts like cosine similarity and dot product similarity. Discover practical techniques for embedding datasets, preparing data, and chunking for optimal results. Gain insights into selecting appropriate embedding lengths and implementing Llama 2 13B with GPTQ in Google Colab. Compare various embedding options, including OpenAI and SBERT, and master the process of creating embeddings from data. Evaluate embedding performance using ChatGPT and analyze results with Llama 13-B and GPT-4. Conclude with expert tips for enhancing embedding performance, including the ColBERT approach, to maximize the potential of embeddings in language model applications.
Embeddings vs Fine-Tuning - Part 1: Understanding and Implementing Embeddings
Overview
Syllabus
Should I use embeddings or fine-tuning?
How does semantic search work?
How to use embeddings with a language model?
The two keys to success with embeddings
How do cosine similarity and dot product similarity work?
How to embed a dataset? Touch Rugby Rules
How to prepare data for embedding?
Chunking a dataset for embeddings
What length of embeddings should I use?
Loading Llama 2 13B with GPTQ in Google Colab
Installing Llama 2 13B with GPTQ
Llama Performance without Embeddings
What embeddings should I use?
How to use OpenAI Embeddings
Using SBERT or "Marco" embeddings
How to create embeddings from data.
Calculating similarity using the dot product
Evaluating performance using embeddings
Using ChatGPT to Evaluate Performance of Embeddings
Llama 13-B Incorrect, GPT-4 Correct
Llama 13-B and GPT-4 Incorrect
Embeddings incorrect AND Llama 13B and GPT-4 Hallucinate
Summary of Embeddings Performance with Llama 2 and GPT-4
Pro tips for further improving performance with embeddings
ColBERT approach to improve embeddings
Top Tips for using Embeddings with Language Models
Taught by
Trelis Research
Related Courses
-
Building a Custom OpenAI Chatbot
-
Mastering Retrieval for LLMs - BM25, Fine-tuned Embeddings, and Re-Rankers
-
Embeddings vs Fine-Tuning - Supervised Fine-tuning - Part 2
-
Synthetic Data Generation and Fine-tuning for OpenAI GPT-4 or Llama 3
-
Fine-tuning Llama 2 for Tone or Style Using Shakespeare Dataset
-
LLama 2 and PEFT Documentation - Building Interactive LLM with Retrieval Augmented Generation
