Explore techniques for monitoring Large Language Models (LLMs) in production environments in this insightful conference talk by Claire Longo. Delve into the challenges of troubleshooting models based on unstructured data and learn about innovative approaches to measure vector and embedding drift for image and language models. Discover how to detect differences between unstructured language datasets and understand those differences using techniques like UMAP. Gain valuable insights from real-world testing across billions of streams and use cases, including the application of Euclidean and Cosine distance methods. Benefit from Longo's expertise as Head of ML Solutions Engineering at Arize and her experience in building and scaling central ML engineering teams at companies like Opendoor and Twilio.
Monitoring LLMs in Production: Beyond the Hype - Part 2
Overview
Syllabus
Beyond the Hype: Monitoring LLMs in Production // Claire Longo // LLMs in Prod Conference Part 2
Taught by
MLOps.community
Related Courses
-
Monitoring Unstructured Data - Lightning Session
-
LLMs vs LMs in Production - Key Differences in Using 100M vs 100B Parameter Models
-
The Confidence Checklist for LLMs in Production
-
Product Engineering for LLMs - Panel 2
-
Building LLM Applications for Production
-
Enabling Defense Missions with Local LLMs - Part 2
