Observing a Large Language Model in Production

Overview

Explore the challenges and solutions of implementing and monitoring a Large Language Model (LLM) in production through this insightful conference talk. Discover how Honeycomb tackled the unique obstacles presented by non-deterministic and inherently unreliable LLM APIs. Learn about effective instrumentation techniques, key performance indicators, and the establishment of Service Level Objectives (SLOs) for LLM-powered features. Gain valuable insights into measuring and iterating on improvements, blending prompt engineering with observability practices to enhance product quality. Acquire practical knowledge on how to effectively monitor and optimize LLM-based features in a production environment, enabling you to build more robust and reliable AI-powered applications.

Syllabus

Observing a Large Language Model in Production - Phillip Carter, Honeycomb

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of Observing a Large Language Model in Production

Taught by

Leveraging SRE and Observability for Building on LLMs

Monitoring Large Language Models in Production - A Scalable and Privacy-Centric Approach

5 Pitfalls of Large Language Models in Production and How to Solve Them

Emerging Architectures for Large Language Model Applications - Building a Custom LLM Application

Never Stop Learning.