Leveraging SRE and Observability for Building on LLMs

Explore the intersection of Site Reliability Engineering (SRE) and observability in the context of Large Language Models (LLMs) in this conference talk from Conf42 Incident Management 2023. Delve into the unique challenges and opportunities presented by LLMs, comparing them to traditional APIs while highlighting their increased unpredictability. Examine the concept of observability and its application to LLM-based systems, including instrumentation techniques and emerging behaviors. Learn about implementing Service Level Objectives (SLOs) for LLM development and gain insights from real-world examples such as Duolingo and Intercom. Discover practical strategies for leveraging SRE principles to build more reliable and observable LLM-powered applications.