Explore the emerging discipline of LLMOps in this 49-minute video lecture from The Full Stack's LLM Bootcamp. Dive into principles and practices for continuous improvement of large language model-powered applications. Learn to compare and evaluate open source and proprietary models, master workflows and tools for iteration and prompt management, and apply test-driven development principles to LLMs. Gain insights on choosing base LLMs, testing strategies, evaluation metrics, deployment, and monitoring. Access downloadable slides and lecture notes for a comprehensive learning experience. The video includes time-stamped sections covering topics from the basics of LLMOps to advanced concepts in test-driven development for LLMs.
Overview
Syllabus
Why LLMOps?
Choosing your base LLM
Proprietary LLMs
Open-source LLMs
Iteration and prompt management
Testing LLMs: Why and why is it hard?
Testing LLMs: What works?
Evaluation metrics for LLMs
Deployment and monitoring
Test-driven development for LLMs
Taught by
The Full Stack
