Explore a comprehensive podcast episode featuring Maxime Beauchemin, founder and CEO of Preset, as he delves into the concept of treating prompt engineering more like code. Learn about Promptimize, an innovative tool for scientifically evaluating prompt effectiveness, and discover its parallels with test suites in software engineering. Gain insights into prompt optimization, deterministic evaluation, and the unique challenges in AI prompt engineering. Understand the advantages of open-sourcing tools in the AI domain and the increasing importance of transparent interactions with language models. Dive into discussions on test-driven prompt engineering, the impact of open source in AI development, and the balance between specialized and generalized models in LLMs.
Treating Prompt Engineering More Like Code - Promptimize and Test-Driven Development for AI
Overview
Syllabus
[] Max introducing the Apache Superset project at Preset
[] Max's preferred coffee
[] Airflow creator
[] Takeaways
[] Please like, share, and subscribe to our MLOps channels!
[] Check Max's first MLOps Podcast episode
[] Promptimize
[] Interaction with API
[] Deterministic evaluation of SQL queries and AI
[] Figuring out the right edge cases
[] Reaction with Vector Database
[] Promptomize Test Suite
[] Promptimize vision
[] The open-source blood
[] Impact of open source
[] Dangers of open source
[] AI-Language Models Revolution
[] Test-driven design
[] Prompt tracking
[] Building Test Suites as Assets
[] Adding new prompt cases to new capabilities
[] Monitoring speed and cost
[] Creating own benchmarks
[] AI feature adding more value to the end users
[] Perceived value of the feature
[] LLMs costs
[] Specialized model versus Generalized model
[] Fine-tuning LLMs use cases
[] Classic Engineer's Dilemma
[] Build exciting tech that's available
[] Catastrophic forgetting
[] Promt driven development
[] Wrap up
Taught by
MLOps.community
