Completed
Choosing VLLM as the Inference Engine
Class Central Classrooms beta
YouTube playlists curated by Class Central.
Classroom Contents
Building Your Own ChatGPT-style LLM AI Infrastructure with Kubernetes
Automatically move to the next video in the Classroom when playback concludes
- 1 John McBride
- 2 Introduction and Background
- 3 Summary of the Blog Post
- 4 The Role of Kubernetes in AI-Enabled Applications
- 5 The Use of TimeScaleDB for Storing Time-Series Data and Vectors
- 6 Migrating to an Open-Source LLM Inference Engine
- 7 Deploying Kubernetes and Setting Up Node Groups
- 8 Choosing VLLM as the Inference Engine
- 9 The Migration Process: Deploying Kubernetes and Setting Up Node Groups
- 10 Choosing the Right Level of Abstraction
- 11 Challenges in Evaluating Language Model Performance
- 12 Considerations for Adopting Kubernetes in Startups
