Discover cost-effective strategies for running machine learning inference in the cloud through this 33-minute conference talk from MLOps World: Machine Learning in Production. Explore GPU fractionalization and the use of Spot instances as presented by Oscar Rovira, Co-founder of Mystic AI. Learn about the benefits and limitations of GPU fractionalization, as well as the value and potential challenges of utilizing Spot instances. Gain insights into how combining these approaches can significantly increase throughput and reduce costs for your GenAI applications, with practical examples provided to illustrate these optimization techniques.
Running Multiple Models on the Same GPU on Spot Instances
MLOps World: Machine Learning in Production via YouTube
Overview
Syllabus
Running Multiple Models on the Same GPU, on Spot Instances
Taught by
MLOps World: Machine Learning in Production
Related Courses
-
Large Language Models with Azure
-
Develop Custom Object Detection Models with NVIDIA and Azure Machine Learning
-
MLOps Platforms: Amazon SageMaker and Azure ML
-
Master EC2 Spot Instances: Ultimate Guide to Massive Savings - Part 21
-
How to Reduce ML Computing Costs: Building Efficient Multi-Cloud Infrastructure
-
AI Inference Workloads - Solving MLOps Challenges in Production
