Model Service Mesh: A New Paradigm for Large-Scale AI Model Service Deployment and Management
CNCF [Cloud Native Computing Foundation] via YouTube
Overview
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the innovative Model Service Mesh (MSM) paradigm for large-scale AI model service deployment and management in cloud-native environments. Delve into the challenges posed by growing AI/ML model complexity and scale, and discover how MSM addresses these issues. Learn about building highly scalable and reliable model delivery systems, featuring dynamic model service routing, unified multi-model management within single endpoints, optimized caching layers, and cache-aware scheduling. Understand how MSM optimizes AI model services in lifecycle management, resource utilization, security enhancement, and observability. Gain insights into creating scalable, secure, and efficient model services in cloud-native settings through this 36-minute presentation by Xi Ning Wang from Alibaba Cloud and Huailong Zhang from Intel China at the CNCF conference.
Syllabus
Model Service Mesh: A New Paradigm for Large-Scale AI Model Servic...- Xi Ning Wang & Huailong Zhang
Taught by
CNCF [Cloud Native Computing Foundation]