Completed
Proposed Solution: Multi-model Inference Service
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Serverless Machine Learning Inference with KFServing
Automatically move to the next video in the Classroom when playback concludes
- 1 Intro
- 2 Inference Stack Evolution PYTORCH
- 3 Model explanation, model pre-post transformers
- 4 GPU Autoscaling the challenge
- 5 Challenge: Increase GPU utilization
- 6 Use Case: Personalized News Monitoring
- 7 Challenge: Deploy many models
- 8 Proposed Solution: Multi-model Inference Service
- 9 Experience from running a serverless inference platform
- 10 Reduce tail latency caused by CPU throttling
- 11 Reduce cold start latency
- 12 Monitoring and Alerting: Control Plane
- 13 Monitoring and Alerting: Access logs
- 14 Monitoring and Alerting: Inference Service metrics
- 15 KFServing Roadmap 2020
- 16 Our Working Group is Open
