Ray Serve for IoT at Samsara: Scaling Cloud Inference for Computer Vision and Sensor Fusion

Explore how Samsara's machine learning infrastructure team leveraged Ray Serve to scale their cloud IoT inference platform in this 11-minute conference talk. Discover pragmatic best practices for creating production serve clusters, including templates for provisioning, common utilities, and dashboards for observability. Learn how Ray Serve enhanced ML engineering development velocity by simplifying complex pipeline composition for specific customer use cases in the IoT space. Gain insights into Samsara's unique machine learning platform, the transition process for teams familiar with Ray, and the handling of diverse data types for computer vision and sensor fusion. Delve into deployment learnings, covering cluster configuration guidelines, essential metrics to monitor, pipeline composition techniques, and cluster maintenance strategies. Benefit from shared utilities, including Terraform for EKS cluster setup, template configurations, and example inference pipelines, as the team gives back to the community.