Cruise.data - A New Dataset Processing Pipeline for Cruise ML

Explore a new ML data pre-processing framework called Cruise.Data in this 31-minute conference talk from Anyscale. Learn how Cruise addresses the challenges of custom data pre-processing for machine learning models in the autonomous vehicle industry. Discover the unique features of Cruise.Data, which combines the best properties of tf.data, the PyTorch ecosystem, and large-scale data processing frameworks. Understand how this innovative system tackles issues such as performance, reliability, and memory usage, particularly when dealing with high-resolution sensor data from autonomous vehicles. Gain insights into how Ray helps scale the Cruise.Data framework and how it enables ML engineers to seamlessly move logic between training and offline batch data processing jobs. Delve into the progress made in building this novel system and its potential impact on improving GPU utilization and overall efficiency in ML model training for autonomous driving applications.