How to Scale Unstructured Data Processing with Ray Data

Discover how to effectively scale unstructured data processing using Ray Data in this 30-minute conference talk from Ray Summit 2024. Join Hao Chen and Praveen Gorthy from Anyscale as they address the growing challenge of handling exponentially increasing unstructured data sizes. Learn about Ray Data on Anyscale's streaming batch model and adaptive scheduling, designed to efficiently manage heterogeneous compute requirements for unstructured data workloads. Explore Anyscale's enhancements to Ray Data, including autoscaling, fault tolerance, and performance optimizations. Watch a live demonstration of developing and scaling an unstructured data processing workflow, and see how Anyscale's observability tools provide real-time insights for pipeline optimization. Gain valuable knowledge for improving performance and cost-efficiency in large-scale unstructured data processing, making this talk essential for data scientists, engineers, and organizations facing similar challenges.