Streamlining AI Workflows with Apache Airflow and Ray

Learn how to create a unified AI infrastructure system through the integration of Apache Airflow and Ray in this conference talk from Ray Summit 2024. Discover solutions to common organizational challenges in AI infrastructure orchestration as Venkata Jagannath from Astronomer and Marwan Sarieddine from Anyscale demonstrate a seamless approach to connecting data processing, model training, and inference platforms. Explore an end-to-end machine learning lifecycle implementation that showcases automated LLM fine-tuning, evaluation, serving, and LoRA deployments using custom data. Gain practical insights for optimizing AI operations, reducing development time, and managing costs effectively in production environments through this 21-minute presentation that bridges the gap between disparate AI development and deployment processes.