Building a Real-Time Feature Store for Machine Learning at iFood

Explore how iFood, Latin America's largest food tech company, built a real-time feature store using Databricks and Spark Structured Streaming. Discover the innovative data processing pipelines that power machine learning models for order completion time estimation, restaurant recommendations, and fraud detection. Learn about the architecture combining event stream processing, Delta Lake Table storage, and Redis low-latency access clusters. Gain insights into iFood's development processes for creating production-grade, reliable, and validated code. Understand the challenges and solutions in handling large-scale data processing for 26 million monthly orders from over 150,000 restaurants. Delve into topics such as two-step aggregation, sparse streaming, materialization jobs, and fading jobs. Uncover the lessons learned and positive outcomes from implementing this advanced feature store system.