Containerizing and Orchestrating Hadoop Infrastructure at Scale

Explore the journey of containerizing and orchestrating Hadoop infrastructure at scale in this 33-minute conference talk by Mithun (Matt) Mathew from Uber. Discover how Uber's Data team transitioned from operating Hadoop on bare-metal to adopting containers for deployment and automation after five years. Learn about the technical challenges encountered during the containerization process, the design of a zero-downtime migration strategy for a system launching over 20 million containers daily, and how this project transformed the team's culture in efficiently operating Hadoop at Uber. Gain insights into powering batch analytics for various use cases, including trip/delivery ETA prediction and decision-making for promotions and pricing. Reference Uber's 2021 blog post for additional information on this transformative project.