Unlocking the Lakehouse with Efficient Data Pipelines - Capital One's Journey

Explore a 22-minute conference talk on building efficient data pipelines for a Lakehouse architecture. Dive into Capital One's journey as a pioneer in data-driven digital transformation and cloud-first enterprise. Learn how they implemented a centralized data lake using the open ecosystem model of Lakehouse to fuel data-driven decision-making. Discover key insights on enabling near real-time data ingestion pipelines using Delta Lake and Databricks. Gain valuable best practices and lessons learned for maximizing the potential of the Databricks platform. Presented by Prabodh Mhalgi, Sr. Lead Data Engineer at Capital One, and Usman Zubair, Lead Technologist at Databricks, this talk offers practical knowledge for data professionals looking to enhance their data pipeline strategies.