Developing Data Pipelines with Branch Deployments - A New Approach

Explore a 15-minute conference talk that challenges traditional development patterns for data pipelines. Learn how Enigma leveraged Databricks and Dagster's branch deployments to create a highly productive workflow for developing data pipelines on production data safely. Discover how developers can work on feature branches, push to branch deployments, and utilize cloud infrastructure and branched storage for improved productivity, more trusted data, and lower costs. Gain insights from Ryan Green, CTO of Enigma, on replicating the fast testing and tight feedback loops of traditional software engineering in data engineering. Access additional resources on data engineering and the Databricks Lakehouse Platform, and connect with Databricks through various social media platforms for more information.