Notorious B.I.G. Data - Wins and Fails When Building a Big Data Platform in the Cloud

Explore the journey of scaling a big data platform in the cloud through this insightful conference talk. Discover how Unacast evolved from processing a few thousand rows of data monthly to handling billions daily over three years. Learn from the experiences shared by Andreas Heim, Unacast's VP Engineering, as he discusses the company's data and engineering history. Gain valuable insights on avoiding common pitfalls such as non-scalable architectures, overburdened systems, and exponentially increasing costs. Uncover strategies for processing massive amounts of data, maintaining transparency in reporting and costs, ensuring GDPR compliance across teams, and optimizing deployment and developer experience. Apply these lessons to your own big data projects and improve your cloud-based data platform architecture.