Scaling Monitoring at Databricks - From Prometheus to M3

Explore the journey of scaling monitoring infrastructure at Databricks, transitioning from Prometheus to M3. Learn about the initial architecture, M3 setup process, migration challenges, and valuable lessons learned. Gain insights into memory usage optimization, internal dashboard development, and strategies for handling upgrades, updates, and metric spikes. Discover capacity planning techniques and future plans for monitoring at Databricks. This comprehensive talk provides a deep dive into the complexities of large-scale monitoring systems and offers practical solutions for similar scaling challenges.