Delight - Improved Apache Spark UI for Performance Troubleshooting

Explore a 21-minute conference talk introducing Delight, a free and cross-platform monitoring dashboard for Apache Spark. Learn how this tool complements the Spark UI by displaying system metrics and Spark information on a unified timeline. Discover how Delight can be installed using an open-source Spark agent and works with any Spark platform. Follow along as the co-founders of Data Mechanics demonstrate real-world performance troubleshooting sessions, showcasing how Delight and Spark UI can jointly identify application bottlenecks. Gain insights into making data engineering pipelines more cost-effective and stable. The talk covers Delight's vision, its functionality, and future developments in Apache Spark monitoring.