Automated Multi-Cloud Large Scale Kubernetes Cluster Lifecycle Management

Learn about automated Kubernetes cluster management at scale in this conference talk that details Databricks' system for managing over 1000 cloud-managed Kubernetes clusters across AWS, Azure, and GCP. Explore the implementation of blue-green cluster rotations and cluster swaps that enable major infrastructure changes and version upgrades with minimal risk. Discover how their system incorporates Kubernetes-style continuous reconciliation for managing swap lifecycles, rapid cluster state detection, and workload migration. Gain insights into orchestrating product workloads and cloud provider APIs for automated cluster swaps, while understanding the challenges and benefits of automating large-scale, multi-cloud Kubernetes upgrades. Master the techniques for achieving staged rollouts, seamless rollbacks, and zero-downtime deployments with minimal operator intervention.