Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

Growing Pain: Scaling to 10K Workflows per Week

CNCF [Cloud Native Computing Foundation] via YouTube

Overview

Explore a conference talk that delves into the challenges of scaling a centralized multi-tenant workflow orchestration platform at Bloomberg. Learn how the engineering team tackled rapid growth, managing spiky submission rates, unpredictable resource requests, and heavy API server demand. Discover the complexities of implementing throttling and garbage collection in a diverse user environment. Gain insights into the unique challenges of maintaining a global Argo installation and fulfilling varied requirements within a single cluster. Examine potential impacts of these scaling issues and uncover generalizable guardrails and mitigation strategies applicable to other batch workloads.

Syllabus

Growing Pain: Scaling to 10K Workflows per Week - Yao Lin & Harris Khan, Bloomberg

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of Growing Pain: Scaling to 10K Workflows per Week

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.