Practical Container Scheduling: Optimizations, Guarantees, and Trade-Offs at Netflix - Lecture

Explore the intricacies of container scheduling in large-scale distributed clusters through this conference talk from Netflix's Senior Software Engineer. Dive deep into the challenges, design, and trade-offs achieved using the open-source scheduling library Fenzo, which takes a holistic approach to provide a nimble scheduling core for various independently evolving clusters. Learn about capacity guarantees, task placement, elasticity, and operational insights for tackling large-scale operations. Discover how Netflix juggles multiple scheduling objectives and constraints, including bin packing, task locality, and capacity guarantees, to efficiently run microservices, batch, and stream processing applications in shared Mesos clusters. Gain valuable insights into multi-goal optimization, cluster autoscaling, and extensibility strategies. Explore fitness functions, hard and soft constraints, and queuing setups used in Netflix's container scheduling process. Understand how to reason about allocation failures and size agent clusters for capacity. This talk provides practical knowledge for engineers working with container scheduling in complex, large-scale environments.