Explore a groundbreaking conference talk on Horus, a novel in-network task scheduler designed for cloud datacenters. Delve into the challenges of scheduling short-lived tasks across distributed workers and discover how Horus overcomes limitations of current rack-level schedulers. Learn about its efficient worker state tracking and distribution among switches, enabling parallel task scheduling at line rate while optimizing quality. Examine the innovative distributed task scheduling policy that minimizes overheads, handles dynamic loads, and avoids task buffering in switches. Compare Horus's performance against state-of-the-art in-network schedulers through testbed experiments and large-scale datacenter simulations. Gain insights into how Horus significantly improves tail response time and throughput in diverse and dynamic workload scenarios.
Overview
Syllabus
NSDI '24 - Horus: Granular In-Network Task Scheduler for Cloud Datacenters
Taught by
USENIX