Harvesting Memory-bound CPU Stall Cycles in Software with MSH

Explore a groundbreaking 16-minute conference talk from USENIX OSDI '24 that introduces MSH, a novel software system for efficiently harvesting memory-bound CPU stall cycles. Learn how researchers from UC Berkeley and ICSI have developed a solution that outperforms traditional simultaneous multithreading (SMT) implementations by offering fine-grained configurability, minimal latency overhead, and full utilization of stall cycles. Discover the innovative co-design approach involving profiling, program analysis, binary instrumentation, and runtime scheduling that enables MSH to achieve up to 72% harvesting throughput of SMT for latency SLOs where SMT must be disabled. Gain insights into how combining MSH with SMT strategically can lead to even higher throughput, revolutionizing the efficiency of datacenter workloads.