Overview
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the critical issue of ordering effects in systems performance measurement through this 17-minute conference talk from USENIX ATC '23. Learn why the common assumption of trial independence in performance studies can lead to biased results and incorrect conclusions. Discover the methodology developed for studying ordering effects, including statistical tests for order dependence. Examine findings from a large-scale empirical study involving nearly 2.3 million measurements from over 1,700 servers collected over 9 months. Gain insights into OrderSage, a tool designed to detect and mitigate ordering effects, and review case studies that reveal previously unknown ordering effects in published research. Understand the importance of considering experiment ordering in performance studies to avoid hidden traps and ensure more accurate results in systems research.
Syllabus
USENIX ATC '23 - Avoiding the Ordering Trap in Systems Performance Measurement
Taught by
USENIX