Full Lifecycle Data Analysis on a Large-scale Leadership Supercomputer

Explore a comprehensive analysis of six years' worth of data from Sunway TaihuLight, the world's 11th-fastest supercomputer, in this 20-minute conference talk from USENIX ATC '24. Delve into the challenges faced by application developers and system administrators in managing complex supercomputer architectures. Gain valuable insights into operational management strategies for High-Performance Computing (HPC) systems, including issues like job hanging and starvation. Examine I/O workload characteristics, such as getattr operation spikes and massive file access patterns. Learn about the methodology, findings, and significance of this study, which analyzed 40 TB of I/O performance data and job running information from a supercomputer with 41,508 nodes. Discover potential applications of this research for future studies and practices in the HPC domain.