Learn how Alibaba achieves exceptional network stability in large-scale computing clusters through a technical presentation that explores advanced monitoring and control techniques. Dive into the unique challenges posed by AI/ML training synchronization operators and discover how the 'DC as a computer' philosophy guides the implementation of unified monitoring systems. Explore high-precision flow monitoring techniques that enable sub-millisecond granularity flow-based rate statistics for identifying micro-congestion points and optimizing congestion control. Understand the implementation of Alternate Marking DSCP (A.M.D) technology and its role in detecting and locating packet loss events across the network with second-level precision.
Network Stability Challenges and Solutions in Large-Scale Computing Clusters
Open Compute Project via YouTube
Overview
Syllabus
The Challenges and Practices of Network Stability in Alibabas Large Scale Computing Clusters
Taught by
Open Compute Project
Related Courses
-
AI Network Challenges and Solutions for High-Performance Computing
-
Large-Scale Deployment of Data Center TCP (DCTCP) - Challenges and Experiences
-
Automated Congestion Management in AI Data Centers - Network Optimization with DCQCN
-
PANAMA: In-Network Aggregation for Shared Machine Learning Clusters
-
Automated Pipeline for Large-Scale Neural Network Training and Inference
-
Demystifying AI and ML Infrastructure for Network Engineers
