Distributed Computing Courses and Certifications

Simons Institute

Local Algorithms: Principles and Applications

Explore insights from Tal Herman's presentation at the Workshop on Local Algorithms, delving into cutting-edge research in computer science and mathematics.

• YouTube
• 25 minutes
• On-Demand
• Free Video

Simons Institute

Sparsification for Communication-Efficient Distributed Symmetry-Breaking

Explore sparsification techniques for efficient distributed algorithms, focusing on local symmetry-breaking problems in graph theory and the CONGEST model.

• YouTube
• 32 minutes
• On-Demand
• Free Video

Devoxx Poland

Challenges Running Planet-Wide Computer: Efficiency

Explore challenges and solutions for running efficient planet-wide computer systems, focusing on scalability and performance optimization techniques.

• YouTube
• 40 minutes
• On-Demand
• Free Video

Truly Universal Compute - Using Bacalhau to Deploy Jobs Cross Region, Cross Cloud and to the Edge

Explore Bacalhau, a unified platform for managing distributed compute jobs across regions, clouds, and edge devices, addressing challenges in data availability, scalability, and security.

• YouTube
• 31 minutes
• On-Demand
• Free Video

Ubuntu OnAir

Arras - MoonRay's Distributed Computational Framework for Ray-Trace Rendering

Explore Arras, a scalable framework enabling distributed computations for ray-trace rendering. Learn how it enhances MoonRay's capabilities, allowing for unlimited computational power and accelerating processor-intensive operations in film production.

• YouTube
• 24 minutes
• On-Demand
• Free Video

CppCon

Distributed Ranges: A Model for Building Distributed Data Structures, Algorithms, and Views

Explore distributed ranges for building scalable data structures and algorithms across clusters. Learn to implement generic, interoperable distributed computing solutions using modern C++ features.

• YouTube
• 54 minutes
• On-Demand
• Free Video

USENIX

MegaScale - Scaling Large Language Model Training to More Than 10,000 GPUs

Innovative system for training large language models on 10,000+ GPUs, addressing efficiency and stability challenges through full-stack optimization and advanced diagnostic tools.

• YouTube
• 16 minutes
• On-Demand
• Free Video

USENIX

CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters

Innovative network-aware scheduler for ML clusters, improving job completion times and reducing network congestion through strategic communication pattern interleaving.

• YouTube
• 16 minutes
• On-Demand
• Free Video

Simons Institute

Conditional Hardness for Massively Parallel Computing Via Distributed Lower Bounds

Explore conditional hardness in massively parallel computing through distributed lower bounds, examining theoretical challenges and algorithmic implications for sublinear algorithms.

• YouTube
• 1 hour 2 minutes
• On-Demand
• Free Video

HOTI - Hot Interconnects Symposium

Characterizing Communication in Distributed Parameter-Efficient Fine-Tuning for LLMs

Explore communication patterns and optimization techniques in distributed parameter-efficient fine-tuning for Large Language Models, focusing on network performance and scalability challenges.

• YouTube
• 29 minutes
• On-Demand
• Free Video

HOTI - Hot Interconnects Symposium

Demystifying the Communication Characteristics for Distributed Transformer Models

Explore communication patterns and optimization strategies for distributed transformer models, focusing on collective operations and performance characteristics in large-scale deployments.

• YouTube
• 34 minutes
• On-Demand
• Free Video

Anyscale

Scaling AI Systems: Challenges and Opportunities

Explore cutting-edge AI developments and industry insights in Kim Hazelwood's keynote address at Ray Summit 2022.

• YouTube
• 15 minutes
• On-Demand
• Free Video

USENIX

OPER: Optimality-Guided Embedding Table Parallelization for Large-scale Recommendation Models

Optimality-guided embedding table parallelization for efficient large-scale recommendation model training and inference, achieving significant speedups over existing frameworks.

• YouTube
• 20 minutes
• On-Demand
• Free Video

USENIX

MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale

Intelligent global scheduling system for ML training workloads across datacenters, optimizing resource utilization and eliminating regional overload through innovative design principles.

• YouTube
• 16 minutes
• On-Demand
• Free Video

Advanced Cyberinfrastructure Training at RPI

Research Ops - Challenges and Practical Solutions for Distributed Scientific Computing

Explore challenges and practical solutions for distributed scientific computing in research operations, focusing on advanced cyberinfrastructure.

• YouTube
• 1 hour 25 minutes
• On-Demand
• Free Video