Distributed Training Courses and Certifications

Anyscale

Best Practices for Productionizing Distributed Training with Ray Train

Explore fault-tolerant distributed training techniques using Ray Train, covering experiment restoration, node failure recovery, cloud storage snapshots, and large model checkpointing for efficient AI workloads.

• YouTube
• 30 minutes
• On-Demand
• Free Video

USENIX

Accelerating Neural Recommendation Training with Embedding Scheduling

Optimizing deep learning recommendation models through embedding scheduling, reducing communication overhead and improving training performance without compromising accuracy.

• YouTube
• 15 minutes
• On-Demand
• Free Video

USENIX

Metis - Fast Automatic Distributed Training on Heterogeneous GPUs

Efficient distributed training system for heterogeneous GPUs. Optimizes parallelism plans, improves resource utilization, and achieves significant speed-ups for large deep learning models.

• YouTube
• 20 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training and Gradient Compression - Lecture 14

Explore distributed training challenges, focusing on bandwidth and latency bottlenecks. Learn gradient compression techniques and delayed gradient averaging to optimize communication in deep learning.

• YouTube
• 58 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training and Gradient Compression - Part I - Lecture 13

Explore distributed training of neural networks, focusing on data parallelism and model parallelism techniques for efficient machine learning on resource-constrained devices.

• YouTube
• 1 hour 1 minute
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training and Gradient Compression - Lecture 14

Explore distributed training challenges, focusing on bandwidth and latency bottlenecks. Learn gradient compression techniques and delayed gradient averaging to optimize communication in deep learning systems.

• YouTube
• 57 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training and Gradient Compression - Part I - Lecture 13

Explore distributed training of neural networks, focusing on data parallelism and model parallelism techniques for efficient machine learning on resource-constrained devices.

• YouTube
• 1 hour 1 minute
• On-Demand
• Free Video

Efficient Data Parallel Distributed Training with Flyte, Spark and Horovod

Learn to optimize distributed training using Flyte, Spark, and Horovod for efficient data parallelism in machine learning workflows.

• YouTube
• 41 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training for Efficient Machine Learning - Part II - Lecture 18

Explore advanced distributed training techniques for machine learning, focusing on efficiency and scalability in large-scale systems.

• YouTube
• 55 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training: Techniques and Strategies - Part II - Lecture 18

Explore advanced distributed training techniques for machine learning, focusing on efficiency and scalability in large-scale AI systems.

• YouTube
• 55 minutes
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training - Part I - Lecture 17

Explore distributed training techniques for efficient machine learning, covering key concepts and strategies for scaling ML models across multiple devices.

• YouTube
• 1 hour 1 minute
• On-Demand
• Free Video

MIT HAN Lab

Distributed Training for Efficient Machine Learning - Part I - Lecture 17

Explore distributed training techniques for machine learning, focusing on data and model parallelism strategies to enhance efficiency and scalability.

• YouTube
• 1 hour 1 minute
• On-Demand
• Free Video

Operationalizing Machine Learning on SageMaker

Learn advanced techniques for deploying professional ML projects on SageMaker, covering resource management, distributed training, high-throughput pipelines, and security in AWS.

• Udacity
• 4 weeks, 4-5 hours a week
• On-Demand
• Paid Course

Custom and Distributed Training with TensorFlow

412 ratings at Coursera

Explore advanced TensorFlow techniques for custom training loops, graph mode optimization, and distributed training on GPUs and TPUs. Gain flexibility and efficiency in model development.

• Coursera
• 1 day 31 minutes
• On-Demand
• Free Online Course (Audit)

Intel

Intel® Solutions Pro – AI in the Cloud

Explore AI workloads in the cloud, covering topics like AI pipelines, performance benchmarking, instance selection, and Federated Learning. Gain hands-on experience through online labs and technical insights.

• Coursera
• 7 hours 25 minutes
• On-Demand
• Free Online Course (Audit)