Explore advanced multi-GPU programming techniques using MPI and OpenACC in this second lecture of the Advanced OpenACC Course. Delve into Message Passing Interface (MPI) concepts, including MPI skeleton, compiling, and launching. Learn through practical examples like the Jacobi solver, and discover strategies for handling multi-GPU nodes. Gain insights into tools for MPI+OpenACC applications, including memory checking with CUDA-memcheck, debugging, and profiling. Examine GPU acceleration of legacy MPI apps and processes sharing GPUs without MPS. Study Hyper-Q/MPS case studies, focusing on UMT, and understand MPS benefits. Master communication and computation overlap techniques, domain decomposition strategies, and their implementation in multi-GPU environments. Conclude with an exploration of Multi-Process Service (MPS) and Unified Virtual Addressing for enhanced GPU utilization.
Advanced Multi-GPU Programming with OpenACC - Lecture 2
Overview
Syllabus
Intro
ADVANCED MULTI-GPU PROGRAMMING WITH MPI AND OPENACC
MESSAGE PASSING INTERFACE - MPI
MPI - SKELETON
MPI Compiling and Launching
EXAMPLE JACOBI
HANDLING MULTI GPU NODES
TOOLS FOR MPI+OPENACC APPLICATIONS
MEMORY CHECKING WITH CUDA-MEMCHECK
DEBUGGING MPI+OPENACC APPLICATIONS
PROFILING MPI+OPENACC APPLICATIONS
GPU ACCELERATION OF LEGACY MPI APPS
PROCESSES SHARING GPU WITHOUT MPS
HYPER-Q/MPS CASE STUDY: UMT
HYPER-Q/MPS CASE STUDIES
MPS SUMMARY
COMMUNICATION + COMPUTATION OVERLAP
DOMAIN DECOMPOSITION STRATEGIES
EXAMPLE: JACOBI SOLVER Multi GPU with the domain decomposition
DOMAIN DECOMPOSITION WITH TILES
ACCESS TO HOMEWORK
MULTI PROCESS SERVICE (MPS)
UNIFIED VIRTUAL ADDRESSING
Taught by
NVIDIA Developer
