ABOUT THE COURSE:This course on parallel programming on scientific applications will enable the students to write parallel programs. The course will provide a overview of HPC hardware, and then move to Message Passing Interface (MPI) and OpenMP programming. GPU tools, such as OpenACC, CuPy, and CUDA, will be introduced in the course. These topics would useful to all students irrespective of their branch.In the course, the algorithms of applications such as Parallel Computational Fluid Dynamics, Parallel Molecular Dynamics would be covered. The HPC tools will then be employed to the above topics.INTENDED AUDIENCE: PG students of Science and Engineering (Specially Physics, Mathematics, Mechanical, Aerospace, Computer science and Chemical Engineering). Advance UG students too can take this course.PREREQUISITES: Basic knowledge of Python and C is required. Also, some knowledge of basic calculus, matrix algebra, and differential equations will help.INDUSTRY SUPPORT: Companies involved in high-performance computing, data science, computational fluid dynamics, computational electrodynamics, data sciences, finance, etc.
Overview
Syllabus
Week 1:Introduction to CPUs
Week 2:Introduction to Memory/Network//Parallel prog
Week 3:Parallel prog contd.
Week 4:Intro to Python /C, Arrays
Week 5:Speeding up C/Fortran/Python programs:
Week 6:Parallel ApplicationsMultithreading
Week 7:MPI Porgramming
Week 8:CUDA programming
Week 9:Cupy, OpenACC programming
Week 10:MPI/OpenMP in C
Week 11:Applications/Visualization/Profiling
Week 12:Case Study
- Introduction to High Performance Computation
- Basic Design
- Processors
- Vectorization
- Multicore Processors
Week 2:Introduction to Memory/Network//Parallel prog
- Memory
- Basics of Operating System (Missing Video
- RAM
- Interconnect
- Parallel Computer Classification
- Classes of Parallelism
- Networks
Week 3:Parallel prog contd.
- Top10 HPC Systems
- Using GPUs for HPC
- Parallel programming fundamentals
- Scaling
- Programming Practices
Week 4:Intro to Python /C, Arrays
- Programming Language
- Classes in Python
- Inheritance
- Modules in Python
- Python Pitfalls
- Python Arrays
- C Arrays 1D
- Higher-DimensionalC Arrays
Week 5:Speeding up C/Fortran/Python programs:
- Python Codes Optimization 1
- C++ Codes Optimization 1
- Python Codes Optimization 2
- C++ Codes Optimization 2
- C++ Codes Optimization 3 (Cache locality)
- Speeding up Using Numba
Week 6:Parallel ApplicationsMultithreading
- Finite Difference Method
- Particle Simulations (Molecular Dynamics)
- Using Multiprocessing Module
- Cache Issues in Multiprocessing
- Using Multithreading Module
- Computing Sum(a*b) with Multithreads
- Computing AX=Y with Multithreads
Week 7:MPI Porgramming
- MPI Using Mpi4py
- Point-To-Point Communication (1)
- Point-To-Point Communication (2)
- Unblocking Send/Receive
- Collective Communication and Reduction Operation
- Send/Receive for Finite Difference Scheme
- MPI Sum with Examples
Week 8:CUDA programming
- Introduction CUDA Programming
- Introduction to C (1)
- Introduction to C (2)
- CUDA Programming (1)
- CUDA Programming (2)
- Examples Using CUDA Programming
- Matrix Multiplication
- Derivative Computation Using Finite Difference
Week 9:Cupy, OpenACC programming
- Cupy Programming
- CUDA Programming with Numba
- Introduction to Open ACC Programming 1
- Introduction to Open ACC Programming 2
- Solving Laplace Equation / OpenACC UPDATE Directives and Clauses
Week 10:MPI/OpenMP in C
- Introduction to MPI in C
- MPI Environment
- Point to Point Communications
- Collective Communication & Reduction Operation
- Using OPENMP
Week 11:Applications/Visualization/Profiling
- FFT
- Spectral Method
- Profiling
- Paraview
Week 12:Case Study
- Project 1: Cahn-Hilliard Equation
- Project 2: Compressible Flow
- Project 3: Ising Model
- Project 4: Magnetohydrodynamic
- Project 5: Molecular Dynamics
- Project 6: Nonlinear Schrodinger Eq.
- Project 7: XY Model
- Summary
Taught by
Prof. Mahendra Verma
