Introduction to High-Performance and Parallel Computing

Overview

This course introduces the fundamentals of high-performance and parallel computing. It is targeted to scientists, engineers, scholars, really everyone seeking to develop the software skills necessary for work in parallel software environments. These skills include big-data analysis, machine learning, parallel programming, and optimization. We will cover the basics of Linux environments and bash scripting all the way to high throughput computing and parallelizing code. We recommend you are familiar with either Fortran 90, C++, or Python to complete some of the programming assignments. After completing this course, you will familiar with: *The components of a high-performance distributed computing system *Types of parallel programming models and the situations in which they might be used *High-throughput computing *Shared memory parallelism *Distributed memory parallelism *Navigating a typical Linux-based HPC environment *Assessing and analyzing application scalability including weak and strong scaling *Quantifying the processing, data, and cost requirements for a computational project or workflow This course can be taken for academic credit as part of CU Boulder’s Master of Science in Data Science (MS-DS) degree offered on the Coursera platform. The MS-DS is an interdisciplinary degree that brings together faculty from CU Boulder’s departments of Applied Mathematics, Computer Science, Information Science, and others. With performance-based admissions and no application process, the MS-DS is ideal for individuals with a broad range of undergraduate education and/or professional experience in computer science, information science, mathematics, and statistics. Learn more about the MS-DS program at https://www.coursera.org/degrees/master-of-science-data-science-boulder.

Syllabus

High-Performance Computing (HPC) for Non-Computer Scientists

Get to know the basics of an HPC system. Users will learn how to work with common high performance computing systems they may encounter in future efforts. This includes navigating filesystems, working with a typical HPC operating system (Linux), and some of the basic concepts of HPC. We will also provide users some key information that is specific to the logistics of this course.

Nuts and Bolts of HPC

During this week we will actually begin to use HPC infrastructure. Some concepts we will learn are - how to load software appropriately onto an HPC system, what the different types of nodes a user can expect to encounter on a system, and how to submit a job to conduct work, such as perform calculations.

Basic Parallelism

In this module, we will introduce users to the nuances of memory on a high performance computing system. We will also cover some ways to conduct work on a system most efficiently. We will also introduce some beginning components of parallel programming.

Evaluating Parallel Program Performance

In this module, we will continue to review topics related to using a high performance computing system most efficiently, including scaling your workflow measuring how efficient your work on a system is, and how to utilize as much of the computing resource as possible.