It is almost impossible to buy a computer that isn't parallel. Even an Apple watch has 2 cores. Some graphics processing chips (GPUs) have more than 1000 specialized compute cores, and some supercomputers incorporate these chips. The Tianhe-2 supercomputer has > 3,000,000 cores and a theoretical peak performance of > 50 Petaflops, i.e., 50,000,000,000,000,000 floating point operations per second. The number of cores/chip will continually increase, and hence parallel computing is needed to make use of whatever system you buy. This is especially true for compute-intensive tasks such as simulations, analyzing large amounts of data, deep learning, or optimizing complicated systems.

*Parallel computers are easy to build - it's the software that takes work.*

- CSE Graduate Students: the software breadth requirement and the general 500-level requirement.
- CSE Undergraduates: "computer oriented technical elective" requirements for the CE and CS degrees.
- Graduate Students outside of CSE: cognate requirements.
- Computational Discovery and Engineering students: it is a core methodology course. Most students in this program should take this course.
- Scientific Computing students: computer science distributional requirements. Most SC students take (and need) this class.
- MS in Data Science: Computation elective.
- CSCS students: an approved course "related to complex systems".
- Various other programs: general computer related distributional requirements.
- All students: you can take it because you want to.

We examine many reasons for poor parallel performance, and a wide range of techniques for improving it. Concepts covered include domain decomposition for geometric and graph-based problems; deterministic, probabilistic and adaptive load balancing; and sychronization. You'll learn why modest parallelization is relatively easy to achieve, and why efficient massive parallelization is quit difficult - in particular, you will continually encounter Amdahl's Law. Examples and programs will be numeric, such as simple stencil calculations for pdes on a grid, and nonnumeric, such as a very simple discrete event simulation, but they do not assume much knowledge in any field and you'll be given all of the requisite information you need to understand the problem. The focus of the class is on techniques and analyses that cut across applications.

Here is a somewhat whimsical overview of parallel computing.

- simulation (such as simulating the climate, the collision avoidance systems on a car, or protein-ligand binding)
- machine learning (such as using GPUs to accelerate neural nets or recommender systems)
- data science (such as algorithms for clustering or pattern matching in massive data sets)
- strategy and optimization (such as programs for playing games or routing traffic)
- software (such as a new technique for optimizing parallel performance)
- algorithms (such as a new theoretical PRAM algorithm for a graph problem, or an MPI implementation of a practical algorithm)
- novel architectures (evaluating quantum computers or synaptic (neuromorphic) systems)
- ?? (students continually come up with topics I never thought of)

- Start thinking in parallel. It's an easy assignment since your brain is a massively parallel system.

Copyright © 2000-2019 Quentin F. Stout |