RCS Workshop 3: Introduction to Parallel Computing using MATLAB: Glossary

Processors are no longer getting much faster

Parallel computing uses multiple cores in one running program

Sequential Batch uses multiple cores for multiple running programs

Speedup is the ratio of sequential runtime to parallel runtime

Linear speedup is p for p processors

Amdahl’s Law calculates limit on speedup for partially parallel code

Threads share memory and reside in a process on the same computer.

Processes do not share memory and can reside on the same or different computers.

OpenMP is an example of multithreaded programming.

MPI is an example of multiprocess programming.

Communication to share data slows down parallel programs.

Processor hardware caches affects performance.

Transferring data between cores and RAM affects performance.

Reading and writing files in parallel affects performace.

Identify candidate code sections for parallelization

Pay attention to random number generation in parallel

The best way to ensure parallel improves performance is to measure

MATLAB manages a parallel pool of processes.

Variables in parallel loops must be independent.

MATLAB classifies parallel loop variables to ensure independence.

The tic/toc command can time sections of MATLAB code.

Monte Carlo simulation is an example that obtain good speedup in parallel.

Prefix Sum is an example that cannot be parallelized using parfor.

Some code can be parallelized, but speedup may be greatly affected by communication overhead.

Vectorized operations apply to an entire matrix or vector.

MATLAB automatically runs some vectorized code in parallel.

Vectorization can be easier and efficient in some cases.

Vectorization may use too much memory for large problems, or not be possible for complex loops.

Preallocating variables, moving constants out of loops, compilation, and memory access patterns are other MATLAB performance optimization techniques.