4. Dichotomy of Parallel Computing Platforms

Dichotomy of Parallel Computing Platforms

  • Explicitly parallel programs must define both concurrency and communication between concurrent tasks.

  • Concurrency is expressed at various levels of granularity, from instruction-level parallelism to process-level parallelism.

  • Different models exist for controlling parallelism, each with corresponding architectural support.

Control Structure of Parallel Programs

  • Parallel processing units may operate under centralized control or independently.

  • SIMD (Single Instruction, Multiple Data) and MIMD (Multiple Instruction, Multiple Data) are two primary control models.

SIMD Processors

  • SIMD processors dispatch the same instruction to multiple processors working on different data.

  • Early parallel computers like Illiac IV, MPP, and MasPar MP-1 used this model.

  • Variants like MMX units in Intel processors and DSP chips also use SIMD.

  • SIMD relies on regular computational structures, such as those in image processing.

  • SIMD programming paradigms often include an "activity mask" to selectively enable/disable operations on data items.

MIMD Processors

  • MIMD processors can execute different programs on different processors.

  • SPMD (Single Program, Multiple Data) is a variant where the same program runs on different processors.

  • Platforms like Sun Ultra Servers, SGI Origin Servers, workstation clusters, and IBM SP utilize MIMD architecture.

SIMD-MIMD Comparison

  • SIMD computers require less hardware due to a single control unit.

  • However, SIMD processors are expensive and have long design cycles.

  • Not all applications are suitable for SIMD processors.

  • Platforms supporting SPMD paradigm can be built from off-the-shelf components quickly and at a lower cost.