buffered-blocking

Programming Using the Message Passing Paradigm

Topic Overview

• Principles of Message-Passing Programming

• The Building Blocks: Send and Receive Operations

• MPI: the Message Passing Interface

• Topologies and Embedding

• Overlapping Communication with Computation

• Collective Communication and Computation Operations

• Groups and Communicators

Principles of Message-Passing Programming

• Logical View of a Machine

  • Consists of p processes, each having its own exclusive address space.

  • Each data element must belong to one partition of the space; hence, data must be explicitly partitioned and placed.

  • All interactions (read-only or read/write) require cooperation of two processes:

  • The process holding the data.

  • The process that wants to access the data.

  • These constraints, while onerous, render the underlying costs explicit to the programmer.

• Programming Paradigms

  • Message-passing programs are often written using:

  • Asynchronous Paradigm: All concurrent tasks execute asynchronously.

  • Loosely Synchronous Model: Tasks or subsets synchronize to perform interactions, while executing completely asynchronously between these interactions.

  • Most message-passing programs are written using the Single Program Multiple Data (SPMD) model.

The Building Blocks: Send and Receive Operations

• Prototype semantics for operations:

  • Send: send(void *sendbuf, int nelems, int dest)

  • Receive: receive(void *recvbuf, int nelems, int source)

  • Code Segment Example:
    plaintext P0 P1 a = 100; receive(&a, 1, 0) send(&a, 1, 1); printf("%d\n", a); a = 0;

  • Semantics of the send operation require that the value received by process P1 must be 100 as opposed to 0.

Non-Buffered Blocking Message Passing Operations

• Send Operation Behavior:

  • Non-buffered blocking sends do not return until the matching receive has been encountered by the receiving process.

  • Issues: Idling and deadlocks due to matching points not being reached similarly.

• Buffered Blocking Sends:

  • In buffered blocking sends, the sender copies data into a designated buffer and returns after the copy operation is completed. The data must also be buffered at the receiving end.

  • Buffers alleviate idling overhead at the expense of additional copying overheads.

## Buffered Blocking Message Passing Operations