HP-MPI User's Guide (11th Edition)

Introduction
MPI concepts
Chapter 110
Nonblocking calls have the same arguments, with the same meaning as
their blocking counterparts, plus an additional argument for a request.
To code a standard nonblocking send, use
MPI_Isend(void *buf, int count, MPI_datatype dtype, int
dest, int tag, MPI_Comm comm, MPI_Request *req);
where
req Specifies the request used by a completion routine
when called by the application to complete the send
operation.
To complete nonblocking sends and receives, you can use MPI_Wait or
MPI_Test. The completion of a send indicates that the sending process is
free to access the send buffer. The completion of a receive indicates that
the receive buffer contains the message, the receiving process is free to
access it, and the status object, that returns information about the
received message, is set.
Collective operations
Applications may require coordinated operations among multiple
processes. For example, all processes need to cooperate to sum sets of
numbers distributed among them.
MPI provides a set of collective operations to coordinate operations
among processes. These operations are implemented such that all
processes call the same operation with the same arguments. Thus, when
sending and receiving messages, one collective operation can replace
multiple sends and receives, resulting in lower overhead and higher
performance.
MPI_Bsend MPI_Ibsend
MPI_Ssend MPI_Issend
MPI_Rsend MPI_Irsend
MPI_Recv MPI_Irecv
Table 1-2 MPI blocking and nonblocking calls
Blocking
mode
Nonblocking
mode