User guide

6–SHMEM Description and Configuration
SHMEM Programs
6-6 IB0054606-02 A
By default mpirun is picked up from the path and is assumed to be called
mpirun. Alternatively, the pathname of mpirun can be specified with the
$SHMEM_MPIRUN environment variable. There is also support for integration with
slurm (see Slurm Integration). The following mpirun commands are supported:
 Open MPI: mpirun
 MVAPICH: mpirun and mpirun_rsh
 MVAPICH2: mpirun and mpirun_rsh
If the shmemrun script is run with -show option, it shows that the command line
was used to invoke mpirun, but will not invoke it. Options that specify the number
of processes and the hosts file are mapped by shmemrun to options that are
accepted by the underlying mpirun. The contents of the host file can be parsed
and regenerated if necessary and options to propagate environment variables are
provided. The rationale for this script is to allow you to use the familiar options
from the mpirun chosen and the options will automatically be remapped as
required for the actual mpirun. This makes it possible to write scripts that call
shmemrun without exposing these details of the underlying mpirun command.
If the shmemrun script finds the special -- option while processing the option list,
that option is deleted and subsequent options and command line arguments are
passed through without any modification. Using this option is useful to prevent
shmemrun from modifying options of the program that are being run.
Running programs without using shmemrun
If you do not wish to use this wrapper script, then you must arrange for the
SHMEM libraries to be found at run time using $LD_LIBRARY_PATH or an
equivalent mechanism, and ensure that PSM support is enabled in your MPI
implementation. The libraries can be found at:
$SHMEM_DIR/lib64/$MPI
Where $SHMEM_DIR denotes the top-level directory of the SHMEM
installation, typically /usr/shmem/qlogic, and $MPI is your choice of
MPI (one of mvapich, mvapich2, or openmpi).
Additionally, the PSM receive thread and back-trace must be disabled using the
following commands:
export PSM_RCVTHREAD=0
export IPATH_NO_BACKTRACE=1