User guide

IB0054606-02 A 7-1
7 Virtual Fabric support in
PSM
Introduction
Performance Scaled Messaging (PSM) provides support for full Virtual Fabric
(vFabric) integration, allowing users to specify IB Service Level (SL) and Partition
Key (PKey), or to provide a configured Service ID (SID) to target a vFabric.
Support for using IB path record queries to the QLogic Fabric Manager during
connection setup is also available, enabling alternative switch topologies such as
Mesh/Torus. Note that this relies on the Distributed SA cache from FastFabric.
All PSM enabled MPIs can leverage these capabilities transparently, but only one
MPI ( Open MPI) is configured to support it natively. Native support here means
that MPI specific mpirun switches are available to activate/deactivate these
features. Other MPIs will require use of environment variables to leverage these
capabilities. With MPI applications, the environment variables need to be
propagated across all nodes/processes and not just the node from where the job
is submitted/run. The mechanisms to do this are MPI specific, but for two common
MPIs the following may be helpful:
 Open MPI: Use –x ENV_VAR=ENV_VAL in the mpirun command line.
Example:
mpirun –np 2 –machinefile machinefile -x
PSM_ENV_VAR=PSM_ENV_VAL prog prog_args
 MVAPICH2: Use mpirun_rsh to perform job launch. Do not use mpiexec
or mpirun. Specify the environment variable and value in the mpirun
command line before the program argument.
Example:
mpirun_rsh –np 2 –hostfile machinefile
PSM_ENV_VAR=PSM_ENV_VAL prog prog_args
Some of the features available require appropriate versions of associated
software and firmware for correct operation. These requirements are listed in the
relevant sections.