HP-MPI Version 2.2.7 for Linux Release Note

Both uDAPL 2.0 and 1.2 can coexist on the same system, and HP-MPI automatically detects
which to use, based on the dat_ia_open call. The MPI_IC_SUFFIXES variable in hpmpi.conf
has several uDAPL_* variables which represent the different libdat2.so and libdat.so
libraries that are dlopened and tested for compatibility with the system /etc/dat.conf entries.
For each library that is loaded, the first ten devices in dat.conf are used in the dat_ia_open
call. If none of those succeed, the next uDAPL library in the MPI_IC_SUFFIXES list is attempted.
Because of the ordering, if uDAPL 1.2 and 2.0 are both available, the uDAPL 2.0 interface will
be selected.
1.2.7.5 XRC support
Extended Reliable Connection (XRC) is a new feature on ConnectX InfiniBand adapters.
Depending on the number of application ranks that are allocated to each host, XRC can
significantly reduce the amount of pinned memory that is used by the InfiniBand driver. Without
XRC, the memory amount is proportional to the number of ranks in the job. With XRC, the
memory amount is proportional to the number of hosts on which the job is being run.
To enable XRC, pass the following options to mpirun:
-srq -e MPI_IBV_XRC=1
OFED version 1.3 or later is required to use XRC.
1.2.7.6 uDAPL 2.0 support
This release adds uDAPL 2.0 spec support. uDAPL 2.0 is not compatible with the previous uDAPL
1.2 version, even at source code level. HP-MPI has built in an automatic version selection based
on the versions specified in the hpmpi.conf file. The default setting in the hpmpi.conf for
uDAPL is libdat2.so libdat.so, which means HP-MPI attempts to use version 2.0 first. If
it fails, then HP-MPI attempts to use version 1.2.
1.2.7.7 iWARP support through uDAPL
HP-MPI supports iWARP through the uDAPL interface. Because of differences in the behavior
of different iWARP systems, HP recommends using HP-MPI 2.2.7 on iWARP. HP-MPI 2.2.7
provides two runtime options to work around these iWARP differences.
HP-MPI 2.2.7 supports both uDAPL 1.0 and 2.0 in OFED 1.3. The user can specify which version
to use, or choose the vendor-specific version. Because iWARP behavior can differ from InfiniBand,
HP recommends HP-MPI 2.2.7 for iWARP hardware.
• During connection setup time, iWARP may require that a first message be sent from the
connect side to the accept side. HP-MPI 2.2.7 has an environment variable to match this
requirement. To send a first message from the connect side to the accept side when running
a uDAPL job, set
-e MPI_UDAPL_MSG1=1
MPI_UDAPL_MSG1 is off by default.
• For iWARP, the byte delivery order within a message is not guaranteed. Use the -sndrcv
option on the mpirun command line to enable the send-receive mode to run uDAPL code.
By default, RDMA is used and requires byte delivery order because the receiver side only
checks memory contents for a new message.
However, these two exceptions may not be applicable, depending on the iWARP card driver. In
that case, the default settings work.
1.2.7.8 InfiniBand card failover
HP–MPI Version 2.2.7 supports InfiniBand card failover. This functionality is always enabled.
An InfiniBand connection is set up between every card pair. During normal operation, short
messages are alternated among the connections in round-robin manner. Long messages are
1.2 What’s in This Version 13