HP-MPI Version 2.2.5 for Linux Release Note

HP-MPI V2.2.5 for Linux Release Note
Known Problems and Workarounds
37
When a background HP-MPI job is run and the shell is terminated, the job may or may
not continue depending on the actual shell used. For /bin/bash, the job is killed. For
bin/sh and /bin/ksh, the job continues. If 'nohup' is used when launching the job, only
background ksh jobs can continue. This behavior may vary depending on your system.
• MPI_IC_ORDER must provide the same definition on every node in a cluster to be effective.
In an HP XC cluster using srun, the environment variables are automatically propagated
by srun. In appfile mode however, the user must explicitly propagate those environment
variables via -e.
% mpirun -e MPI_IC_ORDER="vapi:TCP" -f appfile -prot
• ISV applications that distribute HP-MPI 2.1.1 or earlier for Linux, including HP XC
systems, have a supported limit of 256 ranks in performance. If a customer’s ISV
application requires more than 256 ranks, the application may encounter a startup
limitation. In order to properly run a job with more than 256 ranks, the ISV will need to
migrate to HP-MPI 2.1.2 or a later release. Please contact your ISV representative to
determine when an updated version of your ISV application will be available.
• Interval timer functionality used by HP-MPI on HP XC can conflict with gprof data
collection phase requirements. Set the following two environment variables to
workaround this issue.
% export MPI_FLAGS=s0
% export GMON_OUT_PREFIX=/tmp/app_name
In the above example, setting MPI_FLAGS disables HP-MPI’s conflicting use of interval
timers. Refer to the mpienv(1) man page for descriptions of MPI_FLAG settings. Note that
this setting also disables message progression monitoring, so use with well-behaved
programs only.
In the above example, the second setting causes gprof data collection files to be named
/tmp/app_name.PID (where PID is the process ID number). The prefix is set arbitrarily
and makes the file unique in cases where the same PID is given on different nodes.
• Setting LD_ASSUME_KERNEL to 2.4.1 will cause HP-MPI to fail on 32-bit apps run on
Opteron or EM64T with RedHat 3.0. HP-MPI does not recommend setting
LD_ASSUME_KERNEL for 32-bit apps.
• At the time of this release, the Mellanox InfiniBand driver has issues with buffers sharing
pages when fork( ) is used. Pinned (locked in memory) pages are normally marked
copy-on-write during a fork. If a page is pinned before a fork and subsequently written to
while RDMA operations are being performed on the same page, silent data corruption can
occur as RDMA operations continue to stream data to a page that has moved. To avoid