HP-MPI Version 2.3.1 for Linux Release Note

To increase the maximum allowed shared memory segment size, enter the following
command as root:
# /sbin/sysctl -w kernel.shmmax=<size in bytes>
To make changes to the kernel.shmmax setting persist across a reboot, add the following
line to the /etc/sysctl.conf file:
kernel.shmmax=<size in bytes>
4.11 Using MPI_FLUSH_FCACHE
The MPI_FLUSH_FCACHE environment variable is silently ignored if the -cpu_bind
option is not specified. This limitation will be removed in a future release. See the
mpienv(1) manpage for more information.
4.12 Using MPI_REMSH
HP-MPI uses the $MPI_REMSH command to launch on remote machines. The commands
constructed are of the general form "$MPI_REMSH <host> -n <command>". If a
remote shell command is desired for which the -n in the above syntax is not appropriate,
you can use a shell script such as the following for the MPI_REMSH command:
#!/bin/sh
host="$1"
shift
shift
ssh $host "$@"
4.13 Increasing Pinned Memory
InfiniBand requires pages to be pinned (locked in memory) for message passing. This
can become a problem when a child process is forked and a pinned page exists in both
the parent's and child's address spaces. Normally a copy-on-write would occur when
one of the processes touches memory on a shared page, and the virtual to physical
mapping would change for that process. In the context of InfiniBand, such a change in
the mapping results in data corruption when an RDMA sends data to the original
physical address.
OFED 1.2 and later (with a fork safety mode enabled) avoids this problem by not using
copy-on-write behavior during a fork for pinned pages. Instead, any access to these
pages by the child process results in a segmentation violation of the child, and the
parent's mapping remains unchanged so that the parent can continue running normally
with no data corruption.
HP-MPI turns on this option by default when the IBV or uDAPL protocols are being
used. If the fork safety mode is not desired, you can turn it off with the MPI environment
variable MPI_IBV_NO_FORK_SAFE=1.
By setting the environment variable MPI_PAGE_ALIGN_MEM=1, HP-MPI page-aligns
and page-pads libc memory allocation requests that are large enough to be pinned
4.11 Using MPI_FLUSH_FCACHE 29