HP-MPI Version 2.2.5 for Linux Release Note

HP-MPI V2.2.5 for Linux Release Note
Known Problems and Workarounds
38
this, the Mellanox driver does not use copy-on-write behavior during a fork for pinned
pages. Instead, access to these pages by the child process will result in a segmentation
violation.
If you believe that your application is experiencing a problem due to this limitation,
HP-MPI provides a feature that may avoid the scenario from occurring. By setting the
environment variable MPI_PAGE_ALIGN_MEM, HP-MPI will page-align and page-pad libc
memory allocation requests which are large enough to be pinned during MPI message
transfer. This can result in slightly more memory being allocated, but reduces the
likelihood that a fork’d process will write to a page of memory that was also being used
for message transfer when a fork call occurred.
It is still possible to experience problems if you allocate a large buffer, start a message
transfer operation on part of the buffer, fork, and then access another part of the buffer.
Nevertheless, the use of MPI_PAGE_ALIGN_MEM will alleviate the most common problems
related to calling fork( ) while memory is pinned.
• 32-bit apps are not supported on InfiniBand when running on either Opteron AMD64 or
Intel EM64T machines.
• 32-bit applications wishing to use InfiniBand on x86_64 systems should consider using
Sockets Direct Protocol (SDP). Set the LD_PRELOAD environment variable before running
an executable that uses sockets:
% setenv LD_PRELOAD /usr/voltaire/lib/sock-redirect.so
This will automatically and transparently convert to the use of SDP.
• The HP-MPI 2.2.5 library for Linux contains a dependency on pthread_self, which
requires libpthread to be included on the link line if the application is linked -static.
The mpicc, mpif90, etc. compiler wrapper scripts automatically add the necessary
-lpthread, but a user who links by hand will have to explicitly add -lpthread.
• On HP XC clusters using Fortran apps, a call to MPI_Abort results in a SIGTERM being
raised by default. The version of srun included with HP XC V2.1 would not see the signal
if the app was Fortran, and teardown would not occur.
The version of srun included with HP XC V3.0 will teardown correctly by default. This fix
was included in SLURM 0.5.0-pre10. If you are using an older version of srun and you
observe this teardown issue, a workaround is provided with the MPI_ABORT_SRUN_SIGNUM
environment variable. Set this to any signal (for example SIGABRT) and when MPI_Abort
is called, that is the signal it will raise instead of SIGTERM, and srun will correctly
teardown in that case.
• Profiling routines built for C calls will no longer cause the corresponding Fortran calls to
be wrapped automatically. In order to profile Fortran routines, separate wrappers need to
be written for the Fortran calls.