HP-MPI Version 2.3.1 for Linux Release Note
Table Of Contents
- HP-MPI V2.3.1 for Linux Release Note
- Table of Contents
- 1 Information About This Release
- 2 New or Changed Features in V2.3.1
- 3 New or Changed Features in V2.3
- 3.1 Options Supported Only on HP Hardware
- 3.2 System Check
- 3.3 Default Message Size Changed For -ndd
- 3.4 MPICH2 Compatibility
- 3.5 Support for Large Messages
- 3.6 Redundant License Servers
- 3.7 License Release/Regain on Suspend/Resume
- 3.8 Expanded Functionality for -ha
- 3.8.1 Support for High Availability on InfiniBand Verbs
- 3.8.2 Highly Available Infrastructure (-ha:infra)
- 3.8.3 Using MPI_Comm_connect and MPI_Comm_accept
- 3.8.4 Using MPI_Comm_disconnect
- 3.8.5 Instrumentation and High Availability Mode
- 3.8.6 Failure Recover (-ha:recover)
- 3.8.7 Network High Availability (-ha:net)
- 3.8.8 Failure Detection (-ha:detect)
- 3.8.9 Clarification of the Functionality of Completion Routines in High Availability Mode
- 3.9 Enhanced InfiniBand Support for Dynamic Processes
- 3.10 Singleton Launching
- 3.11 Using the -stdio=files Option
- 3.12 Using the -stdio=none Option
- 3.13 Expanded Lightweight Instrumentation
- 3.14 The api option to MPI_INSTR
- 3.15 New mpirun option -xrc
- 4 Known Issues and Workarounds
- 4.1 Running on iWarp Hardware
- 4.2 Running with Chelsio uDAPL
- 4.3 Mapping Ranks to a CPU
- 4.4 OFED Firmware
- 4.5 Spawn on Remote Nodes
- 4.6 Default Interconnect for -ha Option
- 4.7 Linking Without Compiler Wrappers
- 4.8 Locating the Instrumentation Output File
- 4.9 Using the ScaLAPACK Library
- 4.10 Increasing Shared Memory Segment Size
- 4.11 Using MPI_FLUSH_FCACHE
- 4.12 Using MPI_REMSH
- 4.13 Increasing Pinned Memory
- 4.14 Disabling Fork Safety
- 4.15 Using Fork with OFED
- 4.16 Memory Pinning with OFED 1.2
- 4.17 Upgrading to OFED 1.2
- 4.18 Increasing the nofile Limit
- 4.19 Using appfiles on HP XC Quadrics
- 4.20 Using MPI_Bcast on Quadrics
- 4.21 MPI_Issend Call Limitation on Myrinet MX
- 4.22 Terminating Shells
- 4.23 Disabling Interval Timer Conflicts
- 4.24 libpthread Dependency
- 4.25 Fortran Calls Wrappers
- 4.26 Bindings for C++ and Fortran 90
- 4.27 Using HP Caliper
- 4.28 Using -tv
- 4.29 Extended Collectives with Lightweight Instrumentation
- 4.30 Using -ha with Diagnostic Library
- 4.31 Using MPICH with Diagnostic Library
- 4.32 Using -ha with MPICH
- 4.33 Using MPI-2 with Diagnostic Library
- 4.34 Quadrics Memory Leak
- 5 Installation Information
- 6 Licensing Information
- 7 Additional Product Information
during MPI message transfer. This results in slightly more memory being allocated,
but reduces the likelihood that a forked process writes to a page of memory that was
also being used for message transfer when a fork call occurred.
4.14 Disabling Fork Safety
Applications running on Linux systems with kernels earlier than 2.6.12 might display
the following warning message:
libibverbs: Warning: fork()-safety requested but init failed
This warning message appears because the HP-MPI library enabling the OFED 1.2 fork
safety feature is not supported by Linux kernels earlier than 2.6.12. It does not affect
the application run. To disable HP-MPI fork safety, set the environment variable
MPI_IBV_NO_FORK_SAFE, as in the following example:
% /opt/hpmpi/bin/mpirun -np 4 -prot -e MPI_IBV_NO_FORK_SAFE=1 \
-hostlist nodea,nodeb,nodec,noded /my/dir/hello_world
4.15 Using Fork with OFED
Applications using fork() might crash on configurations with InfiniBand using OFED
on kernels earlier than v2.6.18. You can avoid known problems with fork() and OFED
in any of the following ways:
• Run on XC V3.2.1 or later, where all known OFED fork() fixes have been made.
• Run on a system with kernel 2.6.18 or later.
• Run InfiniBand with non-OFED drivers. (This option is not available on
configurations with ConnectX InfiniBand Host Channel Adapters where OFED is
required.)
4.16 Memory Pinning with OFED 1.2
The initial release of OFED 1.2 contains a bug that causes the memory pinning function
to fail after certain patterns of malloc and free. The symptom, which is visible from
HP-MPI, might be any of several error messages such as:
> prog.x: Rank 0:1: MPI_Get: Unable to pin memory for put/get
This bug has already been fixed in OFED 1.3, but if you are running with the initial
release of OFED 1.2, the only workaround is to set MPI_IBV_NO_FORK_SAFE=1.
4.17 Upgrading to OFED 1.2
When upgrading to OFED 1.2 from earlier versions, the installation script might not
stop the previous OFED version before uninstalling it. Therefore, stop the old OFED
stack before upgrading to OFED 1.2. For example:
/etc/init.d/openibd stop
30 Known Issues and Workarounds