HP-MPI Version 2.3.1 for Linux Release Note
Table Of Contents
- HP-MPI V2.3.1 for Linux Release Note
- Table of Contents
- 1 Information About This Release
- 2 New or Changed Features in V2.3.1
- 3 New or Changed Features in V2.3
- 3.1 Options Supported Only on HP Hardware
- 3.2 System Check
- 3.3 Default Message Size Changed For -ndd
- 3.4 MPICH2 Compatibility
- 3.5 Support for Large Messages
- 3.6 Redundant License Servers
- 3.7 License Release/Regain on Suspend/Resume
- 3.8 Expanded Functionality for -ha
- 3.8.1 Support for High Availability on InfiniBand Verbs
- 3.8.2 Highly Available Infrastructure (-ha:infra)
- 3.8.3 Using MPI_Comm_connect and MPI_Comm_accept
- 3.8.4 Using MPI_Comm_disconnect
- 3.8.5 Instrumentation and High Availability Mode
- 3.8.6 Failure Recover (-ha:recover)
- 3.8.7 Network High Availability (-ha:net)
- 3.8.8 Failure Detection (-ha:detect)
- 3.8.9 Clarification of the Functionality of Completion Routines in High Availability Mode
- 3.9 Enhanced InfiniBand Support for Dynamic Processes
- 3.10 Singleton Launching
- 3.11 Using the -stdio=files Option
- 3.12 Using the -stdio=none Option
- 3.13 Expanded Lightweight Instrumentation
- 3.14 The api option to MPI_INSTR
- 3.15 New mpirun option -xrc
- 4 Known Issues and Workarounds
- 4.1 Running on iWarp Hardware
- 4.2 Running with Chelsio uDAPL
- 4.3 Mapping Ranks to a CPU
- 4.4 OFED Firmware
- 4.5 Spawn on Remote Nodes
- 4.6 Default Interconnect for -ha Option
- 4.7 Linking Without Compiler Wrappers
- 4.8 Locating the Instrumentation Output File
- 4.9 Using the ScaLAPACK Library
- 4.10 Increasing Shared Memory Segment Size
- 4.11 Using MPI_FLUSH_FCACHE
- 4.12 Using MPI_REMSH
- 4.13 Increasing Pinned Memory
- 4.14 Disabling Fork Safety
- 4.15 Using Fork with OFED
- 4.16 Memory Pinning with OFED 1.2
- 4.17 Upgrading to OFED 1.2
- 4.18 Increasing the nofile Limit
- 4.19 Using appfiles on HP XC Quadrics
- 4.20 Using MPI_Bcast on Quadrics
- 4.21 MPI_Issend Call Limitation on Myrinet MX
- 4.22 Terminating Shells
- 4.23 Disabling Interval Timer Conflicts
- 4.24 libpthread Dependency
- 4.25 Fortran Calls Wrappers
- 4.26 Bindings for C++ and Fortran 90
- 4.27 Using HP Caliper
- 4.28 Using -tv
- 4.29 Extended Collectives with Lightweight Instrumentation
- 4.30 Using -ha with Diagnostic Library
- 4.31 Using MPICH with Diagnostic Library
- 4.32 Using -ha with MPICH
- 4.33 Using MPI-2 with Diagnostic Library
- 4.34 Quadrics Memory Leak
- 5 Installation Information
- 6 Licensing Information
- 7 Additional Product Information

NOTE: Do not use MPICH compatibility mode to produce a single executable to run
under both MPICH and HP-MPI.
For more information on MPICH compatibility, see the HP-MPI User's Guide.
3.5 Support for Large Messages
The current MPI standard allows the data transferred using standard API calls to be
greater than 2 GB. For example, if you call MPI_Send() that contains a count of 1024
elements that each have a size of 2049 KB, the resulting message size in bytes is greater
than what could be stored in a signed 32-bit integer. In some cases, previous versions
of HP-MPI handled these incorrectly. HP-MPI V2.3 has been updated to fix these
deficiencies.
Additionally, some users working with extremely large data sets on 64-bit architectures
need to explicitly pass a count that is greater than the size of a 32-bit integer. The current
MPI-2.1 standard does not accommodate this option. Until the standards committee
releases a new API that does, HP-MPI V2.3 provides new APIs to handle large message
counts. These new APIs are extensions to the MPI-2.1 standard and will not be portable
across other MPI implementations. These new APIs contain a trailing L. For example,
to pass a 10 GB count to an MPI send operation, MPI_SendL() must be called, not
MPI_Send().
IMPORTANT: These interfaces will be deprecated when official APIs are included in
the MPI standard.
The other API through which large integer counts can be passed into HP-MPI calls is
the Fortran autodouble -i8 interface (which is also nonstandard). This interface has
been supported in previous HP-MPI releases, but historically had the limitation that
the values passed in must still fit in 32-bit integers because the large integer input
arguments were cast down to 32-bit values. For HP-MPI V2.3, that restriction is removed.
To enable HP-MPI support for these extensions to the MPI-2.1 standard,
-non-standard-ext must be added to the command line of the HP-MPI compiler
wrappers (mpiCC, mpicc, mpif90, mpif77), as in the following example:
% /opt/hpmpi/bin/mpicc -non-standard-ext large_count_test.c
The -non-standard-ext flag must be passed to the compiler wrapper during the
link step of building an executable.
For a complete list of the new large message interfaces supported, see Appendix A of
the HP-MPI V2.3 Release Note at:
http://docs.hp.com/en/T1919-90017/T1919-90017.pdf
3.5 Support for Large Messages 15