HP-MPI Version 2.2.5.1 for Linux Release Note Manufacturing Part Number: T1919-90013 E0807 Release © Copyright 1979-2007 © Hewlett-Packard Development Company, L.P.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental, or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
Intel® and Itanium® are trademarks or registered trademarks of the Intel Corporation or its subsidiaries in the United States and other countries. AMD Opteron and combinations thereof are trademarks of Advanced Micro Devices. FLEXlm® is a registered trademark of Globetrotter Software and Macrovision Corporation. SilverStorm is a trademark of SilverStorm Technologies, Inc. Mellanox is a registered trademark of Myricom, Inc. InfiniBand is a trademark and service mark of the InfiniBand Trade Association.
HP-MPI V2.2.5.1 for Linux Release Note Announcement HP-MPI V2.2.5.1 for Linux Release Note Announcement This release note describes the release of HP-MPI Version 2.2.5.1 for Linux. HP-MPI 2.2.5.1 is the August 2007 release of HP-MPI, Hewlett-Packard’s fully functional implementation of the Message Passing Interface standard for Linux. HP-MPI 2.2.5.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version What’s in This Version HP-MPI is a high-performance implementation of the Message Passing Interface standard. HP-MPI complies fully with the MPI-2 standard. HP-MPI provides an application programming interface and software libraries to support parallel, message-passing applications that are efficient, portable, and flexible. HP-MPI is available as both archive and shared libraries. Use of shared libraries is recommended.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version Table 1 Systems Supported (Continued) Platform Interconnects HP XC3000 Clusters Myrinet InfiniBand Ethernet* HP XC4000 Clusters Myrinet InfiniBand Elan4 Ethernet* HP XC6000 Clusters InfiniBand Elan4 Ethernet* * Ethernet includes 10baseT, 100baseT, GigE, and 10GbE. NOTE 32-bit applications are not supported on Mellanox InfiniBand (VAPI) when running on Opteron AMD64 or Intel EM64T machines.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version Table 2 Protocol Interconnect Support Matrix for V2.2.5.1 Option Supported OS Supported Platforms NIC Version IB Driver Version Shared memory on SMP N/A Linux 2.4, 2.6 Kernels IA64, i386, x86_64 N/A N/A OpenFabrics -IBV Linux 2.6 Kernel IA64, i386, x86_64 Any IB card OFED 1.0, 1.1, 1.2 Mellanox VAPI -VAPI Linux 2.4, 2.6 Kernels IA64, i386, x86_64 Any Mellanox card VAPI 3.2, 4.1 uDAPL Standard -UDAPL Linux 2.4, 2.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version • Portland Group 6.2, 7.0 Directory Structure All HP-MPI for Linux files are stored in the /opt/hpmpi directory. The directory structure is organized as described in Table 3. If you choose to move the HP-MPI installation directory from its default location in /opt/hpmpi, set the MPI_ROOT environment variable to point to the new location.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version • Technical application providers using HP-MPI protect their software investment using a highly efficient, portable tool for parallel application development. HP-MPI 2.2.5.1 for Linux includes the following features (which are described in detail in the next section): • OFED V1.2 support • Enhanced CPU bind support Description of Features The following section provides brief descriptions of the features included in this release.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version ll Bind each rank to CPU each is currently running on. For NUMA-based systems, the following options are also available: ldom Schedule ranks on ldoms according to packed rank id. cyclic Cyclic dist on each ldom according to packed rank id. block Block dist on each ldom according to packed rank id. rr Same as cyclic, but consider ldom load average. fill Same as block, but consider ldom load average.
HP-MPI V2.2.5.1 for Linux Release Note What’s in This Version % mpirun -e MPI_CPU_SPIN=4 -cpu_bind -np 32 ./linpack • MPI_CPU_AFFINITY is an alternative method to using -cpu_bind on the command line for specifying binding strategy. The possible settings are LL, RANK, MAP_CPU, MASK_CPU, LDOM, CYCLIC, BLOCK, RR, FILL, PACKED, SLURM, AND MAP_LDOM. • MPI_BIND_MAP allows specification of the integer CPU numbers, ldom numbers, or CPU masks. These are a list of integers separated by commas (,).
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds Known Problems and Workarounds • InfiniBand requires memory to be pinned (locked in memory) for message passing. This can become a problem when a child process is forked and a pinned page exists in both the parent's and child's address spaces. Normally a copy-on-write would occur when one of the processes touches memory on a shared page, and the virtual to physical mapping would change for that process.
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds • The initial release of OFED 1.2 contains a bug which causes the memory pinning function to fail after certain patterns of malloc and free. The symptom which is visible from HP-MPI could be any of several error messages such as: > prog.x: Rank 0:1: MPI_Get: Unable to pin memory for put/get This bug has already been fixed in OFED, but if you are running with the initial release of OFED 1.
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds • The nofile limit on large linux clusters needs to be increased in /etc/security/limits.conf % soft nofile 1024 For larger clusters, we recommend — 2048 for clusters of 1900 cores or less — 4096 for clusters of 3800 cores or less — 8192 for clusters of 7600 cores or less — etc. • In order to use appfiles on HP XC Elan clusters, MPI_USESRUN must be set to 1 and the line of the appfile may only differ in host name and rank count.
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds When a background HP-MPI job is run and the shell is terminated, the job may or may not continue depending on the actual shell used. For /bin/bash, the job is killed. For /bin/sh and /bin/ksh, the job continues. If 'nohup' is used when launching the job, only background ksh jobs can continue. This behavior may vary depending on your system.
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds HP-MPI complies with the MPI-1.2 standard, which defines bindings for Fortran 77 and C, but not Fortran 90 or C++. HP-MPI also complies with the C++ binding definitions detailed in the MPI-2 standard. However, the C++ bindings provided are not thread safe and should not be used with the HP-MPI threaded libraries (i.e. libmtmpi). HP-MPI does not provide bindings for Fortran 90.
HP-MPI V2.2.5.1 for Linux Release Note Known Problems and Workarounds Try setting the environment variable LIBELAN_TPORT_BIGMSG to an appropriate message size to resolve the resource issue. If this doesn’t eliminate the error, contact Quadrics for the fix (which is unavailable at the time of this release).
HP-MPI V2.2.5.1 for Linux Release Note Installation Requirements Installation Requirements HP-MPI 2.2.5.1 for Linux is supported on HP ProLiant and HP Integrity servers running Red Hat Enterprise Linux AS 3, 4, and 5, or SuSE Linux Enterprise Server 9 and 10 operating systems; as well as HP XC3000, HP XC4000 and HP XC6000 Clusters. You must install the correct HP-MPI product for your system. HP-MPI requires a minimum of 90MB of disk space in /opt.
HP-MPI V2.2.5.1 for Linux Release Note Installation Instructions Installation Instructions 1. Become root by running the su command and entering the superuser password. 2. Place the downloaded file into the /tmp directory (or another suitable directory). 3. Run the rpm command to install the package. % rpm -ivh /tmp/hpmpi-2.02.05.01-20070708r.[i386|ia64|x86_64].rpm Or use the following command to overwrite a previous version: % rpm -Uvh /tmp/hpmpi-2.02.05.01-20070708r.[i386|ia64|x86_64].
HP-MPI V2.2.5.1 for Linux Release Note Licensing Policy Licensing Policy HP-MPI 2.2.5.1 for Linux uses FLEXlm® licensing technology. A license is required to use HP-MPI for Linux. Licenses can be purchased from HP’s software depot at http://www.hp.com/go/softwaredepot, or contact your HP representative. Demo licenses for HP-MPI are also available from HP’s software depot. No HP-MPI license is required at runtime on an HP XC system. Licensing HP-MPI 2.2.5.1 for Linux uses FLEXlm licensing technology.
HP-MPI V2.2.5.1 for Linux Release Note Licensing Policy If the license needs to be placed in another location which would not be found by the above search, the user may set the environment variable LM_LICENSE_FILE to explicitly specify the location of the license file. For more information, see http://licensing.hp.com. Installing License Files A valid license file contains the system hostid and the associated license key. License files can be named either as license.dat or any name with extension of *.
HP-MPI V2.2.5.1 for Linux Release Note Licensing Policy SERVER myserver 0014c2c1f34a DAEMON HPQ INCREMENT HP-MPI HPQ 1.0 permanent 16 BE468B74B592 \ NOTICE="License Number = AAAABBBB2222" SIGN=9AB4034C6CB2 The result is a valid license for 24 ranks.
HP-MPI V2.2.5.1 for Linux Release Note Product Documentation Product Documentation The HP-MPI Documentation Kit is an optional product (product number B6281AA) consisting of the following hardcopy books: • MPI: The Complete Reference (2 volume set) Product number B6011-96012 • HP-MPI User’s Guide (Eleventh Edition) Product number B6060-96024 The HP-MPI User’s Guide and HP-MPI Release Notices are available online • in /opt/hpmpi/doc after you install the product. • at http://docs.hp.com.
HP-MPI V2.2.5.1 for Linux Release Note Product Documentation Table 4 Category Man Page Categories Man Pages Description General MPI.1 Describes the general features of HP-MPI Compilation mpicc.1 mpiCC.1 mpif77.1 mpif90.1 Describes the available compilation utilities Runtime mpiclean.1 mpidebug.1 mpienv.1 mpiexec.1 mpijob.1 mpimtsafe.1 mpirun.1 mpirun.all.1 mpistdio.1 autodbl.1 Describes runtime utilities, environment variables, debugging, thread-safe and diagnostic libraries.
HP-MPI V2.2.5.1 for Linux Release Note Product Packaging Product Packaging HP-MPI is packaged as an optional software product installed in /opt/hpmpi.
HP-MPI V2.2.5.1 for Linux Release Note Software Availability in Native Languages Software Availability in Native Languages There is no information on non-English languages for HP-MPI for Linux systems.
