HP-MPI Version 2.2.
© Copyright 1979-2008 © Hewlett-Packard Development Company, L.P Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose.
Table of Contents 1 Information About This Release.....................................................................................9 1.1 Announcement..................................................................................................................................9 1.2 What’s in This Version.......................................................................................................................9 1.2.1 Platforms Supported.......................................................
List of Figures 1-1 IB Port Failover..............................................................................................................................
List of Tables 1-1 1-2 1-3 5-1 Systems Supported..........................................................................................................................9 Interconnect Support for Version 2.2.7..........................................................................................10 Directory Structure........................................................................................................................11 Manpage Categories................................................
1 Information About This Release 1.1 Announcement This release note describes the release of HP-MPI Version 2.2.7 for Linux. Version 2.2.7 is the April 2008 release of HP-MPI, the HP fully functional implementation of the Message Passing Interface standard for Linux. Version 2.2.7 for Linux is supported on HP ProLiant and HP Integrity servers running Red Hat Enterprise Linux AS 4 and 5, or SuSE Linux Enterprise Server 9 and 10 operating systems, as well as HP XC3000, HP XC4000 and HP XC6000 Clusters. 1.
Table 1-1 Systems Supported (continued) Platform Interconnects Interconnect information for XC systems is available at http://docs.hp.com/en/linuxhpc.html in the release notes, installation guides, and hardware preparation guides. 1 Ethernet includes 10baseT, 100baseT, GigE, 10GbE, and 10GbE with RNIC. 1.2.2 Interconnects Supported HP-MPI supports a wide variety of high performance protocols. Table 1-2 Interconnect Support for Version 2.2.
Table 1-3 Directory Structure Subdirectory Contents bin Command files for the HP-MPI utilities, gather_info script help Source files for the example programs include Header files lib/linux_ia32 HP-MPI Linux 32-bit libraries lib/linux_ia64 HP-MPI Linux 64-bit libraries for Itanium lib/linux_amd64 HP-MPI Linux 64-bit libraries for Opteron and EM64T newconfig/ Configuration files and release notes share/man/man1* manpages for the HP-MPI utilities share/man/man3* manpages for the HP-MPI libra
• • New flag -dbgspin (page 18) Improved message rate (page 18) 1.2.7 Description of Benefits and Features The following section provides brief descriptions of the features included in this release. For more information on HP-MPI, refer to the HP-MPI User’s Guide available at http://docs.hp.com. 1.2.7.
Both uDAPL 2.0 and 1.2 can coexist on the same system, and HP-MPI automatically detects which to use, based on the dat_ia_open call. The MPI_IC_SUFFIXES variable in hpmpi.conf has several uDAPL_* variables which represent the different libdat2.so and libdat.so libraries that are dlopened and tested for compatibility with the system /etc/dat.conf entries. For each library that is loaded, the first ten devices in dat.conf are used in the dat_ia_open call.
striped over all the connections. When one of the connections is broken, a warning is issued, but HP-MPI continues to use the rest of the healthy connections to transfer messages. If all the connections are broken, HP-MPI issues an error message. 1.2.7.9 InfiniBand port failover support A multi-port InfiniBand channel adapter can use Automatic Path Migration (APM) to provide network high availability.
1.2.7.10 InfiniBand support with MPI_Comm_connect() and MPI_Comm_accept() HP–MPI Version 2.2.7 supports MPI_Comm_connect() and MPI_Comm_accept() over InfiniBand processes using the IBV protocol. Both sides must have InfiniBand support enabled and use the same InfiniBand parameter settings. MPI_Comm_connect() and MPI_Comm_accept() need a port name, which is the IP and port at the root process of the accept side. First, a TCP connection is established between the root process of both sides.
fw_ver: node_guid: .... max_pkeys: local_ca_ack_delay: port: 1 state: max_mtu: .... phys_state: GID[ 0]: port: 2 state: max_mtu: .... phys_state: GID[ 0]: 4.7.0 0008:f104:0396:a56c 64 15 PORT_ACTIVE (4) 2048 (4) LINK_UP (5) fe80:0000:0000:0000:0008:f104:0396:a56d PORT_ACTIVE (4) 2048 (4) LINK_UP (5) fe80:0000:0000:0001:0008:f104:0396:a56e The subnet ID is contained in the first 16 digits of the GID. The second 16 digits of the GID are the interface ID.
pp.x: Rank 0:1: MPI_Init: The IB ports choosen for IB connection setup do not have the same subnet_prefix. Please provide a port GID that all nodes have IB path to it by MPI_IB_PORT_GID pp.x: Rank 0:1: MPI_Init: You can get port GID using 'ibv_devinfo -v' 1.2.7.13 Control of first port selection By default, HP-MPI distributes the processes onto the available InfiniBand ports in a round-robin manner using the local rank ID of each process.
I is 0,...,ncard-1 k is 0,...,nport[next_card]-1 m is card count so far 1.2.7.14 Expanded lightweight instrumentation functionality This release extends the HP-MPI lightweight instrumentation feature in a number of ways. Instrumentation is now supported when using -ha.
2 Known Problems and Workarounds • • • OFED 1.3 documents minimum supported firmware revisions for a variety of InfiniBand adapters. If unsupported firmware is used, HP-MPI might experience issues with abnormal application teardown. Codes which call either MPI_Comm_spawn() or MPI_Comm_spawn_multiple() may not be able to locate the commands to spawn on remote nodes.
• • The MPI_FLUSH_FCACHE environment variable is silently ignored if the -cpu_bind option is not specified. This limitation will be removed in a future release. See the mpienv(1) manpage for more information. HP-MPI uses the $MPI_REMSH command to launch on remote machines. The commands constructed are of the general form "$MPI_REMSH -n ".
• uDAPL may experience issues on multi-card systems. uDAPL will function on the second card if it is on a separate subnet. But, the second card will not work if it is on the same subnet as the first card. For multi-card systems set: /sbin/sysctl -w net.ipv4.conf.all.arp_ignore=2 • The nofile limit on large linux clusters needs to be increased in /etc/security/limits.
• • • • The HP-MPI Version 2.2.7 library for Linux contains a dependency which requires libpthread. The mpicc, mpif90, etc. compiler wrapper scripts automatically add the necessary -lpthread, but manually linked applications must explicitly add -lpthread. Profiling routines built for C calls will not cause the corresponding Fortran calls to be wrapped automatically. To profile Fortran routines, separate wrappers need to be written for the Fortran calls. HP-MPI complies with the MPI-1.
3 Installation Information 3.1 Installation Requirements HP-MPI Version 2.2.7 for Linux is supported on HP ProLiant and HP Integrity servers running Red Hat Enterprise Linux AS 4 and 5, or SuSE Linux Enterprise Server 9 and 10 operating systems; as well as HP XC3000, HP XC4000 and HP XC6000 Clusters. You must install the correct HP-MPI product for your system. HP-MPI requires a minimum of 90MB of disk space in /opt. 3.2 Installation Instructions 1. 2. 3.
4 Licensing Information 4.1 Licensing Policy A license is required to use HP-MPI for Linux. Licenses can be purchased from the HP software depot at http://www.hp.com/go/softwaredepot, or by contacting your HP representative. Demo licenses for HP-MPI are also available from the HP software depot. No separate HP-MPI license is required on an HP XC system. 4.2 License Installation and Testing HP-MPI Version 2.2.7 for Linux uses FLEXlm® licensing technology. A license file can be named either as license.
(“MPI BUG: Valid MPI license not found in search path”) 4.2.3 Merging Licenses Newer HP-MPI licenses use the INCREMENT feature which allows separate HP-MPI licenses to be used in combination by concatenating files. For example: License 1: SERVER myserver 0014c2c1f34a DAEMON HPQ INCREMENT HP-MPI HPQ 1.0 permanent 8 9A40ECDE2A38 \ NOTICE="License Number = AAAABBBB1111" SIGN=E5CEDE3E5626 License 2: SERVER myserver 0014c2c1f34a DAEMON HPQ INCREMENT HP-MPI HPQ 1.
5 Additional Product Information 5.1 Product Documentation The HP-MPI Documentation Kit is an optional product (product number B6281AA) consisting of the following hardcopy books: • • MPI: The Complete Reference (2 volume set) product number B6011-96012 HP-MPI User’s Guide (Eleventh Edition) product number B6060-96024 The HP-MPI User’s Guide and HP-MPI Release Notices are available from the following resources: • • • $MPI_ROOT/doc (After you install the product.) http://docs.hp.com http://www.hp.
