HP-MPI Version 2.
© Copyright 1979-2009 © Hewlett-Packard Development Company, L.P Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents 1 Information About this Release......................................................................................9 1.1 Announcement..................................................................................................................................9 1.2 Product Information..........................................................................................................................9 1.2.1 Platforms Supported......................................................
6 Running HP-MPI from HPCS........................................................................................27 7 Running HP-MPI on Windows.....................................................................................29 8 Submitting Jobs............................................................................................................31 8.1 General Information for Submitting Jobs........................................................................................31 8.
15 Product Documentation..............................................................................................55 16 Software Availability in Native Languages.............................................................
List of Figures 6-1 6 Job Allocation................................................................................................................................
List of Tables 1-1 10-1 11-1 15-1 Directory Structure..........................................................................................................................9 mpicc Utility..................................................................................................................................45 mpif90 Utility................................................................................................................................47 Manpage Categories.........................
1 Information About this Release 1.1 Announcement This release note describes the release of HP-MPI Version 2.0 for Windows®. HP-MPI V2.0 is the June 2009 release of HP-MPI, the HP fully functional implementation of the Message Passing Interface standard for Windows. HP-MPI V2.0 for Windows is supported on servers and workstations running HPC Server 2008 (HPCS), Vista, Server 2008 (32-bit or 64-bit), Server 2003 (32-bit or 64-bit), or XP (32-bit or 64-bit).
Table 1-1 Directory Structure (continued) Subdirectory Contents etc hpmpi.conf file help Source files for the example programs, Visual Studio Property pages include\32 32-bit header files include\64 64-bit header files lib HP-MPI libraries licenses Repository for HP-MPI license file man HP-MPI manpages in HTML format sbin HP-MPI Remote Launch service 1.2.3 Documentation Changes This release note might be updated periodically.
2 New or Changed Features HP-MPI V2.0 for Windows includes the following new or changed features. For more information, refer to the manpages in C:\Program Files (x86)\Hewlett-Packard\HP-MPI\man. 2.1 Improved Startup Performance HP-MPI V2.0 for Windows has greatly improved startup and shutdown time for HP-MPI jobs. These improvements are for both HPC submitted jobs and Windows XP/Vista/2003/2008 jobs using the HP-MPI Remote Launch service. Job launch and shutdown scales well to 1024 nodes. 2.
NOTE: Previous versions of HP-MPI Remote Launch services do not work with HP-MPI V2.0. If you are running a service remotely using a network share for MPI_ROOT, you must uninstall the old HP-MPI Remote Launch service and re-install the new service. If you are installing HP-MPI locally on all nodes, this should not be an issue. 2.8 Windows HPC Scheduler API HP-MPI V2.0 for Windows uses the new Windows HPC Scheduler API, which allows more scheduler options such as job templates, node groups, and unit type.
scheduler/launcher feature of HP-MPI. This flag is only effective if used by the HP-MPI automatic job scheduler/launcher. Replaces -ccpin. -hpcout Assigns the job standard output file to the given file name when starting a job through the automatic job scheduler/launcher feature of HP-MPI. This flag is only effective if used by the HP-MPI automatic job scheduler/launcher. Replaces -ccpout.
for saving the job as a job description file. You must submit the configuring job to begin execution. -hpcunit node [core|socket|node] Specifies the unit of entities that can be scheduled as defined by the HPC scheduler. The defaults is core. NOTE: -hpcunit node is the replacement for -nodex. The difference is -nodex requires -np x to include all cores for the nodes. -hpcunit node -np y, y accurately reflects a single rank on y nodes. The -nodex option is deprecated.
NOTE: The mpirun -pwcheck option, along with other HP-MPI password options, run with HP-MPI Remote Launch Service, and do not refer to Windows HPC user passwords. When running through Windows HPC scheduler (with -hpc), you might need to cache a password through the Windows HPC scheduler. For more information, see the Windows HPC job command.
2.16 Appfile, Hostlist, and Hostfile Support for Windows 2008 HP-MPI V2.0 for Windows contains a variety of new launch options for the mpirun.exe command. The -hostfile and -hostlist options enable you to launch the same executable across a collection of hosts. The -f option and an appfile enable applications to launch that consist of multiple programs or that run on multiple hosts. 2.17 Support for InfiniBand Drivers for Windows 2008 HP-MPI for Windows V2.
mpidiag authentication options are the same as mpirun authentication options. These include: -pass, -cache, -clearcache, -iscached, -token/-tg, -package/-pk. 2.20 MPI Compiler Wrapper Script Options • -mpicc -mpicxx -mpif90 These options serve the same purpose as the MPI_CC, MPI_CXX, and MPI_F90 variables for telling the mpicc, mpicxx, and mpif90 wrapper scripts which compiler to use; but they provide the ability to change the compiler from the command line.
3 Installation Requirements HP-MPI V2.0 for Windows is supported on the following Microsoft operating systems: • • 32-bit or 64-bit HPC Server 2008 (HPCS)/Server 2003/Vista 32-bit or 64-bit XP On Windows 2008, all systems which will be used to run or submit HP-MPI jobs must be members of the same active directory domain. You must install and start the HP-MPI Remote Launch service to run on all Windows 2008 systems except Windows HPC Server 2008.
4 Installation Instructions HP-MPI must be installed on all nodes which are used to launch or run jobs. The installation directory used must be the same on all nodes. HP strongly recommends that you uninstall any previous versions of HP-MPI for Windows before installing V2.0. To install HP-MPI, log in as Administrator, and follow the instructions for either an interactive or command line installation described below. 4.1 Standard Interactive Installation Double-click Setup HP-MPI-2.0-060409.
4.3 Installation for non-Windows HPC Cluster Using Remote Launch Service HP-MPI must be installed on all systems in the same location. To run multihost, the HP-MPI Remote Launch service must also be installed and started on all systems. Install HP-MPI as Administrator. To install HP-MPI, double-click HP-MPI-2.0-060409.exe, and follow the instructions. On the Select Component screen, select the second option HP-MPI libraries, Tools and Services for Windows 2008.
C:\> "%MPI_ROOT%\sbin\HPMPIWin32Service.exe" -start Service started... The HP-MPI Remote Launch service runs continually as a Windows service, listening on a port for HP-MPI requests from remote mpirun.exe jobs. This port must be the same port on all machines, and is established when the service starts. The default TCP port is 8636. If this port is not available or to change the port, include a port number as a parameter to -i.
To delete the unins000.exe, enter the following on one line: headnode> clusrun /all del "C:\Program Files (x86)\Hewlett-Packard\HP-MPI \unins000.exe" When the HP-MPI directory is clean, you can proceed with installing the new version of HP-MPI.
5 Configuring Your Environment The default install directory location for HP-MPI for Windows is one of the following directories: On 64-bit Windows: C:\Program Files (x86)\Hewlett-Packard\HP-MPI On 32-bit Windows: C:\Program Files\Hewlett-Packard\HP-MPI The default install defines the system environment variable MPI_ROOT, but does not put "%MPI_ROOT%\bin" in the system path or your user path.
NOTE: Set these environment variables on the mpirun task. Environment variables can also be set using the flag /env. For example: C:\> job add JOBID /numprocessors:1 /env:"MPI_ROOT=\\shared\alternate\location" ...
6 Running HP-MPI from HPCS There are two ways to run HP-MPI under HPCS: command line and scheduler GUI. Both ways can be used to access the functionality of the scheduler. The command line scheduler options are similar to the GUI options. The following instructions are in the context of the GUI, but equivalent command line options are also listed.
7 Running HP-MPI on Windows To run HP-MPI on Windows XP/Vista/2003/2008 (non-HPCS) systems, use the appfile mode or the -hostlist/-hostfile flags. For remote processes to have access to network resources (such as file shares), a password, which is used to create processes on the remote nodes, must be provided. The password is SSPI encrypted before being sent across the network. Passwords are provided using the -pass or -cache flags. To check for valid cached passwords, use the -pwcheck option.
8 Submitting Jobs 8.1 General Information for Submitting Jobs The section includes general information for submitting jobs either from the GUI or the command line. As part of the mpirun task submitted, the following flags are commonly used with mpirun: Automatically creates an appfile which matches the HPCS -hpc job allocation. The number of ranks run will equal the number of processors requested. -np N Indicates the number of ranks to execute.
-hostlist node1[:X,node2:Y,...] Indicates specific cluster resources to use for the job. Include a comma separated list, and specify the number of ranks/node by following the hostname with ':X' where X indicates the number of ranks. This enables the application or test applications to run on a specific set of nodes and ranks. For example, to run a single rank/node: "%MPI_ROOT%\bin\mpirun" -hpc -hostlist node1,node2,node3 rank.exe This command runs a single rank specifically on node1, node2, and node3.
4. Select Task List (from the left menu list) then left-click Add (on the right menu list) to create a new task within the job. 8.
5. In the above example, the following line has been added into the Command line: by selecting the text box and entering: "%MPI_ROOT%\bin\mpirun.exe" -hpc -netaddr 172.16.150.0 -TCP \\node\share\pallas.exe 6. Specify stdout, stderr, and stdin (if necessary). In the above example, the stderr and stdout files are specified using HPCS environment variables defined by the job. This is an easy way to create output files unique for each task. \\node\share\%CCP_JOBNAME%-%CCP_TASKCONTEXT%.out 7.
8. To change task properties such as Resources and Environment Variables, highlight the task in the New Job window, and change the task properties in the lower portion of the window. IMPORTANT: Set the Exclusive entry to True so the job manager will schedule the MPI ranks evenly across the allocated resources. 9. To restrict the run to a set of machines, select the nodes in the Resource Selection window. 8.
NOTE: This step is not necessary. The job will select from any available processors if this step is not done. NOTE: For convenience, Job Description Files can be created and saved by clicking Save Job as .... 10. To run the job, click Submit. 8.3 Running HP-MPI from the Command Line To perform the same steps on the command line, execute 3 commands: 1. 2. C:\> job new /exclusive:true [options] 3.
C:\> job add 242 /stdout:"\\node\share\%CCP_JOBNAME%-%CCPTASKCONTEXT%.out" /stderr:"\\node\share\%CCP_JOBNAME%-%CCPTASKCONTEXT%.err" /exclusive:true "%MPI_ROOT%\bin\mpirun" -hcp -prot -netaddr 172.16.150.20/24 -TCP \\node\share\rank.exe -arg1 -arg2 C:\> job submit /id:242 8.4 Automatic Job Submittal HP-MPI has added functionality which then automatically creates, schedules, and submits an mpirun command to the scheduler from the command line.
In the example above,mpirun is instructed to run 6 ranks across the machines mpihpc1 and mpihpc2 with the layout having ranks 0, 2, and 4 on mpihpc1 and ranks 1, 3, and 5 on mpihpc2. So mpirun creates a HPCS job allocation specifically requesting hosts mpihpc1 and mpihpc2, and launches the task onto those nodes.
9 Compiling and Running Applications This section provides instructions for building, compiling, and running applications. 9.1 Compiling and Running Your First Application To quickly become familiar with compiling and running HP-MPI programs, start with the C version of the hello_world program. This program is called hello_world.c and prints out the text string "Hello world! I’m r of s on host" where r is a process’s rank, s is the size of the communicator, and host is the host on which the program is run.
Hello Hello Hello Hello world! world! world! world! I'm I'm I'm I'm 1 3 0 2 of of of of 4 4 4 4 on on on on banach1 banach1 banach1 banach1 9.3 Building and Running Multihost on Windows HPCS Clusters The following is an example of basic compilation and run steps to execute hello_world.c on a cluster with 16-way parallelism. To build and run hello_world.c on a HPCS cluster: 1. 2. 3. Change to a writable directory on a mapped drive. Share the mapped drive to a folder for the cluster.
2. Submit a script. Verify MPI_ROOT is set in the environment (See the mpirun manpage for more information): C:\> job add 4288 /numprocessors:1 /env:MPI_ROOT="%MPI_ROOT%" /exclusive:true /stdout:\\node\path\to\a\shared\file.out /stderr:\\node\path\to\a\shared\file.err path\submission_script.vbs Where submission_script.vbs contains code such as: Option Explicit Dim sh, oJob, JobNewOut, appfile, Rsrc, I, fs Set sh = WScript.CreateObject(“WScript.Shell”) Set fs = CreateObject(“Scripting.
• C/C++ — Additional Include Directories Set to "%MPI_ROOT%\include\[32|64]" • Linker — Additional Dependencies Set to libhpmpi32.lib or libhpmpi64.lib depending on the application. • Additional Library Directories Set to "%MPI_ROOT%\lib" 9.5 Building and Running on a Windows 2008 Cluster Using Appfiles The example teaches you the basic compilation and run steps to execute hello_world.c on a cluster with 4-way parallelism. To build and run hello_world.
2. Submit the job to HPCS with the following command: X:\demo> mpirun -hpc -f appfile 3. Analyze hello_world output. HP-MPI prints the output from running the hello_world executable in non-deterministic order. The following is an example of the output. Hello Hello Hello Hello world! world! world! world! I'm I'm I'm I'm 2 1 0 3 of of of of 4 4 4 4 on on on on n02 n01 n01 n02 More information about using appfiles is available in Chapter 3 of the HP-MPI User’s Guide. 9.
3. Submit the job to HPCS. X:\demo> "%MPI_ROOT%\bin\mpirun" -hpc -hostfile hfname -np 8 hello_world.exe Nodes are allocated in the order that they appear in the hostfile. Nodes are scheduled cyclically, so if you have requested more ranks than there are nodes in the hostfile, nodes are used multiple times. 4. Analyze hello_world output. HP-MPI prints the output from running the hello_world executable in non-deterministic order.
10 C Command-Line Basics The utility "%MPI_ROOT%\bin\mpicc" is included to aid in command line compilation. To compile with this utility, set MPI_CC to the path of the command line compiler you want to use. Specify -mpi32 or -mpi64 to indicate if you are compiling a 32- or 64-bit application. Specify the command line options that you normally pass to the compiler on the mpicc command line. The mpicc utility adds additional command line options for HP-MPI include directories and libraries.
To embed the manifest file into .exe/.dll, use the mt tool. For more information, see the Microsoft/Visual Studio mt.exe tool. The following example shows how to embed a .manifest file into an application: C:\> "%MPI_ROOT%\bin\mpicc.bat" -mpi64 /MD hello_world.c C:\> mt -manifest hello_world.exe.manifest -outputresource:hello_world.
11 Fortran Command-Line Basics The utility "%MPI_ROOT%\bin\mpif90" is included to aid in command line compilation. To compile with this utility, set MPI_F90 to the path of the command line compiler you want to use. Specify -mpi32 or -mpi64 to indicate if you are compiling a 32- or 64-bit application. Specify the command line options that you normally pass to the compiler on the mpif90 command line. The mpif90 utility adds additional command line options for HP-MPI include directories and libraries.
However, if your application is written to explicitly handle the autodoubled datatypes (for example, if a variable is declared real and the code is compiled -r8 and corresponding MPI calls are given MPI_DOUBLE for the datatype), then the autodouble related command line arguments should not be passed to mpif90.bat at link time (because that would cause the datatypes to be automatically changed).
12 InfiniBand Setup For InfiniBand documentation, see the Interconnect Documentation section of the HP High Performance Computing website at: http://www.docs.hp.com/en/highperfcomp.
13 Known Issues and Workarounds This chapter describes known issues and workarounds for this release. 13.1 Calling Conventions The Fortran interface provided in HP-MPI assumes a style calling convention of C-by- reference. That is, function names are not decorated as _function@bytes, arguments are passed by reference, string length arguments are passed by value after the other arguments, and the caller cleans the stack.
• You installed HP-MPI to a local disk on all compute nodes. 13.7 PMPI_* Calling Convention on Windows On Windows, the mpi.h header specifies the __stdcall calling convention for all the MPI_* functions and the PMPI_* wrapper interface. This means a wrapper which might ordinarily look like: int MPI_Barrier(MPI_Comm_world comm) { ... return(PMPI_Barrier(comm)); } Would instead need to be written as: int __stdcall MPI_Barrier(MPI_Comm_world comm) { ... return(PMPI_Barrier(comm)); } 13.
14 Licensing 14.1 Licensing Policy for Windows HP-MPI V2.0 for Windows uses FLEXlm licensing technology. A license is required to use HP-MPI for Windows. Licenses can be purchased from the HP software depot at http://www.hp.com/go/ softwaredepot or contact your HP representative Demo licenses for HP-MPI are also available from the HP software depot. HP-MPI has an Independent Software Vendor (ISV) program that allows participating ISVs to freely distribute HP-MPI with their applications.
(“MPI BUG: Valid MPI license not found in search path”) 54 Licensing
15 Product Documentation The HP-MPI Documentation Kit is an optional product (product number B6281AA) consisting of the following hardcopy books: • • MPI: The Complete Reference (2 volume set) HP-MPI User’s Guide The HP-MPI User’s Guide and HP-MPI Release Notices are available online • • • in C:\Program Files (x86)\Hewlett-Packard\HP-MPI\doc after you install the product. at http://docs.hp.com. at http://www.hp.com/go/hpmpi.
16 Software Availability in Native Languages There is no information on non-English languages for HP-MPI for Windows systems.
