HP-MPI Version 1.1 for Windows Release Note Manufacturing Part Number: BA683-90003 E0407 Release U.S.A. © Copyright 1979-2007 © Hewlett-Packard Development Company, L.P.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental, or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
Intel® and Itanium are trademarks or registered trademarks of the Intel Corporation or its subsidiaries in the United States and other countries. Voltaire is a registered trademark of Voltaire, Inc. InfiniBand is a trademark of the InfiniBand® Trade Association. FLEXlm® is a registered trademark of Globetrotter Software and Macrovision Corporation. All other trademarks and copyrights referred to are the property of their respective owners.
HP-MPI V1.1 for Windows Release Note Announcement HP-MPI V1.1 for Windows Release Note Announcement This release note describes the release of HP-MPI Version 1.1 for Windows. HP-MPI V1.1 is the April 2007 release of HP-MPI, Hewlett-Packard’s fully functional implementation of the Message Passing Interface standard for Windows. HP-MPI V1.1 for Windows is supported on servers and workstations running Windows Compute Cluster Pack (CCP), Windows Server 2003, or Windows XP Professional.
HP-MPI V1.1 for Windows Release Note What’s in This Version What’s in This Version HP-MPI is a high-performance implementation of the Message Passing Interface standard. HP-MPI complies fully with the MPI-1.2 standard and provides MPI-2 one-sided functionality. HP-MPI provides an application programming interface and software libraries to support parallel, message-passing applications that are efficient, portable, and flexible. HP-MPI uses shared libraries.
HP-MPI V1.1 for Windows Release Note What’s in This Version If you choose to move the HP-MPI installation directory from its default location, set the MPI_ROOT environment variable to point to the new location.
HP-MPI V1.
HP-MPI V1.1 for Windows Release Note What’s in This Version • mpiexec, mpiclean, mpirun.all • MPICH compatibility mode • Deferred deregistration of memory on RDMA networks • -l option (change user ID for job execution) Description of features The following section provides brief descriptions of the new features included in this release. For more information on HP-MPI, refer to the HP-MPI User’s Guide available at http://docs.hp.
HP-MPI V1.1 for Windows Release Note What’s in This Version -ccpblock Uses block scheduling to place ranks on allocated nodes. Nodes will be processed in the order they were allocated by the scheduler, with each node being fully populated up to the total number of CPUs before moving on to the next node. Only valid when the -ccp option is used. May not be used with the -f, -hostfile, or -hostlist options. -ccpcluster Specifies the headnode of the CCP cluster that should be used to run the job.
HP-MPI V1.1 for Windows Release Note What’s in This Version flag is rarely necessary as the client (mpirun) and the server (HP-MPI Remote Launch service) will negotiate the security package to be used for authentication. -token -tg Authenticates to this token with the HP-MPI Remote Launch service. Some authentication packages require a token name. The default is no token. -pass Prompts the user for his domain account password. Used to authenticate and create remote processes.
HP-MPI V1.1 for Windows Release Note What’s in This Version Support DDR IBAL drivers This release supports Double Data Rate (DDR) InfiniBand. The proper DDR drivers must be installed in order to take advantage of the DDR functionality. No additional environment variables or options need to be used for DDR. Debugging applications This release provides a browser-based tutorial on how to debug applications that use HP-MPI in the Windows environment.
HP-MPI V1.1 for Windows Release Note What’s in This Version -sys Authenticates with the remote service and returns remote system information, including node name, CPU count, and username. -ps [username] Authenticates with the remote service, and lists processes running on the remote system. If a username is included, only that user’s processes are listed. -dir Authenticates with the remote service and lists the files for the given path.
HP-MPI V1.1 for Windows Release Note Installation Instructions Installation Instructions HP-MPI must be installed on all nodes which will be used to launch or run jobs. The installation directory used must be the same on all nodes. We strongly recommend that you uninstall any previous versions of HP-MPI for Windows before installing V1.1. To install HP-MPI, log in as Administrator and follow the instructions for either an interactive or command line installation below.
HP-MPI V1.1 for Windows Release Note Installation Instructions To copy the license file to all nodes: headnode> clusrun /all copy "\\netshare\path\mpi.lic" ^ "C:\Program Files (x86)\Hewlett-Packard\HP-MPI\licenses" Installation for Windows 2003/XP and Remote Launch service HP-MPI must be installed on all systems in the same location. To run multihost, the HP-MPI Remote Launch service must also be installed and started on all systems. Install HP-MPI as Administrator.
HP-MPI V1.1 for Windows Release Note Installation Instructions /tasks="RegisterService,!RegisterService\StartService" During the installation, this option will register the HP-MPI Remote Launch service, but will not start the service. To start the service select Administration Tools then Services. Highlight HP-MPI Remote Launch, and select Start. Remote Launch service information The HP-MPI Remote Launch service is located in "%MPI_ROOT%\sbin\HPMPIWin32Service.exe".
HP-MPI V1.1 for Windows Release Note Installation Instructions C:\> "%MPI_ROOT%\sbin\HPMPIWin32Service.exe" -start Service started... For additional HP-MPI Remote Launch service options, use -help. Usage: hpmpiwin32service.
HP-MPI V1.1 for Windows Release Note Installation Instructions To verify all files have been deleted: headnode> clusrun /all dir "C:\Program Files (x86)\Hewlett-Packard\HP-MPI" The only files you should see are the 'licenses' directories. If you see any other directories or files, delete these by hand. It is not uncommon to see a directory missed because a user was running HP-MPI during the uninstall. It is very common to see the uninstall .exe or .dat file not removed.
HP-MPI V1.1 for Windows Release Note Configuring Your Environment Configuring Your Environment The default install directory location for HP-MPI for Windows is one of the following directories: On 64-bit Windows: C:\Program Files (x86)\Hewlett-Packard\HP-MPI On 32-bit Windows: C:\Program Files\Hewlett-Packard\HP-MPI The default install will define the system environment variable MPI_ROOT, but will not put "%MPI_ROOT%\bin" in the system path or your user path.
HP-MPI V1.1 for Windows Release Note Configuring Your Environment From the GUI, use the Task Properties window, Environment tab to set the desired environment variable. NOTE These environment variables should be set on the mpirun task. Environment variables can also be set using the flag /env. For example: > job add JOBID /numprocessors:1 /env:"MPI_ROOT=\\shared\alternate\location" ...
HP-MPI V1.1 for Windows Release Note Running HP-MPI from CCS Running HP-MPI from CCS There are two ways to run HP-MPI under CCS: command line and scheduler GUI. Both approaches can be used to access the functionality of the scheduler. The command line scheduler options are similar to the GUI options. The following instructions are in the context of the GUI, but equivalent command line options are also listed.
HP-MPI V1.1 for Windows Release Note Running HP-MPI from CCS Figure 1 Job Allocation To run an MPI application, submit the mpirun command to the scheduler. HP-MPI uses the environment of the task and job where mpirun is executing to launch the required mpids that start the ranks. It’s important that mpirun uses only a single processor for its task within the job so the resources can be used by the other processes within the MPI application.
HP-MPI V1.1 for Windows Release Note Running HP-MPI on Windows 2003/XP Running HP-MPI on Windows 2003/XP To run HP-MPI on Windows 2003/XP systems use appfile mode or the -hostlist/-hostfile flags. For remote processes to have access to network resources (such as file shares), a password which is used to create processes on the remote nodes must be provided. The password is SSPI encrypted before being sent across the network. Passwords are provided using the -pass or -cache flags.
HP-MPI V1.1 for Windows Release Note Submitting jobs Submitting jobs The section includes general information for submitting jobs either from the GUI or the command line. As part of the mpirun task submitted, the following flags are commonly used with mpirun: -ccp Automatically creates an appfile which matches the CCS job allocation. The number of ranks run will equal the number of processors requested.
HP-MPI V1.1 for Windows Release Note Submitting jobs • CCP_TASKCONTEXT - Task ‘content’ (jobid.taskid) • CCP_TASKID - Task ID Included in the help directory is an example template (.xml file). To use this template, change the processor count on the Submit Jobs window, Processors tab, and edit the Tasks command to include flags and the rank executable. The job will run with ranks being equal to the number of processors, or the number of nodes if the -nodex flag is used.
HP-MPI V1.1 for Windows Release Note Submitting jobs NOTE 26 Examples were generated using CCP V1.0.
HP-MPI V1.1 for Windows Release Note Submitting jobs 4. On the Processors tab, select the total number of processors to allocate to the job (usually the number of ranks).
HP-MPI V1.1 for Windows Release Note Submitting jobs 5. Select the Tasks tab and enter the 'mpirun' command as the task. Then highlight the task and select edit. In the above example, the following line has been added into the "Command Line:" by selecting the text box and clicking Add. "%MPI_ROOT%\bin\mpirun.exe" -ccp -netaddr 172.16.150.0 -TCP ^ \\node\share\pallas.exe NOTE 28 Unselecting "Use job’s allocated processors" and setting the processors count to 1 now will eliminate Step 7.
HP-MPI V1.1 for Windows Release Note Submitting jobs 6. Specify stdout, stderr, and stdin (if necessary) on the Tasks tab. In the above example, the stderr and stdout files are specified using CCS environment variables defined by the job. This is an easy way to create output files unique for each task. \\node\share\%CCP_JOBNAME%-%CCP_TASKCONTEXT%.
HP-MPI V1.1 for Windows Release Note Submitting jobs 7. On the Task Properties window, select the Processors tab and set to one processor for the mpirun task. NOTE In Step 5, you can unselect the "Use job’s allocated processors" box and set the processors count to 1. This eliminates setting the processor count in the task window as shown here in Step 7. 8. To set environment variables for the MPI job, use the Environment tab in the Task Properties window. 9. Select OK on the Task Properties window.
HP-MPI V1.1 for Windows Release Note Submitting jobs 10. If you want to restrict the run to a set of machines, on the Submit Job window select the Advanced tab and set the desired machines. NOTE This step is not necessary. The job will select from any available processors if this step is not done. 11. To run the job, select the Submit on the Submit Job window. For convenience, generic templates can be created and saved using Save As Template in the Submit Job window.
HP-MPI V1.1 for Windows Release Note Submitting jobs Running HP-MPI from command line To perform the same steps via command line, execute 3 commands: 1. job new [options] 2. job add JOBID mpirun [mpirun options] 3. job submit /id:JOBID For example: > job new /jobname:[example job]/numprocessors:12 /projectname:HPMPI Job Queued, ID: 242 This will create a job resource and return a jobid, but not submit it. > job add 242 /stdout:"\\node\share\%CCP_JOBNAME%-%CCPTASKCONTEXT%.
HP-MPI V1.1 for Windows Release Note Submitting jobs -hostlist Indicates what nodes to use for the job. This hostlist may be delimited with spaces or commas. If spaces are used as delimiters anywhere in the hostlist, it may be necessary to place the entire hostlist inside quotes to prevent the command shell from interpreting it as multiple options. -np N Indicates the number of ranks to execute. The -np option is required if none of -f, -hostfile, or -hostlist have been specified.
HP-MPI V1.1 for Windows Release Note Building and running applications Building and running applications Compiling and running your first application To quickly become familiar with compiling and running HP-MPI programs, start with the C version of the hello_world program. This program is called hello_world.c and prints out the text string "Hello world! I’m r of s on host" where r is a process’s rank, s is the size of the communicator, and host is the host on which the program is run.
HP-MPI V1.1 for Windows Release Note Building and running applications > "%MPI_ROOT%\bin\mpicc" -mpi64 hello_world.c Specify the bitness using -mpi64 or -mpi32 for mpicc to link in the correct libraries. Verify you are in the correct 'bitness' compiler window. Using -mpi64 in a Visual Studio 32-bit command window will not work. NOTE Step 3. Run the hello_world executable file: > "%MPI_ROOT%\bin\mpirun" -np 4 hello_world.exe where -np 4 specifies 4 as the number of processors to run. Step 4.
HP-MPI V1.1 for Windows Release Note Building and running applications Copyright Microsoft Corporation. All rights reserved. /out:hello_world.exe "/libpath:C:\Program Files (x86)\Hewlett-Packard\HP-MPI\lib" /subsystem:console libhpmpi64.lib libmpio64.lib hello_world.obj Step 4. Create a new job requesting the number of CPUs to use. Resources are not yet allocated, but the job is given a JOBID number which is printed to stdout: > job new /numprocessors:16 Job queued, ID: 4288 Step 5.
HP-MPI V1.1 for Windows Release Note Building and running applications > job add 4288 /numprocessors:1 ^ /env:MPI_ROOT="%MPI_ROOT%" ^ /stdout:\\node\path\to\a\shared\file.out ^ /stderr:\\node\path\to\a\shared\file.err path\submission_script.vbs Where submission_script.vbs contains code such as: Option Explicit Dim sh, oJob, JobNewOut, appfile, Rsrc, I, fs Set sh = WScript.CreateObject(“WScript.Shell”) Set fs = CreateObject(“Scripting.FileSystemObject”) Set oJob = sh.exec(“%MPI_ROOT%\bin\mpi_nodes.
HP-MPI V1.1 for Windows Release Note Building and running applications Go to VS Project, select View, select Property Manager and expand the project. This will display the different configurations and platforms set up for builds. Include the appropriate property page (HPMPI.vsprops for 32-bit applications, HPMPI64.vsprops for 64-bit applications) in the Configuration/Platform section. Select this page by either double-clicking the page or by right-clicking on the page and selecting Properties.
HP-MPI V1.1 for Windows Release Note Building and running applications -h n01 -np 2 \\node01\share\path\to\hello_world.exe -h n02 -np 2 \\node01\share\path\to\hello_world.exe Step 2. For the first run of the hello_world executable, use -cache to cache your password: > "%MPI_ROOT%\bin\mpirun" -cache -f appfile Password for MPI runs: When typing, the password is not echoed to the screen. The HP-MPI Remote Launch service must be registered and started on the remote nodes.
HP-MPI V1.1 for Windows Release Note Building and running applications X:\demo> mpirun -ccp -f appfile Step 3. Analyze hello_world output. HP-MPI prints the output from running the hello_world executable in non-deterministic order. The following is an example of the output. Hello Hello Hello Hello world! world! world! world! I'm I'm I'm I'm 2 1 0 3 of of of of 4 4 4 4 on on on on n02 n01 n01 n02 More information about using appfiles is available in Chapter 3 of the HP-MPI User’s Guide.
HP-MPI V1.1 for Windows Release Note Building and running applications Any additional runs of ANY HP-MPI application from the same node and same user account will not require a password: X:\demo> "%MPI_ROOT%\bin\mpirun" -hostlist n01:2,n02:2 ^ hello_world.exe Hello Hello Hello Hello world! world! world! world! I'm I'm I'm I'm 1 3 0 2 of of of of 4 4 4 4 on on on on n01 n02 n01 n02 Running with a hostfile using CCS Perform Steps 1 and 2 from “Building and running on a single host” on page 34. Step 1.
HP-MPI V1.1 for Windows Release Note Building and running applications Running with a hostlist using CCS Perform Steps 1 and 2 from “Building and running on a single host” on page 34. Step 1. Change to a writable directory on a mapped drive. The mapped drive should be to a shared folder for the cluster. Step 2. Submit the job to CCS, including the list of nodes on the command line. X:\demo> "%MPI_ROOT%\bin\mpirun" -ccp -hostlist n01,n02,n03,n04 ^ -np 8 hello_world.
HP-MPI V1.1 for Windows Release Note Building and running applications > job submit /id:4288 Job 4288 has been submitted. Step 3. Run the application(s) as a task in the allocation, optionally waiting for each to finish before starting the following one. > "%MPI_ROOT%\bin\mpirun" -ccp -ccpwait -jobid 4288 ^ \\node\share\hello_world.exe mpirun: mpirun: mpirun: mpirun: NOTE Submitting job to ccp scheduler on this node HPMPI Job 4288 submitted to cluster mpiccp1 Waiting for HPMPI Job 4288 to finish...
HP-MPI V1.1 for Windows Release Note C command line basics C command line basics The utility "%MPI_ROOT%\bin\mpicc" is included to aid in command line compilation. To compile with this utility, set MPI_CC to the path of the command line compiler you want to use. Specify -mpi32 or -mpi64 to indicate if you are compiling a 32- or 64-bit application. Specify the command line options that you would normally pass to the compiler on the mpicc command line.
HP-MPI V1.1 for Windows Release Note C command line basics and the linker options: /libpath:"%MPI_ROOT%\lib" /subsystem:console The above assumes the environment variable MPI_ROOT is set. For example, to compile hello_world.c from the Help directory using Visual Studio (from a Visual Studio 2005 command prompt window): cl hello_world.c /I"%MPI_ROOT%\include\64" /link /out:hello_world.exe ^ /libpath:"%MPI_ROOT%\lib" /subsystem:console libhpmpi64.
HP-MPI V1.1 for Windows Release Note Fortran command line basics Fortran command line basics The utility "%MPI_ROOT%\bin\mpif90" is included to aid in command line compilation. To compile with this utility, set MPI_F90 to the path of the command line compiler you want to use. Specify -mpi32 or -mpi64 to indicate if you are compiling a 32- or 64-bit application. Specify the command line options that you would normally pass to the compiler on the mpif90 command line.
HP-MPI V1.1 for Windows Release Note Fortran command line basics ifort compute_pi.f /I"%MPI_ROOT%\include\64" /link /out:compute_pi.exe ^ /libpath:"%MPI_ROOT%\lib" /subsystem:console libhpmpi64.lib autodouble (automatic promotion) HP-MPI supports automatic promotion of Fortran datatypes using any of the following arguments (some of which are not supported on all Fortran compilers). /integer_size:64 /4I8 -i8 /real_size:64 /4R8 /Qautodouble -r8 If these flags are given to the mpif90.
HP-MPI V1.1 for Windows Release Note InfiniBand‘ setup InfiniBand setup This release of HP-MPI has been tested with Voltaire 2.2.493.409. For Windows CCS clusters purchased from HP, Voltaire drivers were either pre-installed on your system, or provided on the Windows Compute Cluster Server 2003 CD-ROM. Drivers are also available for download from Voltaire’s site at http://www.voltaire.com/partner_login.htm.
HP-MPI V1.1 for Windows Release Note Myrinet MX setup Myrinet MX setup Installing the Myrinet MX driver The MX driver can be installed using the Device Manager. It can be reached by opening the System Control Panel applet and selecting the Hardware tab. The Myricom Myrinet Adapter should be listed as an unknown PCI device rooted off Other Devices. Right click on PCI Device and select Properties to bring up the PCI Device Properties page.
HP-MPI V1.1 for Windows Release Note Myrinet MX setup Select Reinstall Driver (although no driver has been previously installed). This will initiate the Hardware Update Wizard. Choose to install from a list or specific location. You will be prompted for the path to the driver files. Enter the path to the directory where the MX files are installed and select Next. Windows will search for compatible devices and should find that the MX driver can be installed.
HP-MPI V1.1 for Windows Release Note Myrinet MX setup The installation of the mapper service begins from the Network Control Panel applet.
HP-MPI V1.1 for Windows Release Note Myrinet MX setup Select the Local Area Connection that corresponds to the Myricom Myrinet Adapter. Right click on the icon and select Properties. For installation of various network components, click on Install.
HP-MPI V1.1 for Windows Release Note Myrinet MX setup The mapper is a service, so select Service then Add. Since this is not a service provided by Windows, choose Have Disk. Select the active mapper and complete the installation by selecting OK. The mapper will automatically run when the system is restarted. The mapper can also be started manually at the command prompt with net start mx_mapper.
HP-MPI V1.1 for Windows Release Note Known Problems and Workarounds Known Problems and Workarounds • The Fortran interface provided in HP-MPI assumes a style calling convention of C-byreference. That is, function names are not decorated as _function@bytes, arguments are passed by reference, string length arguments are passed by value after the other arguments, and the caller cleans the stack.
HP-MPI V1.1 for Windows Release Note Installation Requirements Installation Requirements HP-MPI V1.1 for Windows is supported on the following Microsoft operating systems: • Microsoft Windows Compute Cluster 2003 • Microsoft Windows Server 2003 • Microsoft XP Professional On Windows 2003 and XP Professional, all systems which will be used to run or submit HP-MPI jobs must be members of the same active directory domain.
HP-MPI V1.1 for Windows Release Note Licensing Policy for Windows Licensing Policy for Windows HP-MPI V1.1 for Windows uses FLEXlm licensing technology. A license is required to use HP-MPI for Windows. Licenses can be purchased from HP’s software depot at http://www.hp.com/go/softwaredepot, or contact your HP representative. Demo licenses for HP-MPI are also available from HP’s software depot.
HP-MPI V1.1 for Windows Release Note Licensing Policy for Windows Installing License Files A valid license file contains the system hostid and the associated license key. License files can be named either as license.dat or any name with extension of *.lic (like mpi.lic, for example). The license file must be copied to the installation directory (default C:\Program Files (x86)\Hewlett-Packard\HP-MPI\licenses) on all runtime systems, and to the license server.
HP-MPI V1.1 for Windows Release Note Product Documentation Product Documentation The HP-MPI Documentation Kit is an optional product (product number B6281AA) consisting of the following hardcopy books: • MPI: The Complete Reference (2 volume set) • HP-MPI User’s Guide The HP-MPI User’s Guide and HP-MPI Release Notices are available online • in C:\Program Files (x86)\Hewlett-Packard\HP-MPI\doc after you install the product. • at http://docs.hp.com. • at http://www.hp.com/go/hpmpi.
HP-MPI V1.1 for Windows Release Note Product Documentation Table 4 Category Runtime Man Page Categories (Continued) Man Pages mpidebug.1 mpienv.1 mpimtsafe.1 mpirun.1 mpistdio.1 autodbl.1 Description Describes runtime utilities, environment variables, debugging, thread-safe and diagnostic libraries.
HP-MPI V1.1 for Windows Release Note Software Availability in Native Languages Software Availability in Native Languages There is no information on non-English languages for HP-MPI for Windows systems.
