Product specifications
Table Of Contents
- Table of Contents
- 1 Introduction
- 2 Feature Overview
- 3 Step-by-Step Cluster Setup and MPI Usage Checklists
- 4 InfiniPath Cluster Setup and Administration
- Introduction
- Installed Layout
- Memory Footprint
- BIOS Settings
- InfiniPath and OpenFabrics Driver Overview
- OpenFabrics Drivers and Services Configuration and Startup
- Other Configuration: Changing the MTU Size
- Managing the InfiniPath Driver
- More Information on Configuring and Loading Drivers
- Performance Settings and Management Tips
- Host Environment Setup for MPI
- Checking Cluster and Software Status
- 5 Using QLogic MPI
- Introduction
- Getting Started with MPI
- QLogic MPI Details
- Use Wrapper Scripts for Compiling and Linking
- Configuring MPI Programs for QLogic MPI
- To Use Another Compiler
- Process Allocation
- mpihosts File Details
- Using mpirun
- Console I/O in MPI Programs
- Environment for Node Programs
- Environment Variables
- Running Multiple Versions of InfiniPath or MPI
- Job Blocking in Case of Temporary InfiniBand Link Failures
- Performance Tuning
- MPD
- QLogic MPI and Hybrid MPI/OpenMP Applications
- Debugging MPI Programs
- QLogic MPI Limitations
- 6 Using Other MPIs
- A mpirun Options Summary
- B Benchmark Programs
- C Integration with a Batch Queuing System
- D Troubleshooting
- Using LEDs to Check the State of the Adapter
- BIOS Settings
- Kernel and Initialization Issues
- OpenFabrics and InfiniPath Issues
- Stop OpenSM Before Stopping/Restarting InfiniPath
- Manual Shutdown or Restart May Hang if NFS in Use
- Load and Configure IPoIB Before Loading SDP
- Set $IBPATH for OpenFabrics Scripts
- ifconfig Does Not Display Hardware Address Properly on RHEL4
- SDP Module Not Loading
- ibsrpdm Command Hangs when Two Host Channel Adapters are Installed but Only Unit 1 is Connected to the Switch
- Outdated ipath_ether Configuration Setup Generates Error
- System Administration Troubleshooting
- Performance Issues
- QLogic MPI Troubleshooting
- Mixed Releases of MPI RPMs
- Missing mpirun Executable
- Resolving Hostname with Multi-Homed Head Node
- Cross-Compilation Issues
- Compiler/Linker Mismatch
- Compiler Cannot Find Include, Module, or Library Files
- Problem with Shell Special Characters and Wrapper Scripts
- Run Time Errors with Different MPI Implementations
- Process Limitation with ssh
- Number of Processes Exceeds ulimit for Number of Open Files
- Using MPI.mod Files
- Extending MPI Modules
- Lock Enough Memory on Nodes When Using a Batch Queuing System
- Error Creating Shared Memory Object
- gdb Gets SIG32 Signal Under mpirun -debug with the PSM Receive Progress Thread Enabled
- General Error Messages
- Error Messages Generated by mpirun
- MPI Stats
- E Write Combining
- F Useful Programs and Files
- G Recommended Reading
- Glossary
- Index

F–Useful Programs and Files
Summary and Descriptions of Useful Programs
F-12 IB6054601-00 H
S
mpirun
mpirun determines whether the program is being run against a QLogic or
non-QLogic driver. It is installed from the mpi-frontend RPM. Sample
commands and results are shown in the following paragraphs.
QLogic-built:
$ mpirun -np 2 -m /tmp/id1 -d0x101 mpi_latency 1 0
asus-01:0.ipath_setaffinity: Set CPU affinity to 1, port 0:2:0
(1 active chips)
asus-01:0.ipath_userinit: Driver is QLogic-built
Non-QLogic built:
$ mpirun -np 2 -m /tmp/id1 -d0x101 mpi_latency 1 0
asus-01:0.ipath_setaffinity: Set CPU affinity to 1, port 0:2:0
(1 active chips)
asus-01:0.ipath_userinit: Driver is not QLogic-built
mpi_stress
This is an MPI stress test program designed to load up an MPI interconnect with
point-to-point messages while optionally checking for data integrity. By default, it
runs with all-to-all traffic patterns, optionally including oneself and one’s local
shared memory (shm) peers. It can also be set up with multi-dimensional grid
traffic patterns; this can be parameterized to run rings, open 2D grids, closed
2D grids, cubic lattices, hypercubes, and so on.
Optionally, the message data can be randomized and checked using CRC
checksums (strong but slow) or XOR checksums (weak but fast). The
communication kernel is built out of non-blocking point-to-point calls to load up the
interconnect. The program is not designed to exhaustively test out different MPI
primitives. Performance metrics are displayed, but should be carefully interpreted
in terms of the features enabled.
This is an MPI application and should be run under mpirun or its equivalent.
The following example runs 16 processes and a specified hosts file using the
default options (all-to-all connectivity, 64 to 4MB messages in powers of two, one
iteration, no data integrity checking):
$ mpirun -np 16 -m hosts mpi_stress
There are a number of options for
mpi_stress
; this one may be particularly useful:
-P
This option poisons receive buffers at initialization and after each receive;
pre-initialize with random data so that any parts that are not being correctly
updated with received data can be observed later.
See the mpi_stress(1) man page for more information.