Product specifications
Table Of Contents
- Table of Contents
- 1 Introduction
- 2 Feature Overview
- 3 Step-by-Step Cluster Setup and MPI Usage Checklists
- 4 InfiniPath Cluster Setup and Administration
- Introduction
- Installed Layout
- Memory Footprint
- BIOS Settings
- InfiniPath and OpenFabrics Driver Overview
- OpenFabrics Drivers and Services Configuration and Startup
- Other Configuration: Changing the MTU Size
- Managing the InfiniPath Driver
- More Information on Configuring and Loading Drivers
- Performance Settings and Management Tips
- Host Environment Setup for MPI
- Checking Cluster and Software Status
- 5 Using QLogic MPI
- Introduction
- Getting Started with MPI
- QLogic MPI Details
- Use Wrapper Scripts for Compiling and Linking
- Configuring MPI Programs for QLogic MPI
- To Use Another Compiler
- Process Allocation
- mpihosts File Details
- Using mpirun
- Console I/O in MPI Programs
- Environment for Node Programs
- Environment Variables
- Running Multiple Versions of InfiniPath or MPI
- Job Blocking in Case of Temporary InfiniBand Link Failures
- Performance Tuning
- MPD
- QLogic MPI and Hybrid MPI/OpenMP Applications
- Debugging MPI Programs
- QLogic MPI Limitations
- 6 Using Other MPIs
- A mpirun Options Summary
- B Benchmark Programs
- C Integration with a Batch Queuing System
- D Troubleshooting
- Using LEDs to Check the State of the Adapter
- BIOS Settings
- Kernel and Initialization Issues
- OpenFabrics and InfiniPath Issues
- Stop OpenSM Before Stopping/Restarting InfiniPath
- Manual Shutdown or Restart May Hang if NFS in Use
- Load and Configure IPoIB Before Loading SDP
- Set $IBPATH for OpenFabrics Scripts
- ifconfig Does Not Display Hardware Address Properly on RHEL4
- SDP Module Not Loading
- ibsrpdm Command Hangs when Two Host Channel Adapters are Installed but Only Unit 1 is Connected to the Switch
- Outdated ipath_ether Configuration Setup Generates Error
- System Administration Troubleshooting
- Performance Issues
- QLogic MPI Troubleshooting
- Mixed Releases of MPI RPMs
- Missing mpirun Executable
- Resolving Hostname with Multi-Homed Head Node
- Cross-Compilation Issues
- Compiler/Linker Mismatch
- Compiler Cannot Find Include, Module, or Library Files
- Problem with Shell Special Characters and Wrapper Scripts
- Run Time Errors with Different MPI Implementations
- Process Limitation with ssh
- Number of Processes Exceeds ulimit for Number of Open Files
- Using MPI.mod Files
- Extending MPI Modules
- Lock Enough Memory on Nodes When Using a Batch Queuing System
- Error Creating Shared Memory Object
- gdb Gets SIG32 Signal Under mpirun -debug with the PSM Receive Progress Thread Enabled
- General Error Messages
- Error Messages Generated by mpirun
- MPI Stats
- E Write Combining
- F Useful Programs and Files
- G Recommended Reading
- Glossary
- Index
4–InfiniPath Cluster Setup and Administration
Other Configuration: Changing the MTU Size
4-18 IB6054601-00 H
S
More information for configuration, starting and stopping the interface, and basic
troubleshooting is found in the QLogic OFED+ User Guide.
Other Configuration: Changing the MTU Size
The Maximum Transfer Unit MTU is set to 4K and enabled in the driver by default.
To see the current MTU size, and the maximum supported by the adapter, type
the command:
$ ibv_devinfo
To change the driver default back to 2K MTU, add this line (as a root user) into
/etc/modprobe.conf (or /etc/modprobe.conf.local):
options ib_ipath mtu4096=0
Restart the driver as described in “Managing the InfiniPath Driver” on page 4-19.
NOTE:
To use 4K MTU, set the switch to have the same 4K default. If you are using
QLogic switches, the following applies:
For the Externally Managed 9024, use 4.2.2.0.3 firmware
(9024DDR4KMTU_firmware.emfw) for the 9024 EM. This has the 4K
MTU default, for use on fabrics where 4K MTU is required. If 4K MTU
support is not required, then use the 4.2.2.0.2 DDR *.emfw file for
DDR externally-managed switches. Use FastFabric to load the firmware
on all the 9024s on the fabric.
For the 9000 chassis, use the most recent 9000 code 4.2.4.0.1. The 4K
MTU support is in 9000 chassis version 4.2.1.0.2 and later. For the 9000
chassis, when the FastFabric 4.3 (or later) chassis setup tool is used,
the user is asked to select an MTU. FastFabric (FF) can then set that
MTU in all the 9000 internally managed switches. The change will take
effect on the next reboot. Alternatively, for the internally managed 9000s,
the ismChassisSetMtu Command Line Interface (CLI) command can
be used. This should be executed on every switch and both
hemispheres of the 9240s.
For reference, see the FastFabric Users Guide version 4.3 and the
SIlverStorm 9000 CLI Reference Guide version 4.2. Both are available from
the QLogic web site.
For other switches, see the vendors’ documentation.