QLogic OFED+ Host Software QLogic Corporation All rights reserved Table of Contents 1 2 3 4 5 6 7 8 Version Changes 2.1 Changes to Hardware Support 2.2 Changes to OS Support 2.3 Changes to Software Components 2.4 Changes to Industry Standards Compliance Bug Fixes Known Issues Additional Information 5.1 Included in this Release 5.2 Operating Systems Supported in this Release 5.3 Lustre and GPFS Versions Verified for this Release 5.4 InfiniBand Host Channel Adapters Supported in this Release 5.
5.1.0.3.14 6.0.0.0.54 6.0.0.1.2 6.0.0.2.7 6.0.1.0.30 6.0.1.1.3 6.0.2.0.28 6.0.2.1.11 For detailed information about any of the previous releases listed, refer to the Release Notes for the specific version. 2.1 Changes to Hardware Support Added hardware support for the following releases: Release 5.1.0.0.49 o QLE7340 o QLE7342 o QMH7342 Release 5.1.0.2.1 o None Release 5.1.0.2.8 o None Release 5.1.0.3.14 o QME7342 Release 6.0.0.1.2 o None Release 6.0.0.2.7 o None Release 6.0.1.0.30 o None Release 6.0.1.1.
2.6.18-164.6.1.el5 Release 5.1.0.3.14 o Red Hat EL5 X86_64 (AMD Opteron, and Intel EM64T) (Update 4) 2.6.18-164.9.1el5 o Platform Cluster Manager 1.2b (RHEL 5.4) 2.6.18-164.6.1.el5 Release 6.0.0.0.54 o Red Hat EL4 X86_64 (AMD Opteron, and Intel EM64T) (Update 7) 2.6.9-78.0.1.ELsmp (Update 8) 2.6.9-89.ELsmp o Red Hat EL5 X86_64 (AMD Opteron, and Intel EM64T) (Update 3) 2.6.18-128.1.1.el5, 2.6.18-128, 1.6.el5, 2.6.18128.1.10.el5, 2.6.18-128.1.14.el5, 2.6.18-128.7.1.el5 (Update 4) 2.6.18-164.6.1.
2.3 Changes to Software Components Changes made to the software components for the following releases: Release 5.1.0.0.49 o QLogic OFED+ Software o QLogic InfiniBand Tools Software Release 5.1.0.2.1 o QLogic OFED+ Software o QLogic InfiniBand Tools Software Release 5.1.0.2.8 o QLogic OFED+ Software o QLogic InfiniBand Tools Software Release 5.1.0.3.14 o QLogic OFED+ Software o QLogic InfiniBand Tools Software Release 6.0.0.0.54 o QLogic OFED+ Software o QLogic InfiniBand Tools Software Release 6.0.0.1.
3 Bug Fixes The following fixes have been made to the QLogic OFED+ Host software package between versions 5.1.0.0.49 and 6.0.2.1.11: Fixed in Release 5.1.0.0.49 o Test case mpi_fork no longer fails when using openmpi. o When installing QLogic OFED+ Host software or QLogic InfiniBand Fabric Suite software packages on a server configured with Ethernet bonding , the server no longer hangs when IPoIB bonding is being installed.
o o Open Subnet Manager (SM) now supports Multicast Forwarding Table Subnet Administration (SA) queries. FastFabric features such as route analysis using iba_report or iba_saquery are available when using opensm. The OFED command ibcheckerrors no longer displays FAILED for xEdge and 12100 switches. Fixed in Release 6.0.0.1.2 None Fixed in Release 6.0.0.2.7 None Fixed in Release 6.0.1.0.30 None Fixed in Release 6.0.1.1.3 None Fixed in Release 6.0.2.0.
Known Issue When canceling out of a Host Channel Adapter firmware update, the following message is shown: HCA update failed. Return code: 1 at /usr/bin/qlgc_firmware_tool line 264, STDIN line 2. Workaround This message can be ignored; it indicates that the Host Channel Adapter firmware was not updated due to the user canceling the operation. For SLES10 SP2, the --32bit option of INSTALL will not work.
Known Issue When installing the QLogic-Basic or QLogic-IFS SW on SLES10SP3, there may be conflicts with software that is already installed on the system. The following message may appear: error: %preun(ofed-1.4.1-0.14.9.x86_64) scriptlet failed, exit status 1 Workaround Manually uninstall the old version OFED before you install the newer software. 1. As root, run the following command: rpm -e --noscripts ofed 2. Re-run the normal installation.
Known Issue Workaround Running MPI microbenchmarks over verbs can exhibit low Use MPIs built for use with PSM, QLogic's performance using the osu_bw and osu_bibw tests. It is not high-performance interface for MPI on QLogic Host Channel Adapters. These known how much this affects applications performance. MPIs are the MPIs that are listed in the mpi-selector --list with "_qlc" as part of the name, or QLogic MPI. The ib_send_bw microbenchmark that comes with OFED Use the -a option. 1.5.
Scientific Linux X86_64: o (Scientific Linux 5.3) 2.6.18-128.el5 o (Scientific Linux 5.4) 2.6.18-164.el5 o (Scientific Linux 5.5) 2.6.18-194.el5 Rocks: o (Rocks 5.3.2) 2.6.18-194.el5 Platform Cluster Manager 2.0.1: o (RHEL 5.5) 2.6.18-194.el5 CPU model of Linux kernel can be identified by uname -m and /proc/cpuinfo as follows: Model EM64T ® Opteron Uname /proc/cpuinfo x86_64 Intel CPUs x86_64 AMD CPUs NOTE: Other combinations (such as i586 uname) are not currently supported. 5.
5.5 Performance Tuning Tuning for verbs performance can be accomplished by making the following changes. 5.5.1 Intel CPU Systems (in general) Set the BIOS using the following procedure: 1. 2. 3. 4. Set PCIe parameter MaxPayLoad = 256 bytes Set PCIe parameter MaxReadRequest = 4096 bytes Disable all C-States. Disable Intel Hyper-Threading technology If the BIOS does not support the MaxPayLoad and MaxReadRequest settings, add the following parameter to the /etc/modprobe.
For 8-core, Intel Harpertown, Xeon 54xx, nodes: options ib_qib singleport=1 pcie_caps=0x51 pcie_coalesce=1 krcvqs=8 rcvhdrcnt=4096 For 16-core, 4x Intel quad-core CPUs, nodes: options ib_qib singleport=1 pcie_caps=0x51 rcvhdrcnt=4096 (Leave all 16 contexts for PSM) 5.5.3 Intel Nehalem or Westmere CPU Systems (DIMM configuration) Compute node memory bandwidth is important for HPC application performance and for storage node performance.
5.5.5 AMD CPU Systems (additions for Lustre/GPFS over verbs RDMA) One of the following parameters sets up the QLogic driver to accomplish these settings for Lustre/GPFS over verbs RDMA.
QLogic OFED+ Host software package version 6.0.1.0.30 includes the following new feature: OFED has been upgraded to version 1.5.1. QLogic OFED+ Host software package version 6.0.0.2.7 and 6.0.0.1.2 do not have any new features. QLogic OFED+ Host software package version 6.0.0.0.54 includes the following new features: OFED has been upgraded to version 1.5. CLI commands INSTALL and iba_config now have a --answer keyword=value option.
By default the installation will no longer start iba_mon. Use of iba_mon is not recommended when using the 6.0 QLogic FM and its new PM capabilities. The QLogic Distributed SA (qlogic_sa) is now available. This feature can be used by MPIs using the QLogic optimized PSM API to obtain proper vFabric, LMC and/or Mesh/Torus path information. CLI command iba_saquery now supports vfinfo, vfinfocsv and vfinfocvs2 output to provide information about virtual fabrics in the QLogic FM.
CLI command depcheck option --without has been added to INSTALL to suppress checking of the OS dependencies. This option can allow an install to be forced even if the OS dependencies are missing. This command should be used with care, because using a forced install will typically result in operational problems due to missing OS libraries or tools.
At the time of this release, Oracle has not certified InfiniBand adapters for RDS. As a result, QLogic is unable to extend support for RDS at this time. At the time of this release, the PathScale Compiler Suite is not supported for SLES11. As a result, QLogic is unable to extend support for this environment. The 10m Amphenol Cables have exhibited inconsistent reliability and are not recommended for use with TrueScale InfiniBand Adapters. 5.
If a non-blank output occurs, then debuginfo is enabled. If a blank line is output, then debuginfo is disabled on the present system. When running QLogic SRP over two ports in round robin mode, performance is degraded. When running verbs-based benchmarks such as ib_send_bw, the following message may appear: BUG: soft lockup - CPU#1 stuck for 10s. This message does not impact completion or correctness of the benchmark.
The following error may occur on rare occasions with Intel MPI/uDAPL: unexpected DAPL event 4008 from ... This problem is caused by a limitation in Intel MPI/uDAPL's dynamic connection mechanism when MPI processes are not paying sufficient attention to incoming interconnect traffic. To work around this problem, add the following to the mpirun command: genv I_MPI_USE_DYNAMIC_CONNECTIONS 0 An SRP write performance bug between QLE7240/QLE7280 and LSI (Engenio) storage systems was fixed.
The MPI sample applications installed in /usr/local/src/mpi_apps do not support the Intel 11 compiler. When using the Intel 11 compiler, the HPL sample application is expected to fail to build. QLogic recommends using the GNU compilers, or earlier versions of the Intel compiler when using these sample applications. Alternatively, for HPL runs, HPL can be found on the web along with alternative Math Libraries that may work with newer versions of the Intel compilers.
