Dynamic Path Failover Driver for AIX FibreQuik Host Bus Adapter Installation and User Guide Cambex Corporation 115 Flanders Road Westborough, MA 01581 Customer support support@cambex.com Document: Date: Rev.
Table of Contents DYNAMIC PATH FAILOVER FOR AIX ................................................ 3 Introduction............................................................................................ 3 SAN Management................................................................................... 4 Storage Management ............................................................................. 4 Path Management ..................................................................................
DECLARATION OF CONFORMITY .................................................... 33 Warning................................................................................................ 33 SAFETY NOTICES ................................................................................... 34 FCC Warning....................................................................................... 34 Safety Notice ........................................................................................
Dynamic Path Failover for AIX Introduction The Cambex FibreQuik™ Fibre Channel host bus adapters provide attachment of Fibre Channel devices to PCI or Micro Channel based servers. They support the SCSI-FCP protocol. When used with the Cambex DPF (Dynamic Path Failover) software, two adapters in a system provide greater throughput and total protection from failure of a fibre channel component.
SAN Management SAN Management is the top layer. It is responsible for assigning the required access for each server to each storage device. The unit of storage is the LUN (Logical Unit Number) which acts the same as a single disk directly attached to the server. In reality, a LUN is made up of portions of a number of disks under the direction of a RAID controller in the storage device.
communicating via a network connection to each managed device. The Cambex Dynamic Path Failover driver is a Path Management component that allows multiple paths to a storage device and provides automatic failover between available paths. Cambex Dynamic Path Failover software provides both a command line interface and access through smit, the standard integrated management tool for AIX. SCSI and FCP Drivers AIX makes two types of device drivers available.
How Preferred Paths work The basic idea of multipathing is fairly simple. Access a disk on one of two paths. If the path fails, move I/O activity to the other path. We build on that basis by adding more factors into the configuration and seeing what effect they have. A path is defined as an end-to-end connection through a SAN, ignoring any intermediate routing points. A complex SAN may have multiple connections from an HBA to a storage device, but this is invisible to the software.
and set up preferences so that the remaining paths to the same controller are preferred over paths to the alternate controller. Example 1) Suppose we have two HBAs, a switch, and two RAID controllers. A total of four cables, and four paths to each LUN (2 HBAs x 2 RAID controllers). We use the RAID controller preference to select which controller each LUN is accessed on. The target preference lets me balance those accesses across the two HBAs.
are accessed by different hosts, we could have the problem of "thrashing". One host tries to move the LUN group to preferred controller A while the other host tries to move the group to preferred controller B. Anti-thrashing code prevents this problem in the multipathing software.
Adapter Specifications Bus Data rate (each dir.) PC2000LC-HPSP PC4000 (Single Port) PC2000LC-A PC4002 (Dual Port) PCI PCI-X 2.0 100/200 100/200/400 MB/s Bus Data rate (each dir.) PC4000e (Single Port) PC4002e (Dual Port) PCI-e x4 100/200/400 MB/s Cable length - copper - multimode optical - single mode optical Voltage Power Shipping weight 30 500 10 3.3 or 5 7.2 10 70 m 3.3 8.0 10 V W oz Trademarks Dynamic Path Failover and FibreQuik are trademarks of Cambex Corporation.
Software and Hardware Installation Driver Installation (command line) It is recommended that the drivers be installed prior to installing the HBAs. The examples assume that the driver CD is mounted at /mnt and that you are installing on AIX 5.1. The name and revision level of the uninstalled driver image may be examined by issuing the command (as root): installp -ld where is the path to the drivers, such as /mnt/AIX6/ for the CD or the name of a driver directory on the hard disk.
The new driver may be installed on the system with the command: installp -acd all where is as described above. example: installp -acd /mnt/AIX6.1 all After installing the driver, see the README file for additional information and release notes.
Driver Installation (smit) The smit path is Software Installation and Maintenance Install and Update Software Install and Update Software by Package Name For item " INPUT device / directory" use /mnt/AIX6.1 or the directory where the image files are located. A list should pop up with the name of the drivers. Select and press enter. Also make sure that the following drivers are loaded from the AIX installation media. devices.fcp.disk.rte devices.fcp.tape.
Hardware Installation Power down the system and install the adapter in a free slot. The PCI cards may be installed in either 32 or 64-bit PCI card slots, however the 32-bit slots may limit performance at high data rates. Make sure that any cables do not block airflow to the card. Observe proper static precautions. Detailed procedures for opening the case and for suggested adapter placement may be found in the Operator Guide for the specific server model.
AutoRecover cbxdpfar, the AutoRecover daemon, is automatically installed and enabled when the DPF driver is installed. Control of the daemon is done using SRC. stopsrc -s cbxdpfar Stops AutoRecover startsrc -s cbxdpfar Starts AutoRecover AutoRecover works by periodically polling the standby paths to each LUN. If the polls fail, the path is marked as failed. If the polls succeed, the path is marked as available for use.
AIX Configuration During power-up initialization, AIX will find and configure the new adapter(s) and any supported devices using the driver's configuration method. Devices are assigned a SCSI ID as they are discovered starting with 0. SCSI ID is not related to loop ID, port ID or to SCSI ID assignment on other hosts. It is a logical value bound to the fibre channel WWN of the device.
Persistent Bindings Persistent Bindings is the term used to indicate the mapping of SCSI ID to FC Worldwide Name (WWN). By keeping a persistent binding, the loop ID or port ID of a device may change and the device will still be properly addressed. In the Cambex AIX driver, persistent binding is automatic. At configuration time, all devices that are found are automatically assigned to a SCSI ID and a persistent binding is created. Persistent bindings are by portname and are kept in the ODM database.
SMIT Commands Commands The adapter management commands available in SMIT are in the devices menu under Cambex Fibre Channel Adapter: List all Cambex FC adapters This provides the name and location of each adapter. Change / Show Characteristics of a Cambex FC Adapter Allows the user to display and modify attributes. Attributes (and default values) include: exec_throttle (256) number of simultaneous I/O operations. dma_window (64) PCI DMA memory window in MB.
sw_fc_class (3) Fibre channel class. devtype_exclude ( ) List of SCSI device types to ignore during configuration. For example, specifying "1,5" would exclude both tape and CD devices. Attributes only used in loop mode include: id (0) Default L_ID for adapter. See table of L_ID to ALPA. login_on_lip (yes) Full login if LIP occurs. full_duplex (no) Use full duplex loop open, OPNyx. Generate Error Report Accesses the AIX error logging facility.
The autorecover daemon will automatically move operation to preferred paths. The preferred path is the combination of path preference values for HBA, device and path. For some arrays, target device preference is read automatically from the array at configuration time. Consult the array user manuals for more information. Change / Show Characteristics of Cambex Managed HBA’s Used to display and change HBA preference for one or more adapters.
for use until the purchased license is received. Issue the commands: cd /usr/lpp/cbxdpf mv license.30day license cfgmgr Licenses for OEM versions of the driver are obtained from the storage vendor who provided the software. The generic version of DPF purchased from Cambex is licensed for use only on Cambex cards and on only one LPAR. A license token is provided with the purchase. Contact Cambex support by email to obtain a license.
• • • failed or offline status is considered first. A path will not be used if failed or offline. Anti-thrashing prevents paths to the wrong controller from causing thrashing. HBA preference - normally all HBAs have equal preference to paths.
Troubleshooting Use the errpt command to identify the cause of failure. The lsdev command will provide the location of the component. For example, if errpt identifies a loose cable on device hdisk1, lsdev will give an address similar to: hdisk1 Available 00-02-01 Cambex SCSI...
Checklist for driver and HBA installation The following commands may be used to isolate problems during installation of the fibre channel driver and card. Type the command shown in bold and verify the output. must give: Fileset Path: /usr/lib/objrepos PC1000.driver.obj Level 01.05.00.00 (or greater) State Committed Description Cambex Fibre Channel Adapter Driver lslpp -l PC1000.driver.
cfgmgr -v >/tmp/cfg.log 2>&1 Should identify the card and configure it. If cfg.log contains the ID 7710002x and no driver is found, then the adapter driver is not installed. During boot, the number displayed when configuring the adapter is hex 0x911 (FC adapter). /usr/lpp/cbxfc/mbtest fscsi1 This test code may be used to get fibre channel status from the adapter or from attached devices. Type the command as above for a list of subcommands.
The message no failover devices available indicates that the attached storage is not yet recognized by the DPF driver. This could mean that a platform kit needs to be installed for your storage or that the storage is not yet configured. /usr/lpp/cbxfc/runtrace This should be run to start a trace if Cambex support requests a trace of the card failure. To stop the trace and generate a trace report, type: trcstop; trcrpt > /tmp/trc.
Error Messages Various errors and notifications will be placed in the AIX error log. These can be read using the errpt command, or automatic notification may be set up. DPF_FAILOVER This indicates that the active path to a device has failed, and that DPF successfully transferred I/O to an alternate path. DPF_STDBYDN This indicates that a path other than the active path has failed to a device.
Upgrading the Driver When upgrading to a newer version of the driver there are two methods: 1) Use rmdev to remove all dependent devices, remove the existing driver(s), then install the new driver(s) and run cfgmgr to recreate the devices. This method allows a new driver to be installed without rebooting the server. File systems which are unmounted before the process may be mounted again afterwards without affecting the data. 2) Install the new driver with devices in place.
Appendix A Identifying Fibre Channel Components Copper DE9 GBIC Copper DE9 Cable 28
Copper HSSDC Cable Duplex SC Optical GBIC 29
Small form factor (SFF) optical LC cable 30
PC2000 LED Meanings Top LED – Transceiver hardware is detecting a signal. Middle LED – Firmware is being synchronized. Bottom LED – Flashing amber when not synchronized.
PC4000 LED Meanings Y G A Y Off On Flash Off Off On/Flash ∗ G Off On Flash Off On/Flash Off A Off On Flash On/Flash Off Off No Power Power Applied Initializing * 1G (Flash indicates active) 2G (Flash indicates active) 4G (Flash indicates active) Flashing in unison indicates initialized, while alternate flashing indicates a firmware error.
Declaration of conformity We Cambex Corporation of Westborough, Massachusetts, USA, declare under our sole responsibility, that the products PC2000LC-HPSP, PC2000LC-A, and PC4000 are in conformity with the following standards: EN 55022:1994/A1:1995/A2:1997 Class A ITE emissions requirements EN 50082-1:1992 EMC residential, commercial and light industrial generic immunity standard Warning This is a Class A product.
Safety Notices FCC Warning This equipment has been tested and found to comply with the limits for a class A computing device pursuant to Part 15 of FCC rules which are designed to provide reasonable protection against interference when operating in a commercial environment. This equipment generates, uses and can radiate radio frequency energy and if not installed and used in accordance with the instruction manual may cause interference to radio communications.
Dynamic Path Failover License Agreement Cambex Dynamic Path Failover (the Software) is a licensed product. The user must consent to the provisions of this license agreement in order to use the Software. Use of the Software beyond the number of purchased licenses is prohibited. The user may not cause or permit copying, modification, rental, sublicense, lease, transfer, reverse engineering, or disassembly or recompilation of the Software, or creation of derivative works based on the Software.
