EMC® Host Connectivity with QLogic Fibre Channel and iSCSI Host Bus Adapters (HBAs) and Converged Network Adapters (CNAs) for the Linux Environment P/N 300-002-803 REV A09 EMC Corporation Corporate Headquarters: Hopkinton, MA 01748-9103 1-508-435-1000 www.EMC.
Copyright © 2001–2008 EMC Corporation. All rights reserved. Published December, 2008 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Contents Preface ............................................................................................................................. 7 Chapter 1 Introduction Purpose of this document................................................................ 12 Host connectivity .............................................................................. 12 Fibre Channel ..............................................................................12 Fibre Channel over Ethernet ...........................
Chapter 4 Installing and Configuring the Linux Host with the QLogic Driver Introduction ....................................................................................... 38 Supported in kernel driver versions .............................................. 39 Installation instructions for the in kernel QLogic driver for Linux 2.4.x kernel.............................................................................. 42 Enabling the QLogic driver in RHEL 3.0 ................................
Chapter 5 Updating the CEE/Menlo Firmware Updating the QLogic CEE /Menlo firmware for FCoE adapters .............................................................................................. 82 Chapter 6 Connecting to the Storage Zoning and connection planning in a fabric environment ......... 84 Planning procedure ....................................................................84 Establishing connectivity to the storage array........................
Chapter 8 Additional Notes Ethernet connectivity over the CNA............................................ 104 Device reconfiguration procedures.............................................. 105 Adapter information for RHEL5 and SLES10 ............................ 106 SNIA API for third-party software (EMC ControlCenter and Solution Enabler)..................................................................... 107 OS upgrade from supporting out of kernel driver to OS version supporting in kernel driver......
Preface As part of an effort to improve and enhance the performance and capabilities of its product line, EMC from time to time releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all revisions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes.
Preface Related documentation Conventions used in this document The following related documents are available on Powerlink: ◆ EMC Host Connectivity Guide for Linux ◆ EMC Linux iSCSI Attach Release Notes ◆ EMC Networked Storage Topology Guide ◆ For information on Navisphere, refer to the Navisphere Management software documentation in the Documentation Library on EMC Powerlink at http://Powerlink.EMC.com. EMC uses the following conventions for special notices.
Preface Where to get help Italic: Used in all text (including procedures) for: • Full titles of publications referenced in text • Emphasis (for example a new term) • Variables Courier Used for: • System output, such as an error message or script • URLs, complete paths, filenames, prompts, and syntax when shown outside of running text Courier bold Used for: • Specific user input (such as commands) Courier italic Used in procedures for: • Variables on command line • User input variables <> Angle br
Preface 10 EMC Host Connectivity with QLogic Fibre Channel and iSCSI (HBAs) and (CNAs) for the Linux Environment
1 Invisible Body Tag Introduction This document describes the procedures for installing an EMC-approved QLogic host bus adapter (HBA) or converged network adapter (CNA) into a Linux host environment and configuring the host for connection to an EMC storage array over Fibre Channel or Fibre Channel over Ethernet (FCoE). ◆ ◆ ◆ ◆ ◆ Purpose of this document ................................................................. Host connectivity ....................................................................
Introduction Purpose of this document This document is meant to assist in the installation and configuration of QLogic host bus adapters (HBAs) and Fibre Channel Over Ethernet (FCoE) converged network adapters (CNAs) in Linux environments. The focus of this document is to enable the integrated Qlogic driver in the Linux distributions for Fibre Channel adapters, and to set up Linux hosts using the EMC-supported driver for Qlogic FCoE adapters, available from the EMC-approved section of the QLogic website.
Introduction ◆ In virtualization environments, where several physical storage and network links are commonly required. The installation of the QLogic FCoE CNA provides the host with an Intel-based 10 gigabit Ethernet interface (using the existing in-box drivers), and an QLogic Fibre Channel adapter interface, which requires the installation of the supported driver revision.
Introduction Zoning When using Linux hosts in a fabric environment, the zoning must be set up as single initiator and single target zoning. A single initiator/single target zone is composed of one adapter and one EMC storage array port. Storage array ports can be shared among adapters; however, each adapter must be in its own zone. Note: Multi-initiator zones are not recommended in a Linux fabric environment.
2 Invisible Body Tag Installation Steps This chapter outlines the prerequisisites for first-time installation, offers a summary of the installation steps with links to the appropriate sections, and provides information on installing the adapter. Review the EMC Support Matrix for the latest information on approved adapters and drivers. ◆ ◆ ◆ Prerequisites for first-time installation ........................................... 16 Summary of installation steps...............................................
Installation Steps Prerequisites for first-time installation In order to complete a first-time installation of the QLogic adpater in your server, you will need the following: Operating system ! ◆ “Operating system” on page 16 ◆ “QLogic SANSurfer and SANSurfer CLI” on page 16 ◆ “BIOS and firmware” on page 16 ◆ “Linux driver” on page 17 Before the adapter is installed, the Linux operating system must be installed and properly configured.
Installation Steps Follow the links to your adapters for the appropriate version. Linux driver The Linux driver for your HBA or CNA per theEMC Support Matrix for your supported configuration. EMC supports both in-kernel and out-of-kernel drivers. Note: The installation of the in-kernel driver occurs when you install your Linux distribution of choice.
Installation Steps Summary of installation steps Table 1describes the procedures for installing an EMC-approved QLogic adapters into a Linux host and configuring the host for connection to an EMC Storage Array over Fibre Channel (FC) or Fibre Channel over Ethernet (FCoE). Table 1 Step Instructions For Fibre Channel, refer to For Fibre Channel over Ethernet (FCoE), refer to 1 Install the adapter .
Installation Steps Table 1 Step Instructions 5 Install the firmware. There are two states: • Wrong firmware Installation steps (page 2 of 2) For Fibre Channel, refer to For Fibre Channel over Ethernet (FCoE), refer to The adapter firmware is part of the Linux driver and cannot be altered.. The adapter firmware is part of the Linux driver and cannot be altered. Proceed to Step 6. “Updating the QLogic CEE /Menlo firmware for FCoE adapters” on page 82 Once corrected, proceed to Step 6.
Installation Steps Installing the adapter Follow the instructions included with your adapter. The adapter installs into a single slot. To connect the cable to the adapter: 1. (Optical cable only) Remove the protective covers on each fiber-optic cable. 2. Plug one end of the cable into the connector on the adapter as shown in the appropriate figure in this step. (The hardware might be rotated 90 degrees clockwise from the orientation shown.
Installation Steps • Fibre Channel over Ethernet converged network adapter (CNA) connectivity options include LC optical and Cisco SFP+, shown next. – LC optical cable: 1 2 3 – Cisco SFP+ (Twinax cable) 3. Plug the other end of the cable into a connector on the storage system or a hub/switch port. 4. Label each cable to identify the adapter and the storage/switch/hub port to which it connects. 5. After connecting all adapters in the server, power up the server.
Installation Steps Servers have several different bus slot types for accepting adapters: ◆ ◆ ◆ ◆ PCI PCI-X PCI-X 2.0 PCI-Express PCI slots can be 32-bit and 64-bit (denoted by their 124-pin or 188-pin connectors.) These slots have plastic "keys" that prevent certain adapters from fitting into them. These keys work with the cutout notches in the adapter edge connector so that only compatible adapters will fit into them. This is done because of the voltage characteristics of the adapter. Inserting a 3.
Installation Steps Figure 2 shows the adapter edge connectors compatible with the PCI slots shown in Figure 1 on page 22. Note adapter 5, which shows a universal adapter edge connector. Universal adapters are compatible with both 3.3 V and 5 V PCI slots. Figure 2 Adapter edge connectors PCI-X (or PCI Extended) slots increase the speed with which data travels over the bus. PCI-X slots appear identical to a 64-bit PCI slot keyed for 3.3 V. (Refer to number 3 in Figure 1 on page 22 and Figure 2.
Installation Steps throughput. Because of how PCI Express slots are keyed, an x1 adapter can be inserted in all four slot types, as the adapter will negotiate with the slot to determine the highest mutually supported number of lanes. However, an adapter requiring x16 lanes will not fit into a smaller slot. Figure 3 PCI Express slots Figure 4 shows x1, x4, and x16 lane slots aligned on a mainboard. You can see how the slots are keyed so that low-lane adapters can fit into larger slots.
Installation Steps QLogic offers adapters for each bus/slot type available. Table 2 shows each of the EMC-supported QLogic adapters, and their respective slot requirements. Be sure to consult both your server user guide and QLogic to ensure that the adapter you want to use is compatible with your server's bus. Slot requirements of EMC-supported QLogic adapters Table 2 Adapter model Protocol PCI spec BUS length Power Slot key QLA2200F FC PCI 2.1 64-bit 3.3V, 5V Universal QLA200 FC PCI-X 1.
Installation Steps 26 EMC Fibre Channel with Emulex HBAs for the Linux v2.6.x Kernel Environment and the v8.
3 Invisible Body Tag Installing and Configuring the BIOS Settings This chapter describes the procedures for installing and configuring the BIOS settings. ◆ ◆ ◆ Verifying and configuring the BIOS settings ................................. 28 Manually setting the topology for QLogic adapters..................... 34 Manually setting the data rate for QLogic adapters .....................
Installing and Configuring the BIOS Settings Verifying and configuring the BIOS settings After the adapter is installed, follow these steps during system boot to verify and configure adapter firmware settings. To use SANsurfer or SANsurfer CLI for this function refer to the SANsurfer or SANsurfer CLI documentation you have downloaded. Refer to the EMC Support Matrix for required BIOS versions for qualified adapters.
Installing and Configuring the BIOS Settings d. Under Adapter Settings, note the BIOS version: – If the banner displays the required version, continue to “EMC recommended adapter BIOS settings” on page 31. – If the banner does not display the required version, upgrade the firmware as described under the “Upgrading the adapter BIOS” on page 29; then proceed to “EMC recommended adapter BIOS settings” on page 31.
Installing and Configuring the BIOS Settings Be sure to check the readme included with the BIOS files to make sure you have all of the appropriate files before proceeding. a. Insert a diskette into a Microsoft Windows 9x machine. b. Open any DOS window. c. At the DOS prompt, format the diskette by entering: format /s a: d. At the DOS prompt, change directory (cd) to the location of the saved zipped file, then extract the file to the diskette.
Installing and Configuring the BIOS Settings website. To update the BIOS using the SANsurfer CLI, refer to the QLogic provided documentation on their website for detailed instructions. To invoke the CLI, run the following command: /opt/QLogic_Corporation/SANsurferCLI/scli When upgrading the BIOS for QLogic 4 GB adapters, the minimum required version of the QLogic SANsurfer CLI is 1.06.16build23.
Installing and Configuring the BIOS Settings NVRAM settings, refer to the SANsurfer or SANsurfer CLI documentation from QLogic.
Installing and Configuring the BIOS Settings QLogic BIOS settings (page 2 of 2) Table 3 EMC recommended settings QLogic default No Multipath functionality With Multipath functionality Link Down Timeout 15 45 15 Extended Error Logging Disabled • Disabled (Do not use debugging) • Enable (Use debugging) • Disabled (Do not use debugging) • Enable (Use debugging) Operation Mode 0 0 0 Interrupt Delay Timer 0 0 0 Selectable Boot Disabled Disabled Disabled (Primary) Boot Port Name, LUN WW
Installing and Configuring the BIOS Settings Manually setting the topology for QLogic adapters The EMC default setting for the topology is set to 2 (Loop preferred; otherwise, point to point). For Linux environments, it is recommended that the Connection Options parameter be set to 1 when attached to a fabric and to 0 when attached to an EMC storage array directly. Follow these steps to set the NVRAM variables for the topology: 1. Boot the host.
Installing and Configuring the BIOS Settings Manually setting the data rate for QLogic adapters The EMC default setting for the data rate on the QLA23xx/QLE23xx adapters is Auto Select mode. If necessary, the mode may be set manually to 1 GB, 2 GB, or Auto Select mode. The EMC default setting for the data rate on the QLA24xx/QLE24xx 4 GB capable adapters is Auto Select mode. If necessary, the mode may be set manually to 1 GB, 2 GB, 4 GB, or Auto Select mode.
Installing and Configuring the BIOS Settings 36 EMC Host Connectivity with QLogic Fibre Channel and iSCSI (HBAs) and (CNAs) for the Linux Environment
4 Invisible Body Tag Installing and Configuring the Linux Host with the QLogic Driver This chapter describes the procedures for installing and configuring the driver. ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ Introduction ........................................................................................ 38 Supported in kernel driver versions ............................................... 39 Installation instructions for the in kernel QLogic driver for Linux 2.4.x kernel ..................................................
Installing and Configuring the Linux Host with the QLogic Driver Introduction Using the QLogic adapter with the Linux operating system requires adapter driver software. The driver functions at a layer below the Linux SCSI driver to present Fibre Channel devices to the operating system as if they were standard SCSI devices. EMC supports the QLogic in kernel default drivers in the distribution in most later operating systems.
Installing and Configuring the Linux Host with the QLogic Driver Supported in kernel driver versions Note: QLogic CNAs are not supported within kernel drivers. Refer to the “Installation instructions for the out of kernel QLogic driver” on page 50 for more information. Table 4 lists some examples of supported operating systems in kernel driver versions. Table 4 OS Supported in kernel driver versions (page 1 of 2) Driver version Supported adapters 1/2 Gb RHEL3 U2 6.07.02-RH2 √ SLES8 SP3 6.05.
Installing and Configuring the Linux Host with the QLogic Driver Table 4 OS Driver version Supported adapters 1/2 Gb 4 Gb 8 Gb SLES10 GA 8.01.04-k √ √ RHEL 4.5 OEL 4.5 8.01.04-d8 √ √ RHEL 4.6 OEL 4.6 8.01.07-d4 √ √ RHEL 4.7 Asianux 2.0 SP3 OEL 4.7 8.02.09-d0-rhel4.7-04 √ √ RHEL5.0 Asianux 3.0 OEL 5.0 8.01.07-k1 √ √ SLES 9 SP4 8.01.07.15 √ √ SLES10 SP1 8.01.07-k3 √ √ RHEL 5.1 Asianux 3.0 SP1 OEL 5.1 8.01.07-k7 √ √ RHEL 5.2 OEL 5.2 8.02.00-k5-rhel5.
Installing and Configuring the Linux Host with the QLogic Driver What next? If the driver version is not listed in Table 4, go to “Installation instructions for the out of kernel QLogic driver” on page 50. If the driver version is listed in Table 4, refer to the appropriate section: ◆ “Installation instructions for the in kernel QLogic driver for Linux 2.4.x kernel” on page 42 ◆ “Installation Instructions for the in kernel QLogic driver in Linux 2.6.
Installing and Configuring the Linux Host with the QLogic Driver Installation instructions for the in kernel QLogic driver for Linux 2.4.x kernel The section contains the following instructions for enabling the QLogic driver: ◆ “Enabling the QLogic driver in RHEL 3.0” on page 42 ◆ “Enabling the QLogic driver in SLES 8” on page 43 Enabling the QLogic driver in RHEL 3.0 To enable this driver, follow these steps: 1. Ensure that the /etc/modules.
Installing and Configuring the Linux Host with the QLogic Driver Enabling the QLogic driver in SLES 8 In order for the driver to be loaded at boot time, the driver must be listed in the /etc/sysconfig/kernel and /etc/modules.conf files and the ramdisk must be updated to reflect the changes. To enable the driver: 1. Edit /etc/sysconfig/kernel: vi /etc/sysconfig/kernel a. Add a reference to the QLogic qla2300.o driver in the INITRD_MODULES line: INITRD_MODULES="scsi_mod sd_mod mptscsih qla2300 reiserfs" b.
Installing and Configuring the Linux Host with the QLogic Driver 3. Create a new ramdisk to reflect the changes made: cd /boot mkinitrd -k vmlinuz-$1 -i initrd-$1 where $1 is the v2.4.x kernel version currently running. Example: cd /boot mkinitrd -k vmlinuz-2.4.21-295-smp -i initrd-2.4.21-295-smp 4. Reboot the system.
Installing and Configuring the Linux Host with the QLogic Driver Installation Instructions for the in kernel QLogic driver in Linux 2.6.x kernels If you are installing the OS after the adapter has been installed in the server, the OS will automatically detect the adapter, change the configure file, and build a RAM disk including the driver.
Installing and Configuring the Linux Host with the QLogic Driver 2. Whenever /etc/modprobe.conf is modified, a new ramdisk should be created to reflect the changes made. Create a new ramdisk image to include the newly added references to the QLogic adapters: cd /boot mkinitrd -v initrd-$1.img $1 where $1 is the v2.6.x kernel version currently running. Example: mkinitrd -v initrd- 2.6.9-55.ELsmp.img 2.6.9-55.ELsmp 3. Reboot the host. Enabling the QLogic driver in RHEL 5 To enable this driver: 1.
Installing and Configuring the Linux Host with the QLogic Driver Enabling the QLogic driver in SLES 9 In order for the driver to be loaded at boot time, the driver must be listed in the /etc/sysconfig/kernel and the ramdisk must be updated to reflect the changes. To enable the driver: 1. Edit /etc/sysconfig/kernel: vi /etc/sysconfig/kernel a. Add a reference to the QLogic driver in the INITRD_MODULES line: INITRD_MODULES="reiserfs qla2xxx_conf qla2xxx" b. Save the changes and quit from vi. 2.
Installing and Configuring the Linux Host with the QLogic Driver cd /boot mkinitrd -k vmlinuz-$1 -i initrd-$1 where $1 is the v2.6.x kernel version currently running. Example: cd /boot mkinitrd -k vmlinuz- 2.6.16.21-0.8-smp -i initrd2.6.16.21-0.8-smp 3. Reboot the system.
Installing and Configuring the Linux Host with the QLogic Driver Supported out of kernel driver versions Note: Only the QLogic v7.07.xx and the v8.01.xx driver provides support for the 4 GB-capable QLA24xx/QLE24xx-series adapters, not the less version driver. Table 5 lists the QLogic out of kernel driver versions supported with corresponding OS updates. These driver versions require manually installation.
Installing and Configuring the Linux Host with the QLogic Driver Installation instructions for the out of kernel QLogic driver This section contains the following information for installing the out of kernel QLogic driver: ◆ “Downloading the QLogic v7.x/v8.x-series driver for the v2.4/v2.6.x kernel” on page 50 ◆ “Preinstallation instructions for the QLogic v7.xx.xx/v8.xx.xx driver” on page 51 ◆ “Method 1: Installing the QLogic v7.xx.xx/v8.xx.
Installing and Configuring the Linux Host with the QLogic Driver Preinstallation instructions for the QLogic v7.xx.xx/v8.xx.xx driver Perform the following steps prior to the installation: 1. Stop all I/O. 2. Unmount all filesystems attached to the QLogic driver. 3. If the Naviagent/CLI is installed and enabled on the host, then the Naviagent/CLI service must be stopped. To stop the Naviagent/CLI service, issue one of the two following commands: /etc/init.
Installing and Configuring the Linux Host with the QLogic Driver Use the QLogic DKMS RPM to compile and install the modular driver for Dell servers and attached to EMC storage arrays. This method requires no manual edits for Dell servers attached to EMC storage arrays. By installing the DKMS RPM, the necessary files will be edited and the driver will be compiled and installed automatically. ◆ To create a modular v7.xx.xx/v8.xx.
Installing and Configuring the Linux Host with the QLogic Driver 1. Boot into the qualified and supported kernel onto which the driver will be installed. 2. Obtain the qla2xxx-v8.xx.xx1-2dkms.tgz package from the EMC-approved section of the QLogic website as instructed under the “Downloading the QLogic v7.x/v8.x-series driver for the v2.4/v2.6.x kernel” on page 50. 3. Uncompress and extract the source files from the tar archive: [root@l82bi205 extra]# tar zxvf qla2xxx-v8.xx.xx-2dkms.tar.
Installing and Configuring the Linux Host with the QLogic Driver Method 2: Installing the QLogic v7.xx.xx/v8.xx.xx driver via the QLogic installation script This section guides you through the process of installing and utilizing the QLogic installation script The script will build and install the driver and will modify the /etc/modprobe.conf.local and /etc/sysconfig/kernel files on SLES hosts.
Installing and Configuring the Linux Host with the QLogic Driver Note: EMC does not support the persistent binding implementation contained within the QLogic driver. As a result, it is recommended that the driver be compiled and installed without persistent binding enabled. This can be accomplished by using the 'dp' switch with the installation script. Proceed with the installation. cd qlafc-linux-8.xx.xx-1-install/.
Installing and Configuring the Linux Host with the QLogic Driver • To invoke the SANsurfer CLI (SCLI). The SANsurfer CLI (SCLI) is a command line interface that provides more flexibility for performing adapter configuration, device discovery, Option ROM and NVRAM updates, etc. RHEL examples An example of the console output reported by the QLogic installation script on RHEL hosts is as follows: .
Installing and Configuring the Linux Host with the QLogic Driver Building default persistent binding using SCLI Configuration saved on HBA port 0. Changes have persistent storage. Please reload the QLA driver module/rebuild the the saved configuration to take effect. Configuration saved on HBA port 1. Changes have persistent storage. Please reload the QLA driver module/rebuild the the saved configuration to take effect. been saved to RAM disk for been saved to RAM disk for Saved copy of /etc/modprobe.
Installing and Configuring the Linux Host with the QLogic Driver Port ID : 48-1B-00 Product Vendor : DGC Product ID : RAID 3 Product Revision : 0207 Node Name : 50-06-01-60-90-60-12-70 Port Name : 50-06-01-68-10-60-12-70 Product Type : Disk Number of LUN(s) : 26 Status : Online --------------------------------------------------------------------------------------------------------------------------------------------------------HBA Port 1 - QLA2342 Port Name: 21-01-00-E0-8B-39-9A-54 Port ID: 6B-0E-00 ------
Installing and Configuring the Linux Host with the QLogic Driver alias scsi_hostadapter2 qla2xxx alias scsi_hostadapter3 qla2300 SLES examples An example of the console output reported by the QLogic installation script on SLES host is as follows: ./qlinstall -i -dp #*********************************************************# # QLogic HBA Linux Driver Installation # # Version: 1.00.00b2pre4 # #*********************************************************# Kernel version: 2.6.5-7.
Installing and Configuring the Linux Host with the QLogic Driver Configuration saved. Changes have been saved to persistent storage. Please reload the QLA driver module/rebuild the RAM disk for the saved configuration to take effect. Configuration saved. Changes have been saved to persistent storage. Please reload the QLA driver module/rebuild the RAM disk for the saved configuration to take effect. Saved copy of /etc/sysconfig/kernel as /usr/src/qlogic/v8.00.03-1/backup/kernel-2.6.5-7.
Installing and Configuring the Linux Host with the QLogic Driver Product Vendor : DGC Product ID : RAID 3 Product Revision : 0207 Node Name : 50-06-01-60-90-60-12-70 Port Name : 50-06-01-62-10-60-12-70 Product Type : Disk Number of LUN(s) : 14 Status : Online ----------------------------------------------------------------------------Path : 0 Target : 2 Device ID : 0x83 Port ID : 61-1A-13 Product Vendor : DGC Product ID : RAID 3 Product Revision : 0207 Node Name : 50-06-01-60-90-60-12-70 Port Name : 50-06-
Installing and Configuring the Linux Host with the QLogic Driver Product Vendor : DGC Product ID : LUNZ Product Revision : 0217 Node Name : 50-06-01-60-88-20-12-BB Port Name : 50-06-01-68-08-20-12-BB Product Type : Disk Number of LUN(s) : 1 Status : Online --------------------------------------------------------------------------------------------------------------------------------------------------------HBA Port 0 - QLA2340 Port Name: 21-00-00-E0-8B-13-77-20 Port ID: 74-3B-13 ----------------------------
Installing and Configuring the Linux Host with the QLogic Driver Target : 3 Device ID : 0x84 Port ID : 6C-1B-13 Product Vendor : DGC Product ID : RAID 3 Product Revision : 0207 Node Name : 50-06-01-60-90-60-12-70 Port Name : 50-06-01-62-10-60-12-70 Product Type : Disk Number of LUN(s) : 14 Status : Online ----------------------------------------------------------------------------Path : 0 Target : 4 Device ID : 0x00 Port ID : 74-4A-13 Product Vendor : DGC Product ID : LUNZ Product Revision : 0206 Node Name
Installing and Configuring the Linux Host with the QLogic Driver An example of the installation script's modifications to the file /etc/modprobe.conf.local is as follows: remove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx && { /sbin/modprobe -r --ignore-remove qla2xxx_conf; } options qla2xxx ql2xfailover=0 ConfigRequired=0 Method 3: Installing the QLogic v7.xx.xx driver via the QLogic RPM This section guides you through the process of installing and utilizing the QLogic driver RPM.
Installing and Configuring the Linux Host with the QLogic Driver Please wait: Preparing qla2300 modular driver build copying kernel-2.4.21-i686-smp.config to /lib/modules/2.4.21-32.0.1.ELsmp/build/.config... Building Kernel Dependencies Make successful. File /usr/src/qla2x00/make.2.4.21-32.0.1.ELsmp.v7.03.00-1.log created. Saving copy of /etc/modules.conf as /etc/modules.conf.orig install -d -o root -g root /lib/modules/2.4.21-32.0.1.ELsmp/kernel/drivers/scsi/ install -o root -g root qla2200.o qla2300.
Installing and Configuring the Linux Host with the QLogic Driver Uninstallation methods for the QLogic v7.xx.xx/v8.xx.xx driver This section describes how to successfully uninstall the QLogic v7.xx.xx/v8.xx.xx driver from a Linux host. The method used to uninstall the driver is dependent upon the installation package: ◆ “Method 1: Uninstalling the QLogicv7.xx.xx/v8.xx.xx driver via QLogic DKMS RPM” on page 66 ◆ “Method 2: Uninstalling the QLogic v7.xx.xx/v8.xx.
Installing and Configuring the Linux Host with the QLogic Driver Method 2: Uninstalling the QLogic v7.xx.xx/v8.xx.xx driver via the QLogic installation script This section provides guidance for uninstalling the QLogic v7.xx.xx/v8.xx.xx driver via the QLogic installation script. The driver may be removed from the system with the 'u' switch for the qlinstall script. The following are example steps to uninstall the QLogic driver. Also refer to the README file in the driver package 1.
Installing and Configuring the Linux Host with the QLogic Driver Method 3: Uninstalling the QLogic v7.xx.xx driver via the QLogic RPM This section provides guidance for uninstalling the QLogic v7.xx.xx driver RPM. The QLogic RPM may be used to remove the driver with the e switch for the RPM command. The following are example steps: 1. Verify the currently loaded QLogic driver RPM: [root@l82bi235 extra]# rpm -qa | grep qla qla2x00-v7.03.00-1 2.
Installing and Configuring the Linux Host with the QLogic Driver QLogic SANsurfer and SANsurfer CLI QLogic's SANsurfer program is a GUI-based utility and the SANsurfer CLI is a text-based utility. Both applications may be installed on any Linux system and used to manage, configure, and update the EMC-approved QLogic adapters. Complete documentation and the EMC-qualified versions of SANsurfer and the SANsurfer CLI are available for download from the EMC-approved section of the QLogic website.
Installing and Configuring the Linux Host with the QLogic Driver 2. Enter tar –xvf qioctl-install.tar 3. Enter qioctl-install –install The following is an example of load IOCTL module on RHLE4 U5 and above with command: modprobe -v qioctlmod RPM packages needed for RHEL5 To run the SANsurfer installer under Redhat 5 Linux, if the default install is selected, the following RPMs need to be installed: compat-libstdc++-33-3.2.3-61..rpm libXp-1.0.0-8..
Installing and Configuring the Linux Host with the QLogic Driver QLogic v7.x and v8.x series driver parameters The QLogic driver contains a number of parameters that may be modified to perform failover functionality or to enhance performance. QLogic v7.x series driver parameters The QLogic and EMC recommended values are in Table 6 and descriptions of the parameters follow the table.
Installing and Configuring the Linux Host with the QLogic Driver Table 6 QLogic v7.
Installing and Configuring the Linux Host with the QLogic Driver Table 6 QLogic v7.x series driver parameters (page 3 of 3) Parameters QLogic default values EMC default recommendations qlFailoverNotifyType 0 0 recoveryTime 10 seconds 10 seconds failbackTime 5 seconds 5 seconds Description of QLogic v7.x-series driver parameters When attaching to CLARiiON and Symmetrix storage systems, EMC recommends that the ConfigRequired and ql2xfailover parameters be set to zero.
Installing and Configuring the Linux Host with the QLogic Driver ◆ ql2xintrdelaytimer: defines the amount of time for the firmware to wait before generating an interrupt to the host as notification of the request completion. ◆ retry_gnnft: defines the number of times to retry GNN_FT in order to obtain the Node Name and PortID of the device list. ◆ ConfigRequired: If set to 1, then only devices configured and passed through the ql2xopts parameter are presented to the OS.
Installing and Configuring the Linux Host with the QLogic Driver ◆ MaxRetriesPerIo: defines the total number of retries to perform before failing the command and returning a DID_NO_CONNECT selection timeout to the OS. ◆ qlFailoverNotifyType: defines the type of failover notification mechanism to use when a failover or failback occurs.
Installing and Configuring the Linux Host with the QLogic Driver Example: alias parport_lowlevel parport_pc alias eth0 eepro100 alias scsi_hostadapter cpqarray alias scsi_hostadapter1 cpqarray alias scsi_hostadapter2 qla2300 alias scsi_hostadapter3 qla2300 options scsi_mod max_scsi_luns=255 scsi_allow_ghost_devices=1 options qla2300 extended_error_logging=1 4. After the modification to /etc/modules.conf has been made, a new ramdisk needs to be created and the host rebooted.
Installing and Configuring the Linux Host with the QLogic Driver Setting the parameter values The values of the general driver parameters may be modified at the driver load time. This modification may be performed at the command line or by adding the parameters to the /etc/modprobe.conf file on RHEL 4.0 hosts and the /etc/modprobe.conf.local file on SLES 9 hosts and including them in the ramdisk.
Installing and Configuring the Linux Host with the QLogic Driver options qla2xxx extended_error_logging=1 3. After the modification to the file appropriate for your distribution, a new ramdisk needs to be created and the host rebooted. • For Red Hat distributions, use: cd /boot mkinitrd -v initrd-$1.img $1 where $1 is the currently running v2.6.x kernel version. Example: cd /boot mkinitrd -v initrd-2.6.9-22.ELsmp.img 2.6.9-22.
Installing and Configuring the Linux Host with the QLogic Driver version: license: description: author: alias: alias: depends: vermagic: 8.01.06 GPL QLogic ISP23xx FC-SCSI Host Bus Adapter driver QLogic Corporation pci:v00001077d00002312sv*sd*bc*sc*i* pci:v00001077d00002300sv*sd*bc*sc*i* qla2xxx 2.6.9-22.ELsmp SMP gcc-3.4 An example of the console output displayed when modinfo is run on the qla2xxx module is as follows: [root@l82bi205 ~]# modinfo qla2xxx filename: /lib/modules/2.6.9-22.
Installing and Configuring the Linux Host with the QLogic Driver parm: ql2xretrycount:Maximum number of mid-layer retries allowed for a command. Default value is 20, parm: qlport_down_retry:Maximum number of command retries to a port that returnsa PORT-DOWN status. parm: ql2xlogintimeout:Login timeout value in seconds. parm: ql2xmaxqdepth:Maximum queue depth to report for target devices. parm: ql2xtgtemul:Enable/Disable target combining emulation.
5 Invisible Body Tag Updating the CEE/Menlo Firmware This chapter provides information on updating the CEE/Menlo firmware for Fibre Channel over Ethernet adapters. ◆ Updating the QLogic CEE /Menlo firmware for FCoE adapters .....
Updating the CEE/Menlo Firmware Updating the QLogic CEE /Menlo firmware for FCoE adapters FCoE adapters include an additional chip component which requires the latest supported firmware. This chip is commonly referred to as a CEE (converged enhanced ethernet) or "Menlo" chip, the purpose of which is to handle the convergence of storage (FC) and network (IP) traffic over a single ethernet interface. To update the CEE/Menlo firmware on the CNAs, follow these steps: 1.
6 Invisible Body Tag Connecting to the Storage This chapter provides information on connecting to the storage. ◆ Zoning and connection planning in a fabric environment ..........
Connecting to the Storage Zoning and connection planning in a fabric environment In a fabric environment, the user should plan for the switch topology, target-to-hosts mapping, and the zone. Planning procedure The recommended procedure is as follows: 1. Draw the connectivity among the hosts, switch, and storage array to verify the correct fabric configuration. 2. Configure the zone capability in the switch.
7 Invisible Body Tag Configuring a Boot Device on an EMC Storage Array EMC supports booting Linux from an EMC storage array through an EMC-qualified QLogic Fibre Channel HBA. (Refer to the EMC Support Matrix for specific HBAs, BIOS revisions, and drivers.) ◆ Cautions and restrictions for booting from EMC storage array........ 86 ◆ Common limitations .......................................................................... Configuring a Symmetrix boot device ...........................................
Configuring a Boot Device on an EMC Storage Array Cautions and restrictions for booting from EMC storage array ! CAUTION If Linux loses connectivity long enough, the disks disappear from the system. To prevent further data from being lost in a situation like this, EMC recommends that the error behavior be changed from continue to remount read-only. To make this change, consult the manpage for tune2fs. A hard reboot is required to bring the system back to a usable state.
Configuring a Boot Device on an EMC Storage Array Symmetrix-specific cautions and restrictions ◆ Lost connection to Symmetrix storage array (pulled or damaged cable connection). ◆ Symmetrix service and upgrade procedures, such as on-line Symmetrix microcode upgrades and/or configuration changes. ◆ Symmetrix director failures, including failed lasers. ◆ Symmetrix storage array power failure. ◆ Storage area network service and upgrade procedures, such as firmware upgrades or hardware replacements.
Configuring a Boot Device on an EMC Storage Array Common limitations Boot configurations must not deviate from the following limitations established by EMC: ◆ The EMC Storage device must have enough disk space to hold the Linux operating system. ◆ A maximum of 32 volumes may be configured on the EMC storage array port that will be used to boot Linux. ◆ EMC recommends that you install, rather than upgrade, the kernel-source and kernel UP, SMP, and Hugemem RPMs.
Configuring a Boot Device on an EMC Storage Array Note: The EMC-recommended method is to use LUN masking. • CLARiiON ghost LUN — If the LUN 0 presented to a SCSI path is not owned by the SP attached to that path, a disconnected LUN 0 (ghost LUN) will be presented. The ghost LUN is not write-enabled so that the MBR cannot be written to it. Always ensure that the boot LUN is owned by the correct SP and does not trespass at any time; trespass of the boot LUN would result in a system crash.
Configuring a Boot Device on an EMC Storage Array ◆ Booting from the SAN requires the use of a Navisphere Management station with the Navisphere Manager or NaviCLI installed. The stations must be separate from the boot server, but networked to the CLARiiON storage array. Symmetrix-specific limitations 90 ◆ When attached to a Symmetrix, the physical-to-logical split must be 1:8 or less (1:7, 1:6, etc.). EMC supports the use of any LUN from 00 through FF as a boot disk.
Configuring a Boot Device on an EMC Storage Array Configuring a Symmetrix boot device This section describes how to install an EMC-qualified version of Linux onto an EMC Symmetrix storage array connected to an Intel-based x86 and x86_64 class systems and AMD Opteron-based x86_64 class systems. Preparing the Symmetrix storage array ◆ It is recommended that Volume Logix be enabled on the Symmetrix storage array for LUN masking purposes.
Configuring a Boot Device on an EMC Storage Array ◆ Disable the BIOS on any other adapters in the system other than the QLogic adapter designated for booting. Configuring the QLogic BIOS for SAN boot After the BIOS is installed and enabled, it must be configured for use for booting from the SAN. The following procedure describes how to configure the BIOS. 1. Connect the Symmetrix Fibre Channel port to the adapter in the lowest-numbered PCI slot in the server.
Configuring a Boot Device on an EMC Storage Array 5. From the Configuration Settings menu, select Adapter Settings and press Enter. 6. From the Host Adapter Settings menu, select Host Adapter BIOS and press Enter to enable it if it is not already enabled. Note: Refer to Table 3 on page 32 for recommended settings. 7. Press ESC to exit the Configuration Settings menu. 8. From the Configuration Settings menu, select Selectable Boot Settings and press Enter. 9.
Configuring a Boot Device on an EMC Storage Array Configuring a CLARiiON boot device This section describes how to install an EMC-qualified version of Linux onto an EMC CLARiiON storage array connected to an Intel-based x86 and x86_64 class systems and AMD Opteron-based x86_64 class systems. Preparing the CLARiiON storage array ◆ It is recommended that Access Logix™ be enabled on the CLARiiON storage array for LUN masking purposes.
Configuring a Boot Device on an EMC Storage Array ◆ SCSI hard disks are allowed in SAN boot configurations. However, the BIOS for the disk's SCSI adapters must be disabled. Any SCSI disks attached to the host should be disconnected during the operating system installation. ◆ For servers with SCSI CD-ROM drives, ensure that the BIOS is enabled on the SCSI channel that includes the CD-ROM. Disable the BIOS on any other integrated SCSI channels.
Configuring a Boot Device on an EMC Storage Array • If there are multiple QLogic adapters, a list of addresses occupied by those adapters appears. Since the EMC storage array is attached to the lowest-numbered PCI slot, select the first adapter from the list; then press Enter. The Fast!UTIL Options menu appears. 4. From the Fast!UTIL Options menu, select Configuration Settings and press Enter. 5. From the Configuration Settings menu, select Adapter Settings and press Enter. 6.
Configuring a Boot Device on an EMC Storage Array 13. When the QLogic banner is displayed (as shown in step 2), press Ctrl-Q. 14. Once the Fast!UTIL loads, select the Configuration Settings menu and press Enter. 15. From the Configuration Settings menu, select the Selectable Boot Settings menu and press Enter. 16. From the Selectable Boot Settings menu, select Selectable Boot Device and press ENTER to enable this option if it is not already enabled. 17. Select Current Boot Node Name and press Enter.
Configuring a Boot Device on an EMC Storage Array Using the installation CDs, boot the server to be installed. The installation may be performed as it would when using an internal adapter/controller driver. Once the installation is complete, use QLogic driver installation script to upgrade the driver to the EMC-qualified driver version as follows: ./qlinstall -up Note: Refer to the EMC Support Matrix for the most recent information on qualified kernel and driver releases as well as architectures.
Configuring a Boot Device on an EMC Storage Array Installing RHEL 5.2 and SLES 10 SP2 operating systems onto a boot device using FCoE Adapters EMC supports booting from an array device in FCoE environments with RHEL 5.2 and SLES 10 SP2 operating systems. Because RHEL 5.
Configuring a Boot Device on an EMC Storage Array 5. Insert the disk (described in Step 1) into either the floppy disk drive or the CD drive, depending on the option selected in Step 4. 6. Click OK, then press ENTER. The SCSI driver is loaded automatically. 7. The Disk Driver window displays, prompting for more drivers to install. Click NO, then press ENTER. 8. Insert the current Linux Red Hat product CD #1 in the CD drive (remove the iso-dd-kit CD first if necessary), then press ENTER. 9.
Configuring a Boot Device on an EMC Storage Array The following message displays: Make sure that CD number 1 is in your drive. 8. Put SLES 10 CD 1 in the drive and press OK. 9. Follow the on-screen instructions to complete the installation. Installing RHEL 5.
Configuring a Boot Device on an EMC Storage Array 102 EMC Host Connectivity with QLogic Fibre Channel and iSCSI (HBAs) and (CNAs) for the Linux Environment
8 Invisible Body Tag Additional Notes This chapter provides additional notes to conside ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ Ethernet connectivity over the CNA ............................................. 104 Device reconfiguration procedures ............................................... 105 Adapter information for RHEL5 and SLES10.............................. 106 SNIA API for third-party software (EMC ControlCenter and Solution Enabler)..............................................................................
Additional Notes Ethernet connectivity over the CNA The QLogic FCoE CNA delivers lossless 10 Gb/s Enhanced Ethernet support with dynamic allocation of networking and storage bandwidth that may be used for either system ethernet or iSCSI traffic, as well as FCoE. The Linux driver that supports the ethernet and iSCSI traffic for this device is ixgbe. The driver will automatically be installed and loaded by your supported Linux distribution.
Additional Notes Device reconfiguration procedures There are three methods to reconfigure the devices added or removed in the system. Method 1: Reboot the system : shutdown -r now Method 2: Remove and reinsert the modular driver For example: modprobe -rv qla2400 modprobe -v qla2400 Method 3: Use QLogic script to dynamically scan the devices. QLogic has the QLogic FC HBA LUN Scan Utility which is available from the EMC-approved site on the QLogic website.
Additional Notes Adapter information for RHEL5 and SLES10 QLogic fully supports upstream driver using sysfs start from RHEL5 and SLES10. QLogic adapter information is not available on /proc file system. To get the QLogic adapter information, you can manually go to /sys file system to probe all the necessary information. QLogic provides a script tool to help. You can download QLogic FC HBA Information Utility from EMC-approved site on the QLogic website.
Additional Notes SNIA API for third-party software (EMC ControlCenter and Solution Enabler) For the OS version that supports in kernel driver, SNIA API library shall be installed in the host to display QLogic adapter information for EMC products such as EMC ControlCenter and Solution Enabler usage. For the OS version that supports out of kernel drivers, the installation script will install the API library as well as FC driver.
Additional Notes OS upgrade from supporting out of kernel driver to OS version supporting in kernel driver When RHEL or SLES is upgraded from supporting the OS version that supports the out of kernel driver to the OS version that supports the in kernel driver, old entries in the configure file will not be delete. For Qlogic in kernel driver, the following features are disabled: ◆ ◆ Persistent binding QLogic failover QLogic driver parameter ConfigRequired and ql2xfailover do not need to be set.
Additional Notes ◆ If PowerPath is installed and enabled on the host, then the PowerPath service must be stopped. To stop the PowerPath service, issue one of the two following commands: /etc/init.d/PowerPath stop or service PowerPath stop ◆ If the QLogic SANsurfer daemon qlremote is installed and enabled on the host, then the qlremote service must be stopped in order for the driver to be removed from the currently running kernel.
Additional Notes running kernel if they are not in use and if other modules are not dependent upon them. The v8.x series driver consists of multiple modules. For example, if the command lsmod is invoked on a server with a QLA2340-E-SP adapters installed, the following three modules will be reported: ◆ qla2xxx_conf - The QLogic Linux driver configuration module containing information regarding persistent binding. ◆ qla2xxx - The low level QLogic Linux adapter driver module.
Additional Notes Device reconfiguration: Device numbering In the Linux kernel, the SCSI addresses are not used in the device names as they are in other types of UNIX (Sun, SGI, HP-UX, and BSD, for example). Block device filenames take the form /dev/sd ln, where l is the letter denoting the physical drive and n is the number denoting the partition on that physical drive. Disk device file names and major and minor numbers are assigned dynamically at boot time or device loading time in the order of discovery.
Additional Notes HPQ server-specific note When using HPQ systems, it is highly recommended that the HPQ SmartStart CD be run to configure the HPQ server prior to installing the Linux operating system. The SmartStart CD is shipped by HPQ with their systems and is a bootable CD that is used to configure HPQ servers. If another operating system is selected other than Linux, there may be problems installing the operating system or using the drivers installed in the kernel.
Additional Notes (CLARiiON Only) disconnected ghost LUNs When a Linux host is attached to both SPs in a CLARiiON storage array, the driver will report a disconnected LUN 0 on SPB and a failure to read the capacity of the device. The Navisphere® Host Agent requires that disconnected LUN 0 be reported properly. A device file name is allocated to the disconnected LUN 0 in the /dev filesystem, but the device cannot be mounted, partitioned, or otherwise accessed.
Additional Notes 114 EMC Fibre Channel with Emulex HBAs for the Linux v2.6.x Kernel Environment and the v8.
A Invisible Body Tag Setting Up External Boot for IBM Blade Server HS40 (8839) This appendix contains information on setting up external boot for IBM Blade Server HS40. ◆ Configure HS40 BladeCenter server to boot from external array....
Setting Up External Boot for IBM Blade Server HS40 (8839) Configure HS40 BladeCenter server to boot from external array IBM HS40 (8839) Blade Servers encounter a dual-port adapter conflict when attempting to configure boot BIOS to boot from an external array. To configure an HS40 BladeCenter server to boot successfully follow the steps below. 1. Create a single zone containing the adapter port from which you want to boot. This prevents any conflicts with the other fibre port. 2.
B Invisible Body Tag Special Instructions This appendix contains special instructions for the following: ◆ ◆ CLARiiON CX200 direct-connect dual-host Oracle9i RAC or RHEL 2.1 Cluster Manager cluster configurations with QLA234x adapters ........................................................................... 118 Setting the FC-AL loop ID for CLARiiON CX200 directconnect Oracle9iRAC and RHEL 2.1 Cluster Manager configurations with QLogic QLA234x-Series adapters ..............
Special Instructions CLARiiON CX200 direct-connect dual-host Oracle9i RAC or RHEL 2.1 Cluster Manager cluster configurations with QLA234x adapters For CLARiiON CX200 direct-connect dual-host Oracle9i RAC or RHEL 2.1 Cluster Manager cluster configurations with QLA234x adapters, the default adapter optic jumper position must be changed. ! CAUTION Modifying the jumper setting without using the recommended firmware and/or drivers may cause a loss of connectivity. 1.
Special Instructions Setting the FC-AL loop ID for CLARiiON CX200 direct-connect Oracle9iRAC and RHEL 2.1 Cluster Manager configurations with QLogic QLA234x-Series adapters The FC-AL Loop ID for QLA234x-series adapters must be set manually when directly attaching RHEL 2.1 hosts in Oracle9iRAC or RHEL 2.1 Cluster Manager configurations. Follow the steps below to enable hard addressing and to set the loop ID on each adapter. Perform this procedure on both nodes in the cluster connected to the CX200.
Special Instructions 120 EMC Host Connectivity with QLogic Fibre Channel and iSCSI (HBAs) and (CNAs) for the Linux Environment
Index B H BIOS Settings 28 Version 28 boot configuration 88 boot device cautions and restrictions 86 boot disk 86 booting from an EMC storage array 86 HBA, installing 20 C CEE/Menlo firmware 82 CLARiiON 113 disconnected ghost LUNs 113 CNA 12 CNA, Ethernet connectivity 104 configuration boot 88 converged network adapter 12 E I insmod 109 M Menlo (CEE) firmware 82 message url http //driverdownloads.qlogic.com/QLogicDriv erDownloads_UI/Oem_EMC.
Index 122 EMC Host Connectivity with QLogic Fibre Channel and iSCSI (HBAs) and (CNAs) for the Linux Environment
