Fibre Channel Adapter for VMware ESX User’s Guide FC0054606-00 A
S Fibre Channel Adapter for VMware ESX User’s Guide Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only.
Table of Contents Preface Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What’s in This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . License Agreements. . . . . . . . . . . . . . . . . . . . . . . . . .
Fibre Channel Adapter for VMware ESX User’s Guide S Configuring the Driver Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deploying NPIV for VMware ESX/ESXi . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring NPIV Virtual Ports from SANsurfer . . . . . . . . . . . . . . . . . . . . . . . Configuring Boot from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 3-6 3-9 3-9 3-11 Troubleshooting Troubleshooting with LEDs. . . . . . . .
Preface QLogic provides a portfolio of Fibre Channel (FC) adapters on the VMware® ESX/ESXi platform. The portfolio of QLogic FC adapters includes 2Gb, 4Gb, and 8Gb adapter offerings on VMware ESX/ESXi in PCI-X™, PCI Express®, and PCI Express ExpressModule™ form factors. This guide provides detailed instructions on the installation, configuration, and troubleshooting of QLogic FC adapters on VMware ESX/ESXi.
S What’s in This Guide This user’s guide provides a brief introduction to the FC adapter offerings from QLogic at a broad level. The main focus of this guide is to explain the FC adapter software driver features, management tools available on VMware ESX/ESXi, and the supported features. This preface covers the intended audience, related materials, document conventions used, license agreements, and technical support.
A Text in ALL CAPITALS indicates the name of a key that you press on the keyboard. For example: On the QLogic BIOS banner screen, press CTRL+Q. Text in italics indicates emphasis, terms, or document titles. For example: Refer to the VMware IO Compatibility Guide. Italicized text within angle brackets indicates user-defined variables. For example: cat /proc/scsi/qla2300/ Text within square brackets indicates optional parameters.
S License Agreements Refer to the QLogic Software End User License Agreement for a complete listing of all license agreements affecting this product. Technical Support Customers should contact their authorized maintenance provider for technical support of their QLogic switch products. QLogic-direct customers may contact QLogic Technical Support; others will be redirected to their authorized maintenance provider.
1 Hardware and Software Components This chapter provides details about QLogic FC adapters for VMware, including: “Hardware Components” on page 1-1 “Software Components” on page 1-3 “QLogic FC Adapters Supported on VMware ESX/ESXi” on page 1-4 Hardware Components A typical QLogic FC adapter includes the following major components: RISC processor Static random access memory (SRAM) Non-volatile memory (Flash) Non-volatile random access memory (NVRAM) (On ISP25xx, NVRAM is built into
S Hardware and Software Components Hardware Components 5 1 NVRAM: Non-Volatile Random Access Memory (aka EEPROM). Where some of the adapter parameter settings are stored. Because NVRAM is a non-volatile type of RAM, data is retained even if the power is shut off. 2 Transceivers: Short for transmitterreceiver, a device (copper or optical) that transmits and receives signals. 4 LEDs: Light Emitting Diodes. Display various active/inactive states of an adapter.
A Hardware and Software Components Software Components Table 1-1. FC Adapter Component Details (Continued) Callout Number 5 Description Non-volatile random access memory (NVRAM) is used to store the per-port adapter configuration parameters. These parameters are typically loaded at manufacturing and should not be modified except by those who have an expert level understanding of the functioning of QLogic FC adapters.
S Hardware and Software Components QLogic FC Adapters Supported on VMware ESX/ESXi Management Tools and Utilities QLogic offers a comprehensive suite of management tools and utilities, including the following: SANsurfer FC HBA Manager provides a graphical user interface for simplified management. SANsurfer FC HBA CLI (command line interface) enables you to automate FC adapter management by means of scripting and interactive mode.
2 Installation This chapter provides the following installation information: “Supported Features” on page 2-1 “Installing the Hardware” on page 2-2 “Installing the Driver” on page 2-3 “Installing SANsurfer Management Tools” on page 2-3 Supported Features When properly installed, QLogic FC adapters for VMware ESX support the following features: Topologies—FC-AL, FC-AL2, point-to-point, and switched fabric. Protocols—FCP-3-SCSI, FC-tape (FCP-2).
S Installation Installing the Hardware Installing the Hardware Adapter hardware installation requires that you open the computer and locate the appropriate bus slot for housing the adapter, as detailed in the following steps. Figure 2-1 shows the hardware placement. ADAPTER LEVER SCREW 64-BIT PCI-X SLOT ADAPTER RETAINING BRACKET OR x4 PCI EXPRESS SLOT (x8 PHYSICAL SLOT OR ABOVE REQUIRED FOR QLE25xx ADAPTERs) SLOT COVERS SYSTEM CHASSIS Figure 2-1 Installing the Adapter To install the adapter: 2-2 1.
A Installation Installing the Driver Installing the Driver The QLogic FC driver ships “inbox” with VMware ESX/ESXi. VMware’s patch process issues all driver updates. For details, refer to the VMware support Web site. Installing SANsurfer Management Tools The QLogic FC management tools include the SANsurfer FC HBA Manager (GUI) and the SANsurfer FC HBA CLI (command line interface).
S Installation Installing SANsurfer Management Tools NOTE: The qlremote agent requires a service called portmap running on the ESX host.
A Installation Installing SANsurfer Management Tools 6. Save the zipped file and then use it to install SANsurfer FC HBA Manager by following the installation wizard. 7. Install the qlremote agent on your ESX host using silent mode as follows: # ./standalone_sansurfer5.0.1b33_linux_install.bin -i silent To install SANsurfer FC HBA Manager on Linux or Windows: 1. Go to the QLogic Driver Downloads / Documentation page at the following URL: http://driverdownloads.qlogic.
Installation Installing SANsurfer Management Tools S Installing SANsurfer FC HBA CLI Follow these steps to download and install the SANsurfer FC HBA CLI utility. To install SANsurfer FC HBA CLI: 1. From the QLogic Web site or the SANsurfer Management Suite CD-ROM, download the scli-x.xx.xx-xx.i386.rpm.gz file to your local machine. (In the file name, x.xx.xx indicates the current version of SANsurfer FC HBA CLI. 2. To uncompress the file, execute the following command: # gunzip scli-x.xx.xx-xx.i386.
3 Configuration QLogic FC adapters on VMware ESX configuration procedures include the following: “Identifying QLogic FC Adapters on VMware ESX” on page 3-1 “Using the Fast!UTIL BIOS Utility” on page 3-4 “Verifying That the Driver is Loaded” on page 3-5 “Configuring the Driver Parameters” on page 3-6 “Deploying NPIV for VMware ESX/ESXi” on page 3-9 “Monitoring NPIV Virtual Ports from SANsurfer” on page 3-9 “Configuring Boot from SAN” on page 3-11 Identifying QLogic FC Adapters on
Configuration Identifying QLogic FC Adapters on VMware ESX S Figure 3-1 VI Client Configuration Page Using Console OS You can use the console OS to identify QLogic FC adapters on a VMware ESX system. To identify QLogic FC adapters using console OS: 1. To list the subdirectories, run the following command: ls /proc/scsi: If the qla2300 directory is present, QLogic adapters are installed on the VMware ESX system.
A Configuration Identifying QLogic FC Adapters on VMware ESX 2. To list the QLogic storage adapters present on the system, run the following command: ls /proc/scsi/qla2300 An example output may be: # ls /proc/scsi/qla2300/ 1 3. 2 3 4 5 6 HbaApiNode To view additional details, run the following command: cat /proc/scsi/qla2300/ Where is one of the adapter numbers shown in Step 2.
Configuration Using the Fast!UTIL BIOS Utility S Using the Fast!UTIL BIOS Utility QLogic FC adapters provide a BIOS utility, Fast!UTIL, which you can invoke at the BIOS. QLogic FC adapters support the Fast!UTIL BIOS utility on VMware ESX systems. Fast!UTIL does not require any installation.
A Configuration Verifying That the Driver is Loaded Verifying That the Driver is Loaded Follow this procedure to ensure that the QLogic FC adapter driver is loaded on the ESX system To verify that the QLogic FC adapter driver is loaded: 1. Log in to the service console as root user. 2. Execute the following command: # esxcfg-module -l | grep qla2300 Depending on the ESX Server version, the listed module will be one of the following: qla2300_707 on ESX 3.0.x qla2300_707_vmw on ESX 3.
S Configuration Configuring the Driver Parameters Configuring the Driver Parameters Use the esxcfg-module command to configure the QLogic FC adapter driver parameters. To configure driver parameters: 1. To list the driver parameters, execute the following command: # esxcfg-module -g qla2300_707_vmw 2. To set the driver parameters, execute the following command: # esxcfg-module -s qla2300_707_vmw where is one of the configurable parameters listed in Table 3-1 on page 3-7.
A Configuration Configuring the Driver Parameters Table 3-1. Configurable Driver Parameters Driver Parameter Description Bind Target persistent binding method. 0 = bind by port name (default); 1 = bind by port ID. extended_error_logging Option to enable extended error logging. 0 = no logging (default); 1 = log errors. max_srbs Maximum number of simultaneous commands allowed for an adapter. qfull_retry_count Number of retries to perform on Queue Full status from device. Default is 16.
S Configuration Configuring the Driver Parameters Table 3-1. Configurable Driver Parameters (Continued) Driver Parameter Description ql2xprocessnotready Option to disable handling of NOT-READY status in the driver. 1 = handled by the driver (default); 0 = disable the handling inside the driver. ql2xretrycount Maximum number of mid-layer retries allowed for a command. Default value in non-failover mode is 20, and in failover mode is 30.
A Configuration Deploying NPIV for VMware ESX/ESXi Deploying NPIV for VMware ESX/ESXi QLogic FC adapters support N_Port ID virtualization (NPIV) for FC SANs. NPIV enables each virtual machine to have its own FC world wide port name (WWPN) by creating multiple virtual adapters on a single physical adapter. VMware ESX 3.5/3i enables NPIV support on QLogic FC adapters. Because the QLogic FC driver shipped inbox on VMware ESX/ESXi supports NPIV, no additional software or hardware installation is required.
S Configuration Monitoring NPIV Virtual Ports from SANsurfer Figure 3-4 SANsurfer FC HBA Manager Showing NPIV When you select an individual virtual port on the Topology page, SANsurfer FC HBA Manager displays additional detail about the selected virtual port on four tabbed pages. The Information page (see Figure 3-5) displays port attributes such as Port ID, Firmware Version, and Actual Data Rate.
A Configuration Configuring Boot from SAN Figure 3-5 Virtual Port Information Page in SANsurfer Configuring Boot from SAN Boot from SAN refers to installing and booting a server’s operating system from a storage device attached to the SAN instead of from a locally attached storage device. The OS is installed on one or more LUNs in the SAN array and the FC adapters are configured to boot from the specified LUN.
S Configuration Configuring Boot from SAN VMware ESX 3.x provides the boot from SAN capability on QLogic FC adapters. For storage array and FC switch configuration, refer to the appropriate user manual for the device. For VMware ESX-specific configuration requirements, refer to the VMware Fibre Channel SAN Configuration Guide located at the following URL: https://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdf To configure the QLogic FC adapter for boot from SAN: 1.
A Configuration Configuring Boot from SAN 8. Enable Selectable Boot WWPN/LUN Boot List No x Yes None specified BIOS configures the first disk drive it finds that is also a LUN 0 as boot device. Yes Specified BIOS scans through the specified boot WWPN or LUN list until it finds a disk drive, and then configures it as the boot device. Device Boot BIOS configures the first disk drive it finds as boot device. a. Choose Selectable Boot Settings, and then press ENTER. b.
Configuration Configuring Boot from SAN S Notes 3-14 FC0054606-00 A
A Troubleshooting This appendix provides the following methods of determining the status of the adapters and finding solutions to common problems: “Troubleshooting with LEDs” on page A-1 “Dynamic Extended Logging” on page A-4 “Troubleshooting with Driver Logs” on page A-5 “Debugging Using Proc Information” on page A-10 “Collecting System Logs for Troubleshooting” on page A-11 “Frequently Seen Issues” on page A-11 Troubleshooting with LEDs QLogic FC adapters have LEDs that can help y
S Troubleshooting Troubleshooting with LEDs Table A-2. QLA234x/QLA210/QLE220 Adapter LED Scheme Green LED Amber LED Activity ❍ ❍ Power off ● ● Power ● ❍ Online ❍ ● Link ❍ Flashing Loss of synchronization Alternate flashing Alternate flashing Firmware error Flashing ❍ Beacon Table A-3.
A Troubleshooting Troubleshooting with LEDs Table A-4. QEM2462 Power and Attention Switch LED Scheme Green Power Activity ❍ Power off Insertion or removal of the adapter is permitted ● Power Insertion or removal of the adapter is not permitted Blinking Hot-plug operation is in progress and insertion or removal of the adapter is not permitted Yellow Attention Activity ❍ Normal operation ● Operational problem at this slot Blinking Slot is being identified at the user’s request Table A-5.
S Troubleshooting Dynamic Extended Logging Dynamic Extended Logging The QLogic driver provides a mechanism to dynamically enable extended error logging about driver operation. These logs provide debugging and error information to help you understand driver behavior regarding issues related to the adapter. These logs are saved in a system log file named vmkernel, in the /var/log/ directory.
A Troubleshooting Troubleshooting with Driver Logs Troubleshooting with Driver Logs All messages from the QLogic FC adapter driver are logged in /var/log/vmkernel. These messages provide additional information about the driver operation: both normal and driver warnings. Messages from the driver are usually preceded by scsi(%d), where %d is the adapter instance number. The following example shows a snippet from a vmkernel file: Nov 12 16:11:49 localhost vmkernel: 0:00:00:04.
S Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) "scsi(%d): 64 Bit PCI Addressing Enabled" Driver is reporting that it has configured the adapter for 64bit PCI bus transfers. "scsi(%d): Verifying loaded RISC code..." Driver is reporting that it has verified the RISC code and it is running. "scsi(%d): Verifying chip..." extended" Driver is reporting that it has verified the chip on the adapter. "scsi(%d): Waiting for LIP to complete...
A Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) Driver received an async ISP system error event from the firmware. Additional information follows the message (that is, mailbox values from the firmware). "scsi(%d): Configuration change detected: value %d." Driver received a change in connection async event from the firmware. Additional information follows the message (that is, mailbox 1 value from the firmware).
S Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) "%s(): **** CMD derives a NULL HA\n" Indicates the command does not point to an adapter structure. "%s(): **** CMD derives a NULL search HA\n" Indicates the command does not point to the adapter structure. "scsi(%ld:%d:%d:%d): now issue ADAPTER RESET.\n" Indicates an adapter reset is being issued to (host:bus:target:lun).
A Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) "WARNING qla2x00: couldn't register with scsi layer\n" Indicates the driver could not register with the SCSI layer, usually because it could not allocate the memory required for the adapter. "WARNING scsi(%d): [ERROR] Failed to allocate memory for adapter\n" Indicates the driver could not allocate all the kernel memory it needed.
S Troubleshooting Debugging Using Proc Information Debugging Using Proc Information The driver /proc/scsi/qla2300/ provides debugging information that can help you troubleshoot problems with the driver. The /proc entry provides firsthand information about the current state of the driver. Table A-8 lists the /proc output fields and provides a description of each. Table A-7.
A Troubleshooting Collecting System Logs for Troubleshooting Collecting System Logs for Troubleshooting To collect all relevant data on the running system into one tar file, execute the following command: # vm-support Frequently Seen Issues Table A-8 lists some common issues and questions and provides tips for their resolution. Table A-8. Frequently Seen Issues Issue Troubleshooting Tip What is the SANsurfer FC HBA Manager password? The password is config.
Troubleshooting Frequently Seen Issues S Notes A-12 FC0054606-00 A
Glossary adapter The board that interfaces between the host system and the target devices. Adapter is synonymous with host bus adapter (HBA), host adapter, and adapter board. arbitrated loop A circular (ring) topology (versus point-to-point) where two or more ports can be interconnected, but only two ports can communicate at a time. All communication passes through all ports connected to the loop. BIOS Basic input output system (typically in Flash PROM).
Fibre Channel Adapter for VMware ESX User’s Guide L2 cache—Secondary (larger) cache. Either on the processor chip or external to the processor. device A target, typically a disk drive. Hardware such as a disk drive, tape drive, printer, or keyboard that is installed in or connected to a system. In FC, a target device. driver The software that interfaces between the file system and a physical data storage device or network media.
A E_Port Expansion port. A port in an FC switch that connects to another FC switch or bridge device by an inter-switch link. E_Ports are used to link FC switches to form a multi-switch fabric. F_Port The “Fabric” port in a FC fabric switch provides a point-to-point link attachment to a single N_Port. F_Ports are intermediate ports in virtual point-to-point links between end ports, for example N_Port to F_Port to F_Port to N_Port using a single FC fabric switch.
S Fibre Channel Adapter for VMware ESX User’s Guide L_Port Loop port. Does arbitrated loop functions and protocols. NL_Ports and FL_Ports are examples of loop-capable ports. See E_Port, F_Port, FL_Port, G_Port, N_Port (Node Port), NL_Port (Node Loop Port). LIP Loop initialization process. The initialization process in an arbitrated loop that occurs when the loop is powered up or a new device is added. One function of a LIP is to assign addresses.
A NL_Port (Node Loop Port) A port capable of arbitrated loop functions and protocols. An NL_Port connects through an arbitrated loop to other NL_Port and at most a single FL_Port. NL_Ports handle creation, detection, and flow of message units to and from the connected systems. NL_Ports are end ports in virtual point-to-point links through a fabric, for example NL_Port to F_Port to F_Port to N_Port using a single FC fabric switch.
Fibre Channel Adapter for VMware ESX User’s Guide port Access points in a device where a link attaches. There are four types of ports, as follows: N_Port (Node Port)—an FC port that supports point-to-point topology. NL_Port (Node Loop Port)—an FC port that supports loop topology. FL_Port—a port in a fabric where an N_Port can attach. FL_Port—a port in a fabric where an NL_Port can attach. port instance The number of the port in the system.
Index A adapters definition of Glossary-1 identifying on VMware ESX 3-1 installing hardware 2-2 supported 1-4 adjunct driver level, definition of Glossary-2 agreements, license viii arbitrated loop, definition of Glossary-1 audience, intended v automated local host management, supported 2-1 B BIOS definition of Glossary-1 searching for SCSI devices 3-12 utility, Fast!UTIL 3-4 blue text, meaning of vii bold text, meaning of vi boot code definition of Glossary-1 software component 1-3 version viewing with pr
S Fibre Channel Adapter for VMware ESX User’s Guide driver definition of Glossary-2 installing 2-3 logs, troubleshooting with A-5 OS 1-3 parameters, configuring 3-6 verifying that it is loaded 3-5 dynamic extended logging A-4 E E_Port (Expansion Port), definition of Glossary-3 e-mailing QLogic support viii end user license agreements viii esxcfg-module command 3-6 extended BIOS data area (EBDA) A-11 extended error logging A-4 F F_Port, definition of Glossary-3 fabric device management interface, definit
A introduction to product v ioctl definition of Glossary-3 timeout value 3-7 ISP controller 1-2 italicized text, meaning of vii L L_Port (Loop Port), definition of Glossary-4 L1 cache Glossary-1 L2 cache Glossary-2 leaf driver level, definition of Glossary-2 LED indicators A-1 license agreements viii link rate, supported 2-1 Linux, SANsurfer, installing on 2-3 LIP async event (driver log message) A-6 definition of Glossary-4 waiting to complete (driver message) A-6 load balancing, definition of Glossary-4
S Fibre Channel Adapter for VMware ESX User’s Guide PID definition of Glossary-5 mid-level A-8 PLOGI (port login) definition of Glossary-5 enabling 3-7 point-to-point definition of Glossary-5 supported topology 2-1 port definition of Glossary-6 instance, definition of Glossary-6 port driver, definition of Glossary-2 port login (PLOGI) definition of Glossary-5 enabling 3-7 preface to guide v proc entry, debugging with A-10 proc output fields A-10 process identifier (PID), definition of Glossary-5 protocols
A T target definition of Glossary-6 devices, max queue depth 3-7 discovery, rescanning for A-11 mode, port A-10 numbers, different assigned A-11 OS, missing A-7 persistent binding method 3-7 technical support, obtaining viii telephoning QLogic support viii terms, definitions of Glossary-1 topologies, supported 2-1 training from QLogic viii troubleshooting A-1 driver logs A-5 dynamic extended logging A-4 frequently seen issues A-11 LED indicators A-1 proc information for debugging A-10 system logs A-11 typog
Corporate Headquarters QLogic Corporation 26650 Aliso Viejo Parkway Aliso Viejo, CA 92656 949.389.6000 Europe Headquarters QLogic (UK) LTD. Quatro House Lyon Way, Frimley Camberley Surrey, GU16 7ER UK www.qlogic.com +44 (0) 1276 804 670 © 2009 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLogic and the QLogic logo are registered trademarks of QLogic Corporation. VMware is a registered trademark of VMware, Inc.
