Part Number:313433501 Shared Virtual Array SVA Path Version 3.
Proprietary Information Statement The information in this document is confidential and proprietary to Storage Technology Corporation and may be used only under the terms of the product license or nondisclosure agreement. The information in this document, including any associated software program, may not be disclosed, disseminated, or distributed in any manner without the written consent of Storage Technology Corporation.
Table of Contents Who Should Read This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Shared Virtual Array Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii How to Obtain Software Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii SVA Administrator for HP-UX Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 3. SVA Path Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 The setsp Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 The spmon Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Configuring Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 5. Diagnosing Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Comparing setsp -a with System Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Cable Failure Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Failover Error Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi SVA Path User’s Guide
Preface This guide describes how to use the SVA Path Intelligent Data Path Management software. SVA Path provides improved performance and data accessibility for the StorageTek Shared Virtual Array (SVA). Who Should Read This Guide This guide is for data administrators, capacity planners, performance specialists, and system administrators. This guide assumes that you are familiar with Shared Virtual Array operations and HP-UX system administration.
• Online (for viewing and printing), at the StorageTek Customer Resource Center (CRC) website at: www.support.storagetek.com. Click on Software and go to the Shared Virtual Array Software list. Note: Access to the CRC site requires a password. To obtain a password, call StorageTek Customer Support at 800-678-4430.
SVA Hardware Publications Shared Virtual Array hardware publications are available from the following sources: • On the SVA Hardware Publications CD-ROM (part number 3118447nn). To order a copy, contact StorageTek Publication Sales and Service at 800-436-5554 or send a fax to 303-661-7367. • Online (for viewing and printing), at the StorageTek Customer Resource Center (CRC) website at: www.support.storagetek.com. Click on Disk Subsystems. Note: Access to the CRC site requires a password.
x SVA Path User’s Guide
Chapter 1. SVA Path Overview This chapter provides an overview of SVA Path Intelligent Data Path Management software and its features. SVA Path offers a new level of data accessibility and improved performance for the SVA 9500. It eliminates the point of failure represented by a single input/output (I/O) path between servers and storage systems and permits I/O to be distributed across multiple paths.
1. by allowing SVA Path to uniformly distribute primary paths among all available I/O paths. This is the default behavior of SVA Path. 2. by manually assigning I/O traffic for a logical drive to a particular path. The administrator with an understanding of the I/O load patterns of his or her applications can optimize performance through an intelligent choice of paths. 3. by enabling automatic load balancing.
Supported SVA Path Configurations SVA Path supports single server configurations. System Requirements Before proceeding to the next chapter, you should verify that your site meets the following minimum requirements (Table 1-1, “SVA Path Minimum System Requirements”). Table 1-1 SAM Support SVA Path Minimum System Requirements Host hardware: All 9000 series computers Host software: HP-UX 10.20 and HP-UX 11.x, 32 and 64 bit HBA 10.2, 11.0 HSC HP A3404B 10.2, 11.
Summary of SVA Path Benefits • Increases potential subsystem throughput by directing I/O through multiple host adapters and SVA channels. Logical drives can be assigned to host bus adapters, manually balancing the I/O load across paths. • Provides continuous access to mission-critical data by insulating server applications from I/O path failures. • Installs easily and is transparent to server applications.
Chapter 2. SVA Path Hardware Setup Fibre Addressing Concepts Host Bus Adapters/Initiators The terms “host bus adapter” and “initiator” mean essentially the same thing. Typically, the HBA is a card within the host that, in its role as initiator, issues commands on the Fibre channel. Domains in Fibre Channel Connection StorageTek uses the concept of “domains” to allow open systems hosts access to blocks of logical devices (the domains) within an SVA.
Figure 2-1 shows two data paths connecting the open systems platform to the attached SVA using Fibre cables. Shared Virtual Array Open Systems Platform ICF Cards Fibre Cable Fibre Cable Domain 0 Domain 1 Domain 2 Domain 3 Fibre Cable Fibre Cable ICF Cards Figure 2-1 Fibre Paths from the Host to the SVA Note: In the above figure, a domain can represent up to 255 logical devices (1 Target × 255 LUNs = 255 Logical Devices). There is a limit of 1024 logical devices with an SVA.
Domain Numbering with Fibre and SVA Path It is not normally advisable to have more than one path from a single host set to the same domain number to a given SVA (in case two hosts attempt to share the same LUN and so corrupt the data stored on it). SVA Path requires exactly this configuration for failover to work. SVA Path manages multiple paths from a single host, using identical domain numbers. Chapter 2.
Configuration in a Mixed SCSI/Fibre Environment Failover can only be accomplished from one SCSI interface to another, or from one Fibre interface to another. Figure 2-2 shows SCSI and Fibre data paths connecting the open systems platform to the attached SVA.
Hardware Preparation Configuring an HP System to See New Devices Perform the following steps on the HP host: Note: If any of the following steps do not produce the expected result, refer to your HP-UX system administrator documentation for instructions on setting up the host platform correctly. 1. For Fibre channel connection, contact your Custome Service Engineer to set domain addresses in the SVA 9500.
Class I H/W Path Driver S/W State H/W Type Description ================================================================================ disk 0 0/0/1/1.2.0 sdisk CLAIMED /dev/dsk/c1t2d0 disk 1 0/0/2/0.2.0 sdisk CLAIMED /dev/dsk/c2t2d0 disk 2 0/0/2/1.2.0 sdisk CLAIMED /dev/dsk/c3t2d0 disk 3 0/6/0/0.0.0 disk 4 0/6/0/0.1.0 sdisk CLAIMED /dev/dsk/c6t0d0 sdisk CLAIMED /dev/dsk/c6t1d0 disk 5 0/6/0/0.2.0 disk 6 0/6/0/0.3.
Chapter 3. SVA Path Operation This chapter describes SVA Path commands and configuration options. There are three basic commands in SVA Path • setsp is used to examine and configure the system’s operating parameters, and as such is the command most often invoked by the user. • spmon monitors path states and implements load balancing. • sppath identifies devices to be placed under SVA Path's control. The setsp Command The setsp command is used for most configuration tasks.
Command options that take arguments (shown in braces after the command) require an argument and should not be run without one. Table 3-1 Option setsp Command Options Effect -a show current device configuration and state -b{0|1} set load balancing for a logical drive. (pages 24 and 27).
Table 3-1 setsp Command Options Option Effect -L show current device configuration according to condition(s) defined by setsp command option parameter(s) (e.g.
Table 3-1 Option -u{0|1|2} setsp Command Options Effect show devices by their configuration status: -u0 shows all available devices; -u1 shows configured disks; -u2 shows unconfigured disks -v runs a command in verbose mode The spmon Command The spmon command is primarily associated with load monitoring and balancing of SVA Path functional devices (FDevs). FDev is a logical disk as viewed by the host operating system, the applications and the users.
equation must be less than the percentage value of balance-threshold to be considered balanced. • reassignment-threshold percentage This parameter determines whether or not a reassignment will be considered worthwhile. • measurement-interval time This parameter accepts an positive integer value with a suffix of "s" (seconds), "m" (minutes), or "h" (hours).
How to Verify Load Balancing In Figure 3-4, “Example setsp -a Output in Determining Load Balancing”, there are four SVA Path devices; spd0, spd1, spd2 and spd3.
In our example, the following command will show that each device in Figure 3-4, “Example setsp -a Output in Determining Load Balancing” is accessible via four paths, using HBAs "glm3", "glm4", “lpfc0” and “lpfc1”: # spmon show # spmon show Total I/O load across monitored HBAs = 103.2s HBA Device I/O Load %load -----------------------------c1 glm3 40.1s 38.8% c2 glm4 20.8s 20.2% c4 lpfc0 21.2s 20.6% c5 lpfc1 21.1s 20.
seconds (with an "s" suffix) if it is less than 300 seconds. For example, 18.1s = 28.1 seconds hours:minutes:seconds (hh:mm:ss) if it is more than 59 minutes. For example, 0:12:42 = 0 hours, 12 minutes, and 42 seconds. • HBA Load identifies the I/O load contributed by traffic to/from this FDev as a percentage of all I/O though the current HBA used to access this FDev. The figure in the right-hand column is a ratio I/O Load / HBA Load, expressed as a percentage.
# spmon show Total I/O load across monitored HBAs = 105.1s HBA Device I/O Load %load -----------------------------c1 glm3 34.1s 32.5% c2 glm4 31.3s 29.8% c4 lpfc0 19.1s 18.1% c5 lpfc1 20.6s 19.6% Device c6t0d0 [STK 9500 0000000010850000] FDev Path I/O Load HBA Load ---------------------------------------------------------spd0:0 0/glm3/sd66 16.9s 34.1s 49.5% Device c6t0d1 [STK 9500 0000000010850001] FDev Path I/O Load HBA Load ---------------------------------------------------------spd1:0 0/glm3/sd67 17.
The sppath Command The sppath command is run automatically at boot after SVA Path is installed. sppath’s main function is to create the configuration file that identifies devices to be put under SVA Path’s control. Its command options are used to display or modify this device set. sppath examines disk devices attached to the system to determine whether any physical devices are accessible via redundant paths and whether those devices should be put under the control of the SVA Path driver.
Table 3-2, “sppath Command Options” gives a synopsis of sppath options. They are explained in greater detail below. Table 3-2 Option sppath Command Options Effect -d display debug information -D clear the list of ignored devices. Should be followed by setsp -g (see note below) -I{cXtYdZ} ignore the device or group of devices specified; devices are specified in the form cX, cXtY, or cXtYdZ.
The -d option displays inquiry data in raw format and is generally used only for debugging. Ignoring and Reclaiming Devices with sppath The -I option accepts symbolic device names corresponding to controllers or specific disks and omits them from /etc/sppath.conf. This prevents them from being put under SVA Path’s control. If a device is to be ignored, all of its paths should be specified with -I options.
Understanding the setsp -a Screen Use the setsp -a command to display the default path configurations.
• Status shows the current state of the path. Good paths are functioning normally. Bad paths have failed. • Primary shows which of the physical I/O paths connecting the device to the host’s host bus adapters (or controllers) is defined as primary (marked by an X). Initially, primary path assignments are distributed evenly among the available paths.
default value is 3000 ms. This value can not be set above 100,000 ms (100s). • FailBack indicates whether failback is enabled for the logical device. When failback is enabled (the default setting of 1), SVA Path will keep testing a path that has failed and return it to service (as the primary path or in sequential load balancing) as soon as the path has been restored. If the default configuration is satisfactory, no reconfiguration of SVA Path is necessary.
Assigning a New Primary Path The syntax for changing an SVA LUN’s primary data path is setsp -l -p where is the spd number (see above) and is the number of the new path. The path number can be obtained by simply counting down setsp’s list of physical paths (in the Path/disk column), starting from zero.
All devices may be included, as they present no conflicts. However, in a multiple-host environment, where all spds are visible to SVA Path on all hosts, spds must be either excluded or included so that a host shares no spds. Use the ID number under the spd column on the setsp -a output to identify devices. Do not use the spdX number.
where is the numerical element of the spd number and is the interval between the retries specified by the retry count parameter. Turning Failback Off and On The syntax for changing the a device’s failback mode is setsp -l -f{0|1} where is the spd number (see “Specifying a Device for setsp” on page 25) and the -f option takes one of two arguments: • -f0 turns failback off; • -f1 turns failback on.
5. Start the new SVA Path device with: # setsp -S -l 6. Unexclude the new device: # setsp -e0 -l Chapter 3.
30 SVA Path User’s Guide
Chapter 4. SVA Path Installation This chapter describes how to install SVA Path for use with SVA 9500. Installation of SVA Path in HP-UX is managed by the SD-UX software management commands. Many of these tools (e.g., swinstall and swremove) can be run either from the command line or from a Graphical User Interface (GUI). The following instructions illustrate using the command line. However, the same tasks can be accomplished through the GUI.
3. Before installing SVA Path, it is imperative that the host is able to see and access all storage devices through all available paths. For example, if you have two HBAs and redundant paths to your storage system, each logical drive should be displayed twice by the command: # ioscan -fnC disk If you can’t see all available storage devices through every path, verify that you have the latest fibre channel or SCSI HBA drivers loaded on your system. 4.
complete all active processes, wait 3 minutes before pressing the “Finish” button on the last panel of the install screens, and before rebooting and unmounting the CDROM. (To monitor active processes, type ps -ef | grep install and wait until all processes have ended before rebooting the machine and unmounting the CD.) 8. Enter the appropriate system command to eject the CDROM. 9. Reboot the host. 10. Verify that all available drives are seen by SVA Path by running the command: # setsp -a 11.
Note: If SVA Path is installed after SVAA, SVA Path will claim the SVAA device. Run the following commands, which will cause SVA Path to ignore the specified device: # sppath -I(cXtYdZ) # setsp -T -l(n) # setsp -g Changes will take effect only after the next reboot of the host. After SVA Path is installed, in addition to its physical path(s) (e.g., /dev/rdsk/c1t0d0s2 and /dev/rdsk/c2t0d0s2 ), a virtual path to the ECAM device will have been created along with an additional device name (e.g.
Installed Files During installation, the files listed in Table 4-1, “Installed SVA Path Files” are placed in your system. All of these files will be removed if SVA Path is uninstalled. Table 4-1 Installed SVA Path Files File Description /sbin/spckd Script to start spcheck /usr/conf/master.d/spd SVA Path driver /usr/conf/lib/libusrdrv.a archive file in which spd is stored /etc/sppath.conf sppath configuration file /etc/spd.
36 SVA Path User’s Guide
SVA Path Device Naming on HP-UX Disk device filenames in HP-UX 10.
SVA Path installations, SVA Path creates a single additional device filename for applications to access the device and manages the original device files transparently to those applications. Note: It is the virtual device file that will be used by applications to access that device. The original, redundant data paths should never be used to access the device, or the data it contains could be corrupted. For example, an SVA might present a single physical disk device to the HP-UX host as c1t0d0 and c2t0d0.
Path is installed, be accessed through the SVA Path device file c81t4d0 (Figure 4-1). Host Applications c81 Virtual Controller SVA Path creates a virtual controller and presents the device to host applications as a single device.
If there are more than one set of redundant paths, there could be more than one device with a given target and LUN number. For this reason, SVA Path creates a different virtual HBA for each set of redundant physical HBAs. For example, if one device is accessible via c1t0d0 and c2t0d0, and another device is accessible via c3t0d0 and c4t0d0, SVA Path would create two new device files c81t0d0 (for c1t0d0 and c2t0d0) and c83t0d0 (for c3t0d0 and c4t0d0).
This section shows how to create a new volume group and add a disk drive to it, using example screens from an HP 800 system running HP-UX 11.00. Definitions of Common Terms The following terms will be used throughout this section: • Volume Group. Volume groups are HP-UX's method for dividing and allocating disk storage space. Volume groups can be used to partition a disk drive into smaller logical volumes (see below).
Preparing the Volume Group Device files for each volume group on an HP 9000 system reside in directories with names /dev/vg*. The following example lists the volume groups on an HP800 system running HP-UX 11.
• Creating the Physical Volume a minor number of the form 0xNN0000, where NN is the twodigit hexadecimal representation of the volume group number (here, 02).
To verify the creation and view the volume group characteristics, use the vgdisplay command: # vgdisplay vg02 --- Volume groups --VG Name VG Write Access VG Status Max LV Cur LV Open LV Max PV Cur PV Act PV Max PE per PV VGDA PE Size (Mbytes) Total PE Alloc PE Free PE Total PVG /dev/vg02 read/write available 255 0 0 16 1 1 2167 2 4 2167 0 2167 0 In the output of vgdisplay, the Total PE field gives the number of physical extents in the volume group.
multiple of the physical extent size that is 4 MB by default. In the example below, a volume group of 4092 MB is created: # lvcreate -L 4092 /dev/vg02 Logical volume "/dev/vg02/lvol1" has been successfully created with character device "/dev/vg02/rlvol1". Logical volume "/dev/vg02/lvol1" has been successfully extended. Volume Group configuration for /dev/vg02 has been saved in /etc/lvmconf/vg02.
The Free PE field in the vgdisplay output shows that the number of free physical extents in the volume group has decreased: # vgdisplay vg02 --- Volume groups --VG Name /dev/vg02 VG Write Access read/write VG Status available Max LV 255 Cur LV 1 Open LV 1 Max PV 16 Cur PV 1 Act PV 1 Max PE per PV 2167 VGDA 2 PE Size (Mbytes) 4 Total PE 2167 Alloc PE 1023 Free PE 1144 Total PVG 0 You can continue to create logical volumes with the lvcreate command until the Free PE field is 0, which indicates that the volum
first export the volume group and them import it using the new special device filenames that SVA Path assigns. In the following example there are two logical volumes in a volume group, vg02. 1. Make note of the minor number of the filename group for the volume group: # ls -l /dev/vg02/group crw-rw-rw- 1 root sys 64 0x020000 Jun 23 13:02 /dev/vg02/group You will need this information to recreate the volume group later in this process. In this example the minor number for logical volume vg02 is 0x020000.
3. Unmount all of the logical volumes in the volume group and deactivate the volume group by entering the following command: # vgchange -a n /dev/vg02 4. Remove the volume group from the system configuration file and delete the associated device files for the logical volumes: # vgexport /dev/vg02 5. Recreate the directory and the device filename for the volume group using the same minor number of the original volume group: # mkdir /dev/vg02 # mknod /dev/vg02/group c 64 0x020000 6.
8. Select the SVA Path devices that correspond to the original device filenames of the physical volumes of the volume group. In this example they are /dev/dsk/c86t0d0 and /dev/dsk/c86t0d1. Use these filenames in the vgimport command to recreate the configuration information for the volume group: # vgimport -v /dev/vg02 /dev/dsk/c86t0d0 /dev/dsk/c86t0d1 Beginning the import process on Volume Group "/dev/vg02". Logical volume "/dev/vg02/lvol1" has been successfully created with lv number 1.
50 SVA Path User’s Guide
Chapter 5. Diagnosing Errors Comparing setsp -a with System Events SVA Path necessarily interfaces with system components at the I/O path level, which does not support sophisticated device error reporting. The application reports failed I/O to /var/adm/syslog/syslog.log as well as when: • a data path is found to have stopped functioning • I/O is redirected to an alternate path • the failed path resumes functioning.
Cable Failure Errors Figure 5-2, “Messages Generated When a Cable Is Pulled” illustrates error messages that will occur in the file /var/adm/syslog/syslog.log. The hardware path of the affected fibre channel interface is identified as "0/5/0/0". Using the output of the ioscan command in Figure 5-3, “Excerpt of ioscan -fn output”, the affected devices can be identified. In this example they are /dev/dsk/c6t0d0, /dev/dsk/c6t0d1, and /dev/dsk/c6t0d2.
Errors for other spd devices, representing the same physical I/O path that was interrupted by the pulled cable, will be reported, as shown in Figure 5-4, “Additional Messages for the Same Pulled Cable”. Jun 30 14:58:28 n4000 vmunix: ALERT: fcT1 (1, 735458, Channel Driver received an interrupt indicating a Link Manager Status Register is 0x980010D0.
shows that every device’s path 0 (those corresponding to device filenames beginning c7) have a Status designation of Bad.
READER'S COMMENT FORM Manual PN: Manual Name: Please check or fill in the items; adding explanations/comments in the space provided.
TO COMPLY WITH POSTAL REGULATIONS, FOLD EXACTLY ON DOTTED LINES AND TAPE (DO NOT STAPLE) NO POSTAGE NECESSARY IF MAILED IN THE UNITED STATES BUSINESS REPLY MAIL FIRST CLASS PERMIT NO. 2 LOUISVILLE, CO U.S.A.
Printed in U.S.
