Red Hat Enterprise Linux 4 DM Multipath DM Multipath Configuration and Administration Edition 1.
Red Hat Enterprise Linux 4 DM Multipath DM Multipath Configuration and Administration Edition 1.0 Landmann rlandmann@redhat.
Legal Notice Copyright © 2010 Red Hat, Inc. T his document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed.
Table of Contents Table of Contents .Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. . . . . . . . . . 1. Audience 4 2. Related Documentation 4 3. Feedback 4 4. Document Conventions 5 4.1. T ypographic Conventions 5 4.2. Pull-quote Conventions 6 4.3. Notes and Warnings 7 .Chapter . . . . . . . . 1. . . .Device . . . . . . . Mapper . . . . . . . . .Multipathing . .
Red Hat Enterprise Linux 4 D M Multipath H K L M N P R S U V W 2 36 36 37 37 37 38 38 38 38 38 38
Table of Contents 3
Red Hat Enterprise Linux 4 D M Multipath Preface T his book describes the Device Mapper Multipath (DM-Multipath) feature of Red Hat Enterprise Linux. 1. Audience T his book is intended to be used by system administrators managing systems running the Linux operating system. It requires familiarity with Red Hat Enterprise Linux. 2.
Preface Be sure to mention the manual's identifier: rh-MPIO(EN)-4.9 (2011-02-16T16:48) By mentioning this manual's identifier, we know exactly which version of the guide you have. If you have a suggestion for improving the documentation, try to be as specific as possible. If you have found an error, please include the section number and some of the surrounding text so we can find it easily. 4.
Red Hat Enterprise Linux 4 D M Multipath labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example: Choose System → Preferences → Mouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, select the Left-handed m ouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).
Preface static int kvm_vm_ioctl_deassign_device(struct kvm *kvm, struct kvm_assigned_pci_dev *assigned_dev) { int r = 0; struct kvm_assigned_dev_kernel *match; mutex_lock(&kvm->lock); match = kvm_find_assigned_dev(&kvm->arch.
Red Hat Enterprise Linux 4 D M Multipath Chapter 1. Device Mapper Multipathing Device Mapper Multipathing (DM-Multipath) allows you to configure multiple I/O paths between server nodes and storage arrays into a single device. T hese I/O paths are physical SAN connections that can include separate cables, switches, and controllers. Multipathing aggregates the I/O paths, creating a new device that consists of the aggregated paths. 1.1.
Chapter 1. D evice Mapper Multipathing Array controller port failure With DM-Multipath configured, a failure at any of these points will cause DM-Multipath to switch to the alternate I/O path. Figure 1.2, “Active/Passive Multipath Configuration with T wo RAID Devices” shows a more complex active/passive configuration with 2 HBAs on the server, 2 SAN switches, and 2 RAID devices with 2 RAID controllers each. Figure 1.2.
Red Hat Enterprise Linux 4 D M Multipath Figure 1.3. Active/Active Multipath Configuration with One RAID Device 1.2. Storage Array Support By default, DM-Multipath includes support for the most common storage arrays that support DMMultipath. T he supported devices can be found in the m ultipath.conf.defaults file. If your storage array supports DM-Multipath and is not configured by default in this file, you may need to add them to the DM-Multipath configuration file, m ultipath.conf.
Chapter 1. D evice Mapper Multipathing T able 1.1. DM-Multipath Components Component Description dm -m ultipath kernel module Reroutes I/O and supports failover for paths and path groups. m ultipath command Lists and configures multipath devices. Normally started up with /etc/rc.sysinit, it can also be started up by a udev program whenever a block device is added or it can be run by the initram fs file system.
Red Hat Enterprise Linux 4 D M Multipath Chapter 2. Multipath Devices Without DM-Multipath, each path from a server node to a storage controller is treated by the system as a separate device, even when the I/O path connects the same server node to the same storage controller. DM-Multipath provides a way of organizing the I/O paths logically, by creating a single multipath device on top of the underlying devices. 2.1.
Chapter 2. Multipath D evices If you want the system-defined user-friendly names to be consistent across all nodes in the cluster, set up all of the multipath devices on one machine. T hen copy the bindings file from that machine to all the other machines in the cluster. T he bindings file is located at /var/lib/m ultipath/bindings by default, but as of RHEL 4.6 and later you can set this value to a different location with the bindings_file parameter of the defaults section of the configuration file. 2.3.
Red Hat Enterprise Linux 4 D M Multipath Chapter 3. Setting Up DM-Multipath T his chapter provides step-by-step example procedures for configuring DM-Multipath. It includes the following procedures: Basic DM-Multipath setup Ignoring local disks Adding more devices to the configuration file 3.1. Setting Up DM-Multipath Before setting up DM-Multipath on your system, ensure that your system has been updated and includes the device-m apper-m ultipath package.
Chapter 3. Setting Up D M-Multipath If the SCSI devices do not appear, ensure that all SAN connections are set up properly. For further information on the m ultipath command and its output, see Section 5.1, “Multipath Command Output”, see Section 5.2, “Multipath Queries with multipath Command”, and see Section 5.3, “Multipath Command Options”. 5.
Red Hat Enterprise Linux 4 D M Multipath [root@rh4cluster1 ~]# multipath -v2 create: SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1 [size=33 GB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 0:0:0:0 sda 8:0 [--------device-mapper ioctl cmd 9 failed: Invalid argument device-mapper ioctl cmd 14 failed: No such device or address create: 3600a0b80001327d80000006d43621677 [size=12 GB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 2:0:0:0 sdb 8:16 \_ 3:0:0:0 sdf 8:80 create: 3600a0b80001327510000009a436215ec
Chapter 3.
Red Hat Enterprise Linux 4 D M Multipath Chapter 4. The DM-Multipath Configuration File By default, DM-Multipath provides configuration values for the most common uses of multipathing. In addition, DM-Multipath includes support for the most common storage arrays that support DM-Multipath. T he default configuration values and the supported devices can be found in the /usr/share/doc/device-m apper-m ultipath-0.4 .5/m ultipath.conf.defaults file. Note T he location and contents of the m ultipath.conf.
Chapter 4. The D M-Multipath Configuration File devices Settings for the individual storage controllers. T hese values overwrite what is specified in the defaults section of the configuration file. If you are using a storage array that is not supported by default, you may need to create a devices subsection for your array. When the system determines the attributes of a multipath device, first it checks the multipath settings, then the per devices settings, then the multipath system defaults. 4.2.
Red Hat Enterprise Linux 4 D M Multipath devnode_blacklist { devnode "^sd[a-z]" } You can use a devnode entry in the blacklist section of the configuration file to specify individual devices to blacklist rather than all devices of specific type; this is not recommended, however. Unless it is statically mapped by udev rules, there is no guarantee that a specific device will have the same name on reboot. For example, a device name could change from /dev/sda to /dev/sdb on reboot.
Chapter 4. The D M-Multipath Configuration File defaults { user_friendly_names path_grouping_policy } yes multibus T able 4.1, “Multipath Configuration Defaults” describes the attributes that are set in the defaults section of the m ultipath.conf configuration file. T hese values are used by DM-Multipath unless they are overwritten by the attributes specified in the devices and m ultipaths sections of the m ultipath.conf file.
Red Hat Enterprise Linux 4 D M Multipath T able 4 .1. Multipath Configuration Defaults Attribute Description udev_dir Specifies the directory where udev device nodes are created. T he default value is /udev. polling_interval Specifies the interval between two path checks in seconds. T he default value is 5. selector Specifies the default algorithm to use in determining what path to use for the next I/O operation. T he default value is round-robin 0.
Chapter 4. The D M-Multipath Configuration File rr_m in_io Specifies the number of I/O requests to route to a path before switching to the next path in the current path group. T he default value is 1000. m ax_fds (RHEL 4.7 and later) Sets the maximum number of open file descriptors for the m ultipathd process. A value of m ax sets the number of open file descriptors to the system maximum.
Red Hat Enterprise Linux 4 D M Multipath 4.4. Multipaths Device Configuration Attributes T able 4.2, “Multipath Attributes” shows the attributes that you can set in the m ultipaths section of the m ultipath.conf configuration file for each specific multipath device. T hese attributes apply only to the one specified multipath. T hese defaults are used by DM-Multipath and override attributes set in the defaults and devices sections of the m ultipath.conf file.
Chapter 4. The D M-Multipath Configuration File T able 4 .2. Multipath Attributes Attribute Description wwid Specifies the WWID of the multipath device to which the m ultipath attributes apply. alias Specifies the symbolic name for the multipath device to which the m ultipath attributes apply. path_grouping_policy Specifies the default path grouping policy to apply to unspecified multipaths.
Red Hat Enterprise Linux 4 D M Multipath flush_on_last_del (RHEL 4.7 and later) If set to yes, the m ultipathd daemon will disable queueing when the last path to a device has been deleted. T he default value is no. m ode (RHEL 4.7 and later) T he mode to use for the multipath device nodes, in octal. T he default value is determined by the process. uid (RHEL 4.7 and later) T he user ID to use for the multipath device nodes. You must use the numeric user ID.
Chapter 4. The D M-Multipath Configuration File need to set the vendor and product parameters. You can find these values by looking at /sys/block/device_name/device/vendor and /sys/block/device_name/device/m odel where device_name is the device to be multipathed, as in the following example: [root@cypher-06 ~]# cat /sys/block/sda/device/vendor WINSYS [root@cypher-06 ~]# cat /sys/block/sda/device/model SF2372 T he additional parameters to specify depend on your specific device.
Red Hat Enterprise Linux 4 D M Multipath T able 4 .3. Device Attributes Attribute Description vendor Specifies the vendor name of the storage device to which the device attributes apply, for example COMPAQ. product Specifies the product name of the storage device to which the device attributes apply, for example HSV110 (C)COMPAQ. bl_product Specifies a regular expression used to blacklist devices by vendor/product.
Chapter 4. The D M-Multipath Configuration File group. T he default value is 1000. rr_weight If set to priorities, then instead of sending rr_m in_io requests to a path before calling selector to choose the next path, the number of requests to send is determined by rr_m in_io times the path's priority, as determined by the prio_callout program.
Red Hat Enterprise Linux 4 D M Multipath Chapter 5. DM-Multipath Administration and Troubleshooting T his chapter will provide information on administering DM-Multipath on a running system. It includes sections on the following topics: Multipath Command Output Multipath Queries with multipath Command Multipath Command Options Multipath Queries with dmsetup Command T roubleshooting with the multipathd Interactive Console Resizing an Online Multipathed Device 5.1.
Chapter 5. D M-Multipath Administration and Troubleshooting Note When a multipath device is being created or modified, the path group status and the dm status are not known. Also, the features are not always correct. When a multipath device is being listed, the path group priority is not known. 5.2. Multipath Queries with multipath Command You can use the -l and -ll options of the m ultipath command to display the current multipath configuration.
Red Hat Enterprise Linux 4 D M Multipath T he following command displays all the device mapper devices and their major and minor numbers. T he minor numbers determine the name of the dm device. For example, a minor number of 3 corresponds to the multipathed device /dev/dm -3.
Chapter 5. D M-Multipath Administration and Troubleshooting Use the following procedure to resize an online multipath device. 1. Resize your physical device. 2. Resize your paths. For SCSI devices, writing a 1 to the rescan file for the device causes the SCSI driver to rescan. You can use the following command: # echo 1 > /sys/block/device_name/device/rescan 3.
Red Hat Enterprise Linux 4 D M Multipath Revision History Revision 1.0-11.4 00 Rebuild with publican 4.0.0 2013-10-31 Rüdiger Landmann Revision 1.0-11 Rebuild for Publican 3.0 2012-07-18 Anthony T owns Revision 4 .9-1 Resolves: #633518 Fixes typo. Wed Feb 16 2011 Steven Levine Revision 1.
Revision History - blacklist, Setting Up DM-Multipath, Configuration File Blacklist - bl_product parameter, Configuration File Devices - devnode_blacklist, Configuration File Blacklist - failback parameter, Configuration File Defaults, Multipaths Device Configuration Attributes, Configuration File Devices - features parameter, Configuration File Defaults, Configuration File Devices - flush_on_last_del parameter, Configuration File Defaults - getuid_callout parameter, Configuration File Defaults, Configurat
Red Hat Enterprise Linux 4 D M Multipath devices section - multipath.
Revision History kpartx command , DM-Multipath Components L local disks, ignoring, Ignoring Local Disks when Generating Multipath Devices LVM physical volumes - multipath devices, Multipath Devices in Logical Volumes lvm.
Red Hat Enterprise Linux 4 D M Multipath P path_checker parameter, Configuration File Defaults, Configuration File Devices path_grouping_policy parameter, Configuration File Defaults, Multipaths Device Configuration Attributes, Configuration File Devices path_selector parameter, Multipaths Device Configuration Attributes, Configuration File Devices polling_interval parameter, Configuration File Defaults prio_callout parameter, Configuration File Defaults, Configuration File Devices product parameter, Confi
