HP P6000 Cluster Extension Software Administrator Guide This guide contains detailed instructions for configuring and troubleshooting HP P6000 Cluster Extension Software in Windows and Linux environments. The intended audience has independent knowledge of related software and of the HP P6000/EVA array and its software.
Legal and notice information © Copyright 2006-2011 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 P6000 Cluster Extension features ......................................................... 9 Integration into cluster software ..................................................................................................... 9 Graphical user interface ........................................................................................................ 9 CLI for easy integration .........................................................................................................
SSL protocol ...................................................................................................................... HP P6000 Command View uses ........................................................................................... SSSU considerations ........................................................................................................... IP network considerations ....................................................................................................
Configuration overview ....................................................................................................... Creating an RHCS P6000 Cluster Extension shared resource ................................................... Using Conga to create a shared resource ....................................................................... Using system-config-cluster to create a shared resource ..................................................... Configure the SMI-S server information ...............
Troubleshooting ............................................................................. 101 Changing the P6000/EVA SMI-S user name and password ......................................................... Log facility ............................................................................................................................. Error return codes ................................................................................................................... Log files ..................
Figures 1 Physical replication using HP P6000 Continuous Access .............................................. 10 2 One-to-one configuration ......................................................................................... 18 3 Consolidated DR site configuration ............................................................................ 19 4 Supported P6000 Cluster Extension storage system configuration .................................. 20 5 Service or application example .........................
Tables 1 Setting resource properties and values in the GUI ........................................................ 34 2 Service or application properties and values .............................................................. 35 3 Common and application objects .............................................................................. 74 4 Document conventions ...........................................................................................
1 P6000 Cluster Extension features HP P6000 Cluster Extension offers protection against application downtime due to a fault, failure, or site disaster by extending a local cluster between data centers over metropolitan distance. P6000 Cluster Extension reinstates critical applications at a remote site within minutes of an adverse event, integrating your open-system clustering software and HP P6000 Continuous Access to automate failover and failback between sites.
Disaster tolerance through geographical dispersion Using two or more storage systems, HP P6000 Continuous Access copies data to a remote data center. Cluster solutions using P6000 Continuous Access disk mirroring are called metropolitan clusters or geographically dispersed clusters. In this arrangement, a server is a member of the same cluster dispersed over two or more sites.
NOTE: Vdisks are virtual disks used in a HP P6000/EVA storage system for the storage of application data. A DR group includes one or more vdisks in a P6000/EVA storage system. The DR group is the unit in which P6000 Cluster Extension failover/failback operations are performed. For more information about DR groups, see the HP P6000 Continuous Access Implementation Guide.
MNS quorum in MSCS An MNS is a single quorum resource that allows you to build clusters with nodes that are geographically separated. In an MNS cluster, unlike in a quorum disk cluster, data is stored on multiple disks across the cluster. The MNS resource ensures that the cluster's configuration data is stored locally on the nodes and that it is kept consistent across the different disks. This allows cluster topologies where the nodes do not need shared access to a quorum disk.
NOTE: IPMI fencing can be used for Integrity servers that do not support RIBCL scripting. Networking in an SLE HA cluster Configuring redundant and independent cluster communication paths is a good way to avoid Split Brain conditions. With redundancy in communication paths, the loss of a single interface or switch does not break the communication between nodes and prevents Split Brain conditions. Administrators can configure multiple independent communication paths.
Failback option HP does not recommend auto failback in configurations with P6000 Cluster Extension because the resource failovers due to storage failure can cause resources to go into an unstable state (failover/failback might toggle the resource between the nodes). SLE HA provides the meta-attribute resource-stickiness to determine how much a resource agent prefers to stay where it is. To disable auto failback, set resource-stickiness to the lowest value compared to the other resource location constraints.
• The HP iLO of each cluster system must be accessible over the network from every other cluster system. To handle infrequent failures of the HP iLO fencing (such as a switch failure), you can set up a backup fence method for redundancy. HP iLO fencing can be used on HP Proliant systems with built-in iLO hardware. For third-party systems, other power control fencing methods can be used. NOTE: IPMI fencing can be used for Integrity servers that do not support RIBCL scripting.
Recovery policy When a resource inside the service fails, the default action is to restart the service on the local node before the failover. In a P6000 Cluster Extension environment, it is always expected to relocate the service during restart. To enable this functionality, set the service recovery policy to relocate.
One-to-one configuration When setting up a one-to-one configuration (see Figure 2 on page 18), consider the following: • Cluster host nodes are split between two or more geographically dispersed data center sites and use redundant, diversely routed network connections for intra-cluster communications. Those links must be as reliable as possible to prevent false failover operations or split-brain situations.
Figure 2 One-to-one configuration . Consolidated DR site configuration P6000 Cluster Extension supports configurations that have a single P6000/EVA storage system in the disaster recovery data center and up to two source P6000/EVA storage systems (the logical configuration must be a one-to-one configuration as shown in Figure 2 on page 18). NOTE: The restrictions outlined in “One-to-one configuration” on page 17 also apply to consolidated configurations.
Figure 3 Consolidated DR site configuration . Processes and components For Windows: P6000 Cluster Extension is shipped as a resource DLL for MSCS. For Linux: P6000 Cluster Extension is shipped as a binary executable with an Open Cluster Framework-compliant script-based resource agent. With both Windows and Linux, installation of P6000 Cluster Extension includes a CLI for custom HP P6000 Continuous Access integration.
HP P6000 Continuous Access Figure 4 on page 20 shows the supported P6000 Cluster Extension configuration. Figure 4 Supported P6000 Cluster Extension storage system configuration . Synchronous replication Using synchronous mode, all write requests from the server are first transferred to the remote storage system. After each I/O has been mirrored in the cache area of the remote array, it is acknowledged to the local storage system. The write request is then acknowledged to the server.
Rolling disaster protection and snapshots/snapclones To implement rolling disaster protection, you can create P6000 Business Copy disk pairs for DR groups locally. The P6000 Business Copy snapshots or snapclones must not be presented to the local cluster node because MSCS uses the disk's signature to identify a cluster disk, and the cluster software cannot differentiate between the original disk and the clone if both are presented to the same server.
User configuration file P6000 Cluster Extension provides a user configuration file to customize failover behavior. You can specify all customizable objects of P6000 Cluster Extension in this file. See “User configuration file” on page 73 for more information. Resource configuration file (Linux only) The P6000 Cluster Extension resource configuration file is used to configure resources created using RHCS or SLE HA. The contents of this file are the same as the UCF.cfg file.
HP SMI-S configuration When using HP SMI-S and P6000 Cluster Extension, note the following requirements: • To function properly, P6000 Cluster Extension requires at least one instance of HP P6000/EVA SMI-S running on a management server. You must, therefore, set up at least one management server with HP SMI-S in each data center.
HP P6000 Command View uses HP P6000 Command View is used to create vdisks and configure the HP P6000/EVA storage system. Using HP P6000 Command View, you can: • Create P6000 Continuous Access DR groups. • Use Replication Solutions Manager or P6000 Command View to view a graphical representation of a DR group setup. • Use Replication Solutions Manager or P6000 Command View to manually switch source and destination roles of DR group members.
2 Configuring P6000 Cluster Extension for Windows After installing P6000 Cluster Extension, use the configuration tool to define the P6000 Cluster Extension setup configuration. Use Cluster Administrator (Windows Server 2003), Failover Cluster Management (Windows Server 2008/2008 R2), or cluster commands in the CLI to add and configure resources. NOTE: For a list of supported operating system versions, see the HP SPOCK website: http://www.hp.com/storage/spock.
Starting the P6000 Cluster Extension configuration tool To start the P6000 Cluster Extension configuration tool: • For Windows Server 2003 or Windows Server 2008/2008 R2: Double-click the HP P6000 CLX Configuration Tool icon on the desktop, or select Start > Programs > Hewlett-Packard > HP P6000 CLX Configuration Tool. • For Server Core or Hyper-V Server: Open a command window and enter CLXEVACONFIG –I. The P6000 CLX Configuration Tool window appears.
Defining P6000 Cluster Extension configuration information using the GUI To define configuration information using the GUI: 1. Open the configuration tool. For instructions, see “Starting the P6000 Cluster Extension configuration tool” on page 26. 2. Click Add in the P6000 Management Server Configuration section to open the Add P6000 Management Server (SMA) dialog box. 3. Enter the following values in the appropriate fields, and then click OK: • • • • 4.
6. Click Add in the Cluster Node Configuration section to open the Add Server dialog box. The available servers are listed. 7. Select the servers that are possible owners of the P6000 Cluster Extension-managed disks, and then click OK. NOTE: See the Microsoft Failover Cluster Management documentation for more information about possible owners. 8. Click OK to close the configuration tool and save the information to all of the configured cluster nodes.
Use the import and export features to copy the P6000 Cluster Extension configuration file () (clxevacfg) from one cluster node to another. Exporting configuration settings using the GUI 1. Open the configuration tool. For instructions, see Starting the P6000 Cluster Extension configuration tool. 2. Click Export. 3. When prompted, choose a save location, enter a file name, and then click Save. 4. Click OK to save and close the configuration tool. Exporting configuration settings using the CLI 1.
• For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI or cluster commands in the CLI. For instructions, see “Adding a P6000 Cluster Extension resource using the Failover Cluster Management GUI (Windows Server 2008/2008 R2)” on page 31 or “Adding a P6000 Cluster Extension resource using the Microsoft CLI cluster commands” on page 31. • For Server Core or Hyper-V Server, use cluster commands in the CLI.
Adding a P6000 Cluster Extension resource using the Failover Cluster Management GUI (Windows Server 2008/2008 R2) Use the procedure in this section to add a resource using the Failover Cluster Management GUI. For instructions on using the CLI, see Adding a P6000 Cluster Extension resource using the Microsoft CLI cluster commands. 1. Add a service or application in the Failover Cluster Management GUI, as described in your Microsoft documentation. 2.
P6000 Cluster Extension resource names and service and application names must consist of only one string and cannot include blank spaces; use underscores or hyphens to connect individual words (for example, CLX_resource, not CLX resource). Changing a P6000 Cluster Extension resource name (Windows Server 2003) In this procedure, you use the Cluster Administrator GUI to change a resource name.
When configuring P6000 Cluster Extension resources, note the following: • If the Cluster Administrator or Failover Cluster Management GUI is used to configure a P6000 Cluster Extension resource, configuring the resource using a user configuration file (UCF file) is not required. • P6000 Cluster Extension creates a file called resource_name.online to indicate that the P6000/EVA resource is online on the local node. This file is created in the directory specified by the ApplicationDir resource property.
For more information about setting resource properties, see your Microsoft documentation. Table 1 Setting resource properties and values in the GUI Property Format Description Value Thorough Resource Health Check Interval (Windows Server 2008/2008 R2) Integer Used to poll “Alive” state for the resource. Decreasing this value allows faster resource failure detection but also consumes more system resources.
Property Format Description Value Pending timeout (GUI) Integer Used to specify the timeout for status resolution. For more information, see Timing considerations for MSCS. Windows Server 2008/2008 R2: 03:00 mm:ss PendingTimeout (CLI) Set this value in the Policies tab of the resource properties window in Failover Cluster Management, or in the Advanced tab of the resource properties window in Cluster Administrator.
Setting P6000 Cluster Extension-specific resource properties Changes to resource properties take effect when the resource is brought online again.
Using Failover Cluster Management to set resource properties (Windows Server 2008/2008 R2) For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI to set resource properties. 1. Open Failover Cluster Management. 2. Double-click the P6000 Cluster Extension resource in the summary pane to open the Properties window. 3. Click the Parameters tab. 4. Make the necessary parameter changes, and then click OK.
Using the resource configuration tool to set resource properties (Server Core and Hyper-V Server) For Server Core or Hyper-V Server, use the P6000 Cluster Extension resource configuration tool to set resource properties. When using the resource configuration tool: • You must run the tool on a Server Core or Hyper-V cluster node. You cannot run the tool on a remote management station. • You cannot use the resource configuration tool to add or delete a resource.
HP P6000 Cluster Extension Software Administrator Guide 39
Configuring management servers To configure management servers from the cluster software Parameters tab or the resource configuration tool: 1. To add a management server (CV Server): a. Click Add to open the Add SMA Data center Assignment dialog box. b. Select a management server from the Server Name list. c. Assign the management server to a data center selected from the Data center list. d. Click OK to save and close the dialog box. 2.
2. To remove a disk array, select the array, and then click Remove. 3. To modify a disk array, select the array, and then click Modify. 4. Click Apply to finish your modifications and leave the window open, or OK to save your changes and close the Properties window. Configuring cluster node data center assignments To configure cluster node data center assignments from the cluster software Parameters tab or the resource configuration tool: 1. To add a cluster node: a.
2. Select a DR group, and then click OK. NOTE: When using P6000 Command View 8.0.2 or later, if you created new DR groups, the groups might not show up immediately in the Select DR Group dialog box. To make a new DR group available in the P6000 Cluster Extension GUI, do one of the following: • Wait 30 minutes after creating the DR group. • Restart the HP CIM Object Manager service after creating the DR group.
Use this dialog box to configure the DR group behavior. 2. • The values you can configure depend on whether the DR group uses synchronous or enhanced asynchronous mode. • The settings in the Replication link up scenario section affect the failover behavior when the P6000 Continuous Access link is up. • The settings in the Replication link down scenario section affect the failover behavior when the P6000 Continuous Access link is down.
2. Update the PreExecScript, PostExecScript, and PostExecCheck values as needed, and then click OK. For more information, see “APPLICATION objects” on page 77. 3. Click Apply to finish your modifications and leave the Properties window open, or OK to save your changes and close the window. Setting P6000 Cluster Extension resource properties using the MMC If you are using Server Core or Hyper-V Server, you can manage a cluster remotely by using the MMC to run Failover Cluster Management.
NOTE: When you configure P6000 Cluster Extension resource properties using the CLI, the properties you enter are not validated, so you must enter the property values accurately, and verify them against the P6000 Cluster Extension documentation.
2. Open the sample UCF.cfg file located in %ProgramFiles%\Hewlett-Packard\Cluster Extension EVA\sample. 3. Update the file with the property values you want to use. For more information on the available properties, see User configuration file and EVA Cluster Extension objects. 4. Save the file and copy it to the following directory on all cluster nodes: %ProgramFiles%\ Hewlett-Packard\Cluster Extension EVA\conf. 5. Bring the P6000 Cluster Extension resource online.
3. Double-click a disk in the summary pane. 4. Click the Dependencies tab, and then click Insert. 5. Select the P6000 Cluster Extension resource in the Resource menu. 6. Click OK to add the selected dependency. Adding dependencies using the CLI The cluster commands in this section can be used with Windows Server 2003, Windows Server 2008/2008 R2, Server Core, or Hyper-V Server.
Disaster-tolerant configuration example using a file share The following example assumes that your environment includes: • Four nodes (host1_DCA, host2_DCA, host3_DCB and host4_DCB) • Two P6000/EVA storage systems with WWNs 50001fe15002B5A0 and 50001fe15002B570 • A DR group configured as \Data Replication\DRG_LUN25 • Two management servers SMAM019FK351K and SMAM019FK001A • An MNS arbitrator node in a third site or separate protected area • A pre-executable clxpre.
Figure 5 on page 49 illustrates failover options and shows a second cluster group CLX_IIS. The resource CLX_FILESHARE is part of the service group CLX_SHARE and must be brought online before the physical disk resources Disk_P6000_LUN25. Figure 5 Service or application example . Figure 6 Resource tree for service or application CLX_SHARE . P6000 Cluster Extension is configured as a single resource to enable read/write access to the physical disk resource used for the CLX_SHARE cluster group.
To configure the resource according to the configuration shown in the Figure 5 on page 49: 1. Log in to the host3_DCB system with the Administrator account. 2. Create the file share service or application with all previously mentioned resources and its dependencies, except the P6000 Cluster Extension resource on host3_DCB. 3. Create a new resource of type P6000 Cluster Extension and add systems host2_DCA, host3_DCB and host4_DCB to its possible owners. 4.
Managing P6000 Cluster Extension resources You can manage resources by bringing them online and offline, or by deleting them. Bringing a resource online Resources are usually brought online automatically when the service or application is brought online. You might need to move the service or application to the node where you want to bring the resource online. When bringing resources online: • For Windows Server 2008/2008 R2, use the GUI, MMC, or CLI.
links are up. Ensure that the underlying infrastructure is in a healthy state before performing live migration. P6000 Cluster Extension has the capability of discovering unfavorable storage-level conditions for performing live migration. In response to these conditions, P6000 Cluster Extension will stop or cancel the live migration process and inform the user. This is accomplished with no VM downtime.
In addition, the service or application's FailoverPeriod value must be higher than the resource's RestartPeriod value and both must be higher than the resource's PendingTimeout value. Microsoft Cluster Server provides two properties to adjust state change recognition/resolution: • IsAlive • LookAlive The software automatically calls the IsAlive function whenever the cluster service calls the LookAlive function. Therefore, both functions must be set to the same value.
• If you see the customized P6000 Cluster Extension Parameters tab when you try to configure a P6000 Cluster Extension resource from a remote management station using the MMC, you must unregister clxevamscsex.dll from the cluster node. Use the command cluster /UNREGADMINEXT:clxevamscsex.dll to unregister the DLL. Unregistering the DLL allows you to configure the resource using the standard Microsoft Properties tab.
3 Configuring P6000 Cluster Extension for Linux P6000 Cluster Extension supports integration with the following cluster software for Linux: • Configuring P6000 Cluster Extension with RHCS, page 56 • Configuring P6000 Cluster Extension with SLE HA, page 64 NOTE: For a list of P6000 Cluster Extension versions and the cluster software versions they support, see the HP SPOCK website: http://www.hp.com/storage/spock.
• For RHCS, the configuration file /etc/opt/hpclxeva/conf/CLXEVA.config is associated with the RHCS service CLXWEB that is configured to use the P6000 Cluster Extension resource agent script. RHCS invokes the resource agent script to start the CLXWEB service. The CLXWEB service checks the DR group member states before the volume groups vgweb and vghtdocs are activated and the web server is started.
Configuration overview 1. Configure the SMI-S server information and copy it to all of the cluster nodes. For instructions, see Configure the SMI-S server information. 2. Create an RHCS shared resource. For instructions, see Creating an RHCS P6000 Cluster Extension shared resource. 3. Create an RHCS service using the P6000 Cluster Extension shared resource. For instructions, see Creating an RHCS service using the P6000 Cluster Extension shared resource.
5. Click Create a Resource to open the Resource Configuration dialog box. 6. Select Script in the Select a Resource Type box. 7. Enter CLXP6000 in the Name box. 8. Enter /usr/share/cluster/clxeva.sh in the File (with path) box. 9. Click OK. 10. Select File > Save to save the configuration changes. The service configuration in /etc/cluster/cluster.conf is updated. 11. Click Send to Cluster to propagate the cluster configuration to the other cluster nodes.
Using Conga to create a service To create a P6000 Cluster Extension service using Conga: 1. 2. 3. 4. 5. Log in to Conga. Click the Cluster tab, and then select Cluster List. Click the name of the cluster you want to administer. Click Services. Click Add a Service. The Add a Service page appears. 6. Enter the service name in the Service name box. IMPORTANT: The service name must match the name that is defined for the APPLICATION property in the P6000 Cluster Extension configuration file CLXEVA.config. 7.
NOTE: To add additional resources to the service, use the Add a child feature. Using system-config-cluster to create a service To create a P6000 Cluster Extension service using system-config-cluster: 1. Start the Cluster Configuration tool. 2. Click the Cluster Configuration tab. 3. Expand the Managed Resources tree. 4. Select Services. The Service properties page appears. 5. Click Create a Service. The Add a Service dialog box appears. 6.
9. To add additional resources to the service, select the P6000 Cluster Extension resource and click Attach a new Private Resource to the Selection. Select the resource to be configured and provide the required resource agent parameters. 10. Click Close to close the Service Management window. 11. Select File > Save to save the configuration changes. The service configuration in /etc/cluster/cluster.conf is updated. 12.
4. Run clxconf to obtain values for CSObjectPathA and CSObjectPathB. clxconf -smis -sg /etc/opt/hpclxeva/conf/CLXEVA.config CLXWEB The values for CSObjectPathA and CSObjectPathB are located in the file etc/opt/hpclxeva/ conf/clxobjpaths.cfg. For example: #cat etc/opt/hpclxeva/conf/clxobjpaths.cfg APPLICATION CLXWEB CSObjectPathA HPEVA_ConsistencySet.InstanceID="50001FE100270C30.6005-08b4 -0010-2e9b-0000-5000-0039-0000" CSObjectPathB HPEVA_ConsistencySet.InstanceID="50001FE100270CA0.
5. Relocate the service to a remote data center node. a. Verify that the DR group is in destination mode on the remote storage system. For example (using SSSU): REMOTE_EVA> show DR_GROUP "\Data Replication\clxwebdrg" The output follows: ... drmmode : destination ... NOTE: Use either SSSU or HP P6000 Command View to check the DR group mode. With SSSU, the DR_GROUP property is labeled drmmode. With HP P6000 Command View, the DR_GROUP property is labeled DR Role. b.
Stopping or disabling an RHCS service To stop a P6000 Cluster Extension service using Cluster User Service Administration (clusvcadm), enter the following command: #clusvcadm –d service name To disable a P6000 Cluster Extension service using Cluster User Service Administration (clusvcadm), enter the following command: clusvcadm -d service name NOTE: For instructions on stopping or disabling a P6000 Cluster Extension service using Conga or the Cluster Configuration Tool, see the RHCS documentation.
2. 3. Create a P6000 Cluster Extension resource using the SLE HA GUI. Use one of the following procedures: • Creating a P6000 Cluster Extension resource for Pacemaker • Creating a P6000 Cluster Extension resource for Heartbeat Test the configuration. For instructions, see Testing the configuration. Creating the P6000 Cluster Extension configuration file Create the P6000 Cluster Extension configuration file.
4. Run clxconf to obtain values for CSObjectPathA and CSObjectPathB. clxconf -smis -sg /etc/opt/hpclxeva/conf/CLXEVA.config CLXWEB The values for CSObjectPathA and CSObjectPathB are located in the file etc/opt/hpclxeva/ conf/clxobjpaths.cfg. For example: #cat etc/opt/hpclxeva/conf/clxobjpaths.cfg APPLICATION CLXWEB CSObjectPathA HPEVA_ConsistencySet.InstanceID="50001FE100270C30.6005-08b4 -0010-2e9b-0000-5000-0039-0000" CSObjectPathB HPEVA_ConsistencySet.InstanceID="50001FE100270CA0.
4. Select the following options for the P6000 Cluster Extension resource: Name Value Class ocf Provider heartbeat Type CLXEVA 5. Configure the instance attributes for the resource by selecting the app parameter. In the Value box, enter the APPLICATION tag name configured in the P6000 Cluster Extension configuration file (/etc/opt/hpclxeva/conf/CLXEVA.config). 6. Configure the start, stop, and monitor operations for the P6000 Cluster Extension resource. 7.
2. Add a group resource for P6000 Cluster Extension with the following settings: Name Value ID Enter a resource group ID. Ordered true Collocated true The Linux HA Management Client prompts you to enter the resource type details. 3. Set the value of the app parameter to the APPLICATION tag name configured in the P6000 Cluster Extension resource configuration file (/etc/opt/hpclxeva/conf/CLXEVA.config). NOTE: The resource hierarchy depends on the order in which resources are added.
1. Verify that the DR group is in destination mode on the remote storage system. For example (using SSSU): REMOTE_EVA> show DR_GROUP "\Data Replication\clxwebdrg" The output follows: ... drmmode : destination ... NOTE: Use either SSSU or HP P6000 Command View to check the DR group mode. With SSSU, the DR_GROUP property is labeled drmmode. With HP P6000 Command View, the DR_GROUP property is labeled DR Role. 2. In the SLE HA GUI, click Management in the left pane.
When a DR group failover occurs, the permission settings of the LUs in the DR group change from read-only to read-write at the destination site. In Linux configurations with the Device Mapper Multipath Software, the hosts do not dynamically detect the LU permission change. In this situation, the disks used in the P6000 Cluster Extension setup fail to come online when the host OS does not detect the LU permission change. As a workaround, configure the P6000 Cluster Extension script multipath_rescan.
Finding the user-friendly name of a multipath device The multipath_rescan.sh script requires that you enter the user-friendly names of the multipath devices. To obtain the user-friendly name of a multipath device: 1. Run the pvs command to view the multipath device names for your volume groups. In the following example, dm-21 and dm-23 are the multipath devices for the volume group vg01: [root@node1 ]# pvs PV VG /dev/dm-21 vg01 /dev/dm-23 vg01 /dev/dm-24 vg02 2.
NOTE: Because the failover environment is dispersed over two or more data centers, the failover time cannot be expected to be the same as that of a single data center with a single shared disk device. Therefore, you must adjust the resource startup timeout value based on failover tests you perform to verify the proper configuration setup. In an SLE HA environment, the timeout values for the start operation can be adjusted.
4 User configuration file and P6000 Cluster Extension objects Objects (also called properties in this document) define the disk array environment and failover/failback behavior. You can customize P6000 Cluster Extension objects in the user configuration file or in the cluster software. User configuration file For the CLI implementation of P6000 Cluster Extension, the user configuration file UCF.cfg is used to configure application service-specific information. UCF.
• COMMON • APPLICATION Objects have one of the following formats: tag A definition of an object; for example, COMMON or APPLICATION integer A number; for example, a timeout value string A name, which can include alphabetic and numeric characters and underscores; for example, an application startup value list A list of space-separated strings, for example, a list of host names (lists of numbers are stored as lists of strings) Text that is a comment starts with the pound (#) symbol and continues until
Object CLI MSCS DataSyncTimeout Linux • * DC_A_Hosts • • • * DC_B_Hosts • • • * DC_A_ManagementHosts • • • * DC_B_ManagementHosts • • • * DC_A_StorageCellWWN • • • * DC_B_StorageCellWWN • • • * DeviceGroup • • • FailsafeMode • • • FailsafeModeOverride • • • MergeCheckInterval • PostExecCheck • • • PostExecScript • • • PreExecScript • • • ResyncWaitTimeout • • • StatusRefreshInterval UseNonCurrentDataOk UseNonCurrentDataOkLinkUp • • • • •
COMMON Format tag Description Distinguishes between general (common) and application-specific objects. Format String Description (Optional) Defines the path to the P6000 Cluster Extension log file. Default value Linux /var/opt/hpclxeva/log LogDir Windows %ProgramFiles%\Hewlett-Packard\Cluster Extension EVA\ log LogLevel Format String Description (Optional) Defines the logging level used by P6000 Cluster Extension.
This object is used by all components within P6000 Cluster Extension that interface with HP SMI-S including the clxconf utility, the Connection Test function in the administrator GUI and the P6000 Cluster Extension resource DLL. You must consider this when setting the SmisRequestTimeout to a value other than the default.
Description Specifies the directory where P6000 Cluster Extension searches for applicationspecific files, such as the force flag or online file. If ApplicationDir is set to a nonexistent drive, P6000 Cluster Extension is unable to create the online file and cannot bring the resource online. Windows If ApplicationDir is not set, P6000 Cluster Extension uses the local %HPCLX_EVA_PATH% values as defined in the registry.
• The resource failed because the FailoverThreshold value has been reached. • The resource failed because the FailoverPeriod timeout value has been reached. CAUTION: Disable subsequent automated failover procedures for recovery failback operations. If you want to use P6000 Cluster Extension capabilities to recover a suspended DR group state during a failover, make sure that subsequent failover operations are disabled in case the recovery attempt fails.
Default value 10 seconds ClusterNotifyWaitTime Format Integer Description Specifies the amount of time that P6000 Cluster Extension will monitor for VM live migration state changes. Default value 5 seconds CSObjectPathA (Required) Format String Description Specifies the HPEVA_ConsistencySet object path for the DR group in data center A. This object path is used in sending management commands directed at the P6000 Continuous Access DR group.
hpclxeva/conf/CLXEVA.config). To obtain the value of this object, run the clxconf utility (see “clxconf” on page 91). CLI-specific behavior You must enter this object in the user configuration file (UCF.cfg) when you use P6000 Cluster Extension's CLI. To obtain the value of this object, run the clxconf utility (see “clxconf” on page 91).
Description Space-separated list of management servers for data center B. P6000 Cluster Extension attempts to establish a management path (HP SMI-S) connection with one of the management servers on the hosts in the list. The first host in the list that establishes a successful connection with P6000 Cluster Extension is used as the management path for the failover operation.
Valid values Disabled (default) Enabled FailsafeMode Override (Optional) Format String Description Specifies whether P6000 Cluster Extension can disable the failsafe mode configured for the DR group during a failover, if necessary. This property is used only if FailsafeMode is enabled for the DR group.
Description Specifies an executable with its full path name to be invoked after the failover action or failover procedure. You can include environment variables in path names. Do not specify arguments to be passed to the executable. PreExecScript (Optional) Format String Description Specifies an executable with its full path name to be invoked before the failover action or failover procedure. You can include environment variables in path names. Do not specify arguments to be passed to the executable.
• Setting this value to NO means that if the data is current in the remote data center, the resource will come online, even though UseNonCurrentDataOk is set to NO. If the data is not current in the remote data center, the resource will not come online.
# DataSyncTimeout 170 # MergeCheckInterval 20 NOTE: When you configure a UCF file, remove the # symbol for the properties you want to configure.
5 CLI commands and utilities Using its CLI and command-line utilities, P6000 Cluster Extension can be integrated with almost any cluster software for Linux and Windows operating systems.
Example 2 This example assumes you have defined an APPLICATION tag named myexchange in the UCF.cfg file and have specified all necessary objects, including the DeviceGroup object, to map the DR group to the application service my-exchange. P6000 Cluster Extension will check the vdisk set mapped to the application service my-exchange, and run the necessary failover procedure to enable read/write access to the vdisks.
Creating the P6000 Continuous Access environment HP support personnel are trained to set up P6000 Continuous Access. You can configure and change DR groups and HP SMI-S server instances using HP P6000 Command View, and the HP P6000 Continuous Access GUI. For more information, see the documents listed in “Related documentation” on page 107. Installing and configuring the HP P6000/EVA SMI-S server If not previously completed, install and configure HP P6000/EVA SMI-S server on all management servers.
0001-6e12-0000-7000-031a-0000" ---- Sample UCF.cfg end Timing considerations P6000 Cluster Extension Software gives priority to storage system operations over cluster software operations. If P6000 Cluster Extension is invoked during a DR group member resynchronization operation (merging) or is gathering information about the remote storage system, it waits until the requested status information is reported. This ensures the priority of data integrity over cluster software failover behavior.
Command-line utilities Command-line utilities are also included with P6000 Cluster Extension to allow configuration of P6000 Cluster Extension on all platforms for both the CLI (clxevarun) and cluster integration. clxconf Description Use clxconf to manually retrieve the consistency set object paths for a DR group and place them in a file called clxobjectpaths.cfg located in the P6000 Cluster Extension conf directory.
---- Sample UCF.
DC_A_ManagementHosts smam019fk351k # active sma DC_B_ManagementHosts clxeva1 # passive sma FailsafeMode disable DC_A_Hosts w2k7 w2k6 DC_B_Hosts w2k5 w2k8 CSObjectPathA HPEVA_ConsistencySet.InstanceID="50001FE15002B5A0. 6005-08b40001-4cf1-0000-6000-061f-0000" CSObjectPathB HPEVA_ConsistencySet.InstanceID="50001FE15002B570. 6005-08b40001-6e12-0000-7000-031a-0000" ---- Sample UCF.cfg end Example 2 RHCS or SLE HA integration configuration # clxconf -smis -sg /etc/opt/hpclxeva/conf/CLXEVA.
6005-08b4-0010-2971-0000-8000-00e9-0000" ---- Sample clxobjectpaths.cfg end Copy the consistency set object path from clxobjpaths.cfg to the P6000 Cluster Extension configuration file.
Example 1 This example shows the consistency set values for the DR group DRG031 on both P6000/EVA storage systems. C:\>cd %HPCLX_EVA_PATH% C:\Program Files\Hewlett-Packard\Cluster Extension EVA>cd bin C:\Program Files\Hewlett-Packard\Cluster Extension EVA\bin> clxgetobjpath CLXEVA1 50001FE15002B5A0 50001FE15002B570 "\Data Replication\DRG031" HPEVA_ConsistencySet.InstanceID="50001FE15002B5A0.6005-08b4 -0001-2b74-0000-6000-021f-0000" HPEVA_ConsistencySet.InstanceID="50001FE15002B570.
HP SMI-S parameters. clxresconf verifies the P6000 Cluster Extension resources of your MS cluster or the defined applications (APPLICATION tag) in the UCF.cfg. File output shows the resource name/application tag name and the status of the configuration. Syntax clxresconf [-mscs|-ucf] Operands –mscs Must be specified to check all P6000 Cluster Extension resources in the MSCS cluster. –ucf If you use the clxevarun CLI integration and created a UCF.cfg file, you can verify the file using this option.
clxpmu main menu Menu choice Description 1) View existing entries View a list of configured management servers in the following format: hostname host IP:SMI-S port number login:loginSSL:[SSLyes/SSLno] 2) Delete existing entry Select a management server to delete from list. This function requires you to enter the password for the selected HP SMI-S server before the operation can be performed.
1. Name (resource name) 2. Reserved 3. DR group name (DeviceGroup object) 4. Link state 5. DR mode (source/destination) 6. Log state 7. Failsafe mode 8. Failsafe lock state 9. Suspend mode state 10. Storage system WWN (local) 11. Storage system WWN (remote) 12. VCS firmware version 13. Application directory path (ApplicationDir object) 14. Log file location (LogDir object) 15. DC_A_Hosts node names 16. DC_B_Hosts node names Pre-executables and post-executables must supply a return code.
CAUTION: If the pre-execution program returns 1, 2, 3 or 5, a properly configured post-executable is not executed. If a failover function fails, the post-executable is not executed. Post-execution return codes Pre-execution return code Text Description 0 POST_OK Post-executable OK; continue. 1 POST_ERROR_GLOBAL Post-executable failed; stop application service cluster-wide. MSCS does not allow cluster-wide or data-center-wide disabling of resources. This is local error behavior.
CLI commands and utilities
6 Troubleshooting To troubleshoot problems with P6000 Cluster Extension, you must understand HP P6000 Continuous Access environments. See the HP P6000 Continuous Access and HP P6000 SMI-S documentation before assuming that a problem has been caused by P6000 Cluster Extension. Changing the P6000/EVA SMI-S user name and password If the HP P6000/EVA SMI-S user name or password is changed through the P6000 Command View GUI or any other method, you must change the information in P6000 Cluster Extension.
Error return code Description global error Returned if the configuration or the disk state do not allow an automatic application service startup process. Manual intervention is required in such cases. MSCS does not allow cluster-wide or data-center-wide disabling of resources. This is local error behavior. When P6000 Cluster Extension is integrated into the cluster software, an error message string and integer value are displayed. For the CLI, a return code is displayed.
(for example the cluster service) have not recognized the %HPCLX_EVA_PATH% variable. To fix this problem, restart the cluster service. • A start error occurs if the APPLICATION name tag value in the P6000 Cluster Extension resource configuration file does not match the service name (RHCS) or the P6000 Cluster Extension resource App (SLE HA). • If an error occurs because needed components of P6000 Cluster Extension are not available, a file is generated that could help with troubleshooting.
MNS needs its own resource monitor (Windows only) The MNS quorum is a regular cluster resource that is monitored by the cluster resource monitor (ResrcMon.exe). To avoid cluster failures because of other hanging resources, HP recommends that the MNS resource run in its own resource monitor process. This can be changed by selecting the MNS quorum resource in the Cluster Administrator GUI.
If you have Secure Path installed and you still cannot see the drive letter when adding the physical disk resource, you can use the Secure Path command spprutil device to find the mapping between the Windows physical disk number and the shown partition numbers. You can add the disk based on this mapping and failover/failback the disk. The correct drive letter will show up when the disk is brought online on the node where you originally created the disk partition and drive letter.
3. Restart the node that was shut down. NOTE: The time to detect a storage outage due to failure of all paths to storage depends on the setting for no_path_retry in the multipath software configuration. A value of fail does not queue I/O in the event of a failure in all paths and returns an immediate failure. For information about the recommended value for your environment, see the DM-Multipath documentation.
7 Support and other resources Contacting HP For worldwide technical support information, see the HP support website: http://www.hp.
• HP Storage System Scripting Utility Reference Guide • HP Enterprise Virtual Array Compatibility Reference You can find these documents on the Manuals page of the HP Business Support Center website: http://www.hp.com/support/manuals In the Storage section, click Storage software, and then select your product. White papers The following white papers are available at www.hp.
Convention Element • Keys that are pressed • Text typed into a GUI element, such as a box Bold text • GUI elements that are clicked or selected, such as menu and list items, buttons, tabs, and check boxes Italic text Text emphasis • File and directory names • System output Monospace text • Code • Commands, their arguments, and argument values • Code variables Monospace, italic text • Command variables Monospace, bold text Emphasized monospace text CAUTION: Indicates that failure to follow direct
http://www.hp.com/support/storagedocsurvey Thank you for your time and your investment in HP storage products.
Glossary array A synonym of storage array, storage system, and virtual array. A group of disks in one or more disk enclosures combined with controller software that presents disk storage capacity as one or more virtual disks. asynchronous A term used to describe computing models that eliminate timing dependencies between sequential processes. In asynchronous replication, the array controller acknowledges that data has been written at the source before the data is copied at the destination.
data movement P6000 Continuous Access provides for data movement services, such as data backup, data migration, data distribution, and data mining. data replication mode The operational mode of a DR group that indicates the capability of I/O to be written to its source and/or its destination. See also DR mode default disk group The disk group that is created when an array is initialized. The minimum number of disks the group can contain is eight. The maximum is the number of installed disks.
DR group log state The current behavior of the log associated with a DR group. In the state options, references to multiple destinations are for future use. There are three possible states: • Normal – No destination is logging or merging. • Logging – At least one destination is logging; none are merging. • Merging – At least one destination is merging. DR group write mode Characterizes how a write from a host is replicated.
Fibre Channel A data transfer architecture designed for mass storage devices and other peripheral devices that require high bandwidth. full copy The process of copying all data written on a source virtual disk directly to the destination virtual disk. If a data block contains all zeros, a zeroing message is sent to the destination array's virtual disk for the corresponding data block. This minimizes the amount of data that must be written to the destination virtual disk.
managed set A selection of resources grouped together for convenient management. For example, you can create a managed set to manage all DR groups whose sources reside in the same rack. management server A server on which HP Enterprise Virtual Array management software is installed, including HP P6000 Command View and HP Replication Solutions Manager, if used. A dedicated management server runs EVA management software exclusively.
SMI-S Storage Management Initiative Specification. snapclone A copy that begins as a fully allocated snapshot and becomes an independent virtual disk. Applies only to the HP P6000/EVA array. snapshot A nearly instantaneous copy of the contents of a virtual disk created without interrupting operations on the source virtual disk. Snapshots are typically used for short-term tasks, such as backups. source vdisk A vdisk that contains original data that is replicated to its destination vdisk.
vdisk A simulated disk drive accessible from hosts attached to the SAN. When it is a member of a DR group, a vdisk is the HSV implementation of a copy set, and it can have two states: normal and copying. virtual disk family A virtual disk and its snapshot, if a snapshot exists, constitute a family. The original virtual disk is called the active disk. When you first create a virtual disk family, the only member is the active disk. VM Virtual Machine. VSC Volume size customization. Also known as CVS.
Glossary
Index A APPLICATION description, 77 ApplicationDir description, 77 ApplicationStartup description, 78 asynchronous replication mode, 20 AutoFailbackType description, 35 B Basic Resource Health Check Interval description, 34 C CLI configuration, 9 ClusterNotifyCheckTime description, 79 UCF requirement, 45 ClusterNotifyWaitTime description, 80 UCF requirement, 45 clxconf command, 91 clxevarun command, 87 clxgetobjpath command, 94 clxpmu command, 96 clxresconf command, 95 command line interface configuring,
Device Mapper Multipath Software Rescanning devices, 69 DeviceGroup description, 82 disaster tolerance, 10 example, 48 document conventions, 108 related documentation, 107 documentation HP website, 108 providing feedback, 109 DR groups, 20 E error return codes failover, 101 errors failover error handling, 103 exporting configuration information, 28 F FailoverPeriod description, 35 FailoverThreshold description, 35 FailsafeMode description, 82 FailsafeMode Override description, 83 FASTFAILBACK value descri
mirrored disks automated redirection, 10 multipath_rescan script, 69 Q quorum considerations Windows, 11 N names changing (Microsoft Cluster Service), 32 changing (MSCS), 32 O objects APPLICATION, 77 COMMON section, 75 P6000 Cluster Extension, 73 P P6000 Business Copy rolling disaster protection, 21 P6000 Command View, 24 P6000 Cluster Extension components, 19 configuring with MSCS, 25 features, 9 integration, 9 P6000 Command View server, 21 P6000 Command View dependencies, 22 Storage System Scripting U
S sample configuration Windows, 48 Secure Socket Layer (SSL) protocol, 23 service or application bouncing, 53 SMI-S server, 21 SmisRequestTimeout description, 76 SSL protocol, 23 start errors troubleshooting, 102 StatusRefreshInterval description, 84 UCF requirement, 45 storage management appliance (SMA), 21 Storage System Scripting Utility, 24 storage systems consolidated DR site configuration, 18 one-to-one configuration, 16, 17 supported configurations, 16 Subscriber's Choice, HP, 107 synchronous replica
