HP 3PAR Cluster Extension Software Administrator Guide Abstract This guide contains detailed instructions for configuring and troubleshooting HP 3PAR Cluster Extension Software in Windows and Linux environments. The intended audience has independent knowledge of related software and of the HP 3PAR storage system and its software.
© Copyright 2012 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 HP 3PAR Cluster Extension features...............................................................7 Integration into cluster software (Graphical user interface)..............................................................7 CLI for easy integration..............................................................................................................7 Disaster tolerance through geographical dispersion.......................................................................
2 Configuring HP 3PAR Cluster Extension for Windows....................................21 Integrating HP 3PAR Cluster Extension with MSCS.......................................................................21 Configuring HP 3PAR Cluster Extension.....................................................................................21 Starting the HP 3PAR Configuration Tool...............................................................................
Logs......................................................................................................................................47 Hyper-V Live Migration log entries.............................................................................................47 3 Configuring HP 3PAR Cluster Extension for Linux...........................................48 HP HP 3PAR Cluster Extension for Linux: Sample configuration......................................................
Post-execution return codes..................................................................................................76 6 Troubleshooting........................................................................................77 Logs......................................................................................................................................77 Windows logs...................................................................................................................
1 HP 3PAR Cluster Extension features HP 3PAR Cluster Extension offers protection against application downtime due to a fault, failure, or site disaster by extending a local cluster between data centers over metropolitan distance. HP 3PAR Cluster Extension reinstates critical applications at a remote site within minutes of an adverse event, integrating your open-system clustering software and HP 3PAR Remote Copy to automate failover and failback between sites.
Figure 1 Physical replication using HP 3PAR Remote Copy HP 3PAR Remote Copy-mirrored disks have a read/write-enabled primary (local) disk and a read-only secondary (remote) disk. Current cluster software products cannot distinguish between read-only and write-enabled disks, and cannot enable disk access if the disk is not write-enabled during the server boot process.
Fully Automatic Failover and Failback Automated failover and failback reduces the complexity involved in a disaster recovery situation. It is protection against the risk of downtime, whether planned or unplanned.
HP 3PAR Cluster Extension supports the following configurations: • 1-to-1 • N-to-1 (the logical configuration must be a one-to-one configuration). See Figure 2 (page 11). • 1-to-N (the logical configuration must be a one-to-one configuration). See Figure 2 (page 11). NOTE: Synchronous long distance and unidirectional remote copy configurations are not supported.
Figure 2 1-to-1 configuration N-to-1 and 1-to-N configurations NOTE: The restrictions outlined in “1-to-1 configuration” (page 10) also apply to N-to-1 configurations. When setting up an N-to-1 or 1-to-N configuration (see Figure 3 (page 12)), consider the following: • HP 3PAR Cluster Extension uses the unit of Remote Copy volume groups as the failover unit.
Figure 3 N-to-1 and 1-to-N configuration Processes and components For Windows Clustering, HP 3PAR Cluster Extension is shipped as a resource DLL. For RHEL and SUSE, HP 3PAR Cluster Extension is shipped as a binary executable that is called by native clustering software when data replication is enabled through the package control script. With both Windows and Linux, the installation of HP 3PAR Cluster Extension also includes a CLI for custom HP 3PAR Remote Copy integration.
Figure 4 Relationship of primary versus secondary volume groups on primary and backup storage servers Remote Copy pairs Remote Copy configurations are based on the relationship between a storage system pair, also known as the Remote Copy pair. Within this pair, the primary storage system is the system that holds the volumes that are copied to a backup or secondary storage system, also known as a remote storage system.
Synchronous replication Using synchronous mode, all write requests from the server are first transferred to the remote storage system. After each I/O has been mirrored in the cache area of the remote storage system, it is acknowledged to the local storage system. The write request is then acknowledged to the server.
NOTE: HP 3PAR Cluster Extension does not support shared disk quorum with Windows Clustering. MNS or Node Majority quorum in Windows Clustering An MNS is a single quorum resource that allows you to build clusters with nodes that are geographically separated. In an MNS cluster, unlike in a quorum disk cluster, data is stored on multiple disks across the cluster.
NOTE: File Share Witness support: • Windows Server 2003 (Service Pack 1 and later) supports only two node clusters when using File Share Witness. • Windows Server 2003 Service Pack 2 supports MNS with File Share Witness by default. • When using Microsoft Windows Server 2003 Service Pack 1 (SP1)-based server cluster or Windows Server 2003 R2-based server cluster, you must apply a Microsoft patch to support MNS with File Share Witness. See your Microsoft documentation for more information.
There are three types of resource constraints: • Resource location: Defines the nodes on which a resource can run, cannot run, or is preferred to be run. • Resource colocation: Defines which resources can or cannot run together on a node. • Resource order: Defines the sequence of actions for resources running on a node. Resource operation attribute SLE HA does not monitor resource health by default. To enable this feature, add the monitor operation to the resource definition.
Disk monitoring For the situations in which disk access is lost or read/write protection is in effect due to storage fencing, application monitoring agents, file system agents, or LVM resource agents detect the IO failure. HP 3PAR Cluster Extension does not monitor the disk access status. RHCS cluster setup considerations Quorum In RHCS, the quorum is based on a simple voting majority of the defined nodes in a cluster. To re-form successfully, a majority of all possible votes is required.
A failover domain can have the following characteristics: • Unrestricted: Specifies that the subset of members is preferred, but the cluster service assigned to this domain can run on any available member. • Restricted: The cluster service is allowed to run only on a subset of failover domain members. • Unordered: The member on which the cluster service runs is chosen from the available list of failover domain members with no preference order.
Configuration tool (clx3parconfig.exe) for Windows HP 3PAR Cluster Extension for Windows uses the configuration tool (clx3parconfig.exe) to store configuration and access information for each configured management server. This tool stores information in the clx3parcfg file.
2 Configuring HP 3PAR Cluster Extension for Windows After installing HP 3PAR Cluster Extension, use the configuration tool to define the HP 3PAR Cluster Extension setup configuration. Use Cluster Administrator (Windows Server 2003), Failover Cluster Management (Windows Server 2008/2008 R2), or cluster commands in the CLI to add and configure resources. Integrating HP 3PAR Cluster Extension with MSCS To integrate HP 3PAR Cluster Extension with MSCS: 1.
IMPORTANT: User Account Control (UAC) is a feature in Windows Server 2008 onwards that can help prevent unauthorized changes to your computer. UAC does this by asking you for permission or an administrator password before performing actions that could potentially affect your computer's operation or that change settings that affect other users. For more information on UAC, see http://windows.microsoft.com/en-US/windows-vista/What-is-User-Account-Control .
Defining the HP 3PAR configuration information using the GUI To define configuration information using the GUI: 1. Open the configuration tool. For instructions, see “Starting the HP 3PAR Configuration Tool” (page 22) 2. Click Add in the Array Configuration section to open the Add Array dialog box.
3. 4. 5. Enter the Array Network Name. Click Browse to select the password file. Click Connect to connect to the array. The array details (Array Name, Array Serial Number, Array Model, and Array ID) and the array target information appears on the Add Array dialog box. NOTE: If the Remote Copy link is down between the array and the defined targets, Cluster Extension will not be able to obtain the target array information. The value in the target array ID will be NA and a warning message is displayed.
8. Select the servers that are possible owners of the HP 3PAR Cluster Extension-managed disks, and click OK. NOTE: See the Microsoft Windows Clustering documentation for more information about possible owners. 9. Click OK. You will be prompted to confirm if you want to automatically copy the configuration file on all the cluster nodes used by Cluster Extension. a. If you select Yes, the configuration information is automatically copied to all cluster nodes used by Cluster Extension.
NOTE: You must create or copy the array password file on all cluster nodes used by Cluster Extension. Ensure that the appropriate path is reflected in the configuration information of each node. During configuration, you will be prompted to confirm if you want to copy the configuration file on all cluster nodes. If you choose to copy the HP 3PAR Cluster Extension configuration file automatically, the same password file location is retained on all cluster nodes.
Importing configuration settings using the GUI 1. 2. 3. 4. 5. Open the configuration tool. For instructions, see “Starting the HP 3PAR Configuration Tool” (page 22). Click Import. When prompted, choose the configuration file, and then click Open. Click OK to save and close the configuration tool. Importing configuration settings using the CLI 1. 2. Open a command window. Enter clx3PARconfig IMPORT /FILE=filepath, where filepath specifies the file location and name.
1. 2. 3. 4. 5. 6. 7. 8. Create a new resource as described in your Microsoft documentation. Enter values for the following fields: • Name: A resource name, for example, clx01. • Description: As appropriate for the resource. • Resource type: Select Cluster Extension 3PAR. • Group: The resource group to be associated with the new resource, for example, Application_ A. Click Next. Add or remove resource owners. Click Next. Do not add any dependencies. Click Next.
Changing a HP 3PAR Cluster Extension resource name (Windows Server 2003) In this procedure, you use the Cluster Administrator GUI to change a resource name. For instructions on using CLI commands to change a resource name, see “Setting HP 3PAR Cluster Extension resource properties using the CLI” (page 38). 1. Open Cluster Administrator. 2. Open the resource Properties window and click the General tab. 3. Enter a new name in the Name field. 4. Click OK to save your changes and close the window.
Setting Microsoft cluster-specific resource and service or application properties Microsoft allows you to set specific failover parameter and threshold values for a service or application, and for a resource. Some of these values must be changed for HP 3PAR Cluster Extension to enable manual recovery actions in case of a disaster. To set Microsoft cluster-specific resource properties: • For Windows Server 2003, use the Cluster Administrator GUI or cluster commands in the CLI.
Table 1 Setting resource properties and values in the GUI (continued) Property Format Description Value Advanced tab of the resource properties window in Cluster Administrator. If a resource fails, attempt restart on current node – Maximum restarts in the specified period (Windows Server 2008/2008 R2) Integer Defines whether a resource 0 (Required) can be 0 (Required) automatically restarted after it has failed.
HP 3PAR Cluster Extension requirements for service or application properties are described in Table 2 (page 32). If no specific value is required, the default value is listed. Set these values in the Failover tab of the service or application properties window (Windows Server 2008/2008 R2), the resource group properties window (Windows Server 2003), or in the CLI. For more information about setting service or application properties, see your Microsoft documentation.
• “Setting HP 3PAR Cluster Extension resource properties using the CLI” (page 38) • “Setting HP 3PAR Cluster Extension properties using a UCF” (page 39) Setting HP 3PAR Cluster Extension resource properties using the GUI This section describes the procedures for setting HP 3PAR Cluster Extension resource properties with a GUI.
4. Make the necessary parameter changes, and then click OK. Using the resource configuration tool to set resource properties (Server Core and Hyper-V Server) To set resource properties for Server Core or Hyper-V Server, use the HP 3PAR Cluster Extension resource configuration tool. When using the resource configuration tool: • You must run the tool on a Server Core or Hyper-V cluster node. You cannot run the tool on a remote management station.
Configuring cluster node data center assignments To configure cluster node data center assignments from the cluster software Parameters tab or the resource configuration tool: 1. To add a cluster node: a. Click Add to open the Add Server Data center Assignment window. b. c. d. 2. 3. 4. Select a cluster node from the Pre-Configured host list. Assign the cluster node to a data center selected from the Data center list. Click OK to save and close the window.
Configuring HP 3PAR storage system To configure HP 3PAR storage systems from the cluster software Parameters tab or the resource configuration tool: 1. To add a storage system: a. Click Add to open the Add Array Data center Assignment window. b. c. d. 2. 3. 4. Select a storage system from the Storage System Name list. Assign the storage system to a data center selected from the Data center list. Click OK to save and close the window.
Use this dialog box to configure the Remote Copy volume group behavior. 2. 3. • The settings in the Replication link up scenario section affect the failover behavior when the HP 3PAR Remote Copy link is up. • The settings in the Replication link down scenario section affect the failover behavior when the HP 3PAR Remote Copy link is down. Select RESYNCWAIT or FASTFAILBACK for the Application Startup Behavior. If you selected RESYNCWAIT in Step 2, set the RESYNCWAIT timeout value. • 4. 5. 6. 7.
2. 3. Update the PreExecScript, PostExecScript, and PostExecCheck values as needed, and then click OK. Click Apply to finish your modifications and leave the Properties window open, or OK to save your changes and close the window. Setting HP 3PAR Cluster Extension resource properties using the MMC If you are using Server Core or Hyper-V Server, you can manage a cluster remotely by using the MMC to run Failover Cluster Management.
Setting HP 3PAR Cluster Extension properties using a UCF You can use a UCF to configure certain HP 3PAR Cluster Extension properties for Windows. Common properties that you can configure in a UCF for cluster integrated CLX and clx3parrun include: • LogLevel • ClusterNotifyCheckTime • ClusterNotifyWaitTime • StatusRefreshTime • HP3PARCLICommandTimeout • HP3PARRCPeriodicLinkDownTimeout IMPORTANT: If you plan to use the default values for these properties, no UCF is required.
5. Click OK to save your changes. Adding dependencies using Failover Cluster Management (Windows Server 2008/2008 R2) You can add dependencies with the GUI on a local node or by using the MMC to run the Failover Cluster Management application. 1. Open Failover Cluster Management. 2. Select a service or application that has HP 3PAR Cluster Extension resource. 3. Double-click a disk in the summary pane. 4. Click the Dependencies tab, and then click Insert. 5.
To add a dependency on a HP 3PAR Cluster Extension resource using the CLI, use the following command: cluster resource physical_disk_resource /adddependency:Cluster_Extension_3PAR_resource The following command adds a dependency on the HP 3PAR Cluster Extension clx_fileshare resource to the physical disk resource Disk_32b_00b: cluster resource Disk_32b_00b /adddependency:clx_fileshare Disaster-tolerant configuration example using a file share The following example assumes that your environment includes: •
Figure 6 Resource tree for service or application CLX_SHARE HP 3PAR Cluster Extension is configured as a single resource to enable read/write access to the physical disk resource used for the CLX_SHARE cluster group. The physical disk resource depends on the HP 3PAR Cluster Extension resource and can be brought online only when the HP 3PAR Cluster Extension resource is already online.
C:\>cluster resource CLX_FILESHARE /prop 10. For Windows Server 2008/2008 R2 only: a. Click the Policies tab of the Failover Cluster Management Properties window. b. For the HP 3PAR Cluster Extension resource property response to resource failure, select If resource fails, do not restart. 11. Bring the service or application online on host3_DCB, and verify that the resource and all other CLX_SHARE service or application resources are brought online. 12.
When deleting resources: • For Windows Server 2008/2008 R2, use the GUI or CLI. • For Server Core or Hyper-V Server, use the CLI or the MMC. • For Windows Server 2003, use the GUI or CLI. For more information on deleting resources, see your Microsoft documentation. Using Hyper-V Live Migration with HP 3PAR Cluster Extension Live migration is a managed failover of VM resources.
• The value of the HP3PARCLICommandTimeout property must be less than the configured resource PendingTimeout value. • The value of HP3PARRCPeriodicLinkDownTimeout property should be set between 200 and 300 seconds based on the time taken by the remote copy to make the VG status to go to the stopped state after the RC link is brought down.
Administration Cluster Extension administration includes remote management of resources and monitoring of system resources and logs. Remote management of HP 3PAR Cluster Extension resources in a cluster You can use the MMC with Failover Cluster Management to manage clusters and configure HP 3PAR Cluster Extension resources.
Logs If the HP 3PAR Cluster Extension log files need to be cleared and reset (for example, to reduce disk space usage), you can delete the files. HP 3PAR Cluster Extension automatically creates new log files. TIP: Archive the log files before deleting them. Hyper-V Live Migration log entries In the HP 3PAR Cluster Extension log file (clxmscs.log), live migration messages include the prefix CLX_LM to help you differentiate live migration issues from HP 3PAR Cluster Extension log messages.
3 Configuring HP 3PAR Cluster Extension for Linux HP 3PAR Cluster Extension supports integration with the following cluster software for Linux: • “Configuring HP 3PAR Cluster Extension with RHCS” (page 50) • “Configuring HP 3PAR Cluster Extension with SLE HA” (page 57) NOTE: For a list of HP 3PAR Cluster Extension versions and the cluster software versions they support, see the HP SPOCK website: http://www.hp.com/storage/spock.
LVM volume groups vgweb and vghtdocs. The sample CLX3PAR.config file shows the contents of the configuration file with the described failover behavior. For SLE HA, the configuration file /etc/opt/hpclx3par/conf/CLX3PAR.config is associated with the SLE HA resource CLXWEB. SLE HA invokes the resource agent script /usr/lib/ocf/ resource.d/heartbeat/CLX3PAR, which checks the Remote Copy volume group member states before the LVM volume groups vgweb and vghtdocs are activated and the web server is started.
Configuring HP 3PAR Cluster Extension After installation, you must define the configuration setup using the HP 3PAR Cluster Extension Configuration Tool for both RHCS and SLES HA cluster software. Defining HP 3PAR configuration information using the CLI You can configure HP 3PAR Cluster Extension using the CLI command clx3PARconfig. Enter clx3PARconfig /?, or clx3PARconfig /help to view usage information. 1.
Creating an RHCS HP 3PAR Cluster Extension shared resource After HP 3PAR Cluster Extension is installed, as described in the HP 3PAR Cluster Extension Installation Guide, use Conga or the Cluster Configuration Tool (system-config-cluster) to create a HP 3PAR Cluster Extension shared resource. This procedure is required as part of the initial HP 3PAR Cluster Extension configuration procedure. After you complete this procedure, you do not need to repeat it when you add services.
9. Click OK. 10. Select File > Save to save the configuration changes. The service configuration in /etc/cluster/cluster.conf is updated. 11. Click Send to Cluster to propagate the cluster configuration to the other cluster nodes. Creating an RHCS service using the HP 3PAR Cluster Extension shared resource After you create a shared resource, create an RHCS service using the HP 3PAR Cluster Extension shared resource. Configuration overview 1. 2. 3.
IMPORTANT: The service name must match the name that is defined for the APPLICATION property in the HP 3PAR Cluster Extension configuration file CLX3PAR.config. 7. 8. 9. Select a failover domain. For information about the failover domain requirements, see Failover domains . Select Relocate for the recovery policy. Click Add a resource to this service to add the HP 3PAR Cluster Extension shared resource. The Add a resource page appears. 10.
11. Click Submit. Conga saves the configuration information and updates all of the other cluster nodes. NOTE: To add additional resources to the service, use the Add a child feature. Using system-config-cluster to create a service To create a HP 3PAR Cluster Extension service using system-config-cluster: 1. Start the Cluster Configuration tool. 2. Click the Cluster Configuration tab. 3. Expand the Managed Resources tree. 4. Select Services. The Service properties page appears. 5. Click Create a Service.
7. Click Add a Shared Resource to this service. The Resource Configuration dialog box appears. 8. 9. Select CLX3PAR in the Select a Resource Type menu, and then click OK. To add additional resources to the service, select the HP 3PAR Cluster Extension resource and click Attach a new Private Resource to the Selection. Select the resource to be configured and provide the required resource agent parameters. 10. Click Close to close the Service Management window. 11.
3. In the configuration file (CLX3PAR.config), enter the appropriate values for: • DC_A_InServStorage_SerNum • DC_B_InServStorage_SerNum • RCVolumeGroupA • RCVolumeGroupB • DC_A_Hosts • DC_B_Hosts See “User configuration file” for more information. For example: COMMON #LogLevel info APPLICATION CLXWEB #DC_A_InServStorage_SerNum 1302871 #DC_B_InServStorage_SerNum 1302835 #RCVolumeGroupA CLX_VG #RCVolumeGroupB CLX_VG.
c. d. Verify that the Remote Copy volume group is in source mode on the remote storage system. Stop the service and verify that the service stopped successfully. #clusvcadm –d CLXWEB 6. After verifying that the CLXWEB service can be moved with RHCS and that the software performs automatic failover of the Remote Copy volume group on each cluster node, start the service on its primary cluster node.
1. 2. 3. Create the configuration file. For instructions, see “Creating the HP 3PAR Cluster Extension configuration file” (page 58) Create a HP 3PAR Cluster Extension resource using the SLE HA GUI. Use the procedure for “Creating a HP 3PAR Cluster Extension resource for Pacemaker” (page 58) Test the configuration. For instructions, see “Testing the configuration” (page 60) Creating the HP 3PAR Cluster Extension configuration file Create the HP 3PAR Cluster Extension configuration file.
1. 2. 3. Start the Linux HA Management Client. Select Add group from the Resources menu, and enter a group ID. Add HP 3PAR Cluster Extension as a primitive group's first resource. NOTE: The resource hierarchy depends on the order in which resources are added. Always add HP 3PAR Cluster Extension resources as the first resource in a group. 4. 5. 6. 7.
10. Set the operation defaults to control failover behavior. To specify that when a resource fails, the resources attempts to restart on the same node or another node in the cluster, use the following settings: Name Value requires nothing on-fail restart timeout 30 11. Set the migration-threshold value. This value defines the number of failures that can occur on a node before the node becomes ineligible to host the resource and the resource fails over to another node.
fail to come online when the host OS does not detect the LU permission change. As a workaround, configure the HP 3PAR Cluster Extension script multipath_rescan.sh as a PostExecScript to rescan the disks before they are brought online. Configuring the rescan script The multipath rescan script is available in the directory /opt/hpclx3par/sample/ multipath_rescan.sh. To configure the script to run as a PostExecScript: 1. Copy the multipath_rescan.
brw-rw---- 1 root disk 253, 17 Nov 18 12:42 mpath18p1 3. Run the multipath -ll command to view the multipath device names and SCSI ID for your volume groups for Red Hat Enterprise Linux. [root@ Node1 ~]# multipath -ll mpath18 (350002ac0192e0f50) dm-9 3PARdata,VV [size=5.0G][features=0][hwhandler=0][rw] \_ round-robin 0 [prio=2][active] \_ 1:0:0:8 sdh 8:112 [active][ready] \_ 1:0:1:8 sdp 8:240 [active][ready] In the above example mpath18 needs to be put in the multipath_rescan.sh file.
4 User configuration file and HP 3PAR Cluster Extension objects Objects (also called properties in this document) define the HP 3PAR storage system environment and failover/failback behavior. You can customize HP 3PAR Cluster Extension objects in the user configuration file or in the cluster software. User configuration file For the CLI implementation of HP 3PAR Cluster Extension, the user configuration file UCF.cfg is used to configure application service-specific information. UCF.
string A name, which can include alphabetic and numeric characters and underscores; for example, an application startup value list A list of space-separated strings, for example, a list of host names (lists of numbers are stored as lists of strings) Text that is a comment starts with the pound (#) symbol and continues until the end of the line. Comments can start on a new line or be part of a line specifying an object.
COMMON objects The COMMON section is used to set the environment of HP 3PAR Cluster Extension. The COMMON tag can appear in the configuration file only once. The COMMON object does not require any value. Objects of the type COMMON can appear only one time. Those objects must be placed after the COMMON tag in the configuration file. If the default values fit your environment, there is no need to specify them in the file.
Default value 250 seconds APPLICATION objects The APPLICATION section defines the failback behavior of HP 3PAR Cluster Extension for each application service. APPLICATION is a multitag that can appear in the configuration file for each application service using HP 3PAR Cluster Extension. The APPLICATION object requires the name of the application service as its value. The objects specified after an APPLICATION tag must appear only once per application.
The ApplicationStartup object can be customized to determine whether an application service starts locally or is transferred back to the remote data center (if possible) to start immediately without waiting for disk pair resynchronization. This object is used only if an application service has already been transferred to the secondary site and no recovery procedure has been applied to the Remote Copy volume group (the Remote Copy volume group has not been recovered and is not in a normal state).
the ResyncWaitTimeout object must be specified, in case HP 3PAR Cluster Extension should wait for resynchronization changes for more or less than 90 seconds, which is the default. Files application_name.forceflag CLX_application_name.online If specified in a user configuration file, resource_name is the value of the APPLICATION tag; otherwise, resource_name is the value of the HP 3PAR Cluster Extension resource name.
Description HP 3PAR Remote Copy volume group name on configured DCB HP 3PAR storage system, containing the application service disk set. NOTE: HP 3PAR InForm Management Console (IMC) will not provide details of RCVolumeGroupA and RCVolumeGroupB. Use HP 3PAR CLI command showrcopy groups to find out the exact name for RCVolumeGroupA and RCVolumeGroupB.
Default value 300 seconds UseNonCurrentDataOk (Optional) Format String Description When the replication link is down, this property specifies whether HP 3PAR Cluster Extension can fail over to the destination site when the data at the destination site might not be current.
5 CLI commands and utilities Using its CLI and command-line utilities, HP 3PAR Cluster Extension can be integrated with almost any cluster software for Windows operating systems.
Related information • clx3PARrun expects only the app_name parameter, which uniquely identifies an application service in the APPLICATION section of the user configuration file. • To force a Remote Copy volume group failover, use the –forceflag option. CAUTION: The forceflag option is an emergency switch used to manually activate your virtual volume.
Installing HP 3PAR InForm Command Line Interface (CLI) HP 3PAR Cluster Extension communicates with the HP 3PAR storage system using the HP 3PAR InForm Command Line Interface (CLI). Ensure that you install the supported version of HP 3PAR Command Line Interface (CLI) on a host where Clx3PARrun can be used. Creating and configuring the user configuration file The CLI expects as an argument the name configured as the APPLICATION tag value.
and the number of remote Copy volume groups managed by HP 3PAR storage systems, the online operation could time out. This can also happen if clx3PARrun is used in a script or called by another program or 3PAR storage system IP cannot be reached. • In the UCF.cfg file, if the ApplicationStartup attribute is set to RESYNCWAIT, HP 3PAR Cluster Extension tries to resynchronize virtual volumes and waits until the Remote Copy volume group is in a synced state.
HP 3PAR Cluster Extension transfers information as CLI arguments to the pre-execution and post-execution programs. You must not specify CLI arguments. Pre-executables and post-executables must be specified by full path in the PreExecScript and PostExecScript objects. If no executable is specified (there is an empty value for the object), no preprocessing or postprocessing is performed. If the object is not defined, then the argument value is empty. The 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
Post-execution return codes Pre-execution return code Text Description 0 POST_OK Post-executable OK; continue. 1 POST_ERROR_GLOBAL Post-executable failed; stop application service cluster-wide. MSCS does not allow cluster-wide or data-center-wide disabling of resources. This is local error behavior. 2 POST_ERROR_DC Post-executable failed; stop application service in this data center. MSCS does not allow cluster-wide or data-center-wide disabling of resources. This is local error behavior.
6 Troubleshooting To troubleshoot problems with HP 3PAR Cluster Extension, one must understand HP 3PAR Remote Copy environments. See the Remote Copy User's Guide for more information on Remote Copy configuration and volume group states. See the HP 3PAR Cluster Extension log for any cluster specific information. However, it always keeps its own log files in its default log location.
Log facility The logging module of HP 3PAR Cluster Extension provides log messages to the cluster software as well as to the HP 3PAR Cluster Extension log file. The HP 3PAR Cluster Extension log file includes disk status information and is located by default in this directory: Windows %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\log\ Linux /var/opt/hpclx3par/log If you need to clear and reset the log file, for example to reduce disk space usage, archive the log file and then delete it.
Line 1: The process begins for a Cluster Extension resource clxres05 using HP 3PAR Cluster Extension version number 2.00.00. Line 2: Cluster Extension internal configuration file version. Line 3 - 5: The warning message for a temporary license. Line 6: The host name where a Cluster Extension resource is being brought online in the data center (A or B). Line 7: The remote data center nodes status. Line 8-28 : Configuration details of the Cluster Extension resource before failover.
Sample log file Line 1: The process begins for a Cluster Extension resource CLX03 using HP 3PAR Cluster Extension version number 2.00.00. Line 2: Cluster Extension internal configuration file version. Line 3 - 5: The warning message for a temporary license. Line 6: The host name where a Cluster Extension resource is being brought online in the data center (A or B). Line 7: The remote data center nodes status. Line 8-28 : Configuration details of the Cluster Extension resource before failover.
Line 33: Creating CLX_application_name.online file to check resource health. Line 34: End of the Cluster Extension execution process. HP 3PAR storage system log or sysmgr log A task, identified by task IDs is generated for each operation in the 3PAR storage system. Task manager commands provides information regarding the tasks in the system. For more information, see Managing Tasks section in HP 3PAR CLI Administrator’s Manual.
When HP 3PAR Cluster Extension is integrated into the cluster software, an error message string and integer value are displayed. For the CLI, a return code is displayed. For more information, see “CLI commands” (page 71). Log files Log files assist you in locating and identifying problems. Timestamps, component IDs, and message IDs are included. • HP 3PAR Cluster Extension-related messages are logged by MSCS to the file %ClusterLog% for Windows 2003.
Failover error handling Windows Cluster automatically fails over resource groups if the system where resource group is running on becomes unavailable. This is part of the cluster functionality. Also, this means that if a problem occurs with the HP 3PAR storage system, a resource group online process will be stopped. The behavior of HP 3PAR Cluster Extension is highly configurable.
Resource disk failure when HP 3PAR Cluster Extension is online In a cluster resource group, if the Cluster Extension resource is online but dependent disk resource is failing: • Ensure that the fibre channel connectivity between the servers and the storage systems is up and running. • Verify that the disk resource is dependent to the Cluster Extension resource.
NOTE: Host persona setting varies based on the Windows operating system installed on the cluster nodes, For more information, see the HP 3PAR Implementation Guide. no_fail_wrt_on_err settings The no_fail_wrt_on_err policy apply only to synchronous Remote Copy. The no_fail_wrt_on_err group policy (default policy) allows writes to a primary Remote Copy volume group even though the link between the systems in the Remote Copy pair is broken.
promote issue), replication I/O does not start from the new primary volumes to the new secondary volumes. In case of Windows OS, CLX will continuously attempt to start the group during the monitoring interval of the CLX resource. In case of RHEL and SUSE, manual start of the volume group is necessary in order to resume the replication IO between the primary and secondary RC volume groups. Cluster Extension Autopass troubleshooting Cluster Extension uses Autopass as a framework for licensing checks.
A data center is down (SLE HA and RHCS) RHCS and SLE HA expect an acknowledgement from the fencing device before services are failed over to another node. In the event of complete site failure, including fencing devices, clusters do not automatically fail over services to surviving cluster nodes at the remote site. Manual intervention is required in this situation. For instructions on bringing a service online, see the cluster software documentation.
7 Support and other resources Contacting HP For worldwide technical support information, see the HP support website: http://www.hp.
Typographic conventions Table 4 Document conventions Convention Element Blue text: Table 4 (page 89) Cross-reference links and e-mail addresses Blue, underlined text: http://www.hp.
Glossary 3PAR Remote Copy A product that allows you to copy virtual volumes from one HP 3PAR storage system to another. CLI Command-line interface. An interface comprised of various commands which are used to control operating system responses. cluster A group of controller nodes connected via the same storage server backplane. The nodes in a cluster operate as a unified system, separate from any other clusters that may share the same service processor. CLX Cluster Extension.
Index Symbols 3PAR Cluster Extension components, 12 features, 7 integration, 7 3PAR Remote Copy support for 3PAR Cluster Extension, 12 volume groups, 13 A APPLICATION description, 66 ApplicationDir description, 66 ApplicationStartup description, 66 AutoFailbackType description, 32 B Basic Resource Health Check Interval description, 30 C CLI configuration, 7 ClusterNotifyCheckTime description, 68 UCF requirement, 39 ClusterNotifyWaitTime description, 68 UCF requirement, 39 clx3PARrun command, 71 command l
description, 65 Hyper-V Live Migration, 44, 47 I importing configuration information, 26 integration with cluster software, 7 IsAlivePollInterval description, 30 Q quorum considerations Windows, 14 R MergeCheckInterval UCF requirement, 39 Microsoft Cluster Service administration, 46, 47 changing resource names, 29 Microsoft Management Console, 30 mirrored disks automated redirection, 8 RCVolumeGroupA description, 68 RCVolumeGroupB description, 68 related documentation, 88 remote management, 30, 38 Wind
user configuration file ApplicationDir, 66 ApplicationStartup, 66 creating, 73 DC_A_Hosts, 68 DC_B_Hosts, 68 FASTFAILBACK value, 67 PostExecCheck, 69 PostExecScript, 69 PreExecScript, 69 RCVolumeGroupA, 68 RCVolumeGroupB, 68 requirements, 14 RESYNCWAIT value, 67 ResyncWaitTimeout, 69 sample, 70 V volume groups, 13 W websites HP , 88 product manuals, 88 Windows Cluster changing resource names, 29 93
