HP Serviceguard Toolkit for Integrity Virtual Servers User Guide Abstract This guide describes instructions for integrating HP-UX vPars and HP Integrity Virtual Machines with HP Serviceguard.
© Copyright 2014 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 HP Serviceguard Toolkit for Integrity Virtual Servers.........................................4 Overview................................................................................................................................4 Advantages.............................................................................................................................4 Dependencies......................................................................................................................
1 HP Serviceguard Toolkit for Integrity Virtual Servers Overview HP Serviceguard toolkit for Integrity Virtual Servers (SG IVS toolkit) is a toolkit that facilitates seamless integration of HP-UX vPars and HP Integrity Virtual Machines with HP Serviceguard. It has a set of shell scripts that aids in creating Serviceguard package for VM or vPar (that is VM or vPar-as-Serviceguard-packages) and helps to start, stop, and monitor VM or vPar on VM or vPar Hosts.
The SG IVS toolkit is a new product that inherits the functionality from the HP Integrity VM Serviceguard toolkit. This section identifies the differences between HP Integrity VM Serviceguard toolkit and SG IVS toolkit. HP Integrity VM Serviceguard toolkit SG IVS Toolkit It is part of HP Integrity VM bundle of B.04.30 and earlier It is a new product from HP and must be installed versions. separately. It supports the legacy and modular style packaging of VM. It supports only modular style packaging.
Advantages of using SG IVS Toolkit B.02.00 over HP Integrity VM Serviceguard toolkit Using SG IVS Toolkit B.02.00, you can: 6 • Configure the monitoring interval of the process instead the fixed interval. • Control the maintenance mode. • Configure the packages using Serviceguard Manager. For more information, see www.hp.com/ go/hpux-serviceguard-docs → HP Serviceguard Manager. • Configure NPIV disk monitoring (using HP Serviceguard patch PHSS_42558).
2 Supported configuration This section explains the supported configuration for VM or vPar-as-Serviceguard-packages. Figure 1 (page 7) represents a basic VM or vPar-as-a-Serviceguard-package (or, VM or vPar guest as packages) model configuration as supported with the SG IVS toolkit. Figure 1 VM/vPar as Serviceguard packages In this configuration, a Serviceguard cluster is formed by using VM or vPar Host systems as nodes in the cluster.
NOTE: • The SG IVS toolkit can only control the startup of the VM or vPar. Therefore, applications within the VM or vPar guest must be started through the defined initialization sequence, by using cmappmgr and cmappserver appropriately, or by using the native operating system commands.
way. However, the shared storage must be defined as VM or vPar guest storage devices that are exclusively used by a specific VM or vPar guest. Both standard LVM and VxVM logical volumes are active and available on only one Serviceguard node at a time. Whole disks, Shared LVM logical volumes, and VxVM logical volumes managed by the Cluster Volumes Manager (CVM) and files on a Cluster Files System can be configured to be active and available on multiple Serviceguard nodes simultaneously.
3 Installing and uninstalling HP Serviceguard toolkit for Integrity virtual servers To install the HP Serviceguard toolkit for Integrity virtual servers: 1. Run the command: # swinstall -s The “SD Install - Software Selection” user interface appears. 2. 3. 4. On the “SD Install - Software Selection” screen, mark the product, “SG-IVS-Toolkit”. To mark the product for installation, select Action > Mark For Install (m) Select Action > Install , to initiate the installation.
Table 1 Files created after SG IVS toolkit is installed. File Name Description tkit_module.sh This is a module script for the HP Serviceguard Toolkit for Integrity Virtual Servers. havtn.sh The script is used to start, stop, and validate Integrity VM or vPar. havtn.mon This script contains functions that monitor vswitchmgr process and status of a VM or vPar guest. cmdeployvpkg This command is used to create and manage Serviceguard package configurations for HP Integrity VM or vPar.
4 Using SG IVS toolkit You can use the SG IVS toolkit to create and maintain packages. Creating packages You must perform the preconfiguration steps, and then configure the VM or vPar guest into an HP Serviceguard package by using either of the following ways: • Using Serviceguard commands • Using SG IVS Toolkit commands Preconfiguration To configure VM or vPar as a Serviceguard package, perform the following preconfiguration steps: NOTE: 1.
5. On the secondary nodes: • 6. 7. On On a. b. Optional: If the vswitch (vlan), identical to the one created on the primary node is not configured in step 2, then you must configure it. the primary node, distribute the virtual machine or vPar to the secondary node the secondary nodes: Verify the virtual machine or vPar operation. Stop the virtual machine or vPar. NOTE: To configure more than two nodes for package configuration: a. Repeat from step 4–f to 7–b. b.
Completed the cluster update # NOTE: Create the VM or vPar by using the SLVM devices, for example /dev/vgsharedA/ rlvdisk0. 3. 4. Create a VM or vPar on the SLVM volume group. Add SLVM volume group as mentioned below in the cluster configuration file: HPVM_VOLUME_GROUP /dev/vgsharedA NOTE: To add the SLVM volume group, you must halt the cluster and packages if they are already running. 5. Create the package for the VM or vPar using the cmdeployvpkg command.
NOTE: If you do not run the package associated with the SLVM before starting the package associated with the virtual machine, a message similar to the following is displayed: # cmrunpkg slvmtest Unable to execute command. Dependency on the following packages not met: hpvm_vgsharedA cmrunpkg: Unable to start some package or package instances Using Serviceguard commands To create a VM or vPar package: 1.
Attributes 5. 6. Description MAINTENANCE_FLAG This attribute is used to indicate if the VM or vPar guest package can run in maintenance mode. The default value is [yes]. MONITOR_INTERVAL This is the time interval (in seconds) between the monitor cycles and the default value is 30 seconds. The toolkit monitors the health of the package every [MONITOR_INTERVAL ]seconds. service_name Name of the service that Serviceguard monitors while the package is up.
NOTE: • This script does not include information about attached I/O devices, such as tape, DVD, burner, and changer devices into the package configuration file. • Run the cmdeployvpkg command on the node where VM is running. Managing packages Running packages To start the VM/vPar guest package use the following command: # cmrunpkg NOTE: After you run the package, to enable the package failover on the configured nodes, run the following command: $ cmmodpkg -e -n ...
Halting packages To halt the package, run the following command: # cmhaltpkg This command invokes the hpvmstop or vparreset -d command on the specific VM or vPar, respectively to stop the package. Deleting packages To delete a package from the cluster, run the cmdeployvpkg command with -U option. For example: #cmdeployvpkg -P vm1 -U NOTE: HP recommends that you do not use cmdeleteconf -p to delete VM or vPar package from HP Serviceguard cluster.
When you run the cmdeployvpkg command, it: 1. Executes the VM or vPar migration command internally. 2. Executes the cmdeleteconf command on this package, after the Online VM migration is successful. 3. Runs the cmdeployvpkg -P {vm_name} -n {source-node} -n {target-node} command. NOTE: A new configuration file has minimum configuration information. Therefore, HP recommends that you run the cmdeployvpkg -P {vm_name} -n {package-nodes} with-appropriate-options. 4.
5 Configuring guest application monitoring service HP Serviceguard VM or vPar guest application monitoring service is a program or an agent configured within a VM or vPar guest that checks the status of applications configured within the same VM or vPar guest. HP supports the monitoring and control of applications within a VM or vPar guest configured as an HP Serviceguard package.
In this architecture, the Application Managers and the Application Servers communicate via SSL connections using the JSSE. The Application Manager cmappmgr initiates and maintains a connection with the application server process cmappserver in the monitored VM or vPar guest using TCP or IP port 5315 for communications. Its primary function is to launch the specified command and monitor the execution of the same by: • Sending requests to cmappserver, to execute specified processes.
on the VM or vPar guest. The number of times HP Serviceguard attempts to restart cmappmgr and the VM or vPar guest application is defined by the service_restart value, set for the cmappmgr service in the guest package. If the number of restart attempts exceeds the service_restart value, HP Serviceguard fails over the VM or vPar guest package to its configured VM or vPar Host failover node, as shown in Figure 4 (page 22).
communications (that is, server public certificate distributed to the client; client public certificate distributed to the server). The default locations for the generated key files are: • For VM or vPar Hosts: /etc • For VM or vPar guests: ◦ HP-UX: /opt/hp/cmappserver Key generation on the VM or vPar Host The following three commands must be executed on one VM or vPar Host to generate the client.private, client.public and temp.key security keys by using the keytool program.
SSL key distribution After the keys are initially created on one VM or vPar Host, all other VM or vPar Hosts must use the same client.public key. Each VM or vPar guest generates its own server.public key, therefore, a unique name for each VM or vPar guest must be used , to rename the server.public key. For example, # mv server.public server_[guest name].public. To distribute keys between VM or vPar Hosts and VM or vPar guests: a. Copy the client.
NOTE: If cmappserver is already installed, use —u instead of -i. • Add /opt/hp/cmappserver to the path. • Verify that Java 1.4.2 or 1.5 or 1.6 is installed and included in the path. For Windows VM or vPar guests (subdirectory windows): 4. • Copy the cmappserver.exe file from the VM or vPar Host to a local directory on the VM or vPar guest. • To install the required files in the directoryC:\Program Files\Hewlett-Packard\ cmappserver in the VM or vPar guest, run the cmappserver.exe file, .
5. To modify the VM or vPar guest package to start or monitor specific applications in the VM or vPar guests, do one of the following: • Manually edit the existing VM or vPar guest package configuration file: To monitor specific applications in a VM or vPar guest, a cmappmgr entry is added to the services section of the VM or vPar guest package for each monitored application.
NOTE: With the HP Serviceguard A.11.20 release, application monitoring is now supported for VM or vPar guests that use online VM or vPar guest migration. It is not supported with the HP Serviceguard A.11.19. To support online VM or vPar guest migration, all service command strings sent to the VM or vPar guest must be unique.
Hit return to confirm this entry: You have enter the following application monitor values: Name: Run command: Application timeout: Service restart: Service halt timeout: ping_mon /apps/ping_mon 10 none 300 Hit return to confirm this entry: Enter an additional application monitor? (y/n):n Checking communications with the application monitor server on : slvm1 Merging application monitor service with existing package configuration file Review and/or modify the package configuration file (optional)? (y/n): y
--[probe_count ] --[polling_interval where: • lan: It is the LAN that needs to be monitored • ip: It is the target IP address of any target, such as the gateway which is common to all package nodes. • guest_logfile: It is the complete path of the user specified guest log file. • host_logfile: It is the complete path of the user specified host log file. • log_level: It is the extent of logging on both guest and host. The default value is 0.
Attribute Name Value Example service_restart number_of_restart 5 service_halt_timeout halt_time_out 300 NPIV Disk monitoring in VM or vPar guest using the SG IVS toolkit If you are using NPIV Disk, use the application monitoring framework to configure disk monitoring to detect a failure of NPIV Disks in VM or vPar guest. When there is a failure in any one of the monitored disks, the VM or vPar package fails over to the configured alternate node.
Upgrading cmappmgr/cmappserver combination You can upgrade the VM or vPar guest monitoring functionality while upgrading HP Serviceguard, in two ways. The difference between these two ways is the number of live migrations compared to the VM or vPar guest application downtime. Process 1: This process takes advantage of how cmappserver works. If cmappserver loses communication with cmappmgr, it does not halt the applications that were being monitored, but there is loss of the monitoring function.
This process: • Disables application monitoring on all node. • Performs a rolling upgrade: the VM or vPar guest applications is stopped when the packages are stopped. • Upgrades cmappserver on all guests. • Re-enables application monitoring. To apply this process to upgrade: 1. Optional: If this is an upgrade from HP Serviceguard A.11.19 to HP Serviceguard A.11.20, ensure that all service calls to the VM or vPar guest are unique, even if it means adding extra spaces. NOTE: 2. 3.
6 Online VM guest package migration Online VM Migration (OVMM) enables a running VM guest and its applications to be moved from one VM Host to another without service interruption. SG IVS toolkit aids OVMM of VM-as-Serviceguard-package for HP-UX guests. NOTE: Online guest package migration is applicable only for VM.
For more information about limitations of Integrity VM on OVMM, see HP Integrity Virtual Machines 4.3: Installation, Configuration, and Administration > Restrictions and Limitations of Online VM Migration. You can also refer to HP-UX vPars and Integrity VM V6 Administrator Guide available at http:// www.hp.com/go/linux-serviceguard-docs.
7 Troubleshooting This section explains some of the problem scenarios that you might encounter while working with the SG IVS toolkit in a Serviceguard Cluster. Problem Scenario Possible Cause Recommended Action If the syslog/package log contains the When you start a package, the VM Check the network or DNS following error: guest OS is still not up, therefore, configurations on the VM Host or VM cmappmgr fails to communicate with guest or the SSL setup.
Problem Scenario ERROR: Failed to validate /etc/ cmcluster/scripts/tkit/vtn/ tkit_module.sh On node node-2, validation of package vpar4 failed with: Failed to set application control for vpar4: hpvmmodify: Cannot set '-j 1 because the guest has at least one DIO device. hpvmmodify: Unable to modify the vPar or VM. ERROR: Set Distributed Control: Failed to set Distributed 1 on node-2 ERROR: Failed in verify_and_distribute ERROR: Failed to validate /etc/cmcluster/scripts/tkit/vtn/tkit_module.
8 Limitations This section lists the limitations of SG IVS toolkit in a Serviceguard Cluster: • The online package conversion does not add the custom services and configurations, such as application monitor services, configured for the package created using the HPVM toolkit. • When VM is transformed to vPar or vice versa, you must delete the existing package and create a new package for that VM/vPar. • Online addition or deletion of devices are not supported with VM/vPar as an SG package configuration.
9 Support and other resources Information to collect before contacting HP Ensure that the following information is available before you contact HP: • Software product name • Hardware product model number • Operating system type and version • Applicable error message • Third-party hardware or software • Technical support registration number (if applicable) How to contact HP Use the following methods to contact HP technical support: • In the United States, see the Customer Service / Contact HP Un
HP authorized resellers For the name of the nearest HP authorized reseller, see the following sources: • In the United States, see the HP U.S. service locator website: http://www.hp.com/service_locator • In other locations, see the Contact HP worldwide website: http://welcome.hp.com/country/us/en/wwcontact.html Documentation feedback HP welcomes your feedback. To make comments and suggestions about product documentation, send a message to: docsfeedback@hp.
IMPORTANT An alert that calls attention to essential information. NOTE An alert that contains additional or supplementary information. TIP An alert that provides helpful information.
A Sample package configuration file for HP Integrity VM guest package This section provides with a sample package configuration file for HP Integrity VM guest package: # # # # # # # ********************************************************************** ****** HIGH AVAILABILITY PACKAGE CONFIGURATION FILE (template) ******* ********************************************************************** ******* Note: This file MUST be edited before it can be used.
service_cmd service_restart service_fail_fast_enabled service_halt_timeout ~ 42 /usr/sbin/cmappmgr -node slvm1 -cmappserver_timeout 10 -service /apps/ping_mon none no 300 Sample package configuration file for HP Integrity VM guest package
Glossary ADF Attribute definition file cDSF cluster Device special file DMP Dynamic Multipathing device Guest OS Operating system instance installed on the VM Integrity VM HP Integrity Virtual Machines product JRE Java Runtime Environment JSSE Java Secure Socket Extension LVM Logical Volume Manager MSE Multi Server Environment NFS Network File System OVMM Online Virtual Machine Migration Package A grouping of application services (individual HP-UX Processes) under Serviceguard Physic
Index C Configuration guest application monitoring service, 20 Serviceguard Guest Application Monitoring architecture, 20 Creating packages pre-configuration, 12 using easy deployment, 16 using Serviceguard commands, 15 I Installation uninstallation, 10 M Managing packages halting packages, 18 maintaining packages, 17 running packages, 17 S SG IVS Toolkit Advantages, 4 Comparison between HPVM toolkit and SG IVS toolkit, 5 Dependecies, 4 Overview, 4 Prerequisites, 4 SG IVS toolkit limitations, 37 Online V
