HP StorageWorks Application Recovery Manager Administrator’s Guide Manual Edition: December 2007 Manufacturing Part Number: T4395-96004 Release A.06.00 © Copyright 2007 Hewlett-Packard Development Company, L.P.
Legal Notices ©Copyright 2007 Hewlett-Packard Development Company, L.P. Hewlett-Packard Company makes no warranty of any kind with regard to this material, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be liable for errors contained herein or for incidental or consequential damages in conection with the furnishing, performance, or use of this material.
Contents 1. Introduction Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Zero Downtime Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Backup Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Instant recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents HP StorageWorks Disk Array XP Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . Microsoft Exchange Server Writer Recovery Specifics . . . . . . . . . . . . . . . . . . . . . . . . . Microsoft Exchange Server 2007 Recovery Specifics . . . . . . . . . . . . . . . . . . . . . . . . . Instant Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .B-4 Known Application Recovery Manager Issues and Workarounds . . . . . . . . . . . . . . . .B-6 Installation Related Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .B-6 Integration Related Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .B-7 Cluster Related Issues . . . . . . . . .
Contents vi
Document History Manual updates may be issued between editions to correct errors or document product changes. To ensure that you receive updated or new editions, subscribe to the appropriate product support service. See your HP sales representative for details. Table 1 Edition History Part Number Manual Edition Product T4395-96004 October 2006 Application Recovery Manager Release A.06.00 T4395-96004 July 2007 Application Recovery Manager Release A.06.00 with patch APPRMWIN_00002.
viii
Conventions The following typographical conventions are used in this manual. Table 2 Convention Italic Meaning Example Book or manual titles, and manual page names Refer to the HP StorageWorks Application Recovery Manager Installation and Licensing Guide for more information. Provides emphasis You must follow these steps. Specifies a variable that you must supply when entering a command At the prompt type: rlogin your_name where you supply your login name.
See the online Help for information about the Application Recovery Manager graphical user interface conventions.
Contact Information General Information General information about Application Recovery Manager can be found at http://www.hp.com/go/AppRM Technical Support Technical support information can be found at the HP Electronic Support Centers at http://www.itrc.hp.com Information about the latest Application Recovery Manager patches can be found at http://www.itrc.hp.com HP does not support third-party hardware and software. Contact the respective vendor for support.
xii
Application Recovery Manager Documentation Application Recovery Manager documentation comes in the form of manuals and online Help. Manuals Application Recovery Manager manuals are available in PDF format. Install the PDF files during the Application Recovery Manager setup procedure by selecting the User Interface component. Once installed, the manuals reside in the \docs directory. You can also find the manuals in PDF format at http://www.hp.com/support/manuals.
The Help Topics provide complete information on how to plan, configure, and use the Application Recovery Manager to backup and restore your applications. The context sensitive (F1) Help provides descriptions of the GUI options which are available in a particular context or dialog window. To access online help from the top-level directory on the installation DVD without installing Application Recovery Manager, unzip AppRM_help.zip and run AppRM_help.chm.
Documentation Map Backup X Concepts/Techniques X X CLI Reference Admin Guide Release Notes Installation Guide Getting Started Online Help The following table shows where to find information of different kinds. Shaded squares are a good place to look first.
xvi
In This Book The HP StorageWorks Application Recovery Manager Administrator’s Guide describes the configuration, backup, and restore tasks for the Application Recovery Manager. Audience The manual is intended for backup administrators who are responsible for configuring the application and backing up and restoring data.
Organization The manual is organized as follows: Chapter 1 “Introduction” on page 1. Chapter 2 “Configuration” on page 19. Chapter 3 “Backup” on page 35. Chapter 4 “Restore” on page 53. Chapter 5 “Troubleshooting” on page 79. Appendix A “Appendix A” on page A-1. Glossary Definition of terms used in this manual.
1 Introduction Chapter 1 1
Introduction Overview Overview A traditional backup process is based on the direct communication between the backup application and the application whose data is backed up. This backup method requires from the backup application an individual interface for each application it backs up. The number of applications on the market is constantly increasing. The necessity of handling application specific features can cause difficulties in backup, restore, and storage activities.
Introduction Overview Figure 1-1 Actors of the Traditional Backup Model Figure 1-2 Actors of the Application Recovery Manager VSS Integration Chapter 1 3
Introduction Overview Backup Model Without using the Volume Shadow Copy Service, Application Recovery Manager has to communicate with each application to be backed up individually. Application Recovery Manager introduces a unified backup and restore interface and provides the coordination among the participants of the backup and restore process.
Introduction Overview instant recovery (IR) functionality, which enables recovery of vast amount of data in minutes rather than hours. This makes ZDB and IR capabilities suitable for high-availability systems and mission-critical applications. Application Recovery Manager ZDB and IR techniques utilize mirror and snapshot technologies of disk-based arrays.
Introduction Overview Advanced features of Application Recovery Manager instant recovery: • Ability to restore on different levels: session or object • Ability to automatically start post-recovery scripts 6 Chapter 1
Introduction Concepts Concepts The Application Recovery Manager integration with MS Volume Shadow Copy Service and Virtual Disk Service provides backup and instant recovery functionality for VSS writers that are certified by Application Recovery Manager. Refer to the online Help for a general description of ZDB (snapshot and split mirror backup) and instant recovery concepts. For a complete list of supported VSS writers and providers refer to the latest support matrices at http://www.hp.
Introduction Concepts • coordinating restore session start • requesting the instant recovery Backup During the Application Recovery Manager backup, Application Recovery Manager does not interact directly with each writer, but through the VSS interface. It uses the VSSBAR agent to coordinate the backup process. The consistency of data is a responsibility of the VSS writer and not dependent on Application Recovery Manager functionality.
Introduction Concepts 8. After the information is stored to the internal database, the replicas are unpresented from the backup system. All replicas are set to read-only and are unpresented from all systems to protect the backed up data, unless you specify to mount the replica on the backup system.
Introduction Concepts Figure 1-4 Transportable VSS Backup Data consistency The filesystem backup does not guarantee application data consistency, only filesystem consistency. Application data consistency can be achieved only by using supported application writers, such as Microsoft Exchange Server 2003 writer.
Introduction Concepts Backup with the XP VSS Hardware Provider There are two configuration modes available for the backup with the XP VSS hardware provider: • VSS Compliant Mode When the XP provider is in the VSS compliant mode, the source volume (P-VOL) and its replica (S-VOL) are in simplex, unpaired state after a backup. Therefore the number of replicas (S-VOLs per a P-VOL) rotated is not limited. Restore from a backup in such a configuration is possible only by switching of the disks.
Introduction Concepts perform another instant recovery from that replica. For more information about the switch method, see “Switch of disks” on page 12. In case of XP integration, this type of restore can be performed if a backup was created with the XP provider in the VSS compliant mode.
Introduction Concepts Copy of replica data with the source volume retained With this method, a new copy from a replica is created in the same disk group in which the source volume resides. When the copy is finished, a switch of disks between the source and the new copy is performed. The old source volume is retained. You can also select to retain the replica and thus enable to perform another restore from this backup. Advantages • The old source volume is retained after restore.
Introduction Concepts Restore Using HP StorageWorks Disk Array XP This type of restore can be performed if a backup was created with the XP provider in the resync mode. With this type of restore, the XP integration agent (SSEA) synchronizes an S-VOL with its P-VOL and then splits the pair during the restore. If the option Wait for the replica to complete is selected, SSEA waits for the re-synchronization or copy process to complete (normal restore). Only then, the source volume is available.
Introduction Concepts The replicated storage group are represented as a new Exchange writer instance, Exchange Replication Service. They are backed up like original or production storage groups. Backups of the replicated storage groups (LCR and CCR copies) are possible only using VSS. Restore With Microsoft Exchange Server 2007 writer you can restore your data not only to the original location (from where the backup was performed) but also to a different location.
Introduction Prerequisites and Limitations Prerequisites and Limitations This is a list of prerequisites and limitations for the Application Recovery Manager MS Volume Shadow Copy integration. Additional limitations and recommendations that are not directly connected to the integration (such as operating system or GUI limitations) or some disk array limitations are listed in “Limitations and Recommendations” on page B-2.
Introduction Prerequisites and Limitations Limitations The integration-specific limitations are the following: • The writer’s data must be on an NTFS filesystem. • The VSS integration backup of writers which store their data on network shared volumes is not supported. • With EVA provider, if there are too many source volumes selected for backup (more than four), it may happen that the creation of snapshots takes too long and VSS fails the snapshots creation process.
Introduction Prerequisites and Limitations 18 Chapter 1
2 Configuration The Application Recovery Manager MS Volume Shadow Copy integration by itself does not require configuration steps on the Application Recovery Manager side.
Configuration Server may require additional configuration steps. For details, see the appropriate sections. You must also configure the VSS and VDS hardware providers. You may check which writers and providers are installed and registered on your system using the following Windows operating system command: • For a list of writers: VSSadmin list writers • For a list of VSS providers: VSSadmin list providers • VDS hardware providers should be present in the list of installed software.
Configuration Configuring HP StorageWorks Enterprise Virtual Array Configuring HP StorageWorks Enterprise Virtual Array Overview This section describes the configuration of the Application Recovery Manager HP StorageWorks Enterprise Virtual Array (EVA) integration. It also provides information on the ZDB database. Prerequisites • HP storage components and licenses: — HP StorageWorks Virtual Controller Software (VCS or XCS) and Command View (CV) EVA.
Configuration Configuring HP StorageWorks Enterprise Virtual Array See the “Appendix B” on page B-1 for general Application Recovery Manager and integration-specific limitations. ZDB Database - SMISDB ZDB database for EVA integration is referred to as SMISDB. It keeps the information about: • Management systems on which Command View EVA runs configured in the Application Recovery Manager cell. For each system, the following is stored: — Hostname as recognized in the IP network.
Configuration Configuring HP StorageWorks Enterprise Virtual Array [-port ] [-namespace ] [-user ] | -list [] | -check []} Adding a Management System To add a management system on which CV EVA runs to the Application Recovery Manager cell and to define the login information for the relevant CV EVA, run: omnidbsmis -ompasswd -add [-ssl] [-port ] [-namespace ] [-user ] [-passwd ] [-check [
Configuration Configuring HP StorageWorks Enterprise Virtual Array Removing a Management System To remove a management system from a list of systems on which CV EVAs are running, with its login and port number information, from the SMISDB, run: omnidbsmis -ompasswd -remove [-port ] [-namespace ] [-user ] Used together with the -port port option, the command will only remove the entries for the specified port.
Configuration Configuring HP StorageWorks Enterprise Virtual Array • If standby and failed management systems have the same hostname, no action is needed. • If standby and failed management systems have different hostnames, remove the failed system from the Application Recovery Manager configuration, and then add the new management system. IMPORTANT It is recommended to use the default port number (5988 for non-SSL and 5989 for SSL-based SMI-S provider connection settings).
Configuration Configuring HP StorageWorks Disk Array XP Configuring HP StorageWorks Disk Array XP This chapter describes configuration of the Application Recovery Manager HP StorageWorks Disk Array XP (XP) integration. Prerequisites • XP components and licenses: — RAID Manager library on the application and backup systems. See the RAID Manager library documentation for installation instructions. RAID Manager Library is firmware-dependant.
Configuration Configuring HP StorageWorks Disk Array XP Figure 2-1 Selecting the HP StorageWorks XP Agent For licensing and remote installation instructions, see the HP StorageWorks Application Recovery Manager Installation and Licensing Guide. • Make sure the same operating system version is installed on the application and backup systems. • Connect the application and backup systems to the same XP array. • Assign LUNs to the respective ports.
Configuration Configuring HP StorageWorks Disk Array XP • Maximum three replicas (S-VOLs) can be created for a source volume (P-VOL) if the XP provider is in resync mode. For the general Application Recovery Manager and additional XP integration limitations, see “Appendix B” on page B-1.
Configuration VSS/VDS Provider Configuration for Backup and Instant Recovery VSS/VDS Provider Configuration for Backup and Instant Recovery Configuring the VSS Hardware Provider for EVA To be able to perform instant recovery, you must configure the VSS and VDS provider. Perform the configuration task before you run the backup for the first time. Refer to the hardware provider documentation for details on how to configure the provider.
Configuration VSS/VDS Provider Configuration for Backup and Instant Recovery NOTE Changing the modes between backup sessions can impact the restore. See “HP StorageWorks Disk Array XP Considerations” on page 55 for details.
Configuration Microsoft Exchange Server Writer Specific Configuration Microsoft Exchange Server Writer Specific Configuration A backup of the Microsoft Exchange database can be considered successful only if the consistency check of the replicated datafiles succeeds. It is recommended that the database stores and transaction logs reside on different source volumes.
Configuration Microsoft Exchange Server Writer Specific Configuration Also in case where the application and backup systems are the same, ensure that the VSS/VDS hardware provider is configured for the same Command View on both EVAs. Otherwise, rotation and deletion of old replicas from other EVA cannot be performed.
Configuration Configuration Check Configuration Check With instant recovery, it is possible to selectively restore separate writer’s components if they reside on separate source volumes. Instant recovery of separate components also requires that the volumes with components data should not contain any other data. The configuration check detects, whether there are more than one component on the volume and whether there is any other data besides the component’s data.
Configuration Configuration Check IMPORTANT If the configuration check is disabled for an instant recovery, you will lose the data that does not belong to a component, but resides on the same volume as the component. Disable configuration check only if instant recovery cannot be performed with an enabled configuration check and only if you are sure that this will not result in a loss of data during the instant recovery.
3 Backup Chapter 3 35
Backup Overview Overview To run backups and restores of the VSS writers and filesystems, you need to configure the Application Recovery Manager backup specifications. To configure the backup, perform the following steps: Configuration Steps 1. Configure disk arrays needed for the backup. 2.
Backup Overview Snapshot Replica This type of replica is produced using snapshot technology, provided by HP StorageWorks Enterprise Virtual Array. In the case of a snapshot replica, no synchronization is required before creating the fixed point-in-time copies because a snapshot is a logical copy of the pointers to the data rather than the data itself. Snapshot replicas are considered to be created almost instantaneously and are immediately available for use.
Backup Overview Limitations • When a cloning process of a source volume is in progress, another snapshot (any type) of that source volume cannot be created. • Only one type of target volume per source volume can exist on an EVA disk array at the same time. For example, the creation of a snapclone may fail if a snapshot or vsnap for the same source volume exists on the array. Such replicas should be deleted first using the Application Recovery Manager command omnidbvss.
Backup Creating Backup Specifications Using GUI Creating Backup Specifications Using GUI The procedure below shows how to back up MS VSS objects using the Application Recovery Manager GUI. Some writers have specific limitations. For writers specific limitations, see: • For Microsoft Exchange Server specifics, see “Microsoft Exchange Server Writer Backup Specifics” on page 45. To create a new backup specification, proceed as follows: 1.
Backup Creating Backup Specifications Using GUI Figure 3-1 Selecting VSS Transportable Backup 5. Specify the following options: • The name of the application system. When backing up cluster-aware writers (such as SQL Server via the MSDE Writer or SqlServerWriter, or Exchange Server in the CCR environment), specify the virtual server name given in the particular writer resource group as the application system. • For transportable backup, the name of the backup system.
Backup Creating Backup Specifications Using GUI • If you clear Track replica for instant recovery, you can specify the replica type: Default The replica type created depends on the provider configuration. Mirror/Clone (Plex) A target volume, independent from its source volume is created: — With EVA provider, snapclone is created. — With XP provider, the replica created depends on the provider mode (clone in case of VSS compliant mode, or mirror in case of resync mode).
Backup Creating Backup Specifications Using GUI Figure 3-2 VSS Transportable Backup Options 6. Select the backup objects you want to back up.
Backup Creating Backup Specifications Using GUI Figure 3-3 Selecting Backup Objects (Exchange 2007 LCR Copy and SQL Writer) NOTE Do not select too many objects with EVA provider because they might reside on more than four source volumes. If the objects reside on more than four source volumes, create several backup specifications where objects do not reside on more than four source volumes. If a writer requires all of its components to be backed up, lower-level items are automatically selected.
Backup Creating Backup Specifications Using GUI The Filesystem backup displays all mounted disks. If another disk is mounted under a disk, the parent disk is displayed twice. The first item is the parent disk (for example c:) and the second one is the container for the mountpoint (for example c:\mnt\1). To select a mounted disk, select the mountpoint.
Backup Creating Backup Specifications Using GUI You have now completed the creation of a MS Volume Shadow Copy Writers backup specification. 9. You can review the newly-created and saved backup specification in the Backup context, under the specified group of backup specifications. 10. You can run backup using one of the following methods: • Schedule the backup of an existing MS Volume Shadow Copy Writers backup specification using the Application Recovery Manager Scheduler.
Backup Creating Backup Specifications Using GUI • If transaction logs reside on different source volume than stores, it enables you to perform a rollforward recovery when only the stores are lost. • Create one backup specification for each storage group and schedule them in a row. Writer Stabilization After each Microsoft Exchange Writer backup, Application Recovery Manager by default waits 10 minutes for Microsoft Exchange Writer to stabilize.
Backup Creating Backup Specifications Using GUI Microsoft Exchange Server 2007 Writer Specifics In an LCR or CCR environment, the replicated storage groups are represented as a new instance of Exchange writer, Exchange Replication Service. The replicated storage groups are backed up in the same way as original production storage groups. You can select any of these storage groups (original or replicated) for a backup. However, you cannot select both in the same backup specification.
Backup Creating Backup Specifications Using GUI To specify the cluster node from where you want to perform backup of any instance residing on this node, right-click an Exchange writer and select the node in the additional options dialog box under Back up any available instance from node. See Figure 3-4 on page 44. When backing up Replication Service instance, it may happen that it cannot be backed up due to the following reasons: • The selected node is not available.
Backup Scheduling the Backup Scheduling the Backup For more detailed information on scheduling, refer to the online Help index keyword “scheduled backups”. To schedule a backup specification, perform the following steps in the Application Recovery Manager GUI: 1. In the HP StorageWorks Application Recovery Manager GUI, switch to the Backup context. 2. In the Scoping Pane, expand Backup, then Backup Specifications. Click MS Volume Shadow Copy Writers.
Backup Scheduling the Backup 6. Click OK to return to the Schedule property page. 7. Click Apply to save the changes.
Backup Running an Interactive Backup Running an Interactive Backup An interactive backup can be started using the Application Recovery Manager GUI by following these steps: 1. In the HP StorageWorks Application Recovery Manager GUI, switch to the Backup context. 2. In the Scoping Pane, expand Backup; then expand the Backup Specifications and the MS Volume Shadow Copy Writers items. 3. Right-click the backup specification you want to run, and then select Start Backup from the pop-up menu.
Backup Running an Interactive Backup 52 Chapter 3
4 Restore Chapter 4 53
Restore Overview Overview Instant recovery restores data directly from a replica to source volumes. All data in the replica is restored (regardless of selections during backup). For instant recovery concepts, see the online Help. HP StorageWorks EVA Considerations • After instant recovery, the replica used in the session can become the source volumes (physical location of the original application data changes). This depends on the restore options you select in the GUI.
Restore Overview HP StorageWorks Disk Array XP Considerations • Restore depends on the XP hardware provider mode which was used during backup. If the VSS compliant mode was used, you can restore your data only using VDS. If the resync mode was used, you can restore your data only using HP StorageWorks Disk Array XP.
Restore Overview The following considerations apply to restore using HP StorageWorks XP only: • Only three first-level mirrors can be used for instant recovery. Six additional (cascading) copies are not supported. • You cannot start another instant recovery using the same disk on the application system at the same time. A session can be started only after the preceding session using the same disk on the application system finishes synchronization.
Restore Microsoft Exchange Server Writer Recovery Specifics Microsoft Exchange Server Writer Recovery Specifics This section provides specific information on the instant recovery point-in-time, and rollforward restore of the Microsoft Exchange Server Writer. With Microsoft Exchange Server 2003, you can restore a whole storage group or individual stores only to the original location.
Restore Microsoft Exchange Server Writer Recovery Specifics To perform a point-in-time restore of Microsoft Exchange Server writer, select the whole storage group. The transaction logs will be replaced with the backed up logs. To perform a rollforward restore, select only the database stores and original location. The existing logs will be applied to the restored databases. However, note that rollforward restore will not be possible if the point-in-time restore of the same session was performed before.
Restore Microsoft Exchange Server Writer Recovery Specifics — Restore to a non-Exchange location and creating Recovery Storage Group (RSG) Data can be restored to a non-Exchange location and after that a Recovery Storage Group can be created for the databases. Exchange Server recognizes the Recovery Storage Group created by Application Recovery Manager, which is called AppRM RSG.
Restore Microsoft Exchange Server Writer Recovery Specifics IMPORTANT If you restore to a non-Exchange location and create RSG, check that Recovery Storage Group does not already exist on this location. If Recovery Storage Group already exists, it will be deleted by Application Recovery Manager. All data stored in the existing Recovery Storage Group will be lost.
Restore Microsoft Exchange Server Writer Recovery Specifics • With the XP VSS provider, if a backup of an LCR or CCR storage group copy is performed in the resync mode, restore of such backup to the original location (original database or Exchange Information Store) by default fails. The reason is that the source volumes, on which the database copy (or Exchange Replication Service) resides, are in the suspended mirror relationship with the replicas, which should be restored to the original database.
Restore Instant Recovery Instant Recovery Refer to the online Help for general information on instant recovery. You can selectively restore separate writer’s components only if they reside on separate target volumes. Instant recovery of separate components also requires that the target volumes with components data should not contain any other data. If you do not select all writer’s components residing on the target volumes to be restored, the restore fails.
Restore Instant Recovery Instant Recovery Procedure 1. In case of Exchange Server 2003 writer, manually dismount the databases to be recovered. 2. In the HP StorageWorks Application Recovery Manager GUI, switch to the Instant Recovery context. 3. In the Scoping Pane, expand MS Volume Shadow Copy Writers under Restore Objects, expand the writer you want to restore and click the backup session from which you want to perform the restore.
Restore Instant Recovery Figure 4-2 Selecting Exchange 2007 Writers Components for Instant Recovery to a Different Location 4. In the Source property page, specify writers and/or components for recovery. Select the configuration check mode. For more information, see “Configuration Check” on page 33. SqlServer Writer: Optionally, select No recovery. This option enables you to apply transaction logs on Microsoft SQL Server after instant recovery.
Restore Instant Recovery You can perform point-in-time or rollforward restore of Microsoft Exchange Server writer. See “Microsoft Exchange Server Writer Recovery Specifics” on page 57 for information. Exchange Server 2007 Writer: If you restore an LCR or CCR copy to the original location, note that restore will be performed to the original database (Exchange Information Store) and not to the database copy (Exchange Replication Service). To perform restore to a different location: a.
Restore Instant Recovery restored to this recovery group. Select the target server first and then from the Original drop-down list, select the stores to be restored. Optionally, select the mount location for the storage group if you want to be different than default, which is C:\Omni_Mnt. Figure 4-3 Restore to Different Location Options (Exchange Writer 2007) 5. Optionally, select the VDS or disk array options under the Options tab. For details, press F1.
Restore Instant Recovery Figure 4-4 Selecting Instant Recovery Options (EVA Integration) Chapter 4 67
Restore Instant Recovery Figure 4-5 Selecting Instant Recovery Options (XP Integration) 6. Click Restore. 7. If you restored Microsoft Exchange Server writers, re-mount the database stores. Also, if you restored your data to AppRM RSG, mount the stores in the RSG before you recover any store from the RSG. 8. If you restored an LCR or CCR copy to the original database, perform additional steps.
Restore Instant Recovery Clusters To perform an instant recovery in the cluster environment, use the above instant recovery procedure. Use virtual hostnames. See “Instant Recovery in a Cluster” on page A-19 for details on performing an Instant Recovery in cluster configurations.
Restore Instant Recovery 70 Chapter 4
5 Internal Database Maintenance Chapter 5 71
Internal Database Maintenance Overview Overview The IDB is an internal database, located on the Cell Manager that keeps information regarding which data is backed up, on which disks it resides, the result of backup and recovery sessions, and which disk arrays are configured. What is the IDB used for? The information stored in the IDB enables the following: • Fast and convenient restore • You are able to browse sessions to be restored. • Backup management • You can verify the result of backup sessions.
Internal Database Maintenance Overview When backing up the IDB, you can choose the backup location, the number of copies to make, and limit the amount of space that the IDB backups can occupy. By setting the upper limits on disk usage or number of kept backups, you will probably not run out of disk space.
Internal Database Maintenance Backing up the IDB Backing up the IDB You must perform Internal Database backups regularly; it is best to perform them daily. Once the IDB backup is performed regularly, the most important preparation for recovery in case of a disaster is done. The IDB recovery is essential for restore of other backed up data if the Cell Manager crashes. It is recommended that you keep several backups of the internal database. Keep at least two backup versions. Use the dbtool.
Internal Database Maintenance Backing up the IDB • Run the following command: at : /EVERY: /INTERACTIVE perl.exe \bin\dbtool.pl -backup Note that the scheduling service must be running if you use the at command.
Internal Database Maintenance Restoring the IDB Restoring the IDB To restore the internal database and configuration, use the dbtool command. The command syntax and examples are provided in the HP StorageWorks Application Recovery Manager Command Line Interface Reference. NOTE In the steps below, it is assumed that the perl.exe command is in the search path list for commands (defined by environment variable PATH). If not, specify the full pathname. 1. Change the directory: cd \bin 2.
Internal Database Maintenance Purging the IDB Purging the IDB By default, the information about sessions is kept in the database for 365 days. You can set the interval after which the sessions are purged by setting the KeepDiskonlySessions variable in the global options file. See the online Help index: “global options file, modifying”.
Internal Database Maintenance Purging the IDB 78 Chapter 5
6 Troubleshooting This chapter lists problems you might encounter when using the Application Recovery Manager Microsoft Volume Shadow Copy integration with ZDB integration.
Troubleshooting For general Application Recovery Manager troubleshooting information, see the online Help.
Troubleshooting Before You Begin Before You Begin ✓ Ensure that the latest official Application Recovery Manager patches are installed. See the online Help index: “patches” on how to verify this. ✓ See the HP StorageWorks Application Recovery Manager Release Notes for general Application Recovery Manager limitations, as well as recognized issues and workarounds. ✓ See http://www.hp.com/support/manuals for an up-to-date list of supported versions, platforms, and other information.
Troubleshooting Checks and Verifications Checks and Verifications ✓ On the application and backup systems, examine system errors reported in: Windows: \log\debug.log ✓ With the XP integration, ensure that RAID Manager Library is correctly installed on both the application and backup systems. Check if the libsvrrm.dll file exists in the directory.
Troubleshooting Common Problems Common Problems Problem Backup or instant recovery aborts due to VDS problems. A backup or instant recovery session aborts with the following error. Failed to load VDS service. This error can appear as a result of a VDS service crash. Actions 1. Stop the Virtual Disk Service (VDS): • Check that Virtual Disk Service is started, stop it using the command net stop vds or via the Control Panel.
Troubleshooting Backup Problems Backup Problems Problem Microsoft Exchange Writer backup failed When you start a backup of a Microsoft Exchange Server 2003 Writer, the following error is displayed: [MAJOR]: Writer ‘Microsoft Exchange Writer’ failed to prepare files for backup. The reason may be the failure of the previous backup and, due to a Microsoft Exchange Server 2003 issue, the Exchange Writer could not perform a proper cleanup. Action Restart the Information Store.
Troubleshooting Backup Problems Problem Snapshots (volume shadow copies) creation fails with EVA provider During a VSS backup, the following error is displayed: [Critical] It was not possible to create Volume Shadow Copies. If there are too many source volumes selected for backup (more than four) it may happen that the creation of snapshots takes too long and VSS fails the snapshots creation process.
Troubleshooting Backup Problems • By Windows system: Figure 6-1 Error Reported by Registry it means that the Windows registry is out of space. If there are too many entries in the Windows system registry, the registry runs out of space and consequently backup fails. Although Application Recovery Manager does not write anything to registry directly, the registry contains entries about all volumes and disks ever presented on the system recorded by the volume storage driver.
Troubleshooting Backup Problems • In a cluster environment on Disk Array XP, when the backup system is a virtual server, or during local backup when the application system (which is the same as the backup system) is a virtual server. In this case, after a failover, replicas on the other (failed-over) system cannot be properly removed from the database. Action To remove the false entry from the VSSDB: 1. In the error messages, find the ID of the session which could not remove the target volume. 2.
Troubleshooting Restore Problems Restore Problems Problem Instant recovery of the Microsoft Exchange Server Writer fails. This problem may occur if the Microsoft Exchange Writer is not in the stable state. Check this by running vssadmin list writers from the command prompt. Action Bring the Exchange Server Writer to a stable state by restarting the Microsoft Exchange Information Store. Problem After restore, the system restart error is displayed.
A Appendix A Appendix A A-1
Appendix A Alternate Paths Support Alternate Paths Support For systems with multiple host adapters and connections to a disk array, the alternate paths solution, configured on the backup system, performs dynamic load balancing and monitors each path to ensure that I/O completes its transactions. If a path between a disk array and a server fails, alternate path software automatically switches to an alternate path, removing the failed path from I/O rotation without data loss.
Appendix A Cluster Configurations Cluster Configurations Application Recovery Manager VSS integrations supports: • Microsoft Cluster Server On XP, if the application system is in a cluster, the backup system must be outside this cluster (it may run in a different cluster, or may not be part of a cluster at all). The reason for this limitation is that during backup, the filesystem/database structure is active on the backup system and would prevent an activation during failover.
Appendix A Cluster Configurations • On the backup system on local disks: Application Recovery Manager VSS integration software component, Application Recovery Manager ZDB agent. Figure A-1 Client on the Application System in a Cluster Cell Manager and Client on the Application System in a Cluster Scenarios • Application or Application Recovery Manager failover during backup: session is restarted automatically.
Appendix A Cluster Configurations Install: • On the application system on all cluster nodes on local disks: application binaries, Application Recovery Manager VSS integration software component, Application Recovery Manager ZDB agent. • On the application system cluster shared disk: application database. Note that this shared disk must be a disk array replicated disk. • On the application system cluster shared disk: Cell Manager.
Appendix A Cluster Configurations Figure A-2 Cell Manager and Client on the Application System in a Cluster Client on the Application System in a Cluster, Cell Manager on the Backup System in a Cluster Scenarios • Application failover during backup: session fails and must be restarted manually. • Application failover before backup: session completes successfully. • Cell Manager failover during backup: failed session is automatically restarted, provided the option Restart backup of all objects is set.
Appendix A Cluster Configurations Install: • On the application system on all cluster nodes on local disks: application binaries, Application Recovery Manager VSS integration software component, Application Recovery Manager ZDB agent. • On the application system cluster shared disk: application database. Note that this shared disk must be a disk array replicated disk. • On the backup system cluster shared disk: Cell Manager.
Appendix A Cluster Configurations Figure A-3 Client on the Application System in a Cluster, Cell Manager on the Backup System in a Cluster Cell Manager on the Backup System in a Cluster Scenarios • Cell Manager failover during backup: session is automatically restarted, provided the option Restart backup of all objects is set. • Cell Manager failover before backup: session completes successfully.
Appendix A Cluster Configurations • On the backup system on all cluster nodes on local disks: Application Recovery Manager VSS integration software component, Application Recovery Manager ZDB agent. Figure A-4 Cell Manager on the Backup System in a Cluster Client on the Application System in a Cluster, Cell Manager in a Cluster Cell Manager is installed in a cluster on any system that is not a backup or application system.
Appendix A Cluster Configurations • Cell Manager failover before backup: session completes successfully. Install: • On the application system on all cluster nodes on local disks: application binaries, Application Recovery Manager VSS integration software component, Application Recovery Manager ZDB agent. • On the application system cluster shared disk: application database. Note that this shared disk must be a disk array replicated disk. • On any system cluster shared disk: Cell Manager.
Appendix A Cluster Configurations Figure A-5 Client on the Application System in a Cluster, Cell Manager in a Cluster Appendix A A-11
Appendix A Cluster Configurations A-12 Appendix A
B Appendix B Appendix B B-1
Appendix B Limitations and Recommendations Limitations and Recommendations This appendix contains general limitations and recommendations for using Application Recovery Manager. Size Limitations Number of Sessions in the Database There can be up to 1,000,000 sessions in the database. At the most, 2,000 backup sessions can be run in one day. Number of Backups Scheduled at One Time The maximum total number of backup sessions running in parallel is 60. The default value is set to 5.
Appendix B Limitations and Recommendations Recovery Manager Installation and Licensing Guide. Limitations on Disk Array Integrations HP StorageWorks Disk Array XP Limitations • Only BC configurations are supported, CA configurations are not supported. • Routine maintenance, including (but not limited to) hot-swapping any field replaceable components like, disk array controllers, FC switches, and/or online firmware upgrades during backups are not supported.
Appendix B Limitations and Recommendations • The GUI on Windows can display a maximum 64000 items in a tree view. • The glossary is not available in online Help on 64-bit versions of Windows. • Application Recovery Manager does not support hostnames with non-ASCII characters. • The name of a backup specification should not exceed 64 characters. • When changing the omnirc file, it is recommended to restart the Application Recovery Manager services on the system.
Appendix B Limitations and Recommendations Regular Maintenance of the VSS Registry Using Microsoft’s Registry Management Tool Microsoft maintains a record of mounts in the registry. This results in registry growth over time, which leads to Volume Shadow Copy import problems. For details, see “Volume shadow copies cannot be imported due to the low space in registry and consequently backup fails.” on page 85. It is recommended to consider using Microsoft’s registry management tool.
Appendix B Known Application Recovery Manager Issues and Workarounds Known Application Recovery Manager Issues and Workarounds Installation Related Issues • If there is not enough free disk space on the destination partition, the push installation of the User Interface fails and reports: [Warning] Installation of User Interface FAILED! Application Recovery Manager Software package installation failed Workaround: The disk space on the destination partition has to be at least 40 MB even if the package is ins
Appendix B Known Application Recovery Manager Issues and Workarounds Integration Related Issues Disk Array Integrations • When running a ZDB on EVA, the process can hang under the following condition: The source disks involved in the backup session have control distributed between 2 or more EVA SMI-S Providers. The processes must be manually stopped. Workaround: Disks involved in an EVA ZDB must be controlled through one HP StorageWorks SMI-S EVA provider only.
Appendix B Known Application Recovery Manager Issues and Workarounds Other Known Issues • Command omnistat -session [session ID] -detail sometimes displays Restore started and Backup started incorrectly. This can result in both parameters appearing to be identical • The following applications are not recommended to be installed together with Application Recovery Manager on the same system: WebQoS, CyberSitter 2000, NEC E-border AUTOSOCKS.
Appendix B Known Non-Application Recovery Manager Issues and Workarounds Known Non-Application Recovery Manager Issues and Workarounds Non-Application Recovery Manager Issues Related to Installation • After installation of Application Recovery Manager A.06.00, Windows may report that some application is not installed or that a reinstall is required. The reason is an error in the Microsoft Installer upgrade procedure. Please read the Microsoft Knowledge Base article Q324906 to solve the problem.
Appendix B Known Non-Application Recovery Manager Issues and Workarounds password are stored in registry as a plain text. • During a VSS Transportable Backup the following error is reported by the VSSBAR on the backup server: Import failed. If the backup server is inspected after the failed session, the snapshots are actually visible as new disks in the Device Manager, as well as in the Disk Manager.
Appendix B Known Non-Application Recovery Manager Issues and Workarounds • A snapshot backup of an Exchange Server 2003 database fails, and event ID 9607 is logged. See Microsoft Knowledge Base article ID 910250 for information on how to resolve this problem. • After a restore session, the following error is reported in the event log: Faulting application vds.exe, version 5.2.3790.2738, faulting module vds.exe, version 5.2.3790.2738, fault address 0x000213c8.
Appendix B Known Non-Application Recovery Manager Issues and Workarounds B-12 Appendix B
Glossary access rights See user rights. application agent A component needed on a client to back up or restore online database integrations. Application Recovery Manager Event Log A central repository of all Application Recovery Manager related notifications. By default, all notifications are sent to the Event Log. The events are logged in the \log\server\Ob2Event Log.txt file on the Cell Manager.
Glossary • Type: backup object type. backup owner Each backup object in the IDB has an owner. The default owner of a backup is the user who starts the backup session. backup session A process that creates a copy of source volumes data on a disk array. The activities are specified in a backup specification.. See also source volume and backup specification. backup set A complete set of integration objects associated with a backup.
Glossary the EVA firmware. See also replica, source volume, and snapshot. Boolean operators The Boolean operators for the full text search functionality of the online Help system are AND, OR, NOT, and NEAR. Used when searching, they enable you to define your query precisely by creating a relationship between search terms. If no operator is specified in a multi-word search, AND is used by default. For example, the query consistency check is equivalent to consistency AND check.
Glossary client backup A backup of all writers and filesystems on a client. Filesystems mounted on the client after the backup specification was created are not automatically detected client or client system Any system configured with any Application Recovery Manager functionality and configured in a cell. cluster-aware application It is an application that supports the cluster Application Programming Interface.
Glossary disks. The Command View EVA software runs on the HP OpenView Storage Management Appliance, and is accessed by a Web browser. See also HP StorageWorks EVA SMI– S Agent. CRS The Cell Request Server process (service), which runs on the Application Recovery Manager Cell Manager, and starts and controls the backup and restore sessions. The service is started as soon as Application Recovery Manager is installed on the Cell Manager.
Glossary makes computer names available to client resolvers querying for name resolution across the Internet. domain controller A server in a network that is responsible for user security and verifying passwords within a group of other servers. Event Logs Files in which Windows logs all events, such as the starting or stopping of services and the logging on and off of users.
Glossary up files. Typically, these files occupy about 20% of the IDB, if filenames are stored. full backup A backup in which all selected objects are backed up, whether or not they have been recently modified. full database backup A backup of all data in a database, not only the data that has been changed after the last backup of the database. A full database backup does not depend on any other backup. global options file A file that allows you to customize Application Recovery Manager.
Glossary (BC) configurations, or can be used as standalone entities. See also BC (HP StorageWorks Disk Array XP specific term) and replica. HP StorageWorks EVA SMI-S Agent An Application Recovery Manager software module that executes all tasks required for the HP StorageWorks Enterprise Virtual Array integration. With the EVA SMI-S Agent, the control over the array is established through HP StorageWorks SMI-S EVA provider, which directs communication between incoming requests and CV EVA.
Glossary Installation Server A computer system that holds a repository of the Application Recovery Manager software packages for a specific architecture. The Installation Server is used for remote installation of Application Recovery Manager clients. instant recovery (ZDB specific term) A process in which a replica, produced by a ZDB-to-disk session, is used to restore the contents of the source volumes to their states at the time at which the replica was created.
Glossary without an operator. This implies that no operator personnel is present to work with the backup application. See also Cluster Continuous Replication and Exchange Replication Service. Local Continuous Replication (Microsoft Exchange Server specific term) Local continuous replication (LCR) is a single-server solution that creates and maintains an exact copy (LCR copy) of a storage group. An LCR copy is located on the same server as the original storage group.
Glossary Magic Packet See Wake ONLAN. mailbox (Microsoft Exchange Server specific term) The location to which e-mail is delivered, which is set up by the administrator for each user. If a set of personal folders is designated as the email delivery location, e-mail is routed from the mailbox to this location. Mailbox Store (Microsoft Exchange Server specific term) A part of the Information Store that maintains information about user mailboxes. A mailbox store consists of a binary rich-text .
Glossary (0, 1 or 2), used to indicate a first-level mirror. See also first-level mirror. object See backup object Object ID (Windows specific term) The object IDs (OIDs) enable access to NTFS 5 files no matter where in the system the files reside. Application Recovery Manager treats the OIDs as alternate streams of the files. online backup A backup performed while a database application remains available for use.
Glossary • COPY - The mirrored pair is currently re-synchronizing. Data is transferred from one disk to the other. The disks do not contain the same data. • PAIR - The mirrored pair is completely synchronized and both disks (the primary volume and the mirrored volume) contain identical data. • SUSPENDED - The link between the mirrored disks is suspended. That means that both disks are accessed and updated independently.
Glossary • private, that is, visible (and accessible for restore) only to the owner of the backup and administrators RAID Redundant Array of Inexpensive Disks. RDBMS Relational Database Management System. RDS The Raima Database Server process (service) runs on the Application Recovery Manager Cell Manager and manages the IDB. The process is started when Application Recovery Manager is installed on the Cell Manager.
Glossary (P-VOL) and its replica (S-VOL) are in the suspended mirror relationship. The maximum number of replicas (S-VOLs per a P-VOL) rotated is three provided that MU range is 0-2 or 0, 1, 2. Restore from a backup in such a configuration is possible only by re-synchronization of an S-VOL with its P-VOL. See also VSS compliant mode, source volume, PVOL, replica, S-VOL, MU number, and replica set rotation. RSM The Application Recovery Manager Restore Session Manager controls the restore session.
Glossary software (for example, system providers) or hardware (local disks, disk arrays). See also shadow copy. shadow copy set (MS VSS specific term) A collection of shadow copies created at the same point in time. See also shadow copy and replica set. shared disks A Windows disk on another system that has been made available to other users on the network. SMBF The Session Messages Binary Files (SMBF) part of the IDB stores session messages generated during backup, restore sessions.
Glossary A replica creation technique, in which a pre-configured set of target volumes (a mirror) is kept synchronized with a set of source volumes until the time at which a replica of the contents of the source volumes is required. Then, the synchronization is stopped (the mirror is split) and a split mirror replica of the source volumes at the time of the split remains in the target volumes. See also split mirror.
Glossary checkpoints and the quorum resource recovery log, which contains the most recent cluster database information. program counter, a user-mode stack, a kernel-mode stack, and a set of register values. Several threads can run at a time within one process. system volume/disk/partition A volume/disk/partition containing operating system files.
Glossary archiving of transaction logs should be enabled to prevent you from losing the transaction log files that are created after the last IDB backup and are necessary for IDB recovery. transportable snapshot (MS VSS specific term) A shadow copy that is created on the application system and can be presented to the backup system where a backup can be performed. See also Microsoft Volume Shadow Copy Service (VSS). unattended operation See lights-out operation.
Glossary StorageWorks Enterprise Virtual Array snapshot functionality. See also source volume and target volume. virtual server A virtual machine in a cluster environment defined in a domain by a network IP name and address. Its address is cached by the cluster software and mapped to the cluster node that is currently running the virtual server resources. This way all requests for a particular virtual server are cached by a specific cluster node.
Glossary WINS server A system running Windows Internet Name Service software that resolves Windows networking computer names to IP addresses. writer (MS VSS specific term) A process that initiates change of data on the original volume. Writers are typically applications or system services that write persistent information on a volume. Writers also participate in the shadow copy synchronization process by assuring data consistency. ZDB See zero downtime backup (ZDB). the instant recovery process.
Glossary G-22
Index Index A advantages, 7 alternate paths, A-2 architecture, 4, 9, 10 B backup , 35–51 architecture, 4 creating backup specifications, 39 flow, 8 local, 9, 40 overview, 37 problems, 84 running, 51 scheduling, 49 starting, 51 transportable, 10, 40, 42 troubleshooting, 84 types, 5 backup flow, 8 backup objects, 43 backup specifications creating, 39 backup types, 5 C checking configuration, 33 VSS integration, 40 checks and verifications, 82 clones on EVA, 37 cluster configurations, A-3 Cluster Continuous
Index backup, 37 P prerequisites EVA integration, 21 VSS integration, 16 XP, 26 problems, 79–88 backup, 84 instant recovery, 88 restore, 88 R replica creation and reuse EVA, 37 restore, 53–69 concepts, 11 instant recovery procedure, 63 problems, 88 procedure, 63 troubleshooting, 88 restore flow, 11 running backups, 51 S scheduling backups, 49 setting login information for EVA, 22 SMISDB, 22 snapclones on EVA, 10, 37 snapshot types on EVA, 37 clones, 37 snapclones, 10, 37 standard snapshots, 10, 37 vsnaps, 1
