Symantec NetBackup™ Deduplication Guide Release 7.
Symantec NetBackup™ Deduplication Guide The software described in this book is furnished under a license agreement and may be used only in accordance with the terms of the agreement. Documentation version 7.0 PN: 20654102 Legal Notice Copyright © 2009 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo, Veritas, and NetBackup are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries.
Symantec Corporation 350 Ellis Street Mountain View, CA 94043 http://www.symantec.
Technical Support Symantec Technical Support maintains support centers globally. Technical Support’s primary role is to respond to specific queries about product features and functionality. The Technical Support group also creates content for our online Knowledge Base. The Technical Support group works collaboratively with the other functional areas within Symantec to answer your questions in a timely fashion.
■ Version and patch level ■ Network topology ■ Router, gateway, and IP address information ■ Problem description: ■ Error messages and log files ■ Troubleshooting that was performed before contacting Symantec ■ Recent software configuration changes and network changes Licensing and registration If your Symantec product requires registration or a license key, access our technical support Web page at the following URL: www.symantec.
Maintenance agreement resources If you want to contact Symantec regarding an existing maintenance agreement, please contact the maintenance agreement administration team for your region as follows: Asia-Pacific and Japan customercare_apac@symantec.com Europe, Middle-East, and Africa semea@symantec.com North America and Latin America supportsolutions@symantec.
Contents Technical Support ............................................................................................... 4 Chapter 1 Introducing NetBackup deduplication ............................ 11 About NetBackup deduplication ...................................................... 11 About NetBackup deduplication options ..................................... 12 How deduplication works ........................................................ 14 Chapter 2 Planning your deployment ...................
Contents Chapter 3 Provisioning the storage .................................................... 37 About provisioning the storage ....................................................... About deduplication storage requirements ....................................... About deduplication storage capacity .............................................. About the deduplication storage paths ............................................. Chapter 4 Installing deduplication ..................................
Contents Deleting a load balancing server configuration file ....................... Removing a load balancing server ............................................. Viewing deduplication storage servers ....................................... Viewing deduplication storage server attributes .......................... Resetting the deduplication registry .......................................... Managing NetBackup Deduplication Engine credentials ......................
Contents Volume state changes to DOWN when volume is unmounted ..................................................................... Errors, delayed response, hangs ................................................ Viewing disk errors and events ....................................................... Deduplication event codes and messages .......................................... Chapter 8 Disaster recovery ............................................................... 103 Preparing for disaster ...
Chapter 1 Introducing NetBackup deduplication This chapter includes the following topics: ■ About NetBackup deduplication About NetBackup deduplication The proprietary Symantec PureDisk deduplication technology powers NetBackup integrated deduplication. Symantec packaged PureDisk into modular components. The components plug-in to NetBackup through the NetBackup OpenStorage framework.
Introducing NetBackup deduplication About NetBackup deduplication About NetBackup deduplication options Deduplication everywhere lets you choose at which point in the backup process to perform deduplication. NetBackup can manage your deduplication wherever you implement it in the backup stream. Figure 1-1 shows the options for deduplication. Table 1-1 describes the options for deduplication.
Introducing NetBackup deduplication About NetBackup deduplication Table 1-1 NetBackup deduplication options Type Description NetBackup Client Deduplication Option With NetBackup client-side deduplication, clients deduplicate their backup data and then send it directly to the storage destination. A media server does not deduplicate the data. NetBackup Client Deduplication is a useful deduplication solution if a client host has unused CPU cycles or if the load balancing servers are overloaded.
Introducing NetBackup deduplication About NetBackup deduplication Table 1-1 NetBackup deduplication options (continued) Type Description PureDisk deduplication NetBackup PureDisk is a deduplication solution for bandwidth-optimized backups of data in remote offices. You can use PureDisk to reduce the amount of backup data that is stored in a datacenter by NetBackup. You use PureDisk interfaces to install, configure, and manage the PureDisk servers, storage pools, and client backups.
Introducing NetBackup deduplication About NetBackup deduplication The following list describes how NetBackup derives unique segments to store: ■ The deduplication engine breaks file 1 into segments A, B, C, D, and E. ■ The deduplication engine breaks file 2 into segments A, B, Q, D, and L. ■ The deduplication engine stores file segments A, B, C, D, and E from file 1 and file segments Q, and L from file 2. The deduplication engine does not store file segments A, B, and D from file 2.
Introducing NetBackup deduplication About NetBackup deduplication
Chapter 2 Planning your deployment This chapter includes the following topics: ■ Planning your deduplication deployment ■ About the deduplication storage type ■ About the NetBackup Media Server Deduplication Option ■ About NetBackup Client Deduplication ■ About NetBackup Deduplication Engine credentials ■ About the network interface for deduplication ■ About firewalls and the deduplication hosts ■ About scaling deduplication ■ About compression and encryption ■ About optimized duplicati
Planning your deployment Planning your deduplication deployment Table 2-1 Deployment overview Deployment task Where to find the information Determine the storage type See “About the deduplication storage type” on page 19. Determine which type of deduplication to use See “About the NetBackup Media Server Deduplication Option” on page 19. See “About NetBackup Client Deduplication” on page 24. Determine the requirements for deduplication hosts See “About deduplication servers” on page 21.
Planning your deployment About the deduplication storage type Table 2-1 Deployment overview (continued) Deployment task Where to find the information Replace a PDDO host or migrate from PDDO to NetBackup deduplication See “Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host” on page 33. See “Migrating from PureDisk to the NetBackup Media Server Deduplication option” on page 34.
Planning your deployment About the NetBackup Media Server Deduplication Option For a backup, the NetBackup client software creates the image of backed up files as for a normal backup. The client sends the backup image to the deduplication storage server, which deduplicates the data. The deduplication storage server writes the data to disk. See “About deduplication servers” on page 21. The NetBackup Media Server Deduplication Option is integrated into NetBackup.
Planning your deployment About the NetBackup Media Server Deduplication Option Figure 2-1 NetBackup media server deduplication PureDisk NetBackup client NetBackup client NetBackup client Load balancing servers Plug-in NetBackup client PureDisk PureDisk Plug-in Plug-in PureDisk Plug-in NetBackup Deduplication Engine PureDisk deduplication pool Media server deduplication pool Deduplication storage server More detailed information is available.
Planning your deployment About the NetBackup Media Server Deduplication Option Table 2-2 Host NetBackup deduplication servers Description Deduplication storage One host functions as the storage server for a deduplication node; server that host must be a NetBackup media server. The storage server does the following: ■ Writes the data to and reads data from the disk storage. ■ Manages that storage. The storage server also deduplicates data.
Planning your deployment About the NetBackup Media Server Deduplication Option About deduplication server requirements All hosts that are used for deduplication must be NetBackup 7.0 or later. Hosts include the master server, the media servers, and the clients. The computer’s CPU and memory constrain how many jobs can run concurrently. Table 2-3 Deduplication server minimum requirements Hardware Requirement CPU CPU speed is the most important factor for performance. Minimum CPU speed should be 2.
Planning your deployment About NetBackup Client Deduplication the installation fails. To upgrade a NetBackup media server that hosts a PDDO agent, you must first remove the PDDO agent. See the NetBackup PureDisk Deduplication Option (PDDO) Guide. Deduplication within each media server deduplication node is supported; global deduplication between nodes is not supported. About NetBackup Client Deduplication With normal deduplication, the client sends the full backup data stream to the media server.
Planning your deployment About NetBackup Client Deduplication Figure 2-2 NetBackup client deduplication NetBackup deduplication client NetBackup deduplication client NetBackup deduplication client PureDisk PureDisk PureDisk Plug-in Plug-in Plug-in PureDisk Plug-in NetBackup PureDisk Deduplication Engine PureDisk deduplication pool Deduplication storage server Media server deduplication pool More detailed information is available. See “Deduplication client components” on page 114.
Planning your deployment About NetBackup Deduplication Engine credentials About client deduplication limitations Client deduplication does not support multiple copies per job. For the jobs that specify multiple copies, the backup images are sent to the storage server and may be deduplicated there. Client deduplication does not support encryption. Client deduplication is not tolerant of high latency network connections.
Planning your deployment About the network interface for deduplication About the network interface for deduplication If the server host has more than one network interface, by default the host operating system determines which network interface to use. However, you can specify which interface NetBackup should use for the deduplication traffic. To use a specific interface, enter that interface name when you configure the deduplication storage server.
Planning your deployment About compression and encryption ■ For the initial full backups of your clients, use the deduplication storage server. For subsequent backups, use load balancing servers. Do not expect the deduplication storage server to be the media server that is used for restores to that client. If a media server deduplicates a client backup, your restore settings may require that the media server also be used for restores to that client. ■ Enable client-side deduplication gradually.
Planning your deployment About optimized duplication of deduplicated data Figure 2-3 Optimized duplication copy example Deduplication node B (destination) Deduplication node A (source) PureDisk Plug-in PureDisk PureDisk Plug-in Plug-in Host B Host C PureDisk Plug-in PureDisk Plug-in Host E NetBackup Deduplication Engine NetBackup Deduplication Engine Host A Host D Host D is configured as a load balancing server for node A The following are the requirements for optimized duplication: ■
Planning your deployment About optimized duplication of deduplicated data If the destination is a PureDisk Deduplication Pool: Configure a media server that accesses the PureDisk Storage Pool Authority host as a load balancing server for the source storage server. To use more than one media server for the optimized copy operation, each additional one must be common between them .If you select more than one, NetBackup balances the optimized copy job load among them.
Planning your deployment About optimized duplication of deduplicated data If you use your destination storage unit to back up clients, you can create a different storage unit for those jobs. In that storage unit, select all of the hosts in that node that you want to use for deduplication. If you select the common server from the destination node in the source node storage unit, NetBackup uses i5 for deduplication.
Planning your deployment About deduplication performance About deduplication performance Many factors affect performance, especially the server hardware and the network capacity. Table 2-4 provides information about performance during backup jobs for a deduplication storage server. The deduplication storage server conforms to the minimum host requirements. Client deduplication or load balancing servers are not used. See “About deduplication server requirements” on page 23.
Planning your deployment Replacing the PureDisk Deduplication Option with Media Server Deduplication on the same host Table 2-4 Deduplication job load performance for a deduplication storage server (continued) When Description Clean up periods Clean up is when the NetBackup Deduplication Engine performs maintenance such as deleting expired backup image data segments. NetBackup maintains the same number of concurrent backup jobs as during normal operation.
Planning your deployment Migrating from PureDisk to the NetBackup Media Server Deduplication option Table 2-5 Replacing a PDDO host with a media server deduplication host Task Procedure Ensure that no activity occurs on the host Deactivate all backup policies that use the host. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I..
Planning your deployment Migrating from another storage type to deduplication Table 2-6 To migrate from PureDisk to NetBackup deduplication Task Procedure Install and configure NetBackup See the NetBackup Installation Guide for UNIX and Linux. Configure NetBackup deduplication See “Configuring deduplication” on page 47. Redirect your backup jobs Redirect your backup jobs to the NetBackup media server deduplication pool. See the NetBackup Installation Guide for Windows.
Planning your deployment Migrating from another storage type to deduplication Table 2-7 Migrating to NetBackup deduplication (continued) Task Procedure Redirect your backup jobs Redirect your backup jobs to the media server deduplication pool storage unit. To do so, change the backup policy storage destination to the storage unit for the deduplication pool. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. See the NetBackup Administrator's Guide for Windows, Volume I.
Chapter 3 Provisioning the storage This chapter includes the following topics: ■ About provisioning the storage ■ About deduplication storage requirements ■ About deduplication storage capacity ■ About the deduplication storage paths About provisioning the storage How to provision the storage is beyond the scope of the NetBackup documentation. For help, consult the storage vendor's documentation. What you choose as your storage destination affects how you provision the storage.
Provisioning the storage About deduplication storage capacity NetBackup requires exclusive use of the disk resources. If the storage is used for purposes other than backups, NetBackup cannot manage disk pool capacity or manage storage lifecycle policies correctly. Therefore, NetBackup must be the only entity that uses the storage. See “About the deduplication storage paths” on page 38.
Provisioning the storage About the deduplication storage paths The path names must use ASCII characters only. The NetBackup Media Server Deduplication Option does not support NFS mounted file systems. Caution: You cannot change the paths after NetBackup configures the deduplication storage server. Therefore, carefully decide during the planning phase where and how you want the deduplicated backup data stored.
Provisioning the storage About the deduplication storage paths
Chapter 4 Installing deduplication This chapter includes the following topics: ■ About installing deduplication ■ About the deduplication license key ■ Licensing NetBackup deduplication ■ Replacing the deduplication storage server host computer ■ Uninstalling media server deduplication About installing deduplication The NetBackup deduplication components are installed by default on the supported host systems. However, you must enter a license key to enable deduplication.
Installing deduplication About the deduplication license key About the deduplication license key NetBackup deduplication is licensed separately from base NetBackup. The NetBackup Deduplication Option license key enables both NetBackup Media Server Deduplication and NetBackup Client Deduplication. The license is a front-end capacity license. It is based on the size of the data to be backed up, not on the size of the deduplicated data.
Installing deduplication Replacing the deduplication storage server host computer For the new host, you must use the same host name. The new host cannot host a deduplication storage server already. Reasons to replace the host include a lease swap or perhaps the current deduplication storage server host does not meet your performance requirements. Warning: The new host must use the same byte order as the old host. If it does not, you cannot access the deduplicated data.
Installing deduplication Uninstalling media server deduplication Table 4-1 How to replace the deduplication storage server host (continued) Task Procedure Get a configuration file template If you did not save a storage server configuration file before the failure, get a template configuration file. See “Getting the storage server configuration” on page 68. Edit the configuration file See “Editing a storage server configuration file” on page 68.
Installing deduplication Uninstalling media server deduplication NetBackup media server deduplication components and the deduplication storage. The host remains a NetBackup media server. This process assumes that all backup images that reside on the deduplication disk storage have expired. Caution: If you uninstall deduplication and valid NetBackup images reside on the deduplication storage, data loss may occur.
Installing deduplication Uninstalling media server deduplication Table 4-2 Disable media server deduplication (continued) On UNIX and Linux, remove On UNIX and Linux systems, remove the following files: files ■ etc/pdregistry.cfg ■ opt/pdag ■ opt/pdshared ■ The hostname.cfg file The file resides in the /usr/openv/lib/ost-plugins directory. The servername is the name of the configured deduplication storage server.
Chapter 5 Configuring deduplication This chapter includes the following topics: ■ Configuring deduplication ■ Configuring a deduplication storage server ■ About deduplication pools ■ Configuring a deduplication pool ■ Configuring a deduplication storage unit ■ Enabling client deduplication ■ Configuring backups ■ Configuring optimized deduplication copy ■ Configuring optimized deduplication copy behavior ■ Adding a load balancing server ■ About the deduplication configuration file ■
Configuring deduplication Configuring a deduplication storage server See the NetBackup Administrator's Guide for Windows, Volume I. See the NetBackup Administrator's Guide for UNIX and Linux, Volume I. Table 5-1 Deduplication configuration tasks Task Procedure Configure a deduplication storage server See “Configuring a deduplication storage server” on page 48. Configure a disk pool See “About deduplication pools” on page 49. See “Configuring a deduplication pool” on page 50.
Configuring deduplication About deduplication pools ■ The type of storage server. For NetBackup media server deduplication, select Media Server Deduplication Pool for the type of disk storage. For a PureDisk deduplication pool, select PureDisk Deduplication Pool for the type of disk storage. ■ The credentials for the deduplication engine. See “About NetBackup Deduplication Engine credentials” on page 26. ■ The storage paths. See “About the deduplication storage paths” on page 38.
Configuring deduplication Configuring a deduplication pool Configuring a deduplication pool When you create a deduplication storage server, you can launch the Disk Pool Configuration Wizard after NetBackup creates the deduplication storage server. If you created the disk pool already, you do not have to follow these instructions. When you configure a disk pool for deduplication, you specify the following: ■ The type of disk pool (PureDisk).
Configuring deduplication Configuring a deduplication pool Table 5-2 Media server deduplication pool properties (continued) Property Description Disk volume For a media server deduplication pool, all disk storage is exposed as a single volume. PureDiskVolume is a virtual name for the storage that is contained within the directories you specified for the storage path and the database path. Available space The amount of space available in the disk pool.
Configuring deduplication Configuring a deduplication storage unit Table 5-2 Media server deduplication pool properties (continued) Property Description Limit I/O streams Select to limit the number of read and write streams (that is, jobs) for each volume in the disk pool. A job may read backup images or write backup images. By default, there is no limit. If you select this property, also configure the number of streams to allow per volume.
Configuring deduplication Configuring a deduplication storage unit Deduplication storage unit properties The following are the configuration options for a PureDisk disk pool storage unit. Table 5-3 Property Deduplication storage unit properties Description Storage unit name A unique name for the new storage unit. The name can describe the type of storage. The storage unit name is the name used to specify a storage unit for policies and schedules. The storage unit name cannot be changed after creation.
Configuring deduplication Configuring a deduplication storage unit Table 5-3 Deduplication storage unit properties (continued) Property Description Maximum concurrent jobs The Maximumconcurrentjobs setting specifies the maximum number of jobs that NetBackup can send to a disk storage unit at one time. (Default: one job. The job count can range from 0 to 256.) This setting corresponds to the Maximum concurrent write drives setting for a Media Manager storage unit.
Configuring deduplication Configuring a deduplication storage unit For example, assume that you have 100 important clients, 500 regular clients, and four media servers. You can use two media servers to back up your most important clients and two media servers to back up you regular clients. The following example describes how to configure a favorable client-to-server ratio: ■ Configure the media servers for NetBackup deduplication and configure the storage. ■ Configure a disk pool.
Configuring deduplication Enabling client deduplication Enabling client deduplication To enable Client Deduplication, set an attribute in the NetBackup master server Client Attributes host properties. To specify the clients that deduplicate backups 1 In the NetBackup Administration Console, expand NetBackup Management > Host Properties > Master Servers. 2 In the details pane, select the master server. 3 On the Actions menu, select Properties.
Configuring deduplication Configuring optimized deduplication copy NetBackup deduplicates the client data that it sends to a deduplication storage unit. Configuring optimized deduplication copy You can configure optimized copy of deduplicated backups. See “About optimized duplication of deduplicated data” on page 28. To configure optimized duplication of deduplicated data 1 Ensure that all requirements are met. See “Optimized deduplication copy requirements” on page 28.
Configuring deduplication Adding a load balancing server ■ Optimized duplication failover. By default, if an optimized duplication job fails, NetBackup does not run the job again. ■ Number of optimized duplication attempts. You can specify the number of times NetBackup retries an optimized deduplication job before it fails the jobs. ■ Storage lifecycle policy retries. If the optimized deduplication job is configured in a storage lifecycle policy, NetBackup retries the job three times.
Configuring deduplication Adding a load balancing server To add a load balancing server 1 In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server 2 Select the deduplication storage server. 3 On the Edit, select Change. 4 In the Change Storage Server dialog box, select the Media Servers tab (Windows) or the Login Credentials tab (UNIX). 5 Select the media server or servers that you want to use as a load balancing server.
Configuring deduplication About the deduplication configuration file About the deduplication configuration file On each host that deduplicates data, a pd.conf file contains configuration settings for the deduplication. You can edit the file to configure advanced settings for that host. If you change the pd.conf file on a host, it changes the settings for that host only. If you want the same settings for all of the hosts that deduplicate data, you must change the pd.conf file on all of the hosts.
Configuring deduplication Editing the deduplication configuration file 4 Save and close the file. 5 Restart the NetBackup Remote Manager and Monitor Service (nbrmms) on the host. pd.conf file settings Table 5-4 describes the deduplication settings that you can configure. Table 5-4 pd.conf file values Setting Default value Possible values DEBUGLOG C:\pdplugin.log Any path (Windows) tmp/pdplugin.log (UNIX) LOGLEVEL 0 0 through 10 Action Writes the log information to the specified file.
Configuring deduplication Editing the deduplication configuration file Setting Table 5-4 pd.conf file values (continued) Default value Possible values Action This keyword is reserved for internal use. PDALIGN Note: Do not change this setting unless directed to by a Symantec Technical Support representative. OPTDUP_TIMEOUT N/A The value expressed Specifies the number of minutes before the in minutes optimized duplication times out. Indicated in minutes.
Configuring deduplication Reconfiguring the deduplication storage server and storage paths Table 5-4 pd.conf file values (continued) Default value Possible values Action OPTDUP_ENCRYPTION 1 1 (on) or 0 (off) Determines if the data to replicate is encrypted before it is sent out over the network. DONT_SEGMENT_TYPES N/A Any file extension Allows a list of file name extensions to be specified. Files in the backup stream that have these extensions are given a single segment if smaller than 16MB.
Configuring deduplication Reconfiguring the deduplication storage server and storage paths Warning: Deleting valid backup images may cause data loss. Table 5-5 Disable media server deduplication Task Procedure Ensure that no deduplication Deactivate all backup policies that use deduplication storage. activity occurs See the NetBackup Administrator's Guide for UNIX and Linux, Volume I See the NetBackup Administrator's Guide for Windows, Volume I..
Chapter 6 Managing deduplication This chapter includes the following topics: ■ Managing deduplication servers ■ Managing NetBackup Deduplication Engine credentials ■ Managing deduplication disk pools ■ Monitoring deduplication activity ■ Monitoring deduplication storage capacity and usage ■ Deleting backup images ■ Disabling deduplication for a client ■ About maintenance processing Managing deduplication servers After you configure deduplication, you can perform various tasks to manage ded
Managing deduplication Managing deduplication servers See “Viewing deduplication storage servers” on page 72. See “Viewing deduplication storage server attributes” on page 72. See “Resetting the deduplication registry” on page 73. Changing deduplication storage server properties You can change the retention period and logging level for the NetBackup Deduplication Manager.
Managing deduplication Managing deduplication servers 6 Change the value. 7 Click OK. Deleting a deduplication storage server If you delete a deduplication storage server, NetBackup disables the deduplication functionality and the storage server functionality on that media server. NetBackup does not delete the media server from your configuration. To delete the media server, use the NetBackup nbemmcmd command.
Managing deduplication Managing deduplication servers To determine deduplication storage server state ◆ Run the following command on the NetBackup master server or a deduplication storage server: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -liststs -storage_server server_name -stype PureDisk –U Windows: install_path\NetBackup\bin\admincmd\nbdevquery -liststs -storage_server server_name -stype PureDisk –U The following is example output: Storage Server : bit Storage Server Type : PureDisk Stor
Managing deduplication Managing deduplication servers To edit the storage server configuration 1 If you did not save a storage server configuration file, get a storage server configuration file. See “Getting the storage server configuration” on page 68. If you get the configuration of a storage server that is unavailable because of a disaster, NetBackup returns a template configuration file. The following is an example of a template configuration file: V6.5.5 V6.5.5 V6.5.5 V6.5.5 V6.5.5 V6.5.5 V6.5.
Managing deduplication Managing deduplication servers Setting the storage server configuration You can set the storage server configuration (that is, configure the storage server) by importing the configuration from a file. Setting the configuration can help you with recovery of your environment. The file should be file of your configuration that you saved. See “Preparing for disaster” on page 103. Alternative, the file may be an edited configuration file.
Managing deduplication Managing deduplication servers After you remove the load balancing server, restart the NetBackup Enterprise Media Manager service. The NetBackup disk polling service may try to use the removed server to query for disk status. Because the server is no longer a load balancing server, it cannot query the disk storage. Consequently, NetBackup may mark the disk volume as DOWN. When the EMM service restarts, it chooses a different deduplication server to monitor the disk storage.
Managing deduplication Managing deduplication servers 4 In the Change Storage Server dialog box, select the Media Servers tab (Windows) or the Login Credentials tab (UNIX). 5 Clear the check box of the media server you want to remove. 6 Click OK. Viewing deduplication storage servers Use the NetBackup Administration Console to view a list of deduplication storage servers already configured.
Managing deduplication Managing deduplication servers To view deduplication storage server attributes ◆ Run the following command on the NetBackup master server or a deduplication storage server: UNIX: /usr/openv/netbackup/bin/admincmd/nbdevquery -liststs -storage_server server_name -stype PureDisk –U Windows: install_path\NetBackup\bin\admincmd\nbdevquery -liststs -storage_server server_name -stype PureDisk –U The following is example output: Storage Server : bit Storage Server Type : PureDisk Storage
Managing deduplication Managing NetBackup Deduplication Engine credentials To reset the deduplication registry file on UNIX and Linux ◆ Enter the following command on the storage server to reset the deduplication registry file: cp -f /usr/openv/pdde/pdconfigure/cfg/userconfigs/pdregistry.cfg /etc/pdregistry.
Managing deduplication Managing NetBackup Deduplication Engine credentials Add the same credentials that you already use in your environment. Another procedure exists to add a load balancing server to your configuration. See “Adding a load balancing server” on page 58.
Managing deduplication Managing deduplication disk pools Determining which media servers have deduplication credentials You can determine which media servers have credentials configured for the NetBackup Deduplication Engine. The servers with credentials are load balancing servers. To determine if NetBackup Deduplication Engine credentials exist 1 In the NetBackup Administration Console, expand Media and Device Management > Credentials > Storage Server.
Managing deduplication Managing deduplication disk pools 3 On the Edit menu, select Change. 4 In the Change Disk Pool dialog box, change properties. See “Media server deduplication pool properties” on page 50. Changing the deduplication pool state Disk pool state is UP or DOWN. To change the state to DOWN, the disk pool must not be busy. If backup jobs are assigned to the disk pool, the state change fails. Cancel the backup jobs or wait until the jobs complete.
Managing deduplication Managing deduplication disk pools To change the state to DOWN, the disk pool in which the volume resides must not be busy. If backup jobs are assigned to the disk pool, the state change fails. Cancel the backup jobs or wait until the jobs complete. To change the deduplication disk volume state 1 Determine the name of the disk volume.
Managing deduplication Managing deduplication disk pools To determine disk pool state 1 In the NetBackup Administration Console, expand Media and Device Management > Device Monitor. 2 Select the Disk Pools tab. 3 The state is displayed in the Status column. Determining the deduplication disk volume state Use the NetBackup nbdevquery command to determine the state of the volume in a deduplication disk pool. The command shows the properties and attributes of the PureDiskVolume.
Managing deduplication Monitoring deduplication activity To view disk pools ◆ In the NetBackup Administration Console, expand Media and Device Management > Devices > Disk Pools. Monitoring deduplication activity You can monitor deduplication activity. See “Monitoring the deduplication rates” on page 80. See “Viewing disk reports” on page 81. See “Monitoring deduplication processes” on page 83. See “Monitoring deduplication logs” on page 83.
Managing deduplication Monitoring deduplication activity applicable) to a load balancing server for deduplication. Typically, deduplication rates for that information are zero or very low.
Managing deduplication Monitoring deduplication activity Table 6-1 Disk reports Report Description Images on Disk The Images on Disk report generates the image list present on the disk storage units that are connected to the media server. The report is a subset of the Images on Media report; it shows only disk-specific columns. The report provides a summary of the storage unit contents. If a disk becomes bad or if a media server crashes, this report can let you know what data is lost.
Managing deduplication Monitoring deduplication activity Monitoring deduplication processes The following are the deduplication processes about which NetBackup reports: ■ NetBackup Deduplication Engine in the Administration Activity Monitor Services tab on Windows systems. On UNIX, the NetBackup Deduplication Engine appears as spoold on the Daemons tab. ■ NetBackup Deduplication Manager in the Activity Monitor Services tab on Windows systems.
Managing deduplication Monitoring deduplication activity ■ A log file for each connection to the engine is stored in a directory structure. The following describes the pathname to a log file for a connection: IP address/application/TaskName/FirstDigitofSessionID/sessionID-current_time_in_seconds.log For example, the following is an example of a crcontrol connection log pathname on a UNIX system: /storage_path/log/spoold/127.0.0.1/crcontrol/Control/2/2916742631-1257956402.
Managing deduplication Monitoring deduplication activity 3 Save the file. 4 Run the following command: install_path\Veritas\pdde\pddb\bin\pg_ctl reload -D dbpath\databases\pddb\data If the command output does not include server signaled, use Windows Computer Management to restart the PostgreSQL Server 8.3 service. PureDisk plug-in log You can configure the location and name of the log file and the logging level. To do so, edit the DEBUGLOG entry and the LOGLEVEL in the pd.conf file.
Managing deduplication Monitoring deduplication activity Windows: install_path\Veritas\NetBackup\logs\nbostpxy. About VxUL logs Some NetBackup commands or processes write messages to their own log files. Other processes use Veritas unified log (VxUL) files. VxUL uses a standardized name and file format for log files. An originator ID (OID) identifies the process that writes the log messages. Table 6-2 shows the NetBackup logs for disk-related activity.
Managing deduplication Monitoring deduplication storage capacity and usage To view and manage VxUL log files, you must use NetBackup log commands. For information about how to use and manage logs on NetBackup servers, see the NetBackup Troubleshooting Guide. Monitoring deduplication storage capacity and usage Several options exist to monitor your deduplication storage capacity and usage. See “About deduplication capacity and usage reporting” on page 87.
Managing deduplication Monitoring deduplication storage capacity and usage Table 6-3 Capacity and usage reporting (continued) Option Description Storage Server window The Storage Server window of the NetBackup Administration Console displays real-time values. To display the window, expand Media and Device Management > Credentials > Storage Servers. Change Storage Server dialog box The Change Storage Server dialog box Properties tab displays storage capacity and usage.
Managing deduplication Monitoring deduplication storage capacity and usage About deduplication container files The deduplication storage implementation allocates container files to hold backup data. Deleted segments can leave free space in containers files, but the container file sizes do not change. Segments are deleted from containers when backup images expire and the NetBackup Deduplication Manager performs clean-up.
Managing deduplication Deleting backup images Deleting backup images Image deletion may be time consuming. Therefore, if you delete images manually, Symantec recommends the following approach. See “Data removal process” on page 118. To delete backup images manually 1 Expire all of the images by using the bpexpdate command and the -notimmediate option. The -notimmediate option prevents bpexpdate from calling the nbdelete command, which deletes the image.
Managing deduplication About maintenance processing ■ NetBackup Deduplication Engine queue processing. Operations that require database updates accumulate in a transaction queue. Twice a day, the NetBackup Deduplication Manager directs the deduplication engine to process the queue as one batch. The schedule is frequency-based. By default, queue processing occurs every 12 hours, 20 minutes past the hour. Queue processing is CPU-bound.
Managing deduplication About maintenance processing
Chapter 7 Troubleshooting This chapter includes the following topics: ■ Troubleshooting installation issues ■ Troubleshooting configuration issues ■ Troubleshooting operational issues ■ Viewing disk errors and events ■ Deduplication event codes and messages Troubleshooting installation issues The following sections may help you troubleshoot configuration issues. See “Installation on SUSE Linux fails” on page 93.
Troubleshooting Troubleshooting configuration issues PDDE_packages/suse/NB_FID2740_LinuxS_x86_20090713_6.6.0.27209/ linuxS_x86/catalog/anb/NB.file_trans failed. Aborting ... Verify that your system is at patch level 2 or later, as follows: cat /etc/SuSE-release SUSE Linux Enterprise Server 10 (x86_64) VERSION = 10 PATCHLEVEL = 2 Troubleshooting configuration issues The following sections may help you troubleshoot configuration issues. See “Cannot configure deduplication storage server” on page 94.
Troubleshooting Troubleshooting operational issues To prohibit reverse host name lookup by using the bpsetconfig command ◆ Enter the following command on each media server that you use for deduplication: echo REVERSE_NAME_LOOKUP = PROHIBITED | bpsetconfig -h host_name The bpsetconfig command resides in the following directories: UNIX: /usr/openv/netbackup/bin/admincmd Windows: install_path\Veritas\NetBackup\bin\admincmd The disk pool wizard does not display a volume The Disk Pool Configuration Wizard do
Troubleshooting Troubleshooting operational issues Backup jobs fail If backup jobs fail with an Error 800: Disk Volume is Down message, examine the disk error logs to determine why the volume was marked DOWN. If the storage server is busy with jobs, it may not respond to master server disk polling requests in a timely manner. A busy load balancing server also may cause this error. Consequently, the query times out and the master server marks the volume DOWN.
Troubleshooting Viewing disk errors and events To change the volume state to UP ◆ Mount the file system After a brief period of time, the volume state changes to UP. No further action is required. Errors, delayed response, hangs Insufficient memory or inadequate host capabilities may cause multiple errors, delayed response, and hangs. See “About deduplication server requirements” on page 23.
Troubleshooting Deduplication event codes and messages Deduplication event codes and messages Table 7-1 Event # Event Severity NetBackup Message example Severity 1000 2 Error Operation configload/reload failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. 1001 2 Error Operation configload/reload failed on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. 1002 4 Warning The open file limit exceeded in server PureDisk:server1.symantecs.
Troubleshooting Deduplication event codes and messages Deduplication event codes and messages (continued) Table 7-1 Event # Event Severity NetBackup Message example Severity 1012 2 Error A request for agent task was denied on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. 1014 1 Critical Task session start request on server PureDisk:server1.symantecs.orgon host server1.symantecs.org got an unexpected error.
Troubleshooting Deduplication event codes and messages Deduplication event codes and messages (continued) Table 7-1 Event # Event Severity NetBackup Message example Severity 1028 1 Critical Service Symantec DeduplicationEngine abort on server PureDisk:server1.symantecs.orgon host server1.symantecs.org. Please check the server log for the probable cause of this error.The application has caught an unexpected signal.
Troubleshooting Deduplication event codes and messages Deduplication event codes and messages (continued) Table 7-1 Event # Event Severity NetBackup Message example Severity 1044 multiple multiple The usage of one or more system resources has exceeded a warning level. Operations will or could be suspended. Please take action immediately to remedy this situation. 1040 2 Error Operation Database recovery failed on server PureDisk:server1.symantecs.org on host server1.symantecs.org.
Troubleshooting Deduplication event codes and messages
Chapter 8 Disaster recovery This chapter includes the following topics: ■ Preparing for disaster ■ Moving images off-site ■ Recovering from a deduplication storage server disk failure ■ Recovering from a permanent deduplication storage server failure ■ Recovering the storage server after NetBackup catalog recovery Preparing for disaster Symantec recommends that you get and save the storage server configuration. Getting and saving the configuration can help you with recovery of your environment.
Disaster recovery Recovering from a deduplication storage server disk failure Recovering from a deduplication storage server disk failure If recovery mechanisms do not protect the disk on which the NetBackup software resides, the deduplication storage server configuration is lost if the disk fails. This topic describes how to recover from a system disk or program disk failure where the disk was not backed up.
Disaster recovery Recovering from a permanent deduplication storage server failure Table 8-1 Process to recover from media server disk failure (continued) Delete the credentials on media servers If you have load balancing servers, delete the NetBackup Deduplication Engine credentials on those media servers. See “Deleting credentials from a load balancing server” on page 75. Add the credentials to the storage server Add the NetBackup Deduplication Engine credentials to the storage server.
Disaster recovery Recovering from a permanent deduplication storage server failure Warning: The new host must use the same byte order as the old host. If it does not, you cannot access the deduplicated data. (In computing, endianness describes the byte order that represents data: big endian and little endian. For example, Sun SPARC processors and Intel processors use different byte orders. Therefore, you cannot replace a Solaris SPARC host with a host that has an Intel processor.
Disaster recovery Recovering the storage server after NetBackup catalog recovery Table 8-2 Process to recover from a permanent media server failure (continued) Task Procedure Configure the storage server Configure the storage server by uploading the configuration from the file you edited. If you saved a configuration file before the storage server failure, use that file. See “Setting the storage server configuration” on page 70.
Disaster recovery Recovering the storage server after NetBackup catalog recovery
Chapter Deduplication architecture This chapter includes the following topics: ■ Deduplication server components ■ Media server deduplication process ■ Deduplication client components ■ Deduplication client backup process ■ About deduplication fingerprinting ■ Data removal process Deduplication server components Figure 9-1 is a diagram of the storage server components.
Deduplication architecture Deduplication server components Figure 9-1 Storage server deduplication components NetBackup Deduplication Engine PureDisk plug-in NetBackup Deduplication Manager Catalog plugin Storage path Database application Database path Data path Catalog metadata path Control flow Table 9-1 describes the components.
Deduplication architecture Media server deduplication process Table 9-1 NetBackup deduplication components (continued) Component Description NetBackup Deduplication Manager The deduplication manager is one of the storage server core components. The deduplication manager maintains the configuration and controls internal processes, optimized duplication, security, and event escalation. The deduplication manager binary file name is spad.
Deduplication architecture Media server deduplication process Figure 9-2 Deduplication to a media server deduplication pool Master server Deduplication storage server bpbrm nbjm bptm PureDisk plug-in NetBackup Deduplication Engine bpbkar Client A PureDisk plug-in Media server deduplication pool bpbkar Client B bptm Load balancing server Control path Data path The following list describes the backup process when a media server deduplicates the backups and the destination is a media serve
Deduplication architecture Media server deduplication process ■ The PureDisk plug-in sends only unique data segments to the NetBackup Deduplication Engine on the storage server. The NetBackup Deduplication Engine writes the data to the media server deduplication pool. Figure 9-3 shows the backup process when a media server deduplicates the backups. The destination is a PureDisk storage pool. A description follows.
Deduplication architecture Deduplication client components Deduplication client components Table 9-2 describes the client deduplication components. Table 9-2 Component Client deduplication components Host PureDisk plug-in Client Description The PureDisk plug-in is the data interface to the NetBackup Deduplication Engine on the deduplication storage server. The PureDisk plug-in does the following: ■ Separates the file’s metadata from the file’s content.
Deduplication architecture Deduplication client backup process Deduplication client backup to a media server deduplication pool Figure 9-4 Master server Control path Data path Deduplication client nbjm bpbkar PureDisk plug-in Proxy server (nbostpxy) bpbrm bptm Proxy plug-in NetBackup Deduplication Engine Deduplication storage server Media server deduplication pool The following list describes the backup process for a deduplication client to a media server deduplication pool: ■ The NetBackup Job
Deduplication architecture Deduplication client backup process ■ The PureDisk plug-in performs file fingerprinting calculations. ■ The PureDisk plug-in sends only unique data segments to the storage server, which writes the data to the media server deduplication pool. Figure 9-5 shows the backup process of a client that deduplicates its own data. The destination is a PureDisk storage pool. A description follows.
Deduplication architecture About deduplication fingerprinting ■ The client nbostpxy process moves the data to the PureDisk plug-in. ■ The PureDisk plug-in retrieves a list of fingerprints from the last full backup for the client from the NetBackup Deduplication Engine. The list is used as a cache so the plug-in does not have to request each fingerprint from the engine. ■ The PureDisk plug-in performs file fingerprinting calculations.
Deduplication architecture Data removal process ■ The PureDisk plug-in reads the backup image and separates the image into files. ■ The PureDisk plug-in separates files into segments and calculates the fingerprint for each file and segment. ■ The plug-in compares each fingerprint against the local fingerprint cache. If the fingerprint is not known in the cache, the plug-in requests that the engine verify if the fingerprint already exists.
Index A about NetBackup deduplication 11 about NetBackup deduplication options 12 appliance deduplication 13 B backup client deduplication process 114 big endian 106 byte order 106 C capacity and usage reporting for deduplication 87 client deduplication about 24 components 114 disabling for a specific client 90 host requirements 25 limitations 26 requirements 25 compression and deduplication 28 configuring a deduplication pool 50 configuring a deduplication storage server 48 configuring a deduplication s
Index deduplication hosts and firewalls 27 client requirements 25 load balancing server 22 server requirements 23 storage server 22 deduplication logs client deduplication proxy plug-in log 85 client deduplication proxy server log 85 deduplication database 84 monitoring 83 NetBackup Deduplication Engine 83 NetBackup Deduplication Manager 84 onfiguration script 83 PureDisk plug-in log 85 VxUL deduplication logs 86 deduplication node about 22 adding a load balancing server 58 removing a load balancing s
Index G garbage collection for deduplication 91 H host requirements 23 how deduplication works 14 I images on disk report 82 initial seeding 32 L license information failure for deduplication 94 license key for deduplication 42 licensing deduplication 42 limitations media server deduplication 23 little endian 106 load balancing server about 22 adding to a deduplication node 58 deleting configuration file 70 for deduplication 22 removing from deduplication node 70 logs client deduplication proxy plug-in
Index PureDisk Deduplication Option replacing with media server deduplication 33 PureDisk plug-in about 110 log file 85 purediskdbuser account 111 R reconfiguring deduplication 63 recovery deduplication storage server 105 from deduplication storage server disk failure 104 Red Hat Linux deduplication processes do not start 95 replacing PDDO with NetBackup deduplication 33 replacing the deduplication storage server 43 reports disk logs 82 disk pool status 82, 88 disk storage unit 82 resetting the dedup
