HP StorageWorks File System Extender Software user guide Part number: T3648-96010 First edition: October 2006
Legal and notice information © Copyright 2005, 2006 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents Intended audience . . . . . . . . . . . . Related documentation . . . . . . . . . Document conventions and symbols HP technical support . . . . . . . . . . . Subscription service . . . . . . . . . . . HP web sites . . . . . . . . . . . . . . . . Documentation feedback . . . . . . . . ......................... . . . . . . . . .. .. .. .. .. .. .. .. . . . . . . . . .. .. .. .. .. .. .. .. . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . .. .. .. .. .. .. .. ..
Reconfiguring communication on external Windows FSE clients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Changing the IP address of an FSE host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4 Managing media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Multiple copying. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Parallel copying. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sequential copying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HSM file system access modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Firebird server problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Communication problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SCSI problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tape library problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
About this guide This guide provides information about: • Getting familiar with basic FSE concepts and various FSE implementation options • Getting familiar with FSE policies and their usage examples • Performing typical tasks needed for operating and monitoring the FSE implementation • Performing typical tasks needed for maintaining and troubleshooting the FSE implementation Intended audience This guide is intended for system administrators with knowledge of: • Linux platform • Windows platform Related d
CAUTION: Indicates that failure to follow directions could result in damage to equipment or data. IMPORTANT: NOTE: TIP: Provides clarifying information or specific instructions. Provides additional information. Provides helpful hints and shortcuts. HP technical support Telephone numbers for worldwide technical support are listed on the HP support web site: http://www.hp.com/support/.
1 Introducing HP StorageWorks File System Extender This chapter includes the following topics: • What is FSE?, page 9 • FSE basics, page 9 • FSE architecture, page 11 What is FSE? HP StorageWorks File System Extender (FSE) is a mass storage oriented software product, based on client-server technology.
enables multiple copying of migrated files, thus satisfying the need for additional backup copies and avoiding recall failures. Library - tape compatibility FSE software is a part of HP StorageWorks File System Extender, and can be used with supported SCSI tape libraries with supported drive types. FSE software supports LTO drive technologies for LTO Ultrium 1, LTO Ultrium 2, and LTO Ultrium 3 media. LTO Ultrium 3 WORM media are supported where a WORM type tape media is required.
HSM file systems can be exported to be accessed via NFS protocol. Access using CIFS protocol is enabled by a specially customized Samba package. Files that are written to or read from an HSM file system through a Samba connection are handled like all other files created or accessed locally on the HSM file system. The customized package enables Samba to notify remote CIFS clients (especially Windows Explorer on Windows platform) when a file is offline, thus preventing unnecessary recalls of offline files.
1. Files from the migration candidate list are copied from the HSM file system to a special storage space on the FSE server, called FSE disk buffer. The FSE disk buffer location is platform-dependent. Linux specific The root directory of the FSE disk buffer is /var/opt/fse/diskbuf. Windows specific The root directory of the FSE disk buffer is %InstallPath%\var\diskbuf. The value of %InstallPath% depends on the path selection during FSE installation. It defaults to C:\Program Files\Hewlett-Packard\FSE. 2.
For further information on the recall process and its configuration, see chapter ”Migration, release, recall, and deletion” on page 123. Deletion Deletion is a process of obsoleting and removing obsolete files from HSM file systems. You can use this process for various purposes, including Information Lifecycle Management (ILM) and freeing disk space of redundant files. The criteria that induce file obsolescence on HSM file systems are defined in the deletion policy.
File System Catalog The File System Catalog (FSC) is a database related to a single FSE partition. It consists of a Data Location Catalog (DLC) and a Name Space Catalog (NSC). The Data Location Catalog contains information about location of files on the FSE media. It stores a full history of file locations. The Name Space Catalog (NSC) contains metadata of files on an HSM file system.
Each configured FSE library has one Library Agent running. The Library Agent handles actions in the FSE library, such as loading and unloading media, and library inventory rescan. An FSE implementation which only uses FSE disk media do not have FSE libraries. FSE drive An FSE drive is a tape drive inside an FSE library, configured in the FSE implementation. It is used to read to and write from FSE media.
systems. User data from local HSM file systems are stored on the media in the attached SCSI tape library and/or on disk media. Distributed implementation In this implementation, FSE clients connect to one FSE server and are physically separated. External clients connect to the server through either a LAN or a private network, and host HSM file systems, while the FSE server does not need to host any HSM file systems.
Client-server communication in a distributed FSE implementation An external FSE client communicates with an FSE server through the network using CORBA services. An external client can connect to only one FSE server. Figure 5 Client-server communication The network must provide a bidirectional connection between the FSE server and each of the external FSE clients. You can verify the network connection using the FSE tool fse_net.
connections or private network connections for FSE client-server communication. You can retrieve the parameters that you will use for network configuration using the FSE fse_net tool. If you are about to configure the interprocess communication in a newly installed FSE implementation, the configuration must be performed before the first startup of FSE.
Generally, FSE daemons (services) are the processes that are running most of the time FSE is operating. An FSE daemon (service) is usually a server process with several sophisticated services. FSE agents are software agents, and are designed to perform defined tasks required by each specific job. The location of the binaries for FSE daemons (services) and FSE agents is platform-dependent. Linux specific FSE daemon and agent binaries are located in the directory /opt/fse/lbin.
FSE daemon (service) Binary Running on... Description Library Agent fse-la-s FSE server Handles actions on FSE libraries, such as loading and unloading media, library inventory rescan, and so on. Back End Agent fse-bea FSE server Handles data transfer between FSE drives and FSE media, executes FSE media formatting and initialization. or FSE agent The following figure demonstrates the interaction of the main FSE components.
Linux specific The FSE utility daemon binaries are located in the directory /opt/fse/lbin. Windows specific The FSE utility service binaries are located in the directory %InstallPath%\bin. The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE. The FSE utility daemons (services) are listed in the following table. FSE utility daemon (service) Binary Running on...
FSE commands have the following characteristics: • One command per FSE resource FSE commands are based on a principle of one command per FSE resource, for example fsepartition, fsepool, fsemedium, fselibrary, fsedrive, and so on. • Common options The majority of FSE commands have similar options, for example --help, --add, --modify, --remove, --list, --show, --status, and so on. • Short and long option variants FSE commands accept short (-o) and long (--option) formats of options.
FSE command Description fselibrary Manages configuration of the FSE libraries. Monitors inventory of the FSE libraries. fsemedium Manages configuration of the FSE media. Formats and initializes FSE media. Performs maintenance tasks on the FSE media. Duplicates or reorganizes FSE media. Recreates a copy of the unreadable data located on damaged FSE media. Modifies current status of the FSE media. fsepartition Manages configuration of the FSE partitions.
access to the data stored in FSE databases and system files. The client part, the FSE Management Console client, runs as an interactive standalone application and provides the configuration, administrative and monitoring features for FSE. The FSE Management Console client can be used from anywhere in the intranet, not necessarily from a system that is part of the FSE implementation.
2 Configuring, starting, and stopping the FSE Management Console This chapter includes the following topics: • Configuring the FSE Management Console, page 25 • Starting and stopping the FSE Management Console server, page 28 • Starting and stopping the FSE Management Console client, page 34 • Starting an interactive session with the Management Console client, page 35 Configuring the FSE Management Console This chapter provides instructions for initial configuration of the FSE Management Console server.
# GUI Server debug level # DEBUG_LEVEL = -d4 -DAll # path to OMNIORB configuration file OMNIORB_CONFIG = /etc/opt/fse/omniORB.
Parameter Description CFG_FILE Specifies the location for the FSE implementation services configuration file. Linux specific On Linux platform, the default full pathname of the file is /etc/opt/fse/services.cfg. Windows specific On Windows platform, the default full pathname of the file is %InstallPath%\etc\services.cfg. DEBUG_LEVEL Specifies the logging level information for the FSE Management Console server (for details, see ” Configuring the logging level” on page 27).
4 | Trc - errors, warnings, and detailed trace # Argument for tracing level after the option -D is literal # For FSE Management Console server Mif[D] – the communication activity between MC server and Management Interface; Cs[D] – the communication activity between MC server and MC client; Pool[D] – activity on Pool object; Flt[D] – activity on Filter objects; All[D] – whole activity for all logical parts of MC server # For FSE Management Console client Comm[D] – communication activity between MC cl
the graphical user interface is used for the start-up, the FSE Management Console server is run in the background mode. Using the command-line options, you can also customize the path of the FSE Management Console server configuration file and set the preferred debugging level. Starting the FSE Management Console server on Linux systems On the Linux platform, the FSE Management Console server can be started using the command-line interface.
Stopping the FSE Management Console server on Linux systems On the Linux platform, the FSE Management Console server can be stopped using the command-line interface. Stopping the FSE Management Console server using the operating system CLI To stop the FSE Management Console server that is running in the background mode, proceed as follows: 1. Run the following command: # /etc/init.d/guisrv stop To stop the FSE Management Console server that is running in the foreground mode, proceed as follows: 1.
Starting the FSE Management Console server on Windows systems On the Windows platform, the FSE Management Console server can be started using the command-line interface or the graphical user interface. The FSE Management Console server execution modes are: • background mode In this mode, the FSE Management Console server is running as an ordinary Windows service.
2 - errors and warnings 3 - errors, warnings and trace 4 - errors, warnings and detailed trace -f - configuration file (default ..\var\cfg\gui\guisrv.conf -[h?] - usage information To start the FSE Management Console server in the background mode, proceed as follows: 1. Change the current directory as shown: C:\>cd %InstallPath%\bin The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE. 2.
For further information about the Services tool, see the Microsoft Windows documentation. Figure 8 Starting and stopping Windows services To start the FSE Management Console server in the foreground mode, proceed as follows: 1. Click Start > Programs > Hewlett-Packard > FSE > FSE GUI–Server (Process). Stopping the FSE Management Console server on Windows systems On the Windows platform, the FSE Management Console server can be stopped using the command-line interface or the graphical user interface.
• While the FSE GUI-Server (Process) window is in focus, press the key combination ALT + F4. Starting and stopping the FSE Management Console client You can start the FSE Management Console client using the command-line interface. Command-line options enable you to customize the path of the FSE Management Console client log file and set the preferred debugging level. You can stop the FSE Management Console client using the graphical user interface.
• The PATH environment variable must contain the path /opt/fse/bin. This is the path of the FSE Management Console client binary. To start the FSE Management Console client, perform the step: 1. Run the following command, specifying the appropriate options. # guiclt Options Stopping the FSE Management Console client on Linux systems On the Linux platform, the FSE Management Console client can be stopped using the graphical user interface.
Configuring, starting, and stopping the FSE Management Console
3 Configuring FSE This chapter includes the following topics: • Before you start, page 37 • Configuration basics, page 38 • Configuration procedure, page 60 • Reconfiguring FSE interprocess communication, page 83 • Changing the IP address of an FSE host, page 89 Before you start FSE provides effective management of your data with its extended virtual disk storage, which is controlled by user-defined policies.
• Consider supported media types (tape, disk) according to your data characteristics. Note that duplicating (tape) media can significantly ease your vaulting policy. • Consider your resource policies. Your system and partition policies need to be synchronized and well balanced if you want to achieve good performance.
[Date = "2002/08/05 15:00:12"] [User = "fse-administrator@company.
Additionally, you are able to revert from the current to any other configuration revision at any time. You only need to retrieve the desired revision from the configuration database, and then reapply it. The applied configuration is again saved as a new configuration revision. CAUTION: Do not directly alter the contents of the Configuration Database, as it may cause failures or even loss of data.
The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE. 1. Locate the desired template in the newconfig directory and copy it to the selected directory, in this example named MyConfiguration, using the following commands: Linux specific # cd MyConfiguration # cp /opt/fse/newconfig/fse-partition.
2. Check the configuration history for the target partition. Use the following command to get the list of all configuration revisions for the target partition in the configuration database. fsepartition --show PartitionName --history 3. Copy the contents of the chosen configuration revision (in our case 4) for FSE partition to a file, named for example MyFile in the current directory: fsepartition --show Partition01 4 > MyFile 4.
reconnection to a particular FSE Management Console server, the Connection dialog box displays each time the FSE Management Console client is started. Figure 9 New connection dialog box NOTE: You can establish a connection to another FSE Management Console server using the FSE Management Console client GUI. For details, see ”Graphical user interface overview” on page 44.
Choosing offline operational mode Limitations Limitations for using the offline operational mode of the FSE Management console client are the following: • In the offline operational mode, you can only perform FSE configuration-related tasks for a particular FSE implementation. All changes are local and can be applied to the FSE implementation in a subsequent online session. • In the FSE Management Console client GUI, the All Media, Unassigned Media, Jobs, Log, and Users containers are not visible.
Graphical user interface layout Figure 10 GUI layout The object tree contains hierarchically organized objects. You can browse the hierarchy by clicking the expand/collapse buttons besides the objects. Expanding a subtree does not select the object at the subtree origin. To select an object, click it. The right pane shows the property page with details and status information for the selected object.
The object tree The object tree lists all manageable objects in the FSE implementation. You can navigate through the tree and use drop-down and pop-up menus for listing, configuring, monitoring, and managing the objects. You can also define object and field filters.
An FSE library configured in the FSE implementation. This object holds the containers for FSE slots and FSE drives configured in the library. When this object is selected, the General tab is displayed on the right pane. The tab shows the current status and the configuration variables of the FSE library. When this object is right-clicked, the following actions are available in the pop-up menu: • Enable or Disable: makes the library available to the FSE implementation or puts the library offline.
medium volumes on the media, size of the data volumes, location of the system volume, and size of the system volume are displayed. When this object is right-clicked, the actions New, New Window from this place, and Properties are available in the pop-up menu. For action descriptions, see ”Actions associated to objects” on page 53. • Media Pools > (virtual object) An FSE media pool configured in the FSE implementation. Holds a list of media assigned to this FSE media pool.
When this object is right-clicked, the actions New Window from this place and Properties are available in the pop-up menu. For action descriptions, see ”Actions associated to objects” on page 53. • Partitions > > Media Pools > (virtual object) This is an FSE media pool configured in the FSE implementation and assigned to the FSE partition PartitionName. This object holds a list of all FSE media assigned to this media pool.
NOTE: The Jobs container is not accessible while the FSE Management Console client is in the offline operational mode. When this object is selected, the list of current FSE jobs is shown in the right pane. For each job the job ID, number of copies that the job can read (recall job) or must write (migration job), the partition name, the job type, current job priority, the starting date and time, and current job status are displayed. IMPORTANT: displayed.
• Duplicate Media: duplicates the medium to another medium that is still unassigned. For description of other available actions, see ”Actions associated to objects” on page 53. • (physical object) An FSE medium volume on the selected FSE medium. Objects of this type are displayed in the Media - Properties window after the action Properties is selected for a particular FSE medium.
In each dialog box , the Options box is only a placeholder and not used in the current FSE product version.
Figure 13 Properties dialog box Actions associated to objects When you right-click a particular object in the object tree, a pop-up menu is displayed. The set of the available menu items depends on the object type and the current object status. The actions (pop-up menu items) that are available for the majority of object and container types are the following: • New: opens a pop-up menu with two options: • From File ... creates a new object of the selected type from configuration files.
The Server menu The menu items of the Server menu are listed and described in the following table. Menu item (action) Action description New Connection Opens the Connection window where you can specify connection settings and create a new connection to an FSE implementation: • • • • • • • • Name: specifies a descriptive name for the connection. This name identifies the remote FSE Management Console server system in the object tree.
Menu item (action) Applicable objects Action description Library Library objects. Displays a submenu with the menu items: • • Partition Partition objects. Displays a submenu with the menu items: • • • • • • • Job Rescan Update Jobs objects. Trigger Deletion Check Medium Check FS Recover FSC Reorg-Scan Mount Umount Displays a submenu with the menu items: • • Change Priority Abort Configuration Library, drive, partition, and media pool objects.
The Windows menu The menu items of the Windows menu are listed and described in the following table. Menu item (action) Action description New Instance Opens a new FSE Management Console client instance in which the initial GUI view is displayed. New Window from this place Opens a new FSE Management Console client instance in which the properties of the currently selected object are displayed in the right pane.
The status bar The status bar provides current status of the FSE Management Console client, and the object count and the status information for the item that is currently selected in the object tree. The status information is only available for the following containers and objects: • containers: Libraries, Slots, Drives, Media Pools, Partitions, All Media, Unassigned Media, Jobs. • objects (virtual and physical): , Log.
The status bar displays the number of configured FSE media in the selected FSE media pool. Additional numbers represent FSE media counts according to their states. The status icons have the following meaning: Status icon Current status free open in use full unusable unreliable uninitialized unknown • Partitions (container) The status bar displays the number of configured FSE partitions in the selected FSE implementation. Additional numbers represent FSE partition counts according to their states.
Keyboard shortcuts The following keyboard shortcuts are available for the most common actions. Menu item (action) Keyboard shortcut New Connection Alt + N Connect Alt + C Disconnect Alt + D Remove Server Alt + R Exit Alt + X Settings Alt + O Properties Alt + Enter FSE Management Console users Access to an FSE implementation from the FSE Management Console client is allowed only to configured users that are stored in the FSE Management Console user database.
Removing an FSE Management Console users To remove an FSE Management Console user, follow the steps: 1. In the FSE Management Console client GUI, in the object tree, select Users. 2. In the property page, right-click on an existing user name and click Delete object. FSE Management Console user properties Each configured FSE Management Console user has the following properties, which are shown in the User Details window: • User name: unique alphanumeric string for identifying the user.
1. Define the values for parameters that must be configured in the FSE configuration file. NOTE: To avoid difficulties like missing or inappropriate parameters, HP recommends that you refer to FSE configuration templates. They already contain all required parameters and provide remarks on appropriate use of configuration variables. 2.
Linux specific On Linux platform, FSE uses a SCSI generic (sg) driver for accessing SCSI libraries and drives. The sg driver is an upper level SCSI subsystem device driver that enables user applications to send SCSI commands to relevant devices. It is capable of addressing up to 256 SCSI devices. When configuring libraries and drives, you need to specify their control device file names: each device file name corresponds to one attached SCSI device.
TIP: You can also check the /proc/scsi/scsi file for a list of available SCSI devices. Windows specific C:\Program Files\Hewlett-Packard\FSE\bin>fse_scsi --scan ** \\.\Scsi0: id:0:0:2:0 SCSI device ID: port 0, ch 0, id 0, lun 0, type Medium Changer 8 Device: HP ** \\.
• DriveIndex • Capabilities • Host • ControlDevice IMPORTANT: You need to assign unique DriveIndex values to the FSE drives in the ascending order, starting with the value 1 for the first FSE drive you add to the FSE system. For configuration examples, see ”Library and tape drive configuration examples” on page 65. You will configure your media pool according to your drive configuration. 5. Add the new FSE library to the FSE implementation.
8. In the [ New Drive ] - Creation window, enter the required parameter values into the available text fields. Figure 15 Configuring an FSE drive 9. Click OK. The following are examples of configuration files of FSE libraries and various FSE tape drives on different platforms. The configuration files are uniform for all supported platforms, with the exception of the variable ControlDevice, which uses a different format for each platform.
# # Linux: /dev/sgN # Windows: id:P:B:T:L # ControlDevice = "/dev/sg0" LibType = SCSI # AIT - Sony AIT Family # LTO - LTO Ultrium MediaFamily = LTO # # Specify the slots that will be used with this library. # You can specify ranges of slot numbers and individual # slot numbers.
Windows specific [FSE Library] [Version = 1] [Date = "2003/04/11 15:24:32"] [User = "fse-administrator@company.com"] # # Specify a name for the tape library and a hostname, where it # resides. Use full host name. # Name = "HP MSL5000 #2 LTO" Host = "fse_server.company.com" # # Control device is identification of library's # (media changer's) on the specified system.
Linux specific [FSE Drive] [Date = "2003/04/11 15:30:12"] [User = "fse-administrator@company.com"] Name = "LTO 01" Library = "ESL9000-LTO" MediaFamily = LTO DriveIndex = "1" Capabilities = () Host "fse_server.company.com" { ControlDevice = "/dev/sg4" } Windows specific [FSE Drive] [Date = "2003/04/11 15:30:12"] [User = "fse-administrator@company.com"] Name = "LTO 01" Library = "ESL9000-LTO" MediaFamily = LTO DriveIndex = "1" Capabilities = () Host "fse_server.company.
DriveIndex = "1" Capabilities = (WORM) Host "fse_server.company.com" { ControlDevice = "/dev/sg6" } Windows specific [FSE Drive] [Date = "2005/04/11 17:40:00"] [User = "fse-administrator@company.com"] Name = "LTO WORM 01" Library = "ESL9000-LTO" MediaFamily = LTO DriveIndex = "1" Capabilities = (WORM) Host "fse_server.company.
• MediaFamily • PoolType • BlockSize • NumOfVolumes • SysVolLocation • SizeOfSysVol For an FSE media pool consisting of disk media, you need to configure the following parameters in its configuration file: • Name • MediaFamily • PoolType • BlockSize • VolumeSize 4. Add the new FSE pool to the FSE implementation.
You can also include the existing non-HSM file systems in the FSE implementation. All data on such file systems is preserved and put under the FSE control. For more information, see ”Integrating existing file systems in the FSE implementation” on page 72. Creating and preparing new HSM file systems On Linux systems, HSM file systems must be located on LVM volumes. On Windows systems, HSM file systems must be located on separate disk volumes.
5. In the Assign Drive Letter or Path window, select either the Assign a drive letter or Mount this volume at an empty folder that supports drive paths option and click Next. Possible choices are shown in the following graphics. Figure 17 Assigning a drive letter Figure 18 Mounting a volume to an empty folder 6. In the next step of the Create Partition Wizard, accept the default options by clicking Next. 7. Click Finish to close the Create Partition Wizard and format the newly created partition.
Phase 4: configuring FSE partitions NOTE: A particular FSE partition is related to a single HSM file system and one or more FSE media pools. Configuring a regular FSE partition on the Linux platform After you have configured FSE media pools and prepared an HSM file system, you can configure an FSE partition as follows: 1. The FSE partition configuration contains the path to the device file of the HSM file system.
# uncomment and configure if Type = WORM : # WORMTimeout = 450s } 5. Decide which FSE media pools should be assigned to the FSE partition, and specify an appropriate value for the Pools variable in the configuration file. CAUTION: Once migrations have already been running on the FSE partition, you cannot change the set of FSE media pools that are assigned to the FSE partition any more. 6. Configure all other parameters in the configuration file.
Configure all other parameters in the configuration file. For more information, see chapter ”Migration, release, recall, and deletion”. Figure 19 Configuring an FSE partition 6. Click OK. Configuring a regular FSE partition on the Windows platform After you have configured FSE media pools and prepared an HSM file system, you can configure an FSE partition as follows: 1.
2. In the above command output, mark and copy the \\?\Volume{VolumeGUID}\ value above the line D:\hsm_fs_folder: \\?\Volume{f6f3e1b6-b2ac-11d7-9c99-00b0d02fadef}\ The next steps depend on the chosen FSE user interface. To configure an FSE partition using the FSE command-line interface, proceed with the following steps: 3. On the consolidated FSE system or the FSE server, locate the directory with the FSE configuration templates.
8. Add the new FSE partition to the FSE implementation. PartitionCfgFileName is the name of the local copy of the FSE partition configuration template: fsepartition --add PartitionCfgFileName Once the partition is configured, a new Partition Manager is started for the newly configured partition. 9.
Configuring a WORM FSE partition FSE provides an option to configure an HSM file system as a WORM (Write Once Read Many) file system. The Type parameter in the FSE partition configuration file defines whether the type of a file system is regular or WORM. IMPORTANT: A WORM FSE partition cannot be changed to a regular FSE partition, but a regular FSE partition can be reconfigured as a WORM FSE partition.
Type = WORM # Regular - regular read/write file system # WORM - Write-Once-Read-Many file system # uncomment and configure if Type = WORM : WORMTimeout = 60s } # # Specify which media pool(s) will be used for storing data to. # # NOTE: The number of pools specified defines the number of copies to # be done. # Pools = ("MyWORMPool1", "MyWORMPool2") The differences between configuration files for WORM FSE partitions located on Linux and Windows host are the same as with regular FSE partitions.
NOTE: The option --force will allow formatting of the FSE medium only if there is no valid FSE user data stored on it (the value of Used[%] for all medium volumes is zero). During formatting, the FSE medium is partitioned into FSE medium volumes. 5. Once the medium is formatted, you must initialize the FSE medium volumes created on it. For more information on the FSE medium volumes, see ”FSE medium volumes” on page 98.
NOTE: The option --force will allow formatting of the FSE medium only if there is no valid FSE user data stored on it (the value of Used[%] for all medium volumes is zero). • LTO media do not need formatting, since they can only contain one FSE medium volume. During formatting, the FSE medium is partitioned into FSE medium volumes. Figure 22 Formatting and initializing an FSE medium 10.Click Confirm. 11.Repeat the steps 8 and 9 for each additional FSE medium that you want to format and initialize.
CAUTION: To avoid obstructing other FSE users and the operation of already existing HSM file systems, it is necessary that you choose the mounting mode as appropriate.
CAUTION: To avoid obstructing other FSE users and the operation of already existing HSM file systems, it is necessary that you choose the mounting mode as appropriate.
The initial configuration must be performed after the FSE software package installation, before the FSE implementation is put into operation for the first time. Afterwards, if the implementation or the network type is changed, the communication settings must be reconfigured. CAUTION: An appropriate configuration of the FSE interprocess communication is of crucial importance for normal FSE operation. Incorrectly configured interprocess communication may lead to a non-operating FSE implementation.
NOTE: SUSE Linux specific On a SUSE Linux system, do not run YaST2 after you have configured this FSE host to use a private network for the FSE interprocess communication. Running YaST2 modifies /etc/hosts in such a way that the subsequent FSE implementation startups fail. Alternatively, you can modify /etc/sysconfig/suseconfig by changing the line CHECK_ETC_HOSTS="yes" to CHECK_ETC_HOSTS="no". You can then run YaST2 without affecting FSE operation, but you cannot modify host names with it. 5.
2. In the omniORB.cfg file, configure the parameters in the section --- Private network parameters --- with the following information: • the FQDN that identifies the system inside the private network • the IP address of the system • the subnet mask All these parameters must be verified against the actual private network configuration. Ensure that the FQDN you specify in omniORB.cfg matches the FQDN specified for the hostname variable in the services.cfg file.
Reconfiguring communication on external Linux FSE clients Ordinary LAN connection If the external Linux FSE clients and the consolidated FSE system or the FSE server will communicate through an ordinary LAN, you have to modify the services.cfg file on each external Linux FSE client, as follows: 1. Ensure that the value of the server variable in services.cfg equals the fully-qualified domain name (FQDN) that identifies the consolidated FSE system or the FSE server inside the LAN.
clientTransportRule = localhost tcp clientTransportRule = 192.168.240.0/255.255.255.0 tcp clientTransportRule = * none # The order of network interfaces to use for opening new server connections: # Only localhost and private network. Others are denied. serverTransportRule = localhost tcp serverTransportRule = 192.168.240.0/255.255.255.0 tcp serverTransportRule = * none The following is an example excerpt from the local /etc/hosts file that matches the above services.cfg and omniORB.cfg files: 123.45.67.
• the FQDN that identifies the consolidated FSE system or the FSE server inside the private network • the IP address of the system • the subnet mask All these parameters must be verified against the actual private network configuration. Ensure that the FQDN you specify in omniORB.cfg matches the FQDN specified for the server variable in the services.cfg file. The following example is an excerpt from a properly configured omniORB.
4. Change the IP address of the host using the operating system tools. 5. If the omniORB.cfg file is configured, change its contents appropriately. For more information, see ”Reconfiguring FSE interprocess communication” on page 83. 6. Start the omniORB naming service and FSE processes with fse --start.
4 Managing media This chapter includes the following topics: • Introduction, page 91 • FSE media pools, page 91 • FSE media, page 98 Introduction The way you manage FSE media can significantly increase the stability and performance of your FSE implementation. Managing media consists of tasks that are related to the media life cycle, from formatting, initialization and optional duplication for data vaulting, to replacing worn-out media and media reorganization.
• Regular media pool This media pool type is the most common one since it is used for regular archiving of the migrated FSE implementation data. It contains FSE media that can be overwritten, reorganized, reinitialized, and reformatted. Regular media pools can contain either tape media or disk media. • WORM media pool Used for WORM media only. On these media, data can be recorded only once, therefore WORM media pool type is particularly suitable for storing permanent archives.
value to DWORD registry setting. See section ”Media management problems” on page 252 for details. Check the following examples for the specifics according to different pool types, and then continue applying the modified templates. Example of an LTO media pool configuration file [FSE Pool] [Version = 1] [Date = "2003/04/10 16:11:26"] [User = "fse-administrator@company.com"] # # Specify the name of the media pool. Media pools are # used by FSE partitions.
SysVolLocation = None SizeOfSysVol = 0MB Example of an LTO WORM media pool configuration file [FSE Pool] [Version = 1] [Date = "2005/04/10 18:00:00"] [User = "fse-administrator@company.com"] # # Specify the name of the media pool. Media pools are # used by FSE partitions. # # NOTE: You have to assign media to pools with fsemedium command.
Example of a disk media pool configuration file [FSE Pool] [Version = 1] [Date = "2003/04/10 17:30:00"] [User = "fse-administrator@company.com"] # # Specify the name of the media pool. Media pools are # used by FSE partitions. # # NOTE: You have to assign media to pools with fsemedium command. # Name = "My_DM_pool" MediaFamily = DISK PoolType = Regular # # Change the values of the on-tape format if the defaults don't suit you.
The command output lists the names and attributes of all currently configured FSE media pools in the FSE implementation, as follows: • the media pool family and type, • the owning partition name, • the medium block size, • the number of volumes on the medium, • the data volume size, • the location of an optional system volume on the medium, • the system volume size. You can specify most of these attributes in the media pool configuration file.
Excerpt from the fsepool --list output Pool Family Type Partition BlkSize[KB] #Vol VSize[MB] SysVol SysVol[MB] my_LTO LTO regular partLTO 128 1 n/a none n/a my_Disk disk regular partLTO 64 1 100 none n/a Excerpt from the fsepool --list --detail output Pool: my_LTO UUID: c8898640-8f8a-44aa-a6fd-47687ac1f791 Family: LTO Type: regular Partition: partLTO PartUUID: BlkSize[KB]: 128 #Vol: 1 VSize[MB]: n/a SysVol: none SysVol[MB]: n/a Key: 1 Pool: my_Disk UUID: f264a661
• the total and available capacity of all system volumes on media in the media pool, • the percentage of system volume usage, • the numbers of “good” (usable), unreliable, unusable, and uninitialized media of the media pool. Excerpt from the fsepartition --status output Pool Size[MB] Avail[MB] Used[%] SysSize[MB] SysAvail[MB] poolLTO 92173 70397 24 0 0 SysUsed[%] #Good #Unreliable #Unusable #Uninit 0 2 0 0 0 FSE media An FSE implementation supports tape media as well as disk media.
Volumes of this type cannot be created on the supported LTO Ultrium media and are not used on disk media. FSE supports up to 255 volumes on disk media. Other specifics of different media types are described in the next section. FSE media types The following is a list of the supported media types and their characteristics: • LTO Ultrium 1, LTO Ultrium 2, LTO Ultrium 3 Used in regular and backup media pools. LTO Ultrium media do not support multiple volumes and do not need to be formatted.
Dedicate a disk with high performance for FSE software installation and HSM file systems. Such a disk serves as a primary storage space of files with high frequency of user access, which may affect the performance of the whole FSE implementation. A performance disk is not meant to be a secondary storage space (file archive storage), thus its capacity is not as important as its high performance, which enables optimal operation of FSE.
You define this parameter in the /etc/exports file on the NFS server for this particular export. HP recommends that you make such exports available only for the FSE server. The following is the example of the appropriate entry in the /etc/exports file: /mnt/disk_medium linux_host.company.
| |-- 00000001 | | |-- 00000001.hdr volume header | | `-- 00000003.vdf volume data file | |-- 00000002 | | |-- 00000001.hdr | | |-- 00000003.vdf | | `-- 00028545.fri | `-- 00000003 | volume number `-- 00000001.hdr fast recovery information file volume contains no data Each medium volume has its own directory named after the corresponding volume number, for example: /var/opt/fse/dm/dm000001/00000001.
NOTE: WORM media can only be formatted once. A medium is formatted using the following command: fsemedium --format Barcode If particular medium has already been used in the FSE implementation or contains certain types of data, you must use forced formatting to format it. This is possible only with non-WORM media. The following command initiates forced formatting for a medium: fsemedium --format Barcode --force For more information on the fsemedium command, see the FSE CLI reference.
For more information on the fsemedium command, see the FSE CLI reference. The following table helps you determine what kind of initialization is allowed for different medium volume types identified by FSE.
A duplicated medium is an exact copy of the original medium.
LTO 2 : 200 GiB * 2 = 400 GiB LTO 3 : 400 GiB * 2 = 800 GiB Note that the factor 2 is used because 2 is the assumed compression ratio of the tape drives. If the compression ratio is different, the factor must be changed appropriately. For information on how to extend FSE disk buffer with additional file systems or volumes, see chapter ”Monitoring and maintaining FSE”, section ”Extending storage space of FSE disk buffer” on page 176.
[11:42:09] Requesting resources... [11:42:09] Resources allocated. [11:48:42] Loading medium 456456 to drive drv1. [11:43:58] Checking medium format. [11:43:58] Blank medium detected. [11:43:48] Medium 456456: Blank volume detected. [11:43:58] Medium 456456, volume 1: Writing data... [11:43:58] Medium 456456, volume 1: Volume written. [11:43:58] Medium 456456, volume 2: Writing data... [11:43:58] Medium 456456, volume 2: Volume written. [11:43:58] Writing finished, releasing resources. [11:43:58] Finished.
The scanning step is run for all FSE media in all FSE media pools that are assigned to a specified FSE partition. This step determines the locations of valid and obsolete data on the already closed (full) and still open FSE medium volumes, according to parameters that you define. It then records the scan results to the Resource Management Database (RMDB).
You can limit the number of tape drives used during the reorganization of a specified FSE partition (MaxNumDrivesRecall variable in the FSE partition configuration file) or system-wide (SystemMaxNumDrivesMaint variable in the FSE system configuration file). The priority of the FSE reorganization jobs is defined by the MaintPriority variable in the FSE partition configuration file.
Scanning media for obsolete data You can run the scanning step of the reorganization process on all media belonging to a particular FSE partition, using the following command: fsepartition --reorg-scan PartitionName "YYYY/MM/DD HH:mm:ss" [NumberOfGenerations] NOTE: You can omit the time part (HH:mm:ss) of the ExpirationDate argument ("YYYY/MM/DD HH:mm:ss"). In this case, the time 00:00:00 is assumed.
Total Total amount of space on the FSE media assigned to the media pool. SlackSpace Amount of FSE slack space (in percentage). FSE slack space is the space on FSE media occupied by migrated file generations considered obsolete (not valid) by the reorganizational scan job. Displaying the FSE media-based statistics To check the statistics for all FSE media that belong to a particular FSE partition: 1. Ensure that the scan job running on the chosen partition has completed. 2.
The following is the syntax of the fsemedium command when used for media reorganization: fsemedium --reorg Barcode [{-V | --volume} VolumeNumber...] [--no-monitor] With monitoring enabled, the command generates an output similar to the following: [11:42:09] Started (JobID: 20040121000002). [11:42:09] Requesting resources... [11:42:09] Resources allocated. [11:42:09] Medium 050003 is already present in drive drv1. [11:42:09] Starting Back End Agent.
052002 2 data 5949 5949 60 open, scanned 052002 3 data 5949 5949 0 empty 052002 4 data 5949 5949 0 empty 052002 5 system 1019 1019 0 open In the example, the first medium volume on the FSE medium with barcode 052002 was recycled during the reorganization job and can be reused. Before recycled medium volumes can be reused, they must be reinitialized. The reinitialization is performed automatically at the end of the FSE reorganization job.
actual layout of the remigrated data on FSE media differs from the original data layout on the unreadable medium. Figure 23 Recreating a copy of damaged migrated data Media Pool 2 F2 F3 F4' F4'' F5 F6 Media Pool 4 F3 F4 F5' F5'' F6 F3 F4 F5' F5'' If alternative migrated copies of the unrecallable data exist in both tape and disk media pools, the redundant copy is recreated from the data stored in the disk media pool.
Determining the unusable media FSE detects read errors when recalling the migrated data from FSE disk or tape media to an HSM file system. After a read error is detected, the status of the FSE medium is automatically set to unusable. CAUTION: Before treating an FSE medium as unreadable, make sure it is not loaded in a faulty tape drive. In certain circumstances, an FSE medium may acquire the unusable status because of problems with the tape drive in which it is loaded.
[12:49:09] Starting Back End Agent. [12:49:58] Medium 050010, volume 3: Writing disk buffer. [13:00:58] Medium 050010, volume 3: Disk buffer written to medium volume. [13:11:50] Writing finished, releasing resources. [11:42:09] Request resources (bc: 050004) ... [11:42:09] Resources allocated. [11:42:09] Medium 050004 is already present in drive drv3. [11:42:09] Starting Back End Agent. [12:21:02] Medium 050004, volume 2: Reading files. [12:45:58] Medium 050004, volume 2: Files written to disk buffer.
Checking the status of media Run fsemedium --list [--detail] to get information on media status for all media configured in the FSE implementation.
Example output of the fsemedium --list command fsemedium --list Barcode Type Pool Status Location #Vol SysVolNo 000045 LTO-1 poolLTO open lib:libLTO:3 9 1 000046 LTO-1 poolLTO open lib:libLTO:5 9 1 000116 LTO-1 poolLTO uninitialized lib:libLTO:2 0 0 656701 unknown uninitialized lib:libLTO:4 0 0 656702 unknown uninitialized lib:libLTO:1 0 0 dm000001 disk poolDisk1 open n/a 9 0 dm000002 disk poolDisk1 open n/a 9 0 dm000003 disk poolDisk2 full n/a 4 0 dm00000
The status of the medium can be one of the following: uninitialized The medium must be initialized before it can be used. free The medium has been initialized, but no user data is stored on it yet. in use The medium is currently in use. open Medium volumes are not full yet and Fast Recovery Information is stored on disk only full Medium volumes are full and Fast Recovery Information is already written on the medium.
Depending on the operation you want to prevent being performed on a medium, you can mark it as either unreliable or unusable. To change a medium condition, you need to know its barcode. Status “good” refers to the usable condition of a medium. It needs to be set manually by the administrator if the medium was previously defective but has been repaired and is ready to be put back to use.
NOTE: FRI is stored in the FRI directory, /var/opt/fse/fri/, after recreating FRI for closed WORM volumes. The existing FRI on the medium volume and system volume will be overwritten with the newly created FRI. If there is not enough space on the medium volumes, the FRI is stored to the FRI directory on disk. The FRI directory path is /var/opt/fse/fri (on Linux systems) or %InstallPath%\var\fri (on Windows systems). Inconsistent or incomplete FRI can be detected with the FSC vs. media check.
Managing media
5 Migration, release, recall, and deletion This chapter includes the following topics: • FSE operation basics, page 123 • Migration, page 127 • Release, page 129 • Recall, page 136 • Deletion, page 137 • Resource allocation, page 140 • System allocation and job priority policy, page 142 • Partition allocation and job priority policy, page 144 • Priority calculation, page 146 • Multiple copying, page 147 • HSM file system access modes, page 148 FSE operation basics How does FSE function? FSE keeps track of
With migration and recall jobs, the Partition Manager executes the appropriate sub-process—the job that executes the defined task. When the FSE job completes its task, it is terminated. Release, however, is a simpler process that does not require an FSE job. It is managed and executed locally by the Hierarchical Storage Manager (HSM). FSE disk buffer FSE tries to unlock files on the HSM file system during migration, and de-allocate FSE media during recall as fast as possible.
Storage space allocation in an extended FSE disk buffer In an FSE implementation where several file systems are assigned to the FSE disk buffer, the Resource Manager tries to maximize usage of these file systems. This is necessary in order to execute FSE jobs in the most efficient manner, and to speed up FSE administrative, maintenance, and backup tasks. For all FSE job types, a round robin scheduler is used to allocate individual file systems to different FSE jobs.
The following graphic shows file-status changes, including transitions of file entries between FSE HSM lists. The MinFileAge parameter is one of the configuration parameters for an FSE partition. It represents the period of time during which a file must be left intact to become a migration candidate. Figure 25 Changes of states of files under FSE Example scenario of migration and release This example presents how the configured migration and release policies affect everyday work of FSE users. 1.
release policies) or at execution of the fsefile --trigger-deletion command (deletion policy). The parameters are described in comments inside the FSE configuration file templates. You need to plan in advance to set the policies according to your environment and to optimize the jobs. The way you set up these policies will affect your FSE implementation’s performance. Migration The relevant configuration parameters for the migration policy are specified in the FSE partition configuration file.
Minimum wait time for migration (MinWaitTime) Definition Minimum wait time for files on the migration candidate list during which no more files are added to the list. Impact If this parameter value is high, for example 8 hours, you will have irregular and substantial migrations. A low value results in frequent, smaller migrations. Maximum wait time for migration (MaxWaitTime) Definition Maximum wait time for migration; if reached, migration is started immediately.
Example situations that trigger migration There are four distinct situations that trigger a new migration job. • Situation 1 There were no files added to the list in the last MinWaitTime period of time and there are more than MinNumMigFiles files on the list and total size of files on the list is at least MinMigSize. • Situation 2 The period of time since the first file was put on the list is greater than MaxWaitTime. • Situation 3 There are at least MaxNumMigFiles on the list.
Note that the regular release job starts immediately after the forced release job has ended (because the system is still filled to the high watermark). Figure 26 Reaching the critical watermark on an HSM file system High watermark High watermark is usually reached when new files are created or copied to the HSM file system. Regular release is started for all files on the release candidate list that have passed their migration or recall retention period, and are larger than MinFileSize.
system is reached or when there are no further valid files on the release candidate list. Files not released remain on the release candidate list. Figure 27 Reaching the high watermark on an HSM file system Low watermark Low watermark is considered during a regular release triggered by reaching the high watermark on the HSM file system. The low watermark is the parameter that defines to what percentage volume the disk space should be freed, and stops the release when it has achieved this level.
NOTE: Zero-sized files remain permanently online. They cannot be migrated nor released. Limitations • Only sets of files can be excluded from the release process. You cannot exclude directories. • When an FSE partition is reconfigured by modifying the ExcludeFromRelease variable, the contents of the corresponding release candidate list is not affected.
MinFileSize = "2kB" ExcludeFromRelease = () NOTE: When configuring the ExcludeFromRelease variable, filenames and filename patterns must be specified in the UTF-8 format. Special pattern-matching characters The filename pattern-matching mechanism built into the FSE conforms to that used in command shells on UNIX systems, with a few differences. Special characters are permitted to simplify the definition of sets of files. The following table lists the special characters, meanings, and usage examples.
Configuring exclusion from release After you have modified the FSE partition configuration template or an already used FSE configuration file with a configured ExcludeFromRelease variable, you have to apply it by running the fsepartition --add or the fsepartition --modify command, respectively. Potential syntax errors in the ExcludeFromRelease parameter specification are immediately reported to the standard command output. For information on the fsepartition command, see the FSE CLI reference.
Migration retention time (MigRetentionTime) Definition A period of time during which a certain file is going to be kept online after migration before it will be released. Impact Consider for how long a migrated file should stay online. Adjust the value according to your data characteristics. If you use the partition for archiving purposes, the value should be low. Partitions with frequently used files should have a higher value set, so that the files do not get released and recalled all the time.
regular release. The release job is stopped when the low watermark is reached or if there are no more files on the release candidate list that match the specified criteria. • Situation 2 A critical watermark is reached on the HSM file system. In this case, all files on the release candidate list (regardless of their size and retention time) are scheduled for forced release. The release job is stopped when the high watermark is reached or if there are no more files to release.
Recall timeout (RecallTimeout) Definition When an application requests an offline file, the RecallTimeout is the time frame given to FSE for bringing the offline file back online. If FSE cannot bring the file online in this time frame, the application is notified about it with an error message, but the recall job continues to run in the background. Impact This parameter only affects implicit recalls, and is ignored by recall jobs that are triggered using the fsefile command.
Windows specific You can schedule a task for this purpose. The progress of the deletion process is logged to the FSE event log, and potential errors are logged to the FSE error log. The list of files that were deleted is written to the HSM file system log file on the FSE client that hosts the FSE partition. The HSM file system log file is called hsmfs_PartitionName.log. The locations of this file is platform-dependent. Linux specific HSM file system log file is located in the directory /var/opt/fse/log.
Examples of a configured deletion policy The following are two excerpts from FSE partition configuration files. They show examples of the configured deletion policies for various platforms of the FSE client, and their explanations.
Starting the deletion job You start a deletion job by running the following command on the FSE client: fsefile --trigger-deletion PartitionName [--force] [--no-monitor] Before running the command, ensure that the HSM file system of the selected FSE partition is mounted. By default, fsefile requires confirmation before starting the deletion process. You can suppress the confirmation request by specifying the --force option. This is particularly useful when using the command within scripts.
• Logical resources: • FSE media cartridges, each medium includes FSE medium volumes • FSE media pools The figure below schematically shows the resources as they are represented in the FSE implementation Figure 28 Example of FSE resources Central FSE resource manager All resources of the FSE implementation are controlled by a central Resource Manager. It manages the Resource Management Database and allocates resources. Resource allocation is used each time an FSE job requires new resources.
Note that resource allocation requests from different partitions can have different drive and priority policies, and that the same media pool cannot be assigned to several partitions. For more information on parameters for resource allocation, see ”System allocation and job priority policy” on page 142 and ”Partition allocation and job priority policy” on page 144.
Maximum number of drives for recall (SystemMaxNumDrivesRecall) Definition Maximum number of drives that can be used for recall by all FSE partitions. Impact The purpose of this parameter is to prevent recall jobs from using all drives in the FSE implementation. By limiting drives for recall, you ensure that drives remain available for other jobs (migration, administrative, recovery…). You should carefully balance resources according to your environment.
Phase factor (PhaseFactor) Definition System-wide factor by which the priority of each multi-phase job is increased. Impact The purpose of this parameter is to increase the priority of multiple phase jobs (migration to multiple copies or recall of files that span multiple media volumes) after some phases are already finished, such as migration phases with multiple copying, where each copy represents one phase.
Maximum number of drives for recall (MaxNumDrivesRecall) Definition Maximum number of drives that can be used for recall by the partition. Impact The purpose of this parameter is to prevent the respective Partition Manager from using all the drives at once for recall jobs. You should carefully balance resources according to your environment. Maximum number of drives for recovery (MaxNumDrivesRecovery) Definition Maximum number of drives that can be used for recovery by the partition.
Priority calculation The resource allocation queue holds resource requests that are sorted and processed by their job priority. The priorities of all jobs in the queue will be recalculated each time a job changes its status or resources are released or newly added to the FSE implementation. These job priorities are dynamically increasing according to their age and phase of the job.
Example of resource allocation The following is an example output of the fsejob --list command, which shows the resource allocation queue with job priorities. Note that jobs that are currently not waiting for resources do not have priorities assigned.
Sequential copying The number of configured FSE media pools determines the number of data copies for the corresponding FSE partition; if you have only one drive configured in FSE or if your FSE partition can use a maximum of one drive for migration at a time (MaxNumDrivesMigration variable in the FSE partition configuration file), your copies will be done sequentially.
• while the FSE implementation is fully operational, after the FSE administrator has manually switched from LAM to FAM using FSE command-line interface. In Limited Access Mode, FSE users and third-party applications can only browse the HSM file system directory tree and access files that are still online. They cannot modify existing data, add new directories and files, or recall offline files. If an application tries to access a file while the HSM file system is operating in LAM, it receives an error.
Migration, release, recall, and deletion
6 Monitoring and maintaining FSE The procedures in this chapter provide instructions for: • About monitoring and maintaining in FSE, page 151 • Monitoring general FSE implementation status, page 151 • Managing FSE partitions, page 152 • Monitoring FSE jobs, page 157 • Recalling older generations of a file, page 160 • Recalling sets of files in an efficient way, page 162 • Retrieving detailed information, page 164 • Extending storage space of FSE disk buffer, page 176 • Checking the consistency of the File
• host and mount point of the belonging HSM file system Example output of the fsesystem --status command Linux specific # fsesystem --status # of Partitions: 2 # of Admin Jobs: 0 Partition Status #Jobs video_captures mounted mail_archive mounted Host Mountpoint 2 fse_host.company.com /fsefs/video_captures 2 fse_host.company.
To display the status of an FSE partition using FSE command-line interface, run the following command: fsepartition --status PartitionName To display the status of an FSE partition using graphical user interface, in the FSE Management Console window, in the object tree, expand the Partitions container and select the desired FSE partition. The retrieved status information will be displayed on a property page in the right pane.
Example output of the fsepartition --status command Linux specific # fsepartition --status video_captures Partition: video_captures Status: mounted Host: fse_host.company.
Windows specific Using the fsepartition --show and mountvol commands determine the corresponding volume name and invoke the following command: C:\>fse --umount MountPoint VolumeName Unmounting an HSM file system located on a Windows host terminates the corresponding HSM process and switches the HSM FS filter operation for managing accesses to the FSE partition to Limited Access Mode (LAM). 3.
2. On the FSE server system, run the following command: fsepartition --remove PartitionName Retrieving detailed information about configured FSE partitions To retrieve detailed information about configured FSE partitions using the FSE command-line interface, run the following command: fsepartition --list --detail An example of this command output is shown in ”Excerpt from an example output of the fsepartition --list --detail command” on page 173.
While an HSM file system is operating in Limited Access Mode, do not invoke the fsefile --recall command in order to recall offline files. In this operational mode offline files should not be recalled manually. Switching to Limited Access Mode Prerequisites • FSE processes on the FSE server and the FSE client must be running. • The HSM file system must be operating in FAM.
• type of the job • current job priority • starting date and time • current job status Retrieved data is the same for both fsejob --list with and without the --detail option, but the data is displayed using different layouts. When the --detail option is specified, each job property is displayed in a separate line, so the command output can be parsed by other commands and scripts. To retrieve data only for a particular job type, specify an additional option that corresponds to the job type.
Started: 2002/12/02 10:22:57 Status: waiting for resources JobID: 20021202000083 Partition: video_2 Type: migration Priority: n/a Started: 2002/12/02 10:23:07 Status: writing to medium JobID: 20021202000084 Partition: scanned Type: migration Priority: n/a Started: 2002/12/02 10:24:34 Status: writing to medium For another example of output of the command fsejob --list --detail, see ”Excerpt from an example output of the fsejob --list --detail command” on page 175.
To display status of a particular FSE job using graphical user interface, in the FSE Management Console window, in the object tree, select the Jobs container. A list of FSE jobs will be displayed in the right pane. Right-click the desired FSE job and then click Properties.
NOTE: Permissions, ownership, and dates change on recall. Creation time, modification time, and access time are set to the current time when the file has been recalled. Linux specific The ownership of the recalled older file generation is set to root. Windows specific The ownership of the recalled older file generation is set to the user, under whose account the FSE service is run. Example output of the fsefile --history command The following are examples of the fsefile --history command and its output.
Example of recall by date Linux specific # fsefile --recall /fse/part01/20030328/root/hsmdb.out --date "2003/03/28 16:22:28" --into /tmp Example of recall by file migration ID Windows specific C:\Program Files\Hewlett-Packard\FSE\bin>fsefile --recall i:\fse\part01\20030328\mylittledoc.doc --migid 268509426688 --into D:\tmp Each file must be recalled separately. You define a recall by specifying a filename or by using the argument combination --id PartitionName FileID, as shown in the examples below.
• Due to an underlying omniORB/CORBA limitation, the upper limit on the number of entries in the file list (set of files to be recalled) is 26000. Efficient recall invocation The fsefile command provides options that are used to efficiently recall sets of files from FSE media. fsefile reads list of files to be recalled either from the file specified on the command line or from standard input.
Example excerpt from the FSE error log file [E HSM(/mnt/fsefs01) ,001030.3042864048 2005/08/26 12:51:32.296; root@fse_client.company.com; i_HSM_i::EfficientRecall, i_hsm_impl.cpp,v 6.9 :705] Recall failed. JobID: 0. HSM Job Idx: 0Files have not a usable copy. Inode FileID Path relative to mount point 0x17bca3e000044002 30 dir1/1/fn__1 0x17bca3e100044003 31 dir1/1/fn__2 0x17bca3e200044004 32 dir1/1/fn__3 0x17bca3e300044005 33 dir1/1/fn__4 Execution stack dump: 00000001.
Retrieving detailed information using command-line interface FSE command-line interface (CLI) includes several commands and options that you can use to retrieve information about the FSE. This information indicates, for example, the parameters of configurable FSE entities, their current states, their usage information, and so on. The scope of the retrieved information depends on the type of entity under consideration. The entities and their monitoring commands are listed in the following table.
VendorID: HP FirmwareRev: 0430 ScsiID: id:0:0:5:0 LoadNr: 26 UnloadNr: 22 Key: 1 Name: lib02 Family: LTO Type: SCSI Host: fse_server.company.com CtrlDev: /dev/sg4 Status: disabled ProductID: MSL6000 Series SerialNo: 3G2BLBD358D0 VendorID: HP FirmwareRev: 0430 ScsiID: id:0:0:4:0 LoadNr: 141 UnloadNr: 138 Key: 2 Name: libLTO01 Family: LTO ...
UsageCount: 137 Capabilities: 0 Key: 1 Name: drv02 Family: LTO Library: lib01 Host: fse_server.company.com CtrlDev: /dev/sg2 Status: online Medium: 000008 VendorID: HP ProductID: Ultrium 2-SCSI SerialNo: HUL2K00119 FirmwareRev: F59W ScsiID: id:0:0:3:0 Index: 2 UsageCount: 20 Capabilities: 0 Key: 2 Name: drv_lto01 Family: LTO ...
UUID: 33d4ef52-7ec8-4926-97f9-909edfe66c41 Family: DISK Type: Regular Partition: part01 PartUUID: 891af3c2-0aa9-4cc2-bd57-b16140691f31 BlkSize[KB]: 64 #Vol: 1 VSize[MB]: 500 SysVol: None SysVol[MB]: n/a Key: 2 Pool: pool03 UUID: 47b5fddc-03f7-4b80-8dab-5d2f6fe18cc1 Family: LTO ...
SlotType: repository DrvName: Status: free Location: lib:lib01:7 SerialNo: 10000000000000000000000001987376 BlockSize: 65536 #Vol: 1 SysVolNo: n/a SysVolSize: n/a CurVol: 1 Key: 6 Barcode: 656701 Type: UNKNOWN Pool: LibName: lib01 SlotAddr: 5 SlotType: repository DrvName: Status: uninitialized Location: lib:lib01:5 SerialNo: BlockSize: 0 #Vol: 0 SysVolNo: 0 SysVolSize: 0 CurVol: 1 Key: 5 Barcode: dm01 Type: disk Pool: pool02 LibName: SlotAddr: SlotType: DrvNam
BlockSize: 65536 #Vol: 50 SysVolNo: 0 SysVolSize: 0 CurVol: 33 Key: 11 Barcode: dm02 Type: disk Pool: pool02 LibName: SlotAddr: SlotType: DrvName: Status: open Location: n/a SerialNo: 58:5 BlockSize: 65536 #Vol: 50 SysVolNo: 0 SysVolSize: 0 CurVol: 12 Key: 12 ...
WriteTime: 1970/01/01 01:00:00 OwriteTime: 2004/03/01 17:41:05 LastVer: 1970/01/01 01:00:00 ValidData: 0 TotalData: 0 Barcode: 000009 VolNum: 2 UUID: eca50ba5-f068-4650-a587-945fcffda287 Type: data Size[MB]: 15815 Avail[MB]: 15815 Used[%]: 0 Status: empty AccessNum: 0 WriteNum: 0 OwriteNum: 0 InitTime: 2004/06/18 12:01:12 AccessTime: 1970/01/01 01:00:00 WriteTime: 1970/01/01 01:00:00 OwriteTime: 2004/03/01 17:41:05 LastVer: 1970/01/01 01:00:00 ValidData: 0 TotalData: 0
OwriteTime: 2004/03/01 17:41:05 LastVer: 1970/01/01 01:00:00 ValidData: 0 TotalData: 0 Barcode: 000009 VolNum: 4 UUID: b8e4d73b-890c-41d0-bb20-eefe19d3f057 Type: data Size[MB]: 17391 Avail[MB]: 17391 Used[%]: 0 Status: empty AccessNum: 0 WriteNum: 0 OwriteNum: 0 InitTime: 2004/06/18 12:01:47 AccessTime: 1970/01/01 01:00:00 WriteTime: 1970/01/01 01:00:00 OwriteTime: 2004/03/01 17:41:05 LastVer: 1970/01/01 01:00:00 ValidData: 0 TotalData: 0 Barcode: 000007 VolNum: 1 U
LastVer: 1970/01/01 01:00:00 ValidData: 0 TotalData: 0 ... Excerpt from an example output of the fsepartition --list --detail command Partition: part01 UUID: 891af3c2-0aa9-4cc2-bd57-b16140691f31 Status: mounted Host: fse_host.company.com Mountpoint: /mnt/fsefs01 FSID: 58:3 Options: regular MigIdTime: 1088756689 MigIdCount: 0 Activity: reorg scan in progress Partition: part2 UUID: bfca4697-a1e0-4881-ba12-953acbb72a02 Status: ready Host: fse_host.company.
Excerpt from an example output of the fsefile --list --detail command FileID: 752194 Status: offline FileName: bigfile FileID: 752195 Status: online FileName: bigfile1 FileID: 42110 Status: online FileName: dir18 FileID: 42111 Status: online FileName: dir19 FileID: 12 Status: offline FileName: linux-2.4.19-64GB-SMP.tar.
Excerpt from an example output of the fsejob --list --detail command JobID: 20031225000001 Partition: partLTO Type: recall Priority: n/a Started: 2003/12/25 18:21:00 Status: reading from medium JobID: 20031225000002 Partition: partLTO Type: recall Priority: n/a Started: 2003/12/25 18:21:00 Status: reading from medium JobID: 20031225000003 Partition: partLTO Type: migration Priority: 4001 Started: 2003/12/25 18:21:00 Status: waiting for resources JobID: 20031225000004 Partitio
To retrieve basic information about a particular FSE resource or FSE job, in the FSE Management Console window, in the object tree, select the appropriate container. The retrieved details will be displayed in the belonging row in the right pane. Figure 33 Retrieving basic information for an FSE drive To retrieve detailed information about a particular FSE resource, in the object tree, select the appropriate object. The retrieved details will be displayed on a property page in the right pane.
Benefits of extended FSE disk buffer In an FSE implementation where FSE disk buffer size is insufficient and causes slow or delayed FSE job execution, extending storage space of the FSE disk buffer can speed up the FSE operation.
For details on restarting, see the following content (depending on the platform of a particular FSE host): • FSE installation guide for Linux, chapter ”Installing FSE software”, section ”Starting the FSE implementation”, subsection ”Restarting FSE processes”. • FSE installation guide for Windows, chapter ”Installing FSE software”, section ”Starting the FSE implementation”, subsection ”Restarting FSE processes”.
Linux specific The FSE check log file is named check_hsmfs_fsc_PartitionName.log and located in the directory /var/opt/fse/log. Windows specific The FSE check log file is named check_hsmfs_fsc_PartitionName.log and located in the directory %InstallPath%\var\log. The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE. For each inconsistent file entry, the full file path is reported to the FSE check log.
• Run consistency check of the File System Catalog and enable automatic correction. For details, see Consistency check of FSC vs. FSE media, page 182. The following mismatches may also result from the above condition. • suspicious A file is reported as suspicious when it exists on the HSM file system but does not have a file ID. It is possible that the FSE implementation is unaware of its existence.
[2003/12/03 15:09:52] 77174 273931007488 /mnt/fsefs_01/A4/linux-2.4.19-ea/REPORTING-BUGS DUMP: FSC MigID 273931007488 HSMFS Notes 273931007488 FileSize 2815 FileName REPORTING-BUGS REPORTING-BUGS 57896 57896 OwnerID 0 MISMATCH [2003/12/03 15:11:51] 578672 273989682176 /mnt/fsefs_01/A1/linux-2.4.
Consistency check of FSC vs. FSE media Consistency check of the File System Catalog against FSE media matches file information from the Data Location Catalog and Name Space Catalog, which are parts of the File System Catalog, with Fast Recovery Information (FRI) on the belonging FSE media. This process can be executed either for all FSE media related to the specified FSE partition or to particular FSE media or FSE medium volumes.
• To start the consistency check of a FSC with particular FSE media or particular FSE medium volumes on the same medium, using FSE command-line interface, run the following command: fsecheck --fsc-media --barcode Barcode... [--volume VolumeNumber...] [--autocorrect [--match-media]] The argument Barcode... contains one or more barcodes for FSE media that should be checked. These FSE media must be related to the same FSE partition. The option --volume introduces the argument VolumeNumber...
Inconsistency types Reported inconsistencies in the FSE check log may be the following: • missing in FSC A file is reported as missing in FSC when its file ID exists in Fast Recovery Information, but not in FSC. To resolve the problem, do one of the following: • Run the File System Catalog recovery. For details, see File System Catalog recovery, page 235. • Run consistency check of the File System Catalog and enable automatic correction. For details, see Consistency check of FSC vs. FSE media, page 182.
MedVolOffset 472 FileType 1 FileSize 41643 MigSize 41643 MigFlags HD SplitFlags L SplitOffset 0 SplitSize 41643 FileName MAINTAINERS OldFileName OwnerID 154336 OldOwnerID 0 [2003/12/03 15:29:18] 578684 274037495040 NewFile Missing in FSC.
[2003/12/03 15:29:47] Finished FSC chk vol num: 3 (checked: 73599, missing in FSC: 0, missing on media: 0, mismatch: 0, no NSC: 0) [2003/12/03 15:29:47] Started FSC chk vol num: 4, medium: dm001 [2003/12/03 15:29:55] Finished FSC chk vol num: 4 (checked: 67326, missing in FSC: 0, missing on media: 0, mismatch: 0, no NSC: 0) [2003/12/03 15:29:55] Started FSC chk vol num: 5, medium: dm001 [2003/12/03 15:30:03] Finished FSC chk vol num: 5 (checked: 75999, missing in FSC: 0, missing on media: 0, mismatch: 0, no
• /var/opt/fse/part • /var/opt/fse/log The HHM default behavior can modified by adjusting the HHM configuration file residing on the consolidated FSE system or FSE server. Windows specific default behavior By default, HHM monitors the FSE system directories on HHM server and its clients, but triggers no action, regardless of the threshold crossing events.
E-mail settings The e-mail settings section is marked by the [mail] string in the configuration file and defines the e-mail recipients related information. The e-mail section settings are relevant only if any of the actions defined in the triggered actions definitions section of the configuration file is either email_ge or email_lt. For more information on triggered actions definitions, see section ”Triggered actions definitions” on page 188.
The actions are named and defined as CLI commands and are not provided as “event-action” pairs. Any action defined within this section can be assigned to any threshold event in the monitored items definitions section of the configuration file. For more information on the monitored items definitions, see section ”Monitored items definitions” on page 190. IMPORTANT: The names email_ge and email_lt are reserved for two built-in actions with the same name.
Triggered actions section of the configuration file [action] # action contains available actions to be taken in case a threshold is crossed. # Two actions are predefined and will be used to send e-mails in case # the threshold is crossed upwards (email_ge) or downwards (email_lt) # free defined actions may follow here. Note that the action is called in # a shell environment. The logfile will contain the output of the command.
NOTE: If current usage of the monitored file systems cannot be determined for some reason, HHM will assume 100% usage and the action defined for the critical threshold will be executed. If a threshold is crossed for the monitored item on a HHM client, the assigned action is executed on: • the same HHM client if the HostName parameter is not specified in the triggered actions definitions section of the configuration file.
Linux specific — monitored items definitions section of the configuration file [sysvar] filesystemtype: SYSTEM mountpoint: %%var%% CriticalThreshold: 95 CriticalActionGE: email-ge CriticalActionLT: email-lt WarnThreshold: 80 WarnActionGE: email-ge WarnActionLT: NONE [syspart] filesystemtype: SYSTEM mountpoint: %%var%%/part CriticalThreshold: 95 CriticalActionGE: email-ge CriticalActionLT: email-lt WarnThreshold: 80 WarnActionGE: email-ge WarnActionLT: NONE [syslog] filesystemtype
#CriticalActionGE: umount-ge, email-ge #CriticalActionLT: mount-lt, #WarnThreshold: 90 #WarnActionGE: email-ge #WarnActionLT: email-lt email-lt # in the above case, HHM will do the following if: # - warning threshold is crossed upwards --> send an e-mail # - critical threshold crossed upwards --> send an e-mail, umount hsmfs # - critical threshold crossed downwards # - warning threshold is crossed downwards --> send an e-mail --> send an e-mail, mount hsmfs # The same rules are valid, if
Windows specific — monitored items definitions section of the configuration file [sysvar] filesystemtype: SYSTEM mountpoint: %%var%% CriticalThreshold: 95 CriticalActionGE: email-ge CriticalActionLT: email-lt WarnThreshold: 80 WarnActionGE: email-ge WarnActionLT: NONE [syspart] filesystemtype: SYSTEM mountpoint: %%var%%\part CriticalThreshold: 95 CriticalActionGE: email-ge CriticalActionLT: email-lt WarnThreshold: 80 WarnActionGE: email-ge WarnActionLT: NONE [syslog] filesystemty
#CriticalActionGE: umount-ge, #CriticalActionLT: mount-lt, #WarnThreshold: 90 #WarnActionGE: email-ge #WarnActionLT: email-lt email-ge email-lt # in the above case, HHM will do the following if: # - warning threshold is crossed upwards --> send an e-mail # - critical threshold crossed upwards --> send an e-mail, umount hsmfs # - critical threshold crossed downwards # - warning threshold is crossed downwards --> send an e-mail --> send an e-mail, mount hsmfs # The same rules are valid, if
FSE log files FSE log files provide a variety of information on FSE events, such as error and status messages, details on FSE job execution, and resource allocation. You can use them to monitor FSE operations, for statistical purposes as well as for backup tracing. The FSE log files are the following: • FSE event log (fse.log) This log records relevant information on FSE-process events that occur during FSE operation. • FSE error log (error.log) This log records error messages generated by FSE processes.
Monitoring and analyzing FSE log files To monitor and analyze FSE error logs, use Log Analyzer. Log Analyzer allows you to monitor FSE error logs in specified time intervals, analyze them based on patterns which are configurable for each individual log file, and send the results of the analysis to specified e-mail addresses or as SNMP traps. The results are collected in a package and sent by e-mail at configurable time intervals. The log files analyzed by Log Analyzer depend on the platform.
To meet your requirements for monitoring the FSE event log file, adjust the parameters in the loganalyzer.config general configuration file and set the analysis rules in the loganalyzerrules.py rules configuration file. Similarly, to meet your requirements for monitoring the Linux system event log file, adjust the parameters in the loganalyzer_messages.config general configuration file and set the analysis rules in the loganalyzerrules_messages.py rules configuration file.
# === USED TERMS === # AnalysisFile: The file which is analyzed by the LogAnalyzer. # Package: The file, which is produced by the LogAnalyzer. # This file will be attached to the email.
# NOTE: You need to provide an absolute path! ActivityLog = /var/opt/fse/log/loganalyzer_messages.log # --- AnalysisFile --# The file which is analyzed by the LogAnalyzer. AnalysisFile = /var/log/messages # --- ProcessFileFromBeginning --# ProcessFileFromBeginning = 1 the beginning. ( For debugging purposes ). # ProcessFileFromBeginning = 0 [DEFAULT] the end ( Waiting for new messages ).
MailText = This is another package from your LogAnalyzer ----------------wbr Your LogAnalyzer ----------------- # --- Subject --# Subject-Part of the mailheader. Subject = # --- Subject Prefix --# Used for MailCurrentLogMessage("Sample Subject"). # If used, this text will be prepended to the text "Sample Subject". #SubjectPrefix = # --- From --# From-Part of the mailheader. From = # --- To --# To-Part of the mailheader.
# Default is 1 minute. Slot = 1 [SNMP] # =============================================================================== # This section describes the settings for the SNMP-features of the LogAnalyzer # =============================================================================== # --- DestinationIP --# The IP-Address of the management unit. # DestinationIP = # CommunityName # The community name of the machine.
Parameter Description ProcessFileFromBeginning Analysis mode: 0 = analyze only the newly-added log messages (default) 1 = analyze the entire log file (for debug purpose) IgnoreCase Case sensitivity of the regular expressions in the rules files: 0 = regular expressions are case sensitive (default) 1 = ignore case in regular expressions ServerName Name or IP address of the SMTP server UseSMTPAuthentication Authentication mode: 0 = disables authentication (default) 1 = enables authentication If SMTP a
The rules configuration file Rules configuration files reside in the directory /etc/opt/fse (on Linux systems) or in the directory %InstallPath%\etc\ (on Windows systems), and contain one or more rules, each rule consisting of the following: • A filter, based on which Log Analyzer monitors the log file specified in the general configuration file. Filters are specified as regular expressions, so that Log Analyzer matches them against the FSE log file entries.
'filter': ".*", 'exclude': "(^\[.{7}\]\[E\]|End of stack dump|^$)", 'action': "SaveCurrentLogMessage()" }, { 'filter': ".*So what.
Action Name Description SaveCurrentLogMessage() The current filtered log message is saved to the temporary file in the directory specified in the general configuration file. DisableActions() All subsequent actions are disabled. EnableActions() Actions are enabled. SendTrap(trap) The current filtered log message is sent as an SNMP trap trap.
The -v option displays the Log Analyzer version. The ConfigFile argument sets the general configuration file to be used. Linux specific To start Log Analyzer to monitor the FSE event log file (/var/opt/fse/log/fse.log), execute the following command: # /opt/fse/lbin/loganalyzer /etc/opt/fse/loganalyzer.config & To start Log Analyzer to monitor the system event log file (/var/log/messages), execute the following command: # /opt/fse/lbin/loganalyzer_messages /etc/opt/fse/loganalyzer_messages.
Create time : 2004/06/17 13:55:07 ============================================= [2004/06/17 13:55:07, linux_fse_server, 009779, SVC, , 0, ] Started (3.0.145 "Beta_02"). [2004/06/17 13:55:07, linux_fse_server, 009787, RMG, , 0, ] Started (3.0.145 "Beta_02"). [2004/06/17 13:55:07, linux_fse_server, 009799, MIF, , 0, ] Started (3.0.145 "Beta_02"). [2004/06/17 13:55:08, linux_fse_server, 009817, LAS, lib01, 0, ] Started (3.0.145 "Beta_02").
[2004/06/17 14:18:52, linux_fse_server, 009817, LAS, lib01, 20040617000002, 000046] Move: D=2 -> S=9 [2004/06/17 14:18:57, linux_fse_server, 009817, LAS, lib01, 20040617000002, 000007] Move: D=2 <- S=7 [2004/06/17 14:19:10, linux_fse_server, 010445, BEA, /dev/sg2, 20040617000002, 000007] Vol 4: Changing to volume 3... [2004/06/17 14:20:06, linux_fse_server, 010445, BEA, /dev/sg2, 20040617000002, 000007] Vol 3: Rewind...
[2004/06/17 14:23:08, linux_fse_server, 009799, MIF:ADMIN, , 20040617000004, 000009] WARNING: Aborted. [2004/06/17 14:23:08, linux_fse_server, 009799, MIF:JOB , , 20040617000004, ] JOB 20040617000004 ABORTED. [2004/06/17 14:23:13, linux_fse_server, 009817, LAS, lib01, 20040617000005, 000010] Move: D=1 <- S=10 [2004/06/17 14:23:13, linux_fse_server, 009799, MIF:ADMIN, , 20040617000005, 000010] WARNING: Aborted.
[2004/07/06 15:28:02, windows_fse_server, 000560, BEA, id:3:0:1:0, 20040706000006, PFY228L2] Vol 1: Writing filemark(s) 1... [2004/07/06 15:28:03, windows_fse_server, 000560, BEA, id:3:0:1:0, 20040706000006, PFY228L2] Vol 1: volume initialized. [2004/07/06 15:28:04, windows_fse_server, 001704, MIF:ADMIN, , 20040706000006, PFY228L2] Finished. [2004/07/06 15:28:04, windows_fse_server, 002688, BEA, id:3:0:1:0, 20040706000005, PFY228L2] Vol 1: Changing to volume 1...
[2004/07/08 16:20:56, windows_fse_server, 002672, CLI:PARTITION, , 0, part1] Remove operation confirmed. [2004/07/08 16:20:56, windows_fse_server, 001712, FSE, , 0, ] Configured volume Volume{6fb11c3a-bed1-11d8-b4f4-000e7ff13328} not found. Will be removed from registry. [2004/07/08 16:20:56, windows_fse_server, 001704, MIF:PARTITION, , 0, part1] Partition removed. [2004/07/08 16:20:57, windows_fse_server, 002068, PMG, part1, 0, ] Stopped (3.0.165 "Beta_02").
Execution stack dump: 00000002. mif_CfgRep::ReplaceClients mif_cfgrep.cpp,v 5.3:449 00000001. i_ManagementInterface_i::AddPartition i_mif_partition.cpp,v 5.13:803 End of stack dump. [W MIF() ,009799.011269 2004/06/17 14:11:30.983; root@linux_fse_server.company.com; mif_CfgRep::ReplaceClients, mif_cfgrep.cpp,v 5.3 :449] Host: "linux_fse_server" resolved to FQDN: "linux_fse_server.company.com" Execution stack dump: 00000002. mif_CfgRep::ReplaceClients mif_cfgrep.cpp,v 5.3:449 00000001.
End of stack dump. [E MIF() ,009799.016390 2004/06/17 14:17:56.861; root@linux_fse_server.company.com; log_EventLogger::Write, log_eventlogger.cpp,v 5.7 :223] [2004/06/17 14:17:56, linux_fse_server, 009799, MIF:ADMIN, , 20040617000001, 000006] ERROR: Problems with Back End Agent (6004: Caught Corba exception. 'CORBA::COMM_FAILURE#IDL:omg.org/CORBA/COMM_FAILURE:1.0'). Execution stack dump: 00000005. log_EventLogger::Write log_eventlogger.cpp,v 5.7:223 00000004. i_AdminJob_i::StartBEA i_adminjob_impl.
[W LAS(lib01) ,009817.006149 2004/06/17 14:17:57.203; root@linux_fse_client.company.com; la_SCSILibrary::EjectMedium, la_scsi.cpp,v 5.4 :507] Medium eject attempted on non-configured drive index: 2 Windows specific ============================================= FSE Error log file Create time : 2004/07/06 15:20:09 ============================================= [W RMG() ,001696.001912 2004/07/06 15:20:09.156; SYSTEM@windows_fse_server.company.com; log_EventLogger::Write, log_eventlogger.cpp,v 5.
End of stack dump. [W RMG() ,001696.001912 2004/07/06 15:20:09.171; SYSTEM@windows_fse_server.company.com; log_EventLogger::Write, log_eventlogger.cpp,v 5.7 :219] [2004/07/06 15:20:09, gtl-host, 001696, RMG, , 0, ] WARNING: Medium (BC:PFY229L2) has no usable volumes for migration Execution stack dump: 00000006. log_EventLogger::Write log_eventlogger.cpp,v 5.7:219 00000005. rm_Operator::SelectMediumVolumeMig librmdb.exx,v 5.11:10301 00000004. rm_Operator::SelectMediumForMig librmdb.exx,v 5.
00000006. log_EventLogger::Write log_eventlogger.cpp,v 5.7:219 00000005. rm_Operator::SelectMediumVolumeMig librmdb.exx,v 5.11:10301 00000004. rm_Operator::SelectMediumForMig librmdb.exx,v 5.11:10556 00000003. rm_Operator::AllocateMig librmdb.exx,v 5.11:10860 00000002. rm_DBThread::Run rm_dbthread.cpp,v 5.0:99 00000001. cmn_Thread::run cmn_thread.cpp,v 5.3:355 End of stack dump. [E MIF() ,001704.002232 2004/07/08 16:05:10.171; SYSTEM@windows_fse_server.company.com; ::yylex, cfg_l.cpp,v 5.
Determining total offline storage capacity You can determine the total amount of offline storage capacity in an FSE implementation using Licence Checker. Licence Checker is a command-line tool that eases reporting of the following status information pertaining to a particular FSE implementation: • Number of FSE clients configured to communicate with the FSE server. • Names and addresses of each FSE client. • Total offline storage capacity managed by the FSE server.
7 Backup, restore, and recovery This chapter includes the following topics: • Introduction, page 219 • Defining your backup strategy, page 219 • Backup, page 221 • Restore, page 230 • Recovery, page 235 Introduction There are several methods that you can use to increase safety of data in the FSE implementation. Nevertheless, if data loss occurs, FSE includes features which help recreating data that has been lost in disaster on any component of the FSE implementation.
Additionally, a well defined FSE backup policy with regular backups prevents the excessive growth of the transaction log files of the Hierarchical Storage Management Databases (HSMDBs) and File System Catalogs (FSCs). The transaction log files are committed into the main databases during the FSE backup process. Recovery methods for FSE components The following table lists FSE components, their descriptions, and provides notes on possible recovery methods.
FSE component Description Recovery method Global configuration files on FSE server and FSE clients Global configuration files are customizable files that hold FSE implementation-wide or FSE client-wide configuration. • Linux specific On Linux platform, global configuration files are located in the directory /etc/opt/fse. Windows specific On Windows platform, global configuration files are located in the directory %InstallPath%\etc.
How it works? You need to ensure that the prerequisites are met and consider the backup process, before you start backing up the FSE implementation. Prerequisites Common • If tape will be used as a backup destination, backup FSE media pool and backup FSE media have to be configured. For more information on the configuration procedures, see section ”Backup media management” on page 223.
# MODULES_LOADED_ON_BOOT="dm-snapshot" The backup process The following steps describe FSE backup process in general. Except for the volume snapshot creation, which is platform specific, the process is the same on all platforms: 1. FSE server is internally suspended. Operation of all running Partition Managers is temporarily blocked, in order to achieve a consistent state of all File System Catalogs (FSCs) and other entities that will be backed up. 2.
Configuring a backup media pool A special media pool, called backup FSE media pool, must be configured before FSE sever can be backed up. The Backup value for the PoolType variable in the media pool configuration file makes FSE distinguish backup media pool from ordinary media pools. Backup media pool can include any of the supported tape media families. Nevertheless, to enable media recycling, it is not allowed to use WORM tape media for backup storage.
For more information on formatting, and initializing media, see sections ”Formatting media” on page 102 and ”Initializing media” on page 103, respectively, and the fsemedium man page. Backup media recycling Backup FSE media with old backup images eventually become no longer useful, as the contained data become out-of-date. If these media are recycled, they can be used for storage of new backup images, and thus dismiss the need for adding new media to the backup FSE media pool.
NOTE: HP recommends that you perform your backups on a daily basis. Starting FSE backup The fsebackup command To start FSE backup, run the fsebackup command on the FSE server. After fsebackup invocation, a backup job is created in the FSE implementation, which performs all steps of the backup process.
Example of backing up to tape The FSE backup image will be stored on tape medium in the backup FSE media pool: fsebackup Example of backing up to disk and tape The FSE backup image will be stored in the current directory on disk, and on tape medium in the backup FSE media pool. Name of the backup image file will be automatically constructed according to the template: fsebackup --file . Activity logging FSE backup process reports its progress to the backup.log file.
[9428 2005/10/28 12:24:35] Finished deleting of HSMDB journal files for all partitions. [9428 2005/10/28 12:24:35] Volume /dev/vg_fse/lvolfsepart: Creating snapshot... [9428 2005/10/28 12:24:36] Volume /dev/vg_fse/lvolfsepart: Snapshot created. [9428 2005/10/28 12:24:36] Volume /dev/vg_fse/lvolfsevar: Creating snapshot... [9428 2005/10/28 12:24:36] Volume /dev/vg_fse/lvolfsevar: Snapshot created. [9428 2005/10/28 12:24:36] Unsuspending partition... [9428 2005/10/28 12:24:36] Partitions fs1: Unsuspended.
[1204 2005/10/28 11:50:16] Partition part1: HSMDB journal files deleted. [1204 2005/10/28 11:50:16] Suspending partition... [1204 2005/10/28 11:50:42] Partition part2: Suspended. [1204 2005/10/28 11:50:42] Deleting HSMDB journal files... [1204 2005/10/28 11:50:43] Partition part2: HSMDB journal files deleted. [1204 2005/10/28 11:50:43] Suspending partition... [1204 2005/10/28 11:51:11] Partition part3: Suspended. [1204 2005/10/28 11:51:11] Deleting HSMDB journal files...
[1204 2005/10/28 11:52:53] Partitions part3: FSC journal files deleted. [1204 2005/10/28 11:52:53] Finished deleting FSC journal files for all partitions. [1204 2005/10/28 11:52:53] Set operation COMPLETE. [1204 2005/10/28 11:52:53] Backup job 20051028000218 is removed [1204 2005/10/28 11:52:53] Backup finished. Aborting FSE backup To abort an ongoing FSE backup process, you need to abort the backup job. In certain cases, aborting the fsebackup command itself is also required. Perform the steps: 1.
Prerequisites Common • On the consolidated FSE system or FSE server, Firebird SuperServer must be running. • Processes on all hosts of the FSE implementation must be stopped. For stopping FSE processes on a particular host, locally run the fse --stop command. Note that the processes must be stopped on the external FSE clients first. • Backup FSE medium that contains the desired backup image must be loaded into an FSE drive.
Starting FSE restore The fserestore command To start FSE restore, run the fserestore command on the FSE server. The command performs all steps of the restore process.
The FSE restore process will not perform actual restore, but will only copy the second and third most recent backup images stored on tape medium to the FSE disk buffer on the FSE server. The process will use the backup FSE medium loaded in the drive with control device /dev/sg1, and will report target location of the backup images in the FSE disk buffer.
[14477 2005/11/07 11:50:45] RMDB restored. [14477 2005/11/07 11:50:45] Deleting temporary files... [14477 2005/11/07 11:50:45] Deleting temporary files completed. [14477 2005/11/07 11:50:45] Restore finished. Windows specific The following FSE restore log file shows progress of an FSE restore process for which a backup image on tape was used as the restore source.
Aborting FSE restore CAUTION: HP strongly dissuades aborting the restore process in which actual restore of the FSE server is performed. If this happens, the FSE server may remain only partially restored, with inconsistencies in the entities that were backed up, which results in an incorrect FSE operation or no operation at all. Such circumstances may more than likely lead to a major data loss in the FSE implementation. Nevertheless, if the above happened, you are advised to contact HP technical support.
3. Enable the FSE partition. 4. Start the recovery of FSC: fserecover --fsc PartitionName For example, to start recovery of the File System Catalog for FSE partition fse_fs01 enter: fserecover --fsc fse_fs01 When running the fserecover --fsc command the following occurs: 1. If FRI is located on disk, which is the case for all data volumes that are not yet full, recovery is done from these temporary FRI files. 2.
3. Start the recovery of the HSM file system by invoking the following command on the consolidated FSE system or FSE server: # fserecover --hsmfs PartitionName Windows specific To recover an HSM file system on a Windows FSE client: 1. Using the fsepartition --show and mountvol commands, determine the corresponding volume name and run the following command: C:\>fse --umount MountPoint VolumeName 2. Format the HSM file system using Computer Management or the format command. 3.
• The object with the highest file ID is recreated on the HSM file system with its original name. • The objects with lower file IDs are recreated with modified names, conforming to the template. OriginalName_HSM_RECOVERY_FileID. The HSM file system filter will detect such recreations as object renaming events. FSE users should manually locate these objects and decide to either delete them, rename them, or move them to another location.
8 Troubleshooting This chapter includes the following topics: • Introduction, page 239 • FSE startup and shut-down problems, page 240 • Firebird server problems, page 244 • Communication problems, page 245 • SCSI problems, page 248 • Tape library problems, page 250 • Media management problems, page 252 • HSM file system mounting problems, page 255 • HSM file system filter problems, page 258 • Migration problems, page 260 • Recall problems, page 263 • Backup, restore, and recovery problems, page 265 • Shari
FSE startup and shut-down problems Description After running the fse --stop command to shut down FSE, you may experience the following situation: - The fse command does not finish (does not return to the command prompt) in a reasonable time. - Local HSM file systems are still mounted. - Local FSE processes are still running. Explanation If a pre-stop script has been configured on this system, the above symptoms may indicate problems with commands in the script.
Description After invocation of the fse --start command on a particular FSE system host, some FSE processes that should be started on the host (consolidated FSE system, FSE server or external FSE client) are not running. (SUSE Linux Enterprise Server system specific) Explanation If the required FSE packages were installed on the host using a single rpm command, some of the packages may not have been installed correctly, due to unresolved dependencies.
Description Each time an FSE command is invoked, the following error message is displayed: ERROR 6008: Cannot connect to Management Interface - probably not running after system reboot. Requested name does not yet exist in the NameService. Giving up. Explanation The local services.cfg file contains the fully qualified domain name (FQDN) of the FSE server host, where MIF is running. If the primary DNS suffix of this host is not set appropriately, the full computer name may contain only the host name.
Description If a consolidated FSE system or FSE server has a power failure, external FSE clients are not automatically reconnected to the server after it is restarted. Explanation n/a Workaround Stop all FSE processes on the FSE server and all external FSE clients. Restart the services on the FSE server first, then restart the processes on the external FSE clients.
Firebird server problems Description While FSE processes are starting up, the following error is reported: ERROR: Firebird SuperServer (Interbase) is not up and running! Explanation Firebird is a third-party software used by the Resource Manager for managing resources in FSE implementations. Firebird software must be installed on the system before you can start FSE processes. For instructions on how to install the Firebird package, see the FSE installation guide for a particular platform.
Communication problems Description CORBA cannot be initialized. After invocation of an FSE command, the following error is reported: Consolidated FSE system or FSE server specific 6000 Cannot initialize CORBA ORB. 'CORBA::COMM_FAILURE' FSE client specific Failed to initialize CORBA Explanation Possible reason for this problem may be that omniNames is not running on the consolidated FSE system or FSE server and, consequently, its host is not accessible to the FSE client.
Description After invocation of an FSE command, the following error is reported: 6001 Problems accessing CORBA Naming Service. Explanation omniNames (CORBA Naming Service daemon (service)) is not running in the FSE implementation. Workaround Proceed as follows: 1. On the consolidated FSE system or FSE server, verify the status of the CORBA Naming Service daemon (service). Linux specific Run the command omniNames --status and inspect its output.
Description After running the fsefile command with any of the following options: --history, --recall, --migrate, --trigger-migration, the command fails and reports: 6004 Caught Corba exception. Explanation See Workaround. Workaround First, try to identify the problem as follows: 1. Find out the status of the FSE partition with the command: fsepartition --status PartitionName 2. Check if the fse-hsm process is running on the corresponding FSE client. Linux specific Use the command ps -ef | grep fse-hsm.
SCSI problems Description On the consolidated FSE system or FSE server, major problems with FSE drives are experienced. (Windows system specific) Explanation See Workaround. Workaround Check the FSE error log for the following message: Access to SCSI device is prevented. Make sure that the device driver is disabled. If this error message is logged, the cause of the problem with the FSE drive is the operating system driver for the tape drive, which has not been disabled.
Description Tape drives or other SCSI devices may not appear at boot time in the /proc/scsi/scsi file although they are connected properly. (Linux system specific) Explanation This problem is caused by the current generic SCSI driver in both supported Linux distributions, Red Hat Enterprise Linux and SUSE Linux Enterprise Server. Workaround Proceed as follows: 1. Add a call to the HP utility hp_rescan by adding the following line to the file /etc/rc.d/rc.
Tape library problems Description A tape library reports the same barcode in different locations. Explanation It could happen that the tape library reports the same medium barcode in a drive and in a slot at the same time. In this case FSE writes an error message to the FSE error log file, and marks both the medium and the drive as unusable. This happens when you manually move the medium from the drive to the slot, and then start the library inventory rescan.
Description A bad drive cannot be removed if a medium is still in it. Explanation When trying to remove a drive from the FSE implementation, the command fsedrive --remove Name reports a problem when trying to unload the medium, and the drive cannot be removed from the system. Workaround If the medium could not be unloaded, you must identify the reason and unload the medium manually.
Media management problems Description Initialization of media with large block size fails with the following error: ERROR: Medium 052007, volume 3: Init failed (12053: SCSI driver error. 'Buffer too large. See error.log for details.'). (Windows system specific) Explanation This problem occurs because some Windows SCSI or FS host bus adapter (HBA) drivers (particularly Adaptec and QLogic) do not support large block sizes by default.
Description If you remove an FSE medium from its FSE media pool without physically removing it from an FSE library, then add it to whichever FSE media pool, and finally start an FSE administrative job to format the medium, the job seems to be blocked before formatting actually begins. Explanation When an FSE administrative job commands the Resource Manager (RM) to allocate a particular FSE medium, the RM cannot execute the allocation unless it knows the medium’s physical location in the FSE library.
Description A reorganizational scan fails with the following message: "Volume {VOLID} has already status 73 set" Explanation This occurs when the reorganization has already finished writing to a target medium volume and was then aborted or terminated with the medium volume staying in the state recycled. Workaround Run the fsemedium --init command for recycled volumes. The source volume's content has already been copied to the target medium and the medium can be initialized.
HSM file system mounting problems Description On a system, where the /etc/fstab file contains an entry for the /fse/fsefs_01 file system with type hsmfs, running the mount /fse/fsefs_01 command produces the following error: # mount /fse/fsefs_01/ mount: wrong fs type, bad option, bad superblock on /dev/vg_fse/fsefs_01, or too many mounted file systems (Linux system specific) Explanation This error is reported if any of the following is true: • FSE daemons are not running.
Description While running the fse --mount command to mount an HSM file system, the command fails and reports an error. (Windows system specific) Explanation There are two possible causes for this problem: • Cause 1: NTFS has already attached to the disk volume that stores the HSM file system, and the HSM file system filter cannot gain control over it. This is the most probable cause for the described problem. • Cause 2: The mount point directory for the HSM file system is not empty.
Description After formatting an FSE partition, the FSE partition is not mounted and HSM does not start. (Windows system specific) Explanation This happens because formatting remounts the volume and the file system filter cannot attach to it. You need to unmount the partition first and only then mount it. Workaround To mount the FSE partition, proceed as follows: 1. Unmount the FSE partition on a client using fse --umount. 2.
HSM file system filter problems Description All ordinary file system operations can be performed on an HSM file system that is currently operating in Limited Access Mode. No file or directory creation, move, deletion, rename, and similar operations are prevented. (Windows system specific) Explanation Most probably the HSM file system filter cannot attach to the HSM file system, because the system native NTFS has already attached itself.
Description Free space on an HSM file system runs out and the following error is written to the kernel ring buffer (displayed by the dmesg command): [CRITICAL] FSE-HSM not listening for FS events! Unmount FS and check FSE logs for more details! (Linux system specific) Explanation n/a Workaround To solve this critical situation, proceed as follows: 1. Check free space on the HSM file system. Invoke the following command: df -h HSMFileSystemMountPoint 1. Stop the FSE implementation: fse --stop 2.
Migration problems Description All migrations are waiting for FSE disk buffer and data transfers to FSE media do not start. Explanation If FSE jobs are aborted, killed or they die for an unknown reason, it may happen that the temporary data which they create is not automatically removed, but is left in the FSE disk buffer, occupying valuable storage space. In such situations, FSE jobs will report FSE disk buffer as being full in the FSE event log.
Description After a migrated file is modified using a third-party application, its new generation is migrated using a different file ID. Explanation The cause for this problem is the way the third-party application is handling the modification process. Such application creates a copy of the current file first and then modifies the copy instead of the original. After the copy is saved, the application renames the copy with the original filename and thus replaces the original.
Description After starting FSE daemons (services) with the command fse --start, the following error appears in the FSE error log: ERROR: *** NO OWNER set for INO... *** File must be migrated again! fileID= file / Explanation While FSE daemons (services) on a particular FSE client are stopped and modifications are made on an HSM file system located on the client, it may happen that a file system event is lost.
Recall problems Description Device problem is reported after a failed recall. The error message logged in the fse-hsm_ProcessID.log file is the following: Linux specific device busy Windows specific 31: A device attached to the system is not functioning. Explanation See Workaround.
Description Rename or move actions on offline files cause file recalls, if performed over a network share from a Windows system with Norton AntiVirus installed. (Windows FSE client specific) Explanation Norton AntiVirus ignores the offline file attribute of files located on remote HSM file systems on the Windows FSE client. The rename or move operation causes the files to be recalled for scanning against viruses.
Backup, restore, and recovery problems Description At the end of an HSM file system recovery, some files are reported as being 'reserved' in the output of the fserecover command. Explanation The word reserved means that file IDs were assigned to new files on the HSM file system, but these files had been either deleted or changed before they could have been migrated using these particular file IDs. Such file IDs were only recorded in the File System Catalog.
Description After the FSE restore process successfully completed, FSE processes cannot be started on the consolidated FSE system or FSE server. Explanation The most probable cause of this problem is that some files on the consolidated FSE system or FSE server were either struck by disaster or accidentally deleted, and these files were not backed up during the FSE backup process. Note that FSE backup images do not include files from the FSE installation which are not updated during FSE operation.
Description After an FSE backup process is aborted or the FSE server is restarted due to power outage, new FSE backup cannot be started. Explanation The aborted FSE backup process did not remove the lock file backup.pid from the lock directory. This file is used to prevent starting another backup process while the first backup process is still running. Workaround After verifying that no other FSE backup process is running at the moment, manually remove the file backup.
Description FSE backup fails and the following entries are reported to the terminal window and logged in the backup.log file (the volume and partition names may be different): [2006/03/02 22:39:20] Volume /dev/vg_fse/part: Creating snapshot... [2006/03/02 22:39:20] ERROR: SnapshotsMgr error. [2006/03/02 22:39:20] Unsuspending partition... [2006/03/02 22:39:20] Partitions fs1: Unsuspended. [2006/03/02 22:39:20] Finished unsuspending for all partitions. [2006/03/02 22:39:20] Backup job stopped.
Sharing problems Description Using Microsoft Windows Services for UNIX (SFU), data on HSM file systems that are mounted to directories cannot be shared. (Windows system specific) Explanation Microsoft Windows Services for UNIX (SFU) cannot share mount points, but only whole disk volumes (drive letters) and regular directories. You cannot use SFU for sharing data on an HSM file system that is mounted to a regular directory or subdirectory in the NTFS file system tree.
Other problems Description Problems with detecting hardware such as SCSI cards or LAN cards. (Linux system specific) Explanation Problems when using an ASUS PR-DL533 motherboard: Linux does not detect hardware correctly. It is a known problem of the Linux ACPI subsystem that it does not work correctly with some motherboards. If specifying the general acpi=off option, it is possible that the ACPI subsystem does not detect the hardware (on-board network card, SCSI card) correctly.
Description Execution of the fsecheck --fsc-media command seems to complete successfully, however, the following message is written to the FSE event log: ERROR: FAILED creating new FRI... ...No space left on device. Explanation The file system that stores the temporary Fast Recovery Information files ran out of free space. Linux specific Fast Recovery Information is located in the directory /var/opt/fse/fri. Windows specific Fast Recovery Information is located in the directory %InstallPath%\var\fri.
Description The following error message is displayed in the FSE error log: [2005/03/02 10:39:56, fse_host, 002680, PMG, , 0, ] CRITICAL ERROR: File: fsc_nsDirEntry.lrf vector at index 6042669 of size 4 was released twice. A DB recovery needs to be done in near future. Pack DB and contact support for further steps. Explanation n/a Workaround Recover the File System Catalog (FSC) of the FSE partition PartitionName. For details, see ”File System Catalog recovery” on page 235.
Description Running the fsefile -l * command on the top directory of an HSM file system results in the following message: Warning: System Volume Information: cannot get file state (5: Access is denied. 'CreateFile'). (Windows system specific) Explanation The message appears because the directory System Volume Information is a special Windows directory which refuses access from other applications. Workaround The command completes successfully and you can ignore the message.
FSE troubleshooting tools FSE provides a set of commands and scripts called FSE tools, designed to perform advanced low-level FSE service and maintenance tasks. These commands communicate directly with FSE daemons (services) and FSE agents, and are usually used for low-level modifications and more extensive monitoring or troubleshooting tasks.
Tool name Description fse_fsc Navigates the File System Catalog (FSC) database. This tool is available on consolidated FSE system and FSE server only. dumprmdb.py Dumps the Resource Management Database into the file rmdb_dump_HostName_DateTime.txt in the current directory on the consolidated FSE system or the FSE server. This tool is available on consolidated FSE system and FSE server only.
Debugging options HSM_DEBUG_LEVEL This variable represents the level of detail that FSE processes will track in their debug files. Note that each debugging level includes all debugging information from lower levels. The following table represents the values the HSM_DEBUG_LEVEL variable can take, their corresponding levels, and their descriptions. Option value Debugging level Description L 1 — Low Very little trace information will be logged in files, producing less debug overhead and small debug files.
Example HSM_DEBUG_FILE=. Debugs can occupy a lot of disk space when troubleshooting an unpredictable problem. Linux specific On Linux platform, if you need more disk space than available in the /var/opt/fse/log/debug directory, change your working directory to /var/opt/fse/log and create a symbolic link to a location with more disk space.
HSM_DEBUG_FS The HSM_DEBUG_FS variable is a numeric value composed of several HSM file system filter debug flags. Set the flags if you want to trace specific information about the HSM file system filter driver. Flags are platform-dependent.
The following tables list the values that the variable can take: Option value Description none Do not leave any disk buffer files (default). corrupt Leave only corrupted disk buffer files on disk. all Leave all disk buffer files on disk. Example export HSM_DEBUG_DISKBUF=corrupt Operation fine-tuning options HSM_USE_MAM This debugging variable cannot be used with the fsesystem --enable-dbg command. It can only be specified in the trace.cfg file.
To use buffered SCSI pass-through mode, uncomment the following line in trace.cfg, or add it if necessary: export HSM_PT_DIRECT=no HSM_BEA_VERIFY_POSITION This debugging variable cannot be used with the fsesystem --enable-dbg command. It can only be specified in the trace.cfg file or used as environment variable. The variable enables tape medium position check functionality of the Back End Agent (BEA). This BEA feature decreases the probability of data loss in SCSI bus reset events.
In all other cases it will be assumed that the medium has already been verified by the previous operation. CAUTION: Data might get written to the wrong medium if it is exchanged manually and if an operation that sends medium change notification from SCSI subsystem to the FSE implementation has been performed. It is therefore hazardous to set this variable to “no” or “false” and perform manual drive operations later on.
1. Start the fsebackup or fserestore command, specifying the --dbg option. This will produce additional debug file for the process, besides the ordinary backup.log or restore.log. The debug file is named fse-backup-Number.log or fse-restore-Number.log, respectively, and created in the directory for the debug files. Linux specific The debug files are located in the directory /var/opt/fse/log/debug. Windows specific The debug files are located in the directory %InstallPath%\var\log\debug.
you describe a wrong problem, you might end up with the original problem still present after technical support have already put significant effort into investigating and finding a solution to another problem. Log and debug traces of the problem Reproduce the problem with debugging enabled — for details, see ”Debugging FSE processes” on page 275. Save the evidence, such as wrong command output or strange error messages, and send it for lab investigation. Note that without knowing the contents of the fse.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 6 Functionality not yet implemented. 7 String too long. 8 Argument missing. 9 Shouldn’t happen at all. 10 Invalid file name. 11 Mutex usage error. 12 Environment variable not set. 13 UUID can’t be parsed. 14 Timeout expired. warning warning 15 Pointer is NULL. 16 String contains spaces at the beginning or at the end or contains new lines. 17 Unknown Type. 18 std::exception. error critical 19 Aborted.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 2004 Event is empty or bad. critical info 2005 Object exists. 2006 Object is NULL or is not created. 2007 I-node is 0. critical info 2008 Device file is not found or device file name is bad. critical info 2009 No job list or list is empty. 2010 Hash is zero. Cannot create HSM cache. 2011 File has no file ID. 2012 One or more configuration parameters are wrong. error error 2013 Read error. 2014 Write error.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert Parser related errors 3000 Parser error. 3001 Unknown variable encountered in validation. 3002 Target class doesn’t have any subgroups. 3003 Input value not valid in conversion. 3004 Type not found in table of possible types. 3005 Missing entry for mandatory variable. 3006 Entry parsed too many times. 3100 Not enough parameters for option or command: . 3101 Syntax error: cannot resolve argument: .
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 4023 Invalid header position. critical info 4024 Invalid circular file header. error error 4025 Invalid journal file header. critical info 4026 Read after the end of relative file. Index overflow. 4027 Journal file missing. error error 4028 Unused vector already in buffer. Interprocess communication related errors 6000 Cannot initialize CORBA ORB. error error 6001 Problems accessing CORBA Naming Service.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 8004 Problems with Library Agent of this library. critical info 8005 Problems with Back End Agent. 8006 Problems with Resource Manager. 8007 Packer Error. 8008 Problems starting/stopping partition managers from MIF. critical info 8009 Remove Error. 8010 Update Error. 8011 Select Error. 8012 Semantic Error. critical info 8013 Syntax Error. 8014 Unknown Status.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 10002 FSC did not return any set of copies for specific file ID. critical info 10003 Job was removed. 10004 Job failed. critical info 10005 Not enough space for disk buffer. warning warning 10006 Registration of HSM is not allowed. 10007 Volume not found. 10008 No files packed for migration. 10010 HSM not registered. critical info 10011 HSM registered, but is not accessible.
Code Description Log Analyzer e-mail alert Log Analyzer SNMP alert 13002 Too many copies in the same generation. critical info 13003 No copies found. critical info 13004 Media volume number exceeds limit. 13005 No media volume file found. 13006 Successive change of data. 13007 Number of generations doesn’t match. 13008 No data. 13009 No NSC element object. 13010 FSC is busy. Possible: Reply from journal in progress. warning warning 13011 FSC DB error.
A Directory layout Directories created during the installation of FSE are presented in the following tables. Windows specific The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE.
Path on Linux platform Path on Windows platform Contents /var/opt/fse/log %InstallPath%\var\log All FSE log files /var/opt/fse/log/debug %InstallPath%\var\log\debug All FSE debug files /var/opt/fse/omniorb %InstallPath%\var\omniorb Files used by the CORBA/omniNames service /var/opt/fse/part %InstallPath%\var\part File System Catalog and Hierarchical Storage Management database and associated journals for each configured FSE partition /var/opt/fse/rmdb %InstallPath%\var\rmdb The Resource Man
B FSE configuration templates This appendix provides contents of default templates for the FSE configuration. The template location is platform-dependent. Linux specific Configuration templates are located in the directory /opt/fse/newconfig/. Windows specific Configuration templates are located in the directory %InstallPath%\newconfig. The value of %InstallPath% depends on the choice made in the FSE installation process. It defaults to C:\Program Files\Hewlett-Packard\FSE.
Template for FSE libraries [FSE Library] [Version = 1] [Date = ""] [User = ""] # # Specify a name for the tape libray and a hostname, where # it resides. Use full host name. # Name = "USER_DEFINED_NAME" Host = "LIBRARY_HOST_NAME" # # Control device is identification of library's (media changer's) # on the specified system.
[Slots] Slots = (1-15, 20) HP StorageWorks File System Extender Software user guide 295
Template for FSE drives [FSE Drive] [Version = 1] [Date = ""] [User = ""] # # Specify a name for the tape drive. # Name = "DRIVE_NAME" # # Specify the tape library name where the drive is placed # and the index of the drive in the library (IMPORTANT!) # Index of the first drive in the library always starts with 1.
# # Linux: /dev/sgN # Windows: id:P:B:T:L # Host "HOST_NAME" { ControlDevice = "CONTROL_DEVICE_FILE" } HP StorageWorks File System Extender Software user guide 297
Template for LTO media pools [[FSE Pool] [Version = 1] [Date = ""] [User = ""] # # Specify the name of the media pool. Media pools are # used by FSE partitions. # # NOTE: You have to assign media to pools with fsemedium command. # Name = "MEDIA_POOL_NAME" MediaFamily = LTO # Regular - regular media pool for FSE partition # WORM - media pool for LTO-3 WORM media # Backup - media pool for backup media PoolType = Regular # # Change the values of the on-tape format if the defaults don't suit you.
SysVolLocation = None SizeOfSysVol = 0MB HP StorageWorks File System Extender Software user guide 299
Template for disk media pools [FSE Pool] [Version = 1] [Date = ""] [User = ""] # # Specify the name of the media pool. Media pools are # used by FSE partitions. # # NOTE: You have to assign media to pools with fsemedium command. # Name = "MEDIA_POOL_NAME" MediaFamily = DISK PoolType = Regular # # Change the values of the on-tape format if the defaults don't suit you. # # NOTE: Some of the parameters will be used only if the tapes are # (re)formatted.
Template for FSE partitions - General part [FSE Partition] [Version = 1] [Date = ""] [User = ""] # # User defined name for an FSE partition. FSE partition # consists of: # * HSM file system # * media pool(s) # Name = "PARTITION_NAME" # # Where is the HSM file system? # Client "HSM_FS_HOST_NAME" { FileSystemID = "_HSM_FS_DEVICE_FILE_NAME_" /dev/mapper/hsmfs1, Linux client # e.g.
Template for FSE partitions - Migration part [Migration] # # The amount of time that a file must inactive to # become a candidate for migration. # MinFileAge = 10m # min=1s, max=2D [RegularMigration] # # Parameters that trigger a migration.
Template for FSE partitions - Release part [Release] # # Parameters that control the release of files on HSM file system: # # Release starts: # # * on HighWaterMark for files that expire retention and are # larger than MinFileSize # # * on CriticalWaterMark for all already migrated files # # * ReleaseInterval means how often the release conditions are checked.
# [abc] Set. # Matches a single character in the set. # If the dash '-' character is to be included, it must # immediately follow the opening bracket '['. # If the closing bracket ']' character is to be included, # it must be preceded by a quote '`' # [a-z] Range. # Matches a single character in the range 'a' to 'z' # Ranges and sets may be combined within the same set of # brackets (e.g. [a-c123] matches any character in the set # {a, b, c, 1, 2, 3}) # \ Quote (linux).
Template for FSE partitions - Recall and Allocation part [Recall] # # Recall will be cancelled if it can't be performed in RecallTimeOut.
Template for FSE partitions - Deletion part [Delete] # # FileExpiration age: min=1D max=UNLIMITED default=UNLIMITED # # Example: # FileExpiration 7D # { # # Path = (/usr/tmp, "\Temp\Temporary Internet Files") } # 306 FSE configuration templates
Template for FSE system configuration [FSE System] [Version = 1] [Date = "2002/01/01 01:01:01"] [User = ""] # # Resource Manager tries to allocate resources to # all jobs that have priorities in the range of # ResourceAllocationThreshold (in percentage) from # the job with the top priority. # ResourceAllocationThreshold = 50% # min=0%, max=100% # # The influence of time factor on job allocation. # TimeStep = 2 # min=0, max=10000 # # Influence of retries on the job priorities.
SystemMaxNumDrivesAdmin = 20 # min=0, max=64 SystemMaxNumDrivesRecovery = 20 # min=0, max=64 SystemMaxNumDrivesMaint = 20 # min=0, max=64 308 FSE configuration templates
Glossary This glossary defines terms used in this guide or related to this product and is not a comprehensive glossary of computer terms. administrative job (admin job) A process, executed by the Management Interface, which performs formatting and initialization of the FSE media. administrator (FSE administrator) A system administrator who installs, configures, monitors, and maintains an FSE implementation.
client (FSE client) A set of FSE daemons (services) responsible for managing the HSM file systems on the FSE partitions. These daemons (services) are: File System Event Manager, Hierarchical Storage Manager, and Service, with additional FSE Windows Service on Windows systems. See also ”external client (FSE external client)”. command (FSE command) An FSE command-line interface command that communicates with FSE system components through the Management Interface.
debug file (FSE debug file) A log file of a single FSE process, which records the execution trace of operations inside the process. FSE debug files are created only if tracing in FSE is enabled (typically on request of technical support personnel of FSE). FSE debug files are located in the directory /var/opt/fse/log/debug (on Linux platform) or in the directory %InstallPath%\var\log\debug (on Windows platform).
event log (FSE event log) A file that records relevant information on events happening in the FSE processes during the operation of the FSE implementation. FSE event log, named fse.log, is located in the directory /var/opt/fse/log (on Linux platform) or in the directory %InstallPath%\var\log (on Windows platform). explicit release Unconditional release, started for a file or set of files on the release candidate list, specified by an FSE user.
File System Event Manager A daemon (service) on the FSE client which receives notification about mount events for the HSM file system and triggers the startup of the appropriate Hierarchical Storage Manager. The File System Event Manager process name is fse-fsevtmgr. forced release Release of all files on an HSM file system, which are on the release candidate list, regardless of their retention time and file size.
HSM file system (HSM FS) A file system, controlled by the FSE software. It is used to store file metadata (name, attributes, permissions) and online files of an FSE partition. HSM file systems represent the primary storage space of an FSE implementation. On the Linux platform, HSM file systems are based on the Linux native Ext3 file system and FSE specific attributes are stored as Ext3 extended attributes. On the Windows platform, HSM file systems are based on the Windows native NTFS file system.
MAM See ”Medium Auxiliary Memory (MAM)”. Management Console (MC) An add-on to the basic FSE product that provides graphical user interface for most of the configuration, operation, monitoring, and management tasks in the FSE implementation. Management Console consists of two separate modules: Management Console server and Management Console client.
Network File System The file-sharing protocol used in UNIX networks. This is one of the protocols that can be used (NFS) for accessing files on an HSM file system from a remote machine, particularly a Linux client. offline file A file whose contents were migrated to one or more FSE media (according to the number of copies defined), and then removed from the HSM file system. The header and extended attributes of an offline file are left intact and remain on the HSM file system.
recall, recall job A process, executed by the Partition Manager, that copies the file contents from the FSE media to the HSM file system. Once the process is completed, the offline file becomes online again. Its file entry is then added to the release candidate list. recovery (FSE recovery) A process, executed by the Partition Manager, that recovers the HSM file system or the File System Catalog (FSC) without requiring FSE backup copies.
restore (FSE restore) A process that recreates complete and consistent state of the FSE server at a particular point in time, using data from the backup image. Restore is required after disaster causes data on the FSE server to be damaged or lost. restore log (FSE restore log) A file that stores information about activity of the FSE restore process. FSE restore log is named restore.
tool (FSE tool) A command that communicates directly with FSE daemons (services) and agents besides Management Interface, designed for low-level modifications and more extensive monitoring and troubleshooting tasks. FSE tools are only intended for use by experienced FSE users (typically on request of technical support personnel of FSE). Note that incorrect use of FSE tools can cause data corruption.
Index A access modes HSM file system 148 switching 156 switching, limitations 156 adding FSE Management Console users 59 adding media FSE media pools 102 agents 18 allocating media 224 architecture 11 audience 7 automatic deletion 137 automatic error detection 10 automatic low storage space detection 10, 186 B backup 221 backup configuration modifying, on Linux 222 backup frequency estimating 225 backup media management 223 backup media pools 92 configuring 224 backup media recycling 225 backup strategy de
default job priority policy 144, 145 default partition allocation policy 145 default recall policy 137 default release policy 136 default system allocation policy 144 defining backup strategy 219 deletion 13, 137 starting 140 deletion policies default 139 examples 139 deletion policy parameters 138 determining total offline storage 218 determining configured media pools 95 directory layout 291 dirty file lists 127 disabling FSE partitions 154 disk media 99 displaying FSE Management Console users 59 displayi
FSE log files 196 FSE backup problems troubleshooting 265 FSE command-line interface using 40 FSE command-line interface (CLI) retrieving detailed information 165 FSE commands examples 165 FSE configuration 37–90 before you start 37 configuring FSE drives 61 configuring FSE libraries 61 configuring FSE media pools 69 configuring FSE partitions 73 mounting HSM file systems 81 overview 61 preparing FSE media 79 preparing HSM file systems 70 templates 39 FSE configuration database 39 FSE configuration files 38
reorganizing media 107 status check 97, 110 FSE media types 99 FSE medium volumes 98 data volumes 98 FSE migration 127 dirty file lists 127 parameters 127 retried migration 127 FSE operation basics 123 policies 123, 126 FSE partition status checking 152 checking, examples 154 FSE partitions 13 disabling 154 enabling 155 File System Catalog (FSC) 14 information about 156 managing 152 removing 155 FSE policies configuring 40, 42 FSE problems, miscellaneous troubleshooting 270 FSE recovery introduction 235 FSE
HSM file system recovery 236 offline operational modes, FSE Management Console client 44 reorganizing media 107 switching access modes 156 Limited Access Mode (LAM) 148 error messages 157 listing FSE jobs 157 FSE jobs, examples 158 loading dm-snapshot module, on SUSE Linux 222 Log Analyzer configuring 197 introduction 197 reconfiguring 207 starting 206 logging 207 FSE media operations 207 low storage space automatic detection 10, 186 low watermarks 131 M maintaining FSE implementation 151 managing FSE part
parallel copying 147 parameters deletion policies 138 FSE migration 127 job priority policies 142, 144 partition allocation 144 recall policies 136 release policies 134 reorganizing media 109 system allocation 142 partition allocation policies default 145 parameters 144 performance disk 99 policies FSE migration 127 FSE operation 123, 126 post-restore steps 235 preparing FSE media 79 preparing HSM file systems 70, 71 prerequisites 236 FSE backup 222 FSE restore 231 HSM file system recovery 236 online operat
FSE Management Console GUI 57 status check media 117, 119 media pools 97 stopping FSE Management Console client 34 FSE Management Console server 28 Subscriber's Choice, HP 8 support contacting 282 supported drives 10 supported media 10 switching access modes 156 system allocation policies default 144 parameters 142 system volumes 98 T tape library problems troubleshooting 250 technical support HP 8 service locator web site 8 templates FSE configuration 39 toolbar FSE Management Console GUI 56 tools FSE tro
