HP-UX Event Manager Administrator’s Guide HP-UX 11i v3 Edition 1 Manufacturing Part Number : 5991-6660 February 2007 © Copyright 2007 Hewlett-Packard Development Company, L.P.
Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services.
Contents About This Document 1. Introduction Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Event Manager Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Event Manager Command Line Utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents 3. Configuring Event Manager Configuring Event Manager Daemon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Event Manager Channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring Event Manager Logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Secondary Logger Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Log Files . . . .
Contents 5
Contents 6
Tables Table 1-1. Event Manager Command-Line Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Table 1-2. Event Manager Administrative Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Tables 8
Figures Figure 1-1. Event Manager Component Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Figure 1-2. Event Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figures 10
About This Document This document describes Event Management (EVM) System. It includes information about configuring and troubleshooting Event Manager on HP-UX platforms. The latest version of this document is available at: http://www.docs.hp.com The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date.
Table 1 HP-UX 11i Release (Continued) Release Identifier B.11.31 Release Name HP-UX 11i v3 Supported Processor Architecture Intel Itanium, PA-RISC Document Organization The HP-UX Event Manager Administrator’s Guide is organized as follows: Chapter 1 Introduction Presents an overview of the Event Management system. Chapter 2 Using Event Manager Describes how to use the Event Management system.
ComputerOut Text displayed by the computer. UserInput Commands and other text that you type. Command A command name or qualified command phrase. Variable The name of a variable that you can replace in a command or function or information in a display that represents several possible values. [ ] The contents are optional in formats and command descriptions. { } The contents are required in formats and command descriptions.
1 Introduction This chapter introduces you to the Event Manager (EVM) system and the components it contains. It also discusses the features of EVM, how this system works, and Event Manager events.
Introduction This chapter addresses the following topics: 16 • “Overview” on page 17 • “Features” on page 17 • “Event Manager Events” on page 27 • “How Event Manager Works” on page 18 • “Event Manager Event Model” on page 28 Chapter 1
Introduction Overview Overview A critical part of UNIX system administrator’s job is to monitor the state of the system, and to be ready to take action when certain unusual conditions occur. Examples of such conditions are when a disk fill is full or a processor reports hardware errors. It is also important to verify that certain routine tasks run successfully each day, and to review certain system configuration values. Such conditions or task completions are known as system events.
Introduction Overview How Event Manager Works This section describes how the different components of Event Manager interact with each other. It also describes the system files used to run Event Manager and any files that are created by Event Manager during normal operations. Figure 1-1 illustrates the Event Manager component model.
Introduction Overview Passive event channels do not post events and must be polled for information. These channels are depicted by the log files handled by the monitor scripts. The primary component of the Event Manager is the evmd daemon, which is initialized when the system is booted to run level 2.
Introduction Overview The get server process, evmget_srv, is a transient (demand) process that executes event retrieval scripts for the various event channels. The evmd daemon runs an instance of evmget_srv whenever a user runs the evmget command. Entities on the left side of the model create posting connections to the daemon to post events.
Introduction Overview Table 1-1 Event Manager Command-Line Utilities Command Description evmpost Accepts a file or stream of text event sources and posts them to the daemon for distribution evmshow Accepts one or more events and outputs them in the specified format evmsort Reads a stream of events and sorts them according to the supplied criteria evmwatch Subscribes to events specified and outputs them as they arrive Table 1-2 lists the administrative commands, which are usually invoked during s
Introduction Overview Table 1-2 Event Manager Administrative Utilities Command Description evmlogger The daemon automatically starts the logger. The logger receives events from the daemon and writes them to each of the logs whose filter string they match. The evmlogger also serves as an event forwarding agent that you can configure to take an action when required. evmreload This command posts control events, which instruct the components to reload their configuration files.
Introduction Overview functions enable programs to post events, send requests and notifications to the daemon, or receive responses and information from the daemon. For more information about the APIs, see the EVM (5) manpage. Event Manager System Files Event manager creates or uses the following system file types: Executable Files The Executable files for Event Manager administrative commands are located in the /usr/sbin directory.
Introduction Overview This file is used to control access to events and event services. For more information about this file, see “Event Authorization” on page 73 and evm.auth (4) Log Files, Working The Log files, the working files, and the local installation files are located Files, and Local in the following subdirectories of /var/evm: Installation Files /var/evm/sockets This directory contains a domain socket node, evmd, and a related lock file, evmd.lck. Local clients use this socket for connection.
Introduction Overview This directory is provided for the installation of local and third-party event template subdirectories. This directory is connected to the system template directory by a symbolic link. /var/evm/adm/channels This directory is provided for the installation of local and third-party event channel scripts. /var/evm/adm/config This directory and its subdirectories contain secondary configuration files for various components.
Introduction Overview /usr/share/evm/templates This directory contains system event template files and subdirectories. NOTE 26 Do not modify the system supplied definition.
Introduction Event Manager Events Event Manager Events An Event Manager event is a binary package of data that contains a set of standard data items, including a name, a timestamp, and information about the poster. An event may contain variable data, which is named and supplied by the poster. For example, an event reporting the failure of a device may hold variables containing the path name and type of the device.
Introduction Event Manager Events Other than the Event Manager, the operating system supports a few other mechanisms through which system components can report event and status information. The system logger, syslog is a familiar example of event management system. It provides simple event distribution facilities for other components to use, and its daemon actively manages the event information it receives.
Introduction Event Manager Events Figure 1-2 Event Model The Event Manager includes command-line utilities that recognize the format of the event. You can use these command-line utilities to perform basic operations at the command prompt or in shell scripts. However, you cannot view an event directly with a text viewer (for example, more ) because an event is a package of binary data.
Introduction Event Manager Events The command-line utilities are designed to be used together in pipelines. For example, you may pipe a set of events from a file in to the sort utility, pipe the output in to the formatting utility, pipe the output of that command in to the more command, or redirect it to a file. The “Using Event Manager” on page 31 provides examples of using commands to monitor and review event activity.
2 Using Event Manager This chapter describes how the Event Manager monitors multiple event sources and combines them into a single event stream. By default, the logger is configured to send e-mails to the superuser when events with a priority of 600 (alert) or greater are posted.
Using Event Manager event log on a daily basis, using the command-line utilities. You can also configure the logger to take other actions, such as sending a pager message according to any criteria you choose. In addition, you can monitor events at your terminal as they occur, using the evmwatch command.
Using Event Manager Starting and Stopping Event Manager Starting and Stopping Event Manager The Event Manager starts automatically at system startup and stops when the system is shut down. The Essential Services Monitor (ESM) daemon, esmd, maintains the availability of essential system daemons, including the daemons, by automatically restarting them. For more information about ESM daemon, see the esmd (1M) manpage. To start the Event Manager and the ESM daemon, complete the following steps: Step 1.
Using Event Manager Starting and Stopping Event Manager NOTE You must use the same PID that you used to stop ESM daemon. You do not need to stop and start Event Manager to change the configuration. You can change the configuration, and run the evmreload command. For more information, see the evmreload (1M) manpage.
Using Event Manager Monitoring Events Monitoring Events The following sections discuss the commands you can use to monitor and review event activity. Displaying Events Using evmshow An Event Manager event is a binary data package, because it must be converted to text before you can display it on a terminal. The evmshow command reads binary events from its stdin stream or from a named file, and outputs the same events in text form to stdout.
Using Event Manager Monitoring Events # cat my_events | evmshow -t "@timestamp [@priority] more 03-Aug-2006 21:06:14 [200] been added @@" | ProcSM : A category "esmd" has You can set up your own show-template to display the items that are important to you, in any format you want. For more information about the data items, see EvmEvent (5).
Using Event Manager Monitoring Events Timestamp : 03-Aug-2006 21:06:14 Format added. [4] : ProcSM: A category "$_catname" has been Reference : cat:evmexp.cat:2300 Variable Items: [5] _catname (STRING) = "esmd" Where: 1. The explanation of the event. In some cases, this data field contains a recommended action to rectify a problem. 2. The Formatted Message section. 3. The Event Data Items section, which lists all the standard data items contained in the event.
Using Event Manager Monitoring Events You can display events that are stored in the various system log files, or monitor them as they occur by using the evmget and evmwatch commands. For more information about these commands, see “Retrieving Stored Events Using evmget” on page 38 and “Monitoring Events” on page 35. Most systems produce a large number of events, many of which report normal operation. Use event filters to limit the display to a set of events that you consider to be important.
Using Event Manager Monitoring Events • Feeds events back to the evmget command which writes them to its stdout stream After all the channel get functions run and all the events are returned, both get-server daemon and the evmget command terminate. NOTE Though events may be stored in log files as lines of text, or in a special binary format, the evmget command returns all events in the form of binary events, which can be passed to evmshow for display.
Using Event Manager Monitoring Events Using the -A Option to Simplify the Command String introduces using the evmget command with the -A option, which makes it possible to retrieve, sort, and display events without building a pipeline. Depending on the size and type of your system and the number of events being logged, event retrieval can take a noticeably long time.
Using Event Manager Monitoring Events which are the data items on which you want to sort the events. The specification is a list of data item names, separated by colons (:). For example: priority:timestamp The preceding specification sorts events by timestamp within priority, so the first group of events that are returned are those with the lowest priority, sorted in their order of occurrence.
Using Event Manager Monitoring Events waits to receive events. As events arrive, the evmwatch command writes them to the standard out stream (stdout) as binary Event Manager events. You cannot display the output of the evmwatch command because it is a stream of binary events. You must use the evmshow command to format the events.
Using Event Manager Monitoring Events The -A option simplifies the command string by running the evmsort command and the evmshow command automatically. The evmwatch command also supports the -A option and automatically runs the evmshow command when you use it.
Using Event Manager Monitoring Events By default, the message is posted as a notice event, with a priority of 200. You can change the priority with the -p option.
Using Event Manager Monitoring Events To create and post a new event, complete the following steps: Step 1. Create the /var/evm/adm/templates/local directory if it does not exist. Step 2. Use a text editor, such as vi, to create the following text file: # This file contains EVM event templates for local # backup notification events. event { name local.admin.backup.ok format "BACKUP: Backup completed OK" priority 200 } event { name local.admin.backup.
Using Event Manager Monitoring Events 200 BACKUP: Backup completed OK 400 BACKUP: Backup failed - code 0 Step 5. Verify that the file is owned by root or bin, and that its permissions are set to 0400, 0600, 0440, or 0640. Correct the permissions by using the chown command and the chmod command, if necessary. Step 6.
Using Event Manager Monitoring Events Step 10. Verify that the events are logged correctly by entering the following commands at the HP-UX prompt: # echo 'event {name local.admin.backup.ok}' | evmpost # echo 'event {name local.admin.backup.failed}' | evmpost # evmget -f '[name local.admin.
Using Event Manager Monitoring Events In the previous example, the input to the evmpost command for the success event is simple, so it is supplied on the same line by using the echo command. For the failure event, the value of the result_code variable must also be supplied. To supply this value, the shell's << syntax provides a more structured multiline form of input. Both forms of input supply source code input to the evmget command through its standard input (stdin) stream.
Using Event Manager Listing a Registered Event Listing a Registered Event You can register an event by adding a template file entry as described in “Event Templates” on page 76, and entering the evmreload command with the -d option to make the events known to the Event Manager daemon, or restarting the system. You can use the evmwatch -i command to retrieve a list of registered events. Pipe the output from the evmwatch -i command to the evmshow command to display the event templates in any desired format.
Using Event Manager Listing a Registered Event Using the -A Option to Simplify the Command String The Event Manager commands are designed to be building blocks, with each command performing a specific operation. This provides you with flexibility in developing shell scripts to manipulate event information. When you enter commands from the command line, you may prefer to simplify the command.
Using Event Manager Logging and Forwarding Events Logging and Forwarding Events The response to an event is any action determined by your site-specific needs and conditions. This response can range from activating alarms or paging responsible personnel, to making a log entry or ignoring an expected occurrence of a regular activity. You can configure the event processing sequence to perform a series of dependent tasks, by using an event output by one task as the trigger to activate the next process.
Using Event Manager Logging and Forwarding Events You can include a suppress group specification in an eventlog statement in the configuration file. When you include such a statement, events meeting the suppression criteria are not entered in the log. One instance of the event is stored, with additional data indicating the number of events and the time of the first and last occurrence of the event. For the explanation of this criterion, see evmlogger.conf (4).
Using Event Manager Logging and Forwarding Events • The logger executes the forwarding command asynchronously. It starts the command and then continues with its normal operation without waiting for the command to finish. The following behaviors are normal: — If multiple forwarders are specified in the logger's configuration file, and the same event is to be handled by more than one forwarder, the logger starts each forwarding command without waiting for the others to finish.
Using Event Manager Introduction to Event Filters Introduction to Event Filters This section introduces event filters and relates them to the evmshow command examples from the previous section. Filtering technique is described in detail in later sections of this document. The full filter syntax is defined in EvmFilter (5). An Event Manager event filter is a text string that informs Event Manager which events you want to retrieve.
Using Event Manager Introduction to Event Filters The wildcard asterisk matches the components sys.unix.procsm.category. To avoid any possibility that the shell expand the wildcard character with filenames, enclose the filter string in single quotes instead of the double quotes. This is always a wise precaution if special characters are used in shell commands. When you filter by name, Event Manager assumes that there is a wildcard .* at the end of the name string, even if it is not included in the command.
Using Event Manager Introduction to Event Filters Advanced Selection and Filtering Techniques This section describes some additional filtering techniques that you can use to further improve event selection, so that you receive only the events in which you are interested.
Using Event Manager Introduction to Event Filters The before and since keywords use similar specifier strings. However, you cannot use wildcard characters and there is no day of the week indicator. For example, the following command discovers events that were posted after 3:00p.m. on July 6, 2002: # evmget -A -f '[since 2002:7:6:15:0:0]' | more The age keyword provides a more convenient and intuitive way to select events according to their timestamps.
Using Event Manager Introduction to Event Filters 15-Apr-1999 15-Apr-1999 15-Apr-1999 15-Apr-1999 15-Apr-1999 [1] 14:19:06 0 14:19:06 1 14:19:06 2 14:19:06 3 14:19:06 5 EVM daemon: Configuration completed EVM daemon: Initialization completed EVM logger: Logger started EVM: Mark event - initial EVM logger: Started eventlog /var/evm/evmlog/evmlog.19990415 [2] Where: 1. The age filter keyword selects all events that have occurred today, as indicated by the timestamp in the first column of data. 2.
Using Event Manager Introduction to Event Filters You can search for all such events by the following command: # evmget -A -f '[name *._hwid]' | more If you know the hardware identifier of a specific device, you can narrow the search for events related to that device by using a command similar to the following: # evmget -A -f '[name *._hwid.4]' | more Using Filter Files You can save a useful filter in a file and recall it by using the Event Manager's indirect filter facility.
Using Event Manager Introduction to Event Filters The evmshow -F command option provides an easy way for you to view the contents of a stored filter. The F option causes the evmshow command to display the filter string and then exit without reading any events. In the following example, the evmshow command displays the contents of the filter named user stored in the evm.evf file: # evmshow -f @evm:user -F ( [name sys.unix.evm.msg.
3 Configuring Event Manager Configuring refers to establishing and maintaining the following configurable resident components: • Chapter 3 The Event Manager daemon, evmd 61
Configuring Event Manager • The channel manager, evmchmgr • The logger, evmlogger Each component recognizes a configuration file that directs its operations. When you install the operating system, the Event Manager is configured to run with default options that are suitable for most installations.
Configuring Event Manager Configuring Event Manager Daemon Configuring Event Manager Daemon The daemon reads the /etc/evmdaemon.conf configuration file at system startup and whenever you issue a reload request by using the evmreload command. For a complete description of the contents and syntax for the configuration file, see evmdaemon.conf (4). Example 3-1 shows some sample entries in the daemon configuration file.
Configuring Event Manager Configuring Event Manager Daemon (counting of events) is then suspended for the hold-off period of four hours (240 minutes). */ # Set up an activity monitor: activity_monitor { name period threshold holdoff } event_count 10 500 240 If you make any changes to the configuration file, you must enter the evmreload command to inform the daemon of these changes. For more information, see evmreload (1M).
Configuring Event Manager Configuring Event Manager Channel Configuring Event Manager Channel An event channel is a source of event information. The channel configuration file, /etc/evmchannel.conf, defines a set of event channels and the functions that operate on the channels, for use by the channel manager, the evmshow command, and the event retrieval process. For more information about configuration file, see evmchannel.conf (4). Example 3-2 shows sample channel configuration file entries.
Configuring Event Manager Configuring Event Manager Channel events whose names do not match the events value of any other channel. */ events * /* Any line beginning with fn_ defines a script that runs for each function. */ fn_get "evmlog_get" fn_details "evmlog_details" fn_explain "evmlog_explain" fn_monitor "evmlog_mon" /* The argument values on this line program to control its operation. older than 7 days are compressed days are deleted. The meanings of to individual channel functions, all cases.
Configuring Event Manager Configuring Event Manager Logger Configuring Event Manager Logger The logger handles storage and forwarding of events, according to entries in the /etc/evmlogger.conf configuration file. For more information about configuration file, see evmlogger.conf (4). Example 3-3 shows sample entries in a logger configuration file. An example of possible customization of the logger is to direct output to a terminal in addition to a log file.
Configuring Event Manager Configuring Event Manager Logger / * If this line is not commented out (by #) and the sample path is replaced by the path name of an existing write-enabled directory, an alternate log file is opened in this directory if the primary directory becomes write-disabled. */ # alternate /your_alternate_fs/evmlog/evmlog.dated /* This line establishes the filtering conditions for events, determining which events are logged by this event log.
Configuring Event Manager Configuring Event Manager Logger # Don't forward mail events through mail /* This line establishes filtering for the events. As with an event log definition, the filter string specifies the set of events that are handled by this forwarder. To prevent an event loop from occurring if the mailer posts high-priority events, signifying a possible problem in the mail subsystem, mail events are explicitly excluded from this forwarder. */ filter "[prio >= 600] & 
