HP-UX Host Intrusion Detection System Version 4.3 administrator guide
Table Of Contents
- HP-UX Host Intrusion Detection System Version 4.3 administrator guide
- Table of Contents
- About This Document
- 1 Introduction
- 2 Configuring HP-UX HIDS
- 3 Getting Started with HP-UX HIDS
- 4 Using the System Manager Screen
- Starting the HP-UX HIDS System Manager
- Stopping the HP-UX HIDS System Manager
- System Manager Components
- Starting HP-UX HIDS Agents
- Getting the Status of Agent Hosts
- Resynchronizing Agent Hosts
- Activating Schedules on Agent Hosts
- Stopping Schedules on Agent Hosts
- Halting HP-UX HIDS Agents
- Accessing Other Screens
- 5 Using the Schedule Manager Screen
- The Schedule Manager
- Configuring Surveillance Schedules
- Configuring Surveillance Groups
- Configuring Detection Templates
- Setting Surveillance Schedule Timetables
- Configuring Alert Aggregation
- Configuring Monitor Failed Attempts
- Configuring Duplicate Alert Suppression
- Viewing Surveillance Schedule Details
- Predefined Surveillance Schedules and Groups
- 6 Using the Host Manager Screen
- 7 Using the Network Node Screen
- 8 Using the Preferences Screen
- A Templates and Alerts
- Alert Summary
- UNIX Regular Expressions
- Limitations
- Template Property Types
- Buffer Overflow Template
- Race Condition Template
- Modification of files/directories Template
- Changes to Log File Template
- Creation and Modification of setuid/setgid File Template
- Creation of World-Writable File Template
- Modification of Another User’s File Template
- Login/Logout Template
- Repeated Failed Logins Template
- Repeated Failed su Commands Template
- Log File Monitoring Template
- B Automated Response for Alerts
- C Tuning Schedules and Generating Alert Reports
- D The Agent Configuration File
- E The Surveillance Schedule Text File
- F Error Messages
- G Troubleshooting
- Troubleshooting
- Agent and System Manager cannot communicate with each other
- Agent complains that idds has not been enabled, yet lsdev shows /dev/idds is present
- Agent does not start on system boot
- Agent halts abnormally, leaving ids_* files and message queues
- Agent host appears to hang and/or you see message disk full
- Agent needs further troubleshooting
- Agent does not start after installation
- Agents appear to be stuck in polling status
- Agent displays error if hostname to IP mapping is not registered in name service
- Aggregated alerts targets or details field are truncated and the same aggregated alert has several entries logged in the IDS_ALERTFILE
- Alert date/time sort seems inconsistent
- Alerts are not being displayed in the alert browser
- Buffer overflow triggers false positives
- Duplicate alerts appear in System Manager
- Getting several aggregated alerts for the same process
- GUI runs out of memory after receiving around 19,000 alerts
- The idsadmin Command needs installed agent certificates
- The idsadmin Command notifies of bad certificate when pinging a remote agent
- IDS_checkInstall fails with a kmtune error
- IDS_genAdminKeys or IDS_genAgentCerts does not complete successfully
- IDS_genAdminKeys or idsgui quits early
- Large files in /var/opt/ids
- Log files are filling up
- No Agent Available
- Normal operation of an application generates heavy volume of alerts
- Reflection X rlogin produces multiple login and logout alerts
- Schedule Manager timetable screen appears to hang
- SSH does not perform a clean exit after idsagent is started
- System Manager appears to hang
- System Manager does not let you save files to specific directories
- System Manager does not start after idsgui is started
- System Manager starts with no borders or title bar in X client programs on Windows
- System Manager times out on agent functions such as Activate and Status Poll
- UNKNOWN program and arguments in certain alert messages
- Using HP-UX HIDS with IPFilter and SecureShell
- Unable to Generate Administrator Keys and Agent Certificates on PA–RISC 1.1 Systems
- Troubleshooting
- H HP Software License

Type XI: String
The Type XI property value is a literal string. Unlike the Type I property, the Type XI property
is not interpreted as a regular expression and only specifies one literal string.
The logfile template property of the Log File Monitoring template is a Type XI property
that specifies the pathname of a logfile. For example, the following specifies that the syslog.log
file should be monitored:
logfile | /var/adm/syslog/syslog.log
NOTE: The value of the property consists of all the characters between (and including) the first
non-whitespace character after the preceeding pipe (|) character and the end of the line.
For examples of regular expressions, see “UNIX Regular Expressions ” (page 114).
Buffer Overflow Template
The vulnerability addressed by this template
A buffer can be a local variable residing on the stack, a dynamically allocated buffer residing on
the heap, or a global variable residing in the process data segment. All buffer overflow attacks
(for example, stack smashing, return-into-libc, execute on heap) attempt to overflow a buffer.
Refer to the Stack Buffer Overflow Protection in HP-UX 11i white paper available at http://
www.docs.hp.com, for a description of buffer overflow attacks on HP-UX. Unusually long
program arguments are carefully modified by an attacker to overflow a buffer for which the
program does not perform bounds checking. By overflowing the buffer, an attacker can modify
the program’s execution flow to execute malicious code and thereby hijack a privileged program.
A hacker can modify a program’s execution flow in several ways, including the following:
• Overflowing a buffer on the stack to modify the return address in an activation record.
• Overflowing a buffer on the heap to modify a free memory header so that the heap memory
allocation code then overwrites a function’s return address.
• Overflowing a buffer in the data segment, to overwrite an adjacent variable containing a
function pointer so that a subsequent dereferencing of the variable results in the execution
of malicious code.
How this template addresses the vulnerability
The Buffer Overflow (BO) template monitors attack patterns that are indicative of various types
of buffer overflow attacks, and reports execute-on-stack buffer overflow attacks detected by the
HP-UX kernel. The template monitors privileged setuid programs where the effective user ID
euid is not equal to the real user ID ruid and the euid is one of the user IDs specified in the
template’s property list of privileged users; for example, root.
Specifically, the template monitors privileged setuid programs for the following:
• The privileged setuid program was invoked with an unusually long program argument.
• The privileged setuid program was invoked with program arguments that contain
nonprintable characters (for example, possible CPU opcodes).
The template also reports when the kernel detects that a program has attempted to execute on
its stack, perhaps as part of a stack buffer overflow attack.
Buffer Overflow Template 121