Host Intrusion Detection System Administrator's Guide Release 3.0

Templates and Alerts
Buffer Overflow Template
Appendix A
134
Buffer Overflow Template
The vulnerability
addressed by this
template
All buffer overflow attacks (e.g., stack smashing, return-into-libc, execute on heap)
attempt to overflow a buffer, where the buffer can be a local variable residing on the
stack or a dynamically allocated buffer residing on the heap or a global variable residing
in the process data segment. See the whitepaper “Stack Buffer Overflow Protection in
HP-UX 11i,” available at http://www.docs.hp.com, for a description of buffer overflow
attacks on HP-UX. Unusually long program arguments are carefully modified by an
attacker to overflow a buffer for which the program does not perform bounds checking.
By overflowing the buffer, an attacker can modify the program’s execution flow to
execute malicious code and thereby hijack a privileged program. Modifying a program’s
execution flow can be accomplished in several ways, including the following:
• Overflowing a buffer on the stack to modify the return address in an activation
record
• Overflowing a buffer on the heap to modify a free memory header so that the heap
memory allocation code subsequently overwrites a function’s return address.
• Overflowing a buffer in the data segment in order to overwrite an adjacent variable
containing a function pointer so that a subsequent dereferencing of the variable
results in the execution of malicious code.
How this template
addresses the
vulnerability
The Buffer Overflow (BO) template monitors attack patterns that are indicative of
various types of buffer overflow attacks and reports execute-on-stack buffer overflow
attacks detected by the HP-UX kernel (starting with HP-UX 11i). The template monitors
privileged setuid programs where the effective user id (euid) is not equal to the real user
id (ruid) and the euid is one of the user ids specified in the template’s property list of
privileged users (e.g., root).
Specifically, the template monitors privileged setuid programs for the following:
• The privileged setuid program was invoked with an unusually long program
argument(s).
• The privileged setuid program was invoked with program argument(s) that contain
non printable characters (e.g., could be CPU opcodes).
The template also reports when the kernel has detected that a program has attempted to
execute on its stack (perhaps as part of a stack buffer overflow attack).
NOTE In HP-UX 11i v1 and later, comprehensive stack buffer overflow protection, which uses a
combination of highly efficient software and existing memory management hardware,
protects against both known and unknown buffer overflow attacks without sacrificing
system performance. This protection is managed with the executable_stack tunable
kernel parameter. You can allow selected programs to execute from the stack by marking
them with the -es option of the chatr command. See the executable_stack (5) and
chatr (1) manpages and the Stack BufferOverflow Protection in HP-UX 11i white paper,
available at http://docs.hp.com