HP-UX IPSec version A.02.01 Administrator's Guide

Troubleshooting HP-UX IPSec
IPsec Operation
Chapter 7196
• Clear Text Packet
If the inbound packet has no AH or ESP (it is a normal IP packet in
clear text), HP-UX IPSec must still determine whether the packet
should be dropped or passed in clear text. HP-UX IPSec checks the
kernel policy engine cache for an existing decision on the action to
take (drop or pass in clear text) for the packet based on the IP
addresses, protocol, and port numbers. If the action is to apply an AH
or ESP transform, HP-UX IPSec sends an audit message to the audit
daemon. This is because the remote system should have established
IPsec SAs before sending the packet.
If no cache entry exists, HP-UX IPSec queries the policy manager
daemon for the appropriate action according to the host IPsec policy
with the filter that best matches the packet (or the default policy, if
no filters match). Again, if the action is to apply an AH or ESP
transform, HP-UX IPSec discards the packet and sends an audit
message to the audit daemon.
Establishing Tunnel Security Associations
If HP-UX IPSec is processing an outbound packet and the selected host
or gateway IPsec policy specifies a tunnel IPsec policy, HP-UX IPSec
checks if it has an existing tunnel SA with the tunnel endpoint. If not, it
must establish a tunnel SA before it establishes the end-to-end
(transport) SA. The procedure for establishing a tunnel SA is similar to
establishing a transport SA (HP-UX IPSec uses/establishes an IKE SA to
establish the IPsec SA), except the IKE entities also include proxy
address information during IPsec SA negotiation. The proxy address
information identifies the end-to-end entities and allows a tunnel
endpoint to determine the identity of the end system or subnet for which
the other tunnel endpoint is establishing the tunnel.
Processing Inbound Tunnel Packets
If HP-UX IPSec is processing an inbound packet, it searches the kernel
SA database for inbound packets for an entry with the same SPI and
source IP address. If one exists, it uses the information in the SA to
decrypt or authenticate the packet. If this is a tunnel SA, HP-UX IPSec
decapsulates the packet (removes the outer IP header) and processes the
IP header for the inner packet. If the destination address in the inner
packet is a local address, HP-UX IPSec searches its host IPsec policies to
determine the next action. If it is not a local address, HP-UX IPSec
searches its gateway IPsec policies to determine the next action. If the