HP-UX IPSec version A.02.01 Administrator's Guide

Troubleshooting HP-UX IPSec
IPsec Operation
Chapter 7194
On an end system (the local system is the source for the outbound
packet), the Policy Manager sequentially searches the host IPsec
policies in priority order for the first policy with an IP packet filter
that matches the packet. If no match is found, HP-UX IPSec uses the
default host IPsec policy.
On a gateway system (the local system is forwarding the outbound
packet), the Policy Manager sequentially searches the gateway IPsec
policies in priority order. If no match is found, HP-UX IPSec uses the
default gateway IPsec policy.
If the transform (action) specified in the matching IPsec policy is to
encrypt or authenticate the IP packets using AH or ESP, IPsec SAs
may already exist for the policy. The new packet can use the existing
IPsec SAs if the IP addresses, ports and protocols match. The new
packet can also use the existing IPsec SAs if both IP addresses match
and host-based keying is enabled (the Exclusive bit is not set for the
policy). Otherwise, new IPsec SAs are established.
3. Establish an IKE SA
Before establishing an IPsec SA, the IKE daemon must establish an
IKE SA with the remote system, if none exists. To establish an IKE
SA, the Policy Manager daemon is queried for an IKE policy, based
on the remote system’s IP address.
As part of the IKE SA negotiations, each system establishes a
Diffie-Hellman value using the Oakley group specified in the IKE
policy. Because the Diffie-Hellman algorithm is vulnerable to
third-party attacks, each system must also authenticate the identity
of the other system using the primary authentication method
specified in the IKE policy (preshared keys or RSA signatures with
security certificates), and verify the remote system’s ID type and ID
value (HP-UX IPSec uses IP addresses by default). The two systems
also negotiate the authentication and encryption algorithms for the
IKE SA, and SA lifetime according to values specified in the IKE
policies.
4. Establish IPsec SAs
Once the IKE SA is established, the IKE daemon uses the secured
channel to establish IPsec SAs with its peers. Two IPsec SAs are
established: one for packets from the local system to the remote
system, and one for packets from the remote system to the local
system.