HP-UX IPSec Version A.03.00 Administrator's Guide

1. The recipient ESP module calculates its own authentication value for the encrypted payload
using its copy of the authentication key (KeyA).
2. The recipient ESP compares its authentication value with the transmitted authentication
value (the HMAC). If the values match, the recipient then uses its copy of the encryption
key (KeyE) to decrypt the encrypted portion of the packet and extract the original payload.
Figure 1-3 ESP Processing
Does
derived HMAC’=
received HMAC?
Data
Encryption
Algorithm
Hash
Algorithm
Hash
Algorithm
Encrypted Data
Encrypted Data
System A System B
Authentication
Value (HMAC)
Encrypted Data
Authentication
Value (HMAC)
keyE
Data
Decryption
Algorithm
HMAC’
Integrity OK
Yes,
decrypt
No
Integrity
Bad
(Reject)
keyE
keyA
keyA
Transport and Tunnel Modes
ESP can be used in transport mode or tunnel mode.
Transport Mode
In transport mode, IPsec inserts the ESP header after the original IP header, and adds the ESP
trailer and authentication value to the end of the packet. Only the IP payload (e.g., TCP, UDP,
or IGMP packet) is secured (encrypted and authenticated). The IP header is not secured. Transport
mode is typically used for end-to-end security. Figure 1-4 shows IPv4 ESP packets in transport
mode.
Figure 1-4 ESP Transport Mode
IP Header
ESP Header Payload
encrypted
authenticated
ESP Trailer ESP Authentication
ESP
Transport
Mode
34 HP-UX IPSec Overview