HP-UX IPSec Version A.03.02.02 Administrator's Guide HP-UX 11i version 2 and HP-UX 11i version 3 (762800-001, April 2014)

Figure 3 ESP processing
Does
derived HMAC’=
received HMAC?
Data
Encryption
Algorithm
Hash
Algorithm
Hash
Algorithm
Encrypted Data
Encrypted Data
System A System B
Authentication
Value (HMAC)
Encrypted Data
Authentication
Value (HMAC)
keyE
Data
Decryption
Algorithm
HMAC’
Integrity OK
Yes,
decrypt
No
Integrity
Bad
(Reject)
keyE
keyA
keyA
Transport and tunnel modes
ESP can be used in transport mode or tunnel mode.
Transport mode
In transport mode, IPsec inserts the ESP header after the original IP header, and adds the ESP trailer
and authentication value to the end of the packet. Only the IP payload (e.g., TCP, UDP, or IGMP
packet) is secured (encrypted and authenticated). The IP header is not secured. Transport mode is
typically used for end-to-end security. Figure 4 shows IPv4 ESP packets in transport mode.
Figure 4 ESP transport mode
IP Header
ESP Header Payload
encrypted
authenticated
ESP Trailer ESP Authentication
ESP
Transport
Mode
Tunnel mode
In tunnel mode , IPsec encloses, or encapsulates, the original IP packet, including the original IP
header, within a second IP datagram. All of the original IP packet, including the original header,
is secured. Tunnel mode is typically used on secure gateways. When ESP is used in tunnel mode
on gateways, the outer, unencrypted IP header contains the IP addresses of the gateways, and the
inner, encrypted IP header contains the end IP source and destination addresses. This prevents
eavesdroppers from detecting or analyzing traffic between the end source and destination addresses.
Figure 5 shows IPv4 ESP packets in tunnel mode.
IPsec protocol suite 35