Router User Manual

2 IPSec and IKE troubleshooting
Nortel Secure Router 8000 Series
Troubleshooting - VAS
2-4 Nortel Networks Inc. Issue 01.01 (30 March 2009)
IPSec encapsulation modes
The SA specifies the protocol encapsulation modes. IPSec has two encapsulation modes:
z
Transport mode: AH/ESP is inserted following the IP header but before all transport
layer protocols or all other IPSec protocols.
Figure 2-1 shows the format of transport
mode packets.
z
Tunnel mode: AH/ESP is inserted before the original IP header but after the new IP
header.
Figure 2-2 shows the format of tunnel mode packets.
Figure 2-1 Format of the transport mode packets
Mode
Protocol
Transport
AH
ESP
AH-ESP
ESP data ESP Tail ESP Auth data
IP Header AH dataTCP Header
IP Header TCP Header
ESP data ESP Tail ESP Auth dataIP Header TCP HeaderAH
Figure 2-2 Format of the tunnel mode packets
Mode
Protocol
Tunnel
AH
ESP
AH-ESP
new IP Header AH dataTCP Header
ESP data ESP Tail ESP Auth datanew IP Header TCP HeaderAH
raw IP Header
new IP Header ESP dataTCP Headerraw IP Header ESP Tail ESP Auth data
raw IP Header
Transport mode is suitable for communication between two hosts or between a host and a
security gateway. In this mode, the two devices that encrypt or decrypt packets must be the
original packet sender and the final receiver respectively.
Tunnel mode is suitable for communication between two security gateways.
Authentication algorithms and encryption algorithms
z
Authentication algorithms
AH and ESP can authenticate the integrity of an IP packet to determine whether the
packet is modified during transmission. Authentication is implemented based on the hash
function. IPSec peers calculate the message summary. If they get the same summaries, it
indicates the packet is integrated and unmodified. The two types of IPSec authentication
algorithms are as follows: