HP-UX IPSec version A.02.00 Administrator's Guide

Troubleshooting HP-UX IPSec
Troubleshooting Scenarios
Chapter 5 175
Determine whether or not the ISAKMP/MM SA is being established by
checking the audit log file. IKMPD error entries with the message Main
Mode processing failed indicate that the ISAKMP/MM SA is not
being established. Informative entries with the message MM
negotiation complete with the peer
ip_address indicate that the
ISAKMP/MM SA is being established.
The IKMPD message QM negotiation timeout, mess ID
hhhh may
indicate that there is an IPSec transform proposal mismatch (the IKE
peer may not respond if it receives an unacceptable transform proposal,
which causes a timeout).
If the ISAKMP/MM negotiation was successful but the ISAKMP/MM SA
was deleted later because an IPSec/QM negotiation failed, go on to
“ISAKMP/MM SA Negotiation Succeeded, IPSec/QM SA Negotiation
Fails (Quick Mode processing failed, QM negotiation timeout)” on
page 177.
If the ISAKMP/MM SA is not being established:
Run the following command:
ipsec_policy (determine the IKE policy)
Check the IKE policy parameters against the parameters configured on
the remote system:
• Oakley (Diffie-Hellman) Group
• (Primary) Authentication Method
• Authentication Algorithm
• Encryption Algorithm
Check the audit file for IKMPD error messages such as MM negotiation
timeout. This may indicate a connectivity problem with the remote
system. However, some ISAKMP/IKE responders will not respond if the
initiating system sends an unacceptable SA proposal, which also causes
timeouts.
Enable a nettl level 4 trace using the command ipsec_admin -traceon
or get a line analyzer trace and verify that the packets are being sent and
received by the correct remote system. Check whether the remote
ISAKMP/IKE entity is responding. ISAKMP always uses UDP port 500
to receive and send ISAKMP packets.