HP-UX IPSec Version A.03.00 Administrator's Guide HP-UX 11i version 2 and HP-UX 11i version 3 HP Part Number: J4256-90043 Published: April 2009 Edition: 1.
© Copyright 2007-2009 Hewlett-Packard Development Company L.P Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents About This Document .....................................................................................................19 Intended Audience................................................................................................................................19 New and Changed Documentation in This Edition.............................................................................19 IKE Policy Changes...............................................................................
Shared Key Encryption..............................................................................................................32 Shared Key Hash Functions.......................................................................................................33 ESP Processing...........................................................................................................................33 Transport and Tunnel Modes.................................................................................
Blue Configuration.....................................................................................................................58 Step 3: Verifying the Batch File Syntax.................................................................................................59 Step 4: Committing the Batch File Configuration and Verifying Operation........................................60 Step 5: Configuring HP-UX IPSec to Start Automatically...................................................................
Step 2: Configuring Tunnel IPsec Policies............................................................................................80 ipsec_config add tunnel Syntax...........................................................................................80 tunnel_policy_name ............................................................................................................80 -tsource and -tdestination tunnel_address .............................................................
ip_addr ..............................................................................................................................97 prefix .................................................................................................................................97 -group group_number ..........................................................................................................97 -hash hash_algorithm ......................................................................................
-alt-fqdn fqdn ................................................................................................................119 -alt-user-fqdn user_fqdn ..........................................................................................119 -key_length number_bits .............................................................................................119 -days number_days ..........................................................................................................119 Example......
Additional Options...................................................................................................................134 Configuring HP-UX IPSec to Start Automatically..............................................................................135 ipsec_config add startup Syntax.......................................................................................135 Stopping HP-UX IPSec.........................................................................................................
Determining if the IKEv2 SA Negotiation Succeeded........................................................154 IKEv1 SA Negotiation Fails or Times Out (phase1 negotiation failed)..........................154 Problem....................................................................................................................................154 Symptoms.................................................................................................................................155 Solution..............
Transform Lifetimes.......................................................................................................................168 HP-UX IPSec Operation......................................................................................................................169 HP-UX IPSec Message Flow for Establishing SAs.........................................................................169 IKE Roles................................................................................................
setkey Argument File.......................................................................................................190 racoon.conf File...................................................................................................................190 psk.txt File...........................................................................................................................191 Tips...........................................................................................................
Banana Configuration....................................................................................................................207 Subnet ESP with Exceptions...............................................................................................................208 Carrot Configuration.....................................................................................................................208 Host IPsec Policies.......................................................................
Configuration Overview.....................................................................................................................224 Requirements.................................................................................................................................224 Serviceguard Heartbeat Requirement and Recommendation......................................................224 Configuration Steps..............................................................................................
Authentication Records on Client1 and Client2.......................................................................241 Step 5: Verifying and Testing the HP-UX IPSec Configuration..........................................................242 Step 6: Configuring HP-UX IPSec Start-up Options...........................................................................243 Step 7: Distributing HP-UX IPSec Configuration Files.......................................................................
List of Figures 1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 1-10 1-11 1-12 1-13 1-14 1-15 1-16 4-1 A-1 A-2 A-3 A-4 B-1 D-1 D-2 D-3 D-4 G-1 16 Shared Key Encryption..................................................................................................................32 Shared Key Hash Function............................................................................................................33 ESP Processing.........................................................................................
List of Tables 1 1-1 1-2 4-1 4-2 4-3 4-4 4-5 7-1 7-2 7-3 7-4 7-5 7-6 7-7 A-1 A-2 G-1 Publishing History........................................................................................................................25 HP-UX IPSec Encryption Algorithms...........................................................................................36 HP-UX IPSec Authentication Algorithms.....................................................................................36 ipsec_config Service Names.......
About This Document This document describes how to install, configure, and troubleshoot HP-UX IPSec. The latest version of this document can be found online at http://docs.hp.com. Intended Audience This document is intended for system and network administrators responsible for installing, configuring, and managing HP-UX IPSec. Administrators are expected to have knowledge of HP-UX and networking concepts, commands, and configuration. This document is not a tutorial.
— — • • • • “Support for Multiple Level Public Key Infrastructures” (page 24) “Certificate Revocation List cron File Change” (page 24) “Support for RFC 4301 Security Processing for ICMP Errors” (page 24) “Profile File Changes” (page 24) “Mobile IPv6 Support Is Obsolete” (page 25) “Gateway Policies Are Obsolete” (page 25) IKE Policy Changes The following sections describe product changes related to IKE policies.
encryption algorithm in the profile file (3DES). The migration utility also converts the policy type to IKEv1 and displays a warning. If you are using an IKE policy with DES encryption to communicate with peers that still support DES, you must modify the peer configuration to use 3DES or an alternate algorithm. NOTE: RFC 4772 deprecates DES. DES is susceptible to brute-force attacks.
default priority value is 10. The utility increments the priority value for each subsequent record by the priority value. Authentication Records Specify the IKE (Key Management Protocol) Version Authentication records now include a kmp (key management protocol) field that specifies the IKE version (IKEv1 or IKEv2). The default IKE version is IKEv1. You can specify both IKE versions. The IKE daemon uses the first version for all negotiations it initiates, and responds to negotiations for both versions.
This feature is useful when configuring host policies for remote subnets where not all nodes in the subnet support IPsec. WARNING! Using the FALLBACK_TO_CLEAR flag is a security risk. It can allow packets from non-secure nodes to communicate with the local system. Support for Multiple Source and Destination Arguments in Host and Tunnel Policies You can specify up to 20 instances of the -source and -destination arguments in the ipsec_config add host and ipsec_config add tunnel commands.
Support for 4096 Bit Key Pairs for Certificates HP-UX IPSec now supports 4096-bit public/private key pairs for certificate-based IKE authentication. The ipsec_config add csr command also supports the argument -key_length 4096. Support for PKCS#12 Certificates HP-UX IPSec supports certificates stored in Public Key Cryptography Standards (PKCS) #12 format (commonly referred to as PKCS#12). A PKCS#12 file can also include the private key for the certificate.
the ipsec_migrate utility, ipsec_migrate saves the existing /var/adm/ipsec/ .ipsec_profile file in the /var/adm/ipsec/backup directory before moving the /var/ adm/ipsec/.ipsec_profile.blank file to /var/adm/ipsec/.ipsec_profile. If you use customized settings in your profile file, edit the /var/adm/ipsec/ .ipsec_profile.blank file with your customized settings before running ipsec_migrate.
What’s in This Document HP-UX IPSec Administrator’s Guide is divided into several chapters, and each contains information about installing, configuring, or troubleshooting HP-UX IPSec. The appendices contain supplemental information. “HP-UX IPSec Overview” This chapter describes product features and topologies. “Installing HP-UX IPSec ” This chapter describes how to verify installation prerequisites and install the product.
ENVIRONMENT VARIABLE The name of an environment variable; for example, PATH. [ERROR NAME] The name of an error, usually returned in the errno variable. Key The name of a keyboard key. Return and Enter both refer to the same key. Term The defined use of an important word or phrase. User input Commands and other text that you type. Variable The name of a placeholder in a command, function, or other syntax display that you replace with an actual value. [] The contents are optional in syntax.
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. This product includes cryptographic software written by Eric Young (eay@cryptsoft.com).
1 HP-UX IPSec Overview This chapter describes HP-UX IPSec features and topologies. It contains the following sections: • “Features” (page 29) • “IPsec Protocol Suite” (page 32) • “HP-UX IPSec Topologies” (page 43) Features The IP security (IPsec) protocol suite was defined by the Internet Engineering Task Force (IETF) to provide security for IP networks. HP-UX IPSec is the HP implementation of IPsec.
• Identity authentication The IKE protocol authenticates the identity of the remote system. HP-UX IPSec supports the following forms of IKE authentication: — — Preshared keys. Digital signatures (RSA signatures), using X.509 version 3 security certificates. Because IKE verifies the identity of the remote system, AH and ESP also provide data origin authentication. • Host-based IPsec topologies HP-UX IPSec is supported on host systems in host-to-host and in host-to-gateway topologies.
— Audit logging HP-UX IPSec maintains an audit log of events, including events that may indicate attempts to compromise network security. — Data reporting utility The ipsec_report utility reports IPsec runtime data, including information about security associations or sessions, and entries in the audit log. — Status reporting utility The ipsec_admin utility reports the status of HP-UX IPSec components.
IPsec Protocol Suite The major components of the IPsec protocol suite can be divided into the following categories: • • • • Encapsulating Security Payload (ESP) header for data confidentiality, data integrity, and data authentication. The ESP header also includes a sequence number that provides a form of replay protection. Authentication Header (AH) for data integrity and authentication. The AH header also includes a sequence number for a form of replay protection.
Shared Key Hash Functions Shared key hash functions (also known as a symmetric key hash functions) are hash functions that take a large block of variable-length data and a shared key as input and produce a small, fixed-length hash value, or authentication code. The IPsec protocol suite uses a specific method for producing the hash value and refers to the authentication value as the Hashed Message Authentication Code (HMAC ).
1. 2. The recipient ESP module calculates its own authentication value for the encrypted payload using its copy of the authentication key (KeyA). The recipient ESP compares its authentication value with the transmitted authentication value (the HMAC). If the values match, the recipient then uses its copy of the encryption key (KeyE) to decrypt the encrypted portion of the packet and extract the original payload.
Tunnel Mode In tunnel mode , IPsec encloses, or encapsulates, the original IP packet, including the original IP header, within a second IP datagram. All of the original IP packet, including the original header, is secured. Tunnel mode is typically used on secure gateways. When ESP is used in tunnel mode on gateways, the outer, unencrypted IP header contains the IP addresses of the gateways, and the inner, encrypted IP header contains the end IP source and destination addresses.
ESP Encryption and Authentication Algorithms HP-UX IPSec ESP supports the encryption algorithms listed in Table 1-1 (page 36) and the authentication algorithms listed in Table 1-2 (page 36). For example, HP-UX IPSec can encrypt an ESP packet using AES and authenticate it using SHA1. Table 1-1 HP-UX IPSec Encryption Algorithms Name Description AES Advanced Encryption Standard (AES) Cipher Block Chaining (CBC) mode encryption using a 128-bit key.
Figure 1-8 AH in Transport Mode IP Header AH Header Payload AH Transport Mode authenticated Tunnel Mode In tunnel mode, IPsec encloses, or encapsulates, the original IP datagram, including the original IP header, within a second IP datagram. All of the original IP datagram, including all fields of the original header, is authenticated. Figure 1-9 shows AH in tunnel mode.
Internet Key Exchange (IKE) Before IPsec sends authenticated or encrypted IP data, both the sender and receiver must agree on the protocols, encryption algorithms and keys to use. HP-UX IPSec uses the Internet Key Exchange (IKE) protocol to negotiate the encryption and authentication methods, generate shared encryption keys, and establish secure communication channels, or Security Associations (SAs).
• Phase 1 During Phase 1, the IKE peers establish the IKE SA. • Phase 2 During Phase 2, the IKE peers establish the IPsec SAs. IKEv1 can use one of two methods, or exchange modes, to establish the IKE SA: • • Main Mode Aggressive Mode In Main Mode negotiations, the IKE peers select IKE parameters (configured in IKE policies) based on the remote system’s IP address in the IP packet header. The IKE peers exchange ID information after they establish a secure, encrypted communication channel.
Figure 1-12 Diffie-Hellman Key Generation Step 1: A and B agree on DiffieHellman group Step 2: A and B each generate public/private values System B System A Public Value Public Value Private Value Private Value Step 3: A and B exchange public values with each other Step 4: A combines its private value with B’s public value B combines its private value with A’s public value A and B now have the same shared secret value.
Perfect Forward Secrecy For efficiency, IKEv2 and IKEv1 can reuse an IKE SA to negotiate multiple IPsec SA pairs. For additional security, you can enable the Perfect Forward Secrecy (PFS) feature. With PFS, the compromise (exposure) of one key exposes only the data protected by that key. When PFS is enabled, the IKE peers perform a Diffie-Hellman exchange and generate new keying material for each IPsec SA pair.
• Security Association (SA) An SA is a secure communications channel and its operating parameters. An IPsec SA must exist to use ESP or AH, and an IKE SA must exist to establish IPsec SAs. Because an IKE SA is required to create an IPsec SA, the IKEv2 protocol also refers to IPsec SAs as “child SAs.” • Key Types HP-UX uses four types of cryptography keys: — Preshared keys. IKE uses the preshared key to authenticate the identity of the remote system for IKE. HP-UX supports ASCII keys for preshared keys.
HP-UX IPSec Topologies You can use IPsec between hosts (end nodes), between gateways, and between a host and a gateway in an IP network. You can install HP-UX IPSec only on end nodes. An HP-UX IPSec system can have the following roles: • • • A host in a host-to-host IPsec topology A host in a host-to-gateway IPsec topology A host in a host-to-host IPsec tunnel topology, frequently referred to as an end-to-end tunnel. End-to-end tunnels are commonly used in iSCSI topologies.
System B on the manufacture’s subnet communicate with a host-to-host IPsec topology. For added security, you can configure filtering on the manufacturer’s firewall so that it checks the traffic to and from system A and allows only IPsec packets between system A and B to pass.
In these scenarios, HP-UX IPSec can secure the host-to-host data path between the gateway application server in the DMZ (B in Figure 1-16) and the backend server (C in Figure 1-16). You must configure filtering on the gateway application server (B) to limit access to the backend servers.
2 Installing HP-UX IPSec This chapter describes installation prerequisites and procedures for installing HP-UX IPSec software.
HP-UX IPSec Product Requirements Prior to installing the HP-UX IPSec product, check that your system can accommodate the following product requirements. Software Requirements HP-UX IPSec requires the following software: • OpenSSL software version A.00.09.07l or later. There are two ways to meet this requirement: — On HP-UX 11i v3 systems, the software bundle SysMgmtMin includes OpenSSL software that meets this requirement.
Step 1: Verifying HP-UX IPSec Installation and Configuration Prerequisites 1. Verify that the operating system version is HP-UX 11i version 3 (B.11.31) or HP-UX 11i v2 Update 2 (v2UD2). To determine the OS version, enter the following command: uname -a 2. Verify that you have the required OpenSSL version installed on your system, as listed in “Software Requirements” (page 48). Enter the following command: # what /usr/bin/openssl 3.
Step 2: Loading the HP-UX IPSec Software Follow the steps below to load HP-UX IPSec software using the swinstall utility. 1. 2. 3. Log in as root. Insert the HP-UX IPSec disk into the appropriate drive, or locate the directory into which you downloaded the software from HP Software Depot. Run the swinstall program using the command: swinstall This opens the Software Selection window and the Specify Source window.
Step 3: Establishing the HP-UX IPSec Password You must set the HP-UX IPSec password after installing the product. HP-UX IPSec uses the password to encrypt configuration information. Use the following command to establish the HP-UX IPSec password: ipsec_admin -newpasswd The ipsec_admin utility prompts you to establish the HP-UX IPSec password: IPSEC_ADMIN: Establishing IPsec password, enter IPsec password: Enter a password. The password must be at least 15 characters long and cannot contain spaces.
Step 4: Completing Post-Installation Migration Requirements If you are migrating from a previous version of HP-UX IPSec, run the ipsec_migrate utility and complete other post-installation migration procedures, as described in Appendix C, “Post-Installation Migration Instructions” (page 201). Removing HP-UX IPSec Use the following procedure to remove HP-UX IPSec: 1. Stop HP-UX IPSec by entering the following command: # ipsec_admin -stop 2. 3. 4.
3 Quick Configuration Procedure and Tips This chapter contains a procedure for quickly configuring HP-UX IPSec for a simple host-to-host topology using IKE with preshared keys. In this procedure, you modify the batch file template /var/adm/ipsec/templates/host-to-host. This chapter also includes configuration tips.
Step 1: Establishing the HP-UX IPSec Password If you have not already established the HP-UX IPSec password, use the following command to establish it: ipsec_admin -newpasswd The ipsec_admin utility prompts you to establish the HP-UX IPSec password: IPSEC_ADMIN: Establishing IPsec password, enter IPsec password: Enter a password. The password must be at least 15 characters long and cannot contain spaces.
Step 2: Modifying the Configuration Batch File Template HP-UX IPSec provides the following configuration batch file templates in the directory /var/ adm/ipsec/templates : • • • • end-to-gateway end-to-end-tunnel host-to-host manual-keys For a simple host-to-host topology, edit the batch file template /var/adm/ipsec/templates/ host-to-host as follows: • Uncomment the appropriate configuration statements. At a minimum, you must uncomment and configure the following items: — Host IPsec policies.
###################################################################### # /var/adm/ipsec/templates/host-to-host # # Sample ipsec_config batch file for securing host-to-host IP packets # using preshared keys. # # Copyright 2009, Hewlett-Packard Development Company L.P. # ###################################################################### # # To use this file: # 1. Uncomment the appropriate configuration statements. # For host-to-host IPsec, you must: # a. Configure at least one host IPsec policy.
# -action ESP_AES128_HMAC_SHA1 # ############################################################################ # Case 2 - Host policy to secure inbound telnet ############################################################################ # #add host \ # -source /32/TELNET -destination /32 \ # -action ESP_AES128_HMAC_SHA1 # ############################################################################ # Case 3 - Host policy to secure all UDP packets
# -hash algorithm: MD5 # -encryption: 3DES # -IKEv1 SA lifetime: 28800 seconds # # If you use IKEv1 as the Key Exchange Protocol and these parameter values # do not meet your requirements, # uncomment the following policy to change the default IKEv1 policy: # #add ikev1 default -group 2 -hash MD5 -encryption 3DES -life 28800 # # IKEv2 : # The pre-loaded default IKEv2 policy has the following parameters: # -Diffie-Hellman Group: 2 # -IKEv2 authentication: HMAC-SHA1 # -IKEv2 encryption: 3DES # -Pseudo Random
Step 3: Verifying the Batch File Syntax Use the following command to verify the contents of the ipsec_config batch file without committing the configuration: ipsec_config batch batch_file_name -nocommit The ipsec_config utility displays the following message to indicate the profile file used: Using default profile file /var/adm/ipsec/.ipsec_profile If there are no syntax errors in the batch file, ipsec_config returns without displaying any other messages.
Step 4: Committing the Batch File Configuration and Verifying Operation Use the following procedure to verify the operation of your HP-UX IPSec configuration. 1. Commit the batch file operations to the configuration database with the following command: ipsec_config batch batch_file_name 2. Verify the contents of the configuration database with the following command: ipsec_config show all 3. Start HP-UX IPSec with following command: ipsec_admin -start 4.
------------- IPsec SA ---------------Sequence number: 2 SPI (hex): 100782 State: MATURE SA Type: ESP with AES128-CBC encryption and HMAC-SHA1 authentication Src IP Addr: 10.2.2.2 Dst IP Addr: 10.1.1.
Step 5: Configuring HP-UX IPSec to Start Automatically After you have verified your HP-UX IPSec configuration is properly operating, you can configure HP-UX IPSec so that it starts automatically at system startup time. TIP: HP recommends that you configure HP-UX IPSec to start automatically at system startup time once you have a known, good HP-UX IPSec configuration. This enables HP-UX IPSec to secure your system at all times.
Configuration Tips and Reminders This section contains configuration tips. • Minimum Configuration Requirements If you are using preshared keys for IKE authentication, your configuration must contain at least the following objects: — Host policy — Authentication record (this contains the preshared key) HP-UX IPSec also requires an IKEv2 policy or an IKEv1 policy. The configuration database includes default IKEv2 and IKEv1 policies that can be used without modification.
4 Configuring HP-UX IPSec This chapter describes how to configure HP-UX IPSec, including preshared key configuration. If you are using RSA signature authentication for IKE, you must also see Chapter 5: “Using Certificates with HP-UX IPSec ” (page 113) for instructions on configuring certificates. This chapter also describes how to maximize HP-UX IPSec security and how to use the HP-UX IPSec configuration utility, ipsec_config.
Maximizing Security A system can have both “public” interface IP addresses and “private” interface IP addresses. A public interface IP address is an IP address configured on a Network Interface Card (NIC) connected to a public network. A private interface IP address is an IP address configured on a NIC connected to a private internal network.
Using ipsec_config The ipsec_config utility adds, deletes and displays HP-UX IPSec configuration objects stored in the configuration database, /var/adm/ipsec/config.db. If HP-UX IPSec is active and running, ipsec_config also adds and deletes configuration information in the runtime configuration database.
Comments Lines starting with a pound sign (#) are interpreted as comments. Comment lines within an operation are not allowed. ipsec_config delete Command The ipsec_config delete command deletes objects from the configuration and runtime databases.
IPsec policies, which are created using the ipsec_config add host command. Each section is delimited by BEGIN and END statements. Creating a Customized Profile File In most topologies, you can use the default values in /var/adm/ipsec/.ipsec_profile. If you want to create a customized profile file, make a copy of this file and edit the copy with a text editor.
Configuration Overview There are eight main configuration components: • Host IPsec Policies Host IPsec policies specify HP-UX IPSec behavior for IP packets sent or received by the local system as an end host. A host IPsec policy contains address specifications used to select the host IPsec policy for a packet.
2. Configure tunnel IPsec policies. See “Step 2: Configuring Tunnel IPsec Policies” (page 80) for a description of this step. Skip this step if the local system is not a tunnel endpoint. 3. Configure authentication records. If you are using preshared key authentication, the authentication records also specify the preshared key values. See “Step 3: Configuring Authentication Records and Preshared Keys” (page 85) for a description of this step.
Step 1: Configuring Host IPsec Policies Host IPsec policies specify HP-UX IPSec behavior for IP packets sent or received by the local system as an end host. Each host IPsec policy includes address specifications used to select the host IPsec policy for a packet, and the action for packets using the policy: pass the packets in clear text, discard the packets, or apply an IPsec transform (AH or ESP) to the packets.
[-protocol protocol_id] [-priority priority_number] [-action PASS|DISCARD|transform_list] [-flags flags] HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec.
ip_addr The ip_addr is the source or destination IP address. You can specify a single IP address, or an address range with two addresses separated by a dash and no spaces (ip_addr-ip_addr). The second address in a range must be higher number than the first. For example, 10.1.1.1-10.1.1.3 matches any of the following addresses: 10.1.1.1, 10.1.1.2, 10.1.1.3. Valid Values: An IPv4 address in dotted-decimal notation or an IPv6 address in colon-hexadecimal notation.
Table 4-1 ipsec_config Service Names (continued) Service Name Port Protocol HTTP-TCP 80 TCP HTTP-UDP 80 UDP NTP 123 UDP REXEC 512 TCP RLOGIN 513 TCP RWHO 513 UDP REMSH 514 TCP REMPRINT 515 TCP SMTP 25 TCP TELNET 23 TCP TFTP 69 UDP -protocol protocol_id The protocol_id is the value or name of the upper-layer protocol that HP-UX IPSec uses in the address filter to select an IPsec policy for a packet. You cannot specify protocol and a service_name in the same policy.
To ensure proper operation of IPv6 networks, the default HP-UX IPSec behavior allows all other ICMPv6 message types to pass in clear text. To discard or secure other ICMPv6 message types, you must specify -protocol ICMPV6 and explicitly specify the message type value using the -dst_icmpv6_type and -src_icmpv6_type arguments. For more information, see “ICMPv6 Message Processing” (page 185).
If you are using dynamic keys, the transform list can contain: • • A list that contains up to 2 AH transforms A list that contains up to 6 ESP transforms. Use a comma to separate multiple transform specifications. The order of transforms in the transform list is significant. The first transform is the most preferable and the last transform is the least preferable. At least one transform must match a transform configured on the remote system.
Default: 0 (infinite). CAUTION: HP recommends that you do not specify an infinite value for lifetime_seconds (0) with a finite value for lifetime_kbytes. -flags flags The flags are additional options for this policy. Join multiple flags with a plus sign (+ ). Table 4-3 Host Policy Flags Flag Description EXCLUSIVE Specifies session-based keying. Session-based keying uses a different pair of IPsec SAs per connection or session.
The following batch file entry configures a host IPsec policy that requires telnet requests (where the local system is the telnet server) from subnet 10.0.0.0 to use ESP with AES128 encryption and HMAC SHA-1 authentication. add host telnet_in -source 0.0.0.0/0/TELNET \ -destination 10.0.0.0/8 -pri 120 \ -action ESP_AES128_HMAC_SHA1 The following batch file entry configures a host IPsec policy for an application that listens for requests on local TCP port 50000. Clients have addresses in the range 10.1.1.
Step 2: Configuring Tunnel IPsec Policies Complete this step only if you are using IPsec tunnels. If you are not using IPsec tunnels, skip this step. Tunnel IPsec policies specify HP-UX IPSec behavior for IP packets tunneled by the local system. In an IPsec tunnel, a tunnel endpoint system encapsulates the original packet in a new IPsec packet with an AH or ESP header.
-tsource and -tdestination tunnel_address The tunnel_address is the IP address for the tunnel endpoint. The -tsource tunnel_address is the local tunnel endpoint; the -tdestination tunnel_address is the remote tunnel endpoint. Valid Values: An IPv4 address in dotted-decimal notation or an IPv6 address in colon-hexadecimal notation. The IP address type (IPv4 or IPv6) must be the same for all the addresses in the policy. HP-UX IPSec does not support unspecified IPv6 addresses.
Default: 32 if ip_addr is a non-zero IPv4 address, 128 if ip_addr is a non-zero IPv6 address, or 0 (match any address) if ip_addr is an all-zeros address (0.0.0.0 or 0::0). You must specify a prefix value if you specify a port or service name as part of the address filter. -protocol protocol_id The protocol is the value or name of the upper-layer protocol that HP-UX IPSec uses in the address filter to select an IPsec policy for a packet.
specifies the transforms acceptable for packets using the policy. The HP-UX IPSec IKE daemon proposes the transform list when negotiating the transform for IPsec Security Associations (SAs) with a remote system. The transform_list in a tunnel policy are tunnel transports applied to packets encapsulated between the tunnel endpoints.
ipsec_config add tunnel my_host_host_tunnel \ -tsource 10.1.1.1 -tdestination 10.2.2.2 \ -source 10.1.1.1 -destination 10.2.2.
Step 3: Configuring Authentication Records and Preshared Keys This section describes how to configure IKE authentication records and preshared keys. You must configure authentication records if you are using certificates or preshared keys for IKE authentication. You do not need to configure or modify authentication records if you are using manual keys or are using HP-UX IPSec only to discard packets. The main components of an authentication record are: • • • Remote IP address. This can be a subnet address.
ipsec_config add auth Syntax You can use the following ipsec_config add auth syntax in most installations: ipsec_config add auth auth_name -remote ip_addr[/prefix] [-kmp ike_version] [-exchange|x AM|MM] [-ltype local_id_type -lid local_id] [ -rtype remote_id_type -rid remote_id] [-local_method method] [-remote_method method] [-preshared preshared_key] [-priority priority] HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec.
A prefix length of 0 bits matches all addresses. Range: 0 - 32 for an IPv4 address; 0 - 128 for an IPv6 address. If you are using manual keys, prefix must be 32 if ip_addr is an IPv4 address or 128 if ip_addr is an IPv6 address. Default: 32 if ip_addr is a non-zero IPv4 address, 128 if ip_addr is a non-zero IPv6 address, or 0 (match any address) if ip_addr is an all-zeros address (0.0.0.0 or 0::0).
TIP: Most IKEv1 implementations use Main Mode by default. The IKE protocol specification requires implementations to support Main Mode; support for Aggressive Mode is optional. -ltype local_id_type and -lid local_id The local_id_type and local_id are the local ID type and value the local system sends to the remote system when negotiating an IKE SA. These values must match what is configured on the remote system.
Table 4-4 Local and Remote ID Types and Values (continued) ID Type ID Value USER-FQDN User-Fully Qualified Domain Name (User-FQDN) in SMTP format (also referred to as RFC 822 email address format), such as user@myhost.hp.com. If you are using certificate-based authentication, this must match the subjectAlternativeName field in the certificate. X500-DN X.500 Distinguished Name (DN; also referred to a ASN.1 DN). This ID type is valid only if you are using certificate-based authentication.
Valid Values: PSK (preshared key) RSASIG (RSA signatures using certificates) Default: The configured value for -remote_method. If the -remote_method argument is not specified, and the -preshared argument is present, the default is PSK. If both the -remote_method and the -preshared argument are not specified, the default is the value for the local-method parameter in the AUTHPolicy-Defaults section of the profile file used.
database, incremented by the automatic priority increment value (priority) specified in the AuthPolicy-Defaults section of the profile file (this policy will be the last authentication record evaluated). The default automatic priority increment value (priority) is 10. If this is the first authentication record created, ipsec_config uses the automatic priority increment value as the priority. -flags flags Specifies additional options for this record.
This matches the following FQDNs: alpha.foo.example.com alpha.beta.foo.example.com It does not match the following FQDNs: foo.example.com example.com User FQDN Specify only the FQDN (do not specify a user name) preceded by an at sign (@) to match any user at that FQDN, or specify the FQDN preceded by a dot (.) to match any user in the subtree domain. For example: -rid @foo.example.com This matches the following user FQDNs: root@foo.example.com user1@foo.example.com The user FQDN value .foo.example.
ipsec_config add auth hostB -remote 10.2.2.2 \ -preshared my_hostA_hostB_key IKEv2 The following command configures an IKEv2 authentication record for preshared key authentication. ipsec_config add auth hostC -remote 10.5.5.5 \ -kmp IKEV2 \ -preshared my_hostA_hostC_key Multihomed Example The following batch file entries IKEv1 configure authentication records with preshared key authentication for a remote multihomed HP-UX IPSec system that has addresses 10.8.8.8 and 11.8.8.
add auth Black -remote 10.10.10.10 \ -ltype IPV4 \ -lid 10.20.20.20 This causes Zebra to send 10.20.20.20 as its local ID, even when it transmits packets on its 192.6.2.1 interface. You do not have to specify remote ID information in the above entry because Black is not multihomed, and uses its IPv4 address as its ID.
Step 4: Configuring IKEv1 and IKEv2 Policies The IKEv1 and IKEv1 policies specify parameters for negotiating IKEv1 and IKEv2 SAs. An IKE SA is required to negotiate an IPsec SA pair with dynamic keys. You do not need to configure or modify IKE policies if you are using manual keys or are using HP-UX IPSec only to discard packets. default IKE Policies The configuration database contains a preloaded default IKEv1 policy and a preloaded default IKEv2 policy.
If you omit the priority argument, ipsec_config assigns a priority value that is set to the current highest priority value (lowest priority) for the appropriate IKE policies (IKEv2 or IKEv2) in the configuration database, incremented by the automatic priority increment value for the appropriate IKE policies. The result is that the new policy will be the last IKEv1 or IKEv2 policy evaluated before the default policy.
NOTE: This argument is not valid for the default IKEv1 policy. The default IKEv1 policy matches all remote addresses. Where: ip_addr The ip_addr is the remote IP address. Valid Values: An IPv4 address in dotted-decimal notation or an IPv6 address in colon-hexadecimal notation. HP-UX IPSec does not support unspecified IPv6 addresses. However, you can use the double-colon (::) notation within a specified IPv6 address to denote a number of zeros (0) within an address. The address must be a unicast address.
Default: The value of the hash parameter in the IKEV1Policy-Defaults section of the profile file used. The default hash parameter value is MD5 in /var/adm/ipsec/.ipsec_profile. -encryption encryption_algorithm The encryption_algorithm is the encryption algorithm for encrypting IKE messages. You can specify multiple encryption_algorithm values, delimited by commas and no spaces, and specified in descending order of preference.
add ikev1 apple -remote 10.1.1.1 -encryption 3DES add ikev1 all_others -remote 10.0.0.
double-colon (::) notation within a specified IPv6 address to denote a number of zeros (0) within an address. The address must be a unicast address. Default: None. prefix The prefix is the prefix length, or the number of leading bits that must match when comparing the remote address with ip_addr. For IPv4 addresses, a prefix length of 32 bits indicates that all the bits in the addresses must match. For IPv6 addresses, a prefix length of 128 bits indicates that all the bits in the addresses must match.
in descending order of preference. At least one encryption algorithm must match an encryption algorithm configured on the remote system. Valid Values: AES128-CBC (128–bit Advanced Encryption Standard CBC) 3DES (triple-DES CBC, three encryption iterations, each with a different 56-bit key, 3DES-CBC) NULL (null encryption) Default: The value of the encryption parameter in the IKEV2Policy-Defaults section of the profile file used. The default encryption parameter value is 3DES in /var/adm/ipsec/ .
ipsec_config add ikev2 Command Example The following command modifies the default IKEv2 policy to use Diffie-Hellman group 5 or group 2, with a higher preference for group 5: ipsec_config add default ikev2 default -group 5,2 102 Configuring HP-UX IPSec
Step 5: Configuring Certificates See Chapter 5: “Using Certificates with HP-UX IPSec ” (page 113) for information on configuring certificate information if you are using RSA signatures for IKE authentication. After you have configured certificate information, go to “Step 6: Configuring the Bypass List (Local IP Addresses)” (page 104).
Step 6: Configuring the Bypass List (Local IP Addresses) The bypass list specifies local IP addresses that IPsec bypasses or ignores. The system does not attempt to find an IPsec policy for packets sent or received using an IP address in the bypass list, and the system processes these packets as if HP-UX IPSec was not enabled. The bypass list improves transmission rates for addresses in the bypass list.
ipsec_config add bypass Syntax You can use the following ipsec_config add bypass syntax to configure preshared keys in most installations: ipsec_config add bypass ip_address HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec.
Step 7: Verifying the Batch File Syntax Use the following command to verify the contents of the ipsec_config batch file without committing the configuration: ipsec_config batch batch_file_name -nocommit The ipsec_config utility displays the following message to indicate the profile file used: Using default profile file /var/adm/ipsec/.ipsec_profile If there are no syntax errors in the batch file, ipsec_config returns without displaying any other messages.
Step 8: Committing the Batch File Configuration and Verifying Operation Use the following procedure to verify the operation of your HP-UX IPSec configuration. 1. Commit the batch file operations to the configuration database with the following command: ipsec_config batch batch_file_name 2. Verify the contents of the configuration database with the following command: ipsec_config show all The ipsec_config utility displays the contents of the configuration database.
You should see two IPsec SAs (one for each direction) and one IKE SA. The output will be similar to the following: ------------- IPsec SA ---------------Sequence number: 1 SPI (hex): BE882 State: MATURE SA Type: ESP with AES128-CBC encryption and HMAC-SHA1 authentication Src IP Addr: 10.1.1.1 Dst IP Addr: 10.2.2.
field. This is the action taken by HP-UX IPSec. Match the transform configured for the IPsec policy pass or discard ). For more information on the ipsec_report command, refer to the ipsec_report(1M) manpage. 7. Verify any entries in the bypass list.
Step 9: Configuring HP-UX IPSec to Start Automatically After you have verified your HP-UX IPSec configuration is properly operating, you can configure HP-UX IPSec so that it starts automatically at system startup time. TIP: HP recommends that you configure HP-UX IPSec to start automatically at system startup time once you have a known, good HP-UX IPSec configuration. This allows HP-UX IPSec to secure your system at all times.
Step 10: Creating Backup Copies of the Configuration Files Create backup copies of the following files, as applicable: • • • The configuration database file, /var/adm/ipsec/config.db Your batch file. If you do not have a batch file, use the ipsec_config export command to create one from the configuration database. See “Exporting the Configuration Database to a Batch File” (page 138) for more information. The configuration profile file. The default profile file is/var/adm/ipsec/.
5 Using Certificates with HP-UX IPSec This chapter describes how to use security certificates with HP-UX IPSec.
Overview You must use security certificates if you are using digital signatures (RSA signatures) for IKE authentication. HP-UX IPSec uses the certificates to obtain cryptography keys for digital signatures and to verify the digital signatures. If you are not using digital signatures for IKE authentication, you can skip this chapter. Security Certificates and Public Key Cryptography Security certificates are used for public key cryptography , also referred to as asymmetric key cryptography.
IKE Authentication with RSA Signatures HP-UX IPSec supports IKE authentication with RSA signatures in IKE SA negotiations where the IKE entities exchange certificates. The initiator sends an authentication “challenge” to the responder: the initiator sends data, including a random number (nonce), encrypted using the responder’s public key.
A CA certificate must be stored in cACertificate;binary attribute. A CRL must be stored in a certificateRevocationList;binary attribute. Some vendors publish the CRL as a certificateRevocationList;binary attribute of a certificationAuthority object. Other vendors publish the CRL as a certificateRevocationList;binary attribute of a cRLDistributionPoint object.
Configuring Certificates Use the following procedure to configure certificates for HP-UX IPSec. You must also complete the configuration tasks for the main product components, as described in Chapter 4: “Configuring HP-UX IPSec” (page 65). You create one certificate for each HP-UX IPSec system using RSA signatures for IKE authentication. If the local system is multihomed (has multiple IP addresses), you create one certificate for the system. 1. 2. 3. 4. 5.
Step 1: (Optional) Getting a Certificate for the Local System There are two methods you can use to obtain a certificate for the local system: • Use the ipsec_config add csr command to create a Certificate Signing Request (CSR) for the local system. The ipsec_config utility generates a public/private key pair and unsigned certificate for the local system. To use this method, the CA must accept CSRs in PKCS#10 format.
commonName : The commonName of the DN in printable string format. This field cannot contain commas and must be 64 bytes or less. country : The two-character ISO 3166-1 code for the country listed in the DN, for example US for United States of America. This field cannot contain commas. organization : The organization of the DN, for example Hewlett-Packard. This field cannot contain commas and must be 64 bytes or less. organizationalUnit : The organizationalUnit for the DN, for example Marketing.
Example The following command creates a CSR for the local host with its IPv4 address as the subjectAlternativeName and the DN cn=myhost,c=us,o=hp,ou=lab as the subjectName: ipsec_config add csr -subject “cn=myhost,c=us,o=hp,ou=lab” \ -alt-ipv4 15.1.1.
Submitting the Certificate Signing Request to the CA Submit the PKCS#10 Certificate Signing Request (CSR) to the CA to request a signed certificate for the local system. The ipsec_config utility stores the CSR in the file /var/adm/ipsec/ ipsec.csr. PKI vendors support different methods for submitting CSRs.
Step 2: Adding the Local Certificate After the CA creates signed certificates for the local system, use the ipsec_config add mycert command to add the certificates to the HP-UX IPSec storage scheme. There are two syntax formats for the ipsec_config add mycert command: • ipsec_config add mycert -file The ipsec_config add mycert -file syntax extracts the system certificate from a file. The file can be in PEM, DER, or PKCS#12 format.
Range: 1 - 65535. Default: 389, the IANA registered TCP port number for LDAP. -base search_base Search base for the certificate, in X.500 Distinguished Name (DN) format, such as C=US,O=HP,OU=Lab. The search base with the search filter appended to it form a search path to the location of the userCertificate attribute in the LDAP directory. If there are spaces in the DN, you must enclose the DN in double quotes (““ ). For example, “C=US,O=My Company,OU=Blue Lab”. Default: None.
Step 3: Adding the CA Certificates Use the ipsec_config add cacert command to add CA certificates to the HP-UX IPSec storage scheme. There are two syntax formats for the ipsec_config add cacert command: • ipsec_config add cacert -file The ipsec_config add cacert -file syntax extracts a CA certificate from a file. The file can be in PEM or DER format. See “ipsec_config add cacert -file Syntax” (page 124).
-ldap server The hostname or address of the LDAP server where the CA certificate is stored. Default: None. -port port_number TCP port number for the LDAP server. Range: 1 - 65535. Default: 389, the IANA registered TCP port number for LDAP. -base search_base Search base for the certificate, in X.500 Distinguished Name (DN) format, such as C=US,O=HP,OU=Lab. The search base with the search filter appended to it form a search path to the location of the cACertificate attribute in the LDAP directory.
Step 4: Adding the CRL Use the ipsec_config add crl command to add a CRL to the HP-UX IPSec storage scheme. There are two syntax formats for the ipsec_config add crl command: • ipsec_config add crl -file The ipsec_config crl -file syntax extracts a CA certificate from a file. The file can be in PEM or DER format. See “ipsec_config add crl -file Syntax” (page 126). • ipsec_config add crl -ldap The ipsec_config add crl -ldap syntax retrieves the certificate from an LDAP database.
Default: 389, the IANA registered TCP port number for LDAP. -base search_base Search base for the CRL, in X.500 Distinguished Name (DN) format, such as C=US,O=HP,OU=Lab. The search base with the search filter appended to it form a search path to the location of the certificateRevocationList attribute in the LDAP directory. The search base and search filter must not overlap. For example, the value O=HP can be part of the search base or the search filter, but not both.
Step 5: Retrieving the CRL Using cron If the CA periodically publishes the CRL to an LDAP directory, you can use the following procedure to automatically retrieve it using the cron utility. 1. 2. Enter the ipsec_config add crl -ldap command if you have not already done so. In addition to retrieving the CRL, this command creates a file in /var/adm/ipsec/ crl_cron directory that contains information about the LDAP server. The files in this directory are used by the /var/adm/ipsec/util/crl.
Configuration Example This example shows the sequence of commands used to configure certificates for HP-UX IPSec on the system hostA. In addition, the administrator must complete the configuration tasks described in Chapter 4: “Configuring HP-UX IPSec” (page 65), such as configuring IPsec polices, authentication rules, and IKE policies. 1. Create a CSR. In this example, the peer is an HP-UX system.
# ipsec_config show mycert Certificate: Data: Version: 3 (0x2) Serial Number: 3 (0x3) Signature Algorithm: md5WithRSAEncryption Issuer: C=US, O=HP, OU=LAB, CN=issuerCA/emailAddress=root@foo .hp.
ipsec_config show cacert The ipsec_config show cacert command also displays the valid date range for each CRL (the lastUpdate and nextUpdate fields). In the following example, the file 5b0152d9.0 contains the CA certificate and the file 5b0152d9.r0 contains the CRL. The subject and issuer name is /C=US/O=HP/OU=LAB/CN=myPKI for both objects. # ipsec_config show cacert In directory /var/adm/ipsec/certstore : CA cert : 5b0152d9.0 - subject : /C=US/O=HP/OU=LAB/CN=myPKI CRL : 5b0152d9.
6 Administering HP-UX IPSec This chapter describes common HP-UX IPSec maintenance procedures.
Starting HP-UX IPSec Use the ipsec_admin -start command to start HP-UX IPSec.
Configuring HP-UX IPSec to Start Automatically HP recommends that you configure HP-UX IPSec to start automatically at system startup time once you have a known, good HP-UX IPSec configuration. This allows HP-UX IPSec to secure your system at all times. Use the ipsec_config add startup command to configure HP-UX IPSec to start automatically at system startup time.
Stopping HP-UX IPSec Use the ipsec_admin -stop command to stop HP-UX IPSec. This command performs the following operations: • • • • Flushes the kernel-resident HP-UX IPSec memory structures. Disables the kernel components. Sends IKE DELETE messages to peer IKE entities for the local system’s inbound SAs. The DELETE messages tell the peer that the local system will no longer accept data for the deleted SAs. Stops the HP-UX IPSec daemons.
Changing HP-UX IPSec Operating Parameters The ipsec_admin command supports the following arguments for changing HP-UX IPSec operating parameters: • • • • • • • • • • auditlvl (audit level) auditdir (audit directory) maxsize (maximum audit file size) newpasswd (HP-UX IPSec password) spi_min (lower bound for inbound, dynamic Security Parameters Index; this argument is valid only with the -start argument) spi_max (upper bound for inbound, dynamic key Security Parameters Index; this argument is valid only with
Exporting the Configuration Database to a Batch File The ipsec_config export command exports the contents of the configuration database to a batch file that you can use as input for the ipsec_config batch command. You can then use the batch file to re-create the configuration database if the database is corrupt or lost (see “Re-Creating the Configuration Database” (page 139)), or use the batch file as a base for creating a similar configuration on another system.
Re-Creating the Configuration Database Use the following procedure to re-create the configuration database file (/var/adm/ipsec/ config.db ). 1. Copy the skeleton database file (/var/adm/ipsec/migration/skeleton.db.020002 ) to /var/adm/ipsec/config.db : cp /var/adm/ipsec/migration/skeleton.db.020002 \ /var/adm/ipsec/config.db 2.
Deleting SA Entries The ipsec_admin -deletesa command deletes security association (SA) information. In normal operation, there is no need for you to do this. However, there are cases when the SA information on the local system is not synchronized with information on a remote system, such as when the IPsec subsystem on a remote system terminates abruptly.
7 Troubleshooting HP-UX IPSec This chapter describes procedures for troubleshooting HP-UX IPSec software. It contains the following sections: • • • • “Troubleshooting Utilities Overview” (page 142) “Troubleshooting Procedures” (page 145) “Reporting Problems” (page 151) “Troubleshooting Scenarios” (page 152).
Troubleshooting Utilities Overview HP-UX IPSec provides three troubleshooting utilities: ipsec_admin Returns status information and allows the administrator to change the audit level, audit file directory, audit file size, and enable or disable level 4 (TCP, UDP, IGMP) data tracing. ipsec_report Reports HP-UX IPSec operating parameters and displays the contents of audit files. The output can be displayed to stdout or sent to a file.
Table 7-3 Getting Policy Information (continued) Task Command Show tunnel IPsec policies in the configuration database. ipsec_config show tunnel Show all tunnel IPsec policies in the configuration. ipsec_report -tunnel Show IKE policies in the configuration database. ipsec_config show ike Show IKE policies loaded by the IKE daemon. ipsec_report -ikev1 ipsec_report -ikev2 Show current policy decisions cached by the kernel policy engine.
Table 7-6 Viewing and Configuring Audit Information (continued) Task Command Change the audit level. ipsec_admin -auditlvl [alert|error|warning|informative|debug] Change the audit file directory. ipsec_admin -audit audit_directory Change the maximum audit file size (in kilobytes). ipsec_admin -m[axsize] max_audit_file_size Configure audit parameters for startup time.
Troubleshooting Procedures This section describes the following troubleshooting procedures: • • • • • • “Checking Status” (page 145) “Isolating HP-UX IPSec Problems from Upper-layer Problems” (page 146) “Checking Policy Configuration” (page 146) “Isolating HP-UX IPSec Problems from Upper-layer Problems” (page 146) “Checking Policy Configuration” (page 146) “Configuring HP-UX IPSec Auditing” (page 147) Checking Status HP-UX IPSec has five main modules: • • • • • IKE (ISAKMP/Oakley) daemon (ikmpd ) Policy
• Queries the policy daemon and reports the tunnel IPsec policies. You can also do this by entering the following command: ipsec_report -tunnel • Queries the policy daemon and reports the interfaces in the bypass list. You can also do this by entering the following command: ipsec_report -bypass • Queries the policy daemon and reports the active (configured UP or DOWN, plumbed) IP interfaces, and whether or not HP-UX IPSec is enabled for each interface.
ipsec_policy -sa 15.1.1.1 -sp 65535 -da 15.2.2.2 -dp 23 \ -p tcp -dir out To determine which policies HP-UX IPSec will use for inbound telnet requests to 15.1.1.1 from system 15.2.2.2 (the local system 15.1.1.1 is the telnet server), you can use the following command: ipsec_policy -da 15.1.1.1 -dp 23 -sa 15.2.2.2 -p tcp -dir in -sp 65535 \ Refer to the ipsec_policy(1M) manpage for more information. NOTE: Both examples shown above include a dummy user-space port number (65535) for the client port.
• • • • changes in security parameters, unknown message types, and changing of the HP-UX IPSec password or audit level. error : Error audit entries report error events including recoverable error conditions, syntax errors, unsupported features, bad packets, and unknown message types. warning : Warning audit entries report non-intrusive security events. informative : Informative audit entries provide detailed event logging for troubleshooting.
max_size is the maximum size for each audit file, in kilobytes. The default is 100 kilobytes. When you modify startup parameters in the configuration database, the changes do not take effect until the next time HP-UX IPSec starts. The startup configuration object includes other operating parameters. Any parameters you do not specify are reset to the default values, including the autoboot flag, which determines if HP-UX IPSec starts automatically at system startup time.
• • In some cases, the IKE daemon does not send a response if there is a mismatch in IKE parameters. IKE uses this strategy to avoid responding to attackers. If you have misconfigured IKE parameters, the IKE responder may not send a response. In this case, the IKE initiator's log file will show multiple retransmissions and an error message. For IKEv1 negotiations, the error message includes the text phase1 negotiation failed due to time up.
Reporting Problems Be sure to include the following information when reporting problems: • A complete description of the problem and any error messages. Include information about: — the local system (IP addresses) — IP addresses of relevant remote systems — routing table information (netstat -rn output) if appropriate Also include a description of what works and what does not work. • • • Output from the ipsec_admin -status command. Output from the ipsec_report -all command.
Troubleshooting Scenarios This section contains information about the following common troubleshooting scenarios, including their symptoms and resolutions: • • • • • • • • • • • “HP-UX IPSec Incorrectly Passes Packets” (page 152) “HP-UX IPSec Incorrectly Attempts to Encrypt/Authenticate Packets” (page 153) “HP-UX IPSec Attempts to Encrypt/Authenticate and Fails” (page 153) “IKEv1 SA Negotiation Fails or Times Out (phase1 negotiation failed)” (page 154) “IKEv2 SA Negotiation Fails or Times Out (retransmissi
HP-UX IPSec Incorrectly Attempts to Encrypt/Authenticate Packets Problem IPsec is attempting to encrypt or authenticate (apply a transform) packets that should not be encrypted or authenticated. Symptoms Link errors (unable to connect or connection timeouts) on traffic that should not be encrypted/authenticated. Solution Run the following commands: ping, linkloop (check connectivity) ipsec_policy or ipsec_report -cache and ipsec_report -host (determine the policy being used) Check the configuration file.
Where n is the IKE version and policy_name is the IKEv1 or IKEv2 policy used. For example: Msg: 1258 From: IKMPD Lvl: INFORMATIVE Event: found ikev1 policy: default Date: Mon Mar 2 22:52:09 2009 Determining if the IKEv1 SA Negotiation Succeeded If you are using IKEv1, the output from ipsec_report -sa ike output shows the IKEv1 SA if the IKE SA negotiation succeeded.
Symptoms The output from the ipsec_report -sa ike command does not show the IKEv1 SA. The audit log contains the error phase1 negotiation failed due to time up. Solution • Use the ipsec_report -audit command to view the audit file entries. The audit message phase1 negotiation failed due to time up can indicate either: — A connectivity problem with the remote system. — A mismatch in IKE configuration. HP-UX and other IKE responders will not respond if the initiator sends an unacceptable SA proposal.
NOTE: The audit file on the initiator may also show an message with the text Event: phase2 negotiation failed due to time up waiting for phase1. This message does not always indicate that the phase 1 negotiation was successful and that the IKE daemon started a phase 2 negotiation (IPsec SA negotiation). The IKE daemon starts a timer for the completion of the phase 2 negotiation before it starts the phase 2 negotiation, independent of the status of the phase 1 negotiation.
stores all values prefixed with 0x as hexadecimal values and stores all other values as ASCII values. The ipsec_config command does not allow spaces, and any double quote marks in the command are added to the key value. If you are using RSA signatures, see “IKE Primary Authentication Fails with Certificates” (page 158).
IKEv1 IPsec SA Error The following audit file entries indicate that the responder rejected the IPSec SA negotiation because the initiator proposed an ESP transform with the AES encryption method, but the responder is configured to use 3DES: Msg: 622 From: IKMPD Lvl: WARNING Date: Tue Mar 17 12:42:55 2009 Event: trns_id mismatched: my:3DES peer:AES Msg: 623 From: IKMPD Event: not matched Lvl: ERROR Date: Tue Mar 17 12:42:55 2009 Msg: 624 From: IKMPD Lvl: ERROR Date: Tue Mar 17 12:42:55 2009 Event: no suit
If you see the message Only preshared key can be used for authentication, then the IKE daemon was unable to validate the local certificate. You can also try using preshared keys for primary authentication. You will need to configure the same preshared key on both systems. Check that you have a certificate for the local system and for the root CA. If you are using chained CAs, you must have a certificate for each CA in the authentication chain between the local system and the remote system.
Corrupt or Missing HP-UX IPSec Configuration Database Problem The HP-UX IPSec configuration database file (/var/adm/ipsec/config.db ) is corrupt or missing. Symptoms The symptom vary according to when the problem is detected. HP-UX IPSec modules will log error messages to the audit log file and user utilities will also display the error messages to stdout.
Security Policy Database Limit Exceeded (Kernel Policy Cache Threshold reached or Kernel Policy Cache Threshold exceeded ) Problem The Security Policy Database (SPD) is near or exceeding the soft or hard size limit. Symptoms The SPD is the HP-UX IPSec runtime policy database, with cached policy decisions for packet descriptors (five-tuples consisting of exact, non-wildcard source IP address, destination IP address, protocol, source port, and destination port).
A Product Specifications This appendix lists the HP-UX IPSec product specifications. This chapter contains the following sections: • “Product Files and Directories” (page 163) • “IPsec RFCs” (page 165) • “Product Restrictions” (page 166) • “HP-UX IPSec Transforms” (page 167) • “HP-UX IPSec Operation” (page 169) Product Files and Directories HP-UX IPSec uses the following files and directories: • /sbin/init.d/ipsec Startup and shutdown script. • /usr/sbin/ikmpd IKE daemon.
• 164 /var/adm/ipsec/certstore Contains the following certificate and CRL files: — mycert.pem Certificate for the local system. — mykey.pem Private key for the local system certificate. — rootcert.pem Softlink to the certificate file for the root CA. — hash.0 CA certificate file, where hash is a hash value generated from the subject name. — hash.r0 CRL file where hash is a hash value generated from the issuer's name.
IPsec RFCs The HP-UX IPSec product conforms to the Internet Engineering Task Force (IETF) RFCs listed in Table A-1 (page 165): Table A-1 Supported IPsec RFCs RFC Number RFC Title RFC 2401 Security Architecture for the Internet Protocol RFC 2402 IP Authentication Header RFC 2403 The Use of HMAC-MD5-96 within ESP and AH RFC 2404 The Use of HMAC-SHA-1-96 within ESP and AH RFC 2406 IP Encapsulating Security Payload (ESP) RFC 2407 The Internet IP Security Domain of Interpretation for ISAKMP RFC 240
Product Restrictions HP-UX IPSec product restrictions are described below: • HP-UX IPSec systems cannot act as IP or IPsec gateways. • The action for the host policy in an end-to-end tunnel topology must be PASS. • HP-UX IPSec does not support security for broadcast addresses, including network broadcast, subnet broadcast, multicast, and anycast addresses.
HP-UX IPSec Transforms Comparative Key Lengths Table A-2 lists the key lengths of AH and ESP algorithms. In general, the longer the key length, the more secure the encryption algorithm will be. AES encryption provides the most secure encryption, but should be used with some form of authentication, such as the ESP-AES128-HMAC-SHA1 authenticated ESP transform.
ESP-AES128-HMAC-MD5 ESP using Advanced Encryption Standard encryption with a 128-bit key (AES128) and HMAC-MD5 to generate an ICV. ESP-AES128-HMAC-SHA1 Authenticated ESP using AES128 encryption and HMAC-SHA1 to generate an ICV. ESP-NULL-HMAC-MD5 ESP header and trailer, but nothing is encrypted. ESP generates an ICV using HMAC-MD5. ESP-NULL-HMAC-SHA1 ESP header and trailer, but nothing is encrypted. ESP generates an ICV using HMAC-SHA1.
HP-UX IPSec Operation To troubleshoot HP-UX IPSec, it is useful to understand a few key points about its operation. This section contains high-level descriptions of the message flow HP-UX IPSec uses when establishing Security Associations (SAs) and how HP-UX IPSec processes packets.
IKEv1 Main Mode In a MM exchange, the IKE entities use six messages to establish the IKE SA: Figure A-1 IKEv1 Main Mode 170 • Message 1: Initiator sends IKE SA proposals The node initiating the IKE exchange (the IKE initiator) sends IKE SA proposals, which contain IKE SA parameters including authentication and encryption algorithms, Oakley (Diffie-Hellman) group number, and lifetimes.
IKEv1 Aggressive Mode In an Aggressive Mode (AM) exchange, the IKE entities use three messages to establish the IKE SA: Figure A-2 IKEv1 Aggressive Mode • Message 1: Initiator sends IKE SA proposals, Diffie-Hellman public value, IKE ID, and authentication data The initiator sends IKE SA parameters, Diffie-Hellman public value, IKE ID, and authentication data.
The traffic selectors specify the endpoints for the IPsec SA by IP address, protocol, and port number. • Message 2: Responder sends accepted IPsec SA proposal, SPI, and traffic IDs In message 2, the responder sends back the accepted IPsec SA proposal and the SPI for the responder's inbound IPsec SA (for packets to the responder from the initiator). • Message 3: Initiator sends hash message to prove liveness In Message 3, the initiator contains a hash of data sent by the responder in message 2.
The IPsec SA proposals include the transformation(s) used (ESP and/or AH). The initiator also sends the SPI to identify the initiator's inbound IPsec SA (for packets to the initiator from the responder). • Message 4: Responder sends IKE ID, authentication data, accepted IPsec SA proposal, and IPsec traffic IDs In message 4, the responder sends it IKE ID information and authentication data. If the IKE authentication method is RSA signatures, the responder sends its certificate.
• Audit daemon, secauditd The audit daemon receives audit messages from the other modules and logs them in an audit file. • User utilities The ipsec_config, ipsec_admin, ipsec_report, and ipsec_policy utilities enable the user to modify the configuration, start and stop HP-UX IPSec, report status, and test policies. Outbound Data Processing The following sections describe outbound data processing.
The policy manager daemon also checks if the host policy specifies the name of a tunnel policy. If no tunnel policy is specified, the policy manager daemon adds an entry for the five-tuple to the kernel policy engine cache and the packet passes in clear text. If the matching host policy specifies the name of a tunnel policy, the policy manager daemon verifies that the packet five-tuple matches any of the source and any of the destination values in the tunnel policy.
high-level description of how HP-UX IPSec establishes IKE and IPsec SAs and uses configuration data in IKE negotiations. Determining the IKE Version To determine the IKE version, the IKE daemon searches the authentication records in priority order for a record with a remote value that matches the remote system’s IP address.
Initiator Sends Message 3 The initiator sends message 3 in the MM exchange, which includes its Diffie-Hellman public value from the group specified by the group value in the IKEv1 policy. If the remote_method value is RSASIG, the message includes a request for the peer's certificate.
Initiator Receives Message 6 When the initiator receives message 6, it: • Verifies that the ID payload from the responder matches the rtype and rid values in the authentication record. • If the remote_method value in the authentication record is RSASIG, the daemon verifies that the contents of the ID payload matches the appropriate field (subjectName or subjectAlternativeName) in the responder's certificate. • Verifies that the hash type is appropriate for the remote_method value.
• • Uses the values in the selected IKEv1 policy to evaluate the IKE SA proposals sent by the initiator as described in “IKE and IPsec SA Proposals” (page 183). Uses the initiator's Diffie-Hellman public value and its Diffie-Hellman private value (from the group specified by the group value in its IKEv1 policy) to calculate a shared secret value. This shared secret value is used as keying material. Responder Sends Message 2 The responder sends message 2 in the AM exchange.
Initiator Sends Message 1 The initiator sends message 1 in the QM exchange that includes information from the following configuration parameters: • action from the host policy as the IPsec SA proposal. If the action value contains multiple transforms, IKE sends multiple IPsec SA proposals. • source (address and port number) and protocol parameters from the host policy as the IPsec initiator traffic selector (initiator client ID).
IKEv2 Negotiations If the IKE version is IKEv2, the negotiation for the initial IPsec SA pair is combined with the negotiation for the IKEv2 SA in messages 3 and 4. Initiator Sends Message 1 The IKE daemon on the initiator selects an IKEv2 policy by searching the IKEv2 policies in priority order and selecting the first policy with a remote address (remote parameter in the policy) that matches the address of the remote system. The IKE daemon sends message 1 in the negotiation.
• the interface used to send the packet as the local ID value and the address type (IPV4 or IPV6) as the ID type. If the local_method value in the authentication record is PSK, the message includes a hash value calculated from the preshared key. If the local_method value is RSASIG, the message includes the local certificate and a digital signature calculated using the certificate private key. • • • action from the host policy, sent as the IPsec SA proposals.
Message 4 also includes the SPI for the inbound IPsec SA on the responder. Initiator Receives Message 4 When the initiator receives message 4, it: • Verifies that the IDr matches the rtype and rid values in the authentication record. • Verifies the authentication data. If the remote_method value in the authentication record is PSK, it verifies the hash value using the preshared key. If the value is RSASIG, it verifies the digital signature using the public key from the responder's certificate.
If the host policy is not shareable (the EXCLUSIVE flag is set), and HP-UX IPSec is the initiator in the IPsec negotiation, the IKE daemon sends one TSi and one TSr with the exact addresses, ports, and protocol that match the packet. Tunnel Policies The behavior for tunnel policies in IPsec SA negotiations using IKEv1 and IKEv2 is described in the sections that follow.
• IP may send ICMP Redirect messages to redirect traffic to a different gateway. The transmission of ICMP Redirect messages is controlled by the IP kernel parameter ip_send_redirects. By default, this feature is enabled on all HP-UX systems. Refer to the ndd(1M) manpage for information on checking or changing this parameter value. • IP may send ICMP Source Quench messages to request the source system to decrease its transmission rate.
If you configure an host policy that specifies the protocol value ALL or ICMPV6 and do not specify an ICMPv6 type (you do not specify -src_icmpv6_type or -dst_icmpv6_type), only the following ICMPv6 messages affected by the policy: • Echo Request • Echo Reply • Mobile Prefix Advertisement • Mobile Prefix Solicitation Syntax If you specify ICMPV6 for the protocol argument in a host policy, you can specify ICMPv6 message type values for the packet filter using the -dst_icmpv6_type and -src_icmpv6_type argumen
B Interoperability This appendix contains following information about using HP-UX IPSec with other IPsec implementations and contains the following sections: • “Microsoft” (page 188) • “Linux” (page 190) • “FreeBSD” (page 192) • “Cisco” (page 198) 187
Microsoft HP-UX IPSec can interoperate with Microsoft IPsec implementations. Versions and Functionalities HP-UX IPSec A.03.
(port 514) service on the Windows 2008 server (10.0.0.208) from the HP-UX system (10.0.0.11): netsh advfirewall consec add rule name=iop-rule enable=yes endpoint1=10.0.0.208 endpoint2=10.0.0.11 protocol=tcp port1=514 action=requireinrequireout auth1=computerpsk auth1psk=MyKey Rules configured using the netsh advfirewall command-line context are bidirectional. IPsec will be used for TCP packets from 10.0.0.11 to port 514 on 10.0.0.208 and for packets in the reverse direction.
Linux HP-UX IPSec can interoperate with Linux IPsec. Version and Functionalities HP tested HP-UX IPSec with Linux IPsec functionality provided by ipsec-tools version 0.7, which is included with Linux kernel versions 2.6 and later.
NOTE: A compression_algorithm statement is required even though IP compression is not used (HP-UX IPSec does not support IP compression). ############## racoon.conf configuration file # see racoon.conf(5) # start racoon with racoon -F -f this_file path pre_shared_key "/root/linux-native-racoon/psk_xport_1/psk.txt"; remote 10.0.0.11 { exchange_mode main; proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method pre_shared_key; dh_group modp1024; } } sainfo address 10.0.0.
FreeBSD HP-UX IPSec can interoperate with FreeBSD IPsec implementations. Version and Functionalities HP tested with the FreeBSD 6.3 release using IPsec functionality provided by the Racoon2 20090218 CVS build.
interface { ike { MY_IP port 500; }; spmd { unix "/var/run/racoon2/spmif"; }; spmd_password "/usr/local/racoon2/etc/racoon2/spmd.pwd"; }; resolver { resolver off; }; include "/usr/local/racoon2/etc/racoon2/default.conf"; include "/usr/local/racoon2/etc/racoon2/transport_ike.conf"; transport_ike.conf File The contents of the transport_ike.conf file are as follows: ######################### ## /usr/local/racoon2/etc/racoon2/transport_ike.
vals.conf File The relevant sections of the vals.conf file are as follows: ## /usr/local/racoon2/etc/racoon2/vals.conf setval { # Preshared key file directory : specify to use preshared keys PSKDIR "/usr/local/racoon2/etc/racoon2/psk"; # Preshared Key file name # You can generate it by pskgen. PRESHRD_KEY "test.psk"; : : ### Transport Mode Settings ### # Your IP Address MY_IPADDRESS "10.0.0.63"; # Peer's IP Address PEERS_IPADDRESS "10.0.0.11"; : : } default.conf File The default.
Racoon2 Configuration The following Racoon 2 configuration files are located in the /usr/local/racoon2/etc/ racoon directory. • racoon2.conf • transport_ike.conf • vals.conf • default.conf Certificate Files In this example, the following certificate-related files are located in the /usr/local/racoon2/ etc/racoon/cert subdirectory: • myPubKey.pem: public key for the local (FreeBSD) system • myPvtKey.pem: (private key for the local (FreeBSD) system • hpuxPeerPubkey.
: : ### Transport Mode Settings ### # Your IP Address MY_IPADDRESS "10.0.0.64"; # Peer's IP Address PEERS_IPADDRESS "10.0.0.11"; : : }; default.conf File The default.conf file installed with Racoon2 is used without modifications. Configuration Example: IKEv2 Using Preshared Keys The following configuration data is for an IKEv2 topology using preshared keys for end-to-end IPsec SAs. The address for the Free BSD 6.3 system is 10.0.0.65. The address for the HP-UX system is 10.0.0.11.
peers_id ipaddr "${PEERS_IPADDRESS}"; peers_ipaddr "${PEERS_IPADDRESS}" port 500; kmp_enc_alg { 3des_cbc; }; kmp_hash_alg { hmac_sha1; }; kmp_dh_group { modp1024; }; ## Use Preshared Key kmp_auth_method { psk; }; pre_shared_key "${PSKDIR}/${PRESHRD_KEY}"; }; selector_index ike_trans_sel_in; }; vals.conf File The relevant sections of the vals.conf file are as follows: ## /usr/local/racoon2/etc/racoon2/vals.
Cisco HP-UX IPSec can interoperate with Cisco IOS IPsec implementations. Version and Functionality HP-UX IPSec has been successfully tested with the following Cisco product: Model 2821, version 12.4. The following functionality was tested: • IKEv1 using preshared key authentication for a host-to-gateway tunnel (HP-UX Host1 to the Cisco router), with end-to-end clear text Example In the following topology, the HP-UX system with address 192.0.0.2 creates an IPsec tunnel to the Cisco router with address 192.
Router (config)# crypto ipsec transform-set aes-sha1 esp-aes 128 esp-sha-hmac Router (cfg-crypto-trans)# mode tunnel Router (cfg-crypto-trans)# exit Define an IPsec policy map: Router(config)# access-list 100 permit ip host 192.1.1.2 host 192.0.0.2 Router(config)# crypto map hpux-1 1 ipsec-isakmp Router (config-crypto-map)# set peer 192.0.0.
C Migrating from Previous Versions of HP-UX IPSec This appendix provides information on migrating to the current version of HP-UX IPSec from previous versions. This appendix contains the following sections: • “Pre-Installation Migration Instructions” (page 201) • “Post-Installation Migration Instructions” (page 201) DES Compatibility HP-UX IPSec version A.03.00 does not support DES encryption.
1. Run the ipsec_migrate utility after you have installed HP-UX IPSec A.03.00. For example: /usr/sbin/ipsec_migrate If the /var/adm/ipsec/ipsec.key file is present, ipsec_migrate prompts for the HP-UX IPSec password before decrypting this file and extracting the contents. The ipsec_migrate utility creates backup copies of the following files and saves them in the files under the /var/adm/ipsec/backup directory: /var/adm/ipsec/.ipsec_profile /var/adm/ipsec/cainfo.txt /var/adm/ipsec/config.
• • • • /var/adm/ipsec/.ipsec_profile file. For host policies, the default action is DISCARD. For tunnel policies, the default action is the ESP_AES128_HMAC_SHA1 transform. Check the priority value in authentication records. In previous releases, authentication records did not have a priority value; if multiple authentication records had a remote IP address value that matched the peer's address, HP-UX IPSec selected the record with the longest IP address prefix.
D HP-UX IPSec Configuration Examples This appendix provides configuration examples for the following topologies: • “Host to Host telnet” (page 206) This section contains example ipsec_config batch files for encrypting and authenticating all telnet traffic between two systems using dynamic keys and preshared keys for IKE authentication.
Host to Host telnet You have two systems, Apple (15.1.1.1 ) and Banana (15.2.2.2 ) on a private, isolated LAN. You want to use authenticated ESP with AES encryption and SHA-1 authentication for all telnet traffic from Apple to Banana, and for all telnet traffic from Banana to Apple. By default, all other network traffic will pass in clear text.You do not have a Public Key Infrastructure, so you can use only preshared keys for IKE primary authentication.
-destination 15.2.2.2/32/TELNET \ -priority 20 -action ESP_AES128_HMAC_SHA1 add host telnetBA \ -source 15.1.1.1/32/TELNET \ -destination 15.2.2.2 \ -priority 30 -action ESP_AES128_HMAC_SHA1 Authentication Record with Preshared Key You must configure the preshared key to use when Apple authenticates system Banana’s identity and to authenticate Apple’s identity to Banana. The ipsec_config batch file entry is listed below: add auth banana -remote 15.2.2.
Subnet ESP with Exceptions You have a system, Carrot, on a LAN with the network address 192.1.1.*. You want to limit access to this LAN from outside nodes. There is one system outside the LAN with IPsec, Potato, that you will allow to communicate with the nodes in your network using AES with SHA1. All other packets from external nodes will be discarded. All nodes within the LAN have HP-UX IPSec installed, except for internal routers.
-protocol ICMP -priority 30 -action pass add host aes_lan -destination 192.1.1.0/24 \ -priority 40 -action ESP_AES128_HMAC_SHA1 add host default -action DISCARD Policy Priority Note the priority of the pass_icmp policy (30) and aes_lan policy (40). The pass_icmp policy MUST have a lower order number (higher priority) than the aes_lan policy.
Host to Gateway On system Blue (15.5.5.5 ), you configure HP-UX IPSec to communicate back to Home1 (17.7.7.7 ) using a secure IPsec tunnel to a gateway (a router), accessed using its 16.6.6.6 address. The end-to-end packets pass in clear text. Blue must use the router as the gateway to Home1. You may need to configure an explicit IP route to Home1 that specifies 16.6.6.6 as the gateway address. The gateway (router) cannot be an HP-UX system. (HP-UX IPSec does not support gateway functionality.
Autoconfiguration Clients The system Server1 has the address 2001:db8:11:11::1111 on the subnet 2001:db8:11:11::/64 . This subnet has three autoconfiguration clients, configured with the user FQDN IKE IDs joe_s@corp.com , mick_j@corp.com , andpaul_s@corp.com . Server1 Configuration The configuration on Server1 specifies the subnet address for the autoconfiguration clients as the remote address.
add auth paul_s \ -remote 2001:db8:11:11::/64 \ (autoconf client subnet addr. ) -ltype FQDN -lid server1.corp.com -rtype USER-FQDN -rid paul_s@corp.com -exchange AM -preshared secret3333 \ -flags AUTOCONF IKE Policy The default IKEv1 policy is used with no modifications. Client Configuration The configuration is the same on each client, except for the local ID in the authentication record. This section lists the configuration for the system with local ID joe_s@corp.com .
E HP-UX IPSec and HP-UX IPFilter This appendix describes configuration requirements when using HP-UX IPSec and HP-UX IPFilter on the same system.
Using HP-UX IPSec with HP-UX IPFilter HP-UX IPSec and HP-UX IPFilter can coexist on the same system. You can configure HP-UX IPSec and HP-UX IPFilter so that there is some overlap in the configurations. However, you must be sure the overlapping configurations do not block each other. HP-UX IPFilter is located below HP-UX IPSec in the networking stack. HP-UX IPFilter processes inbound IP packets before HP-UX IPSec and processes outbound packets after HP-UX IPSec.
F Using Manual Keys This appendix describes how to configure and troubleshoot manual keys for IPsec SAs. Manual keys are an alternative to IKE. Instead of dynamically generating and distributing cryptography keys for ESP and AH, the cryptography keys are static and manually distributed. Manual keys are typically used only when the remote system does not support IKE.
Configuring Manual Key SAs You specify information for manual key SAs with -in and -out statements in host and tunnel policies: -in manual_key_sa_specification -out manual_key_sa_specification The format for manual_key_sa_specification is: ESP/spi /auth_key /enc_key [/iv] ESP indicates the transform is an ESP transform. spi is the decimal or hexadecimal (prefixed by 0x) Security Parameters Index (SPI) number, used to identify the Security Association (SA).
od -Ax -Nnn /dev/random nn is the number of bytes to extract from the random number generator.
Manual Key Configuration Example You want to secure rlogin sessions from the system Dog (10.2.2.2 ) to the system Cat (10.4.4.4 ) using manual keys. There is no configuration for rlogin sessions from Cat to Dog; these sessions will use the default host IPsec policy and pass in clear text. Dog Configuration The ipsec_config batch file on Dog contains only one host IPsec policy.
Troubleshooting Manual Key Problems Troubleshooting manual key problems can be difficult because there are no IKE negotiations and no IKE audit messages. Symptoms Link errors (unable to connect ) and timeouts. The output from the ipsec_report -sa ipsec command shows the SAs, but attempts to exchange data with the remote system fail.
Examining STREAMS Logging Records You can use the strace utility to view STREAMS log records, or use the following procedure to examine the nettl log file for entries logged by the HP-UX IPSec STREAMS modules. 1. Execute the following command to determine the current nettl log file (the default is /var/adm/nettl.LOG000) and the current log classes for the STREAMS subsystem: nettl -ss The default STREAMS log classes are error and disaster.
G HP-UX IPSec and Serviceguard HP-UX IPSec can secure HP-UX Serviceguard network traffic. This appendix describes how to configure HP-UX IPSec as an Serviceguard package service so a package will fail or fail over if HP-UX IPSec terminates.
Introduction An Serviceguard cluster is a networked group of HP 9000 or Integrity servers (host systems known as nodes) with redundant hardware and software so that a single point of failure does not significantly disrupt service. Application packages (individual HP-UX processes) can be grouped together in failover packages .
• • automatically establish IPsec SAs with the adoptive node as needed, without operator intervention. HP-UX IPSec is compatible with redundant network interfaces. If Serviceguard fails over from a primary interface to a standby interface, HP-UX IPSec will continue operating and use the standby interface with no change in the operation of the Security Associations (SAs). HP-UX IPSec includes a monitor script, /var/adm/ipsec/ipsec_status.
Configuration Overview Requirements To use HP-UX IPSec with Serviceguard, your topology must meet the following requirements: • The same version of HP-UX IPSec must be installed on all cluster nodes. • Serviceguard version A.11.16 or later must be installed on all cluster nodes. • All cluster nodes must have the same HP-UX IPSec configuration database file. Serviceguard Heartbeat Requirement and Recommendation • • You must allow Serviceguard heartbeat messages to pass in clear text.
• “Step 7: Distributing HP-UX IPSec Configuration Files” (page 244) After you have tested the HP-UX IPSec configuration, distribute the IPsec configuration files to the other nodes in the cluster. NOTE: You must maintain the same HP-UX IPSec configuration information on each cluster node. If you make HP-UX configuration changes after you have started the cluster, you must make the same changes on all cluster nodes.
Step 1: Configuring HP-UX Host IPsec Policies for Serviceguard Overview Use the procedure described in Chapter 4, “Step 1: Configuring Host IPsec Policies” (page 72) to configure host IPsec policies, with the following additional requirements: • Configure PASS host IPsec policies for all packets sent between the heartbeat IP addresses. This ensures that Serviceguard does not unnecessarily reform the cluster because of delays introduced by HP-UX IPSec.
Configuring PASS Host IPsec Policies for Intracluster Messages Configure a PASS host IPsec policy (host IPsec policy with -action PASS ) for each pair of heartbeat IP addresses in the cluster to ensure that Serviceguard heartbeat and intracluster control messages pass in clear text. Since the IPsec configuration database is the same for all cluster nodes, you must configure a PASS host IPsec policy for each heartbeat IP address pair in the cluster.
Serviceguard Quorum Server If you are using a Quorum Server for the Serviceguard cluster, configure HP-UX IPSec so it does not discard packets listed in the sections below. Cluster Node IPsec Policies for Quorum Server For each cluster node, configure host IPsec policies so HP-UX IPSec does not discard (the transform list contains any transform except DISCARD ) the packets listed below. If HP-UX IPSec is not installed on the Quorum Server, configure PASS host IPsec policies for these packets.
For remote execution of the cmscancl command, HP-UX IPSec must not discard the following packets: Source IP Address Destination IP Address Protocol cluster node address (or wildcard) remote command client TCP address Source Port Destination Port 514 0 Remote Command Client Host IPsec Policies If HP-UX IPSec is installed on the remote command clients, configure host IPsec policies for the packets listed below with actions (PASS or transform lists) that match the policies on the cluster nodes.
Source IP Address Destination IP Address cluster node address (or wildcard) cluster node address (or wildcard) Protocol Source Port Destination Port SMH Management Station TCP address 2381 0 SMH Management Station UDP address 2381 0 SMH Management Station Host IPsec Policies If HP-UX IPSec is installed on the SMH Management Station, configure host IPsec policies for the packets listed below with actions (PASS or transform lists) that match the policies on the cluster nodes.
Standalone Serviceguard Manager Host IPsec Policies If HP-UX IPSec is installed on the standalone Serviceguard Manager system, configure host IPsec policies for the packets listed below with actions (PASS or transform lists) that match the policies on the cluster nodes.
Cluster Object Manager (COM) If you are using a Cluster Object Manager (COM) on a system outside the cluster to provide connections to COM clients, such as Serviceguard Manager clients, configure HP-UX IPSec so it does not discard the packets listed in the sections that follow. Cluster Node Host IPsec Policies for COM For each cluster node, configure host IPsec policies so HP-UX IPSec does not discard (the action is not DISCARD ) the packets listed below.
SMH Management Station Host IPsec Policies If HP-UX IPSec is installed on the SMH Management Station, configure host IPsec policies for the packets listed below with actions (PASS or transform lists) that match the policies on the cluster nodes.
Table G-1 Serviceguard Port Numbers and Protocols (continued) Port Protocols Service 5304 TCP HA Cluster Commands (hacl-local). Used as the destination port between the cluster nodes. 5305 TCP HA Cluster Test (hacl-test). Used as the destination port between cluster nodes. 5315 TCP HA Polling for cmappserver (hacl-poll). Used as the destination port between cluster nodes. 5408 TCP HA Distributed Lock Manager (ha-dlm). Used as the destination port between cluster nodes.
Step 2: Configuring HP-UX IPSec IKE policies Configure IKE policies as described in Chapter 4, “Step 4: Configuring IKEv1 and IKEv2 Policies” (page 95). In most cases, you can use the default IKEv1 or IKEv2 policy without changes. Cluster IKE policies The cluster nodes must have IKE policies with remote address specifications for the cluster clients. Cluster Client IKE policies The cluster clients must have IKE policies with remote address specifications that include the package addresses.
Step 3: Configuring Authentication Records for Preshared Keys This section describes configuration requirements for authentication records if you are using preshared keys for IKE authentication. If you are not using preshared keys for IKE authentication, go to “Step 4: Configuring Authentication Records for Certificates” (page 239). The preshared key information must be the same on all nodes in the cluster. Configure authentication records with preshared keys on one Serviceguard cluster node.
-ltype FQDN -lid mycluster.hp.com \ -rtype IPV4 -rid 15.5.5.5 \ -psk my_client2_key add auth client1 -remote 15.4.4.4 -ltype IPV4 -lid 15.1.1.1 add auth client2 -remote 15.5.5.5 -ltype IPV4 -lid 15.1.1.1 If the cluster clients were multihomed, you would also add entries for the additional addresses on the cluster clients, and specify local ID type and local ID value arguments. Authentication Records on Client1 On client1, the ipsec_config batch file contains the following entries.
Preshared Keys Configuration on Client2 Client2 has the following preshared keys configured: 238 Remote IP Address Key 16.98.98.98 (pkgA) client2_key 16.99.99.
Step 4: Configuring Authentication Records for Certificates This section describes configuration requirements for authentication records if you are using security certificates (RSA signatures) for IKE authentication. If you are not using security certificates for IKE authentication, go to “Step 5: Verifying and Testing the HP-UX IPSec Configuration” (page 242). All nodes in an Serviceguard cluster share the same certificate and IKE ID configuration.
NOTE: If all clients are in the same subnet and use FQDNs or X.500 DNs with a common base or IP addresses for IDs, you can use a subtree or address range remote ID to configure one authentication record for all clients. For more information, see “Subtree and Address Range Remote ID Matching” (page 91). Cluster Clients On each cluster client, configure an authentication record for each package address in the cluster.
-rtype IPV4 -rid 15.4.4.4 add auth client2 -remote 15.5.5.5 -kmp IKEV1 \ -ltype FQDN -lid mycluster.hp.com \ -rtype IPV4 -rid 15.5.5.5 Authentication Records on Client1 and Client2 On each cluster client, the ipsec_config batch file contains the following entries. The authentication records use the default local ID type (IPV4) and default local ID value (the IP address of the interface used to communicate with the remote system).
Step 5: Verifying and Testing the HP-UX IPSec Configuration Start and verify HP-UX IPSec on the cluster node on which you configured IPsec using the procedure in Chapter 4, “Step 8: Committing the Batch File Configuration and Verifying Operation” (page 107).
Step 6: Configuring HP-UX IPSec Start-up Options HP-UX IPSec must be running on all nodes in the cluster before Serviceguard starts. After you have verified the configuration, you can configure HP-UX IPSec to start automatically at system startup time. See Chapter 4, “Step 9: Configuring HP-UX IPSec to Start Automatically” (page 110) to configure HP-UX IPSec to start automatically at system boot-up time.
Step 7: Distributing HP-UX IPSec Configuration Files After you have verified and tested the HP-UX IPSec configuration on the configuration node, distribute the HP-UX IPSec configuration database file, /var/adm/ipsec/config.db, to the other nodes in the cluster. NOTE: Do not redistribute the configuration database file if HP-UX IPSec is running. If you need to modify the configuration while HP-UX IPSec is running on the cluster, use an ipsec_config batch file to make changes on one system.
NOTE: If you do not want to use the above procedure, you can use the following methods to securely distribute the private key: • Transfer the private key file using SSH FTP. • Copy the private key file to removable media and physically transfer the file to the other cluster nodes. crontab File If the CRL is stored in an LDAP directory and you want to automatically retrieve the CRL periodically, you must also modify the root user’s crontab file (/var/spool/cron/crontabs/ root ) on each cluster node.
Step 8: Configuring Serviceguard Configure Serviceguard according to the Serviceguard product documentation, with the additional requirements listed below. Verify the Serviceguard configuration using the cmcheckconf command, as described in the Serviceguard product documentation. Cluster Configuration HP strongly recommends that you do not secure heartbeat messages using IPsec (with AH or ESP).
Step 9: Starting HP-UX IPSec and Serviceguard HP-UX IPSec must be running on all cluster nodes with the same HP-UX IPSec configuration files before you start the Serviceguard cluster. Use the following procedure to start HP-UX IPSec and Serviceguard. 1. Start HP-UX IPSec. There are two ways to start HP-UX IPSec: • Manually, using the ipsec_admin -start command. • Automatically, at system boot-up time.
Glossary 3DES Triple Data Encryption Standard. A symmetric key block encryption algorithm that encrypts data three times, using a different 56-bit key each time (168 bits are used for keys). 3DES is suitable for bulk data encryption. AES Advanced Encryption Standard. Uses a symmetric key block encryption. HP-UX IPSec supports AES with a 128-bit key. AES is suitable for encrypting large amounts of data.
ESP The ESP (Encapsulating Security Payload) protocol provides confidentiality (encryption), data authentication, and an anti-replay service for IP packets. When used in tunnel mode, ESP also provides limited traffic flow confidentiality. filter The parameters in an IPsec policy that HP-UX IPSec uses to select the policy applied to an IP packet. The parameters are the source and destination IP addresses, protocol, and source and destination port numbers. HMAC Hashed Message Authentication Code.
Aggressive Mode (AM) A mode used in IKEv1 Phase 1 negotiations to establish IKE SAs. AM is less secure than Main Mode because the IKE peers exchange identity information before establishing a secure channel, but requires the IKE peers to exchange fewer packets (three instead of six). The IKE protocol specification does not require implementations to support AM. Main Mode (MM) A mode used in IKEv1 Phase 1 negotiations to establish IKE SAs.
Security Parameters Index (SPI) A numerical index used with the source IP address to identify a Security Association (SA) on the receiving system. Each SPI and source IP address pair must be unique on the receiving system. SHA1 (Secure Hash Algorithm-1). Authentication algorithm that generates a 160-bit message digest using a 160-bit key. IPsec truncates the message digest to 96 bits. SPI See Security Parameters Index..
Index Symbols 3DES (Triple Data Encryption Standard), 36, 249 configuring in host IPsec policies, 77 configuring in IKEv1 policies, 98 configuring in IKEv2 policies, 100 configuring in tunnel IPsec policies, 82 key length, 167 A AES (Advanced Encryption Standard), 36, 249 configuring in host IPsec policies, 77 configuring in IKEv2 policies, 100, 101 configuring in tunnel IPsec policies, 82 key length, 167 recommendation, 36 Aggressive Mode (AM), 39 configuring in authentication records, 87 SA, 250 (see als
group, 249 configuring in IKEv1 policies, 97 configuring in IKEv2 policies, 100 digital signature, 40 using with IPsec, 30 disk requirements, 48 Distinguished Name specifying in authentication record, 89 specifying in CSR, 118 DMZ securing with IPsec, 45 E EEXIST error message, 219 Encapsulating Security Payload (see ESP) encryption algorithms, 167 configuring in IKEv1 policies, 98 configuring in IKEv2 policies, 100 definition, 249 keys, 169 generating, 216 error messages can't find matching selector, 157
topologies, 43 IPsec policy configuring overview, 70 default, 72 definition, 250 selection process, 72 IPsec QM SA, 250 (see also IPsec SA) ipsec_admin, 60, 107, 142, 147 -auditlvl option, 144 -status option, 142 to change audit directory, 144 to change audit level, 144 to get HP-UX IPSec status, 142 ipsec_config add auth examples, 92, 93 syntax, 85 ipsec_config add bypass example, 105 syntax, 105 ipsec_config add cacert example, 124 syntax, 124 LDAP, 124 ipsec_config add crl syntax, 126 ipsec_config add cs
configuring in IKEv1 policies, 97 configuring in tunnel IPsec policies, 82 key length, 167 RFC, 165 MM SA, 250 (see also IKE SA) configuring in tunnel IPsec policies, 81 pseudo-random function configuring in IKEv2 policies, 101 public key, 40, 114 using with IPsec, 30 N QM SA, 250 (see also IPsec SA) Quick Mode (QM) definition, 251 Quick Mode SA, 250 (see also IPsec SA) ndd, 151 netstat, 151 no proposal chosen error message, 156, 158 no suitable policy found error message, 158 O Oakley, 250 group confi
processing, 180, 182 startup options configuring, 62, 110, 135 status report, 60, 107 verifying, 145 subjectAlternativeName specifying in CSR, 119 subjectName specifying in CSR, 118 subnets configuring policies for ICMP messages, 185 swinstall(1M), 50 swlist(1M), 49 NIS, 166 User FQDN specifying in authentication record, 89 user FQDN specifying in CSR, 119 V verifying the installation, 60, 107 T tools survey, 142 tracing disabling, 142 enabling, 142 layer 4, 146, 151 traffic selectors configuring in tunn
