HP Auto Port Aggregation Administrator's Guide HP-UX 11i v3 HP Part Number: J4240-90049 Published: January 2010 Edition: January 2010, E1001
© Copyright 2010 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents 1 Introduction......................................................................................................................9 Conceptual overview..............................................................................................................................9 Link aggregate...................................................................................................................................9 Load balancing....................................................
Administering HP APA.................................................................................................39 Modifying a link aggregate...................................................................................................................39 Modifying a failover group...................................................................................................................39 Deleting a link aggregate.............................................................................
A Product specifications..................................................................................................87 B HP APA configuration files..........................................................................................89 hp_apaconf file......................................................................................................................................89 hp_apaportconf file...............................................................................................
List of Figures 1-1 1-2 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 4-1 4-2 D-1 D-2 6 SMH Main Page.............................................................................................................................14 HP APA Main Page.......................................................................................................................15 Sample Enterprise Intranet Client/Server Configuration..............................................................
List of Tables 5-1 A-1 C-1 C-2 Legacy Commands and Corresponding nwmgr Commands.......................................................54 HP APA and LAN Monitor Capabilities.......................................................................................87 lan900 Events and Proactive Failover (Equal Network Costs)....................................................103 lan900 Events and Proactive Failover (Unequal Network Costs)...............................................
List of Examples 4-1 4-2 4-3 5-1 5-2 5-3 5-4 6-1 B-1 8 Sample Link Aggregate 32-Bit Statistics........................................................................................45 Sample Link Aggregate 64-Bit Statistics........................................................................................46 Sample Link Aggregate 32-Bit Statistics Monitoring....................................................................47 nwmgr Output (No Options).................................................
1 Introduction HP Auto Port Aggregation (APA) is a software product that creates link aggregates, often called trunks, which provide a logical grouping of two or more physical ports into a single fat pipe. This port arrangement provides more data bandwidth than would otherwise be available and enables you to build large bandwidth logical links into the server that are highly available and completely transparent to the client and server applications.
• The physical ports in the link aggregation use the same MAC address. By default, the unique MAC address for a specific link aggregate is determined by using the MAC address of one of the ports in the link aggregate. All ports will use the same MAC address. The January 2010 release of APA for HP-UX 11i v3 (B.11.31.40) enables you to assign a fixed MAC address for the link aggregate.
1. 2. 3. Data Flow Lookup — The load distribution algorithm determines an index into a hash table that includes the physical port through which the outbound data flow is forwarded. Data Flow Physical Port Assignment — If the hash index for the data flow has not been assigned a physical port (the entry is empty), a physical port in the link aggregate is assigned to that specific hash index. The physical port is selected on a Round Robin basis.
For each failover group, if you assign a cost value to one link, you must assign a cost value to all other links in the group. If you do not specify a cost value for any of the failover group's links, the failover group uses the default failover behavior based on priority. During certain LAN Monitor events (for example, link failure and link recovery), the normalized port cost might change on the active or standby links.
• • • You can use VLANs over one link aggregate to serve multiple workgroups. This also enables broadcast traffic to be isolated within the same broadcast domain, offering improved security for workgroups. The same link aggregate or failover group can offer different level of service for each user using ToS. You gain more flexibility in how you deploy link aggregates and failover groups. You can create, remove, and modify VLANs over link aggregates and failover groups without rebooting the system.
1. From a character-cell terminal or terminal window, enter: # smh If the DISPLAY environment variable is set, HP SMH opens in the default web browser. If the environment variable is not set, HP SMH opens in the TUI. Figure 1-1 shows the SMH Main page. Figure 1-1 SMH Main Page 2. 14 Click Tools. The Tools page displays.
3. Scroll down and click Auto Port Aggregation in the Network Interfaces Configuration tool. The APA main page displays as shown in Figure 1-2. Figure 1-2 HP APA Main Page To exit HP SMH, click Sign Out. For more information about the System Management Homepage, see smh(1M) and the online help. nwmgr command You can also use the nwmgr command from the HP-UX command line prompt to make changes to HP APA. By default, those changes are not preserved across reboots.
2 Installing the APA software This chapter describes the information required in order to install APA on your system. Installation requirements 1. 2. 3. Log in to the HP-UX server as superuser. Confirm that the /usr/bin, /usr/sbin, and /sbin directories are in your PATH by using the echo $PATH command. Use the uname -a command to determine the HP-UX version of your system. Product dependencies See the HP-UX Auto Port Aggregation (APA) Release Notes for required patches and known problems and workarounds.
NOTE: HP APA only supports failover groups of LAN interfaces when running on blade hardware. Operating system requirements HP-UX 11i v3 Software requirements For the December 2008 release of HP APA for HP-UX 11i v3 (B.11.31.30) and later releases, if you want to IPoIB interfaces for failover groups, HP APA requires the following software: • DLPI Patch PHNE_36869 or later • IB-4X Driver version B.11.31.
1. Verify that the product was installed by issuing the following command: # swlist -l product | grep -i HP-APA Output similar to the following displays: HP-APA-FMT HP-APA-KRN HP-APA-LM HP-APA-NETMOD HP-APA-RUN B.11.31.20 B.11.31.20 B.11.31.20 B.11.31.20 B.11.31.20 HP HP HP HP HP Auto-Port Auto-Port Auto-Port Auto-Port Auto-Port Aggregation Aggregation Aggregation Aggregation Aggregation APA formatter product. kernel products. LM commands. nwmgr/NCweb libraries. APA command products.
3 Configuring APA This chapter describes how to configure HP APA on your system. This includes: • Reviewing sample HP APA configurations • Preparing for the configuration by gathering information • Configuring systems in sample configurations • Configuring a link aggregate • Configuring a failover group • Configuring the link partner • Performing post-configuration tasks HP APA configuration examples This section shows some sample HP APA configurations.
Figure 3-1 Sample Enterprise Intranet Client/Server Configuration Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 HP ProCurve 8000 Switch Hub 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Hub 0 1 2 3 4 5 6 7 Internet or large enterprise environments using routers You can use HP APA link aggregation successfully in certain environments employing routers.
Figure 3-2 Sample Router and Server Configuration (No Switch) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation Router or Switching Router Intranet Internet Figure 3-3 (page 24) shows a sample router and server configuration with a switch. In this configuration, the switch might present problems because switches typically use a MAC address load-balancing algorithm.
Figure 3-3 Sample Router and Server Configuration (Switch) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 Switch Router or Switching Router Intranet Internet Server-to-Server (Back-to-Back) Figure 3-4 (page 25) shows a sample server-to-server configuration. You create server-to-server aggregations by directly connecting the physical ports in one server's link aggregation to the physical ports in the other server's link aggregation.
Figure 3-4 Sample Server-to-Server Configuration (Back-to-Back) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation Run Attn. Fault RemotePower Hot Standby for high availability Figure 3-5 (page 26) shows a sample MANUAL (Hot Standby) mode configuration. These link aggregations provide high availability network access with an active link and a standby link. NOTE: HP strongly recommends using failover groups (LAN_MONITOR mode) rather than Hot Standby mode.
Figure 3-5 Sample Hot Standby Configuration for High Availability Run Attn. Fault RemotePower Primary Standby Switch or Hub HP APA 2-Port Hot Standby Link Aggregation Switch or Hub Server-to-Server with switch (not recommended) Figure 3-6 (page 27) shows a sample server–to–server HP APA link aggregation configuration with a switch between the servers.
Figure 3-6 Sample Server-to-Server Configuration with Switch (Not Recommended) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 Switch HP APA 2–4 Port Link Aggregation Run Attn. Fault RemotePower Failover group Figure 3-7 (page 28) shows a sample failover group (LAN_MONITOR mode) configuration.
Figure 3-7 Sample Failover Group (LAN_MONITOR) Configuration Run Active Attn. Fault RemotePower Run LAN Monitor 2-Port Failover Group Fault RemotePower LAN Monitor 3-Port Failover Group 2 Standby Links Standby Switch or Hub Active Attn.
Figure 3-8 Sample Failover Group Using Link Aggregates Configuration Run Attn. Fault Remote Power Run LAN Monitor failover group using HP APA link aggregations. Standby LinkAgg Active LinkAgg Attn. Fault Remote Power Active LinkAgg Standby hp ProCurve hp ProCurve hp ProCurve hp ProCurve Switch 8000 Switch 8000 Switch 8000 Switch 8000 LAN Monitor failover group using an HP APA link aggregation as the active link and one physical link as a standby link.
Figure 3-9 HP APA Configuration Worksheet HP APA Configuration Worksheet Aggregate Instance Number Mode FEC_AUTO LACP_AUTO LAN_MONITOR Not_Enabled Failover Priority Based Cost Based Instance Number: Hardware Path: Interface Type: MANUAL Priority: Cost: Advanced Parameters (Link Aggregates) Load Distribution Algorithm LB_MAC LB_IP LB_PORT HOT_STANDBY Group Capability (FEC_AUTO only) Key (LACP_AUTO only) Advanced Parameters (Failover Groups) Dead Count Polling Interval Rapid ARP on of
Mode The configuration mode of the link aggregate or failover group. Your choice will be determined by the capabilities of the link partner (for example, switch, router, or server) to which the link aggregate physical interfaces will be connected. See your link partner's documentation to determine which modes it supports. Check the mode you want to use. The following choices are available: FEC_AUTO Automatically start the FEC protocol on the physical port.
Priority The port priority in a failover group (LAN_MONITOR mode) using priority-based failover or cost-based failover. If you assign a priority value to one link, you must assign a priority value to all links in the failover group. Leave this blank if you do not want to assign a priority value; HP APA will assign it. Cost The port cost in a failover group (LAN_MONITOR mode) using cost-based failover.
Recommended configuration: Servers that need highly available network interfaces. CAUTION: For HP Serviceguard configurations, the member links of an APA Hot Standby link aggregate must be bridged to function properly. If they are not, the node or cluster might go down when a member link fails. Group Capability An integer value that determines which network physical ports can be aggregated into a common FEC_AUTO mode link aggregate.
Specify a whole number of seconds because the value you specify is rounded up to the next whole number of seconds. The value must be a valid integer. The rapid ARP interval multiplied by the rapid ARP count must be less than or equal to 60 seconds. Rapid ARP Count The number of gratuitous ARP packets sent rapidly. The valid range is 5–60, inclusive. The default value is 10. The value must be a valid integer. The rapid ARP interval multiplied by the rapid ARP count must be less than or equal to 60 seconds.
6. 7. Select the Configuration Options you want. Click OK to create a link aggregate and set the attributes. An Operation Successful page appears (Figure 3-11), confirming the creation of the link aggregate and the updating of the APA configuration. Figure 3-11 Operation Successful Page 8. Click OK. The APA main page displays, showing the newly created link aggregate. By default, SMH changes the current APA configuration and saves the information and APA configuration to the /etc/rc.config.
1. Click Create Failover Group on the APA main page to open the Create Failover Group page as shown in Figure 3-12. Figure 3-12 Create Failover Group Page 2. 3. 4. Select a Link Aggregate Instance, or accept the default. If you select a different value, the display refreshes. Select a Failover Policy, if you do not want to accept the default. If you select a different value, the display refreshes. Select the ports to include in the failover group.
hp_apaportconf, and /etc/lanmon/lanconfig.ascii files, which maintains the Create Failover Group changes across reboots. After you configure the failover group, configure an IP address to enable it. See “Configuring an IP address” (page 37) for more information. Configuring an IP address After you configure HP APA, you must configure an IP address on the interface. To configure an IP address for a link aggregate or failover group, do the following: 1. 2. 3. 4. 5. 6.
3. On the server side, create the link aggregates or failover groups. Then, create the VLANs over the link aggregate or failover group. After a link aggregate is in use, you can create, modify, and delete VLANs over it. Users can leave behind VLANs after the link aggregate or failover group has been cleared. If the link aggregate or failover group has been cleared, users can neither create VLANs nor modify any VLANs on the link aggregate or failover group. Users can only delete the VLANs.
4 Administering HP APA Administering HP APA consists of performing the following tasks: • “Modifying a link aggregate” (page 39) • “Modifying a failover group” (page 39) • “Deleting a link aggregate” (page 39) • “Deleting a failover group” (page 41) • “Displaying the current APA configuration” (page 41) • “Displaying the link aggregate details” (page 41) • “Modifying HP APA global parameters” (page 42) • “Performing Critical Resource Analysis” (page 43) • “Stopping FEC or LACP protocol” (page 43) • “Logging
Figure 4-1 Delete Link Aggregate Page 3. 4. 40 Select the Configuration Options you want, if different from the default. Click OK to delete the link aggregate. An Operation Successful page appears (Figure 4-2), confirming the deletion of the link aggregate and the updating of the APA configuration.
Figure 4-2 Operation Successful Page (Delete) 5. Click OK. The APA main page displays. Deleting a failover group To delete a failover group, do the following: 1. 2. 3. 4. 5. Select a failover group (a link aggregate with LAN_MONITOR mode) in the SMH APA main page to display the detailed view of the failover group. Click Delete Failover Group in the details pane to display the Delete Failover Group page. Select the Configuration Options you want, if different from the default.
2. 3. Click View Detail in the details pane to display the View Detail page containing the details of the selected link aggregate. Click OK to display the APA main page when you are finished viewing the information. Modifying HP APA global parameters The following HP APA global parameters are stored in the /etc/rc.config.d/hp_apaconf file: Default Port Mode The default configuration mode for link aggregates. The supported values are as follows: MANUAL Disables FEC and IEEE 802.
By default, SMH changes the current APA configuration and saves the information and APA configuration to the /etc/rc.config.d/hp_apaconf, /etc/rc.config.d/ hp_apaportconf, and /etc/lanmon/lanconfig.ascii files, which maintains the modified APA global parameter changes across reboots. Performing Critical Resource Analysis Critical Resource Analysis (CRA) provides usage analysis for interfaces (ports and link aggregates). This includes applications that are using the interfaces.
• Anytime its Operation Status (ifOper) status is DOWN (the link cannot carry network traffic) or If the link is part of an automatic link aggregate and the protocol has failed to maintain the membership of the port in the link aggregate. • Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar . . .
statistic, the sum of all the physical ports corresponding statistic counter is added to the link aggregates counter. For a detailed description of the statistics fields, see RFC 1213 for 32-bit statistics and RFC 2863 for 64-bit statistics. When a physical port is removed from a link aggregate, each of its statistics is added to the corresponding link aggregate statistic. When a physical port is added to a link aggregate, its current statistics are subtracted from the link aggregates statistics.
Example 4-2 Sample Link Aggregate 64-Bit Statistics # nwmgr --st -I 901 -S apa *** lan901 64 bit MIB statistics: Interface Name = lan901 PPA Number = 901 Description = lan901 Hewlett-Packard LinkAggregate Interface MTU Size = 1500 Speed = 3 Gbps Station Address = 0x0012794371B4 Administration Status = UP Operation Status = UP Last Change = Thu Sep 28 18:35:11 2006 Inbound Octets = 1353732 Inbound Unicast Packets = 0 Inbound Multicast Packets = 0 Inbound Broadcast Packets = 0 Inbound Discards = 0 Inbound Err
--interval seconds By default, the counters are displayed every second. Minimum value is 1 second. --iteration count By default, the monitor action continues until you enter Ctrl+c. Minimum value is 1. For more information, see nwmgr_apa(1M). Example 4-3 shows the 32-bit statistics monitoring.
5 Using the nwmgr command In addition to the preferred SMH GUI and TUI interfaces for configuring and managing HP APA, HP-UX provides the nwmgr command for managing all LAN-based and IB-based network interfaces. The nwmgr command serves as a single tool for performing all network interface-related tasks, offering the functionality provided by the lanscan, lanadmin, linkloop, lanapplyconf, and landeleteconf commands. NOTE: The lanadmin, lanscan, and linkloop commands are deprecated.
• • • • • • • • • • Save the current HP APA configuration to the HP APA configuration files (--saved option) View all APA interfaces (--get option, the default) View all or specified attributes of an APA or physical LAN interface View all or specified APA global attributes (-q global or -A attribute option, respectively) View APA interface statistics (--st option) Reset an APA interface (--reset option) Reset the MIB statistics for an APA interface (--reset option) Rebalance the load among member links in
configuration infromation for link aggregate lan900 produces the following human-readable output: # nwmgr -c lan900 Class Parent APA Mode Instance PPA State ======== ====== ============ =========== lan900 Up LAN_MONITOR Load Membership Balancing ========= ========== LB_HS 4,5 In contrast, the following command produces output that scripts can parse: # nwmgr -c lan900 --sc apa!lan900#parent_ppa#current# apa!lan900#apa_state#current#Up apa!lan900#mode#current#LAN_MONITOR apa!lan900#lb#current#LB_HS apa!lan9
Standby 4 5 The specified instance is up, and is ready to take traffic in its parent link aggregate. A list of the members of the specified link aggregate. In the previous example, the values 9, 21, 22, 14, 13* mean lan9, lan21, lan22, lan14, and lan13 are the member of the lan900. The asterisk character (*) after 13 indicates lan13 is in a down state. The active LAN instance in the specified link aggregate.
Example 5-3 nwmgr Verbose Output (Link Aggregate) # nwmgr -S apa -I 920 -v lan920 current values: Speed = 2 Gbps Full Duplex MTU = 1500 Virtual Maximum Transmission Unit = 0 MAC Address = 0x00127942FC38 Network Management ID = 46 Features = Linkagg Interface IPV4 Recv CKO IPV4 Send CKO VLAN Support VLAN Tag Offload 64Bit MIB Support Load Distribution Algorithm = LB_MAC Mode = LACP_AUTO Parent PPA = APA State = Up Membership = 13,14 Active Port(s) = 13,14 Not Ready Port(s) = Key = 920 Operational Key = 920
Deleting an HP APA interface Before attempting to delete an HP APA interface, ensure that there are no applications or upper layer protocols active on the interface. Perform a critical resource analysis operation on the interface, by entering: # nwmgr --cra -c lanaggregate_instance The --cra operation displays all applications and commands currently assigned to the interface.
Table 5-1 Legacy Commands and Corresponding nwmgr Commands (continued) Task Legacy Command nwmgr Command Remove all ports from a link aggregate lanadmin -X -c 900 nwmgr -d -A links=all -I 900 -S apa Remove all ports from a failover group landeleteconf -g lan900 nwmgr -d -A links=all -c lan900 Remove specific ports from a lanadmin -X -d 1 2 900 link aggregate nwmgr -d -A links=1,2 -I 900 -S apa Update the load balancing lanadmin -X -l LB_MAC 900 algorithm and group lanadmin -X -g 900 900 900 capab
Table 5-1 Legacy Commands and Corresponding nwmgr Commands (continued) 56 Task Legacy Command nwmgr Command Get statistics lanadmin -g 900 nwmgr --st -c lan900 Monitoring statistics apa-monitor -p 5 nwmgr --st monitor -S apa -I 900 Reset an APA interface lanadmin -r 900 nwmgr -r -c lan900 Reset statistics lanadmin -c 900 nwmgr -r --st -c lan900 View basic help lanadmin -x -h 900 nwmgr -h -S apa View verbose help lanadmin -X -H 900 nwmgr -h -v -S apa Clear data flows on a link aggregat
6 Troubleshooting HP APA This chapter contains a diagnostic map to help you solve problems that might occur when you use HP APA. Use this chapter with the appropriate HP and third-party switch documentation to solve as many problems as possible at your level. Operation Network problems can occur for a number of reasons. The diagnostic map in this chapter help you to isolate the problem.
System booted without errors? NO YES Edit the /etc/rc.log file and search for the string hpapa. This takes you to the beginning of the HP APA start-up section. If there are failover groups configured on your system, the following message indicates an error: ERROR: lanapplyconf failed If there are no failover groups configured on your system, you can ignore the error message. If you see error messages about link aggregate attributes, do the following: 1.
NIC operating? Verify that the NIC is operating. Enter the following command: NO # nwmgr -v [-c lanx | -S driver-name] If the Interface State is DOWN, do the following: YES 1. Check the status of the Link LED. If the LED is off, check the connection to the switch. Make sure that the switch is configured in the correct mode and is autonegotiating, if necessary. Reset the NIC with following command: # nwmgr -r -c lanxx 2. 3.
link aggregate problems (FEC mode)” (page 70) for FEC_AUTO mode link aggregates, or “Solving failover group problems” (page 74) for failover groups. Solving link aggregate problems (MANUAL mode) Link partner configured? NO YES Link aggregate configured? NO Verify that the link aggregate is configured by issuing the nwmgr command. Link aggregate names begin with lan900 and have the subsystem and interface type as hp_apa.
The Membership column shows the ports that compose the link aggregate. If the link aggregate contains ports that you do not want in the aggregate, delete the ports by issuing the following command: # nwmgr -d -A links=port-number -c lanxxx where port-number can be a list of multiple ports numbers separated by commas and xxx is the instance number of the link aggregate (for example, 900).
NOTE: If you are using the HOT_STANDBY load distribution algorithm, be sure the switch side port is not in any trunk. Network reachable? NO If a remote host's network is not reachable, the following message is displayed in response to the ping command: network is unreachable Complete the following steps: YES 1. 2. Host known? Ensure that the network devices are configured properly on the local host, using the netstat -i command.
# arp hostname If the entry is wrong or incomplete, enter the correct station address by using the arp command. See arp(1M) for more information. 5. 6. 7. 8. File access successful? NO Make sure the network devices are configured properly on the local host, using the netstat -i command. Verify that the routing tables on the local host are correct, using the netstat -r command. Use the ping command to determine whether the IP router is reachable.
If the connection terminates abnormally or a network application appears to hang, complete the following steps: 1. Test the network to determine whether the problem is on the local host, remote host, or a host on the path between the two. YES 2. After you identify the host with the problem, do the following: a. Confirm that the NIC is properly configured. Verify that the broadcast address and address mask for the local host are correct. b.
2. Correct ports in link aggregate? NO Verify that the link partner is configured to run the IEEE 802.3ad LACP protocol. Make any changes, if necessary, and wait 10 seconds. Then, verify that the link aggregate is enabled. Verify the ports in the link aggregate by issuing the following command: # nwmgr -A all -c lanxxx -S apa YES where xxx is the link aggregate instance number (for example, 900). The Membership column shows the ports that compose the link aggregate.
1. Verify the type, speed, duplex mode, MTU size, and checksum offload capability of each port you want with the following command: # nwmgr -v -c lanxxx where xxx is the instance number for the port. All ports must have the same values. NOTE: You cannot aggregate a port that has no CKO capability with a port that has CKO capability, even if its CKO capability is currently disabled. To change an attribute, do the following: a.
3. Verify that the port's key value matches the value of the link aggregate. If it does not, do the following: a. Set the port mode to MANUAL with the following command: # nwmgr -s -A mode=MANUAL -I port-number -S apa b. Set the key value and mode with the following command: # nwmgr -s -A key=value, mode=LACP_AUTO -I port-number -S apa c. d. The ports then negotiate with the link partner (approximately 10 seconds) and join the link aggregate. Repeat this step for each port you want in the aggregate.
4. 5. Host reachable? file to see if nis is specified as a service for the hosts database entry. If it is not, add it. Also, verify that the DNS service has information about the remote host. If your site uses the NIS name service for name-to-address translation, look in the /etc/nsswitch.conf file to see if nis is specified as a service for the hosts database entry. If it is not, add it. Also, verify if the NIS service has information about the remote host. If your /etc/nsswitch.
3. 4. 5. telnet command successful? NO YES Verify that the /etc/hosts.equiv file is set up correctly. Verify that the directory and file protection on the files to be copied or the .rhosts file on the remote system are correct. If you are using NFS, see the appropriate NFS documentation. If the telnet command is not successful, there is a problem with the Transport Layer (OSI Layer 4). Do the following: 1.
If you suspect or know that you are experiencing packet loss over your connection, do the following: 1. LACP mode link aggregates require time to negotiate with the switch. You might drop some packets during this time. YES 2. Check the trunk is configured correctly on the switch. Make sure that ports connected to the ports on the server are assigned to the LACP trunk. If the LACP trunk is assigned to any other ports on the server, this can cause packets to be dropped. 3. Check the switch port configuration.
2. Correct ports in link aggregate? NO Verify that the link partner is configured to run the Cisco Fast EtherChannel protocol. Make any changes, if necessary, and wait 10 seconds. Then, verify that the link aggregate is enabled. Verify the ports in the link aggregate by issuing the following command: # nwmgr -A all -c lanxxx -S apa where xxx is the link aggregate instance number. YES The Membership column shows the ports that compose the link aggregate.
If there are ports in the Not Ready Port(s) field, verify the port's operation status by issuing the following command: # nwmgr --st -c lanxxx where xxx is the port number (for example, 1, 4, or 10). If the Operation Status is UP, see the steps for Link partner configured?. If the status is down, check the cabling from the NIC to the switch and see the steps for NIC operating?.
3. 4. Verify the remote host is running, using the ping command. If the remote host does not respond, ask the host's system administrator to start the host. For additional information about the ping command, see “Testing access to Internet network hosts” (page 80). Verify that an entry exists for the remote host in your system's ARP cache by entering the following command: # arp hostname If the entry is wrong or incomplete, enter the correct station address by using the arp command.
3. Transfer a file to a different remote host on the network. Use the netstat command to check for lost packets. If network congestion does not appear to be the cause, contact your HP representative. See “Reporting problems ” (page 83) for more information. If the connection terminates abnormally or a network application appears to hang, complete the following steps: NO 1. Test the network to determine whether the problem is on the local host, remote host, or a host on the path between the two. YES 2.
If the APA State is Down, do the following: 1. 2. Correct ports in failover group? NO Check that all ports in the failover group are connected to the switch. Verify that the link partner is configured for no trunking. Make any changes, if necessary, and wait 10 seconds. Then, verify that the failover group is up. Verify the ports in the failover group by issuing the following command: # nwmgr -S apa YES where xxx is the failover group instance number.
• • • 5. The port is already in a link aggregate or failover group. Find another port to use. The port is not an Ethernet port or is not supported by HP APA. The port is down. Wait for the port to come up. Verify that the port is connected to the switch correctly. If it is, reset the port.
Active port correct? NO Verify that the active port is the one you intended by using the following command: # nwmgr -v -c lanxxx where xxx is the failover group instance number. YES If the active port is not the intended one and the intended port is listed in the Connected Port(s) field, the intended port is connected to the current active port.
where port is the port instance number and xxx is the failover group instance number. The message indicates the reason the port is not connected with the others. For cost-based failover groups, enter the following command: # nwmgr -s -A links=port -A port_pri=value -A port_cost=value -c lanxxx The message indicates the reason the port is not connected with the others. 3.
2. 3. 4. Perform a loopback test on your own system by using the ping command. If this is successful, your system is operating properly to the Network Layer (OSI Layer 3). Verify the remote host is running, using the ping command. If the remote host does not respond, ask the host's system administrator to start the host. For additional information about the ping command, see “Testing access to Internet network hosts” (page 80).
3. Transfer a file to a different remote host on the network. Use the netstat command to check for lost packets. If network congestion does not appear to be the cause, contact your HP representative. See “Reporting problems ” (page 83) for more information. If the connection terminates abnormally or a network application appears to hang, complete the following steps: NO 1. Test the network to determine whether the problem is on the local host, remote host, or a host on the path between the two. YES 2.
When using the ping command for fault isolation, first test the local host to verify it is running. If the local host returns the data correctly, use the ping command to test remote hosts farther and farther away from the local host. If you do not specify command options, the ping command displays the results of each ICMP request in sequence, the number of bytes received from the remote host, and the round-trip time. If the output indicates lost packets, note the percentage.
packets going through the network and loopback and header information. Log messages are written to the /var/adm/nettl.LOG000 file. To format and view the contents of this file, enter the following command: # netfmt -v -f /var/adm/nettl.LOG000 | more Look for Error and Disaster messages. APA also supports the logging of status messages to the syslog.log file. See “Logging messages to the syslog.log file” (page 43) for information on enabling this feature.
• To stop LACP tracing, enter: nettl -traceoff all -entity HP_APALACP • To format the tracefile into the file /tmp/traceout, enter: nettl -f /tmp/tracefile.TRC0 > /tmp/traceout Whenever the TSO status of a link aggregate or failover group changes, a warning message is logged in the nettl.LOG000 file.
Gathering information To gather information to report a problem, complete the following steps: 1. Write a complete description of the problem. Describe the events leading up to and including the problem. Attempt to describe the source and symptoms of the problem. Include in your description: HP-UX commands used; communication subsystem commands used; job streams; result codes and messages; and data that can reproduce the problem.
7 Support and other resources Contacting HP Documentation feedback HP welcomes your feedback. To make comments and suggestions about product documentation, send a message to: http://www.hp.com/bizsupport/feedback/ww/webfeedback.html Include the document title and manufacturing part number. All submissions become the property of HP. New and changed information in this edition The January 2008 release of HP APA for HP-UX 11i v3 provides IPoIB support and support for fixed MAC address assignment.
%, $, or # A percent sign represents the C shell system prompt. A dollar sign represents the system prompt for the Bourne, Korn, and POSIX shells. A number sign represents the superuser prompt. audit(5) A manpage. The manpage name is audit, and it is located in Section 5. Command A command name or qualified command phrase. Computer output Text displayed by the computer. Ctrl+x A key sequence.
A Product specifications Table A-1 summarizes the capabilities of HP APA and LAN Monitor.
1 MANUAL mode: Can be Load Balancing or Non-Load Balancing • Load Balancing: MAC, IP, or LB_PORT algorithm • Non–Load Balancing: Hot Standby mode 2 3 4 5 6 88 Load Balancing is configurable for outbound traffic on Ethernet links only. Links must be of the same speed and type (100Base-T or 1000Base-T). LACP requires full duplex (FD) operation of the links. You cannot reset the speed, duplex mode, or MTU size over a link aggregate. This is for the total of link aggregates and failover groups combined.
B HP APA configuration files HP APA uses the following configuration files: • /etc/rc.config.d/hp_apaconf • /etc/rc.config.d/hp_apaportconf • /etc/lanmon/lanconfig.ascii The following sections describe each file and its parameters. For configuration examples that describe editing these files, see Appendix C (page 97). hp_apaconf file The /etc/rc.config.d/hp_apaconf file contains the following general HP APA configuration parameters for link aggregates.
HP_APA_GROUP_CAPABILITY HP_APA_HOT_STANDBY HP_APA_INTERFACE_NAME HP_APA_KEY HP_APA_LACP_SYSTEM_ID_MODE HP_APA_LACP_TIMEOUT HP_APA_LOAD_BALANCE_MODE For FEC_AUTO only. An integer value used to determine which network physical ports can be aggregated into a common PAgP link aggregate. Set the group capability to be the same for all network physical ports in the same link aggregate. Ports going to different link aggregates must have different group capabilities.
HP_APA_MAX_LINKAGGS HP_APA_STATION_ADDRESS HP_APA_USE_SYSLOG Sets the maximum number of link aggregates for the server. The valid range is from 5 to 50. The default value is 50. This value takes effect only after a reboot. Specifies the MAC address to assign to the specified link aggregate.
capability must match the group capability of the desired aggregate that the user wants the port to join. This value is set in the /etc/rc.config.d/hp_apaconf file. “Link aggregate advanced parameters” (page 32) describes how to choose the group capability for link aggregates. Ports going to different link aggregates should have different group capabilities. The default group capability is 5. HP_APAPORT_INTERFACE_NAME HP_APAPORT_KEY Name of physical interface. For example, lan0 and lan1.
Example B-1 Sample lanconfig.ascii Configuration File #*********************************************** #******* LAN MONITOR CONFIGURATION FILE #*** For complete details about the parameters and how #*** to set them, consult the lanqueryconf(1m) manpage #*** or your manual. #*** All timeout values are microseconds; they are rounded up #*** or down appropriately.
FAILOVER_GROUP STATIONARY_IP STATIONARY_MAC STANDBY PRIMARY lan900 193.33.33.33 0x021122334455 lan9 7 lan6 5 FAILOVER_GROUP STATIONARY_IP PRIMARY STANDBY STANDBY STANDBY lan901 195.55.55.55 lan4 5 lan1 3 lan2 3 lan3 3 FAILOVER_GROUP STATIONARY_IP PRIMARY STANDBY lan902 fe80::2 lan11 5 lan10 3 The lanconfig.ascii configuration file contains the following fields: The name for a single LAN Monitor failover group. You must FAILOVER_GROUP specify a name for each LAN Monitor failover group on the system.
port (if present) are moved to the failover group. In previous versions of HP APA, the port must have IP address configured and must be the highest priority or lowest cost. In the HP-UX 11i v3 version of HP APA, you can assign the port any priority or cost values. You also need not configure an IP address on it. HP recommends that you always assign the port the highest priority or the lowest cost in a failover group. Specifies a standby port for the FAILOVER_GROUP.
parameter, see “Failover group advanced parameters” (page 33). If you specify the POLLING_INTERVAL, DEAD_COUNT, LM_RAPID_ARP_INTERVAL, and LM_RAPID_ARP_COUNT fields once in the lanconfig.ascii file before all failover, the values apply to all failover groups. If you specify them multiple times in the file, the values apply only to those failover groups that follow these fields; you can set different values for failover groups.
C Configuring HP APA by editing files This appendix provides information about editing the different configuration files to configure the following: • Link aggregates • Failover groups Editing configuration files for link aggregates This section shows how to configure the three modes associated with port aggregation. For release-specific information, see the release notes on your system in the /opt/networkdocs directory or on the web at: http://www.docs.hp.
3. Activate the new configuration by entering: # # # # /sbin/init.d/hplm stop 1 /sbin/init.d/hpapa stop 2 /sbin/init.d/hpapa start /sbin/init.d/hplm start 3 1 2 3 If failover groups are also configured. This can interrupt traffic on existing link aggregates. For configuring failover groups. FEC_AUTO port configuration mode To configure load balancing and group capability for link aggregate 901 with MAC-based load balancing, do the following: 1. Edit the /etc/rc.config.
HP_APAPORT_KEY[1]=902 HP_APAPORT_CONFIG_MODE[1]=LACP_AUTO NOTE: Set the LACP_AUTO key in the previous examples to be the same as that of the link aggregate to which it belongs. Ports going to different link aggregates should have different keys. 3. Activate the new configuration by entering: # # # # /sbin/init.d/hplm stop 1 /sbin/init.d/hpapa stop 2 /sbin/init.d/hpapa start /sbin/init.d/hplm start 3 1 2 3 If failover groups are also configured. This can interrupt traffic on existing link aggregates.
POLLING_INTERVAL DEAD_COUNT LM_RAPID_ARP LM_RAPID_ARP_INTERVAL LM_RAPID_ARP_COUNT FAILOVER_GROUP STATIONARY_IP STANDBY PRIMARY 10000000 3 off 1000000 10 lan900 192.1.1.1 lan11 3 lan10 5 7. Compare the /etc/lanmon/lanconfig.ascii file with the /tmp/filename file. If they are not the same, decide which file you want to use. If you want to use the file in /tmp, copy that file to /etc/lanmon/lanconfig.ascii. 8. Edit the /etc/lanmon/lanconfig.
1. Connect lan1 and lan2 to the same switch or to different switches or hubs on the same subnet. NOTE: 2. Be sure that trunking is not enabled on the switch ports.
8. Use the lanapplyconf command to build the failover group, and check it with the nwmgr -S apa and netstat -in commands: # lanapplyconf Reading ASCII file /etc/lanmon/lanconfig.
2. Assign a priority to the aggregates. If you prefer lan900 to be the active port when both link aggregates have the same number of operational ports, assign lan900 a higher priority. If you do not, the active port changes when one of the link aggregates becomes more efficient (has more operational ports or higher bandwidth). In this example, you assign lan900 a priority of 5 and lan901 a priority of 3. 3. Edit the lanconfig.ascii configuration file. FAILOVER_GROUP lan902 STATIONARY_IP 192.19.20.
3. Edit the configuration file. FAILOVER_GROUP lan902 STATIONARY_IP 10.0.1.60 # Primary/Standby Interface Priority : Cost # –––––––––––––––––––––––+–––––––––––––+–––––––––––––––– PRIMARY lan900 8 : 2 STANDBY lan901 7 : 3 Table C-2 lists lan900 events for this example and how proactive failover determines the active port for lan902. Table C-2 lan900 Events and Proactive Failover (Unequal Network Costs) Event lan901 Normalized Port Cost Active Port Failover group constructed1 2/200, or .
D VLANs over APA using HP Procurve switches This appendix describes how to create VLANs over trunks and ports on an HP Procurve switch. For more information about creating VLANs on HP Procurve switches, see Procurve, Series 6400cl Switches, Series 5300xl Switches, Series 3400cl Switches, Advanced Traffic Management Guide at: ftp://ftp.hp.com/pub/networking/software/6400-5300-3400-AdvTrafficMgmt-Oct2005-59906051.
4. Verify that the VLAN with ID 333 is tagged to Trk3. Enter: HP ProCurve Switch 5308xl# show vlan 333 Status and Counters - VLAN Information - Ports - VLAN 333 802.1Q VLAN ID : 333 Name : vlan_333 Status : Port-based Voice : No Port Information Mode Unknown VLAN Status ---------------- -------- ------------ ---------Trk3 Tagged Learn Up VLAN 333 has been successfully tagged to Trk3. On the server side, complete the following steps: 1. Create link aggregate lan900 using ports lan3 and lan4.
Figure D-2 VLAN over a Failover Group Run lan901 Atn. Fault lan5 Remote Power lan6 Failover Group lan902: lan901 (Primary), lan5, 6 (Standby) Switch1 Switch2 Trk4 Vlan 333 Vlan 334 F17 F18 Vlan 333 Vlan 334 As the first step in creating the failover group, associate the ports in link aggregate lan901 to Trk4 on Switch 1; set the trunk type to trunk (MANUAL trunking). Connect ports lan5 and lan6 to ports F17 and F18, respectively, on Switch 2.
3. Verify that the VLANs with IDs 333 and 334 are tagged correctly. Enter: Switch 1# show vlan 333 Status and Counters - VLAN Information - Ports - VLAN 333 802.1Q VLAN ID : 333 Name : vlan_333 Status : Port-based Voice : No Port Information Mode ---------------- -------Trk4 Tagged Switch 1# show vlan 334 Status and Counters - VLAN Unknown VLAN Status ------------ ---------Learn Up Information - Ports - VLAN 334 802.
On the server side, complete the following steps: 1. Create link aggregate lan901. 2. Create failover group lan902. 3. Create VLANs 333 and 334 over lan902. Enter: # nwmgr -a -S vlan -A vlanid=333, ppa=902 VLAN interface lan5000 successfully configured.
E Switch configuration information This appendix provides some information on how to configure the following switches for use with HP APA: • Alteon • Cisco • Extreme • Procurve See your switch documentation for complete information. Alteon switches Alteon switches have 8 ports. Fiber blades support 1500 and 9000 MTU (jumbo frames).
Creating an LACP link aggregation Switch1#enable Password:password Switch1#config term config command 1 . . Ctrl+z Switch1(config-if)#interface range gigabitethernet 3/1 - 3 Switch1(config-if-range)#no ip address Switch1(config-if-range)#channel-protocol lacp Switch1(config-if-range)#channel-group 1 mode active Creating a port-channel interface Port-channel 1 Switch1(config-if-range)#end Switch1#config term config command . .
Port-channel: Po1 (Primary Aggregator) -----------Age of the Port-channel = 00d:01h:46m:08s Logical slot/port = 14/1 Number of ports = 3 Port state = Port-channel Ag-Inuse Protocol = LACP Ports in the Port-channel: Index Load Port EC state No of bits ------+------+------+------------------+----------2 49 Gi3/1 Active 3 0 92 Gi3/2 Active 3 1 24 Gi3/3 Active 2 Time since last port bundled: 00d:01h:44m:15s Gi3/1 Displaying a Port in a link aggregation Switch1#show interface GigabitEthernet 3/3 etherchanne
Port Gi3/2 System ID Port Number 00000,0000.0000.00000x0 LACP Partner Port Priority 0 Partner Oper Key 0x0 Age 297s Flags SP Partner Port State 0x0 Partner's information: Partner Partner System ID Port Number 00000,0030.6ef4.
Configuring LACP link aggregates ExtremeWare v. 7.5 enables you to configure LACP. Use contiguous ports and let the first port be the master.
HP ProCurve Switch 4108GL# Make sure that the ports you think you are aggregating on the switch are the right ones. This can be combined with the process of creating switch aggregations. Do the following: 1. Log in to the switch. 2. Enter menu at the prompt. 3. Select Switch Configuration. 4. Select Port/Trunk Settings. 5. Verify which cards or ports are connected to the switch ports that you intend to configure. Do the following: a.
4. The following prompt displays: HP9304(config)# TIP: Enter [command name] ? to display the syntax for a command. Configuring HP APA To configure HP APA to use the Procurve 9304/8 switch, do the following: 1. Verify port connections before configuring APA. Do the following: a. Log in to the switch using the previous steps. b. Enter the interface e [slot]/[port] command. For example: HP9304(config)#interface e 2/1 c. Enter the following: HP9304(config-if-e100-2/1)#disable d.
2. Configure HP APA. Do the following: a. Unplug all LAN connections to be aggregated. b. Create the aggregations. They must be either 2 or 4 port trunks. For two-port aggregations, the beginning port number must be odd. For four-port aggregations, the beginning port number must be a number divisible by 4 plus 1 (for example, 1, 5, 9, 13). Use the following command to create a four-port aggregation: HP9304(config)#trunk server e 1/5 to 1/8 c. Save the configuration. Enter: HP9304(config)#write mem d.
Glossary A administrative key An integer value that determines which network physical ports can be aggregated into a common LACP link aggregate. The key must match the key of the desired aggregate that the user wants the port to join aggregate A group. For APA, a group of up to eight ports (32 ports for LACP aggregates) makes one link aggregate. There can be 50 link aggregates per computer. aggregation See aggregate. APA Auto Port Aggregation ARP Address Resolution Protocol.
M MAC Media Access Control MANUAL The default mode of port configuration. Can be performed by using SMH (preferred) or by editing two configuration files. MIB Management Information Base N network adapter A network device which has one or more network physical ports. network physical port The communications channel formed when you attach a network cable between a specific network port (adapter card) and a LAN device. P PAgP See Port Aggregation Protocol.
Index A aggregate instance number, 30 algorithm HOT_STANDBY, 32 LB_IP, 32 LB_MAC, 32 LB_PORT, 32 load balancing, 10 APA Main page, 15 application hang (failover group), 80 hang (FEC_AUTO mode), 74 hang (LACP_AUTO mode), 69 hang (MANUAL mode), 64 C caution HP Serviceguard and HOT_STANDBY mode, 33 lanqueryconf command, 99 CKO and link aggregates, 66 command ioscan, 81 lanqueryconf, 99 nettl, 81 nwmgr, 15, 38, 49 ping, 80 swlist, 18 swremove, 19 configuration, 29 (see also sample configuration) displaying cur
features, 9 FEC protocol stopping, 43 FEC_AUTO mode, 31 configuration example, 98 configuring by editing files, 97–99 verifying link partner configuration, 70 file editing for APA configuration, 97–99 hp_apaconf, 38, 89–91 hp_apaportconf, 38, 91–92 netconf, 99 file access verifying (failover group), 79 verifying (FEC_AUTO mode), 73 verifying (LACP_AUTO mode), 68 verifying (MANUAL mode), 63 FIRST_APA_PORT, 42 FIRST_LACP_PORT, 42 G global parameter changing, 42 default port mode, 42 LACP SYSID mode, 42 LACP
lanconfig.
resetting a data flow, 54 verifying HP APA configuration, 38 verifying TSO support, 12 O Operation Successful page, 35 delete link aggregate, 41 outbound data transfer, 32 P packet loss, 70 packet ordering, 11 parameter, 42 (see also global parameter) permission denied message in failover group, 79 in FEC_AUTO mode link aggregates, 73 in LACP_AUTO mode link aggregate, 68 in MANUAL mode link aggregate, 63 ping command, 80 polling interval specifying in lanconfig.
System Management Homepage (see SMH) T TCP Segmentation Offload (see TSO) telnet command verifying success (failover group), 79 verifying success (FEC_AUTO mode), 73 verifying success (LACP_AUTO mode), 69 verifying success (MANUAL mode), 63 tip displaying Procurve command syntax, 117 specifying group capability, 33 troubleshooting, 57 TSO and failover group, 12 and link aggregate, 12 status messages, 83 verifying support, 12 U unknown host message in failover group, 78 in FEC_AUTO mode link aggregates, 72
