HP Auto Port Aggregation Administrator's Guide HP-UX 11i v1, 11i v2 HP Part Number: J4240-90046 Published: September 2008 Edition: September 2008, E0908
© Copyright 2008 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents About This Document.........................................................................................................9 1 Introduction...................................................................................................................13 Conceptual Overview...........................................................................................................................13 Link Aggregate.............................................................................
What Happens During Start Up?..........................................................................................................48 4 Using the lanadmin Command..................................................................................51 Set Options............................................................................................................................................52 Display Options...........................................................................................
E Switch Configuration Information.............................................................................119 Alteon Switches...................................................................................................................................119 Cisco 6509 Switches.............................................................................................................................119 Configuring a Single Port..........................................................................
List of Figures 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 3-13 3-14 3-15 3-16 3-17 3-18 3-19 3-20 3-21 3-22 3-23 3-24 3-25 3-26 3-27 3-28 D-1 D-2 6 Sample Enterprise Intranet Client/Server Configuration..............................................................24 Sample Router and Server Configuration (No Switch).................................................................25 Sample Router and Server Configuration (Switch).....................................................................
List of Tables 1-1 4-1 A-1 C-1 C-2 Interoperability with HP Serviceguard.........................................................................................17 Summary of lanadmin -x and -X Options.....................................................................................51 HP APA and LAN Monitor Capabilities.......................................................................................95 lan900 Events and Proactive Failover (Equal Network Costs).......................................
List of Examples 5-1 5-2 6-1 B-1 8 Sample Link Aggregate 32-Bit Statistics........................................................................................66 Sample Link Aggregate 64-Bit Statistics........................................................................................66 Sample ioscan –f Output...............................................................................................................90 Sample lanconfig.ascii Configuration File.....................................
About This Document This document (formerly titled HP Auto Port Aggregation Support Guide) describes how to install, configure, and troubleshoot HP Auto Port Aggregation (APA) on HP-UX Version 11.0, 11i v1, 11i v2 platforms. Document updates can be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, subscribe to the appropriate product support service. See your HP sales representative for details. This document is not a tutorial.
Typographic Conventions This document uses the following typographical conventions: %, $, or # audit(5) Command Computer output Ctrl+x ENVIRONMENT VARIABLE [ERROR NAME] Key Term User input Variable [] {} ... | WARNING CAUTION IMPORTANT NOTE A percent sign represents the C shell system prompt. A dollar sign represents the system prompt for the Bourne, Korn, and POSIX shells. A number sign represents the superuser prompt. A manpage. The manpage name is audit, and it is located in Section 5.
• • • HP Auto Port Aggregation (APA) Release Notes Performance and Scalability White Paper Using APA to Build a Screaming Fast Network Server Connection Publishing History The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The document part number will change when extensive changes are made.
1 Introduction HP Auto Port Aggregation (APA) is a software product that creates link aggregates, often called trunks, which provide a logical grouping of two or more physical ports into a single fat pipe. This port arrangement provides more data bandwidth than would otherwise be available and enables you to build large bandwidth logical links into the server that are highly available and completely transparent to the client and server applications.
NOTE: The December 2005 and later releases of APA for HP-UX 11i v2 (B.11.23.10) enable you to combine 2 to 8 physical link ports into one link aggregate. • The physical ports in the link aggregation use the same MAC address. The unique MAC address for a specific link aggregate is determined by using the MAC address of one of the ports in the link aggregate. All ports will use the same MAC address. When a physical port is removed from a link aggregate, the port's MAC address is reset to its own MAC address.
1. 2. 3. Data Flow Lookup — The load distribution algorithm determines an index into a hash table that includes the physical port through which the outbound data flow is forwarded. Data Flow Physical Port Assignment — If the hash index for the data flow has not been assigned a physical port (the entry is empty), a physical port in the link aggregate is assigned to that specific hash index. The physical port is selected on a Round Robin basis.
During certain LAN Monitor events (for example, link failure and link recovery), the normalized port cost might change on the active or standby links. When these events occur, the normalized port cost of the active link and the standby links are compared. If a standby link has a lower normalized port cost than the active link, the standby link becomes the active link even if the current active link is UP.
• • The same link aggregate or failover group can offer different level of service for each user using ToS. You gain more flexibility in how you deploy link aggregates and failover groups. You can create, remove, and modify VLANs over link aggregates and failover groups without rebooting the system. This enables you to configure networking on a server without disrupting other users. For more information on managing and using VLANs, see HP-UX VLAN Administrator's Guide and your switch documentation.
lanadmin Command You can also use the lanadmin command from the HP-UX command line to make changes to HP APA. By default, those changes are not preserved across reboots. For more information about the lanadmin command and using it to administer APA, see lanadmin(1M) and Chapter 4 (page 51), respectively. Manually Editing Configuration Files Some sections of this manual describe the system files that are updated or modified when you perform an administrative task.
2 Installing the APA Software This chapter describes the information required in order to install APA on your system. Installation Requirements 1. 2. 3. 4. Log in to the HP-UX server as superuser. Confirm that the /usr/bin, /usr/sbin, and /sbin directories are in your PATH by using the echo $PATH command. Use the uname -a command to determine the HP-UX version of your system. Install the required patches for your system as described in the “Required Patches” section of the release notes.
Operating System Requirements HP-UX 11.0, 11i v1, or 11i v2. Software Requirements For the December 2005 release of HP APA for HP-UX 11i v2 (B.11.23.10) and later releases, if you want to use 8 ports for trunking HP APA requires the following software: • Transport Optional Upgrade (TOUR) 3.0 • Streams Advance Release (STAR) 1.0 • IPFilter version A.03.05.12, if you use IP Filter Installing the Software Skip this section if you ordered product option 0D1—preinstallation. 1.
1. Verify that the product was installed by issuing the following command: # swlist -l product | grep -i HP-APA Output similar to the following displays: HP-APA-FMT HP-APA-KRN HP-APA-LM HP-APA-NETMOD HP-APA-RUN B.11.23.40 B.11.31.20 B.11.31.20 B.11.31.20 B.11.31.20 HP HP HP HP HP Auto-Port Auto-Port Auto-Port Auto-Port Auto-Port Aggregation Aggregation Aggregation Aggregation Aggregation APA formatter product. kernel products. LM commands. nwmgr/NCweb libraries. APA command products.
3 Configuring APA This chapter describes how to configure HP APA on your system. This includes: • Reviewing sample HP APA configurations • Preparing for the configuration by gathering information • Configuring systems in sample configurations • Configuring a link aggregate • Configuring a failover group • Configuring the link partner • Performing post-configuration tasks HP APA Configuration Examples This section shows some sample HP APA configurations.
Figure 3-1 Sample Enterprise Intranet Client/Server Configuration Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 HP ProCurve 8000 Switch Hub 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Hub 0 1 2 3 4 5 6 7 Internet or Large Enterprise Environments Using Routers You can use HP APA link aggregation successfully in certain environments employing routers.
Figure 3-2 Sample Router and Server Configuration (No Switch) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation Router or Switching Router Intranet Internet Figure 3-3 (page 26) shows a sample router and server configuration with a switch. In this configuration, the switch might present problems because switches typically use a MAC address load-balancing algorithm.
Figure 3-3 Sample Router and Server Configuration (Switch) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 Switch Router or Switching Router Intranet Internet Server-to-Server (Back-to-Back) Figure 3-4 (page 27) shows a sample server-to-server configuration. You create server-to-server aggregations by directly connecting the physical ports in one server's link aggregation to the physical ports in the other server's link aggregation.
Figure 3-4 Sample Server-to-Server Configuration (Back-to-Back) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation Run Attn. Fault RemotePower Hot Standby for High Availability Figure 3-5 (page 28) shows a sample MANUAL (Hot Standby) mode configuration. These link aggregations provide high availability network access with an active link and a standby link. NOTE: HP strongly recommends using failover groups (LAN_MONITOR mode) rather than Hot Standby mode.
Figure 3-5 Sample Hot Standby Configuration for High Availability Run Attn. Fault RemotePower Primary Standby Switch or Hub HP APA 2-Port Hot Standby Link Aggregation Switch or Hub Server-to-Server with Switch (Not Recommended) Figure 3-6 (page 29) shows a sample server–to–server HP APA link aggregation configuration with a switch between the servers.
Figure 3-6 Sample Server-to-Server Configuration with Switch (Not Recommended) Run Attn. Fault RemotePower HP APA 2–4 Port Link Aggregation hp ProCurve Switch 8000 Switch HP APA 2–4 Port Link Aggregation Run Attn. Fault RemotePower Failover Group Figure 3-7 (page 30) shows a sample failover group (LAN_MONITOR mode) configuration.
Figure 3-7 Sample Failover Group (LAN_MONITOR) Configuration Run Active Attn. Fault RemotePower Run LAN Monitor 2-Port Failover Group Fault RemotePower LAN Monitor 3-Port Failover Group 2 Standby Links Standby Switch or Hub Active Attn.
Figure 3-8 Sample Failover Group Using Link Aggregates Configuration Run Attn. Fault Remote Power Run LAN Monitor failover group using HP APA link aggregations. Standby LinkAgg Active LinkAgg Attn. Fault Remote Power Active LinkAgg Standby hp ProCurve hp ProCurve hp ProCurve hp ProCurve Switch 8000 Switch 8000 Switch 8000 Switch 8000 LAN Monitor failover group using an HP APA link aggregation as the active link and one physical link as a standby link.
Figure 3-9 HP APA Configuration Worksheet HP APA Configuration Worksheet Aggregate Instance Number Mode FEC_AUTO LACP_AUTO LAN_MONITOR Not_Enabled Failover Priority Based Cost Based Instance Number: Hardware Path: Interface Type: MANUAL Priority: Cost: Advanced Parameters (Link Aggregates) Load Distribution Algorithm LB_MAC LB_IP LB_PORT HOT_STANDBY Group Capability (FEC_AUTO only) Key (LACP_AUTO only) Advanced Parameters (Failover Groups) Dead Count Polling Interval Rapid ARP on of
NOTE: For HP-UX 11.0 releases, use 100 as the starting instance number. Mode The configuration mode of the link aggregate or failover group. Your choice will be determined by the capabilities of the link partner (for example, switch, router, or server) to which the link aggregate physical interfaces will be connected. See your link partner's documentation to determine which modes it supports. Check the mode you want to use.
Priority The port priority in a failover group (LAN_MONITOR mode) using priority-based failover or cost-based failover. If you assign a priority value to one link, you must assign a priority value to all links in the failover group. Leave this blank if you do not want to assign a priority value; HP APA will assign it. For the December 2005 and later releases of HP APA for HP-UX 11i v1 (B.11.11.20) and PHNE_33116 (B.11.11.17) patch release.
Recommended configuration: Servers that need highly available network interfaces. CAUTION: For HP Serviceguard configurations, the member links of an APA Hot Standby link aggregate must be bridged to function properly. If they are not, the node or cluster might go down when a member link fails.
Rapid ARP Interval The number of microseconds between rapid gratuitous ARP messages. The range of valid values is 1000000–4000000, inclusive (1 second to 4 seconds). The default value is 1000000 (1 second). Specify a whole number of seconds because the value you specify is rounded up to the next whole number of seconds. The value must be a valid integer. The rapid ARP interval multiplied by the rapid ARP count must be less than or equal to 60 seconds.
NOTE: The starting PPA number for link aggregates varies with the operating system installed: for HP-UX 11.0, it is 100; and for all versions of 11i, it is 900. Figure 3-10 Displaying Link Aggregates 4. Click on the link aggregate to be configured. From the Actions pull-down menu choose the Configure Link Aggregate option. A window similar to Figure 3-11 will appear.
5. Click Advanced Options to display window similar to Figure 3-12. Figure 3-12 Link Aggregate Advanced Options 6. 7. 8. Verify the correct settings for the load-balancing (distribution) algorithm. If required, make the necessary changes. Change the group capability of this link aggregate as required. The group capability must be a nonzero number that differs from the group capability of any other link aggregate.
Figure 3-14 Modify Network Physical Port Attributes 12. In this window, make the necessary changes and click OK. 13. Repeat steps 8 through 12 to configure the remaining ports to be in the aggregate. Figure 3-15 shows a sample FEC_AUTO configuration using ports 0 and 1 with a group capability of 7. Figure 3-15 Example of Configured Link Aggregates 14. From the List pull-down menu, choose the Link Aggregates Supported by HP APA option. A window similar to Figure 3-16 will appear.
Figure 3-16 Status of Configured Link Aggregate is UP 15. Click once again on the link aggregate being configured. Then from the Actions pull-down menu, choose the Configure Link Aggregate option. A window like that in Figure 3-17 will appear.
16. Fill in the desired IP address and subnet mask to be used for the link aggregate. Then click OK. A window like that in Figure 3-18 will appear. Figure 3-18 Link Aggregate with Configured IP Address Perform one of the following: • If the link aggregate has a status of UP the configuration is complete. Exit SAM. Verify that the proper ports are configured in the proper link aggregation with the lanadmin -x -v 900 command.
Figure 3-19 Link Aggregates Supporting HP APA 4. Choose the Network Physical Ports that Support HP APA option from the List pull-down menu. The port configuration mode (shown in column 7 in Figure 3-20) determines the mode for the link aggregate. In Figure 3-20, the configuration mode for the ports that support HP APA is, by default, set to MANUAL mode. You might need to adjust the horizontal scroll bar to see all the window's fields. Figure 3-20 Network Physical Ports Supporting APA 5.
NOTE: Set the group capability to be the same as that of the link aggregate to which it belongs. Ports going to different link aggregates must have different group capability numbers. For the September 2007 release of HP APA for HP-UX 11i v1 (B.11.11.30), you no longer need to assign a group capability number to ports to be included in a MANUAL mode link aggregate. 6. 7. Highlight the port to configure by clicking on the port designated by its PPA.
Figure 3-23 Configuring Link Aggregates 12. Fill in the desired IP address and subnet mask to be used for the link aggregate. 13. Click the Advanced Options button to display a window similar to Figure 3-24. Figure 3-24 Link Aggregate Advanced Options 14. Verify the correct settings for the Group Capability (11i v1), assigned in step 4, and either Hot Standby or the load balancing (distribution) algorithm (not both). Make the necessary changes, if required. Then click OK. 15.
Figure 3-25 Adding Ports to or Deleting Ports from Link Aggregate 16. Highlight a port to be included in the link aggregate. Then use < to move it into the link aggregate. Figure 3-26 shows the result of moving lan0 and lan3 into the link aggregate, lan900. Figure 3-26 Adding Ports to or Deleting Ports from Link Aggregate 17. Click OK to see the Configure Link Aggregates window (Figure 3-27) with all the link aggregate information filled in.
Figure 3-27 Configured Link Aggregates Display 18. Clicking OK displays the original window (Figure 3-28). It now shows lan900 configured as a MANUAL mode link aggregate. Figure 3-28 Link Aggregate Displays with Status UP 19. The link aggregate is configured. Configure the next MANUAL mode link aggregate, or exit by choosing the Exit option from the File pull-down menu. SAM permanently saves the HP APA configuration values in the /etc/rc.config.d/ hp_apaconf and /etc/rc.config.
Configuring an IP Address After you configure HP APA, you must configure an IP address on the interface. To configure an IP address for a link aggregate, use the following command: # ifconfig lanxxx IP_address up Where xxx is the link aggregate PPA number. To make this change permament, edit the /etc/rc.config.d/netconf file and include the appropriate entry. For more information, see ifconfig(1M).
1. To verify which link aggregates have been configured, enter either of the following commands: # lanscan # lanscan -v 2. The Hardware Path column contains the link aggregates. Link aggregate names begin with LinkAgg. The Hdw State column shows the link aggregate state. UP indicates the link aggregate is operational. DOWN indicates the link aggregate was initialized but not configured.
4. HP APA software starts Cisco's FEC or IEEE 802.3ad LACP protocol is started on the ports that have been marked as such. No action is taken for the ports that have Cisco's FEC or IEEE 802.3ad LACP turned off. 5. LAN Monitor ASCII file is processed Process /etc/lanmon/lanconfig.ascii to create failover groups. If this file has not been created, no action is taken. After the system is up and running, you can distinguish a link aggregate from a normal (unaggregated) port by running the lanscan command.
4 Using the lanadmin Command You can use the lanadmin command to administer HP APA. The command enables you to do the following tasks from the command line: • Display HP APA settings (-x option (lowercase x) with options) • Change the HP APA configuration temporarily (-X option (uppercase X) with options) You can also use lanadmin interactively. CAUTION: The lanadmin command does not preserve changes across reboots. Use SAM or edit the configuration files to permanently save the changes.
Set Options To manage link aggregates and add ports to a link aggregate, use the following syntax and options with the -X (uppercase) option: lanadmin [-X -a portPPA [portPPA]] [-X -c linkAggregatePPA] [-X -d portPPA [portPPA]...
-k -X -k portPPA admin_key anyLinkAggregatePPA Sets the Administrative Key for an LACP_AUTO port or LACP_AUTO link aggregate. Applies to LACP_AUTO only. An Administrative Key and Operational Key are associated with each port. The Operational Key is used for forming aggregations in autonegotiation. The Administrative Key can be set at anytime. At the port level, the Operational Key only copies the Administrative Key value when the LACP protocol is not running.
-y -X -y on | off linkAggregatePPA Turns Hot Standby on or off for the link aggregate with the specified linkAggregatePPA number. For a description of Hot Standby, see the load distribution algorithms in “Link Aggregate Advanced Parameters” (page 34) and the the description of HP_APA_HOT_STANDBY in “hp_apaconf File” (page 97). Turn on sets link aggregate load-balancing mode to LB_HOT_STANDBY. Turn off sets link aggregate load-balancing mode to LB_MAC.
• The ports that are not ready to be used in a link aggregation or failover group. A port can be in this list due to: — The link state of the NIC is down. — The NIC is experiencing a hardware problem, which is causing unexpected failures. — The NIC has been configured to run an APA auto protocol and it has not reached a stable state. • For failover groups, a list of the ports in the "ready" list that have connectivity to the current active port.
. . -n -x -n anyLinkAggregatePPA Displays the status for all APA capable ports using any link aggregate in the command. For example: Number of elements=5 [0] ppa : 1 [0] reserved2 value : -1 [0] state : 1 -p -x -p portPPA anyLinkAggregatePPA Displays the present status of a port with the specified portPPA number.
-t -x -t portPPA anyLinkAggregatePPA Displays the current port priority for the port with the specified portPPA number. For LACP_AUTO, this shows the priority value for the port. For HOT_STANDBY, this shows which port is used for traffic and which for standby. This option is not applicable to PAgP (FEC_AUTO). -v -x -v linkAggregatePPA Displays the status for a given link aggregate with the specified linkAggregatePPA number.
• To manually configure ports with PPAs 6, 7, 8, and 9 into a link aggregate lan901, enter the following command: lanadmin -X -a 6 7 8 9 901 CAUTION: Be careful when using the -a suboption. This might lead to an invalid link aggregate. This command gives you full control over forming any link aggregate that you want.
To add ports lan1 and lan2 (which are not aggregated) to link aggregate lan901 (which is not enabled), with the group capability set to 901, do the following: 1. Configure the switch. Configure your switches' trunking mode to match the mode being used on the server. The mode is Cisco Fast EtherChannel (FEC). 2. Configure the server: a. Set the link aggregate's group capability to 901. Enter: lanadmin -X -g 901 901 900 b.
reset = Reset LAN Interface to execute its selftest specific = Go to Driver specific menu Enter command: display LAN INTERFACE STATUS DISPLAY Tue , Oct 1,2002 14:07:10 PPA Number Description Type (value) MTU Size Speed Station Address Administration Status (value) Operation Status (value) Last Change = = = = = = = = = 901 lan901 Hewlett-Packard LinkAggregate Interface ethernet-csmacd(6) 1500 300000000 0x108318b927 up(1) up(1) 102502046 For link-aggregate MIB statistics, press to continue Inbound
quit = Terminate the Administration, return to shell Enter command: port Enter Port PPA number: 1 **** PORT NUMBER: 1 ******* pagpEnabled : ENABLED ppMyData.deviceId : 00108318b907 ppMyData.distReq : LEARNCAP_AGPORT ppMyData.portPriority : 0 ppMyData.sentPortIfIndex : 1 ppMyData.groupCapability : 901 ppMyData.groupIfIndex : 1 ppNoPagpTimerI : 0 ppNoTransTimerQ : 0 ppTHToTATimerS : 0 ppSlowHelloTimerA : 7 ppPartnerCount : 1 ++++++++ PARTNER DATA ++++++++ PARTNER 0 ppPartnerData.
5 Administering HP APA Administering HP APA consists of performing the following tasks: • “Modifying HP APA Global Parameters” (page 63) • “Logging Messages to the syslog.log File” (page 63) Modifying HP APA Global Parameters The following HP APA global parameters are stored in the /etc/rc.config.d/hp_apaconf file: Default Port Mode The default configuration mode for link aggregates. The supported values are as follows: MANUAL Disables FEC and IEEE 802.3ad LACP on all ports that support HP APA.
messages are a subset of the messages APA logs to the nettl log file, but are reworded for readability and understandability. For information on configuring APA to log status messages to the /var/adm/syslog/ syslog.log file, see “Modifying HP APA Global Parameters” (page 63). Examples This section contains sample status messages that might appear in the syslog.log file.
packets (lan1 is active) 2 Mar 3 08:58:03 hpserver1 vmunix: APA/LM: FOG:lan900 - Proactive Failover Occurred \ (lan901 is active) 3 1 2 3 Connectivity among links in the failover group was lost. The link in the failover group started receiving poll packets; it has recovered connectivity. Proactive failover occurred in the failover group. For detailed log messages, use the NetTL subsystem. See nettl(1M) for more information.
Example 5-1 Sample Link Aggregate 32-Bit Statistics # lanadmin -g 900 LAN INTERFACE STATUS DISPLAY Fri, Jul 30, 2007 12:20:23 PPA Number Description Type (value) MTU Size Speed Station Address Administration Status (value) Operation Status (value) Last Change Inbound Octets Inbound Unicast Packets Inbound Non-Unicast Packets Inbound Discards Inbound Errors Inbound Unknown Protocols Outbound Octets Outbound Unicast Packets Outbound Non-Unicast Packets Outbound Discards Outbound Errors Outbound Queue Length S
6 Troubleshooting HP APA This chapter contains a diagnostic map to help you solve problems that might occur when you use HP APA. Use this chapter with the appropriate HP and third-party switch documentation to solve as many problems as possible at your level. Operation Network problems can occur for a number of reasons. The diagnostic map in this chapter help you to isolate the problem.
System booted without errors? NO YES Edit the /etc/rc.log file and search for the string hpapa. This takes you to the beginning of the HP APA start-up section. If there are failover groups configured on your system, the following message indicates an error: ERROR: lanapplyconf failed If there are no failover groups configured on your system, you can ignore the error message. If you see error messages about link aggregate attributes, do the following: 1.
NIC operating? Verify that the NIC is operating. Enter the following command: NO # lanscan If the Hardware State is DOWN, do the following: YES 1. Check the status of the Link LED. If the LED is off, check the connection to the switch. Make sure that the switch is configured in the correct mode and is autonegotiating, if necessary. Reset the NIC with following command: # lanadmin -r linkAggPPA 2. 3.
Problems (FEC Mode)” (page 79) for FEC_AUTO mode link aggregates, or “Solving Failover Group Problems” (page 83) for failover groups. Solving Link Aggregate Problems (MANUAL Mode) Link partner configured? NO YES Link aggregate configured? NO Verify that the link aggregate is configured by issuing the lanscan command. Link aggregate names begin with LinkAgg. If the link aggregate is not configured, use the SAM to configure it. See “Configuring a Link Aggregate” (page 36) for more information.
# lanadmin -X -d -portPPA linkAggPPA If there are ports you wanted in the link aggregate that are not members of the link aggregate, do the following: 1. If FEC or LACP is enabled on the port, stop autonegotiation with the following command: # lanadmin -X -p portPPA MANUAL linkAggPPA 2. Verify the type, speed (including duplex), and MTU size of each port you want with the following command: # lanadmin -g portPPA 3. All ports must have the same values.
1. 2. Host known? Ensure that the network devices are configured properly on the local host, using the netstat -i command. Verify that the routing tables on the local host are correct, using the netstat -r command. If a host is not known, the following message is displayed: NO unknown host Complete the following steps: YES 1. 2. 3. 4. 5. Host reachable? Verify that the user is trying to reach the remote host using a valid host name.
7. 8. File access successful? NO Verify that the local host's address-to-name translation for the remote host is correct by using the nslookup command. Inspect the routers along the path to the remote host to determine whether they have security features enabled that prevent you from reaching the remote host. If a file cannot be accessed using the rcp or rsh command, the following message is displayed: permission denied Complete the following steps: YES 1. 2. 3. 4. 5.
Problem still exists? 2. Report your problem to HP. See “Reporting Problems ” (page 92). After you identify the host with the problem, do the following: a. Confirm that the NIC is properly configured. Verify that the broadcast address and address mask for the local host are correct. b. Make sure the local host's /etc/hosts file has the correct IP address for the local host. c. Make sure the cabling from the local host to the network is intact and properly connected. d.
The Active Ports PPA # field shows the ports that compose the link aggregate. If the link aggregate contains ports that you do not want in the aggregate, delete the ports by issuing the following command: # lanadmin -X -d portPPA linkAggPPA where portPPA can be a list of multiple ports numbers separated by a space.
# lanadmin -X -k value linkAggPPA 3. Verify that the port's key value matches the value of the link aggregate. If it does not, do the following: a. Set the port mode to MANUAL with the following command: # lanadmin -X -p MANUAL portPPA b. Set the key value and mode with the following commands: # lanadmin -X -k value portPPA # lanadmin -X -p LACP_AUTO portPPA c. d. All ports active? NO The ports then negotiate with the link partner (approximately 10 seconds) and join the link aggregate.
4. 5. Host reachable? entry. If it is not, add it. Also, verify that the DNS service has information about the remote host. If your site uses the NIS name service for name-to-address translation, look in the /etc/nsswitch.conf file to see if nis is specified as a service for the hosts database entry. If it is not, add it. Also, verify if the NIS service has information about the remote host. If your /etc/nsswitch.
4. 5. telnet command successful? NO YES Verify that the directory and file protection on the files to be copied or the .rhosts file on the remote system are correct. If you are using NFS, see the appropriate NFS documentation. If the telnet command is not successful, there is a problem with the Transport Layer (OSI Layer 4). Do the following: 1. Verify whether you have a problem with a pseudoterminal driver (pty) on your system by issuing the ftp command to the remote host.
If you suspect or know that you are experiencing packet loss over your connection, do the following: 1. LACP mode link aggregates require time to negotiate with the switch. You might drop some packets during this time. YES 2. Check the trunk is configured correctly on the switch. Make sure that ports connected to the ports on the server are assigned to the LACP trunk. If the LACP trunk is assigned to any other ports on the server, this can cause packets to be dropped. 3. Check the switch port configuration.
# lanadmin -X -d portPPA linkAggPPA If there are ports you wanted in the link aggregate that are not members of the link aggregate, do the following: 1. Verify the type, speed (including duplex), MTU size, and checksum offload of each port you want with the following command: # lanadmin -g portPPA All ports must have the same values. NOTE: You cannot aggregate a port that has no CKO capability with a port that has CKO capability, even if its CKO capability is currently disabled.
3. Verify that the port's group capability value matches the value of the link aggregate. If it does not, do the following: a. Set the port mode to MANUAL with the following command: # lanadmin -X -p MANUAL portPPA b. Set the group capability and mode values with the following commands: # lanadmin -X -g value portPPA # lanadmin -X -p FEC_AUTO portPPA c. d. All ports active? NO The ports then negotiate with the link partner (approximately 10 seconds) and join the link aggregate.
4. 5. Host reachable? entry. If it is not, add it. Also, verify that the DNS service has information about the remote host. If your site uses the NIS name service for name-to-address translation, look in the /etc/nsswitch.conf file to see if nis is specified as a service for the hosts database entry. If it is not, add it. Also, verify if the NIS service has information about the remote host. If your /etc/nsswitch.
4. 5. telnet command successful? NO YES Verify that the directory and file protection on the files to be copied or the .rhosts file on the remote system are correct. If you are using NFS, see the appropriate NFS documentation. If the telnet command is not successful, there is a problem with the Transport Layer (OSI Layer 4). Do the following: 1. Verify whether you have a problem with a pseudoterminal driver (pty) on your system by issuing the ftp command to the remote host.
Link aggregate configured? NO Verify that the failover group is configured by issuing the following command: # lanadmin -x -i linkAggPPA If the failover group is not configured (Link Aggregation Mode is MODE_UNDEFINED), edit the configuration files to configure it. For more information, see “Editing Configuration Files for Failover Groups” (page 107).
• • • • • All ports UP? NO The port has IP addresses or VLANS configured on it. Unconfigure them from the port and start LAN_MONITOR again. The port is not in full-duplex mode. Make sure the port on the server and on the switch are set to full-duplex or to autonegotiate. Start LAN_MONITOR again. The port is already in a link aggregate or failover group. Find another port to use. The port is not an Ethernet port or is not supported by HP APA. The port is down. Wait for the port to come up.
For cost-based failover groups, edit the configuration files and verify the port cost for both ports. If the active port priority is greater than the intended port priority, modify the priority value for the intended port (making it greater than the current active port). If the active port cost is less than the intended port cost, modify the cost value for the intended port (making it less than the current active port). For more information, see “Editing Configuration Files for Failover Groups” (page 107).
5. Host reachable? is specified as a service for the hosts database entry. If it is not, add it. Also, verify if the NIS service has information about the remote host. If your /etc/nsswitch.conf file lists files as the only name-to-address translation mechanism, the /etc/hosts file does not have information on the remote host. See nsswitch.conf(4) for more information. If a remote host is not reachable, the following message is displayed: NO host is unreachable Complete the following steps: YES 1. 2.
telnet command successful? NO YES If the telnet command is not successful, there is a problem with the Transport Layer (OSI Layer 4). Do the following: 1. Verify whether you have a problem with a pseudoterminal driver (pty) on your system by issuing the ftp command to the remote host. If the command is successful, there is a problem with a pty on your system. Contact your HP representative. See “Reporting Problems ” (page 92) for more information. 2.
Testing Access to Internet Network Hosts Use the ping command to test your system's ability to reach a host on the Internet network. The ping command has the following syntax: /usr/sbin/ping [options] [hostname] The ping command sends an Internet Control Message Protocol (ICMP) echo request to the host specified. When the request is successful, the remote host sends the data back to the local host where is printed to stdout.
Example 6-1 Sample ioscan –f Output # ioscan -f Class I H/W Path Driver S/W State H/W Type Description ==================================================================== bc 0 root CLAIMED BUS_NEXUS bc 1 8 ccio CLAIMED BUS_NEXUS I/O Adapter ba 0 8/4 GSCtoPCI CLAIMED BUS_NEXUS PCI Bus Bridge - GSCtoPCI lan 4 8/4/1/0 igelan CLAIMED INTERFACE HP PCI 1000Base-T Core lan 5 8/4/2/0 igelan CLAIMED INTERFACE HP PCI 1000Base-T Core ba 1 8/8 GSCtoPCI CLAIMED BUS_NEXUS PCI Bus Bridge - GSCtoPCI lan 6 8/8/1/0 igelan C
To access the GUI version of the logging and tracing utility, enter the following command: nettladm There are three levels of logging: • • • At link aggregate level At Cisco's FEC level At IEEE 802.3ad LACP level The following examples show how to perform different tasks from the command line: • To turn on all logging at link aggregate level, enter: nettl -log 0xf -e HP_APA • To turn on all logging at Cisco's FEC level, enter: nettl -log 0xf -e HP_APAPORT • To turn on all logging at IEEE 802.
Process ID : [ICS] Subsystem : HP_APA User ID ( UID ) : -1 Log Class : INFORMATIVE Device ID : 903 Path ID : 0 Connection ID : 0 Log Instance : 0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <4015> HP Auto-Port Aggregation product enabled TCP Segmentation Offload capability for link aggregation 903. VMTU = 32160. For more information on using the command line interface and the GUI version, see nettl(1M) and nettladm(1M), respectively.
• • • • 4. 5. 6. 7. 8. /etc/rc.config.d/netconf /etc/rc.config.d/hp_apaconf /etc/rc.config.d/hp_apaportconf /etc/lanmon/lanconfig.ascii Enter the dmesg command and record messages about the status of the NIC. Enter the lanscan -v command and record the output. Enter the display command of the lanadmin diagnostic on the HP APA interface and record the output. Record all error messages and numbers that appear at the user terminal and the system console. Save all network log files.
A Product Specifications Table A-1 summarizes the capabilities of HP APA and LAN Monitor.
1 MANUAL mode: Can be Load Balancing or Non-Load Balancing • Load Balancing: MAC, IP, or LB_PORT algorithm • Non–Load Balancing: Hot Standby mode 2 3 4 5 6 7 8 96 For HP-UX 11.0 and 11i v1, the maximum is 4. Load Balancing is configurable for outbound traffic on Ethernet links only. Links must be of the same speed and type (100Base-T or 1000Base-T). LACP requires full duplex (FD) operation of the links. You cannot reset the speed, duplex mode, or MTU size over a link aggregate. For HP-UX 11.
B HP APA Configuration Files HP APA uses the following configuration files: • /etc/rc.config.d/hp_apaconf • /etc/rc.config.d/hp_apaportconf • /etc/lanmon/lanconfig.ascii The following sections describe each file and its parameters. For configuration examples that describe editing these files, see Appendix C (page 105). hp_apaconf File The /etc/rc.config.d/hp_apaconf file contains the following general HP APA configuration parameters for link aggregates.
HP_APA_HOT_STANDBY physical ports can be aggregated into a common PAgP link aggregate. Set the group capability to be the same for all network physical ports in the same link aggregate. Ports going to different link aggregates must have different group capabilities. This value must match the value of HP_APAPORT_GROUP_CAPABILITY in the /etc/ rc.config.d/hp_apaportconf file. Configure a link aggregate for Hot Standby mode. This parameter is mutually exclusive with the HP_APA_LOAD_BALANCE_MODE parameter.
NOTE: Some IP applications might not have TCP/UDP port numbers, or the TCP/UDP port numbers are encrypted. For such cases, even if you choose LB_PORT, it will fall back to the default LB_MAC. HP_APA_MANUAL_LA HP_APA_MAX_LINKAGGS HP_APA_USE_SYSLOG Manually set ports for the specified link aggregate. Specify the ports with PPA numbers. Separate each port by a comma (,). Sets the maximum number of link aggregates for the server. The valid range is from 5 to 50. The default value is 50.
to choose the group capability for link aggregates. Ports going to different link aggregates should have different group capabilities. The default group capability is 5. HP_APAPORT_INTERFACE_NAME HP_APAPORT_KEY Name of physical interface. For example, lan0 and lan1. For LACP_AUTO only. An integer value that determines which network physical ports can be aggregated into a common LACP link aggregate. Set the key to be the same for all network physical ports.
Example B-1 Sample lanconfig.ascii Configuration File #*********************************************** #******* LAN MONITOR CONFIGURATION FILE #*** For complete details about the parameters and how #*** to set them, consult the lanqueryconf(1m) manpage #*** or your manual. #*** All timeout values are microseconds; they are rounded up #*** or down appropriately.
FAILOVER_GROUP STATIONARY_IP STANDBY PRIMARY lan900 193.33.33.33 lan9 3 lan6 5 FAILOVER_GROUP STATIONARY_IP PRIMARY STANDBY STANDBY STANDBY lan901 195.55.55.55 lan4 5 lan1 3 lan2 3 lan3 3 FAILOVER_GROUP STATIONARY_IP PRIMARY STANDBY lan902 fe80::2 lan11 5 lan10 3 The lanconfig.ascii configuration file contains the following fields: NODE_NAME The name of the system as obtained by gethostname. This must be the first line in the file. The name for a single LAN Monitor failover group.
• • • The LAN interface name (for example, lan0 and lan1). The port priority that will be assigned to the port. The port with the highest priority is the port. A cost value for the LAN interface preceded by a colon (:). This is an optional field. See “Proactive Failover” (p. 15) for more information. The number of microseconds between polling messages. You can specify this keyword multiple times in the configuration file. A link aggregate's polling interval is set to the last specified interval.
C Configuring HP APA by Editing Files This appendix provides information about editing the different configuration files to configure the following: • Link aggregates • Failover groups Editing Configuration Files for Link Aggregates This section shows how to configure the three modes associated with port aggregation. For release-specific information, see the release notes on your system in the /opt/networkdocs directory or on the web at: http://www.docs.hp.
3. Activate the new configuration by entering: # # # # /sbin/init.d/hplm stop 1 /sbin/init.d/hpapa stop 2 /sbin/init.d/hpapa start /sbin/init.d/hplm start 3 1 2 3 If failover groups are also configured. This can interrupt traffic on existing link aggregates. For configuring failover groups. FEC_AUTO Port Configuration Mode To configure load balancing and group capability for link aggregate 901 with MAC-based load balancing, do the following: 1. Edit the /etc/rc.config.
HP_APAPORT_KEY[1]=902 HP_APAPORT_CONFIG_MODE[1]=LACP_AUTO NOTE: Set the LACP_AUTO key in the previous examples to be the same as that of the link aggregate to which it belongs. Ports going to different link aggregates should have different keys. 3. Activate the new configuration by entering: # # # # /sbin/init.d/hplm stop 1 /sbin/init.d/hpapa stop 2 /sbin/init.d/hpapa start /sbin/init.d/hplm start 3 1 2 3 If failover groups are also configured. This can interrupt traffic on existing link aggregates.
POLLING_INTERVAL DEAD_COUNT LM_RAPID_ARP LM_RAPID_ARP_INTERVAL LM_RAPID_ARP_COUNT FAILOVER_GROUP STATIONARY_IP STANDBY PRIMARY 10000000 3 off 1000000 10 lan900 192.1.1.1 lan11 3 lan10 5 7. Compare the /etc/lanmon/lanconfig.ascii file with the /tmp/filename file. If they are not the same, decide which file you want to use. If you want to use the file in /tmp, copy that file to /etc/lanmon/lanconfig.ascii. 8. Edit the /etc/lanmon/lanconfig.
1. Connect lan1 and lan2 to the same switch or to different switches or hubs on the same subnet. NOTE: 2. Be sure that trunking is not enabled on the switch ports.
8. Use the lanapplyconf command to build the failover group, and check it with the lanscan -q and netstat -in commands: # lanapplyconf Reading ASCII file /etc/lanmon/lanconfig.ascii Creating Fail-Over Group lan900 # lanscan -q 4 0 3 900 1 2 901 902 903 904 # netstat -in Name Mtu Network Address Ipkts lan0 1500 191.1.1.0 191.1.1.1 1794 lo0 4136 127.0.0.0 127.0.0.1 390 lan900 1500 192.1.1.0 192.1.1.
2. Assign a priority to the aggregates. If you prefer lan900 to be the active port when both link aggregates have the same number of operational ports, assign lan900 a higher priority. If you do not, the active port changes when one of the link aggregates becomes more efficient (has more operational ports or higher bandwidth). In this example, you assign lan900 a priority of 5 and lan901 a priority of 3. 3. Edit the lanconfig.ascii configuration file. FAILOVER_GROUP lan902 STATIONARY_IP 192.19.20.
3. Edit the configuration file. FAILOVER_GROUP lan902 STATIONARY_IP 10.0.1.60 # Primary/Standby Interface Priority : Cost # –––––––––––––––––––––––+–––––––––––––+–––––––––––––––– PRIMARY lan900 8 : 2 STANDBY lan901 7 : 3 Table C-2 lists lan900 events for this example and how proactive failover determines the active port for lan902. Table C-2 lan900 Events and Proactive Failover (Unequal Network Costs) Event lan901 Normalized Port Cost Active Port Failover group constructed1 2/200, or .
D VLANs over APA Using HP Procurve Switches For the September 2006 and later releases of HP APA for HP-UX 11i v2 (B.11.23.20), this appendix describes how to create VLANs over trunks and ports on an HP Procurve switch. For more information about creating VLANs on HP Procurve switches, see Procurve, Series 6400cl Switches, Series 5300xl Switches, Series 3400cl Switches, Advanced Traffic Management Guide at: ftp://ftp.hp.com/pub/networking/software/6400-5300-3400-AdvTrafficMgmt-Oct2005-59906051.
4. Verify that the VLAN with ID 333 is tagged to Trk3. Enter: HP ProCurve Switch 5308xl# show vlan 333 Status and Counters - VLAN Information - Ports - VLAN 333 802.1Q VLAN ID : 333 Name : vlan_333 Status : Port-based Voice : No Port Information Mode Unknown VLAN Status ---------------- -------- ------------ ---------Trk3 Tagged Learn Up VLAN 333 has been successfully tagged to Trk3. On the server side, complete the following steps: 1. Create link aggregate lan900 using ports lan3 and lan4.
Figure D-2 VLAN over a Failover Group Run lan901 Atn. Fault lan5 Remote Power lan6 Failover Group lan902: lan901 (Primary), lan5, 6 (Standby) Switch1 Switch2 Trk4 Vlan 333 Vlan 334 F17 F18 Vlan 333 Vlan 334 As the first step in creating the failover group, associate the ports in link aggregate lan901 to Trk4 on Switch 1; set the trunk type to trunk (MANUAL trunking). Connect ports lan5 and lan6 to ports F17 and F18, respectively, on Switch 2.
3. Verify that the VLANs with IDs 333 and 334 are tagged correctly. Enter: Switch 1# show vlan 333 Status and Counters - VLAN Information - Ports - VLAN 333 802.1Q VLAN ID : 333 Name : vlan_333 Status : Port-based Voice : No Port Information Mode ---------------- -------Trk4 Tagged Switch 1# show vlan 334 Status and Counters - VLAN Unknown VLAN Status ------------ ---------Learn Up Information - Ports - VLAN 334 802.
On the server side, complete the following steps: 1. Create link aggregate lan901. 2. Create failover group lan902.For HP-UX 11i v2 and earlier versions, the primary port in the failover group must have an IP address configured on it. 3. Create VLANs 333 and 334 over lan902. Enter: # lanadmin -V create vlanid 333 902 Successfully configured. lan5000: vlanid 333 name UNNAMED pri 0 tos 0 tos_override IP_HEADER pri_override CONF_PRI ppa 902 # lanadmin -V create vlanid 334 902 Successfully configured.
E Switch Configuration Information This appendix provides some information on how to configure the following switches for use with HP APA: • Alteon • Cisco • Extreme • Procurve See your switch documentation for complete information. Alteon Switches Alteon switches have 8 ports. Fiber blades support 1500 and 9000 MTU (jumbo frames).
Creating an LACP Link Aggregation Switch1#enable Password:password Switch1#config term config command 1 . . Ctrl+z Switch1(config-if)#interface range gigabitethernet 3/1 - 3 Switch1(config-if-range)#no ip address Switch1(config-if-range)#channel-protocol lacp Switch1(config-if-range)#channel-group 1 mode active Creating a port-channel interface Port-channel 1 Switch1(config-if-range)#end Switch1#config term config command . .
Port-channel: Po1 (Primary Aggregator) -----------Age of the Port-channel = 00d:01h:46m:08s Logical slot/port = 14/1 Number of ports = 3 Port state = Port-channel Ag-Inuse Protocol = LACP Ports in the Port-channel: Index Load Port EC state No of bits ------+------+------+------------------+----------2 49 Gi3/1 Active 3 0 92 Gi3/2 Active 3 1 24 Gi3/3 Active 2 Time since last port bundled: 00d:01h:44m:15s Gi3/1 Displaying a Port in a Link Aggregation Switch1#show interface GigabitEthernet 3/3 etherchanne
Port Gi3/2 System ID Port Number 00000,0000.0000.00000x0 LACP Partner Port Priority 0 Partner Oper Key 0x0 Age 297s Flags SP Partner Port State 0x0 Partner's information: Partner Partner System ID Port Number 00000,0030.6ef4.
Configuring LACP Link Aggregates ExtremeWare v. 7.5 enables you to configure LACP. Use contiguous ports and let the first port be the master.
HP ProCurve Switch 4108GL# Make sure that the ports you think you are aggregating on the switch are the right ones. This can be combined with the process of creating switch aggregations. Do the following: 1. Log in to the switch. 2. Enter menu at the prompt. 3. Select Switch Configuration. 4. Select Port/Trunk Settings. 5. Verify which cards or ports are connected to the switch ports that you intend to configure. Do the following: a. On the computer, enter the following command for each port: llp lanx b.
4. The following prompt displays: HP9304(config)# TIP: Enter [command name] ? to display the syntax for a command. Configuring HP APA To configure HP APA to use the Procurve 9304/8 switch, do the following: 1. Verify port connections before configuring APA. Do the following: a. Log in to the switch using the previous steps. b. Enter the interface e [slot]/[port] command. For example: HP9304(config)#interface e 2/1 c. Enter the following: HP9304(config-if-e100-2/1)#disable d.
2. Configure HP APA. Do the following: a. Unplug all LAN connections to be aggregated. b. Create the aggregations. They must be either 2 or 4 port trunks. For two-port aggregations, the beginning port number must be odd. For four-port aggregations, the beginning port number must be a number divisible by 4 plus 1 (for example, 1, 5, 9, 13). Use the following command to create a four-port aggregation: HP9304(config)#trunk server e 1/5 to 1/8 c. Save the configuration. Enter: HP9304(config)#write mem d.
Glossary A administrative key An integer value that determines which network physical ports can be aggregated into a common LACP link aggregate. The key must match the key of the desired aggregate that the user wants the port to join aggregate A group. For APA, a group of up to eight ports (32 ports for LACP aggregates) makes one link aggregate. There can be 50 link aggregates per computer. aggregation See aggregate. APA Auto Port Aggregation ARP Address Resolution Protocol.
M MAC Media Access Control MANUAL The default mode of port configuration. Can be performed by using SMH (preferred) or by editing two configuration files. MIB Management Information Base N network adapter A network device which has one or more network physical ports. network physical port The communications channel formed when you attach a network cable between a specific network port (adapter card) and a LAN device. P PAgP See Port Aggregation Protocol.
Index abnormal termination (FEC_AUTO mode), 83 abnormal termination (LACP_AUTO mode), 78 abnormal termination (MANUAL mode), 73 packet loss, 79 A aggregate instance number, 32 algorithm HOT_STANDBY, 34 LB_IP, 34 LB_MAC, 34 LB_PORT, 34 load balancing, 14 application hang (failover group), 88 hang (FEC_AUTO mode), 83 hang (LACP_AUTO mode), 78 hang (MANUAL mode), 73 B binary file caution, 103 C caution binary file, 103 HP Serviceguard and HOT_STANDBY mode, 35 lanadmin -a option, 58 lanadmin command, 51 lanq
configuring with lanadmin, 58 configuring with SAM, 36–41 verifying link partner configuration, 79 file editing for APA configuration, 105–107 hp_apaconf, 48, 97–99 hp_apaportconf, 48, 99–100 netconf, 107 file access verifying (failover group), 87 verifying (FEC_AUTO mode), 82 verifying (LACP_AUTO mode), 77 verifying (MANUAL mode), 73 FIRST_APA_PORT, 63 FIRST_LACP_PORT, 63 G global parameter default port mode, 63 LACP SYSID mode, 63 LACP timeout, 63 maximum link aggregates, 63 Use syslog, 63 group (see fai
verifying TSO support, 16 lanconfig file, 103 lanconfig.
in FEC_AUTO mode link aggregates, 82 in LACP_AUTO mode link aggregate, 77 in MANUAL mode link aggregate, 73 physical port displaying using SAM, 38, 42 modifying attributes using SAM, 39 modifying with SAM, 43 ping command, 89 polling interval specifying in lanconfig.
lanadmin command, 59 tip displaying Procurve command syntax, 125 specifying group capability, 35 troubleshooting, 67 TSO and failover group, 16 and link aggregate, 16 status messages, 91 verifying support, 16 U unknown host message in failover group, 86 in FEC_AUTO mode link aggregates, 81 in LACP_AUTO mode link aggregate, 76 in MANUAL mode link aggregate, 72 V VLAN and failover group, 16, 114 and link aggregate, 16, 113 configuring, 47 133
