Windows Server 2008 SP2 and Windows Server 2008 R2 on HP Integrity Servers Network Adapter Teaming White Paper HP Part Number: 5992-5629 Published: October 2009
© Copyright 2009 Hewlett-Packard Development Company, L.P. Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents 1 Introduction......................................................................................................................9 Abstract...................................................................................................................................................9 How to Read This Document.................................................................................................................9 Section Layout.................................................
Network Adapter Miniport Driver............................................................................................30 Teaming Intermediate Driver.....................................................................................................30 HP Network Configuration Utility............................................................................................30 HP Teaming and Layer 2 Versus Layer 3 Addresses.......................................................................
Dual Channel Applications........................................................................................................78 Recommended Configurations for Dual Channel Environments.............................................78 802.3ad Dynamic Dual Channel Load Balancing............................................................................79 802.3ad Dynamic Dual Channel Configuration.........................................................................79 Automatic...........................
List of Figures 3-1 3-2 3-3 3-4 3-5 3-6 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 4-13 4-14 4-15 4-16 4-17 4-18 4-19 4-20 4-21 4-22 4-23 4-24 4-25 4-26 4-27 A-1 A-2 6 HP Integrity Network Adapter Teaming......................................................................................15 Using Multi-homing for server network redundancy..................................................................16 Using NIC teaming for server network redundancy.............................................
List of Tables 3-1 3-2 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 4-13 4-14 4-15 4-16 5-1 Basic versus Advanced teaming features......................................................................................22 Dual Channel capabilities comparison..........................................................................................23 Heartbeat Frame Format...............................................................................................................36 802.
1 Introduction Abstract This white paper provides a high- and low-level discussion of the technology behind HP Integrity Network Adapter Teaming for HP Integrity servers running Microsoft® Windows®. HP Integrity Network Adapter Teaming is software-based technology used by server administrators and network administrators to increase a server’s network availability and performance.
This white paper specifically discusses HP Integrity Network Adapter Teaming for Windows Server 2008 with Service Pack 2 (SP2) and Windows Server 2008 R2. For technical information on network adapter teaming for other operating systems, refer to www.hp.com.
2 An Executive Overview of Teaming What is HP Integrity Network Adapter Teaming? HP Integrity Network Adapter Teaming is software-based technology used by server administrators and network administrators to increase a server’s network availability and performance. HP Integrity Network Adapter Teaming provides network adapter, network port, network cable, switch, and communication path fault recovery technology, in addition to, transmit and receive load balancing technology.
• administrator to provide a switch port to the server that has 100% uptime. While network administrators can achieve high availability in the network, they can’t promise 100% availability for the switch and switch port to which the server’s network adapter port is connected. Switch failures, like server failures, do occur. Switch misconfigurations, like server misconfigurations, do occur.
today’s servers ship standard with at least two network adapter ports, many built directly onto the motherboard (in other words, LAN On Motherboard or LOM). The SA need only install HP Integrity Network Adapter Teaming software, select at least two network adapter ports, and create a team. Everything else works the same, both inside the server (server’s protocol stack) and outside of the server (the server’s clients).
3 Teaming Fundamentals for the First-Time User A Technical Overview of HP Integrity Network Adapter Teaming HP Integrity Network Adapter Teaming provides fault tolerance and load balancing across a team of two or more network adapter ports. The term “team” refers to the concept of multiple network adapters (teamed ports) working together as a single network adapter, commonly referred to as a virtual network adapter or virtual NIC interface.
Figure 3-2 Using Multi-homing for server network redundancy 1.1.1.3 1.1.1.1 1.1.1.4 1.1.1.2 HP Integrity Server Server has redundant path for outgoing traffic. From the server’s IP perspective, either adapter provides access to the network. 1.1.1.5 Switch Clients How do clients know the server has two or more IP addresses? Which IP address represents the server for DNS/NetBIOS/etc.
Figure 3-3 Using NIC teaming for server network redundancy Integrity Server Teaming Driver 1.1.1.1 Clients 1.1.1.3 Switch Co ns o le 1.1.1.4 1.1.1.5 •Server has redundant path for outgoing traffic. From the Server’s IP perspective, single “virtual” adapter provides access to the network. •Provides redundant connectivity from Clients to Server for same Server IP address. Server is known by single IP for DNS/NetBIOS/etc.
Figure 3-4 Teaming types and functionality Advanced Teaming Basic Teaming Receive Load Balancing Basic Teaming Basic Teaming Transmit Load Balancing Switch Fault Tolerance* Switch Fault Tolerance* Network Adapter Fault Tolerance Network Fault Tolerance Only (NFT) Network Fault Tolerance Only with Preference Order Transmit Load Balancing with Fault Tolerance (TLB) Transmit Load Balancing with Fault Tolerance and Preference Order Switch-assisted Load Balancing with Fault Tolerance (SLB) 802.
Transmit Load Balancing with Fault Tolerance (TLB) Transmit Load Balancing with Fault Tolerance (TLB) is a team type that allows the server to load balance its transmit traffic. TLB is switch independent and supports switch fault tolerance by allowing the teamed ports to be connected to more than one switch in the same LAN. With TLB, traffic received by the server is not load balanced. The primary teamed port is responsible for receiving all traffic destined for the server.
different switches and one of the switches fail, the team remains available via the group attached to the functional switch. Switch-assisted Dual Channel Load Balancing is an advanced feature of HP Integrity Network Adapter Teaming. Refer to “Basic Teaming Versus Advanced Teaming” for more information. 802.3ad Dynamic Dual Channel Load Balancing 802.3ad Dynamic Dual Channel Load Balancing, referred to as Dynamic Dual Channel, is identical to Dual Channel except that the switch must support the IEEE 802.
Figure 3-5 How to choose the best team type What type of NIC Team do I Need? NIC Teaming is not needed I need NIC Fault Tolerance and/or Load Balancing NO Team Type Automatically set to IEEE 802.3ad YES YES Choose Team Type “Network Fault Tolerant Only” Choose Team Type “Network Fault Tolerant Only with Preference Order” NO I want to manually configure the Team type I need to designate the order in which ports become Primary NO All NICs connected to a single IEEE 802.
Basic Teaming Versus Advanced Teaming You can choose to use the basic HP Integrity Network Adapter Teaming features like NFT, TLB, and SLB, or the advanced teaming features such as Dual Channel, Dynamic Dual Channel, Active Path, and Fast Path. The following table summarizes these Basic and Advanced features. Table 3-1 Basic versus Advanced teaming features Basic Teaming Features NFT √ NFT with Preference √ TLB √ TLB with Preference √ SLB √ 802.
the team’s Primary port among one or more teamed ports with validated connectivity to the preferred Spanning Tree root switch. Fast Path is extremely important for configurations with teamed ports connected to more than one switch. • Dual Channel – Switch-assisted Dual Channel Load Balancing, simply referred to as Dual Channel, is a special team type designed by HP Engineering to accomplish everything that NFT, TLB and SLB team types accomplish all in a single team type (refer to Table 3-2).
NCU Main Page The NCU main page, labeled HP Network Configuration Utility Properties, is used to create and dissolve teams, monitor the overall health of teams and individual network adapters, and to export team configuration settings to an XML file. The main page is also used to access the Properties pages for team settings and individual NIC settings. Refer to Figure 3-6.
Team Properties Page Tabs • • • • • • • Teaming Controls – This tab is used to change the team name, select team type, select the Transmit Load Balancing algorithm, change port preference order for NFT with Preference teams, and to assign group membership for Dual Channel teams. Advanced Redundancy – This tab is used to manage Active Path, Fast Path, and monitor Fast Path port cost and path cost.
Getting Started Example Deployment of Basic Teaming A typical deployment of basic teaming might include the following steps: 1. Perform the initial installation. For example, complete the steps described above in the section:“Basic Deployment Steps for HP Integrity Network Adapter Teaming”. 2. Select the network adapter ports for a team. From the NCU main page, highlight two or more network adapter ports. Then, click the Team icon. 3. Set the team type to Automatic. a.
4. 4. Enable and configure advanced redundancy mechanisms. a. Select the Advanced Redundancy tab from the Team Properties page. b. Enable Active Path Failover by checking the box. c. Select Community Address ARP as the Echo Node Response Mechanism. d. Type the IP address of the designated Echo Node (for example, gateway router) in the Echo Node IP Address field. e. Select an unused IP address on the same subnet as the Echo Node and type it in the Community Probe IP Address field.
4 The Mechanics of Teaming for the Advanced User Section Objectives and Prerequisites This section is intended to provide an in-depth technical discussion of the mechanics of HP Integrity Network Adapter Teaming for the advanced user. This section assumes the reader has read the previous sections in this white paper and has hands-on experience in the deployment and usage of HP Integrity Network Adapter Teaming.
Teaming Software Components HP Integrity Network Adapter Teaming consists of three components: the Miniport driver, Intermediate driver, and configuration GUI. Network Adapter Miniport Driver The Miniport driver used with the HP network adapter is e1g6064.sys or e1e6064.sys or b57nd60i.sys, depending on the adapter in use. Teaming Intermediate Driver The Intermediate driver is CPQTEAM.SYS, and is used for all teaming functions involving HP Integrity supported adapters.
event, because the protocol address is directly assigned to the Intermediate (teaming) driver, and not to the Miniport driver. When transmitting frames, the current Primary adapter always transmits using the team’s MAC address as the Layer 2 address and the team’s Protocol address as the Layer 3 address. Non-Primary adapters always transmit using the MAC address assigned to them by the teaming driver and using the team’s protocol address as the Layer 3 address.
Figure 4-1 Overview of NFT communication ARP Table: Destination MAC/Source MAC/Destination IP/Source IP ARP Table: 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D E A 1.1.1.1 1.1.1.2 E B 1.1.1.1 1.1.1.3 E C 1.1.1.1 1.1.1.4 E D 1.1.1.1 1.1.1.5 Client A IP Address = 1.1.1.2 MAC = A Client B Destination MAC/Source MAC/Destination IP/Source IP A E 1.1.1.2 1.1.1.1 B E 1.1.1.3 1.1.1.1 C E 1.1.1.4 1.1.1.1 D E 1.1.1.5 1.1.1.1 IP Address = 1.1.1.
Figure 4-2 Scenario 4–A: A device pings an NFT team on the same Layer 2 network Ethernet MAC = A IP = 1.1.1.1 MAC = B Primary MAC = E Non-Primary Red NFT IP = 1.1.1.2 Blue (server) 1. Red transmits a broadcast ARP request asking for Blue’s MAC address. A user on Red issues the command ping 1.1.1.2 to initiate a ping to Blue. First, Red determines whether or not Blue is on the same Layer 2 network.
4. Blue transmits a broadcast ARP request asking for Red’s MAC address. NOTE: The following step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that matches 1.1.1.1. If Blue does not find one, then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address. NOTE: The following step will not occur if Step 4 does not take place.
• teamed port under test is able to receive the heartbeat frame, the test succeeds. This feature can be disabled, but fault tolerance will be reduced. Switch MAC table updates with team MAC address – The Primary Port must ensure that the switch has the team’s MAC address on the correct port, and that the switch keeps the team’s MAC address in its MAC table [or Content Addressable Memory (CAM) table].
frame. Therefore, a transmit path validation heartbeat frame may be transmitted once every 6 seconds (if the transmit path validation interval timer is set to the default of 3 seconds) on a port that has not transmitted anything. If the port does successfully transmit a heartbeat or any other kind of frame, its internal status is reset to 0.
Table 4-2 802.1Q Tagged Heartbeat Frame Format (continued) Source MAC address “MAC address of Teamed port” 6 bytes 802.
unplugged/disconnected. This results in the HP Integrity server being effectively disconnected from the entire network except for Switch A. Only the 50 users on Switch A can access the server, whereas the 50 users on Switch B, the 250 users connected via the Core Switch, and the router connection to any external network can no longer communicate with the server.
the Echo Node will always resolve the team’s IP address to a single port in the team, which would prevent Echo Node probe responses from being sent to every port in the team. Active Path uses a special frame as the Echo Node probe. This frame is identical to a normal ARP request frame except for one field – the Sender’s IP address field. In an Echo Node probe, the sender’s IP address field is intentionally left blank.
modifications to the designated Echo Node were required. Third, a frame type had to be chosen that would not cause loss of communication for normal data traffic between the Echo Node and the team since, in many environments, the Echo Node could be another server, another client, a switch, or a router functioning as the team’s gateway. To comply with the above requirements, Directed ARP was designed to use a modified ARP request frame as the Echo Node probe.
Table 4-4 Directed ARP Echo Node Probe REPLY Frame Format (continued) ARP operation 0x002 (ARP response) 2 bytes Source MAC address “Echo Node’s MAC Address” 6 bytes Source IP address “Echo Node’s IP Address” 4 bytes Destination MAC address “MAC address of Teamed port” 6 bytes Destination IP address 0.0.0.
the particular Echo Node. In contrast to Directed ARP, each individual teamed port is not necessarily required to transmit its own Echo Node request, yet all teamed ports individually receive an Echo Node probe reply. With Community Address ARP, the team’s Primary port transmits an Echo Node probe request to the designated Echo Node. The Echo Node then transmits an Echo Node reply that is received by every teamed port in that team.
1. 2. 3. Validate individual connectivity for a teamed port There is no special configuration on the designated Echo Node The Echo Node probe frame can’t cause loss of connectivity for the team Community Address ARP was designed to work with IP stacks that fail to respond to the Directed ARP probe packets. Community Address ARP allows practically any network device to function as the Echo Node. Although Directed ARP works with most IP stacks, certain routers fail to respond to the Directed ARP probes.
Table 4-6 Community Address ARP Echo Node Probe REPLY Frame Format (continued) Protocol address length 0x04 1 byte ARP operation 0x002 (ARP response) 2 bytes Source MAC address “Echo Node’s MAC Address” 6 bytes Source IP address “Echo Node’s IP Address” 4 bytes Destination MAC address “Community Probe MAC from GUI” 6 bytes Destination IP address “Community Probe IP from GUI” 4 bytes Checksum “varies” 4 bytes Community Address ARP’s Designated Echo Node: An Echo Node is any network device
Active Path Configuration To use Active Path, an SA must enable it on a per-team basis from the Advanced Redundancy tab located on the HP Integrity Network Adapter Teaming configuration GUI. Next, the SA should select which Active Path mechanism to use – Directed ARP or Community Address ARP from the Echo Node Response Mechanism drop-down box. For both Directed ARP and Community Address ARP, an IP address for an appropriate Echo Node must be provided in the Echo Node IP Address field.
Figure 4-5 Active Path failover configuration: Directed ARP Community Address ARP Configuration: To enable Active Path’s Community Address ARP mechanism, select Community Address ARP from the Echo Node Response Mechanism drop-down box. In addition to supplying the Echo Node IP address, it is necessary to supply a Community Probe IP Address. Community Address ARP guidelines: • Use the same Echo Node, Community Probe IP, and Community Probe MAC address for as many servers as possible on the same subnet.
Figure 4-6 Active Path failover configuration: Community Address ARP Fast Path Fast Path is a mechanism used by HP Integrity Network Adapter Teaming to intelligently and proactively determine the best teamed ports to use to provide optimum network connectivity for the server.
• • • Link loss detection – “Do I have physical link?” Transmit path validation – “ Can I successfully transmit a frame onto the network?” Receive path validation – “Can I successfully receive any kind of frame from the network?” These basic redundancy mechanisms do not provide for redundancy beyond the network adapter port in the team.
Figure 4-7 Upstream link failures cause server isolation 50 USERS ACCESS TO SERVER 250 USERS NO ACCESS TO SERVER Co ns o le Switch A HP NIC TEAMING IP= 1.1.1.1 NIC 1 Primary Co ns o le Core Switch NO ACCESS TO SERVER Switch B NIC 2 Non-Primary External Network o le ns Co Router NO ACCESS TO SERVER Server 50 USERS NO ACCESS TO SERVER In this case, the Fast Path mechanism, designed and developed by HP Engineering is an alternative solution.
Figure 4-8 Upstream link failures cause server receive bandwidth bottleneck Link 5 Gigabit Ethernet HP NIC TEAMING Co ns o le Switch A NIC 1 Primary Gigabit Ethernet Link 2 100 Mbps Link 3 NIC 2 Non-Primary Link 1 Gigabit Ethernet Gigabit Ethernet Link 4 Core Switch Co ns o le Switch B Server The Fast Path mechanism is a solution to this problem.
The solution to this problem is the Fast Path mechanism designed and developed by HP Engineering. In addition to Fast Path detecting full connectivity loss, as described in the first example, Fast Path can also detect when a server’s teamed ports have different hop count paths to the Core Switch (root switch).
Figure 4-10 Fast Path operation overview Fast Path Configuration Fast Path is enabled from the Advanced Redundancy tab in NCU by checking the Fast Path Failover checkbox (refer to Figure 4-11). Fast Path can also be monitored from the Advanced Redundancy tab. Under the Team Members section, the Port/Path Cost column provides real-time monitoring information. The Port Cost value (in other words, 4 in Figure 4-11) is the Spanning Tree cost associated with the current speed of the teamed port.
Figure 4-11 Fast Path configuration and monitoring Fast Path has two Spanning Tree Protocol configuration options: • Enabled or disabled, as discussed in the previous section • Spanning Tree Protocol (STP) type Both of these settings are located on the Advanced Redundancy tab in NCU. Refer to Figure 4-11. There are two predominate Spanning Tree Protocols (STP) in use today. They are IEEE 802.1D Spanning Tree Protocol and Cisco® Per VLAN Spanning Tree Plus (PVST+) Protocol.
1. 2. 3. 4. 5. Switch ports connected to teamed ports should be configured to bypass Spanning Tree’s block/listen/learn stages. These stages are not needed for ports connected to end node devices (for example, servers, PCs, etc.) Switch ports connected to teamed ports MUST transmit BPDUs. Without BPDUs, Fast Path will not have the information it needs to make decisions.
Table 4-7 Redundancy mechanism comparison chart (continued) Detects good ports versus best ports Mechanism can be manually disabled √ √ √ √ √ Failover Events Link Loss When a network port is a member of a team and loses physical link (in other words, wire fault, lost link light), the teaming driver prevents the port from being used by the team.
teamed port from a set of teamed ports that all have validated connectivity to the main network segment. As a result, Fast Path can initiate a failover from one teamed port operating as the Primary port to another non-Primary teamed port for either of two reasons. The first failover is caused by a teamed port losing connectivity with the main network segment, in which case the team will label the teamed port as Split LAN and will disable the teamed port for use by the team.
requires fault tolerance in case of a network adapter malfunction, but does not have a demand for receiving or transmitting more than the capacity of the Primary port.) Recommended Configurations for NFT Environments HP recommends the following: • Heartbeats should be enabled (default). • MAC addresses should not be manually set to a locally administered address (LAA) via the Microsoft Adapter Properties User Interface.
However, traffic received by the server is not load balanced, meaning the Primary teamed port is responsible for receiving all traffic destined for the server (refer to Figure 4-12). As with NFT, there are two types of team members, Primary and Non-Primary ports. The Primary port transmits and receives frames and the non-Primary ports only transmit frames. Figure 4-12 Overview of TLB communication ARP Table: Client A 1.1.1.1 = E IP Address = 1.1.1.2 1.1.1.3 = B 1.1.1.4 = C 1.1.1.
Figure 4-13 Scenario 4–B: A device pings a TLB team on the same Layer 2 network Ethernet MAC = A IP = 1.1.1.1 MAC = B Primary MAC = E Non-Primary Red TLB IP = 1.1.1.2 Blue (server) 1. Red transmits a broadcast ARP request asking for Blue’s MAC address. A user on Red issues the command ping 1.1.1.2 to initiate a ping to Blue. First, Red determines whether or not Blue is on the same Layer 2 network.
4. Blue transmits a broadcast ARP request asking for Red’s MAC address. NOTE: The following step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that matches 1.1.1.1. If Blue does not find one, then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address. NOTE: The following step will not occur if Step 4 does not take place.
Figure 4-14 Transmit Load Balancing Method Configuration Table 4-8 Transmit Load Balancing method comparison Preserves Frame Transmission Order Load Balances through Router Automatic √ √ TCP Connection √ √ Destination IP address √ √ Destination MAC address √ Round Robin Guarantees Equal Load Balancing HP Recommendation √ √ TLB Automatic Method Automatic is a load-balancing method that is designed to preserve frame ordering.
the Automatic mode. By deploying this method now, future upgrades will automatically take advantage of the new intelligence. TLB TCP Connection Method TCP Connection is also a load-balancing method that is designed to preserve frame ordering. This method will load balance outbound traffic based on the TCP port information in the frame’s TCP header. This load-balancing method combines the TCP source and destination ports to identify the TCP conversation.
Table 4-10 Load Balancing based on Destination IP Address (four- and five-port teams) (continued) 000 network port 1 000 network port 1 001 network port 2 001 network port 2 010 network port 3 010 network port 3 011 network port 4 011 network port 4 100 network port 1 100 network port 5 101 network port 2 101 network port 1 110 network port 3 110 network port 2 111 network port 4 111 network port 3 If the Destination IP Address algorithm is chosen, and the frame doesn’t have
Figure 4-15 Scenario 4-C: TLB team using IP address for load balancing algorithm MAC = C IP = 1.1.1.3 Green (Router) MAC = D IP = 2.2.2.3 Network 1.0.0.0 Network 2.0.0.0 Ethernet MAC = F IP = 1.1.1.4 Yellow Ethernet MAC = B Primary MAC = A IP = 1.1.1.1 MAC = E Non-Primary TLB IP = 2.2.2.1 Red Blue (server) TLB Destination MAC Address Method Destination MAC Address is another load-balancing method that will attempt to preserve frame ordering.
Table 4-11 Load balancing based on Destination MAC Address (two- and three-port teams) (continued) 011 network port 2 011 network port 1 100 network port 1 100 network port 2 101 network port 2 101 network port 3 110 network port 1 110 network port 1 111 network port 2 111 network port 2 Table 4-12 Load balancing based on Destination MAC Address (four- and five-port teams) Four-Port Team Five-Port Team Destination MAC Transmitting Port Destination MAC Transmitting Port 000 network
Figure 4-16 Scenario 4-D: TLB team using MAC address for load-balancing algorithm Ethernet MAC = F IP = 1.1.1.4 MAC = A IP = 1.1.1.1 MAC = B Primary Yellow MAC = E Non-Primary Red TLB IP = 1.1.1.2 Blue (server) In summary, special consideration should be given when choosing the MAC address-based load-balancing algorithm in an environment where the server and clients are separated by a Layer 3 device such as a router.
ports are equally used. HP recommends that the implications of this method of load balancing be carefully considered before deployment. TLB Redundancy Mechanisms The basic and advanced redundancy mechanisms used by TLB are identical to NFT. Refer to the section, “NFT Redundancy Mechanisms”, for more information. TLB Network Adapter Failure Recovery With TLB, the recovery mechanism provided is very similar to the NFT failover methods discussed previously.
• switches. This helps prevent switch uplink failure scenarios that leave team members in separate broadcast domains. TLB teams that communicate with TCP/IP network devices via a router should use the Automatic, TCP port or, IP address-based load balancing algorithm (configured via the NCU).
Figure 4-17 Overview of SLB communication ARP Table: Client A 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D IP Address = 1.1.1.2 Destination MAC/Source MAC/Destination IP/Source IP E A 1.1.1.1 1.1.1.2 N100NT5.SYS MAC = E MAC = A Destination MAC/Source MAC/Destination IP/Source IP A E 1.1.1.2 1.1.1.1 Client B E B 1.1.1.1 Destination MAC/Source MAC/Destination IP/Source IP B E 1.1.1.3 1.1.1.1 Destination MAC/Source MAC/Destination IP/Source IP N100NT5.SYS MAC = E E C 1.1.1.1 1.1.1.
As an example, a detailed discussion of Cisco’s EtherChannel load-balancing algorithm is described below. Most of the information about how Cisco switches load-balance traffic on a port trunk is applicable to other switch vendors’ Port Trunking technology. Cisco EtherChannel Cisco's Fast EtherChannel (FEC) and Gigabit EtherChannel (GEC) technology is a MAC layer (Layer 2) load-balancing technology using two or more switch ports grouped together as one logical switch port.
Table 4-14 Example of load-balancing algorithms #2 (continued) Packet 2 - Destination MAC address: 00-00-00-00-00-01 frame is transmitted out port 2 Last binary bit = 0000 0001 Method 2: SOURCE-BASED ALGORITHM Packet 1 - Source MAC address: 00-00-00-00-00-03 frame is transmitted out port 2 Last binary bit = 0000 0001 Packet 2 - Source MAC address: 00-00-00-00-00-02 frame is transmitted out port 1 Last binary bit = 0000 0000 Method 3: XOR ALGORITHM (best method to use on switch) Packet 1 - Source MAC a
SLB Redundancy Mechanisms The only redundancy mechanisms used by SLB are link loss and transmit validation heartbeats. None of the advanced redundancy mechanisms (for example, Active Path and Fast Path) are supported by SLB. The advanced redundancy mechanisms require the ability to predict and control which ports in a team will receive a particular frame. Since the switch connected to an SLB team controls how the team receives frames, the advanced redundancy mechanisms would not function reliably.
• technology has been designed to allow for flexibility. Therefore, the NCU may allow configuration of an SLB team that will not work correctly with a particular vendor’s switch. The switch’s load-balancing algorithm should be set to XOR or SOURCE-BASED but not DESTINATION-BASED (refer to the sections, “Switch-assisted Load Balancing with Fault Tolerance (SLB)” and “Cisco EtherChannel”).
Figure 4-18 TLB does not provide receive load balancing Switch NIC 1 (A) Client ARP Table: 1.1.1.1 = A PRIMARY HP Network Adapter Team Client ARP Table: 1.1.1.1 = A Switch Redundancy NIC 2 (B) SECONDARY (TLB) Client ARP Table: 1.1.1.1 = A NIC 3 (C) SECONDARY 1.1.1.1 Switch NIC 4 (D) SECONDARY HP Integrity Server • Maximum Server Bandwidth Transmit = 4000 Mbit Receive = 1000 Mbit Client ARP Table: 1.1.1.
With Dual Channel, a server can have full transmit and receive load balancing with switch redundancy (refer to Figure 4-20). Figure 4-20 Dual Channel & Dynamic Dual Channel provide full load balancing and switch redundancy Switch Client ARP Table: 1.1.1.1 = A Port Channel NIC 1 (A) Client 1 HP Network PRIMARY Adapter Team (Dual Channel) 1.1.1.1 Client ARP Table: 1.1.1.1 = A Client 2 NIC 2 (A) SECONDARY Switch Switch Redundancy Client ARP Table: Port Channel NIC 3 (B) SECONDARY 1.1.1.
Figure 4-21 Overview of Dual Channel communication ARP Table: ARP Table: Client A 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D IP Address = 1.1.1.2 Destination MAC/Source MAC/Destination IP/Source IP 1.1.1.1 Group A MAC = A 1.1.1.2 Switch 2 A Destination MAC/Source MAC/Destination IP/Source IP A E 1.1.1.2 1.1.1.1 Client B IP Address = 1.1.1.3 Destination MAC/Source MAC/Destination IP/Source IP E B 1.1.1.1 1.1.1.3 N100NT5.SYS MAC = E CPQTEAM.SYS Dual Channel Team IP Address = 1.1.1.
Figure 4-22 Dual Channel configuration Dual Channel Transmit Balancing Algorithm The transmit load-balancing algorithms used by Dual Channel are the same methods used by TLB. Refer to “Transmit Load Balancing with Fault Tolerance (TLB)” for a detailed discussion.
The client’s ARP table entries expire after a few minutes; as a result, the clients will re-ARP for the server’s IP address. The clients may or may not get the same IP address for the server depending on the results from the load-balancing algorithm. • Periodic non-cross trunk load balancing when server transmits ARP request: Periodically, the server must broadcast an ARP request for the MAC address of an IP address to which it needs to talk.
• • • Make sure that each group in a Dual Channel team is connected to a different switch. If both groups are attached to the same switch, switch redundancy is lost. Whenever a Dual Channel team is created with an even number of teamed ports, evenly distribute the ports between both groups. Ensure all ports in the same group are connected to the same switch. 802.3ad Dynamic Dual Channel Load Balancing 802.3ad Dynamic Dual Channel Load Balancing (Dynamic Dual Channel) functions exactly like Dual Channel.
Table 4-16 Team type and redundancy mechanism compatibility (continued) Receive validation heartbeats √ √ √ √ Active Path √ √ √ √ Fast Path √ √ √ √ Mechanism Priority Most team types can support multiple redundancy mechanisms simultaneously. This allows an implementer to deploy any combination of redundancy mechanisms depending on what the network environment will support.
mechanism, then NIC 2 will be chosen as the team’s Primary port in the scenario above and NIC 1 will be disabled for use by the team. If Active Path is selected as the higher priority mechanism, the result is the opposite – NIC 1 will be chosen as the team’s Primary port and NIC 2 will be disabled for use by the team.
Figure 4-25 Teamed port Information tab Listed below are the possible status conditions with definitions: • Available – The team member is functioning normally. • Not Teamed – The adapter is not part of the team. The most likely cause is adding an adapter to the team but not applying the change. • Unknown – The team member's status could not be determined. • Wire Fault – The member does not have link. • Not Joined – The member cannot be joined in the team because it has an incompatible setting.
• Failed (LACP Standby) – The team member has failed because the team has more members than the switch supports in the LACP protocol. The port is blocked by the switch. Failed (Rx Heartbeat) – The team member is not receiving frames. Failed (Tx Heartbeat) – A failure occurred while attempting to send a frame on the team member. Failed (Multiple) – The team member has multiple failed conditions. • • • Team State The NCU reports a Team State on the Information tab of each configured team.
HP Integrity Network Adapter Teaming and Advanced Networking Features When adapter ports are members of a team, some advanced features are no longer configurable on the port’s Advanced Settings tab. These advanced features are either promoted to the team’s Advanced Settings tab because the features are compatible and supported by all team members, or not promoted because one or more ports do not support the features.
feature is promoted to the team’s Advanced Settings tab with the lowest size configured for the team members. For example, if there are two adapters, one configured for 4088 bytes and the other for 9014 bytes, and they are teamed, the Maximum Frame Size feature is set to 4088 bytes. A setting of 1514 bytes is equal to Jumbo Frames being disabled. Maximum Frame Size greater than 1514 is equal to Jumbo Frames being enabled. Jumbo Frames are supported on NC Series gigabit adapters only.
4–7 HP Integrity Network Adapter Teaming Configuration and Deployment with Integrity Essentials Rapid Deployment Pack (RDP) One of the methods of deployment for HP Integrity Network Adapter Teaming is to use HP Integrity Essential’s Rapid Deployment Pack (RDP) remote scripting feature. This feature allows an SA to deploy a specific teaming configuration to any number of target servers in a single action.
5 Teaming Feature Matrix Table 5-1 Teaming feature matrix NFT & NFT with Preference TLB & TLB with Preference SLB & 802.
Table 5-1 Teaming feature matrix (continued) Can team ports of different media √ √ √ √ Load balances TCP/IP √ √ √ Load balances non-IP traffic √ √ √ Supports load balancing by destination IP address, destination MAC address, TCP port, and Round Robin √ √ √ √ √* √ √ All teamed ports within a team transmit frames with the same MAC address All ports within a team utilize the same IP address(es) on the network √ √ * Dual Channel and Dynamic Dual Channel transmit frames with the same sour
A – Overview of Network Addressing and Communication Understanding the concepts of network addressing is the key to understanding how HP’s Network Adapter Teaming works. This section provides a brief overview of network addressing as a baseline for explaining how HP’s Network Adapter Teaming can create one logical network port from a team of two or more ports. Layer 2 Versus Layer 3 Devices on a computer network use unique addresses, much like telephone numbers, to communicate with each other.
These scenarios provide a baseline of typical network addressing and communication using IP. This baseline will be referred to later in this document to differentiate how HP Integrity Network Adapter Teaming functions in these same scenarios. By understanding the differences in simple examples such as these (without HP’s Network Adapter Teaming technology involved), implementers will have a better understanding of how HP’s Network Adapter Teaming technology may work in their environment.
NOTE: This step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that corresponds to 1.1.1.1. If Blue does not find one (in other words, ARP cache timed out since last communication with Red), then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address (A). NOTE: The following step will not occur if Step 4 does not take place.
is. First, Red checks its own ARP cache for an entry that matches 1.1.1.3. If Red does not have an entry cached, then it must broadcast an ARP request frame on network 1.0.0.0 asking Green to respond and provide its MAC address. 2. Green transmits a unicast ARP reply to Red providing its MAC address (C). Green sees the ARP request and responds with a unicast ARP reply to Red. Also, Green enters Red’s MAC address and IP address into its ARP cache.
address as the source MAC address and Blue’s IP address as the source IP address. Green receives the ping reply and determines that the frame is meant for Red because of the Layer 3 address (IP). 10. Green transmits a broadcast ARP request on Network 1.0.0.0 asking for Red’s MAC address. NOTE: This step will not occur if Green’s ARP table still contains an entry for Red resulting from Steps 1 and 2. Green looks in its ARP cache for a MAC address for Red.
B – Frequently Asked Questions B.1 HP Integrity Network Adapter Teaming Frequently Asked Questions B.B.1.1 Why is traffic not being load balanced out of my server? (or) Why is traffic not being load balanced during backups? Either TLB or SLB is required for load balancing of transmit traffic. NFT will not provide for any type of load balancing.
B.B.1.9 B.B.1.10 B.B.1.11 B.B.1.12 B.B.1.13 How do I uninstall HP Integrity Network Adapter Teaming? HP Integrity Network Adapter Teaming can be uninstalled by opening the properties page of any network interface under Network and Dial-up Connections (Microsoft UI). Select HP Network Teaming and Configuration and click the UNINSTALL button. Always dissolve any existing teams using the NCU before attempting an uninstall of HP Integrity Network Adapter Teaming.
• Understanding Requirements for Failover Clusters: http://technet2.microsoft.com/windowsserver2008/en/library/ 6fe86b30-6820-43a2-8321-3e7b51cbe7631033.mspx • Network Adapter Teaming and Server Clustering: http://support.microsoft.com/kb/254101 B.B.1.15 My switch is set up for Switch-assisted Load Balancing (Port Trunking) but my network adapters are not.
switch vendor redundancy mechanisms to make a single switch highly redundant. For example, Cisco provides an option called High Availability on some switches. This option allows a Cisco switch to have redundant Supervisor modules.
C – Overview of Utilities Included with HP Integrity Network Adapter Teaming In the Component Pack (CPnnnnn.exe) for HP Integrity Network Adapter Teaming there are several command line utilities provided for special teaming-related management functions. Below is a list of the utilities with a brief description of each. Consult the latest HP Integrity Network Adapter Teaming Component Pack for the latest utilities and associated documentation. • Cqniccmd.
D – Glossary The following networking and teaming terms are used in this document: 802.1D Refers to the IEEE 802.1D specification. This is the original Spanning Tree specification. 802.3ad Dynamic Refers to the IEEE 802.3ad specification. This specification provides for manual and automatic (dynamic) port grouping for fault tolerance and load balancing between two network devices.
GEC Gigabit EtherChannel. A method of load balancing that both transmits and receives traffic across multiple Gigabit Ethernet (1000 Mbps) connections between two devices. Developed by Cisco Systems. Refer to SLB. GUI Graphical User Interface HP Teaming and Configuration GUI Now referred to as the HP Network Configuration Utility (NCU). Refer to NCU. IEEE Institute of Electrical and Electronics Engineers. A standards body for, among other things, network communications and protocols.
NFT Network Fault Tolerance. A team of network ports that transmits and receives on only one port with all other ports in standby. NIC Network Interface Card. Synonymous with network adapter. NIC teaming A phrase referring to HP Integrity Network Adapter Teaming. OSI Model Open Systems Interconnect Model. The seven-layer model developed by the International Standards Organization (ISO) that outlines the mechanisms used by networked devices to communicate with each other.
