HP FlexFabric 5900CP Solutions Configuration Guide Abstract This document describes HP FlexFabric 5900CP switch usage in converged Ethernet and storage network configurations. It provides information about switch features and storage configurations, including the device support.
© Copyright 2014, Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 HP FlexFabric 5900CP switch overview.........................................................4 Features and benefits................................................................................................................4 HP FlexFabric 5900CP network architectures................................................................................5 2 Port types and switch modes........................................................................7 Port types.................................
1 HP FlexFabric 5900CP switch overview The HP FlexFabric 5900CP switch is a multipurpose converged L2/L3 switch with 48 converged ports and four 40 GbE QSFP+ ports. This switch provides seamless integration of lossless 10GbE networks into server-edge environments and core environments through FCoE, DCB, Fibre Channel, and many other Ethernet network protocols. It is a full featured switch and a software license is not required.
• No additional feature licenses. The following four switch modes are available: ◦ Standard—Configurable DCB switch ◦ FCF—FC/FCoE initiator; target; FlexFabric SAN switch; F, VF, VE, and E port connectivity (4K zones); FSPF ◦ NPV—Gateway for FC/FCoE multi-vendor connectivity ◦ Transit—FIP-snooping DCB aggregation switch • 1.28 Tb/s switching capacity • 952.32 Million PPS throughput, integrated 9 MB packet buffer • 10GbE cut-through latency < 1.
(ISLs) (VE_Ports) to other 5900CP or 5900AF switches, the 6125XLG, or via NPV (VNP_Ports) to other HP FCoE switches that support NPIV. When configured as Fibre Channel ports, the converged ports support device connectivity (F_Port) or switch connectivity via inter-switch links (ISLs) (E_Ports) to other 5900CP switches, or via NPV (NP_Ports) to other HP Fibre Channel switches that support NPIV.
2 Port types and switch modes To support LAN and SAN environments, the HP FlexFabric 5900CP switch utilizes multiple port types and switch modes. Port types The HP FlexFabric 5900CP switch supports multiple port types. Table 1 (page 7) lists the port types supported.
FCoE • HP FlexFabric 5900CP VF_Port to a VN_Port on a node. • HP FlexFabric 5900CP (NPIV) VF_Port to a VNP_Port on an NPV switch or module. • HP FlexFabric 5900CP VE_Port to the VE_Port on another HP FlexFabric 5900CP, HP 5900AF, HP 6125XLG, or other FCF capable HP Networking switch or switches. If the primary mode of the HP FlexFabric 5900CP switch is set to FCF, the converged ports can be set to F_Port or E_Port, but cannot be set to NP_Port.
3 Storage use-cases The HP FlexFabric 5900CP switch supports several converged Ethernet/FCoE and Fibre Channel storage configurations. To simplify understanding and implementation of these configurations, a set of use-case topology designs are defined. The use-cases describe recommended ways to use the HP FlexFabric 5900CP switch, switch modes, and port types in different server-storage deployment scenarios. Some of the use-cases show multiple types of connectivity within the same configuration.
Table 2 HP FlexFabric 5900CP use-cases Major SAN Fabric Use-Case Variant 5900CP Server Connect Storage Connect Switch Mode #1 Single-Tier Fabric, Rack Servers 1A: Rack Server, FCoE Storage Rack/CNA 1B: Rack Server, FC Storage Rack/CNA/HBA Native FC #2 Single-Tier Fabric, Blade System 2A: BladeSystem Dual-hop, FCoE Storage Blade/VC Native FCoE 2B: BladeSystem FC/FCoE, FC Storage Blade/FC Native FC #3 Single-Tier Fabric, NPV Gateway 3A: Rack Server, FC NPV Gateway, FC Storage Rack/CNA/HBA FC vi
Figure 4 HP FlexFabric 5900CP FCF Use-Case 1A FCoE Storage LAN IRF VN_Port VN_Port Expand switches horizontally as required VF_Port IRF* VF_Port 5900CP FCF FCoE Converged Ethernet/FCoE Native Ethernet FCoE IRF* between 5900CP’s (optional) Server/CNA 5900CP1A Figure 5 HP FlexFabric 5900CP FCF Use-Case 1B FC Storage LAN IRF N_Port N_Port Expand switches horizontally as required F_Port IRF* F_Port 5900CP FCF FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s (optional
Figure 6 HP FlexFabric 5900CP FCF Use-Case 2A FCoE Storage LAN VN_Port IRF VN_Port Expand switches horizontally as required VF_Port VF_Port IRF* 5900CP FCF FCoE Converged Ethernet/FCoE Native Ethernet FCoE IRF* between 5900CP’s or 6125XLGs (optional) Blade Server/CNA Blade Server/CNA 5900CP2A Converged Ethernet/FCoE server connection options (Use-case 2A): 12 • Virtual Connect 20/40 F8 • Virtual Connect Flex - 10/10D, FCoE • 6125XLG, FCoE (NPV) • Virtual Connect FlexFabric 10Gb/24–Port (8G
Figure 7 HP FlexFabric 5900CP FCF Use-Case 2B FC Storage LAN IRF N_Port N_Port Expand switches horizontally as required F_Port F_Port IRF* 5900CP FCF FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s or 6125XLGs (optional) Blade Server/CNA Blade Server/CNA or HBA 5900CP2b Converged or Native FC server connection options (Use-case 2B): • Virtual Connect 20/40 F8 • Virtual Connect Flex - 10/10D, FCoE • Virtual Connect FlexFabric 10Gb/24–Port (8Gb FC, -10GbE FCoE/VC
Figure 8 HP FlexFabric 5900CP NPV Use-Case 3A FC Storage LAN N_Port IRF N_Port Fabric-A Fabric-B F_Port Expand switches horizontally as required F_Port IRF* NP_Port Fabric-A, Fabric-B HP 5900CP, HP Brocade, Cisco, or QLogic FC Switches NP_Port 5900CP NPV FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s (optional) Server/HBA Server/CNA 5900CP3A Figure 9 HP FlexFabric 5900CP NPV Use-Case 3B FC Storage LAN N_Port IRF N_Port Fabric-A Expand switches horizontally as
Converged or Native FC server connection options (Use-cases 3A and 3B): • Virtual Connect 20/40 F8 • Virtual Connect Flex - 10/10D, FCoE • Virtual Connect FlexFabric 10Gb/24–Port (8Gb FC, -10GbE FCoE/VC) • Virtual Connect 8Gb 24-Port Fibre Channel Module for c-Class BladeSystem (Brocade) • Virtual Connect 8Gb 20-port Fibre Channel Module for c-Class BladeSystem • Blade Switch: HP 6125XLG, Ethernet/FCoE (6125XLG in NPV mode) • Blade Switch: HP 8/24c SAN Switch (Brocade, Access Gateway mode) •
Figure 11 HP FlexFabric 5900CP FCF Maximum FCoE Hop Count Use-Case 4A FCoE Storage VN_Port 8 Expand switches horizontally as required.
Figure 12 HP FlexFabric 5900CP FCF Maximum FC Hop Count Use-Case 4B Expand switches horizontally as required.
Figure 13 HP FlexFabric 5900CP FCF Maximum FCoE Hop Count Use-Case 5A FCoE Storage VN_Port 8 Expand switches horizontally as required. LAN VN_Port VF_Port 7 6 5 4 VF_Port VE _Ports 3 8 VE _Ports VE_Ports 1 7 6 2 5 VE_Port 4 3 IRF* VE_Port 2 Blade Server/CNA 1 5900CP FCF Native FCoE Expand switches vertically as required, up to 8 switches (7 hops) total, including 6125XLG or VC Flex-10/10D. Each hop pair optionally IRF, except VC-Flex modules.
Figure 14 HP FlexFabric 5900CP FCF Maximum FC Hop Count Use-Case 5B FC Storage Expand switches horizontally as required.
4 IRF usage with storage The HP FlexFabric 5900CP switch supports IRF for use with Ethernet and storage networks. For Ethernet networks, you can configure up to nine HP FlexFabric 5900CP switches in a single IRF domain. For storage networks, including FCoE, you can configure up to two switches per fabric in an IRF domain. When using IRF, NIC teaming is supported and is utilized for LAN traffic.
• Must be specific on device placement to allow desired access such as CNA to storage • Allows Ethernet usage of the IRF links Considerations • Not considered a NSPOF configuration, IRF spans virtual fabrics • Firmware updates are considered disruptive to both VSANs, requiring scheduled downtime. Figure 16 Dual-redundant storage fabric configuration, 2-Switch IRF for Ethernet-only FC and FCoE Storage Fabric-AB “Virtual Switch” VSAN1 Server/CNA IRF VSAN2 Server/HBA 5900CP5B 3.
Figure 17 Dual-redundant storage fabric configuration, 4-Switch IRF for Ethernet-only FC and FCoE Storage Fabric-B “Virtual Switch” Fabric-A “Virtual Switch” VSAN1 VSAN2 VSAN3 Server/CNA 4. 5900CP7C Dual-redundant storage fabric configuration with two stacked switches in IRF domains—Implemented with two HP FlexFabric 5900CP switches in each fabric configured in two IRF domains for high availability and NSPOF. See Figure 18 (page 23).
irf-port 1/2 port group interface FortyGigE1/0/50 interface FortyGigE1/0/50 priority-flow-control auto priority-flow-control no-drop dot1p 3 irf-port 2/2 port group interface FortyGigE2/0/50 interface FortyGigE2/0/50 priority-flow-control auto priority-flow-control no-drop dot1p 3 Figure 18 Dual-redundant storage fabric configuration, two stacked switches in IRF domains FC and FCoE Storage Fabric-A “Virtual Switch” VSAN 2 VSAN 2 Server/CNA Fabric-B “Virtual Switch” VSAN 3 Server/HBA VSAN 3 5900CP
5 c-Class BladeSystem interconnects The HP FlexFabric 5900CP switch supports connectivity to multiple HP server interconnect options including Virtual Connect modules/switches and BladeSystem switches. See Table 1 (page 7). For Virtual Connect modules, Flex-10/10D and FlexFabric 10 Gb/24-port, connectivity is through FIP Snooping to the HP FlexFabric 5900CP switch in FCF mode. For BladeSystem switches, connectivity is configured based on the BladeSystem switch used.
6 HP FlexFabric 5900CP Storage Network Support The following section describes the operating system and the hardware support for the HP FlexFabric 5900CP switch when used in a storage network configuration. For more information about current HP FlexFabric 5900CP storage support, see the HP SPOCK website at http://www.hp.com/ storage/spock. You must sign up for an HP Passport to enable access.
• HP QMH2572 8Gb FC HBA • HP BLc QLogic QMH2562 8Gb FC HBA Opt • HP 82Q 8Gb Dual Port PCI-e FC HBA • HP PCIe 2-port 8Gb FC SR (QLogic) HBA • HP 81Q PCI-e FC HBA • HP PCIe 1-port 8Gb FC SR (QLogic) HBA • HP LPe1205A 8Gb FC HBA Opt • HP BLc Emulex LPe1205 8Gb FC HBA Opt • 82E 8Gb Dual-port PCI-e FC HBA • HP 81E 8Gb SP PCI-e FC HBA HP FlexFabric NPV Switch Support Converged Network Switches Connectivity to the following FCoE switches via HP FlexFabric 5900CP NPV mode is supported: • Cisco
HP FlexFabric HP Storage System Support HP FlexFabric HP Storage System Support: • 3PAR StoreServ 10400/10800 FCoE/Fibre Channel/iSCSI • 3PAR 7200/7400/7450 FCoE/Fibre Channel/iSCSI • 3PAR F-Class (F200/F400) Fibre Channel • P6300/P6350/P6500/ P6550 FCoE/Fibre Channel/iSCSI • P9500 FCoE/Fibre Channel • P2000 G3 Fibre Channel • MSA 2040 (G4) Fibre Channel • EVA 4400 (with or without embedded switch) Fibre Channel • EVA 6400/8400 Fibre Channel • StoreVirtual P4330 and 4730 FC HP FlexFabric
7 Software and Firmware update process Firmware Update Process – Non ISSU The non-ISSU firmware update process is disruptive if you have implemented a single SAN fabric or a dual SAN fabric using logically separate fabrics. See “IRF usage with storage” (page 20). If you have implemented a dual-redundant NSPOF SAN, the firmware process is considered partially disruptive, requiring fabric failover. To update the firmware using non-ISSU procedure: 1. Ensure that the latest firmware is obtained via hp.com 2.
8 Management Software The primary management interface for the HP FlexFabric 5900CP switch is through the Command Line Interface (CLI). To access the CLI, see “Using the CLI” (page 29). All the management commands necessary for HP FlexFabric 5900CP switch are available through the CLI. Management support for HP FlexFabric 5900CP switch is also provided with HP IMC (Intelligent Management Center) software.
3. 4. Configure scheme authentication for VTY login users. The default authentication scheme is password authentication. Assign a user role to VTY login users. The default role assigned is network-operator.
A HP FlexFabric 5900CP system architecture considerations Parameter name/type Parameter value Remarks Fabric logins per FCF 255 E-port utilizes one Login, reduce the number by one for each e-port connected switch. FSPF for hop by hop routing, static route support, up to 255 routes. The maximum number of static routes allowed in a VSAN is 256. VSAN's per switch 16 Virtual SANs with overlapping address space up to 16. The maximum number of VSANs, including the default VSAN, allowed on a switch is 16.
Parameter name/type Parameter value Remarks to avoid storage traffic flow over IRF links. Set the maximum number of Selected link-aggregation selected-port ports for the aggregation group. maximum number 32 By default, the maximum number of selected ports for an aggregation group is 32. Set the minimum number of Selected ports for the aggregation group. By default, the minimum number of selected ports for the aggregation group is not specified.
Parameter name/type Parameter value Remarks Upper limit of concurrent logins using access-limit max-user-number 16 the same user name recommended By default, the number of concurrent logins is not limited for the local user. This command takes effect only when local accounting is configured for the local user. It does not apply to FTP, SFTP, or SCP users who do not support accounting. Configure password control attributes Depends on environment for the local user.
B Support information The HP FlexFabric 5900CP switch provides various display capabilities. You can use the displays to identify issues by viewing configurations, port status summaries, and more in-depth information about each port. By default, the switch updates a logfile once a day. You can generate a g-zipped tar file using the display diagnostic CLI command and export this file for review. Two files are required for support: • the configuration file—startup.cfg • logfile—logfile.
In the following examples, two configuration files are named after the switch number. The config87.cfg file is the primary file representing how the switch is presently configured. dir Directory of flash: 0 -rw8215552 Mar 10 2014 15:51:10 5900_5920-cmw710-boot-r2308p01.bin 1 -rw52940800 Mar 10 2014 15:53:08 5900_5920-cmw710-system-r2308p01.bin 2 -rw22736 Jul 23 2013 20:46:13 config87-npv.cfg 3 -rw32344 Mar 11 2014 23:19:59 config87.cfg 4 -rw223240 Mar 11 2014 23:19:59 config87.
C HP FlexFabric 5900CP switch configuration file This appendix shows some sections from the HP FlexFabric 5900CP switch configuration file. Use this as a guide for creating the configuration file for your specific implementation.
zoneset activate name 5900CP_vsan100 # # vlan 1 # vlan 1001 description ToLAN # There shall be an FCoE enabled VLAN associated with each vsan configured regardless of port-type FC and Ethnernet/FCoE vlan 4001 description ToSAN-A fcoe enable vsan 100 # Definition of the FCoE and other queues qos map-table dot1p-lp import 0 export 0 import 1 export 0 import 2 export 0 import 3 export 1 import 4 export 0 import 5 export 0 import 6 export 0 import 7 export 0 # acl 4000 for FCoE - acl 3000 for iSCSI use acl n
remark dot1p 3 # qos policy DCBX classifier DCBX behavior DCBX mode dcbx # Configure Console Connectivity line class aux user-role network-admin # line class vty user-role network-operator # line aux 0 1 user-role network-admin idle-timeout 0 0 # line vty 0 15 authentication-mode scheme user-role network-admin user-role network-operator # execute set authentication password simple and it will put it in the config file similar to below.
execute password simple and it will put it in the config file similar to below. password hash $h$6$im0rLRRRYOJHIwCY$zjEOgLLx7RF5dm5GDg5h4Fc 6zJGeVYkDomKh9VKVnqq3NCW9QmsrGAlOOPWZTy/EUu tU0KLgW9HQismv1+PAOg== Setup FTP/Telnet/SSH services service-type ftp service-type ssh telnet terminal authorization-attribute user-role network-admin authorization-attribute user-role network-operator # ftp server enable # # This is the physical management port.
stp edged-port lldp tlv-enable dot1-tlv dcbx qos trust dot1p qos wrr be group 1 byte-count 15 qos wrr af1 group 1 byte-count 15 qos wrr af2 group sp qos wrr af3 group sp qos wrr af4 group sp qos wrr ef group sp qos wrr cs6 group sp qos wrr cs7 group sp qos apply policy DCBX outbound # Standard FC interface - to change the ethernet port to FC use the port-type fc command. Supported FC port modes [auto, e, f], auto is the default mode.
fc mode np port trunk vsan 100 bind interface FortyGigE 1/0/49 # To create an aggregation group follow these steps: 1. 2. 3. Create the link-aggregation interface. a. interface Bridge-Aggregation X (where X is a number) b. port link-type trunk c. port trunk permit vlan 1 102 1001 d. link-aggregation mode dynamic Associate each of the member ports. a. interface Ten-GigabitEthernet 1/0/1 b. port link-aggregation group X c. interface Ten-GigabitEthernet 1/0/2 d.
interface Ten-GigabitEthernet 1/0/2 port link-mode bridge port link-type trunk port trunk permit vlan 1 1001 4001 port trunk pvid vlan 1001 priority-flow-control auto priority-flow-control no-drop dot1p 3 lldp tlv-enable dot1-tlv dcbx qos trust dot1p qos wrr be group 1 byte-count 15 qos wrr af1 group 1 byte-count 15 qos wrr af2 group sp qos wrr af3 group sp qos wrr af4 group sp qos wrr ef group sp qos wrr cs6 group sp qos wrr cs7 group sp qos apply policy DCBX outbound port link-aggregation group 1 # 42 H
Glossary This glossary defines acronyms and terms used in this guide. It is not a comprehensive glossary of computer terms. A ACL Access Control List. A list of permissions attached to an object. An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects. Each entry in a typical ACL specifies a subject and an operation. B BB_credit Buffer-to-buffer credits.
such as PFC configuration and link bandwidth. You can configure DCB to set specific bandwidth to be allocated to packets depending on their priority values. F F_ID Fabric_Identifier. An entity consisting of one or more switches that interconnect various Nx_Ports attached to it, and capable of routing frames using only the D_ID information in an FC-2 frame header. An identifier assigned to each fabric in an inter-fabric routing environment. F_Port Fabric Port.
ISSU In-Service Software Upgrade. A comprehensive transparent software upgrade capability for network switches. IRF assisted ISSU will reboot one unit in the IRF system, wait for it to come back online, then reboot another unit in the IRF system. When a unit is rebooting, it is really down, so any host which is single-wire connected to this unit will be offline. The ISSU process assumes hosts or peer devices are dual-connected to two different IRF members.
P PFC Priority flow control. An extension of the 802.3x mechanism and defined in IEEE 802.1Qbb to define a lossless Ethernet connection. The PAUSE frame contains an 8–bit bit mask of the 802.1p priorities (specifying which traffic classes should be paused) and a timer for each priority specifying how long the traffic in that priority class should be paused.
to the same VLAN. Traffic between VLANs must be routed. LAN port VLAN membership is assigned manually on a port-by-port basis. vN_Port Virtual Node Port. vN_Ports are CNAs on node ports and FCoE storage ports. vN_Port MAC address Virtual Node Port MAC address. vN_Port MAC address is the Name_Identifier of a vN_Port. The MAC address used by an ENode for a particular address identifier during FC operation using FCoE frames.
