HP FlexFabric 5900CP Solutions Configuration Guide Abstract This document describes HP FlexFabric 5900CP switch usage in converged Ethernet and storage network configurations. It provides information about switch features and storage configurations, including the device support.
© Copyright 2014, Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 HP FlexFabric 5900CP switch overview.........................................................4 Features and benefits................................................................................................................4 HP FlexFabric 5900CP network architectures................................................................................5 2 Port types and switch modes........................................................................6 Port types.................................
1 HP FlexFabric 5900CP switch overview The HP FlexFabric 5900CP switch is a ToR (Top of Rack) converged L2/L3 switch with 48 converged ports and four 40GbE QSFP ports. This switch provides seamless integration of 10GbE networks into existing server-edge environments, such as FCoE DCB and Fibre Channel. It is a full featured switch and a software license is not required.
◦ NPV—Gateway for FC/FCoE multi-vendor connectivity ◦ Transit—FIP-snooping DCB aggregation switch • 1.28 Tb/s switching capacity • 952.32 Million PPS throughput, integrated 9 MB packet buffer • 10GbE cut-through latency < 1.5 µs (64Byte packets) • CLI and iMC/VAN/VFM (Intelligent Management Center) fabric management • HP IIIAS (Intelligent Infrastructure Analyzer Software) support • L2/L3, IPv4/IPv6 dual stack, TRILL, VEPA • Cloud and SDN ready (OpenFlow 1.3.
2 Port types and switch modes To support LAN and SAN environments, the HP FlexFabric 5900CP switch utilizes multiple port types and switch modes. Port types The HP FlexFabric 5900CP switch supports multiple port types. Table 1 (page 6) lists the port types supported.
3. NPV/Gateway Mode—When a switch operates in this mode, it is called as an NPV (N_Port Virtualizer) switch. In this mode, the following HP FlexFabric 5900CP converged port connections are available: Fibre Channel • HP FlexFabric 5900CP F_Port to an N_Port on a node. • HP FlexFabric 5900CP NP_Port to an F_Port on an FCF (NPIV) switch. FCoE • HP FlexFabric 5900CP VF_Port to a VN_Port on a node. • HP FlexFabric 5900CP VNP_Port to a VF_Port on an FCF (NPIV) switch.
3 Storage use-cases The HP FlexFabric 5900CP switch supports several converged Ethernet/FCoE and Fibre Channel storage configurations. To simplify understanding and implementation of these configurations, a set of use-case topology designs are defined. The use-cases describe recommended ways to use the HP FlexFabric 5900CP switch, switch modes, and port types in different server-storage deployment scenarios. Some of the use-cases show multiple types of connectivity within the same configuration.
Table 2 HP FlexFabric 5900CP use-cases Major SAN Fabric Use-Case Variant 5900CP Server Connect Storage Connect Switch Mode #1 Single-Tier Fabric, Rack Servers 1A: Rack Server, FCoE Storage Rack/CNA 1B: Rack Server, FC Storage Rack/CNA/HBA Native FC #2 Single-Tier Fabric, Blade System 2A: BladeSystem Dual-hop, FCoE Storage Blade/VC Native FCoE 2B: BladeSystem FC/FCoE, FC Storage Blade/FC Native FC #3 Single-Tier Fabric, NPV Gateway 3A: Rack Server, FC NPV Gateway, FC Storage Rack/CNA/HBA FC vi
Figure 5 HP FlexFabric 5900CP FCF Use-Case 1B FC Storage LAN IRF N_Port N_Port Expand switches horizontally as required F_Port IRF* F_Port 5900CP FCF FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s (optional) Server/CNA Server/HBA 5900CP1b Figure 6 HP FlexFabric 5900CP FCF Use-Case 2A FCoE Storage LAN VN_Port IRF VN_Port Expand switches horizontally as required VF_Port IRF* VF_Port 5900CP FCF FCoE Converged Ethernet/FCoE Native Ethernet FCoE IRF* between 590
Converged Ethernet/FCoE server connection options: • Virtual Connect Flex - 10/10D, FCoE • 6125XLG, FCoE (NPV) • Virtual Connect FlexFabric 10Gb/24–Port (8Gb FC, 10GbE FCoE/Ethernet) Figure 7 HP FlexFabric 5900CP FCF Use-Case 2B FC Storage LAN IRF N_Port N_Port Expand switches horizontally as required F_Port IRF* F_Port 5900CP FCF FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s (optional) 5900CP2b Converged or Native FC server connection options: • Virtual Con
Figure 8 HP FlexFabric 5900CP NPV Use-Case 3A FC Storage LAN N_Port IRF N_Port Fabric-A Fabric-B F_Port Expand switches horizontally as required NP_Port Fabric-A, Fabric-B HP 5900CP, HP Brocade, Cisco, or QLogic FC Switches F_Port IRF* NP_Port 5900CP NPV FCoE/FC Converged Ethernet/FCoE Native FC Native Ethernet IRF* between 5900CP’s (optional) Server/HBA Server/CNA 5900CP3A Figure 9 HP FlexFabric 5900CP NPV Use-Case 3B FC Storage LAN N_Port IRF N_Port Fabric-A Expand switches horizontally as
Converged or Native FC server connection options: • Virtual Connect Flex - 10/10D, FCoE • Virtual Connect FlexFabric 10Gb/24–Port (8Gb FC, -10GbE FCoE/VC) • Virtual Connect 8Gb 24-Port Fibre Channel Module for c-Class BladeSystem (Brocade) • Blade Switch: HP 6125XLG, Ethernet/FCoE (6125XLG in NPV mode) • Blade Switch: HP 8/24c SAN Switch (Brocade) • Blade Switch: MDS 8Gb 24-port Switch (Cisco) Figure 10 HP FlexFabric 5900CP NPV Use-Case 3C FC Storage LAN N_Port Cisco Nexus 55xx FCoE/FC N_Port
4 IRF usage with storage The HP FlexFabric 5900CP switch supports IRF for use with Ethernet and storage networks. For Ethernet networks, you can configure up to nine HP FlexFabric 5900CP switches in a single IRF domain. For storage networks, including FCoE, you can configure up to two switches per fabric in an IRF domain. When using IRF, NIC teaming is supported and is utilized for LAN traffic.
Figure 12 Dual-redundant storage fabric configuration, 2-Switch IRF for Ethernet-only FCoE Storage Fabric-AB “Virtual Switch” VSAN1 IRF VSAN2 5900CP5A 3. Dual-redundant fabric SAN with 4-switch IRF for Ethernet—Implemented with two HP FlexFabric 5900CP switches in each fabric configured in two IRF domains for high availability and NSPOF. See Figure 13 (page 16).
Figure 13 Dual-redundant storage fabric configuration, 4-Switch IRF for Ethernet-only FCoE Storage Fabric-A “Virtual Switch” VSAN1 VSAN2 Fabric-B “Virtual Switch” VSAN3 VSAN4 5900CP7A 16 IRF usage with storage
5 c-Class BladeSystem interconnects The HP FlexFabric 5900CP switch supports connectivity to multiple HP server interconnect options including Virtual Connect modules/switches and BladeSystem switches. See Table 1 (page 6). For Virtual Connect modules, Flex-10/10D and FlexFabric 10 Gb/24-port, connectivity is through FIP Snooping to the HP FlexFabric 5900CP switch in FCF mode. For BladeSystem switches, connectivity is configured based on the BladeSystem switch used.
6 HP FlexFabric 5900CP Storage Network Support The following section describes the operating system and the hardware support for the HP FlexFabric 5900CP switch when used in a storage network configuration. For more information about current HP FlexFabric 5900CP storage support, see the HP SPOCK website at http://www.hp.com/ storage/spock. You must sign up for an HP Passport to enable access.
HP FlexFabric NPV Switch Support Converged Network Switches Connectivity to the following FCoE switches via HP FlexFabric 5900CP NPV mode is supported: • Cisco Nexus 5548UP/5596UP FCoE Converged Network Switch • Cisco Nexus 5010/5020 FCoE Converged Network Switch NOTE: Note: Merged fabric interoperability (VE_Port) is not supported between HP FlexFabric 5900CP switch and other FCoE switches.
7 Software and Firmware update process (Non-ISSU and ISSU) Firmware Update Process – Non ISSU To update the firmware using non-ISSU procedure: 1. Ensure that the latest firmware is obtained via hp.com 2. Use the ftp client on the switch to download a. ftp then login to the server. b. bin c. get d. boot-loader file flash:/ slot 1 main The following message appears: This command will set the main startup software images.
8 Management Software The primary management interface for the HP FlexFabric 5900CP switch is through the Command Line Interface (CLI). To access the CLI, see “Using the CLI” (page 21). All the management commands necessary for HP FlexFabric 5900CP switch are available through the CLI. Management support for HP FlexFabric 5900CP switch is also provided with HP IMC (Intelligent Management Center) software.
3. 4. Configure scheme authentication for VTY login users. The default authentication scheme is password authentication. Assign a user role to VTY login users. The default role assigned is network-operator.
A HP FlexFabric 5900CP system architecture considerations Parameter name/type Parameter value Remarks Fabric logins per FCF 255 E-port utilizes one Login, reduce the number by one for each e-port connected switch. FSPF for hop by hop routing, static route support, up to 255 routes. The maximum number of static routes allowed in a VSAN is 256. VSAN's per switch 16 Virtual SANs with overlapping address space up to 16. The maximum number of VSANs, including the default VSAN, allowed on a switch is 16.
Parameter name/type Parameter value Remarks to avoid storage traffic flow over IRF links. Set the maximum number of Selected link-aggregation selected-port ports for the aggregation group. maximum number 32 By default, the maximum number of selected ports for an aggregation group is 32. Set the minimum number of Selected ports for the aggregation group. By default, the minimum number of selected ports for the aggregation group is not specified.
Parameter name/type Parameter value Remarks Upper limit of concurrent logins using access-limit max-user-number 16 the same user name recommended By default, the number of concurrent logins is not limited for the local user. This command takes effect only when local accounting is configured for the local user. It does not apply to FTP, SFTP, or SCP users who do not support accounting. Configure password control attributes Depends on environment for the local user.
B Support information The HP FlexFabric 5900CP switch provides various display capabilities. You can use the displays to identify issues by viewing configurations, port status summaries, and more in-depth information about each port. By default, the switch updates a logfile once a day. You can generate a g-zipped tar file using the display diagnostic CLI command and export this file for review. Two files are required for support: • the configuration file—startup.cfg • logfile—logfile.
dir Directory of flash:/logfile 0 -rw10483710 Mar 14 2014 15:36:30 524288 KB total (330800 KB free) The flash:/startup.cfg or your logfile.log [HP]display logfile summary Log file: Enabled Log file size quota: 10 MB Log file directory: flash:/logfile Writing frequency: 24 hour 0 min 0 sec [HP]display diagnostic Save or display diagnostic information (Y=save, N=display)? [Y/N]:y Please input the file name(*.tar.gz)[flash:/diag.tar.
C HP FlexFabric 5900CP switch configuration file This appendix shows some sections from the HP FlexFabric 5900CP switch configuration file. Use this as a guide for creating the configuration file for your specific implementation.
zoneset activate name 5900CP_vsan100 # # vlan 1 # vlan 1001 description ToLAN # # 25 The vsan defined is being tied to the vlan vlan 4001 description ToSAN-A fcoe enable vsan 100 # ##################################################### # ############################### Definition of the FCoE queue qos map-table dot1p-lp import 0 export 0 import 1 export 0 import 2 export 0 import 3 export 1 import 4 export 0 import 5 export 0 import 6 export 0 import 7 export 0 # ############################### # #########
acl number 3000 rule 0 permit tcp destination-port eq 3260 ################################################## # ######################################### Configure DCBx traffic behavior DCBX remark dot1p 3 # qos policy DCBX classifier DCBX behavior DCBX mode dcbx # ######################################### # Configure Console Connectivity line class aux user-role network-admin # line class vty user-role network-operator # line aux 0 1 user-role network-admin idle-timeout 0 0 # line vty 0 15 authentication
LLDP management address - used for VC connectivity Each management interface must be in its own subnet interface Vlan-interface1 ip address 10.10.10.1 255.255.255.0 # # domain system # domain default enable system # # user-group system # local-user admin class manage execute password simple and it will put it in the config file similar to below.
##################################################### Settings for typical FCoE port If setting up multiple ports simultaneously use the range command Ex.
port trunk vsan 100 bind interface Ten-GigabitEthernet1/0/1 # ############################################ # ####################################################### MAC binding when devices coming from a FIP snooping or standard DCB device interface Vfc200 port trunk vsan 100 bind interface Bridge-Aggregation1 mac 6c3b-e5af-ad09 # ####################################################### # ############################################# Setup a Vfc to uplink from NPV switch to FCF or NPV switch interface vfc1
#To create an aggregation group follow these steps: # 1. Create the link-aggregation interface # a)interface Bridge-Aggregation X (where X is a number) # b) port link-type trunk # c) port trunk permit vlan 1 102 1001 # d) port trunk pvid vlan 1001 # e) link-aggregation mode dynamic # 2. Associate each of the member ports # a)interface Ten-GigabitEthernet 1/0/1 # b)port link-aggregation group X # c)interface Ten-GigabitEthernet 1/0/2 # d)port link-aggregation group X # 3.
lldp tlv-enable dot1-tlv dcbx qos trust dot1p qos wrr be group 1 byte-count 15 qos wrr af1 group 1 byte-count 15 qos wrr af2 group sp qos wrr af3 group sp qos wrr af4 group sp qos wrr ef group sp qos wrr cs6 group sp qos wrr cs7 group sp qos apply policy DCBX outbound port link-aggregation group 1 # interface Ten-GigabitEthernet 1/0/2 port link-mode bridge port link-type trunk port trunk permit vlan 1 1001 4001 port trunk pvid vlan 1001 priority-flow-control auto priority-flow-control no-drop dot1p 3 lldp t
