Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.5 Deploying a pair of S5048F-ON switches for a VxRail 4.5 cluster Abstract This document provides Dell EMC Networking OS9 switch configuration examples and topology options for a VxRail 4.5 cluster deployment.
Revisions Revisions Date Description March 2019 Changed switch node-facing interface settings from “flowcontrol rx on tx on” to “flowcontrol rx on tx off” as a best practice. August 2018 Initial release The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Revisions Table of contents Revisions.............................................................................................................................................................................2 1 2 Introduction ...................................................................................................................................................................5 1.1 Supported switches and operating systems .....................................................................
Revisions 7.1.4 Configure VLANs ..............................................................................................................................................24 7.2 Dual switch without VLT ...................................................................................................................................26 7.2.1 General settings................................................................................................................................................26 7.
Introduction 1 Introduction VxRail sits at the forefront of a fundamental shift in IT infrastructure consumption – away from applicationspecific, “build-your-own” infrastructure and toward virtualized, general-purpose, engineered systems. Dell EMC and VMware have embraced this shift with the VxRail hyper-converged appliance. VxRail has a simple, scale-out architecture that leverages VMware vSphere and VMware vSAN to provide server virtualization and software-defined storage.
Introduction 1.1 Supported switches and operating systems The examples provided in this deployment guide use VxRail 4.5 nodes connected to Dell EMC Networking S5048F-ON switches running Dell Networking OS9. Dell EMC Networking supports the following switch and OS combinations for VxRail 4.0 and later: Supported Dell EMC Networking switches and operating systems 6 Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.
Introduction 1.2 Typographical conventions The CLI and GUI examples in this document use the following conventions: 1.3 Monospace Text CLI examples Underlined Monospace Text CLI examples that wrap the page Italic Monospace Text Variables in CLI examples Bold Monospace Text Commands entered at the CLI prompt, or to highlight information in CLI output Bold text GUI fields and information entered in the GUI Attachments This document in .pdf format includes switch configuration file attachments.
Hardware overview 2 Hardware overview This section briefly describes the hardware used to validate this deployment. Appendix A contains a complete listing of hardware and software validated for this guide. 2.1 Dell EMC Networking S5048F-ON The S5048F-ON is a 1-Rack Unit (RU), multilayer switch with 48x25GbE, and 6x100GbE or 72x25GbE ports. This guide uses two S5048F-ONs as leaf switches. Dell EMC Networking S5048F-ON 2.
Hardware overview 2.3.2 Dell EMC VxRail V Series node VxRail V Series nodes are graphics-ready for uses such as high-end 2D/3D visualization. V Series nodes support up to 40 CPU cores, 1024GB memory, 3 GPUs, and 24TB hybrid or 46TB all-flash storage in a 2-RU form factor. 2.3.3 Dell EMC VxRail S Series node VxRail S Series nodes provide expanded storage capacity for collaboration, data, and analytics.
Hardware overview P, V, S, and E Series VxRail nodes also include a 1GbE BASE-T integrated Dell Remote Access Card (iDRAC) for OOB management. Note: P, V, and S Series nodes support additional network adapters in PCIe slots. See the Dell EMC VxRail Network Guide for more information. 10 Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.
Topology options 3 Topology options VxRail may be deployed using a single or dual switch topology. Using a single switch provides the lowest initial cost, but creates a single point of failure. A dual switch configuration helps ensure high availability by eliminating this single point of failure. A dual switch configuration may be used with or without VLT. Dell EMC recommends a dual switch configuration with VLT. The sections that follow explain the different options. 3.
Topology options • • • • Provides link-level resiliency Assures high availability Allows a single device to use a LAG across two upstream switches Provides Layer 2 multipathing When combined with Virtual Router Redundancy Protocol (VRRP), a dual switch topology with VLT provides more efficient Layer 3 forwarding than if VLT is not used.
Topology options 3.2 Single switch In a single switch topology, all VxRail nodes connect to a single switch. This topology is lower initial cost, but it is not recommended as the switch becomes a single point of failure. S5048F-ON 1 2 VxRail node 1 1 2 25GbE connections VxRail node 2 1 2 VxRail node n Rack 1 Single switch topology 13 Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.
VxRail in the data center 4 VxRail in the data center 4.1 Leaf-spine network Dell EMC recommends using a leaf-spine network in the data center with leaf switches configured as VLT peers. The switches and VxRail nodes covered in this guide are shown in Rack 1 in Figure 10, and are incorporated into a data center’s leaf-spine network. Spine1 Spine2 Leaf-2A Leaf-2B L3 L2 L3 S5048F-Leaf1B S5048F-Leaf1A VLTi Leaf-nA Leaf-nB VLTi VLTi Additional VxRail clusters, servers, storage, etc.
VxRail in the data center 4.2 OOB management network The out-of-band (OOB) management network is an isolated network for remote management of servers, switches, and storage devices via their dedicated hardware management ports. It is also commonly used to carry heartbeat messages sent between switches configured as VLT peers.
Configuration planning 5 Configuration planning 5.1 VLANs and IP addresses VLANs and IP addresses used for VxRail node traffic must be planned before switch configuration and VxRail deployment can begin. VxRail node traffic is divided into four or more VLANs as shown in Table 1.
Configuration planning 5.2 VxRail network configuration table Information provided in the VxRail network configuration table is used during VxRail deployment. The values used for this deployment example are shown in the right column. The VLANs and IP addresses used are based on the information from Table 2. VxRail network configuration table Row 1 VxRail 2 3 System Category Description Values used Management VLAN ID The recommended is untagged traffic on the Native VLAN.
Configuration planning Row Description Values used 39 Subnet mask 255.255.255.0 40 VLAN ID 1613 VM Network name and VLAN ID VM_Network_A, 1614 VM Network name and VLAN ID vRealize Log Insight™ hostname VM_Network_B, 1615 vxinsight 44 vRealize Log Insight IP address 172.16.11.103 45 Syslog server (instead of Log Insight) 41 42 43 Category VM Networks … (unlimited number) Solutions Logging This deployment uses four VxRail nodes.
Configuration planning An NTP server is not required but is recommended. If an NTP server is not provided, VxRail uses the time that is set on ESXi host #1. Note: For this deployment guide, the PowerEdge server used as the jump box also provides DNS and NTP services for the VxRail cluster. It is connected to both leaf switches for redundancy using Microsoft Windows NIC teaming (switch-independent load balancing). 19 Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.
Switch configuration prerequisites 6 Switch configuration prerequisites 6.1 Check switch OS version S5048F-ON switches must be running OS version 9.12(1.0) or later. Run the show version command to check the OS version. Dell EMC recommends upgrading to the latest release available on the Dell Networking Customer Support Portal (account required). Dell#show version Dell EMC Real Time Operating System Software Dell EMC Operating System Version: 2.0 Dell EMC Application Software Version: 9.12(1.
Configure switches 7 Configure switches This section covers switch configuration for S5048F-ON switches in the different topologies. Commands for all examples are provided as attachments to this guide. 7.1 Dual switch with VLT This example uses a four-node VxRail cluster connected to a pair of switches configured with VLT as shown below. Dell EMC recommends using this topology.
Configure switches General settings – dual switch with VLT 7.1.2 S5048F-Leaf1A S5048F-Leaf1B enable configure terminal enable configure terminal hostname S5048F-Leaf1A hostname S5048F-Leaf1B interface ManagementEthernet 1/1 ip address 100.67.172.36/24 no shutdown management route 0.0.0.0/0 100.67.172.254 interface ManagementEthernet 1/1 ip address 100.67.172.35/24 no shutdown management route 0.0.0.0/0 100.67.172.254 ntp server 100.67.10.20 ntp server 100.67.10.
Configure switches 7.1.
Configure switches Configure VLT – dual switch with VLT 7.1.
Configure switches 25 S5048F-Leaf1A S5048F-Leaf1B interface Vlan 1612 description vMotion no ip address tagged twentyFiveGigE 1/17-1/20 no shutdown interface Vlan 1612 description vMotion no ip address tagged twentyFiveGigE 1/17-1/20 no shutdown interface Vlan 1613 description vSAN no ip address tagged twentyFiveGigE 1/17-1/20 no shutdown interface Vlan 1613 description vSAN no ip address tagged twentyFiveGigE 1/17-1/20 no shutdown interface Vlan 1614 description VM_Network_A no ip address tagged tw
Configure switches 7.2 Dual switch without VLT This example uses a four-node VxRail cluster connected to a switch pair without VLT as shown below. S5048F-Leaf1A S5048F-Leaf1B 1/20 1/19 1/18 1/17 1/17 1/18 1/19 1/20 LACP 25GbE connections VxRail node 1 100GbE LACP connections, ports 1/53-54 VxRail node 2 VxRail node 3 The jump box/DNS server, not shown, is connected to port 1/9 on each switch.
Configure switches 7.2.2 S5048F-Leaf1A S5048F-Leaf1B interface ManagementEthernet 1/1 ip address 100.67.172.36/24 no shutdown management route 0.0.0.0/0 100.67.172.254 interface ManagementEthernet 1/1 ip address 100.67.172.35/24 no shutdown management route 0.0.0.0/0 100.67.172.254 ntp server 100.67.10.20 ntp server 100.67.10.
Configure switches 7.2.
Configure switches 7.2.4 S5048F-Leaf1A S5048F-Leaf1B interface Port-channel 127 description To Leaf1B no ip address portmode hybrid switchport no shutdown interface Port-channel 127 description To Leaf1A no ip address portmode hybrid switchport no shutdown Configure VLANs In this section, VLANs are configured per Table 2. Assign an IP address to the in-band management VLAN. Add the interfaces connected to the VxRail nodes and jump box as untagged members of the in-band management VLAN.
Configure switches 7.
Configure switches 7.3.1 General settings Configure the hostname, OOB management IP address and default gateway. Specify an NTP server accessible by the switch. Enable LLDP for troubleshooting. Enable RSTP as a precaution against loops. General settings – single switch S5048F-ON enable configure terminal hostname S5048F-ON interface ManagementEthernet 1/1 ip address 100.67.172.36/24 no shutdown management route 0.0.0.0/0 100.67.172.254 ntp server 100.67.10.
Configure switches S5048F-ON portmode hybrid switchport spanning-tree rstp edge-port no shutdown interface twentyFiveGigE 1/10 description JumpBox_DNS no ip address portmode hybrid switchport spanning-tree rstp edge-port no shutdown interface twentyFiveGigE 1/17 description Node1_Port1 no ip address portmode hybrid switchport spanning-tree rstp edge-port flowcontrol rx on tx off no shutdown interface twentyFiveGigE 1/18 description Node1_Port2 no ip address portmode hybrid switchport spanning-tree rstp edg
Configure switches S5048F-ON no shutdown interface twentyFiveGigE 1/22 description Node3_Port2 no ip address portmode hybrid switchport spanning-tree rstp edge-port flowcontrol rx on tx off no shutdown interface twentyFiveGigE 1/23 description Node4_Port1 no ip address portmode hybrid switchport spanning-tree rstp edge-port flowcontrol rx on tx off no shutdown interface twentyFiveGigE 1/24 description Node4_Port2 no ip address portmode hybrid switchport spanning-tree rstp edge-port flowcontrol rx on tx off
Configure switches Configure VLANs – single switch S5048F-ON interface Vlan 1611 description InBand_Mgmt ip address 172.16.11.
Switch validation 8 Switch validation After switches are configured and devices are connected, the switch CLI is used to validate the network configuration. This section provides a list of the most common commands and their output for the examples used in this guide. 8.1 General validation commands The commands and output shown below are for S5048F-Leaf1A in the dual switch with VLT topology. The output of its peer, S5048F-Leaf1B, is similar.
Switch validation Note: The command output for the dual switch without VLT topology is shown below. In this example, LAG 127 is the LACP port channel used as the switch interconnect. S5048F-Leaf1A#show interfaces port-channel brief Codes: L - LACP Port-channel O - OpenFlow Controller Port-channel A - Auto Port-channel I - Internally Lagged L 8.1.
Switch validation Note: For the dual switch without VLT topology, the VLAN portion of the command output is as follows: * NUM 1 1611 Status Inactive Active Description Q Ports InBand_Mgmt 1612 Active vMotion 1613 Active vSAN 1614 Active VM_Network_A 1615 Active VM_Network_B U U T T T T T T T T Po127(Hu 1/53,1/54) Tf 1/9,1/17-1/20 Po127(Hu 1/53,1/54) Tf 1/17-1/20 Po127(Hu 1/53,1/54) Tf 1/17-1/20 Po127(Hu 1/53,1/54) Tf 1/17-1/20 Po127(Hu 1/53,1/54) Tf 1/17-1/20 For the single switch topo
Switch validation 8.1.5 show vrrp brief In this deployment, VRRP is configured on the in-band management VLAN, 1611, to provide gateway redundancy for management traffic. The output from the show vrrp brief command shows the local and virtual IP addresses and whether the switch is in the master or backup role. The switch configured with the largest priority value, shown in the Pri column, becomes the master.
Switch validation 8.2.2 show vlt backup-link This command is used to verify VLT peers are communicating on the backup link over the OOB management network. The Destination is the management IP address of the peer. The Peer HeartBeat status must be Up. S5048F-Leaf1A#show vlt backup-link VLT Backup Link ----------------Destination: 100.67.172.
Deploy VxRail 9 Deploy VxRail A laptop computer or jump box with a web browser for the VxRail user interface is required. It is either plugged into a leaf switch or able to logically reach the VxRail in-band management VLAN from elsewhere on the network. By default, the initial VxRail Manager IP address is 192.168.10.200/24. After initial configuration, the address changes to its new address on the in-band management network. The new VxRail Manager address used in this guide is 172.16.11.
Validated components A Validated components The following tables include the hardware, software, and firmware used to configure and validate the examples in this guide. A.1 Dell EMC Networking Switches Switches and OS versions A.2 Qty Item OS Version 2 Dell EMC Networking S5048F-ON leaf switches 9.12.1.0 1 Dell EMC Networking S3048-ON management switch 9.13.0.3 VxRail P570 nodes A cluster of four VxRail P570 nodes was used to validate the examples in this guide.
Validated components A.3 VxRail Appliance software This deployment guide was developed using VxRail Appliance software 4.5.200. It consists of the component versions shown in Table 18. VxRail Appliance software component versions 42 Item Version VxRail Manager 4.5.200.7769314 VMware ESXi 6.5.0 Update 1 Patch 36, build 7388607 VMware vCenter Server Appliance 6.5 Update 1e, build 7515524 VMware vSAN 6.6.1 Patch 02 VMware vRealize Log Insight 4.
VxRail network adapter traffic optimization B VxRail network adapter traffic optimization VxRail optimizes network traffic by splitting traffic across rNDC uplinks and by using NIOC shares. Based on traffic type, uplinks are configured as active or standby, and NIOC shares are used to allocate network bandwidth. Uplink load balancing and NIOC shares are automatically configured by VxRail.
Technical resources C Technical resources Dell TechCenter is an online technical community where IT professionals have access to numerous resources for Dell EMC software, hardware, and services.
Support and feedback D Support and feedback Contacting Technical Support Support Contact Information Web: http://www.dell.com/support Telephone: USA: 1-800-945-3355 Feedback for this document We encourage readers to provide feedback on the quality and usefulness of this publication by sending an email to Dell_Networking_Solutions@Dell.com. 45 Dell EMC Networking OS9 Switch Configuration Guide for VxRail 4.
