Dell EMC Networking OS10 Enterprise Edition Deployment Guide with Cisco ACI Abstract This document provides steps for the configuration and deployment of Dell EMC Networking switches running Dell EMC OS10 Enterprise Edition, into a Cisco ACI environment.
Revisions Date Description December 2018 Initial release The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. © 2018 Dell Inc. or its subsidiaries. All Rights Reserved.
Table of contents Revisions............................................................................................................................................................................. 2 1 2 Introduction ................................................................................................................................................................... 5 1.1 Typographical conventions .............................................................................................
9.3 A B 4 Verify connectivity between VMs ......................................................................................................................30 Additional information .................................................................................................................................................31 A.1 Reset OS10EE switches to factory defaults .....................................................................................................31 A.
1 Introduction Dell EMC Networking is committed to providing customers with modern data center networking technology to be the foundation for digital transformation. Customers can choose from a wide range of industry-standard network applications, operating systems, and hardware platforms to realize cost savings and improvement in service agility.
1.2 Attachments This document in .pdf format includes one or more file attachments. To access attachments in Adobe Acrobat Reader, click the icon in the left pane halfway down the page, and then click the icon.
2 Hardware overview This section briefly describes the hardware that is used to validate the deployment examples in this document. Appendix B contains a complete listing of hardware and software that is validated for this guide.
2.4 Cisco Nexus C93180YC-EX The Cisco Nexus C93180YC-EX switch is a 1-RU switch with forty-eight 1/10/25GbE ports and six 40/100GbE ports. A pair of Cisco Nexus C93180YC-EX switches are used as Cisco ACI leaf switches in the demonstrated environment in this guide. 2.5 Cisco Nexus C9336-PQ The Cisco Nexus C9336-PQ switch is a 2-RU switch with thirty-six 40GbE QSFP+ ports. One Cisco Nexus C9336-PQ switch is used as a Cisco ACI spine switch in the demonstrated environment in this guide. 2.
3 Cisco Application Centric Infrastructure (ACI) Cisco ACI is an application focused, software-defined networking solution utilizing both software and traditional switching hardware. The solution is an overlay on Cisco’s high-performance switches, operating in an ACI mode managed by a controller. The Cisco Application Policy Infrastructure Controller (APIC) is a central management appliance that handles policy, visibility, security, and overall network control for the ACI environment.
4 Dell EMC ToR switches with Cisco ACI environment overview The example that is shown in this paper covers deploying S5248F-ON switches connected to a Cisco ACI environment. By integrating the ToR switch pair into an ACI environment, compute resources within the rack can use ACI gateways and access ACI resources. The validated Cisco ACI environment includes a pair of Nexus C93180YC-EX switches as leaf switches as shown in Figure 5.
4.1 Validated environment In this example, two S5248F-ON switches are joined to an existing Cisco ACI environment. The rack contains two S5248F-ON and four PowerEdge R730xd servers. The connections between the ACI environment and the S5248F-ON switches consist of double-sided multichassis link aggregation group (MLAG). The double-sided MLAG connection consists of a vPC on the Cisco ACI side and a VLT port channel on the S5248F-ON side.
The existing Cisco ACI environment has two PowerEdge R730xd rack servers that are directly connected to the ACI leaf switches. These rack servers are in a VMware vSphere cluster, with a vCenter VM named mgmtvc01 on R730xd-01 as shown in Figure 6. Integrating the S5248F-ON switches into the Cisco ACI environment enables the four PowerEdge R730xd servers in the rack to join the existing VMware vSphere cluster. This enables all hosts and VMs to communicate using the relevant networks.
5 OOB Management network Network topological designs are not complete without a layer for management traffic. The OOB management network is a separate network for management traffic only. Administrators use the OOB management network to configure, manage, and monitor devices such as servers and switches. Payload traffic that is initiated by network end users does not traverse the OOB management network. Switches used for management are 1GbE.
Figure 8 shows how the OOB management network was configured for the demonstrated environment, alongside the connections used for in-band production traffic. There are two OOB management connections for each server: one that is used for ESXi management and one that is used for providing connectivity to the server iDRAC. The production in-band uplinks to the ACI environment are not shown.
6 Cisco APIC configuration The Cisco APIC configuration includes the ports that are connected to the R730xd rack servers and the vPC that connects to the S5248F-ON VLT port channel. Included are the configuration of the ACI fabric interfaces, switches, and application-level elements such as ACI endpoint groups (EPGs) and bridge domains.
APIC configuration steps that are used in the validated environment are provided in the attachment titled OS10EE Deployment Guide - Cisco APIC configuration steps. See the Cisco ACI documentation for detailed APIC configuration instructions. Note: In this environment, the 100 GbE ports on the ACI leaf switches are used as downlinks to the S5248FON switches. By default, the 100 GbE switch ports are designated for fabric connections, such as connections to ACI spine switches.
7 Configure S5248F-ON switches The following section outlines the configuration commands that are issued to the Dell EMC Networking S5248F-ON ToR switches. 7.1 OOB management configuration The OOB port for the Dell EMC Networking S5248F-ON is mgmt 1/1/1. The following commands show how to configure this OOB port when using OS10EE. The switches start at their factory default settings as described in Appendix A.1. Note: The S5248F-ON CLI is accessible through the console port or by using SSH.
VLT configuration 7.3 S5248F-1 S5248F-2 interface range ethernet 1/1/53-1/1/54 description VLTi no shutdown no switchport interface range ethernet 1/1/53-1/1/54 description VLTi no shutdown no switchport vlt-domain 127 backup destination 100.67.166.32 discovery-interface ethernet 1/1/531/1/54 vlt-domain 127 backup destination 100.67.166.33 discovery-interface ethernet 1/1/531/1/54 Configure the VLANs The VLAN settings that are used during deployment for this environment are shown in Table 8.
VLAN configuration 7.
7.5 Switch pair uplink configuration Configure the VLT port channel and interfaces connecting to the ACI leaf switches. Configure and enable uplink failure detection (UFD).
8 vCenter configuration overview The existing ACI environment has two PowerEdge R730xd rack servers that are connected to the ACI leaf switches. These servers are in a vSphere cluster named Management. After the Dell EMC Networking switches are deployed, servers in the Dell EMC ToR environment can communicate with the vCenter and other servers that are located in the ACI environment. The servers are joined to the vSphere cluster by an administrator as shown in Figure 10.
A VDS named VDS-Mgmt, along with five distributed port groups, one for each VLAN, are used as shown in Figure 11. VDS and port groups that are used in the validated environment Note: For each port group in the VDS in this example, both uplinks are active and the load balancing method that is used is Route based on physical NIC load as recommended in VMware Validated Design Documentation.
9 Verify configuration This section covers methods to verify the Dell EMC ToR and ACI environment is configured properly. 9.1 Validation using OS10EE CLI The CLI commands that are shown in this section are available to help validate the configuration. The commands and output that are shown below are from an 5248F-ON switch. The CLI output from the second S5248F-ON, not shown, is similar. Note: The S5248F-ON CLI is accessible using SSH. The default username and password are both admin. 9.1.
9.1.2 show vlt [domain-id] vlt-port-detail command The show vlt domain-id vlt-port-detail command shows the VLT port channel status for both VLT peers. The VLT in this example is connected to the Cisco ACI vPC. It is automatically configured in port channel 100, and it consists of two ports on each switch.
9.1.5 show spanning-tree brief command The show spanning-tree brief command validates that STP is enabled on the leaf switches. All interfaces are forwarding (FWD), as shown in the Sts column. S5248F-1# show spanning-tree brief Spanning tree enabled protocol rstp with force-version rstp Executing IEEE compatible Spanning Tree Protocol Root ID Priority 4096, Address 54bf.64be.f540 Root Bridge hello time 2, max age 20, forward delay 15 Bridge ID Priority 4096, Address 54bf.64be.
9.2 Cisco ACI validation 9.2.1 Verify vPC configuration Verify the vPC connection from the Cisco ACI fabric to the S5248F-ON VLT, shown in Figure 6, is up and properly configured to enable designated VLANs and EPGs. This is done as follows: 1. In the APIC GUI, go to Fabric > Inventory > Pod name > Leaf name > Interfaces > vPC Interfaces and navigate to the applicable port channel/vPC policy group as shown in Figure 12. Cisco ACI vPC port channel and interfaces 2.
4. With the port channel/vPC interface policy group selected in the left pane, click VLANs at the top of the right pane as shown in Figure 13. Cisco ACI vPC port channel VLANs and EPGs 5. Verify that the port channel includes all required VLANs, and EPGs are mapped to the correct VLANs. Repeat steps 1 through 5 for the remaining leaf switch.
9.2.2 Verify physical interface configuration The physical, host-connected, interfaces in the validated environment are those connected directly to the PowerEdge R730xd servers as shown in Figure 6. Verify the physical interfaces from the Cisco ACI fabric to the servers are up and properly configured to enable designated VLANs and EPGs. This configuration is done as follows: 1. In the APIC GUI, go to Fabric > Inventory > Pod 1 > Leaf name > Interfaces > Physical Interfaces as shown in Figure 14.
4. Verify the interface includes all required VLANs and EPGs. Repeat for remaining interfaces as needed. Repeat steps 1 through 4 for the remaining leaf switch. 9.2.3 Verify ACI is learning endpoints To verify that ACI is learning endpoints, do the following: 1. In the APIC GUI, go to Tenants > Tenant name > Application Profiles > Application Profile name > Application EPGs > select an Application EPG. 2. Click Operational at the top of the right pane as shown in Figure 16.
9.3 Verify connectivity between VMs In ACI, by default, communication flows freely within EPGs, but not between EPGs. To enable inter-EPG communication, contracts are configured on the APIC. This example is configured for unrestricted inter-EPG communication as shown in steps 17 through 19 in the attachment titled OS10EE Deployment Guide Cisco APIC configuration steps. Connectivity is verified by pinging between the VMs shown in Figure 6.
A Additional information A.1 Reset OS10EE switches to factory defaults To reset OS10EE switches back to the factory default configuration, enter the following commands: OS10# delete startup-configuration Proceed to delete startup-configuration [yes/no(default)]:yes OS10# reload System configuration has been modified. Save? [yes/no]:no Proceed to reboot the system? [confirm yes/no]:yes The switch reboots with default configuration settings. A.
B Validated components B.1 Dell EMC Networking switches Dell EMC Networking switches and operating system versions Qty Item Version 2 Dell EMC Networking S5248F-ON ToR switches 10.4.2 2 Dell EMC Networking S4148U-ON ToR switches 10.4.2 1 Dell EMC Networking S3048-ON OOB management switch 10.4.1.2 Note: Validation of the S4148U-ON switches was performed by replacing the S5248F-ON switches within the same test environment that is detailed within this document. B.
C Technical resources Dell EMC Networking Guides Manuals and documents for Dell EMC Networking S3048-ON Manuals and documents for Dell EMC Networking S5248F-ON Manuals and documents for Dell EMC Networking S4148U-ON Manuals and documents for Dell EMC Networking S4112F-ON 33 Dell EMC Networking OS10 Enterprise Edition Deployment Guide with Cisco ACI
D Support and feedback Contacting Technical Support Support Contact Information Web: http://www.dell.com/support Telephone: USA: 1-800-945-3355 Feedback for this document Readers are encouraged to provide feedback on the quality and usefulness of this publication by sending an email to Dell_Networking_Solutions@Dell.
