VXLAN and BGP EVPN Configuration Guide for Dell EMC SmartFabric OS10 Release 10.5.0 June 2020 Rev.
Notes, cautions, and warnings NOTE: A NOTE indicates important information that helps you make better use of your product. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. © 2019 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries.
Contents Chapter 1: VXLAN .........................................................................................................................6 VXLAN concepts..................................................................................................................................................................7 VXLAN as NVO solution.....................................................................................................................................................
show mac address-table count nve........................................................................................................................ 34 show mac address-table count virtual-network.................................................................................................. 34 show mac address-table extended......................................................................................................................... 35 show mac address-table nve............................
Specify the controller reachability information...................................................................................................148 Assign interfaces to be managed by the controller...........................................................................................149 Service Nodes.............................................................................................................................................................149 View replicators............................
1 VXLAN A virtual extensible LAN (VXLAN) extends Layer 2 (L2) server connectivity over an underlying Layer 3 (L3) transport network in a virtualized data center. A virtualized data center consists of virtual machines (VMs) in a multi-tenant environment. OS10 supports VXLAN as described in RFC 7348. VXLAN provides a L2 overlay mechanism on an existing L3 network by encapsulating the L2 frames in L3 packets.
● N3248TE-ON Topics: • • • • • • • • • • • VXLAN concepts VXLAN as NVO solution Configure VXLAN L3 VXLAN route scaling Overlay ECMP for L3 prefix routes DHCP relay on VTEPs View VXLAN configuration VXLAN MAC addresses VXLAN commands VXLAN MAC commands Example: VXLAN with static VTEP VXLAN concepts Network virtualization overlay (NVO) An overlay network extends L2 connectivity between server virtual machines (VMs) in a tenant segment over an underlay L3 IP network.
Switch-scoped VLAN A VLAN that is mapped to a virtual network ID (VNID) in OS10. All member ports of the VLAN are automatically added to the virtual network. ● You can map only one VLAN ID to a virtual network. ● Ideally suited for existing tenant VLANs that stretch over an IP fabric using VXLAN. Port-scoped VLAN A Port,VLAN pair that maps to a virtual network ID (VNID) in OS10. Assign an individual member interface to a virtual network either with an associated tagged VLAN or as an untagged member.
2. Configure an IP address on the Loopback interface in INTERFACE mode. The IP address allows the source VTEP to send VXLAN frames over the L3 transport network. ip address ip-address/mask 3. Return to CONFIGURATION mode. exit 4. Enter NVE mode from CONFIGURATION mode. NVE mode allows you to configure the VXLAN tunnel endpoint on the switch. nve 5. Configure the Loopback interface as the source tunnel endpoint for all virtual networks on the switch in NVE mode. source-interface loopback number 6.
1. Assign a VLAN to the virtual network in VLAN Interface mode. interface vlan vlan-id virtual-network vn-id 2. Configure port interfaces as trunk members of the VLAN in Interface mode. interface ethernet node/slot/port[:subport] switchport mode trunk switchport trunk allowed-vlan vlan-id exit The local physical ports assigned to the VLAN transmit packets over the virtual network.
1. Create a reserved VLAN ID to assign untagged traffic on member interfaces to a virtual network in CONFIGURATION mode. The VLAN ID is used internally for all untagged member interfaces on the switch that belong to virtual networks. virtual-network untagged-vlan untagged-vlan-id 2. Configure port interfaces as trunk members and remove the access VLAN in Interface mode. interface ethernet node/slot/port[:subport] switchport mode trunk no switchport access vlan exit 3.
network IP addresses in different subnets. If you do not assign the virtual-network interface to a tenant VRF, it is assigned to the default VRF. interface virtual-network vn-id ip vrf forwarding tenant-vrf-name ip address ip-address/mask no shutdown exit 4. Configure an anycast gateway IPv4 or IPv6 address for each virtual network in INTERFACE-VIRTUAL-NETWORK mode. This anycast IP address must be in the same subnet as the IP address of the virtual-network interface in Step 3.
Table 1. MAC address for all VTEPs (continued) Virtual network VNID 13 VTEP Anycast gateway MAC address VTEP 3 00.11.22.33.44.55 VTEP 1 00.11.22.33.44.55 VTEP 2 00.11.22.33.44.55 VTEP 3 00.11.22.33.44.55 ● Configure a unique IP address on the virtual-network interface on each VTEP across all virtual networks. Configure the same anycast gateway IP address on all VTEPs in a virtual-network subnet. For example: Table 2.
Configure VLT (Optional) To use VXLAN in a VLT domain, configure the VLT domain — including the VLT Interconnect (VLTi) interfaces, backup heartbeat, and VLT MAC address — as described in the OS10 Enterprise Edition User Guide in the Virtual link trunking section. Required VLT VXLAN configuration: ● The IP address of the VTEP source Loopback interface must be same on the VLT peers.
Table 3.
Overlay ECMP for L3 prefix routes If a tenant IP subnet extends between more than one VTEP, all VTEPs which host the subnet advertise its reachability using Type-5 prefix routes. ● A VTEP that is locally connected to the tenant subnet uses local routes instead of remote routes. ● A VTEP that is not locally connected to the tenant subnet installs the remote routes using an ECMP group of overlay next-hops.
2. Configure a static IP route to the DHCP server interface in the tenant VRF. OS10(config)# ip route vrf tenant01 40.1.1.0/24 interface vlan40 3. Configure DHCP relay on the virtual-network interface of the tenant VRF. OS10(config)# interface virtual-network 10 OS10(conf-if-vn-10)# ip helper-address 40.1.1.1 vrf tenant01 View VXLAN configuration Use show commands to verify the VXLAN configuration and monitor VXLAN operation.
1000 2000 857/8570 457/3570 257/23709 277/13709 NOTE: Using flex counters, OS10 may display additional packets in the Output field number, but the additional packets do not transmit. For an accurate count, use the Output Bytes number. View the VXLAN remote VTEPs OS10# show nve remote-vtep summary Remote-VTEP State ---------------------2.2.2.2 up OS10# show nve remote-vtep Codes: DP - MAC-learn Dataplane, CP - MAC-learn Controlplane, UUD - Unknown-Unicast-Drop IP Address: 2.2.2.
C C C 100.33.0.0/16 100.65.0.0/16 100.97.0.0/16 via 100.33.1.4 via 100.65.1.4 via 100.97.1.
Table 4. Display VXLAN MAC addresses (continued) Command Description interface ethernet node/slot/port:subport: Displays only MAC addresses learned on the specified interface. interface port-channel number: Displays only MAC addresses learned on the specified port channel. static: Displays only static MAC addresses. dynamic: Displays only dynamic MAC addresses.
Table 5. Clear VXLAN MAC addresses Command Description clear mac address-table dynamic virtual-network [interface {ethernet node/slot/port:subport | port-channel number} | local | vn-id [address mac-address | local]] Clears all MAC addresses learned on all VXLAN virtual networks. interface ethernet node/slot/port:subport: Clears only MAC addresses learned on the specified interface. interface port-channel number: Clears only MAC addresses learned on the specified port channel.
Parameters Default balancedoverlayrouting Reserve routing entries for balanced VXLAN tenant routing: ● S4048T-ON and S6010-ON: 24576 entries ● S4100-ON series: 16384 entries ● S5200-ON series switches: 32768 entries scaledoverlayrouting Reserve routing entries for scaled VXLAN tenant routing: ● S4048T-ON and S6010-ON: 36864 entries ● S4100-ON series: 24576 entries ● S5200-ON series switches: 53248 entries disableoverlayrouting Allocate 0 next-hop entries for overlay routing and all next-hop entries f
ip virtual-router address Configures an anycast gateway IP address for a VXLAN virtual network. Syntax Parameters ip virtual-router address ip-address address ipaddress Enter the IP address of the anycast L3 gateway. Default Not configured Command mode INTERFACE-VIRTUAL-NETWORK Usage information Configure the same anycast gateway IP address on all VTEPs in a VXLAN virtual network.
port-channel number Assign the specified port channel to a virtual network. untagged Assign untagged traffic on an interface or port channel to a virtual network. vlan-tag vlan-id Assign tagged traffic on the specified VLAN to a virtual network. Default Not configured Command mode VIRTUAL-NETWORK Usage information Use this command to assign traffic on the same VLAN or interface to different virtual networks. The no version of this command removes the configured value.
Supported releases 10.4.2.0 or later show hardware overlay-ecmp-profile mode Displays the ECMP resources available for VXLAN overlay routing. Syntax Parameters show hardware overlay-ecmp-profile mode [all] all View the number of ECMP groups available in all overlay ECMP profiles, including the default profile. Default Not configured Command mode EXEC Usage information On OS10 switches, L3 VXLAN overlay routing supports ECMP processing.
balanced-overlay-routing scaled-overlay-routing Supported releases 32768 53248 32768 12288 8192 12288 8192 4096 show hardware overlay-routing-profile mode Overlay Setting Mode Next-hop Entries Current default-overlay-routing 8192 Underlay Next-hop Entries 57344 Overlay L3 RIF Entries 2048 Underlay L3 RIF Entries 14336 Next-boot default-overlay-routing 57344 2048 14336 8192 10.4.3.
Command mode EXEC Usage information Use this command to display the IP address, operational state, and configured VXLANs for each remote VTEP. The remote MAC learning and unknown unicast drop settings used for each VXLAN ID (VNI) also display. Example OS10# show nve remote-vtep summary Remote-VTEP State ----------------------2.2.2.2 up OS10# show nve remote-vtep Codes: DP - MAC-learn Dataplane, CP - MAC-learn Controlplane, UUD Unknown-Unicast-Drop IP Address: 2.2.2.
200 300 Supported releases 2 300 1.1.1.1 1.1.1.1 2.2.2.2 2.2.2.2 10.4.2.0 or later show virtual-network Displays a virtual-network configuration, including all VXLAN configurations. Syntax show virtual-network [vn-id] Parameters vn-id Enter a virtual-network ID, from 1 to 65535. Default Not configured Command mode EXEC Usage information Use this command to display the VNID, port members, source interface, and remote tunnel endpoints of a VXLAN virtual network.
show virtual-network interface counters Displays packet statistics for a member port, port channel, or VLAN in VXLAN virtual networks. Syntax Parameters show virtual-network interface {ethernet node/slot/port:subport | portchannel number} [vlan vlan-id] counters interface ethernet node/slot/ port[:subport ] Enter the port information for an Ethernet interface. interface port-channel number Enter a port-channel number, from 1 to 128. vlan vlan-id (Optional) Enter a VLAN ID, from 1 to 4093.
ethernet1/1/1 ethernet1/1/1 Supported releases 200 300 2000 3000 10.4.2.0 or later show virtual-network vlan Displays the VXLAN virtual networks where a VLAN is assigned. Syntax show virtual-network vlan vlan-id Parameters vlan vlan-id Enter a VLAN ID, from 1 to 4093. Default Not configured Command mode EXEC Usage information Use this command to verify the VXLAN virtual networks where a VLAN is assigned, including the port members connected to downstream servers.
source-interface loopback Configures a dedicated Loopback interface as the source VTEP. Syntax Parameters source-interface loopback number loopback number Enter the Loopback interface used as the source interface of a VXLAN virtual tunnel, from 0 to 16383. Default Not configured Command mode NVE-INSTANCE Usage information The IP address of the Loopback interface serves as the source IP address in encapsulated packets transmitted from the switch as an NVE VTEP.
Usage information Example Supported releases The untagged VLAN ID is used internally for all untagged member interfaces that belong to virtual networks. You cannot use the reserved untagged VLAN ID for a simple VLAN bridge or for tagged traffic on member interfaces of virtual networks. The no version of this command removes the configured value. OS10(config)# virtual-network untagged-vlan 10 10.4.2.0 or later vxlan-vni Assigns a VXLAN ID to a virtual network.
clear mac address-table dynamic virtual-network Clears MAC addresses learned on all or a specified VXLAN virtual network. Syntax Parameters clear mac address-table dynamic virtual-network [interface {ethernet node/ slot/port:subport | port-channel number} | local | vn-id [address macaddress | local]] interface ethernet node/slot/ port[:subport ] Clear all MAC addresses learned on the specified interface. interface port-channel number Clear all MAC addresses learned on the specified port channel.
Usage information Example Supported releases Use this command to display the number of MAC address entries learned on all VLANs and VXLAN virtual networks. OS10# show mac address-table count extended MAC Entries for all vlans : Dynamic Address Count : 10 Static Address (User-defined) Count : 2 Total MAC Addresses in Use: 12 10.4.2.0 or later show mac address-table count nve Displays the number of MAC addresses learned on a VXLAN virtual network or from a remote VXLAN tunnel endpoint.
interface ethernet node/slot/ port[:subport ] Display the number of MAC addresses learned on the specified interface. interface port-channel number Display the number of MAC addresses learned on the specified port channel. vn-id Display the number of MAC addresses learned on the specified virtual network, from 1 to 65535. Default Not configured Command mode EXEC Usage information Use this command to display the number of MAC address entries learned on virtual networks in the MAC address table.
10000 10000 10000 10000 20000 20000 20000 20000 Supported releases 1 500 4000 100 100 300 300 300 aa:bb:cc:dd:f0:03 aa:bb:cc:dd:f0:03 aa:bb:cc:dd:f0:03 00:00:00:00:00:11 00:00:00:00:00:44 00:00:00:00:00:55 00:00:00:00:00:77 00:00:00:00:00:22 00:00:00:00:00:33 00:00:00:00:00:66 00:00:00:00:00:88 static static static dynamic dynamic dynamic dynamic dynamic dynamic dynamic dynamic port-channel1000 port-channel1000 port-channel1000 ethernet1/1/31:1 port-channel1000 port-channel10 VxLAN(32.1.1.
dynamic Display only dynamic MAC addresses. address macaddress Display only information about the specified MAC address. Enter the MAC address in EEEE.EEEE.EEEE format. interface ethernet node/slot/ port[:subport ] Display only MAC addresses learned on the specified interface. interface port-channel number Display only MAC addresses learned on the specified port channel.
Figure 2. Static VXLAN use case VTEP 1 Leaf Switch 1. Configure the underlay OSPF protocol Do not configure the same IP address for the router ID and the source loopback interface in Step 2. OS10(config)# router ospf 1 OS10(config-router-ospf-1)# router-id 172.16.0.1 OS10(config-router-ospf-1)# exit 2. Configure a Loopback interface OS10(config)# interface loopback0 OS10(conf-if-lo-0)# no shutdown OS10(conf-if-lo-0)# ip address 192.168.1.1/32 OS10(conf-if-lo-0)# ip ospf 1 area 0.0.0.
3. Configure the Loopback interface as the VXLAN source tunnel interface OS10(config)# nve OS10(config-nve)# source-interface loopback0 OS10(config-nve)# exit 4.
OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# no switchport mtu 1650 ip address 172.16.2.0/31 ip ospf 1 area 0.0.0.0 exit 8. Configure VLT Configure a dedicated L3 underlay path to reach the VLT Peer in case of network failure OS10(config)# interface vlan4000 OS10(config-if-vl-4000)# no shutdown OS10(config-if-vl-4000)# ip address 172.16.250.1/30 OS10(config-if-vl-4000)# ip ospf 1 area 0.0.0.
OS10(config-if-vn-10000)# ip address 10.1.0.231/16 OS10(config-if-vn-10000)# ip virtual-router address 10.1.0.100 OS10(config-if-vn-10000)# no shutdown OS10(config-if-vn-10000)# exit OS10(config)# interface virtual-network 20000 OS10(config-if-vn-20000)# ip vrf forwarding tenant1 OS10(config-if-vn-20000)# ip address 10.2.0.231/16 OS10(config-if-vn-20000)# ip virtual-router address 10.2.0.100 OS10(config-if-vn-20000)# no shutdown OS10(config-if-vn-20000)# exit VTEP 2 Leaf Switch 1.
OS10(conf-if-po-10)# switchport access vlan 200 OS10(conf-if-po-10)# exit OS10(config)# interface OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# ethernet1/1/5 no shutdown channel-group 10 mode active no switchport exit OS10(config)# interface port-channel20 OS10(conf-if-po-20)# no shutdown OS10(conf-if-po-20)# switchport mode access OS10(conf-if-po-20)# switchport access vlan 200 OS10(conf-if-po-20)# exit OS10(config)# interface OS10(conf-if-eth1/1/6)# OS10
Configure a VLT domain OS10(config)# vlt-domain 1 OS10(conf-vlt-1)# backup destination 10.16.150.
4. Configure VXLAN virtual networks with a static VTEP OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vxlan-vni 10000 OS10(config-vn-vxlan-vni)# remote-vtep OS10(config-vn-vxlan-vni-remote-vtep)# OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(config-vn-20000)# vxlan-vni 20000 OS10(config-vn-vxlan-vni)# remote-vtep OS10(config-vn-vxlan-vni-remote-vtep)# OS10(config-vn-vxlan-vni)# exit OS10(config-vn-20000)# exit 192.168.1.1 exit 192.168.1.
OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# no switchport mtu 1650 ip address 172.18.2.0/31 ip ospf 1 area 0.0.0.0 exit 9.
Configure an anycast L3 gateway OS10(config)# ip virtual-router mac-address 00:01:01:01:01:01 Configure routing with an anycast gateway IP address for each virtual network OS10(config)# interface virtual-network 10000 OS10(config-if-vn-10000)# ip vrf forwarding tenant1 OS10(config-if-vn-10000)# ip address 10.1.0.233/16 OS10(config-if-vn-10000)# ip virtual-router address 10.1.0.
OS10(conf-if-po-10)# no switchport access vlan OS10(conf-if-po-10)# exit OS10(config)# interface OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# ethernet1/1/5 no shutdown channel-group 10 mode active no switchport exit OS10(config)# interface port-channel20 OS10(conf-if-po-20)# no shutdown OS10(conf-if-po-20)# switchport mode trunk OS10(conf-if-po-20)# no switchport access vlan OS10(conf-if-po-20)# exit OS10(config)# interface OS10(conf-if-eth1/1/6)# OS10(co
OS10(conf-if-po-10)# exit OS10(config)# interface port-channel20 OS10(conf-if-po-20)# vlt port-channel 20 OS10(conf-if-po-20)# exit Configure VLTi member links OS10(config)# interface OS10(conf-if-eth1/1/3)# OS10(conf-if-eth1/1/3)# OS10(conf-if-eth1/1/3)# ethernet1/1/3 no shutdown no switchport exit OS10(config)# interface OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# ethernet1/1/4 no shutdown no switchport exit Configure a VLT domain OS10(config)# vlt-domain 1 OS10(conf-vlt-1)
OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# no switchport ip address 172.16.1.1/31 ip ospf 1 area 0.0.0.0 exit OS10(config)# interface OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# ethernet1/1/2 no shutdown no switchport ip address 172.17.1.1/31 ip ospf 1 area 0.0.0.
2 BGP EVPN for VXLAN Ethernet Virtual Private Network (EVPN) is a control plane for VXLAN that reduces flooding in the network and resolves scalability concerns. EVPN uses MP-BGP to exchange information between VTEPs. EVPN was introduced in RFC 7432 and is based on BGP MPLS-based VPNs. RFC 8365 describes VXLAN-based EVPN. The MP-BGP EVPN control plane provides protocol-based remote VTEP discovery, and MAC and ARP learning. This configuration reduces flooding related to L2 unknown unicast traffic.
Table 6. Differences between Static VXLAN and VXLAN BGP EVPN Static VXLAN VXLAN BGP EVPN To start sending and receiving virtual-network traffic to and from a remote VTEP, manually configure the VTEP as a member of the virtual network. No manual configuration is required. Each remote VTEP is automatically learned as a member of a virtual network from the EVPN routes received from the remote VTEP. After a remote VTEP address is learned, VXLAN traffic is sent to, and received from, the VTEP.
Leaf nodes are typically top-of-rack (ToR) switches in a data center network. They act as the VXLAN tunnel endpoints and perform VXLAN encapsulation and decapsulation. Leaf nodes also participate in the MP-BGP EVPN to support control plane and data plane functions. Control plane functions include: ● Initiate and maintain route adjacencies using any routing protocol in the underlay network. ● Advertise locally learned routes to all MP-BGP EVPN peers.
the export RT associated with the EVI. A receiving VTEP downloads information in the BGP EVPN route to EVIs that have a matching import RT value. You can autogenerate or manually configure the RT import and export for each EVI. In auto-EVI mode, RT autogenerates. In manual EVI configuration mode, you can autogenerate or manually configure the RT. The RT consists of a 2-octet type and a 6-octet value.
3. Configure EVPN, including the VNI, RD, and RT values associated with the EVPN instance. 4. Verify the BGP EVPN configuration. Configuration 1. Configure BGP to advertise EVPN routes. EVPN requires that you establish MP-BGP sessions between leaf and spine nodes in the underlay network. On each spine and leaf node, configure at least two BGP peering sessions: ● A directly connected BGP peer in the underlay network to advertise VTEP and Loopback IP addresses using the IPv4 unicast address family.
e. Enable the peer session with the BGP neighbor in ROUTER-BGP-NEIGHBOR mode. no shutdown f. Configure the L2 VPN EVPN address family for VXLAN host-based routing to the BGP peer in ROUTER-BGP-NEIGHBOR mode. address-family l2vpn evpn g. Enable the exchange of L2VPN EVPN addresses with the BGP peer in ROUTER-BGP-NEIGHBOR mode. activate h. Return to ROUTER-BGP mode. exit i. Enter IPv4 address-family configuration mode from ROUTER-BGP mode. address-family ipv4 unicast j.
a. Enable the EVPN control plane in CONFIGURATION mode. evpn b. Manually create an EVPN instance in EVPN mode. The range is from 1 to 65535. evi id c. Configure the Route Distinguisher in EVPN EVI mode. rd {A.B.C.D:[1-65535] | auto} Where: ○ rd A.B.C.D:[1-65535] configures the RD with a 4-octet IPv4 address then a 2-octet-number. ○ rd auto automatically generates the RD. d. Configure the RT values in EVPN EVI mode.
Minimum time between advertisement runs is 30 seconds Minimum time before advertisements start is 0 seconds Capabilities received from neighbor for IPv4 Unicast: MULTIPROTO_EXT(1) ROUTE_REFRESH(2) CISCO_ROUTE_REFRESH(128) 4_OCTET_AS(65) MP_L2VPN_EVPN Capabilities advertised to neighbor for IPv4 Unicast: MULTIPROTO_EXT(1) ROUTE_REFRESH(2) CISCO_ROUTE_REFRESH(128) 4_OCTET_AS(65) MP_L2VPN_EVPN Prefixes accepted 1, Prefixes advertised 1 Connections established 2; dropped 0 Last reset never Prefixes ignored due
ICMPv6 protocol operation. They advertise as EVPN Type-2 BGP route updates to other VTEPs, each of whom then imports and installs them as ARP/IPv6 neighbor entries in the dataplane. To enable efficient traffic forwarding on a VTEP, OS10 supports distributed gateway routing. A distributed gateway allows multiple VTEPs to act as the gateway router for a tenant subnet. The VTEP that is located nearest to a host acts as its gateway router. To 1. 2. 3.
Configure Symmetric IRB for VXLAN BGP EVPN Before you start 1. Follow the procedure in Configure VXLAN to: ● Configure the VXLAN overlay network. ● Enable routing for VXLAN virtual networks. Integrated Routing and Bridging (IRB) is automatically enabled. ● Enable an overlay routing profile with the number of reserved ARP table entries for VXLAN overlay routing. For a sample configuration, see Example: VXLAN with static VTEP. 2.
Inclusive Multicast : 110.111.170.107 IRB : Enabled(VRF-TENANT-1) OS10# show evpn evi 20000 EVI : 20000, State : up Bridge-Domain : Virtual-Network 20000, VNI 20000 Route-Distinguisher : 1:110.111.170.195:20000(auto) Route-Targets : 0:20000:16797216(auto) both Inclusive Multicast : IRB : Enabled(VRF-TENANT-1) Display the EVPN Type 2 routes for host MAC/IP addresses show evpn mac-ip Type -(lcl): Local (rmt): remote EVI Mac Address Type Seq No Host-IP Interface/Next-Hops 10000 00:00:0b:0b:0b:0a lcl 0 10.10.
*>r Route distinguisher: 4.4.4.4:101 VNI:101 [2]:[0]:[48]:[14:18:77:25:6f:4d]:[32]:[11.11.11.2]/224 4.4.4.4 0 100 32768 *>r Route distinguisher: 3.3.3.3:102 VNI:102 [2]:[0]:[48]:[14:18:77:25:8f:6d]:[32]:[12.12.12.1]/224 3.3.3.3 0 100 0 100 101 ? *> Route distinguisher: 3.3.3.3:101 [3]:[0]:[32]:[3.3.3.3]/152 3.3.3.3 0 100 0 100 101 ? *>r Route distinguisher: 4.4.4.4:101 [3]:[0]:[32]:[4.4.4.4]/152 4.4.4.4 0 100 32768 ? *>r Route distinguisher: 4.4.4.4:102 [3]:[0]:[32]:[4.4.4.4]/152 4.4.4.
Figure 4. BGP EVPN in VLT domain VXLAN BGP commands activate (l2vpn evpn) Enables the exchange of L2 VPN EVPN address family information with a BGP neighbor or peer group.
Usage Information Example Supported Releases Use this command to exchange L2 VPN EVPN address information for VXLAN host-based routing with a BGP neighbor. The IPv4 unicast address family is enabled by default. Use the no activate command to disable an address family with a neighbor. OS10(conf-router-neighbor)# address-family l2vpn evpn unicast OS10(conf-router-bgp-neighbor-af)# activate 10.2.
sender-side-loop-detection Enables the sender-side loop detection process for a BGP neighbor. Syntax sender-side-loop-detection Parameters None Default Enabled Command Mode ROUTER-BGP-NEIGHBOR-AF Usage Information This command helps detect routing loops, based on the AS path before it starts advertising routes. To configure a neighbor to accept routes use the neighbor allowas-in command. The no version of this command disables sender-side loop detection for that neighbor.
*> Route distinguisher: 110.111.170.107:64536 [3]:[0]:[32]:[110.111.170.107]/152 110.111.170.107 0 100 101 ? OS10# show BGP router Neighbor State/Pfx 3.3.3.3 4.4.4.4 5.5.5.5 6.6.6.6 0 100 ip bgp l2vpn evpn summary identifier 2.2.2.2 local AS number 4294967295 AS MsgRcvd MsgSent Up/Down 4294967295 4294967295 4294967295 4294967295 2831 2364 4947 2413 9130 9586 8399 7310 05:57:27 05:56:43 01:10:39 05:51:56 504 504 11514 504 OS10# show ip bgp l2vpn evpn neighbors BGP neighbor is 3.3.3.
Received 20 messages 1 opens, 0 notifications, 0 updates 19 keepalives, 0 route refresh requests Sent 20 messages 1 opens, 1 notifications, 0 updates 18 keepalives, 0 route refresh requests Minimum time between advertisement runs is 30 seconds Minimum time before advertisements start is 0 seconds Capabilities received from neighbor for IPv4 Unicast: MULTIPROTO_EXT(1) ROUTE_REFRESH(2) CISCO_ROUTE_REFRESH(128) 4_OCTET_AS(65) MP_L2VPN_EVPN(1) Extended Next Hop Encoding (5) Capabilities advertised to neighbor
Supported releases 10.4.2.0 or later VXLAN EVPN commands advertise Advertises the IP prefixes learned from external networks and directly connected neighbors into EVPN. Syntax advertise {ipv4 | ipv6} {connected | static| ospf | bgp} [route-map mapname] Parameters ● ● ● ● ● ● ● Default None Command Mode EVPN-VRF Usage Information EVPN uses Type 5 route advertisements. Use the advertise command to specify the types of learned routes to use in EVPN Type 5 advertisements in a tenant VRF.
○ The RT auto-configures as Type 2 from the 4-byte AS and the 2-byte EVI—Type encoded as 0x0202. Example Supported releases OS10(config)# evpn OS10(config-evpn)# auto-evi 10.4.2.0 or later evi Creates an EVPN instance (EVI) in EVPN mode. Syntax evi id Parameters id Enter the EVPN instance ID, from 1 to 65535.
auto Configure the RD to automatically generate. Default Not configured Command mode EVPN-EVI and EVPN-VRF Usage information A RD maintains the uniqueness of an EVPN route between different EVPN instances. Configure a route distinguisher in a tenant VRF used for EVPN symmetric IRB traffic. The RD auto-configures as Type 1 from the overlay network source IP address and the auto-generated EVPN instance ID. The rd auto command is not supported in EVPN-VRF mode.
Parameters value {import | export | both} Configure an RT import or export value, or both values in the format 2-octetASN:4-octet-number or 4-octet-ASN:2-octet-number. ● The 2-octet ASN or number is 1 to 65535. ● The 4-octet ASN or number is 1 to 4294967295. auto Configure the RT import and export values to automatically generate. asn4 (Optional) Advertises a 4-byte AS number in RT values.
show evpn evi Displays the configuration settings of EVPN instances. Syntax show evpn evi [id] Parameters id — (Optional) Enter the EVPN instance ID, from 1 to 65535. Default Not configured Command mode EXEC Usage information Use this command to verify EVPN instance status, associated VXLAN virtual networks and the RD and RT values the BGP EVPN routes use in the EVI. The status of integrated routing and bridging (IRB) and the VRF used for EVPN traffic also display.
Local MAC Address Count : Remote MAC Address Count : 1 2 OS10# show evpn mac evi 811 next-hop 80.80.1.8 count EVI 811 next-hop 80.80.1.8 MAC Entries : Remote MAC Address Count : 2 Supported releases 10.4.2.0 or later show evpn mac-ip Displays the BGP EVPN Type 2 routes used for host MAC-IP address binding.
106 106 14:18:77:25:6f:84 14:18:77:25:6f:84 lcl lcl 0 0 16.16.16.2 2001:16::16:2 OS10# show evpn mac-ip evi 104 Type EVI 104 104 104 104 -(lcl): Local (rmt): remote Mac-Address 14:18:77:25:4d:b9 14:18:77:25:4d:b9 14:18:77:25:6e:b9 14:18:77:25:6e:b9 Type rmt rmt lcl lcl Seq-No 0 0 0 0 Host-IP Interface/Next-Hop 14.14.14.1 95.0.0.3 2001:14::14:1 95.0.0.3 14.14.14.
show evpn vrf Displays the VRF instances used to forward EVPN routes in VXLAN overlay networks. Syntax show evpn vrf [vrf-name] Parameters vrf-name — (Optional) Enter the name of a non-default tenant VRF instance. Default Not configured Command mode EXEC Usage information Use this command to verify the tenant VRF instances used in EVPN instances to exchange BGP EVPN routes in VXLANs.
40 40 40 vrf_40 OS10# show evpn vrf l3-vni vrf_30 VRF : vrf_30, State : up L3-VNI : 3030 Route-Distinguisher : 1:80.80.1.1:3030(auto) Route-Targets : 0:200:268435557(auto) both Remote VTEP : 4.4.4.4 Supported releases 10.5.1.0 or later show evpn vxlan-vni Displays the VXLAN overlay network for EVPN instances. Syntax show evpn vxlan-vni [vni] Parameters vni — (Optional) Enter the VXLAN virtual-network ID, from 1 to 16,777,215.
vrf Creates a non-default VRF instance for EVPN symmetric IRB traffic. Syntax vrf vrf-name Parameters ● vrf-name — Enter the name of a non-default tenant VRF; 32 characters maximum. Default Not configured Command Mode EVPN Usage Information Configure a non-default VRF for symmetric IRB for each tenant VRF. The tenant VRF is created using the ip vrf command when you enable overlay routing with IRB; see Enable overlay routing between virtual networks.
Figure 5. VXLAN BGP EVPN use case VTEP 1 Leaf Switch 1. Configure a Loopback interface for the VXLAN underlay using same IP address as the VLT peer OS10(config)# interface loopback0 OS10(conf-if-lo-0)# no shutdown OS10(conf-if-lo-0)# ip address 192.168.1.1/32 OS10(conf-if-lo-0)# exit 2.
3. Configure VXLAN virtual networks OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vxlan-vni 10000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(config-vn-20000)# vxlan-vni 20000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-20000)# exit 4.
OS10(config-router-bgp-100)# address-family ipv4 unicast OS10(config-router-bgp-af)# redistribute connected OS10(config-router-bgp-af)# exit 8. Configure eBGP for the IPv4 point-to-point peering OS10(config-router-bgp-100)# neighbor 172.16.1.
12. Configure VLT Configure a dedicated L3 underlay path to reach the VLT Peer in case of a network failure OS10(config)# interface vlan4000 OS10(config-if-vl-4000)# no shutdown OS10(config-if-vl-4000)# ip address 172.16.250.
Configure routing on the virtual networks OS10(config)# interface OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# virtual-network 10000 ip vrf forwarding tenant1 ip address 10.1.0.231/16 ip virtual-router address 10.1.0.
OS10(conf-if-eth1/1/5)# exit OS10(config)# interface port-channel20 OS10(conf-if-po-20)# no shutdown OS10(conf-if-po-20)# switchport mode trunk OS10(conf-if-po-20)# switchport access vlan 200 OS10(conf-if-po-20)# exit OS10(config)# interface OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# ethernet1/1/6 no shutdown channel-group 20 mode active no switchport exit 6.
OS10(config-router-neighbor)# update-source loopback1 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# no activate OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# address-family l2vpn evpn OS10(config-router-bgp-neighbor-af)# activate OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# neighbor
OS10(conf-vlt-1)# vlt-mac aa:bb:cc:dd:ee:ff OS10(conf-vlt-1)# exit Configure UFD with uplink VLT ports and downlink network ports OS10(config)# uplink-state-group OS10(conf-uplink-state-group-1)# OS10(conf-uplink-state-group-1)# OS10(conf-uplink-state-group-1)# OS10(conf-uplink-state-group-1)# OS10(conf-uplink-state-group-1)# 1 enable downstream ethernet1/1/1-1/1/2 upstream port-channel10 upstream port-channel20 exit Configure iBGP IPv4 peering between VLT peers OS10(config)# router bgp 100 OS10(config-ro
OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(config-vn-20000)# vxlan-vni 20000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-20000)# exit 4. Configure unused VLAN ID for untagged membership OS10(config)# virtual-network untagged-vlan 1000 5.
OS10(configure-router-bgp-af)# redistribute connected OS10(configure-router-bgp-af)# exit 9. Configure eBGP for the IPv4 point-to-point peering OS10(config-router-bgp-100)# neighbor 172.18.1.1 OS10(config-router-neighbor)# remote-as 101 OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# neighbor 172.18.2.
OS10(config-evpn-evi-10000)# route-target auto OS10(config-evpn-evi-10000)# exit OS10(config-evpn)# evi 20000 OS10(config-evpn-evi-20000)# OS10(config-evpn-evi-20000)# OS10(config-evpn-evi-20000)# OS10(config-evpn-evi-20000)# OS10(config-evpn)# exit vni 20000 rd auto route-target auto exit 13.
Configure iBGP IPv4 peering between VLT peers OS10(config)# router bgp 100 OS10(config-router-bgp-100)# neighbor 172.16.250.11 OS10(config-router-neighbor)# remote-as 100 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 14.
5.
OS10(config-router-neighbor)# remote-as 101 OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 10. Configure a Loopback interface for BGP EVPN peering different from the VLT peer IP address OS10(config)# interface loopback1 OS10(conf-if-lo-1)# no shutdown OS10(conf-if-lo-1)# ip address 172.19.0.
Configure a VLTi VLAN for the virtual network OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vlti-vlan 100 OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(conf-vn-20000)# vlti-vlan 200 OS10(conf-vn-20000)# exit Configure a dedicated L3 underlay path to reach the VLT Peer in case of a network failure OS10(config)# interface vlan4000 OS10(config-if-vl-4000)# no shutdown OS10(config-if-vl-4000)# ip address 172.16.250.
Create a tenant VRF OS10(config)# ip vrf tenant1 OS10(conf-vrf)# exit Configure an anycast gateway MAC address OS10(config)# ip virtual-router mac-address 00:01:01:01:01:01 Configure routing on the virtual networks OS10(config)# interface OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# virtual-network 10000 ip vrf forwarding tenant1 ip address 10.1.0.234/16 ip virtual-router address 10.1.0.
OS10(conf-router-bgp-101)# neighbor 172.17.1.0 OS10(conf-router-neighbor)# remote-as 100 OS10(conf-router-neighbor)# no shutdown OS10(conf-router-neighbor)# address-family ipv4 unicast OS10(conf-router-neighbor-af)# no sender-side-loop-detection OS10(conf-router-neighbor-af)# exit OS10(conf-router-neighbor)# exit OS10(conf-router-bgp-101)# neighbor 172.18.1.
OS10(conf-router-neighbor)# address-family l2vpn evpn OS10(conf-router-neighbor-af)# no sender-side-loop-detection OS10(conf-router-neighbor-af)# activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-bgp-101)# neighbor 172.19.0.
OS10(conf-router-bgp-101)# neighbor 172.18.2.0 OS10(conf-router-neighbor)# remote-as 100 OS10(conf-router-neighbor)# no shutdown OS10(conf-router-neighbor)# address-family ipv4 unicast OS10(conf-router-neighbor-af)# no sender-side-loop-detection OS10(conf-router-neighbor-af)# exit OS10(conf-router-neighbor)# exit OS10(conf-router-bgp-101)# neighbor 172.19.2.
OS10(conf-router-neighbor)# remote-as 100 OS10(conf-router-neighbor)# send-community extended OS10(conf-router-neighbor)# update-source loopback1 OS10(conf-router-neighbor)# no shutdown OS10(conf-router-neighbor)# address-family ipv4 unicast OS10(conf-router-neighbor-af)# no activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-neighbor)# address-family l2vpn evpn OS10(conf-router-neighbor-af)# no sender-side-loop-detection OS10(conf-router-neighbor-af)# activate OS10(conf-router-neighbor-af)# exit
64 bytes from 10.2.0.10: icmp_seq=4 ttl=63 time=0.944 ms 64 bytes from 10.2.0.10: icmp_seq=5 ttl=63 time=0.806 ms --- 10.2.0.10 ping statistics --5 packets transmitted, 5 received, 0% packet loss, time 4078ms rtt min/avg/max/mdev = 0.806/0.851/0.944/0.051 ms root@HOST-A:~# 5. Check connectivity between host A and host C root@HOST-A:~# ping 10.1.0.20 -c 5 PING 10.1.0.20 (10.1.0.20) 56(84) bytes of 64 bytes from 10.1.0.20: icmp_seq=1 ttl=64 64 bytes from 10.1.0.20: icmp_seq=2 ttl=64 64 bytes from 10.1.0.
Figure 6. VXLAN BGP EVPN with multiple AS VTEP 1 Leaf Switch 1. Configure a Loopback interface for the VXLAN underlay using same IP address as the VLT peer OS10(config)# interface loopback0 OS10(conf-if-lo-0)# no shutdown OS10(conf-if-lo-0)# ip address 192.168.1.1/32 OS10(conf-if-lo-0)# exit 2.
3. Configure VXLAN virtual networks OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vxlan-vni 10000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(config-vn-20000)# vxlan-vni 20000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-20000)# exit 4.
OS10(config-router-bgp-99)# address-family ipv4 unicast OS10(config-router-bgp-af)# redistribute connected OS10(config-router-bgp-af)# exit 8. Configure eBGP for the IPv4 point-to-point peering OS10(config-router-bgp-99)# neighbor 172.16.1.1 OS10(config-router-neighbor)# remote-as 101 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-99)# neighbor 172.16.2.
OS10(config-evpn-evi-20000)# route-target 100:20000 import OS10(config-evpn-evi-20000)#exit OS10(config-evpn)# 12. Configure VLT Configure a dedicated L3 underlay path to reach the VLT Peer in case of a network failure OS10(config)# interface vlan4000 OS10(config-if-vl-4000)# no shutdown OS10(config-if-vl-4000)# ip address 172.16.250.
Configure an anycast gateway MAC address OS10(config)# ip virtual-router mac-address 00:01:01:01:01:01 Configure routing on the virtual networks OS10(config)# interface OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# virtual-network10000 ip vrf forwarding tenant1 ip address 10.1.0.231/16 ip virtual-router address 10.1.0.
OS10(config)# interface OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# OS10(conf-if-eth1/1/5)# ethernet1/1/5 no shutdown channel-group 10 mode active no switchport exit OS10(config)# interface port-channel20 OS10(conf-if-po-20)# no shutdown OS10(conf-if-po-20)# switchport mode trunk OS10(conf-if-po-20)# switchport access vlan 200 OS10(conf-if-po-20)# exit OS10(config)# interface OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# OS10(conf-if-eth1/1/6)# ethern
OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# no activate OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# address-family l2vpn evpn OS10(config-router-bgp-neighbor-af)# activate OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# exit OS10(config-router-bgp-99)# neighbor 172.202.0.
OS10(conf-if-eth1/1/4)# no switchport OS10(conf-if-eth1/1/4)# exit Configure the VLT domain OS10(config)# vlt-domain 1 OS10(conf-vlt-1)# backup destination 10.16.150.
2. Configure the Loopback interface as the VXLAN source tunnel interface OS10(config)# nve OS10(config-nve)# source-interface loopback0 OS10(config-nve)# exit 3. Configure VXLAN virtual networks OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vxlan-vni 10000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit OS10(config)# virtual-network 20000 OS10(config-vn-20000)# vxlan-vni 20000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-20000)# exit 4.
OS10(conf-if-eth1/1/1)# mtu 1650 OS10(conf-if-eth1/1/2)# ip address 172.18.2.0/31 OS10(conf-if-eth1/1/2)# exit 8. Configure eBGP OS10(config)# router bgp 100 OS10(config-router-bgp-100)# router-id 172.18.0.1 OS10(config-router-bgp-100)# address-family ipv4 unicast OS10(configure-router-bgp-af)# redistribute connected OS10(configure-router-bgp-af)# exit 9. Configure eBGP for the IPv4 point-to-point peering OS10(config-router-bgp-100)# neighbor 172.18.1.
OS10(config-evpn-evi-10000)# rd 192.168.2.1:10000 OS10(config-evpn-evi-10000)# route-target 99:10000 import OS10(config-evpn-evi-10000)# route-target 100:10000 both OS10(config-evpn-evi-10000)#exit OS10(config-evpn)# evi 20000 OS10(config-evpn-evi-20000)# vni 20000 OS10(config-evpn-evi-20000)# rd 192.168.2.1:20000 OS10(config-evpn-evi-20000)# route-target 99:20000 import OS10(config-evpn-evi-20000)# route-target 100:20000 both OS10(config-evpn-evi-20000)#exit OS10(config-evpn)# 13.
Configure iBGP IPv4 peering between VLT peers OS10(config)# router bgp 100 OS10(config-router-bgp-100)# neighbor 172.16.250.11 OS10(config-router-neighbor)# remote-as 100 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 14.
5.
OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 10. Configure a Loopback interface for BGP EVPN peering different from the VLT peer IP address OS10(config)# interface loopback1 OS10(conf-if-lo-1)# no shutdown OS10(conf-if-lo-1)# ip address 172.19.0.1/32 OS10(conf-if-lo-1)# exit 11. Configure BGP EVPN peering OS10(config)# router bgp 100 OS10(config-router-bgp-100)# neighbor 172.201.0.
OS10(conf-vn-20000)# vlti-vlan 200 OS10(conf-vn-20000)# exit Configure a dedicated L3 underlay path to reach the VLT Peer in case of a network failure OS10(config)# interface vlan4000 OS10(config-if-vl-4000)# no shutdown OS10(config-if-vl-4000)# ip address 172.16.250.
Configure routing on the virtual networks OS10(config)# interface OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# virtual-network10000 ip vrf forwarding tenant1 ip address 10.1.0.234/16 ip virtual-router address 10.1.0.
OS10(conf-router-neighbor)# exit OS10(conf-router-bgp-101)# exit 4. Configure a Loopback interface for BGP EVPN peering OS10(config)# interface loopback1 OS10(conf-if-lo-1)# no shutdown OS10(conf-if-lo-1)# ip address 172.201.0.1/32 OS10(conf-if-lo-1)# exit 5. Configure BGP EVPN peer sessions OS10(config)# router bgp 101 OS10(conf-router-bgp-101)# neighbor 172.16.0.
Spine Switch 2 1.
OS10(conf-router-neighbor)# update-source loopback1 OS10(conf-router-neighbor)# no shutdown OS10(conf-router-neighbor)# address-family ipv4 unicast OS10(conf-router-neighbor-af)# no activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-neighbor)# address-family l2vpn evpn OS10(conf-router-neighbor-af)# activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-bgp-102)# neighbor 172.17.0.
2. Verify EVPN configurations and EVPN parameters LEAF1# show evpn evi EVI : 10000, State : up Bridge-Domain : Route-Distinguisher : Route-Targets : Inclusive Multicast : IRB : EVI : 20000, State : up Bridge-Domain : Route-Distinguisher : Route-Targets : Inclusive Multicast : IRB : LEAF1# Virtual-Network 10000, VNI 10000 1:192.168.1.1:10000 0:99:10000 both, 0:100:10000 import 192.168.2.1 Enabled(tenant1) Virtual-Network 20000, VNI 20000 1:192.168.1.1:20000 0:99:10000 both, 0:100:10000 import 192.168.2.
rtt min/avg/max/mdev = 0.640/0.669/0.707/0.041 ms root@HOST-A:~# NOTE: Follow Steps 1 to 6 to check ping connectivity between combinations of other hosts, and between hosts through different virtual-network IP addresses. Example: VXLAN BGP EVPN — Centralized L3 gateway The following VXLAN with BGP EVPN example uses a centralized Layer 3 gateway to perform virtual-network routing. It is based on the sample configuration in Example: VXLAN BGP EVPN — Multiple AS topology.
Figure 7. VXLAN BGP EVPN with centralized L3 gateway NOTE: This centralized L3 gateway example for VXLAN BGP EVPN uses the same configuration steps as in Example: VXLAN BGP EVPN — Multiple AS topology. Configure each spine and leaf switch as in the Multiple AS topology example, except: ● Because VTEPs 1 and 2 operate only in Layer 2 VXLAN mode, do not configure IP switching in the overlay network.
Create a tenant VRF OS10(config)# ip vrf tenant1 OS10(conf-vrf)# exit Configure an anycast gateway MAC address OS10(config)# ip virtual-router mac-address 00:01:01:01:01:01 Configure routing on the virtual networks OS10(config)# interface OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# OS10(conf-if-vn-10000)# virtual-network10000 ip vrf forwarding tenant1 ip address 10.1.0.233/16 ip virtual-router address 10.1.0.
S4048T-ON, S6010-ON, and the S4100-ON series, routing after decapsulation is performed only between virtual networks. You can connect an egress virtual network to a VLAN in an external router, which connects to the external network. In the following example, VLT domain 1 is a VLT VTEP. VLT domain 2 is the border leaf VLT VTEP pair. All virtual networks in the data center network are configured in all VTEPs with virtual-network IP and anycast IP gateway addresses.
Figure 8. VXLAN BGP EVPN with border leaf gateway NOTE: This border leaf gateway example for VXLAN BGP EVPN uses the same configuration steps as in Example: VXLAN BGP EVPN — Multiple AS topology. Configure each spine and leaf switch as in the Multiple AS topology example and add the following additional configuration steps on each VTEP. VTEP 1 Leaf Switch 14. Configure a dedicated VXLAN virtual network.
15. Configure routing on the virtual network. OS10(config)# interface virtual-network 500 OS10(conf-if-vn-10000)# ip vrf forwarding tenant1 OS10(conf-if-vn-10000)# ip address 10.5.0.231/16 16. Configure a static route for outbound traffic sent to the anycast MAC address of the dedicated virtual network. OS10(config)#ip route 0.0.0.0/0 10.5.0.100 VTEP 2 Leaf Switch 14. Configure a dedicated VXLAN virtual network.
18. Configure a static route for outbound traffic sent to VLAN 200. OS10(config)#ip route 0.0.0.0/0 10.10.0.3 VTEP 4 Leaf Switch 14. Configure a dedicated VXLAN virtual network. OS10(config)# virtual-network 500 OS10(config-vn-500)# vxlan-vni 500 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit 15. Configure an anycast gateway MAC address on the boder leaf VTEP. This MAC address must be different from the anycast gateway MAC address configured on non-border-leaf VTEPs.
VTEP 1 Leaf Switch 1. Configure a Loopback interface for the VXLAN underlay using same IP address as the VLT peer OS10(config)# interface loopback0 OS10(conf-if-lo-0)# no shutdown OS10(conf-if-lo-0)# ip address 192.168.1.
2. Configure the Loopback interface as the VXLAN source tunnel interface OS10(config)# nve OS10(config-nve)# source-interface loopback0 OS10(config-nve)# exit 3. Configure the VXLAN virtual network OS10(config)# virtual-network 10000 OS10(config-vn-10000)# vxlan-vni 10000 OS10(config-vn-vxlan-vni)# exit OS10(config-vn-10000)# exit 4.
OS10(config-router-bgp-100)# neighbor 172.16.2.1 OS10(config-router-neighbor)# remote-as 101 OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 9.
Configure the VLT port channel OS10(config)# interface port-channel10 OS10(conf-if-po-10)# vlt-port-channel 10 OS10(conf-if-po-10)# exit Configure the VLTi member links OOS10(config)# interface ethernet1/1/3 OS10(conf-if-eth1/1/3)# no shutdown OS10(conf-if-eth1/1/3)# no switchport OS10(conf-if-eth1/1/3)# exit OS10(config)# interface OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# ethernet1/1/4 no shutdown no switchport exit Configure the VLT domain OS10(config)# vlt-domain 1 OS10(c
In EVPN mode, configure the router MAC used by remote VTEPs as the destination address in VXLAN encapsulated packets sent to the switch. Configure a dedicated VXLAN VNI for symmetric IRB for each tenant VRF. OS10(config)# evpn OS10(config-evpn)# router-mac 00:01:02:03:04:05 OS10(config-evpn)# vrf tenant1 OS10(config-evpn-vrf-tenant1)# vni 3000 OS10(config-evpn-vrf-tenant1)# route-target 65535:30000 both OS10(config-evpn-vrf-tenant1)# exit OS10(config-evpn)# exit OS10(config)# 15.
6. Configure upstream network-facing ports OS10(config)# interface OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# ethernet1/1/1 no shutdown no switchport mtu 1650 ip address 172.17.1.0/31 exit OS10(config)# interface OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# ethernet1/1/2 no shutdown no switchport mtu 1650 ip address 172.17.2.0/31 exit 7.
OS10(config-router-neighbor)# send-community extended OS10(config-router-neighbor)# update-source loopback1 OS10(config-router-neighbor)# no shutdown OS10(config-router-bgp-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# no activate OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# address-family l2vpn evpn OS10(config-router-bgp-neighbor-af)# activate OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-rou
OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 13.
4. Configure unused VLAN ID for untagged membership OS10(config)# virtual-network untagged-vlan 1000 5.
10. Configure a Loopback interface for BGP EVPN peering different from VLT peer IP address OS10(config)# interface loopback1 OS10(conf-if-lo-1)# no shutdown OS10(conf-if-lo-1)# ip address 172.18.0.1/32 OS10(conf-if-lo-1)# exit 11. Configure BGP EVPN peering OS10(config)# router bgp 100 OS10(config-router-bgp-100)# neighbor 172.201.0.
Configure the VLT port channel OS10(config)# interface port-channel20 OS10(conf-if-po-20)# vlt-port-channel 20 OS10(conf-if-po-20)# exit Configure VLTi member links OOS10(config)# interface ethernet1/1/3 OS10(conf-if-eth1/1/3)# no shutdown OS10(conf-if-eth1/1/3)# no switchport OS10(conf-if-eth1/1/3)# exit OS10(config)# interface OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# ethernet1/1/4 no shutdown no switchport exit Configure the VLT domain OS10(config)# vlt-domain 1 OS10(conf-
In EVPN mode, configure the router MAC used by remote VTEPs as the destination address in VXLAN encapsulated packets sent to the switch. Configure a dedicated VXLAN VNI for symmetric IRB for each tenant VRF. OS10(config)# evpn OS10(config-evpn)# router-mac 00:01:02:03:04:06 OS10(config-evpn)# vrf tenant1 OS10(config-evpn-vrf-tenant1)# vni 3000 OS10(config-evpn-vrf-tenant1)# route-target 65535:30000 both OS10(config-evpn-vrf-tenant1)# exit OS10(config-evpn)# exit OS10(config)# 16.
OS10(config-route-map)# match ip address prefix-list deny_v4_host_routes OS10(config-route-map)# exit OS10(config)# router bgp 100 OS10(config-router-bgp-100)# vrf tenant1 OS10(config-router-bgp-100-vrf)# address-family ipv4 unicast OS10(configure-router-bgpv4-vrf-af)# redistribute l2vpn evpn route-map deny_v4_host_routes OS10(configure-router-bgpv4-vrf-af)# end Use the following configuration to advertise the local connected routes on the border-leaf switches to external device: OS10(config)# router bgp 10
7. Configure upstream network-facing ports OS10(config)# interface OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# OS10(conf-if-eth1/1/1)# ethernet1/1/1 no shutdown no switchport mtu 1650 ip address 172.19.1.0/31 exit OS10(config)# interface OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# ethernet1/1/2 no shutdown no switchport mtu 1650 ip address 172.19.2.0/31 exit 8.
OS10(config-router-neighbor)# send-community extended OS10(config-router-neighbor)# update-source loopback1 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# address-family ipv4 unicast OS10(config-router-bgp-neighbor-af)# no activate OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-neighbor)# address-family l2vpn evpn OS10(config-router-bgp-neighbor-af)# activate OS10(config-router-bgp-neighbor-af)# allowas-in 1 OS10(config-router-bgp-neighbor-af)# exit OS10(config-router-
OS10(conf-uplink-state-group-1)# downstream ethernet1/1/1-1/1/2 OS10(conf-uplink-state-group-1)# upstream port-channel20 OS10(conf-uplink-state-group-1)# exit Configure iBGP IPv4 peering between the VLT peers OS10(config)# router bgp 100 OS10(config-router-bgp-100)# neighbor 172.16.250.10 OS10(config-router-neighbor)# remote-as 100 OS10(config-router-neighbor)# no shutdown OS10(config-router-neighbor)# exit OS10(config-router-bgp-100)# exit 14.
18. Configure BGP session with external router on the border-leaf VTEPs OS10(config)# router bgp 100 OS10(config-router-bgp-100)# vrf tenant1 OS10(config-router-bgp-100-vrf)# neighbor 10.10.0.3 OS10(config-router-vrf-neighbor)# remote-as 102 OS10(config-router-vrf-neighbor)# no shutdown OS10(config-router-vrf-neighbor)# end 19.
OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(conf-if-eth1/1/2)# OS10(config)# interface OS10(conf-if-eth1/1/3)# OS10(conf-if-eth1/1/3)# OS10(conf-if-eth1/1/3)# OS10(conf-if-eth1/1/3)# OS10(config)# interface OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# no switchport ip address 172.17.1.1/31 exit ethernet1/1/3 no shutdown no switchport ip address 172.18.1.1/31 exit ethernet1/1/4 no shutdown no switchport ip address 172.19.1.1/31 exit 2.
OS10(conf-router-neighbor)# update-source loopback1 OS10(conf-router-neighbor)# no shutdown OS10(conf-router-neighbor)# address-family ipv4 unicast OS10(conf-router-neighbor-af)# no activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-neighbor)# address-family l2vpn evpn OS10(conf-router-neighbor-af)# no sender-side-loop-detection OS10(conf-router-neighbor-af)# activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-bgp-101)# neighbor 172.17.0.
OS10(conf-if-eth1/1/3)# OS10(config)# interface OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# OS10(conf-if-eth1/1/4)# exit ethernet1/1/4 no shutdown no switchport ip address 172.19.2.1/31 exit 2. Configure eBGP OS10(config)# router bgp 101 OS10(config-router-bgp-101)# router-id 172.202.0.1 OS10(config-router-bgp-101)# address-family ipv4 unicast OS10(configure-router-bgpv4-af)# redistribute connected OS10(configure-router-bgpv4-af)# exit 3.
OS10(conf-router-neighbor-af)# activate OS10(conf-router-neighbor-af)# exit OS10(conf-router-bgp-101)# neighbor 172.17.0.
LEAF1# LEAF1# show evpn vrf l3-vni VRF : tenant1, State : up L3-VNI : 3000 Route-Distinguisher : 1:192.168.1.1:3000(auto) Route-Targets : 0:65535:30000 both Remote VTEP : 192.168.2.1 LEAF1# 3. Verify BGP EVPN neighborship between leaf and spine nodes LEAF1# show ip bgp l2vpn evpn summary BGP router identifier 172.16.0.1 local AS number 100 Neighbor AS MsgRcvd MsgSent Up/Down State/Pfx 172.201.0.1 101 1132 1116 13:29:00 27 172.202.0.1 101 1131 1118 13:29:02 28 LEAF1# 4.
3 Controller-provisioned VXLAN OS10 supports VXLAN provisioning using an Open vSwitch Database (OVSDB) controller. Currently, the only supported OVSDB controller is the VMware NSX controller. In a controller-provisioned VXLAN, the controller manages VXLAN-related configurations and other control-plane operations, such as MAC address propagation. NOTE: Controller-provisioned VXLAN is not supported on S5148F-ON and S3048-ON switches.
Topics: • • • • Configure controller-provisioned VXLAN Configure and control VXLAN from VMware vCenter Example: VXLAN with a controller configuration VXLAN Controller commands Configure controller-provisioned VXLAN To configure the NSX controller, follow these steps on each OS10 VTEP: 1. Configure the source interface used for controller-based VXLAN provisioning. Assign an IPv4 address to a loopback interface. Assign the loopback interface to an NVE instance.
Assign interfaces to be managed by the controller In a VTEP, explicitly assign interfaces for an OVSDB controller to manage. Before you assign the interface, consider the following: ● The interface must be in Switchport Trunk mode. ● The interface must not be a member of any VLAN ● The interface must not be a member of a port-channel When the above conditions are not met when assigning the interfaces to be managed by the controller, the system returns error messages.
Since VTEP relies on service nodes to replicate BUM traffic, we need a mechanism to monitor the connectivity between the VTEP and the service nodes. BFD can be used to monitor the connectivity between the VTEP and service nodes, and detects failures. The NSX controller provides parameters, such as the minimum TX and RX interval, and the multiplier, to initiate the BFD session between the VTEP and the service nodes. To establish a BFD session, enable the BFD on the controller and the VTEP.
● Show output with details about the replicators available for the VNID. OS10# show nve replicators vnid 10009 Codes: * - Active Replicator BFD Status:Enabled Replicators State ----------------------2.2.2.3 Up 2.2.2.2* Up *— indicates the replicator to which the VTEP sends the BUM traffic for the specific VNID. Configure and control VXLAN from VMware vCenter You can configure and control VXLAN from the VMware vCenter GUI. Complete the following steps: 1.
If successfully establishing connectivity between the VTEP and the NSX controller, the console displays the current connection status between the controller and the management IP address of the VTEP. 3. Create a logical switch. You can create a logical network that acts as the forwarding domain for virtualized and nonvirtualized server workloads on the physical and virtual infrastructure. The following steps configure the logical switch for NSX controller management. a.
4. Create a logical switch port that provides a logical connection point for a VM interface (VIF) and a L2 gateway connection to an external network. 5. (Optional) Enable or disable BFD globally. The following steps enable or disable BFD configuration in the controller. a. Click Service Definitions from the left navigation pane. b. Click the Hardware Devices tab. c. Click the Edit button in the BFD Configuration. d.
After you configure a VMware NSX controller on a server VM, connect to the controller from the VXLAN gateway switch. For more information about the NSX controller configuration in the VTEP, see Configure a connection to an OVSDB controller. For more information about NSX controller configuration, see the NSX User Guide from VMware. Example: VXLAN with a controller configuration This example shows a simple NSX controller and an hardware OS10 VTEP deployed in VXLAN environment.
● Configure the NSX controller in VMware vCenter. For more information about configuring the NSX controller using the GUI, see the Configure and control VXLAN from the VMware vCenter. You must configure an OS10 VTEP with the controller configuration so that the VTEP can communicate with the NSX controller. The NSX controller handles configurations and control plane operations in the VXLAN environment. VTEP 1 1. Configure the OSPF protocol in the underlay.
3. Create an NVE instance and configure a Loopback interface as the VXLAN source tunnel interface. OS10(config)# nve OS10(config-nve)# source-interface loopback 1 4. Specify the NSX controller reachability information. OS10(config-nve)# controller ovsdb OS10(config-nve-ovsdb)# ip 10.16.140.182 port 6640 ssl OS10(config-nve-ovsdb)# max-backoff 10000 OS10(config-nve-ovsdb)# exit 5. Assign interfaces to be managed by the controller.
Replicators State ----------------------13.0.0.5 Up 13.0.0.3 Up 13.0.0.2 Up To view the remote VTEP status, use the show nve remote-vtep command. OS10# show nve remote-vtep IP Address: 13.0.0.2, State: up, Encap: VxLAN VNI list: ,6000 IP Address: 13.0.0.3, State: up, Encap: VxLAN VNI list: ,6000 IP Address: 13.0.0.5, State: up, Encap: VxLAN VNI list: ,6000 IP Address: 202.0.0.
VNI list: ,6000 IP Address: 13.0.0.3, VNI list: ,6000 IP Address: 13.0.0.5, VNI list: ,6000 IP Adress: 200.0.0.1, VNI list: 6000 State: up, Encap: VxLAN State: up, Encap: VxLAN State: up, Encap: Vxlan VXLAN Controller commands controller ovsdb Changes the mode to CONFIGURATION-NVE-OVSDB from where you can configure the controller parameters.
Supported releases 10.4.3.0 or later max-backoff Configures a time interval, in milliseconds (ms). This is the duration the switch waits between the connection attempts to the controller. Syntax max-backoff interval Parameters interval—Enter the amount of time, in ms. This is the duration the switch waits between the connection attempts to the controller, from 1000 to 180000 ms.
Parameters None Default None Command mode EXEC Usage information This command is available only for the sysadmin and secadmin roles. This command generates the SSL certificate and restarts the OVSDB server to start using the newly generated certificate. Example Supported releases OS10# nve controller ssl-key-generate 10.4.3.0 or later show nve controller Displays information about the controller and the controller-managed interfaces.
MIIDgDCCAmgCAQMwDQYJKoZIhvcNAQENBQAwgYExCzAJBgNVBAYTAlVTMQswCQYD VQQIDAJDQTEVMBMGA1UECgwMT3BlbiB2U3dpdGNoMREwDwYDVQQLDAhzd2l0Y2hj YTE7MDkGA1UEAwwyT1ZTIHN3aXRjaGNhIENBIENlcnRpZmljYXRlICgyMDE4IFNl cCAyMyAwMzo0NzoyMCkwHhcNMTgwOTI0MTYzMDUyWhcNMjgwOTIxMTYzMDUyWjCB iTELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRUwEwYDVQQKDAxPcGVuIHZTd2l0 Y2gxHzAdBgNVBAsMFk9wZW4gdlN3aXRjaCBjZXJ0aWZpZXIxNTAzBgNVBAMMLGRl bGwgaWQ6MGVlZmUwYWMtNGJjOC00MmVmLTkzOTEtN2RlMmMwY2JmMTJjMIIBIjAN BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsMlD4c4fWwy+5t6VScjiz
show ovsdb-tables mac-local-ucast Displays information about local MAC address entries including each MAC address, IP address, local switch name, and VNID. Syntax show ovsdb-tables mac-local-ucast Parameters None Default None Command mode EXEC Usage information This command is available only for netadmin, sysadmin, and secadmin roles.
Example OS10# show ovsdb-tables manager Count : 3 Manager table _uuid inactivity_probe is_connected max_backoff other_config status target ------------------------------------ ---- ------------ ---------------------- ------------------------------478ec8ca-9c5a-4d29-9069-633af6c48002 [] false 1000 {} {state=BACKOFF} "ssl:10.16.140.171:6640" 52f2b491-6372-43e0-98ed-5c4ab0ca8542 [] true 1000 {} {sec_since_connect="37831", sec_since_disconnect="37832", state=ACTIVE} "ssl:10.16.140.
4 Support resources The Dell EMC Support site provides a range of documents and tools to assist you with effectively using Dell EMC devices. Through the support site you can obtain technical information regarding Dell EMC products, access software upgrades and patches, download available management software, and manage your open cases. The Dell EMC support site provides integrated, secure access to these services. To access the Dell EMC Support site, go to www.dell.com/support/.