Dell PowerEdge Configuration Guide for the M I/O Aggregator October 2013
Notes, Cautions, and Warnings NOTE: A NOTE indicates important information that helps you make better use of your computer. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. indicates potential damage to hardware or loss of data if instructions are not followed. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. Information in this publication is subject to change without notice.
1 About this Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com 4 Aggregator Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Log Messages in the Internal Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Disabling System Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Auto-Detection of the DCBX Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 DCBx Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 DCBx Prerequisites and Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 DCBX Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Debugging DCBx on an Interface . . . . . . . . . . .
www.dell.com | support.dell.com 8 Internet Group Management Protocol (IGMP). . . . . . . . . . . . . . . . . . . . . . . . . . . 119 IGMP Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 IGMP Version 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 IGMP Version 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iSCSI Optimization: Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Default iSCSI Optimization Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 11 Link Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 How the LACP is Implemented on an Aggregator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Uplink LAG . . . . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com Important Point to Remember . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Setting up SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Creating a Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Read Managed Object Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Setting the Time for the Software Clock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Synchronizing the Hardware Clock Using the Software Clock . . . . . . . . . . . . . . . . 236 Setting the Time Zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Setting Daylight Savings Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 19 Uplink Failure Detection (UFD) . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com Displaying Stack Port Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 x Displaying Stack Member Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Application Core Dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Mini Core Dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 About this Guide Objectives This guide describes the supported protocols and software features, and provides configuration instructions and examples, for the Dell Networking M I/O Aggregator running FTOS version 8.3.17.4. The M I/O Aggregator is installed in a Dell PowerEdge M1000e Enclosure. For information about how to install and perform the initial switch configuration, refer to the Getting Started Guides on the Dell Support website at http://support.dell.com/manuals.
www.dell.com | support.dell.com Conventions This document uses the following conventions to describe command syntax: Convention Description keyword Keywords are in bold and must be entered in the CLI as listed. parameter Parameters are in italics and require a number or word to be entered in the CLI. {X} Keywords and parameters within braces must be entered in the CLI. [X] Keywords and parameters within brackets are optional.
2 Configuration Fundamentals The Dell Networking operating software (FTOS) command line interface (CLI) is a text-based interface through which you can configure interfaces and protocols. The CLI is structured in modes for security and management purposes. Different sets of commands are available in each mode, and you can limit user access to modes using privilege levels. In FTOS, after you enable a command, it is entered into the running configuration file.
www.dell.com | support.dell.com CLI Modes Different sets of commands are available in each mode. A command found in one mode cannot be executed from another mode (with the exception of EXEC mode commands preceded by the command do; for more information, refer to do Command and EXEC Privilege Mode commands). The FTOS CLI is divided into three major mode levels: • • • EXEC mode is the default mode and has a privilege level of 1, which is the most restricted level.
Navigating CLI Modes The FTOS prompt changes to indicate the CLI mode. The following table lists the CLI mode, its prompt, and information about how to access and exit this CLI mode. You must move linearly through the command modes, with the exception of the end command, which takes you directly to EXEC Privilege mode and the exit command moves you up one command mode level.
www.dell.com | support.dell.com The following illustration shows how to change the command mode from CONFIGURATION mode to INTERFACE configuration mode. Figure 2-3. Changing CLI Modes FTOS(conf)# interface tengigabitethernet 1/2 FTOS(conf-if-te-1/2)# New command prompt do Command Enter an EXEC mode or EXEC privilege mode command from any CONFIGURATION mode (such as CONFIGURATION, INTERFACE, etc.) without returning to EXEC mode by preceding the EXEC mode command with the command do.
Figure 2-5. Undoing a command with the no Command FTOS(conf)# interface managementethernet 0/0 FTOS(conf-if-ma-0/0)# ip address 192.168.5.6/16 Assign an IP address FTOS(conf-if-ma-0/0)# FTOS(conf-if-ma-0/0)# FTOS(conf-if-ma-0/0)#show config ! interface ManagementEthernet 0/0 ip address 192.168.5.
www.dell.com | support.dell.com Figure 2-7. Keyword? Command Example FTOS(conf)#cl? clock FTOS(conf)#cl • partial keyword plus “?” for matching keywords A keyword followed by [space]? lists all of the keywords that can follow the specified keyword. Figure 2-8.
Table 2-2. Short-Cut Keys and their Actions (continued) Key Combination Action CNTL-P Recalls commands, beginning with the last command. CNTL-R Re-enters the previous command. CNTL-U Deletes the line. CNTL-W Deletes the previous word. CNTL-X Deletes the line. CNTL-Z Ends continuous scrolling of command outputs. Esc B Moves the cursor back one word. Esc F Moves the cursor forward one word. Esc D Deletes all characters from the cursor to the end of the word.
www.dell.com | support.dell.com Figure 2-9. Filtering Command Outputs with the grep Command FTOS(conf)#do show stack-unit stack unit 0 stack-port all 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause all stack-ports all pfc details | grep 0 Rx Rx Rx Rx Rx Rx pkts pkts pkts pkts pkts pkts Note: FTOS accepts a space or no space before and after the pipe.
Multiple Users in Configuration Mode FTOS notifies all users in the event that there are multiple users logged into CONFIGURATION mode. A warning message indicates the username, type of connection (console or vty), and in the case of a vty connection, the IP address of the terminal on which the connection was established.
12 | Configuration Fundamentals www.dell.com | support.dell.
3 Getting Started This chapter contains the following sections: • • • • • • • • • • • Front Panel Port Numbering Console access Boot Process Configure a Host Name Access the System Remotely Configure the Enable Password Configuration File Management File System Management View the Command History Upgrading FTOS When the boot process is complete, the console monitor displays the Dell Networking operating software (FTOS) banner and EXEC mode prompt (Figure 3-3).
www.dell.com | support.dell.com Front Panel The following example shows the I/O Aggregator (also known as aggregator) front panel: Figure 3-1. Front Panel of the M I/O Aggregator Flex IO Module in Top Expansion Slot Flex IO Module in Bottom Expansion Slot USB Storage Port 40GbE QSFP+ Ports on Base Module USB Console Port Port Numbering When installed in a PowerEdge M1000e Enclosure, Aggregator ports are numbered 1 to 56 and consist of internal server-facing ports, uplink ports, and stacking ports.
Uplink Ports Ports 33 to 56 are external ports used for uplinks and numbered from the bottom to the top of the switch as follows: • • • The two base module ports operate by default in standalone 4x10GbE mode and are numbered 33 to 36 and 37 to 40. Ports on the 2-Port 40-GbE QSFP+ module operate only in 4x10GbE mode: • In the bottom expansion slot, ports are numbered 41 to 44 and 45 to 48. • In the top expansion slot, ports are numbered 49 to 52 and 53 to 56.
www.dell.com | support.dell.com Figure 3-2.
Step Task (continued) Note: Terminal settings on the console port cannot be changed in the software and are set as follows: • • • • • 9600 baud rate No parity 8 data bits 1 stop bit No flow control External Serial Port with a USB Connector The following table lists the pin assignments. Table 3-1.
www.dell.com | support.dell.com Figure 3-3. Completed Boot Process syncing disks... done unmounting file systems... unmounting /f10/flash (/dev/ld0e)... unmounting /usr (mfs:31)... unmounting /lib (mfs:23)... unmounting /f10 (mfs:20)... unmounting /tmp (mfs:15)... unmounting /kern (kernfs)... unmounting / (/dev/md0a)... done rebooting... þ NetLogic XLP Stage 1 Loader Built by build at tools-sjc-01 on Fri Mar 16 Navasota IOM Boot Selector Label 4.0.0.
Figure 3-4. Completed Boot Process (Contd.) ########################################## # # # u-boot: for Navasota board # # # ########################################## Initialized CPLD on CS3 Detected [XLP308 (Lite) Rev A0] CPLD reg 06 val 0xf7 This is a NAVASOTA ... Initializing I2C0: speed = 30 KHz, prescaler = 0x0377 -- done. Initializing I2C1: speed = 100 KHz, prescaler = 0x0109 -- done.
www.dell.com | support.dell.com Figure 3-5. 20 Completed Boot Process (Contd.) FTOS>00:00:30: %STKUNIT0-M:CP %IFMGR-5-IFM_ISCSI_ENABLE: iSCSI has been enabled causing flow control to be enabled on all interfaces.
Figure 3-6. Completed Boot Process (Contd.
www.dell.com | support.dell.com Figure 3-7. Completed Boot Process (Contd.
Access the Aggregator Remotely Configuring the Aggregator for Telnet access is a three-step process: 1. Configure an IP address for the management port. Refer to Configure the Management Port IP Address. 2. Configure a management route with a default gateway. Refer to Configure a Management Route. 3. Configure a username and password. Refer to Configure a Username and Password. Configure the Management Port IP Address Assign IP addresses to the management ports in order to access the system remotely.
www.dell.com | support.dell.com Configure a Management Route Define a path from the system to the network from which you are accessing the system remotely. Management routes are separate from IP routes and are only used to manage the system through the management port. To configure a management route, follow this step: Step 1 Task Command Syntax Command Mode Configure a management route to the network from which you are accessing the system.
Dell Networking recommends using the enable secret password. To configure an enable password: Task Command Syntax Command Mode Create a password to access EXEC Privilege mode. enable [password | secret] [level level] [encryption-type] password CONFIGURATION level is the privilege level, is 15 by default, and is not required. encryption-type specifies how you are inputting the password, is 0 by default, and is not required. • • • 0 is for inputting the password in clear text.
www.dell.com | support.dell.com Copy Files to and from the System The command syntax for copying files is similar to UNIX. The copy command uses the format copy source-file-url destination-file-url. Note: For a detailed description of the copy command, refer to the FTOS Command Line Reference Guide. • • To copy a local file to a remote system, combine the file-origin syntax for a local file location with the file-destination syntax for a remote file location (Table 3-2).
Figure 3-9. Copying a file to a Remote System Local Location Remote Location FTOS#copy flash://FTOS-EF-8.2.1.0.bin ftp://myusername:mypassword@10.10.10.10//FTOS/FTOS-EF-8.2.1.0 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 27952672 bytes successfully copied Figure 3-10 shows an example of using the copy command to import a file to the Dell Networking system from an FTP server. Figure 3-10.
www.dell.com | support.dell.
Figure 3-11. Restoring the Factory Default Settings FTOS#restore factory-defaults stack-unit 0 nvram *********************************************************************** * Warning - Restoring factory defaults will delete the existing * * persistent settings (stacking, fanout, etc.) * * After restoration the unit(s) will be powercycled immediately.
www.dell.com | support.dell.com Figure 3-12.
Figure 3-13. Tracking Changes with Configuration Comments FTOS#show running-config Current Configuration ... ! Version E8-3-17-38 ! Last configuration change at Tue Jul 24 20:33:08 2012 by default ! boot system stack-unit 1 primary tftp://10.11.9.21/dv-m1000e-2-b2 boot system stack-unit 1 default system: A: boot system gateway 10.11.209.
www.dell.com | support.dell.com You can change the default file system so that file management commands apply to a particular device or memory. To change the default storage location: Task Command Syntax Command Mode Change the default directory. cd directory EXEC Privilege You can change the default storage location to the USB Flash (Figure 3-15). File management commands then apply to the USB Flash rather than the internal Flash. Figure 3-15.
Figure 3-16. show command-history Command Example FTOS# show command-history [5/18 21:58:32]: CMD-(TEL0):[enable]by admin from vty0 (10.11.68.5) [5/18 21:58:48]: CMD-(TEL0):[configure]by admin from vty0 (10.11.68.5) - Repeated 1 time. [5/18 21:58:57]: CMD-(TEL0):[interface port-channel 1]by admin from vty0 (10.11.68.5) [5/18 21:59:9]: CMD-(TEL0):[show config]by admin from vty0 (10.11.68.5) [5/18 22:4:32]: CMD-(TEL0):[exit]by admin from vty0 (10.11.68.
34 | Getting Started www.dell.com | support.dell.
4 Aggregator Management This chapter explains the different protocols or services used to manage an Aggregator including: • • • • • • • • Logging Disabling System Logging File Transfer Services Terminal Lines Telnet to Another Network Device Recovering from a Forgotten Password Recovering from a Forgotten Enable Password Recovering from a Failed Start Logging FTOS tracks changes in the system using event and error messages.
www.dell.com | support.dell.com Disabling System Logging By default, logging is enabled and log messages are sent to the logging buffer, all terminal lines, console, and syslog servers. To enable and disable system logging: Task Command Syntax Command Mode Disable all logging except on the console. no logging on CONFIGURATION Disable logging to the logging buffer. no logging buffer CONFIGURATION Disable logging to terminal lines. no logging monitor CONFIGURATION Disable console logging.
Changing System Logging Settings You can change the default settings of the system logging by changing the severity level and the storage location. The default is to log all messages up to debug level, that is, all system messages. By changing the severity level in the logging commands, you control the number of system messages logged.
www.dell.com | support.dell.com Displaying the Logging Buffer and Logging Configuration To display the current contents of the logging buffer and the logging settings for the system, enter the show logging command in EXEC privilege mode (Figure 4-1). Figure 4-1.
Configuring a UNIX Logging Facility Level You can save system log messages with a UNIX system logging facility. To configure a UNIX logging facility level, use the following command in CONFIGURATION mode: Command Syntax Command Mode Purpose logging facility [facility-type] CONFIGURATION Specify one of the following parameters.
www.dell.com | support.dell.com Enabling Time Stamps on Syslog Messages By default, syslog messages do not include a time/date stamp stating when the error or message was created. To have FTOS include a timestamp with the syslog message, use the following command syntax in CONFIGURATION mode: Command Syntax Command Mode Purpose service timestamps [log | debug] [datetime [localtime] [msec] [show-timezone] | uptime] CONFIGURATION Add timestamp to syslog messages.
Enabling the FTP Server To enable the system as an FTP server, use the following command in CONFIGURATION mode: Command Syntax Command Mode Purpose ftp-server enable CONFIGURATION Enable FTP on the system. To view the FTP configuration, enter the show running-config ftp command in EXEC privilege mode (Figure 4-3). Figure 4-3.
www.dell.com | support.dell.com Terminal Lines You can access the system remotely and restrict access to the system by creating user profiles. The terminal lines on the system provide different means of accessing the system. The virtual terminal lines (VTY) connect you through Telnet to the system. Telnet to Another Network Device To telnet to another device (Figure 4-4): Task Command Syntax Command Mode Telnet to the stack-unit.
Recovering from a Forgotten Password If you configure authentication for the console and you exit out of EXEC mode or your console session times out, you are prompted to re-enter the password. If you forget your password, follow these steps: Step Task Command Syntax Command Mode 1 Log onto the system using the console. 2 Power-cycle the Aggregator by using the CMC interface or removing it from the M1000e and re-inserting it in the chassis.
www.dell.com | support.dell.com Step Task Command Syntax Command Mode 4 Set the system parameters to ignore the enable password when the system reloads. ignore enable-password BOOT USER 5 Reload the system. reload BOOT USER 6 Configure a new enable password. copy flash://startup-config.bak running-config EXEC Privilege 7 Configure a new enable password. enable {secret | password) CONFIGURATION 8 Save the running-config to the startup-config.
Figure 4-5. Recovering from a Failed Start: Example U-Boot 2010.03-rc1(Dell Force10) Built by build at tools-sjc-01 on Thu May 31 23:53:38 2012 IOM Boot Label 4.0.1.0 DRAM: 2 GB Initialized CPLD on CS3 Detected [XLP308 (Lite+) Rev A0] Initializing I2C0: speed = 30 KHz, prescaler = 0x0377 -- done. Initializing I2C1: speed = 100 KHz, prescaler = 0x0109 -- done.
46 | Aggregator Management www.dell.com | support.dell.
5 Data Center Bridging (DCB) On an I/O Aggregator, data center bridging (DCB) features are auto-configured in standalone mode. You can display information on DCB operation by using show commands. Note: DCB features are not supported on an Aggregator in stacking mode.
www.dell.com | support.dell.com Data center bridging satisfies the needs of the following types of data center traffic in a unified fabric: • • • LAN traffic consists of a large number of flows that are generally insensitive to latency requirements, while certain applications, such as streaming video, are more sensitive to latency. Ethernet functions as a best-effort network that may drop packets in case of network congestion.
Figure 5-1. Priority-Based Flow Control PFC is implemented as follows in the Dell Networking operating software (FTOS): • • • • • • • • • PFC supports buffering to receive data that continues to arrive on an interface while the remote system reacts to the PFC operation. PFC uses the DCB MIB IEEE802.1azd2.5 and the PFC MIB IEEE802.1bb-d2.2. PFC is supported on specified 802.1p priority traffic (dot1p 0 to 7) and is configured per interface.
www.dell.com | support.dell.com • PFC uses the DCB MIB IEEE802.1azd2.5 and the PFC MIB IEEE802.1bb-d2.2. If DCBx negotiation is not successful (for example, due to a version or TLV mismatch), DCBx is disabled and you cannot enable PFC or ETS. Configuring Priority-Based Flow Control Priority-based flow control (PFC) provides a flow control mechanism based on the 802.1p priorities in converged Ethernet traffic received on an interface and is enabled by default when you enable DCB.
Step Task Command Syntax Command Mode 4 Enable the PFC configuration on the port so that the priorities are included in DCBx negotiation with peer PFC devices. The default is PCFC mode is on. pfc mode on DCB INPUT POLICY 5 (Optional) Enter a text description of the input policy. The maximum is 32 characters. description text DCB INPUT POLICY 6 Exit DCB input policy configuration mode. exit DCB INPUT POLICY 7 Enter interface configuration mode.
www.dell.com | support.dell.com • • You can enable link-level flow control on the interface (refer to Ethernet Pause Frames). To delete the input policy, first disable link-level flow control. PFC is then automatically enabled on the interface because an interface is by default PFC-enabled. PFC still allows you to configure lossless queues on a port to ensure no-drop handling of lossless traffic (refer to Configuring Lossless Queues).
Figure 5-2. Enhanced Transmission Selection ETS uses the following traffic groupings to select multiprotocol traffic for transmission: • • • • Priority group: A group of 802.1p priorities used for bandwidth allocation and queue scheduling. All 802.1p priority traffic in a group must have the same traffic handling requirements for latency and frame loss. Group ID: A 4-bit identifier assigned to each priority group. The range is from 0 to 7.
www.dell.com | support.dell.com Different traffic types have different service needs. Using ETS, you can create groups within an 802.1p priority class to configure different treatment for traffic with different bandwidth, latency, and best-effort needs. For example, storage traffic is sensitive to frame loss; interprocess communication (IPC) traffic is latency-sensitive.
Figure 5-3.
www.dell.com | support.dell.com • • A DCB input policy with PFC disabled A DCB output policy with ETS disabled Figure 5-4 shows a default interface configuration with DCB enabled and link-level flow control enabled. When the first Aggregator interface with DCB disabled receives an LLDP packet with a DCBx TLV advertisement, DCB is enabled on the interface and on all uplink interfaces. Figure 5-4.
Lossless Traffic Handling In auto-DCB-enable mode, Aggregator ports operate with the auto-detection of DCBx traffic. At any moment, some ports may operate with link-level flow control while others operate with DCB-based PFC enabled. As a result, lossless traffic is ensured only if traffic ingresses on a PFC-enabled port and egresses on another PFC-enabled port.
www.dell.com | support.dell.com QoS dot1p Traffic Classification and Queue Assignment DCB supports PFC, ETS, and DCBx to handle converged Ethernet traffic that is assigned to an egress queue according to the following quality of service (QoS) methods: • • Important: of two Honor dot1p: dot1p priorities in ingress traffic are used at the port or global switch level.
How Priority-Based Flow Control is Implemented Priority-based flow control provides a flow control mechanism based on the 802.1p priorities in converged Ethernet traffic received on an interface and is enabled by default. As an enhancement to the existing Ethernet pause mechanism, PFC stops traffic transmission for specified priorities (CoS values) without impacting other priority classes. Different traffic types are assigned to different priority classes.
www.dell.com | support.dell.com • • • • • • • • Traffic in priority groups is assigned to strict-queue or WERR scheduling in an ETS output policy and is managed using the ETS bandwidth-assignment algorithm. FTOS de-qeues all frames of strict-priority traffic before servicing any other queues. A queue with strict-priority traffic can starve other queues in the same port. ETS-assigned bandwidth allocation and scheduling apply only to data queues, not to control queues.
• • New ETS configurations are ignored and existing ETS configurations are reset to the previously configured ETS output policy on the port or to the default ETS settings if no ETS output policy was previously applied. ETS operates with legacy DCBx versions as follows: • In the CEE version, the priority group/traffic class group (TCG) ID 15 represents a non-ETS priority group.
www.dell.com | support.dell.com DCBx Operation DCBx performs the following operations: • • • • Discovers DCB configuration (such as PFC and ETS) in a peer device. Detects DCB misconfiguration in a peer device; that is, when DCB features are not compatibly configured on a peer device and the local switch. Misconfiguration detection is feature-specific because some DCB features support asymmetric configuration.
• If the received peer configuration is not compatible with the currently configured port configuration, the link with the DCBx peer port is disabled and a syslog message for an incompatible configuration is generated. The network administrator must then reconfigure the peer device so that it advertises a compatible DCB configuration. The internally propagated configuration is not stored in the switch's running configuration.
www.dell.com | support.dell.com Configuration Source Election When an auto-upstream or auto-downstream port receives a DCB configuration from a peer, the port first checks to see if there is an active configuration source on the switch. • • If a configuration source already exists, the received peer configuration is checked against the local port configuration. If the received configuration is compatible, the DCBx marks the port as DCBx-enabled.
Auto-Detection of the DCBX Version The Aggregator operates in auto-detection mode so that a DCBx port automatically detects the DCBx version on a peer port. Legacy CIN and CEE versions are supported in addition to the standard IEEE version 2.5 DCBX. A DCBx port detects a peer version after receiving a valid frame for that version.
www.dell.com | support.dell.com Figure 5-7. DCBX Sample Topology DCBx Prerequisites and Restrictions • • 66 | DCBx requires LLDP in both send (TX) and receive (RX) mode to be enabled on a port interface. If multiple DCBx peer ports are detected on a local DCBx interface, LLDP is shut down. The CIN version of DCBx supports only PFC, ETS, and FCOE; it does not support iSCSI, backward congestion management (BCN), logical link down (LLD), and network interface virtualization (NIV).
DCBX Error Messages An error in DCBx operation is displayed using the following syslog messages: LLDP_MULTIPLE_PEER_DETECTED: DCBX is operationally disabled after detecting more than one DCBX peer on the port interface. LLDP_PEER_AGE_OUT: DCBX is disabled as a result of LLDP timing out on a DCBX peer interface. DSM_DCBX_PEER_VERSION_CONFLICT: A local port expected to receive the IEEE, CIN, or CEE version in a DCBX TLV from a remote peer but received a different, conflicting DCBX version.
www.dell.com | support.dell.com Verifying DCB Configuration Use the show commands in Table 5-2 to display DCB configurations and statistics. Table 5-2. Displaying DCB Configurations Command Output show dcb [stack-unit unit-number] (Figure 5-8) Displays data center bridging status, number of PFC-enabled ports, and number of PFC-enabled queues. On the master switch in a stack, you can specify a stack-unit number. Valid values: 0 to 5.
Figure 5-10.
www.dell.com | support.dell.com Table 5-3. 70 show interface pfc summary Command Description Field | Description Remote is enabled, Priority list Remote Willing Status is enabled Operational status (enabled or disabled) of peer device for DCBx exchange of PFC configuration with a list of the configured PFC priorities. Willing status of peer device for DCBx exchange (Willing bit received in PFC TLV): enabled or disabled.
Table 5-3. show interface pfc summary Command Description Field Description Input Appln Priority TLV pkts Number of Appln Priority TLVs received. Output Appln Priority TLV pkts Number of Appln Priority TLVs transmitted. Error Appln Priority TLV pkts Number of Appln Priority error packets received.
www.dell.com | support.dell.com Figure 5-11.
FTOS# show interfaces tengigabitethernet 0/34 ets Interface TenGigabitEthernet 0/34 Max Supported PG is 4 Number of Traffic Classes is 8 Admin mode is on detail Admin Parameters : -----------------Admin is enabled PG-grp Priority# Bandwidth TSA -----------------------------------------------0 0,1,2,3,4,5,6,7 100% ETS 1 2 3 4 5 6 7 Remote Parameters : ------------------Remote is disabled Local Parameters : -----------------Local is enabled PG-grp Priority# Bandwidth TSA ------------------------------------
www.dell.com | support.dell.com Table 5-4. 74 show interface ets detail Command Description Field | Description Admin mode ETS mode: on or off. When on, the scheduling and bandwidth allocation configured in an ETS output policy or received in a DCBx TLV from a peer can take effect on an interface. Admin Parameters ETS configuration on local port, including priority groups, assigned dot1p priorities, and bandwidth allocation.
Figure 5-12.
www.dell.com | support.dell.com Figure 5-14.
Example: PFC and ETS Operation Table 5-5. show interface dcbx detail Command Description Field Description Interface Interface type with chassis slot and port number. Port-Role Configured DCBx port role: auto-upstream or auto-downstream. DCBx Operational Status Operational status (enabled or disabled) used to elect a configuration source and internally propagate a DCB configuration. The DCBx operational status is the combination of PFC and ETS operational status.
www.dell.com | support.dell.com Table 5-5. show interface dcbx detail Command Description Field Description Total DCBX Frame errors Number of DCBx frames with errors received. Total DCBX Frames unrecognized Number of unrecognizable DCBx frames received. PFC TLV Statistics: Input PFC TLV pkts Number of PFC TLVs received. Output PFC TLV pkts Number of PFC TLVs transmitted. Error PFC pkts Number of PFC error packets received. PFC Pause Tx pkts Number of PFC pause frames transmitted.
Figure 5-15. Example: PFC and ETS Applied to LAN, IPC, and SAN Priority Traffic dot1p priority 3 dot1p priority 0, 1, 2, 4, 5 Converged LAN, SAN, and IPC traffic dot1p priority 0-7 SAN priority 3 IPC priority 4 QoS Traffic Classification: On the Aggregator, the service-class dynamic dot1p command has been used in Global Configuration mode to map ingress dot1p frames to the queues shown in Table 5-6. For more information, refer to dcb enable auto-detect on-next-reload Command Example.
www.dell.com | support.dell.com Table 5-6. Example: dot1p-Queue Assignment dot1p Value in Incoming Frame Queue Assignment 0 0 1 0 2 0 3 1 4 2 5 3 6 3 7 3 Lossless SAN traffic with dot1p priority 3 is assigned to queue 1. Other traffic types are assigned the 802.1p priorities shown in Table 5-7 and the bandwidth allocations shown in Table 5-8. Table 5-7.
Table 5-8. Example: priority group-bandwidth Assignment Priority Group Bandwidth Assignment IPC 5% SAN 50% LAN 45% Hierarchical Scheduling in ETS Output Policies On an Aggregator, ETS supports up to three levels of hierarchical scheduling. For example, ETS output policies with the following configurations can be applied: • • • Priority group 1 assigns traffic to one priority queue with 20% of the link bandwidth and strict-priority scheduling.
82 | Data Center Bridging (DCB) www.dell.com | support.dell.
Skippy812 6 Dynamic Host Configuration Protocol (DHCP) The Aggregator is auto-configured to operate as a DHCP client. The DHCP server, DHCP relay agent, and secure DHCP features are not supported. DHCP Overview Dynamic host configuration protocol (DHCP) is an application layer protocol that dynamically assigns IP addresses and other configuration parameters to network end-stations (hosts) based on configuration policies determined by network administrators.
www.dell.com | support.dell.com DHCP Packet Format and Options DHCP uses the user datagram protocol (UDP) as its transport protocol. The server listens on port 67 and transmits to port 68; the client listens on port 68 and transmits to port 67. The configuration parameters are carried as options in the DHCP packet in type, length, value (TLV) format; many options are specified in RFC 2132.
Assigning an IP Address Using DHCP When a client joins a network: 1. The client initially broadcasts a DHCPDISCOVER message on the subnet to discover available DHCP servers. This message includes the parameters that the client requires and might include suggested values for those parameters. 2. Servers unicast or broadcast a DHCPOFFER message in response to the DHCPDISCOVER that offers to the client values for the requested parameters.
www.dell.com | support.dell.com FTOS Behavior: DHCP is implemented in FTOS based on RFC 2131 and 3046. DHCP Client An Aggregator is auto-configured to operate as a DHCP client. The DHCP client functionality is enabled only on the default VLAN and the management interface. A DHCP client is a network device that requests an IP address and configuration parameters from a DHCP server.
Releasing and Renewing DHCP-based IP Addresses On an Aggregator configured as a DHCP client, you can release a dynamically-assigned IP address without removing the DHCP client operation on the interface. You can later manually acquire a new IP address from the DHCP server as follows: Task Command Syntax Command Mode Release a dynamically-acquired IP address while retaining the DHCP client configuration on the interface.
www.dell.com | support.dell.com Figure 6-4. show ip dhcp lease FTOS# show ip dhcp Interface ========= Ma 0/0 Vl 1 Lease-IP Def-Router ======== ========= 0.0.0.0/0 0.0.0.0 10.1.1.254/24 0.0.0.0 ServerId ======== 0.0.0.0 10.1.1.
Figure 6-5.
www.dell.com | support.dell.com Figure 6-6 shows an example of the packet- and event-level debug messages displayed for the packet transmissions and state transitions on a DHCP client interface when you release and renew a DHCP client. Figure 6-6.
DHCP Client on a Management Interface The following conditions apply on a management interface that operates as a DHCP client: • • • • • • • The management default route is added with the gateway as the router IP address received in the DHCP ACK packet. This is required to send and receive traffic to and from other subnets on the external network. This route is added irrespective both when the DHCP client and server are in the same or different subnets.
www.dell.com | support.dell.com DHCP Client on a VLAN The following conditions apply on a VLAN that operates as a DHCP client: • • The default VLAN 1 with all ports auto-configured as members is the only L3 interface on the Aggregator. When the default management VLAN has a DHCP-assigned address and you reconfigure the default VLAN ID number, the Aggregator: • Sends a DHCP release to the DHCP server to release the IP address. • Sends a DHCP request to obtain a new IP address.
Configure Secure DHCP DHCP as defined by RFC 2131 provides no authentication or security mechanisms. Secure DHCP is a suite of features that protects networks that use dynamic address allocation from spoofing and attacks. • • • • Option 82 DHCP Snooping Dynamic ARP Inspection Source Address Validation Option 82 RFC 3046 (the relay agent information option, or Option 82) is used for class-based IP address assignment.
www.dell.com | support.dell.com DHCP Snooping DHCP snooping protects networks from spoofing. In the context of DHCP snooping, all ports are either trusted or untrusted. By default, all ports are untrusted. Trusted ports are ports through which attackers cannot connect. Manually configure ports connected to legitimate servers and relay agents as trusted.
Add a Static Entry in the Binding Table To add a static entry in the binding table, follow this step: Task Command Syntax Command Mode Add a static entry in the binding table.
www.dell.com | support.dell.com To view the DHCP snooping statistics, use the show ip dhcp snooping command (Figure 6-7). Figure 6-7. Command example: show ip dhcp snooping FTOS#show ip dhcp snooping IP IP IP IP DHCP DHCP DHCP DHCP Snooping Snooping Mac Verification Relay Information-option Relay Trust Downstream : : : : Disabled. Disabled. Disabled. Enabled.
To view the number of entries in the table, use the show ip dhcp snooping binding command. This output displays the snooping binding table created using the ACK packets from the trusted port (Figure 6-8). Figure 6-8. Command example: show ip dhcp snooping binding FTOS#show ip dhcp snooping binding Codes : S - Static D - Dynamic IP Address MAC Address Expires(Sec) Type VLAN Interface ======================================================================== 10.1.1.
www.dell.com | support.dell.com • denial of service—an attacker can send fraudulent ARP messages to a client to associate a false MAC address with the gateway address, which blackholes all internet-bound packets from the client. Note: Dynamic ARP inspection (DAI) uses entries in the L2SysFlow CAM region, a sub-region of SystemFlow. One CAM entry is required for every DAI-enabled VLAN. You can enable DAI on up to 16 VLANs on a system.
To see how many valid and invalid ARP packets have been processed, use the show arp inspection statistics command (Figure 6-10). Figure 6-10.
www.dell.com | support.dell.com IP Source Address Validation IP source address validation prevents IP spoofing by forwarding only IP packets that have been validated against the DHCP binding table. A spoofed IP packet is one in which the IP source address is strategically chosen to disguise the attacker. For example, using ARP spoofing, an attacker can assume a legitimate client’s identity and receive traffic addressed to it.
To enable IP+MAC source address validation, follow these steps: Step Task Command Syntax Command Mode 1 Allocate at least one FP block to the ipmacacl CAM region. cam-acl l2acl CONFIGURATION 2 Save the running-config to the startup-config. copy running-config startup-config EXEC Privilege 3 Reload the system. reload EXEC Privilege 4 Enable IP+MAC Source Address Validation.
www.dell.com | support.dell.
7 FIP Snooping FIP snooping is auto-configured on an Aggregator in standalone mode. You can display information on FIP snooping operation and statistics by entering show commands.
www.dell.com | support.dell.com To ensure similar Fibre Channel robustness and security with FCoE in an Ethernet cloud network, the Fibre Channel over Ethernet initialization protocol (FIP) establishes virtual point-to-point links between FCoE end-devices (server ENodes and target storage devices) and FCoE forwarders (FCFs) over transit FCoE-enabled bridges.
Figure 7-1. FIP discovery and login between an ENode and an FCF FIP Snooping on Ethernet Bridges In a converged Ethernet network, intermediate Ethernet bridges can snoop on FIP packets during the login process on an FCF. Then, using ACLs, a transit bridge can permit only authorized FCoE traffic to be transmitted between an FCoE end-device and an FCF. An Ethernet bridge that provides these functions is called a FIP snooping bridge (FSB).
www.dell.com | support.dell.com Dynamic ACL generation on an Aggregator operating as a FIP snooping bridge functions as follows: • • • • Global ACLs are applied on server-facing ENode ports. Port-based ACLs are applied on ports directly connected to an FCF and on server-facing ENode ports. Port-based ACLs take precedence over global ACLs. FCoE-generated ACLs take precedence over user-configured ACLs. A user-configured ACL entry cannot deny FCoE and FIP snooping frames.
The following sections describe how to configure the FIP snooping feature on a switch that functions as a FIP snooping bridge so that it can perform the following functions: • • • • • Perform FIP snooping (allowing and parsing FIP frames) globally on all VLANs or on a per-VLAN basis. Set the FCoE MAC address prefix (FC-MAP) value used by an FCF to assign a MAC address to an FCoE end-device (server ENode or storage device) after a server successfully logs in.
www.dell.com | support.dell.com FIP Snooping on VLANs FIP snooping is enabled globally on an Aggregator on all VLANs: • • • • FIP frames are allowed to pass through the switch on the enabled VLANs and are processed to generate FIP snooping ACLs. FCoE traffic is allowed on VLANs only after a successful virtual-link initialization (fabric login FLOGI) between an ENode and an FCF. All other FCoE traffic is dropped.
FIP Snooping Prerequisites On an Aggregator, FIP snooping requires the following conditions: • • A FIP snooping bridge requires DCBX and PFC to be enabled on the switch for lossless Ethernet connections (refer to Data Center Bridging (DCB)). Dell recommends that you also enable ETS; ETS is recommended but not required. DCBX and PFC mode are auto-configured on Aggregator ports and FIP snooping is operational on the port.
www.dell.com | support.dell.com Displaying FIP Snooping Information Use the show commands in Table 7-1 to display information on FIP snooping. Table 7-1.
Figure 7-3. show fip-snooping sessions Command Example FTOS#show fip-snooping sessions Enode MAC Enode Intf aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 FCoE MAC 0e:fc:00:01:00:01 0e:fc:00:01:00:02 0e:fc:00:01:00:03 0e:fc:00:01:00:04 0e:fc:00:01:00:05 Table 7-2.
www.dell.com | support.dell.com Figure 7-5. show fip-snooping enode Command Example FTOS# show fip-snooping enode Enode MAC Enode Interface ----------------------d4:ae:52:1b:e3:cd Te 0/11 Table 7-3. FCF MAC ------54:7f:ee:37:34:40 Description ENode MAC MAC address of the ENode. ENode Interface Slot/ port number of the interface connected to the ENode. FCF MAC MAC address of the FCF. VLAN VLAN ID number used by the session. FC-ID Fibre Channel session ID assigned by the FCF.
Figure 7-7.
www.dell.com | support.dell.com Figure 7-8.
Table 7-5. show fip-snooping statistics Command Descriptions Field Description Number of Vlan Requests Number of FIP-snooped VLAN request frames received on the interface. Number of VLAN Notifications Number of FIP-snooped VLAN notification frames received on the interface. Number of Multicast Discovery Solicits Number of FIP-snooped multicast discovery solicit frames received on the interface.
www.dell.com | support.dell.com Figure 7-9. show fip-snooping system Command Example FTOS# show fip-snooping system Global Mode FCOE VLAN List (Operational) FCFs Enodes Sessions : : : : : Enabled 1, 100 1 2 17 Note: NPIV sessions are included in the number of FIP-snooped sessions displayed. Figure 7-10.
FIP Snooping Example Figure 7-11 shows an Aggregator used as a FIP snooping bridge for FCoE traffic between an ENode (server blade) and an FCF (ToR switch). The ToR switch operates as an FCF and FCoE gateway. Figure 7-11. Example: FIP Snooping on an Aggregator In Figure 7-11, DCBX and PFC are enabled on the Aggregator (FIP snooping bridge) and on the FCF ToR switch. On the FIP snooping bridge, DCBX is configured as follows: • • A server-facing port is configured for DCBX in an auto-downstream role.
www.dell.com | support.dell.com Debugging FIP Snooping To enable debug messages for FIP snooping events, enter the debug fip-snooping command. Task Command Command Mode Enable FIP snooping debugging on for all or a specified event type, where: all enables all debugging options. acl enables debugging only for ACL-specific events. error enables debugging only for error conditions. ifm enables debugging only for IFM events. info enables debugging only for information events.
8 Internet Group Management Protocol (IGMP) On an Aggregator, IGMP snooping is auto-configured. You can display information on IGMP by using show commands. Multicast is based on identifying many hosts by a single destination IP address. Hosts represented by the same IP address are a multicast group. The internet group management protocol (IGMP) is a Layer 3 multicast protocol that hosts use to join or leave a multicast group.
www.dell.com | support.dell.com Figure 8-1. IGMP Version 2 Packet Format Preamble IHL Version (4) TOS (0xc0) Total Length Start Frame Delimiter Destination MAC Flags Frag Offset Source MAC TTL (1) Protocol (2) Padding IP Packet Ethernet Type Header Checksum Src IP Addr Dest IP Addr FCS Options (Router Alert) Type Padding Max.
IGMP Version 3 Conceptually, IGMP version 3 behaves the same as version 2. However, there are differences: • Version 3 adds the ability to filter by multicast source, which helps the multicast routing protocols avoid forwarding traffic to subnets where there are no interested receivers. To enable filtering, routers must keep track of more state information, that is, the list of sources that must be filtered.
www.dell.com | support.dell.com Joining and Filtering Groups and Sources Figure 8-4 shows how multicast routers maintain the group and source information from unsolicited reports. 1. The first unsolicited report from the host indicates that it wants to receive traffic for group 224.1.1.1. 2. The host’s second report indicates that it is only interested in traffic from group 224.1.1.1, source 10.11.1.1.
Leaving and Staying in Groups Figure 8-5 shows how multicast routers track and refresh state changes in response to group-and-specific and general queries. 1. Host 1 sends a message indicating it is leaving group 224.1.1.1 and that the included filter for 10.11.1.1 and 10.11.1.2 are no longer necessary. 2. The querier, before making any state changes, sends a group-and-source query to see if any other host is interested in these two sources; queries for state-changes are retransmitted multiple times.
www.dell.com | support.dell.com How IGMP Snooping is Implemented on an Aggregator • • • • • • • • • • IGMP snooping is enabled by default on the switch. FTOS supports version 1, version 2, and version 3 hosts. FTOS IGMP snooping is based on the IP multicast address (not on the Layer 2 multicast MAC address). IGMP snooping entries are stored in the Layer 3 flow table instead of in the Layer 2 forwarding information base (FIB). FTOS IGMP snooping is based on draft-ietf-magma-snoop-10.
Figure 8-6. show ip igmp groups Command Example FTOS# show ip igmp groups Total Number of Groups: 2 IGMP Connected Group Membership Group Address Interface 226.0.0.1 Vlan 1500 226.0.0.1 Vlan 1600 FTOS#show ip igmp groups detail Interface Group Uptime Expires Router mode Last reporter Last reporter mode Last report received Group source list Source address 1.1.1.2 Member Ports: Po 1 Vlan 1500 226.0.0.1 00:00:21 Never INCLUDE 1.1.1.
www.dell.com | support.dell.com Figure 8-7.
9 Interfaces This chapter describes the auto-configuration of 1 Gigabit (1GbE) and 10 Gigabit Ethernet (10GbE) interfaces (physical and logical) on an I/O Aggregator.
www.dell.com | support.dell.com Interface Auto-Configuration An Aggregator auto-configures interfaces as follows: • • • • All interfaces operate as layer 2 interfaces at 10GbE in standalone mode. FlexIO module interfaces support only uplink connections. You can only use the 40GbE ports on the base module for stacking. • By default, the two fixed 40GbE ports on the base module operate in 4x10GbE mode with breakout cables and support up to eight 10GbE uplinks.
Viewing Interface Information To view interface status and auto-configured parameters use show commands. The show interfaces command in EXEC mode lists all configurable interfaces on the chassis and has options to display the interface status, IP and MAC addresses, and multiple counters for the amount and type of traffic passing through the interface. If you configure a port channel interface, the show interfaces command lists the interfaces configured in the port channel.
www.dell.com | support.dell.com To view which interfaces are enabled for Layer 3 data transmission use the show ip interfaces brief command in EXEC Privilege mode. In Figure 9-2, the TenGigabitEthernet interface 1/5 is in Layer 3 mode because an IP address has been assigned to it and the interface’s status is operationally up. Figure 9-2.
Disabling and Re-enabling a Physical Interface By default, all port interfaces on an Aggregator are operationally enabled (no shutdown) to send and receive Layer 2 traffic. You can reconfigure a physical interface to shut it down by entering the shutdown command. To re-enable the interface, enter the no shutdown command.
www.dell.com | support.dell.com Layer 2 Mode On an Aggregator, physical interfaces, port channels, and VLANs auto-configure to operate in Layer 2 mode. Figure 9-4 shows the basic configuration found in a Layer 2 interface. Note: Layer 3 (network) mode is not supported on Aggregator physical interfaces, port channels, and VLANs. Only management interfaces operate in Layer 3 mode. Figure 9-4.
The Aggregator supports the management ethernet interface as well as the standard interface on any front-end port. You can use either method to connect to the system. Configuring a Management Interface On the Aggregator, the dedicated management interface provides management access to the system.You can configure this interface with FTOS, but the configuration options on this interface are limited. You cannot configure gateway addresses and IP addresses if it appears in the main routing table of FTOS.
www.dell.com | support.dell.com To display the configuration for a given port, use the show interface command from EXEC Privilege mode (Figure 9-5). To display the routing table for a given port, use the show ip route command from EXEC Privilege mode. Figure 9-5.
Configuring a Static Route for a Management Interface When an IP address used by a protocol and a static management route exists for the same prefix, the protocol route takes precedence over the static management route.
www.dell.com | support.dell.com VLAN Membership A virtual LAN (VLANs) is a logical broadcast domain or logical grouping of interfaces in a LAN in which all data received is kept locally and broadcast to all members of the group. In Layer 2 mode, VLANs move traffic at wire speed and can span multiple devices. FTOS supports up to 4093 port-based VLANs and one default VLAN, as specified in IEEE 802.1Q.
Port-Based VLANs Port-based VLANs are a broadcast domain defined by different ports or interfaces. In FTOS, a port-based VLAN can contain interfaces from different stack units within the chassis. FTOS supports 4094 port-based VLANs. Port-based VLANs offer increased security for traffic, conserve bandwidth, and allow switch segmentation. Interfaces in different VLANs do not communicate with each other, adding some security to the traffic on those interfaces.
www.dell.com | support.dell.com Configuring VLAN Membership By default, all Aggregator ports are member of all (4094) VLANs, including the default untagged VLAN 1. You can use the CLI or CMC interface to reconfigure VLANs only on server-facing interfaces (1 to 32) so that an interface has membership only in specified VLANs. To assign an Aggregator interface in Layer 2 mode to a specified group of VLANs, use the vlan tagged and vlan untagged commands.
Displaying VLAN Membership To view the configured VLANs, enter the show vlan command in EXEC privilege mode: Figure 9-8.
www.dell.com | support.dell.com Adding an Interface to a Tagged VLAN Figure 9-9 shows an example of how to add a tagged interface (Te1/7) to a VLAN (VLAN 2). Figure 9-9. Adding an Interface to Another VLAN FTOS(conf-if-te-1/7)# vlan tagged 2 FTOS(conf-if-te-1/7)# exit FTOS(conf)# exit FTOS# show vlan id 2 Enter the vlan tagged command to add interface Te 1/7 to VLAN 2. Enter the show vlan command to verify that interface Te 1/7 is a tagged member of VLAN 2.
Adding an Interface to an Untagged VLAN To move an untagged interfaces from the default VLAN to another VLAN, use the vlan untagged command as shown in Figure 9-10. Figure 9-10. Moving an Untagged Interface to Another VLAN FTOS(conf)# interface tengigabit 0/16 Enter the vlan untagged command to FTOS(conf-if-te-0/16)# vlan untagged 4 as an untagged member of VLAN 4.
www.dell.com | support.dell.com Port Channel Definition and Standards Link aggregation is defined by IEEE 802.3ad as a method of grouping multiple physical interfaces into a single logical interface—a link aggregation group (LAG) or port channel. A LAG is “a group of links that appear to a MAC client as if they were a single link” according to IEEE 802.3ad. In FTOS, a LAG is referred to as a port channel interface.
Port channels can contain a mix of 1000 or 10000 Mbps Ethernet interfaces. The interface speed (100, 1000, or 10000 Mbps) used by the port channel is determined by the first port channel member that is physically up. FTOS disables the interfaces that do not match the interface speed set by the first channel member. That first interface may be the first interface that is physically brought up or was physically operating when interfaces were added to the port channel.
www.dell.com | support.dell.com Displaying Port Channel Information To view the port channel’s status and channel members in a tabular format, use the show interfaces command in EXEC Privilege mode (Figure 9-11). port-channel brief Figure 9-11.
Figure 9-12. show interface port-channel Command Example FTOS#show interface port-channel Port-channel 1 is a dynamically-created port channel based on the Port-channel 1 is up, line protocol is up NIC teaming configuration in connected servers learned via LACP.
www.dell.com | support.dell.com Interface Range An interface range is a set of interfaces to which other commands may be applied, and may be created if there is at least one valid interface within the range. Bulk configuration excludes from configuring any non-existing interfaces from an interface range. A default VLAN may be configured only if the interface range being configured consists of only VLAN ports.
Create a Multiple-Range Figure 9-14. Creating a Multiple-Range Prompt FTOS(conf)#interface range tengigabitethernet 0/5 - 10 , tengigabitethernet 0/1 , vlan 1 FTOS(conf-if-range-te-0/5-10,te-0/1,vl-1)# Exclude a Smaller Port Range If the interface range has multiple port ranges, the smaller port range is excluded from the prompt. Figure 9-15.
www.dell.com | support.dell.com You can display interface statistics with the monitor interface command. This command displays an ongoing list of the interface status (up/down), number of packets, traffic statistics, etc. Command Syntax Command Mode Purpose monitor interface interface EXEC Privilege View interface statistics.
Figure 9-18. monitor interface Command Example FTOS#monitor interface tengig 3/1 Dell Force10 uptime is 1 day(s), 4 hour(s), 31 minute(s) Monitor time: 00:00:00 Refresh Intvl.
www.dell.com | support.dell.com Maintenance Using TDR The time domain reflectometer (TDR) is supported on all Dell Networking switch/routers. TDR is an assistance tool to resolve link issues that helps detect obvious open or short conditions within any of the four copper pairs. TDR sends a signal onto the physical cable and examines the reflection of the signal that returns.
Flow Control Using Ethernet Pause Frames An Aggregator auto-configures to operate in auto-DCB-enable mode (Refer to Data Center Bridging: Auto-DCB-Enable Mode). In this mode, Aggregator ports detect whether peer devices support converged enhanced Ethernet (CEE) or not, and enable DCBX and PFC or link-level flow control accordingly: • • Interfaces come up with DCB disabled and link-level flow control enabled to control data transmission between the Aggregator and other network devices.
www.dell.com | support.dell.com MTU Size The Aggregator auto-configures interfaces to use a maximum MTU size of 12,000 bytes. If a packet includes a Layer 2 header, the difference in bytes between the link MTU and IP MTU must be enough to include the Layer 2 header. For example, for VLAN packets, if the IP MTU is 1400, the link MTU must be no less than 1422: 1400-byte IP MTU + 22-byte VLAN Tag = 1422-byte link MTU The MTU range is 592-12000, with a default of 1554.
Auto-Negotiation on Ethernet Interfaces Setting Speed and Duplex Mode of Ethernet Interfaces By default, auto-negotiation of speed and duplex mode is enabled on 1GbE and 10GbE Ethernet interfaces on an Aggregator. The local interface and the directly connected remote interface must have the same setting. Auto-negotiation is the easiest way to accomplish these settings, as long as the remote interface is capable of auto-negotiation.
www.dell.com | support.dell.com Figure 9-19.
Not supported Not supported (Error message is (Error message is thrown) (% Error: Speed 100 thrown) not supported on this (% Error: Speed 100 not interface, config supported on ignored Te 0/49) this interface, config ignored Te 0/ 49) Not supportedNot supported % Error: Speed 100 not supported on this interface, interface-config Supported mode interface-config Supported mode Supported Supported Supported Supported Not Supported Not supported negotiation auto interface-config Supported mode duplex
www.dell.com | support.dell.com Figure 9-21 shows the auto-negotiation options. Figure 9-21.
In EXEC mode, the show interfaces switchport command displays only interfaces in Layer 2 mode and their relevant configuration information. The show interfaces switchport command (Figure 9-23) displays the interface, whether the interface supports IEEE 802.1Q tagging or not, and the VLANs to which the interface belongs. Figure 9-23. show interfaces switchport Command Example FTOS#show interfaces switchport Name: TenGigabitEthernet 13/0 802.
www.dell.com | support.dell.com Clearing Interface Counters The counters in the show interfaces command are reset by the clear counters command. This command does not clear the counters captured by any SNMP program. To clear the counters, use the following command in EXEC Privilege mode: Command Syntax Command Mode Purpose clear counters [interface] EXEC Privilege Clear the counters used in the show interface commands for all VRRP groups, VLANs, and physical interfaces or selected ones.
10 iSCSI Optimization An Aggregator enables internet small computer system interface (iSCSI) optimization with default iSCSI parameter settings (Default iSCSI Optimization Values) and is auto-provisioned to support: • • Detection and Auto configuration for Dell EqualLogic Arrays iSCSI Optimization: Operation To display information on iSCSI configuration and sessions, you can use show commands. iSCSI optimization enables quality-of-service (QoS) treatment for iSCSI traffic.
www.dell.com | support.dell.com • • • • If you configured flow-control, iSCSI uses the current configuration. If you did not configure flow-control, iSCSI auto-configures flow control. iSCSI monitoring sessions — the switch monitors and tracks active iSCSI sessions in connections on the switch, including port information and iSCSI session information. iSCSI QoS — A user-configured iSCSI class of service (CoS) profile is applied to all iSCSI traffic.
Information Monitored in iSCSI Traffic Flows iSCSI optimization examines the following data in packets and uses the data to track the session and create the classifier entries that enable QoS treatment: • • • • • • • Initiator’s IP Address Target’s IP Address ISID (Initiator defined session identifier) Initiator’s IQN (iSCSI qualified name) Target’s IQN Initiator’s TCP Port Target’s TCP Port If no iSCSI traffic is detected for a session during a user-configurable aging period, the session data clears.
www.dell.com | support.dell.com • • • Link-level flow control is enabled on PFC disabled interfaces. iSCSI session snooping is enabled. iSCSI LLDP monitoring starts to automatically detect EqualLogic arrays. iSCSI optimization requires LLDP to be enabled. LLDP is enabled by default when an Aggregator auto-configures.
To display information on iSCSI optimization, use the show commands in Table 10-2 t Table 10-2. Displaying iSCSI Optimization Information Command Output show iscsi (Figure 10-2) Displays the currently configured iSCSI settings. show iscsi sessions (Figure 10-3) Displays information on active iSCSI sessions on the switch. show iscsi sessions detailed [session isid] (Figure 10-4) Displays detailed information on active iSCSI sessions on the switch.
www.dell.com | support.dell.com Figure 10-4. show iscsi sessions detailed Command Example 164 FTOS# show isci sessions detailed Session 0 : ----------------------------------------------------------------------------Target:iqn.2010-11.com.ixia:ixload:iscsi-TG1 Initiator:iqn.2010-11.com.ixia.ixload:initiator-iscsi-2c Up Time:00:00:01:28(DD:HH:MM:SS) Time for aging out:00:00:09:34(DD:HH:MM:SS) ISID:806978696102 Initiator Initiator Target Target Connection IP Address TCP Port IP Address TCPPort ID 10.10.0.
11 Link Aggregation The I/O Aggregator auto-configures with link aggregation groups (LAGs) as follows: • • All uplink ports are automatically configured in a single port channel (LAG 128). Server-facing LAGs are automatically configured if you configure server for link aggregation control protocol (LACP)-based NIC teaming (Network Interface Controller (NIC) Teaming). No manual configuration is required to configure Aggregator ports in the uplink or a server-facing LAG.
www.dell.com | support.dell.com Server-Facing LAGs Server-facing ports are configured as individual ports by default. If you configure a server NIC in standalone, stacking, or VLT mode for LACP-based NIC teaming, server-facing ports are automatically configured as part of dynamic LAGs. The LAG range 1 to127 is reserved for server-facing LAGs.
LACP Example Figure 11-1 shows an example of how LACP operates in an Aggregator stack by auto-configuring the uplink LAG 128 for the connection to a top of rack (ToR) switch and a server-facing LAG for the connection to an installed server that you configured for LACP-based NIC teaming. Figure 11-1.
www.dell.com | support.dell.com The show outputs in this section for uplink LAG 128 and server-facing LAG 1 refer to the example shown in Figure 11-1. 168 Figure 11-2.
Figure 11-3. show lacp 128 Command Example FTOS# show lacp 128 Port-channel 128 admin up, oper up, mode lacp Actor System ID: Priority 32768, Address 0001.e8e1.e1c3 Partner System ID: Priority 32768, Address 0001.e88b.
www.dell.com | support.dell.com Figure 11-4.
Figure 11-5.
172 | Link Aggregation www.dell.com | support.dell.
12 Layer 2 The Aggregator supports CLI commands to manage the MAC address table: • • Clearing MAC Address Entries Displaying the MAC Address Table The Aggregator auto-configures with support for Network Interface Controller (NIC) Teaming. Note: On an Aggregator, all ports are configured by default as members of all (4094) VLANs, including the default VLAN. All VLANs operate in Layer 2 mode.
www.dell.com | support.dell.com Task Command Syntax Command Mode Clear a MAC address table of dynamic entries.
Figure 12-1. Redundant NICs with NIC Teaming X Port 0/1 MAC: A:B:C:D A:B IP: 1.1.1.1 k Active Lin Port 0/5 fnC0025mp MAC Address Station Move When you use NIC teaming, consider that the server MAC address is originally learned on Port 0/1 of the switch (Figure 12-2). If the NIC fails, the same MAC address is learned on Port 0/5 of the switch. The MAC address is disassociated with the one port and re-associated with another in the ARP table; in other words, the ARP entry is “moved”.
176 | Layer 2 www.dell.com | support.dell.
13 Link Layer Discovery Protocol (LLDP) An Aggregator auto-configures to support the link layer discovery protocol (LLDP) for the auto-discovery of network devices. You can use CLI commands to display acquired LLDP information, clear LLDP counters, and debug LACP operation. Overview LLDP—defined by IEEE 802.1AB—is a protocol that enables a local area network (LAN) device to advertise its configuration and receive configuration information from adjacent LLDP-enabled LAN infrastructure devices.
www.dell.com | support.dell.com There are five types of TLVs (Table 13-1). All types are mandatory in the construction of an LLDPDU except Optional TLVs. You can configure the inclusion of individual Optional TLVs. Table 13-1. Type Type, Length, Value (TLV) Types TLV Description 0 End of LLDPDU Marks the end of an LLDPDU. 1 Chassis ID The Chassis ID TLV is a mandatory TLV that identifies the chassis containing the IEEE 802 LAN station associated with the transmitting LLDP agent.
Organizationally Specific TLVs Organizationally specific TLVs can be defined by a professional organization or a vendor. They have two mandatory fields (Figure 13-3) in addition to the basic TLV fields (Figure 13-1): • Organizationally Unique Identifier (OUI)—a unique number assigned by the IEEE to an organization or vendor. OUI Sub-type—These sub-types indicate the kind of information in the following data field. The sub-types are determined by the owner of the OUI. • Figure 13-3.
www.dell.com | support.dell.com Table 13-2. Optional TLV Types Type TLV Description IEEE 802.3 Organizationally Specific TLVs 127 MAC/PHY Configuration/Status Indicates the capability and current setting of the duplex status and bit rate, and whether the current settings are the result of auto-negotiation. This TLV is not available in the FTOS implementation of LLDP, but is available and mandatory (non-configurable) in the LLDP-MED implementation.
TIA Organizationally Specific TLVs The Dell Networking system is an LLDP-MED Network Connectivity Device (Device Type 4). Network connectivity devices are responsible for: • • transmitting an LLDP-MED capabilities TLV to endpoint devices storing the information that endpoint devices advertise Table 13-3 list the five types of TIA-1057 Organizationally Specific TLVs. Table 13-3.
www.dell.com | support.dell.com LLDP-MED Capabilities TLV The LLDP-MED Capabilities TLV communicates the types of TLVs that the endpoint device and the network connectivity device support. LLDP-MED network connectivity devices must transmit the Network Policies TLV. • • The value of the LLDP-MED Capabilities field in the TLV is a 2 octet bitmap (Figure 13-4), each bit represents an LLDP-MED capability (Table 13-4). The possible values of the LLDP-MED Device Type is listed in Table 13-5.
LLDP-MED Network Policies TLV A network policy in the context of LLDP-MED is a device’s virtual local area network (VLAN) configuration and associated Layer 2 and Layer 3 configurations, specifically: • • • • VLAN ID VLAN tagged or untagged status Layer 2 priority DSCP value The application type is represented by an integer (the Type integer in Table 13-6), which indicates a device function for which a unique network policy is defined.
www.dell.com | support.dell.com Figure 13-5.
LLDP Operation On an Aggregator, LLDP operates as follows: • • • • • • LLDP is enabled by default. LLDPDUs are transmitted and received by default. LLDPDUs are transmitted periodically. The default interval is 30 seconds. LLDPDU information received from a neighbor expires after the default Time to Live (TTL) value: 120 seconds. FTOS supports up to eight neighbors per interface. FTOS supports a maximum of 8000 total neighbors per system.
www.dell.com | support.dell.com Viewing Information Advertised by Adjacent LLDP Agents 186 | To display brief information about adjacent devices, use the show lldp neighbors command (Figure 13-9). To display all of the information that neighbors are advertising, use the show lldp neighbors detail command (Figure 13-10). Figure 13-9.
Figure 13-10.
www.dell.com | support.dell.com Clearing LLDP Counters 188 | You can clear LLDP statistics that are maintained on an Aggregator for LLDP counters for frames transmitted to and received from neighboring devices on all or a specified physical interface. To clear LLDP counters, enter the clear lldp counters command.
Debugging LLDP The debug lldp command allows you to view the TLVs that your system is sending and receiving. • • Use the debug lldp brief command to view a readable version of the TLVs. Use the debug lldp detail command to view a readable version of the TLVs plus a hexadecimal version of the entire LLDPDU. Figure 13-11.
www.dell.com | support.dell.com Relevant Management Objects FTOS supports all IEEE 802.1AB MIB objects. • • • • Table 13-7. Table 13-7 lists the objects associated with received and transmitted TLVs. Table 13-8 lists the objects associated with the LLDP configuration on the local agent. Table 13-9 lists the objects associated with IEEE 802.1AB Organizationally Specific TLVs. Table 13-10 lists the objects associated with received and transmitted LLDP-MED TLVs.
Table 13-8.
www.dell.com | support.dell.com Table 13-9. LLDP 802.1 Organizationally Specific TLV MIB Objects TLV Type TLV Name TLV Variable System LLDP MIB Object 127 Port-VLAN ID PVID Local lldpXdot1LocPortVlanId Remote lldpXdot1RemPortVlanId 127 Port and Protocol VLAN ID port and protocol VLAN supported Local port and protocol VLAN enabled PPVID 127 VLAN Name VID VLAN name length VLAN name Table 13-10.
Table 13-10.
www.dell.com | support.dell.com Table 13-10.
14 Port Monitoring The Aggregator supports user-configured port monitoring. See Configuring Port Monitoring for the configuration commands to use. Port monitoring copies all incoming or outgoing packets on one port and forwards (mirrors) them to another port. The source port is the monitored port (MD) and the destination port is the monitoring port (MG).
www.dell.com | support.dell.com Port Monitoring The Aggregator supports multiple source-destination statements in a monitor session, but there may only be one destination port in a monitoring session (Message 2). Message 2 One Destination Port in a Monitoring Session Error Message % Error: Only one MG port is allowed in a session. The number of source ports supported in a port pipe is equal to the number of physical ports in the port pipe.
Figure 14-2 shows ports 0/25 and 0/26 that belong to port pipe 1 with a maximum of four destination ports. Figure 14-2.
www.dell.com | support.dell.com Configuring Port Monitoring To configure port monitoring, use the following example: Step Task Command Syntax Command Mode 1 Verify that the intended monitoring port has no configuration other than no shutdown (Figure 14-3). show interface EXEC Privilege 2 Create a monitoring session using the command monitor session from CONFIGURATION mode (Figure 14-3).
In Figure 14-4, the host and server are exchanging traffic which passes through the uplink interface 1/1. Port 1/1 is the monitored port and port 1/42 is the destination port, which is configured to only mirror traffic received on tengigabitethernet 1/1 (host-originated traffic). Figure 14-4.
200 | Port Monitoring www.dell.com | support.dell.
15 Simple Network Management Protocol (SNMP) Network management stations use the simple network management protocol (SNMP) to retrieve or alter management data from network elements. A datum of management information is called a managed object; the value of a managed object can be static or variable. Network elements store managed objects in a database called a management information base (MIB).
www.dell.com | support.dell.com Setting up SNMP FTOS supports SNMP version 1 and version 2 which are community-based security models. The primary difference between the two versions is that version 2 supports two additional protocol operations (informs operation and snmpgetbulk query) and one additional object (counter64 object). Creating a Community For SNMPv1 and SNMPv2, you must create a community to enable the community-based security in FTOS.
Dell Networking supports RFC 4001, Textual Conventions for Internet Work Addresses that defines values representing a type of internet address. These values display for ipAddressTable objects using the snmpwalk command. In the following figure, the value “4” displays in the OID before the IP address for IPv4. >snmpwalk -v 2c -c public 10.11.195.63 1.3.6.1.2.1.4.34 IP-MIB::ip.34.1.3.1.4.1.1.1.1 = INTEGER: 1107787778 IP-MIB::ip.34.1.3.1.4.2.1.1.1 = INTEGER: 1107787779 IP-MIB::ip.34.1.3.2.16.254.128.0.0.0.0.
www.dell.com | support.dell.com Displaying the Ports in a VLAN Using SNMP FTOS identifies VLAN interfaces using an interface index number that is displayed in the show interface as shown in Figure 15-5. vlan output, Figure 15-5.
Figure 15-6 shows the output for an Aggregator. All hex pairs are 00, indicating that no ports are assigned to VLAN 10. In Figure 15-7, Port 0/2 is added to VLAN 10 as untagged. And the first hex pair changes from 00 to 04. Figure 15-7.
www.dell.com | support.dell.com Table 15-1. MIB Objects for Fetching Dynamic MAC Entries in the Forwarding Database MIB Object OID dot1dTpFdbTable .1.3.6.1.2.1.17.4.3 Description MIB List the learned unicast MAC addresses on the default VLAN. Q-BRIDGE MIB dot1qTpFdbTable .1.3.6.1.2.1.17.7.1.2. List the learned unicast MAC addresses on 2 non-default VLANs. dot3aCurAggFdb Table .1.3.6.1.4.1.6027.3.2. List the learned MAC addresses of aggregated 1.1.5 links (LAG).
To fetch the learned MAC address of a port-channel use dot3aCurAggFdbTable. The instance number is the decimal conversion of the MAC address concatenated with the port-channel number. Figure 15-10.
www.dell.com | support.dell.com Starting from the least significant bit (LSB): • • • • • • the first 14 bits represent the card type the next 4 bits represent the interface type the next 7 bits represent the port number the next 5 bits represent the slot number the next 1 bit is 0 for a physical interface and 1 for a logical interface the next 1 bit is unused For example, the index 44634369 is 10101010010001000100000001 in binary.
Monitor Port-channels Note: The interface index does not change if the interface reloads or fails over. If the unit is renumbered (for any reason) the interface index changes during a reload. To check the status of a Layer 2 port-channel, use f10LinkAggMib (.1.3.6.1.4.1.6027.3.2). Below, Po 1 is a switchport and Po 2 is in Layer 3 mode. [senthilnathan@lithium ~]$ snmpwalk -v 2c -c public 10.11.1.1 .1.3.6.1.4.1.6027.3.2.1.1 SNMPv2-SMI::enterprises.6027.3.2.1.1.1.1.1.1 = INTEGER: 1 SNMPv2-SMI::enterprises.
www.dell.com | support.dell.com For L3 LAG, you do not have this support. SNMPv2-MIB::sysUpTime.0 = Timeticks: (8500842) 23:36:48.42 SNMPv2-MIB::snmpTrapOID.0 = OID: IF-MIB::linkDown IF-MIB::ifIndex.33865785 = INTEGER: 33865785 SNMPv2-SMI::enterprises.6027.3.1.1.4.1.2 = STRING: "OSTATE_DN: Changed interface state to down: Tengig 0/ 0" 2010-02-10 14:22:39 10.16.130.4 [10.16.130.4]: SNMPv2-MIB::sysUpTime.0 = Timeticks: (8500842) 23:36:48.42 SNMPv2-MIB::snmpTrapOID.0 = OID: IF-MIB::linkDown IF-MIB::ifIndex.
The status of the MIBS is as follows: $ snmpwalk -c public -v 2c 10.16.130.148 1.3.6.1.2.1.47.1.1.1.1.2 SNMPv2-SMI::mib-2.47.1.1.1.1.2.1 = "" SNMPv2-SMI::mib-2.47.1.1.1.1.2.2 = STRING: "PowerConnect I/O-Aggregator" SNMPv2-SMI::mib-2.47.1.1.1.1.2.3 = STRING: "Module 0" SNMPv2-SMI::mib-2.47.1.1.1.1.2.4 = STRING: "Unit: 0 Port 1 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.5 = STRING: "Unit: 0 Port 2 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.6 = STRING: "Unit: 0 Port 3 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.
www.dell.com | support.dell.com SNMPv2-SMI::mib-2.47.1.1.1.1.2.81 = STRING: "Unit: 1 Port 14 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.82 = STRING: "Unit: 1 Port 15 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.83 = STRING: "Unit: 1 Port 16 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.84 = STRING: "Unit: 1 Port 17 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.85 = STRING: "Unit: 1 Port 18 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.86 = STRING: "Unit: 1 Port 19 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.
16 Stacking Overview An Aggregator auto-configures to operate in standalone mode. To use an Aggregator in a stack, you must manually configure it using the CLI to operate in stacking mode. Stacking is supported only on the 40GbE ports on the base module. Stacking is limited to six Aggregators in the same or different m1000e chassis. To configure a stack, you must use the CLI. Stacking provides a single point of management for high availability and higher throughput.
www.dell.com | support.dell.com Figure 16-15. A Two-Aggregator Stack Stack Management Roles The stack elects the management units for the stack management: • • Stack master: primary management unit Standby: secondary management unit The master holds the control plane and the other units maintain a local copy of the forwarding databases.
Stack Master Election The stack elects a master and standby unit at bootup time based on MAC address. The unit with the higher MAC value becomes master. To view which switch is the stack master, use the show system command. Figure 16-16 shows sample output from an established stack. A change in the stack master occurs when: • • • You power down the stack master or bring the master switch offline. A failover of the master switch occurs. You disconnect the master switch from the stack.
www.dell.com | support.dell.com MAC Addressing All port interfaces in the stack use the MAC address of the management interface on the master switch. The MAC address of the chassis in which the master Aggregator is installed is used as the stack MAC address. The stack continues to use the master’s chassis MAC address even after a failover. The MAC address is not refreshed until the stack is reloaded and a different unit becomes the stack master.
Stacking Port Numbers By default, each Aggregator in Standalone mode is numbered stack-unit 0. Stack-unit numbers are assigned to member switches when the stack comes up. Figure 16-17 shows the numbers of the 40GbE stacking ports on an Aggregator. Figure 16-17. Stack Groups on an Aggregator Stack Unit 0 / Port 37 Stack Unit 0 / Port 33 Configuring a Switch Stack To configure and bring up a switch stack, follow these steps: 1.
www.dell.com | support.dell.com Stacking Prerequisites Before you cable and configure a stack of Aggregators, review the following prerequisites: • • • • • • All Aggregators in the stack must be powered up with the initial or startup configuration before you attach the cables. All stacked Aggregators must run the same FTOS version. The minimum FTOS version required is 8.3.17.0. To check the FTOS version that a switch is running, use the show version command.
Cabling Procedure The following cabling procedure uses the stacking topology in Figure 16-15. To connect the cabling: 1. Connect a 40GbE base port on the first Aggregator to a 40GbE base port on another Aggregator in the same chassis. 2. Connect a 40GbE base port on the second Aggregator to a 40GbE port on the first Aggregator. The resulting ring topology allows the entire stack to function as a single switch with resilient fail-over capabilities.
www.dell.com | support.dell.com Note: If the stacked switches all reboot at approximately the same time, the switch with the highest MAC address is automatically elected as the master switch. The switch with the next highest MAC address is elected as standby. As each switch joins the stack, it is assigned the lowest available stack-unit number from 0 to 5. The default configuration of each stacked switch is stored in the running configuration of the stack.
Resetting a Unit on a Stack Use the following reset commands to reload any of the member units or the standby in a stack. If you try to reset the stack master, an error message is displayed: Reset of master unit is not allowed. Task Command Syntax Command Mode Reload a stack unit from the master switch reset stack-unit unit-number EXEC Privilege Reset a stack-unit when the unit is in a problem state.
www.dell.com | support.dell.com Verifying a Stack Configuration Using LEDs Table 16-2 lists the status of a stacked switch according to the color of the System Status light emitting diodes (LEDs) on its front panel. Table 16-2. System Status LED on a Stacked Switch Color Meaning Blue The switch is operating as the stack master or as a standalone unit. Off The switch is a member or standby unit. Amber The switch is booting or a failure condition has occurred.
Figure 16-18. show system brief Command Example FTOS# show system brief StStack MAC : 00:1e:c9:f1:00:9b -- Stack Info -Unit UnitType Status ReqTyp CurTyp Version Ports -----------------------------------------------------------------------------------0 Management online I/O-Aggregator I/O-Aggregator 8-3-17-46 56 1 Standby online I/O-Aggregator I/O-Aggregator 8-3-17-46 56 2 Member not present 3 Member not present 4 Member not present 5 Member not present Figure 16-19.
www.dell.com | support.dell.com Figure 16-20. show inventory optional-module Command Example FTOS# show inventory optional-module Unit Slot Expected Inserted Next Boot Power ----------------------------------------------------------------0 0 SFP+ SFP+ AUTO Good 0 1 QSFP+ QSFP+ AUTO Good * - Mismatch Figure 16-21.
Figure 16-24.
www.dell.com | support.dell.com Troubleshooting a Switch Stack Troubleshooting Commands To perform troubleshooting operations on a switch stack, use the commands in Table 16-4 on the master switch. Table 16-4. Troubleshooting Stack Commands Command Output show system stack-ports (Figure 16-25) Displays the status of stacked ports on stack units.
Figure 16-26.
www.dell.com | support.dell.com Figure 16-27.
Master Switch Fails Problem: The master switch fails due to a hardware fault, software crash, or power loss. Resolution: A failover procedure begins: 1. Keep-alive messages from the Aggregator master switch time out after 60 seconds and the switch is removed from the stack. 2. The standby switch takes the master role. Data traffic on the new master switch is uninterrupted. Protocol traffic is managed by the control plane. 3. A member switch is elected as the new standby.
www.dell.com | support.dell.com Stack Unit in Card-Problem State Due to Incorrect FTOS Version Problem: A stack unit enters a Card-Problem state because the switch has a different FTOS version than the master unit (Figure 16-29). The switch does not come online as a stack unit. Resolution: To restore a stack unit with an incorrect FTOS version as a member unit, disconnect the stacking cables on the switch and install the correct FTOS version.
Upgrading a Switch Stack To upgrade all switches in a stack with the same FTOS version, follow these steps: Step Task Command Syntax Command Mode 1 Copy the new FTOS image to a network server. 2 Download the FTOS image by accessing an interactive CLI that requests the server IP address and image filename, and prompts you to upgrade all member stack units. Specify the system partition on the master switch into which you want to copy the FTOS image. Valid partition values are a: and b:.
www.dell.com | support.dell.com Upgrading a Single Stack Unit Upgrading a single stacked switch is necessary when the unit was disabled due to an incorrect FTOS version. This procedure upgrades the image in the boot partition of the member unit from the corresponding partition in the master unit.
17 Broadcast Storm Control On the Aggregator, the broadcast storm control feature is enabled by default on all ports, and disabled on a port when an iSCSI storage device is detected. Broadcast storm control is re-enabled as soon as the connection with an iSCSI device ends. Broadcast traffic on Layer 2 and Layer 3 interfaces is limited or suppressed during a broadcast storm. You can view the status of a broadcast-storm control operation by using the show io-aggregator broadcast storm-control status command.
234 | Broadcast Storm Control www.dell.com | support.dell.
18 System Time and Date The Aggregator auto-configures the hardware and software clocks with the current time and date. If necessary, you can manually set and maintain the system time and date using the CLI commands described in this chapter.
www.dell.com | support.dell.com Setting the Time for the Software Clock You can change the order of the month and day parameters to enter the time and date as time day month year. You cannot delete the software clock. The software clock runs only when the software is up. The clock restarts, based on the hardware clock, when the switch reboots.
Setting the Time Zone Universal time coordinated (UTC) is the time standard based on the International Atomic Time standard, commonly known as Greenwich Mean time. When determining system time, you must include the differentiator between the UTC and your local timezone. For example, San Jose, CA is the Pacific Timezone with a UTC offset of -8.
www.dell.com | support.dell.com Setting Daylight Savings Time FTOS supports setting the system to daylight savings time once or on a recurring basis every year. Setting Daylight Saving Time Once Set a date (and time zone) on which to convert the switch to daylight savings time on a one-time basis.
Setting Recurring Daylight Saving Time Set a date (and time zone) on which to convert the switch to daylight savings time on a specific day every year. If you have already set daylight savings for a one-time setting, you can set that date and time as the recurring setting using the clock summer-time time-zone recurring command.
www.dell.com | support.dell.com Command Syntax Command Mode Purpose • • • • • • • • • end-week: If you entered a start-week, enter one of the following as the week that daylight savings ends: week-number: enter a number from 1 to 4 as the number of the week to end daylight savings time. first: enter the keyword first to end daylight savings time in the first week of the month. last: enter the keyword last to end daylight savings time in the last week of the month.
19 Uplink Failure Detection (UFD) Feature Description Uplink Failure Detection (UFD) provides detection of the loss of upstream connectivity and, if used with NIC teaming, automatic recovery from a failed link. A switch provides upstream connectivity for devices, such as servers. If a switch loses its upstream connectivity, downstream devices also lose their connectivity.
www.dell.com | support.dell.com Figure 19-1. Uplink Failure Detection How Uplink Failure Detection Works UFD creates an association between upstream and downstream interfaces. The association of uplink and downlink interfaces is called an uplink-state group. An interface in an uplink-state group can be a physical interface or a port-channel (LAG) aggregation of physical interfaces. An enabled uplink-state group tracks the state of all assigned upstream interfaces.
Figure 19-2. Uplink Failure Detection Example If only one of the upstream interfaces in an uplink-state group goes down, a specified number of downstream ports associated with the upstream interface are put into a link-down state. This number is user-configurable and is calculated by the ratio of upstream port bandwidth to downstream port bandwidth in the same uplink-state group.
www.dell.com | support.dell.com Important Points to Remember When you configure Uplink Failure Detection, the following conditions apply: • You can configure up to sixteen uplink-state groups. By default, no uplink-state groups are created. An uplink-state group is considered to be operationally up if it has at least one upstream interface in the link-up state. An uplink-state group is considered to be operationally down if it has no upstream interfaces in the link-up state.
Configuring Uplink Failure Detection To configure Uplink Failure Detection, follow these steps: Step 1 Command Syntax and Mode Description uplink-state-group group-id Creates an uplink-state group and enabling the tracking of upstream links on the switch/router. Valid group-id values are 1 to 16. Command Mode: CONFIGURATION To delete an uplink-state group, enter the no uplink-state-group group-id command.
www.dell.com | support.dell.com Step 5 Command Syntax and Mode Description defer-timer seconds Specifies the time (in seconds) to wait for the upstream port channel (LAG 128) to come back up before server ports are brought down. The range is from 1 to 120. Command Mode: UPLINK-STATE-GROUP 6 description text (Optional) Enters a text description of the uplink-state group. Maximum length: 80 alphanumeric characters.
Message 1 shows the Syslog messages displayed when you clear the UFD-disabled state from all disabled downstream interfaces in an uplink-state group by entering the clear ufd-disable uplink-state-group group-id command. All downstream interfaces return to an operationally up state.
www.dell.com | support.dell.com Displaying Uplink Failure Detection To display information on the Uplink Failure Detection feature, enter any of the following show commands: Show Command Syntax Description show uplink-state-group [group-id] [detail] Displays status information on a specified uplink-state group or all groups. Valid group-id values are 1 to 16. Command Mode: EXEC detail displays additional status information on the upstream and downstream interfaces in each group (see Figure 19-3).
Figure 19-3.
www.dell.com | support.dell.com Figure 19-4.
Sample Configuration: Uplink Failure Detection Figure 19-7 shows a sample configuration of Uplink Failure Detection on a switch/router in which you: • • • • • • Configure uplink-state group 3. Add downstream links TenGigabitethernet 0/1, 0/2, 0/5, 0/9, 0/11, and 0/12. Configure two downstream links to be disabled if an upstream link fails. Add upstream links TenGigabitethernet 0/3 and 0/4. Add a text description for the group. Verify the configuration with various show commands.
www.dell.com | support.dell.com Figure 19-7.
20 Upgrade Procedures To view the requirements for upgrading the Dell Networking operating software (FTOS) on an Aggregator, refer to the FTOS Release Notes for the M I/O Aggregator. Follow the procedures in the FTOS Release Notes for the software version you wish to upgrade to. Direct any questions or concerns about FTOS upgrades to the Dell Networking Technical Support Center. You can reach technical support: • • • On the Web: www.force10networks.com/support/ By email: support@force10networks.
254 | Upgrade Procedures www.dell.com | support.dell.
21 Debugging and Diagnostics The chapter contains the following sections: • • • • • • • • • • • • Debugging Aggregator Operation Software show Commands Offline Diagnostics Trace Logs Show Hardware Commands Environmental Monitoring Buffer Tuning Troubleshooting Packet Loss Application Core Dumps Mini Core Dumps TCP Dumps Restoring the Factory Default Settings Debugging and Diagnostics | 255
www.dell.com | support.dell.com Debugging Aggregator Operation This section describes common troubleshooting procedures to use for error conditions that may arise during Aggregator operation. All interfaces on the Aggregator are operationally down Symptom: All Aggregator interfaces are down. Resolution: Ensure that port channel 128 is up and that the Aggregator-facing port channel on the top-of-rack switch is correctly configured. Steps to Take: 1.
Broadcast, unknown multicast, and DLF packets switched at a very low rate Symptom: Broadcast, unknown multicast, and DLF packets are switched at a very low rate. By default, broadcast storm control is enabled on an Aggregator and rate limits the transmission of broadcast, unknown multicast, and DLF packets to 1Gbps. This default behavior is designed to avoid unnecessarily flooding these packets on all (4094) VLANs on all Aggregator interfaces (default configuration).
www.dell.com | support.dell.com Flooded packets on all VLANs are received on a server Symptom: All packets flooded on all VLANs on an Aggregator are received on a server, even if the server is configured as a member of only a subset of VLANs. This behavior happens because all Aggregator ports are, by default, members of all (4094) VLANs. Resolution: Configure a port that is connected to the server with restricted VLAN membership. Steps to Take: 1.
Auto-configured VLANs do not exist on a stacked Aggregator Symptom: When an Aggregator is configured and used in a stack, traffic does not flow and the VLAN auto-configuration on all ports is lost. This behavior happens because an Aggregator in stacking mode does not support auto-configured VLANs. Only VLANs that were previously manually configured are retained on the master stack unit. Resolution: You must manually configure VLAN membership on each stack-unit port. Steps to Take: 1.
www.dell.com | support.dell.com Software show Commands Use the show version and show system stack-unit 0 commands as a part of troubleshooting an Aggregator’s software configuration in a standalone or stacking scenario. Table 21-1. Software show Commands Command Description show version Display the current version of FTOS software running on an Aggregator. show system stack-unit 0 Display software configuration on an Aggregator in stacking mode. Figure 21-1.
Figure 21-2. show system stack-unit 0 Command Example FTOS#show system stack-unit 0 -- Unit 0 -Unit Type : Management Unit Status : online Next Boot : online Required Type : I/O-Aggregator - 34-port GE/TE (XL) Current Type : I/O-Aggregator - 34-port GE/TE (XL) Master priority : 0 Hardware Rev : Num Ports : 56 Up Time : 17 hr, 8 min FTOS Version : 8-3-17-15 Jumbo Capable : yes POE Capable : no Boot Flash : A: 4.0.1.0 [booted] B: 4.0.1.0bt Boot Selector : 4.0.0.
www.dell.com | support.dell.com Offline Diagnostics The offline diagnostics test suite is useful for isolating faults and debugging hardware.The diagnostics tests are grouped into three levels: • • • Level 0—Level 0 diagnostics check for the presence of various components and perform essential path verifications. In addition, they verify the identification registers of the components on the board. Level 1—A smaller set of diagnostic tests.
Figure 21-3. Taking a Stack Unit Offline FTOS#offline stack-unit 2 Warning - Diagnostic execution will cause stack-unit to reboot after completion of diags. Proceed with Offline-Diags [confirm yes/no]:y 5w6d12h: %STKUNIT0-M:CP %CHMGR-2-STACKUNIT_DOWN: Stack unit 2 down - stack unit offline 5w6d12h: %STKUNIT0-M:CP %IFMGR-1-DEL_PORT: Removed port: Tengig 2/1-48 FTOS#5w6d12h: %STKUNIT1-S:CP %IFMGR-1-DEL_PORT: Removed port: Tengig 2/1-48 2.
www.dell.com | support.dell.com On a Standby unit, you can reach the TRACE_LOG_DIR files only by using the show file command from the flash://TRACE_LOG_DIR directory. Note: Non-management Member units do not support this functionality. Figure 21-5. Command Example FTOS#dir flash://TRACE_LOG_DIR Directory of flash:/TRACE_LOG_DIR 1 2 3 drwx drwx -rwx 4096 4096 100583 Jan 17 2011 15:02:16 +00:00 . Jan 01 1980 00:00:00 +00:00 ..
Table 21-2. show hardware Commands Command Description show hardware stack-unit {0-5} cpu party-bus statistics View input and output statistics on the party bus, which carries inter-process communication traffic between CPUs. show hardware stack-unit {0-5} drops unit {0-0} port {33-56} View the ingress and egress internal packet-drop counters, MAC counters drop, and FP packet drops for the stack unit on per port basis.
www.dell.com | support.dell.com Figure 21-6. show interfaces transceiver Command Example FTOS#show int ten 0/49 transceiver SFP is present SFP 49 Serial Base ID fields SFP 49 Id = 0x03 SFP 49 Ext Id = 0x04 SFP 49 Connector = 0x07 SFP 49 Transceiver Code = 0x00 0x00 0x00 0x01 0x20 0x40 0x0c 0x01 SFP 49 Encoding = 0x01 SFP 49 BR Nominal = 0x0c SFP 49 Length(9um) Km = 0x00 SFP 49 Length(9um) 100m = 0x00 SFP 49 Length(50um) 10m = 0x37 SFP 49 Length(62.
Recognize an Over-Temperature Condition An over-temperature condition occurs for one of two reasons: • • The card genuinely is too hot. A sensor has malfunctioned. Inspect cards adjacent to the one reporting condition to discover the cause. • • If directly adjacent cards are not a normal temperature, suspect a genuine overheating condition. If directly adjacent cards are a normal temperature, suspect a faulty sensor. When the system detects a genuine over-temperature condition, it powers off the card.
www.dell.com | support.dell.com Figure 21-8.
The simple network management protocol (SNMP) traps and OIDs in Table 22-3 provide information about environmental monitoring hardware and hardware components. Table 21-3. SNMP Traps and OIDs OID String OID Name Description chSysPortXfpRecvPower OID to display the receiving power of the connected optics. chSysPortXfpTxPower OID to display the transmitting power of the connected optics. chSysPortXfpRecvTemp OID to display the Temperature of the connected optics. Receiving power .1.3.6.1.4.1.6027.
www.dell.com | support.dell.com All ports support eight queues, four for data traffic and four for control traffic. All eight queues are tunable. Physical memory is organized into cells of 128 bytes. The cells are organized into two buffer pools—a dedicated buffer and a dynamic buffer. • • Dedicated buffer is reserved memory that cannot be used by other interfaces on the same ASIC or by other queues on the same interface.
Deciding to Tune Buffers Dell Networking recommends exercising caution when configuring any non-default buffer settings, as tuning can significantly affect system performance. The default values work for most cases. As a guideline, consider tuning buffers if traffic is very bursty (and coming from several interfaces). In this case: • • • Reduce the dedicated buffer on all queues/interfaces. Increase the dynamic buffer on all interfaces.
www.dell.com | support.dell.com You cannot allocate more than the available memory for the dedicated buffers. If the system determines that the sum of the configured dedicated buffers allocated to the queues is more than the total available memory, the configuration is rejected, returning a syslog message similar to the following. Table 21-4.
Figure 21-11. Displaying Buffer Profile Allocations FTOS#show running-config interface tengigabitethernet 2/0 ! interface TenGigabitEthernet 2/0 no ip address mtu 9252 switchport no shutdown buffer-policy myfsbufferprofile FTOS#show buffer-profile detail int tengig 0/10 Interface Tengig 0/10 Buffer-profile fsqueue-fp Dynamic buffer 1256.00 (Kilobytes) Queue# Dedicated Buffer Buffer Packets (Kilobytes) 0 3.00 256 1 3.00 256 2 3.00 256 3 3.00 256 4 3.00 256 5 3.00 256 6 3.00 256 7 3.
www.dell.com | support.dell.com Using a Pre-Defined Buffer Profile FTOS provides two pre-defined buffer profiles, one for single-queue (for example, non-QoS) applications, and one for four-queue (for example, QoS) applications. Task Command Mode Apply one of two pre-defined buffer profiles for all port pipes in the system. buffer-profile global [1Q|4Q] CONFIGURATION You must reload the system for the global buffer profile to take effect (Message 3).
Sample Buffer Profile Configuration The two general types of network environments are sustained data transfers and voice/data. Dell Networking recommends a single-queue approach for data transfers (Figure 22-12). Figure 21-12.
www.dell.com | support.dell.com Displaying Drop Counters The show hardware stack-unit 0–11 drops [unit 0 [port 0–63]] command assists in identifying which stack unit, port pipe, and port is experiencing internal drops (Figure 22-13) and (Figure 22-14). Figure 21-13.
Figure 21-14.
www.dell.com | support.dell.com Figure 21-15.
Displaying Stack Port Statistics The show hardware stack-unit stack-port command displays input and output statistics for a stack-port interface (Figure 22-17). Figure 21-17.
www.dell.com | support.dell.com Application Core Dumps Application core dumps are disabled by default. A core dump file can be very large. Due to memory requirements, the file can only be sent directly to an FTP server. It is not stored on the local flash. To enable full application core dumps, use the following command: Task Command Syntax Command Mode Enable RPM core dumps and specify the shutdown mode.
Figure 21-19.
www.dell.com | support.dell.com TCP Dumps TCP dump captures CPU bound control plane traffic to improve troubleshooting and system manageability. When enabled, a TCP dump captures all the packets on the local CPU, as specified in the CLI. You can save the traffic capture files to flash, FTP, SCP, or TFTP. The files saved on the flash are located in the flash://TCP_DUMP_DIR/Tcpdump_/ directory, and labeled tcpdump_*.pcap. There can be up to 20 Tcpdump_ directories.
• • When you restore the units in stand-alone mode, the units remain in stand-alone mode after the restoration. After the restore is complete, the units power cycle immediately. Figure 4-9 shows an example of using the restore factory-defaults command to restore the Factory Default Settings. Figure 21-21.
284 | Debugging and Diagnostics www.dell.com | support.dell.
22 Standards Compliance This chapter contains the following sections: • • • IEEE Compliance RFC and I-D Compliance MIB Location Note: Unless noted, when a standard cited here is listed as supported by Dell Networking operating software (FTOS), FTOS also supports predecessor standards. One way to search for predecessor standards is to use the http://tools.ietf.org/ website.
www.dell.com | support.dell.com 286 RFC and I-D Compliance The following standards are supported by FTOS on an Aggregator and are grouped by related protocol. The columns showing support by platform indicate which version of FTOS first supports the standard.
General IPv4 Protocols RFC# Full Name 791 Internet Protocol 792 Internet Control Message Protocol 826 An Ethernet Address Resolution Protocol 1027 Using ARP to Implement Transparent Subnet Gateways 1042 A Standard for the Transmission of IP Datagrams over IEEE 802 Networks 1519 Classless Inter-Domain Routing (CIDR): an Address Assignment and Aggregation Strategy 1812 Requirements for IP Version 4 Routers 2131 Dynamic Host Configuration Protocol 3021 Using 31-Bit Prefixes on IPv4 Point-to-
www.dell.com | support.dell.
Network Management (continued) RFC# Full Name ruzin-mstp-mib-02 (Traps) Definitions of Managed Objects for Bridges with Multiple Spanning Tree Protocol sFlow.org sFlow Version 5 sFlow.
www.dell.com | support.dell.com MIB Location 290 | Force10 MIBs are under the Force10 MIBs subhead on the Documentation page of iSupport: https://www.force10networks.com/csportal20/KnowledgeBase/Documentation.aspx You also can obtain a list of selected MIBs and their OIDs at the following URL: https://www.force10networks.com/csportal20/MIBs/MIB_OIDs.aspx Some pages of iSupport require a login. To request an iSupport account, go to: https://www.force10networks.com/CSPortal20/Support/AccountRequest.
