Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide Cisco MDS NX-OS Release 6.2(1) April 2013 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
CONTENTS New and Changed Information Preface xiii xv Audience xv Organization xv Document Conventions xvi Related Documentation xvii Release Notes xvii Regulatory Compliance and Safety Information xvii Compatibility Information xvii Hardware Installation xvii Software Installation and Upgrade xvii Cisco NX-OS xvii Command-Line Interface xviii Intelligent Storage Networking Services Configuration Guides Troubleshooting and Reference xviii Obtaining Documentation and Submitting a Service Request CHA
Contents TE Port 2-4 TF Port 2-4 TNP Port 2-5 SD Port 2-5 ST Port 2-5 Fx Port 2-5 B Port 2-5 Auto Mode 2-5 Interface States 2-6 Administrative States 2-6 Operational States 2-6 Reason Codes 2-6 Graceful Shutdown 2-9 Port Administrative Speeds 2-10 Autosensing 2-10 Frame Encapsulation 2-10 Beacon LEDs 2-11 Speed LEDs 2-11 Bit Error Thresholds 2-11 SFP Transmitter Types 2-12 TL Ports 2-13 TL Port ALPA Caches 2-14 Port Guard 2-14 Port Monitor 2-15 Port Monitor Port Guard 2-16 Port Group Monitor 2-16 Local Swi
Contents Configuring Interface Modes 2-22 Configuring System Default Port Mode F 2-23 Configuring ISL between Two Switches 2-24 Configuring 10-Gbps FC Mode 2-24 Configuring Port Administrative Speeds 2-25 Configuring Port Speed Group 2-26 Configuring the Interface Description 2-26 Specifying a Port Owner 2-27 Configuring Beacon Mode 2-27 Disabling Bit Error Threshold 2-28 Configuring Switch Port Attribute Default Values 2-28 Configuring TL Ports 2-29 Manually Inserting Entries into the ALPA Cache 2-29 Clea
Contents Displaying Management Interface Configuration Displaying VSAN Interface Information 2-55 CHAPTER 3 Configuring Fibre Channel Interfaces 2-55 3-1 Information About Fibre Channel Interfaces 3-1 Generations of Modules and Switches 3-1 Port Groups 3-3 Port Rate Modes 3-5 Dedicated Rate Mode 3-7 Shared Rate Mode 3-8 Dedicated Rate Mode Configurations for the 8-Gbps Modules Port Speed 3-10 Dynamic Bandwidth Management 3-10 Out-of-Service Interfaces 3-11 Oversubscription Ratio Restrictions 3-11 Ban
Contents Verifying Fibre Channel Interfaces Configuration 3-43 Displaying Interface Capabilities 3-44 Displaying SFP Diagnostic Information 3-45 Configuration Examples for Fibre Channel Interfaces 3-45 Configuration Example for 48-Port 8-Gbps Module Interfaces Configuration Example for 24-Port 8-Gbps Module Interfaces Configuration Example for 4/44-Port 8-Gbps Module Interfaces Configuration Example for 48-Port 4-Gbps Module Interfaces Configuration Example for 24-Port 4-Gbps Module Interfaces CHAPTER 4
Contents Configuring Performance Buffers 4-22 Configuring Extended BB_credits 4-23 Enabling Buffer-to-Buffer Credit Recovery 4-24 Enabling the Buffer-to-Buffer State Change Number Configuring Receive Data Field Size 4-24 Verifying BB_Credit Configuration CHAPTER 5 Configuring Trunking 4-25 5-1 Information About Trunking 5-1 Trunking E Ports 5-1 Trunking F Ports 5-2 Key Concepts 5-3 Trunking Protocols 5-3 Trunk Modes 5-4 Trunk-Allowed VSAN Lists and VF_IDs 5-5 Guidelines and Limitations 5-7 General
Contents Interface Addition to a PortChannel 6-8 Forcing an Interface Addition 6-9 Interface Deletion from a PortChannel 6-9 PortChannel Protocols 6-9 Channel Group Creation 6-10 Autocreation 6-11 Manually Configured Channel Groups 6-12 Prerequisites for PortChannels 6-12 Guidelines and Limitations 6-13 General Guidelines and Limitations 6-13 Generation 1 PortChannel Limitations 6-14 F and TF PortChannel Limitations 6-14 Valid and Invalid PortChannel Examples 6-15 Default Settings 6-16 Configuring Port
Contents Traffic Map 7-7 Disruptive 7-8 Multiple VSAN Support 7-8 Guidelines and Limitations 7-8 NPV Guidelines and Requirements 7-8 NPV Traffic Management Guidelines 7-9 DPVM Configuration Guidelines 7-9 NPV and Port Security Configuration Guidelines 7-10 Configuring N Port Virtualization 7-10 Enabling N Port Identifier Virtualization 7-10 Configuring NPV 7-10 Configuring NPV Traffic Management 7-12 Configuring List of External Interfaces per Server Interface 7-12 Enabling the Global Policy for Disrup
Contents Enabling Port Tracking 9-3 Information About Configuring Linked Ports 9-3 Binding a Tracked Port Operationally 9-4 Information About Tracking Multiple Ports 9-4 Tracking Multiple Ports 9-5 Information About Monitoring Ports in a VSAN 9-5 Monitoring Ports in a VSAN 9-5 Information AboutForceful Shutdown 9-6 Forcefully Shutting Down a Tracked Port 9-6 Displaying Port Tracking Information 9-6 9-8 Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide OL-29284-01, Release 6.
Contents Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide xii OL-29284-01, Release 6.
New and Changed Information As of Cisco MDS NX-OS Release 4.2(1), software configuration information is available in new feature-specific configuration guides for the following information: • System management • Interfaces • Fabric • Quality of service • Security • IP services • High availability and redundancy The information in these new guides previously existed in the Cisco MDS 9000 Family CLI Configuration Guide and in the Cisco MDS 9000 Family Fabric Manager Configuration Guide.
Table 1 New and Changed Features (continued) Feature GUI Change Description FlexAttach Disable FlexAttach Added information about disabling FlexAttach. Port Group Monitoring Enhancements Check Oversubscription Added information about > Monitor monitoring a selected port group. Changed in Release Where Documented 5.0(1a) Chapter 8, “Configuring FlexAttach Virtual pWWN” 5.
Preface This preface describes the audience, organization, and conventions of the Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide. It also provides information on how to obtain related documentation. Audience This guide is for experienced network administrators who are responsible for configuring and maintaining the Cisco MDS 9000 Family of multilayer directors and fabric switches.
Chapter Title Description Chapter 7 Configuring N Port Virtualization Provides an overview of N Port Virtualization and includes quidelines and requirements for configuring and verifying NPV. Chapter 8 Configuring FlexAttach Virtual pWWN FlexAttach virtual pWWN feature facilitates server and configuration management. In a SAN environment, the server installation or replacement, requires interaction and coordination among the SAN and server administrators.
Related Documentation The documentation set for the Cisco MDS 9000 Family includes the following documents. To find a document online, use the Cisco MDS NX-OS Documentation Locator at: http://www.cisco.com/en/US/docs/storage/san_switches/mds9000/roadmaps/doclocater.
• Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide • Cisco MDS 9000 Family NX-OS Security Configuration Guide • Cisco MDS 9000 Family NX-OS IP Services Configuration Guide • Cisco MDS 9000 Family NX-OS Intelligent Storage Services Configuration Guide • Cisco MDS 9000 Family NX-OS High Availability and Redundancy Configuration Guide • Cisco MDS 9000 Family NX-OS Inter-VSAN Routing Configuration Guide • Cisco MDS 9000 Family Cookbook for Cisco MDS SAN-OS Command-Line Interface
CH A P T E R 1 Interfaces Overview • Trunks and PortChannels, page 1-1 • Fibre Channel Port Rate Limiting, page 1-1 • Extended Credits, page 1-2 • N Port Virtualization, page 1-2 • FlexAttach, page 1-2 Trunks and PortChannels Trunking, also known as VSAN trunking, is a feature specific to switches in the Cisco MDS 9000 Family. Trunking enables interconnect ports to transmit and receive frames in more than one VSAN, over the same physical link. Trunking is supported on E ports and F ports.
Chapter 1 Interfaces Overview Extended Credits Extended Credits Full line-rate Fibre Channel ports provide at least 255 buffer credits standard. Adding credits lengthens distances for Fibre Channel SAN extension. Using extended credits, up to 4095 buffer credits from a pool of more than 6000 buffer credits for a module can be allocated to ports as needed to greatly extend the distance for Fibre Channel SANs.
Chapter 1 Interfaces Overview FlexAttach Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide OL-29284-01, Release 6.
Chapter 1 Interfaces Overview FlexAttach Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 1-4 OL-29284-01, Release 6.
CH A P T E R 2 Configuring Interfaces • Information About Interfaces, page 2-1 • Prerequisites for Interfaces, page 2-18 • Guidelines and Limitations, page 2-18 • Default Settings, page 2-20 • Configuring Interfaces, page 2-20 • Verifying Interfaces Configuration, page 2-41 Information About Interfaces The main function of a switch is to relay frames from one data link to another.
Chapter 2 Configuring Interfaces Information About Interfaces • Port Group Monitor, page 2-16 • Local Switching, page 2-16 • Slow Drain Device Detection and Congestion Avoidance, page 2-17 • Management Interfaces, page 2-17 • VSAN Interfaces, page 2-18 Interface Description For the Fibre Channel interfaces, you can configure the description parameter to provide a recognizable name for the interface.
Chapter 2 Configuring Interfaces Information About Interfaces • Note The operational status represents the current status of a specified attribute like the interface speed. This status cannot be changed and is read-only. Some values may not be valid when the interface is down (for example, the operational speed). When a module is removed and replaced with the same type of module, the configuration is retained.
Chapter 2 Configuring Interfaces Information About Interfaces TL Port In translative loop port (TL port) mode, an interface functions as a translative loop port. It may be connected to one or more private loop devices (NL ports). TL ports are specific to Cisco MDS 9000 Family switches and have similar properties as FL ports.
Chapter 2 Configuring Interfaces Information About Interfaces TNP Port In trunking NP port (TNP port) mode, an interface functions as a trunking expansion port. It may be connected to a trunked F port (TF port) to create a link to a core NPIV switch from an NPV switch to carry tagged frames. SD Port In SPAN destination port (SD port) mode, an interface functions as a switched port analyzer (SPAN). The SPAN feature is specific to switches in the Cisco MDS 9000 Family.
Chapter 2 Configuring Interfaces Information About Interfaces port or NL port mode. If the interface is attached to a third-party switch, it operates in E port mode. If the interface is attached to another switch in the Cisco MDS 9000 Family, it may become operational in TE port mode (see Chapter 5, “Configuring Trunking”). TL ports and SD ports are not determined during initialization and are administratively configured.
Chapter 2 Configuring Interfaces Information About Interfaces Table 2-3 Note Reason Codes for Interface States Administrative Configuration Operational Status Reason Code Up Up None. Down Down Administratively down—If you administratively configure an interface as down, you disable the interface. No traffic is received or transmitted. Up Down See Table 2-4. Only some of the reason codes are listed in Table 2-4.
Chapter 2 Configuring Interfaces Information About Interfaces Table 2-4 Reason Codes for Nonoperational States Reason Code (long version) Description Applicable Modes Link failure or not connected The physical layer link is not operational. All SFP not present The small form-factor pluggable (SFP) hardware is not plugged in. Initializing The physical layer link is operational and the protocol initialization is in progress.
Chapter 2 Configuring Interfaces Information About Interfaces Table 2-4 Reason Codes for Nonoperational States (continued) Reason Code (long version) Description Isolation due to ELP failure The port negotiation failed. Isolation due to ESC failure The port negotiation failed. Isolation due to domain overlap The Fibre Channel domains (fcdomain) overlap. Isolation due to domain ID assignment failure The assigned domain ID is not valid.
Chapter 2 Configuring Interfaces Information About Interfaces A graceful shutdown is not possible in the following situations: Note • If you physically remove the port from the switch. • If in-order delivery (IOD) is enabled (for information about IOD, refer to the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide). • If the Min_LS_interval interval is higher than 10 seconds. For information about FSPF global configuration, refer to the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide.
Chapter 2 Configuring Interfaces Information About Interfaces You can set the frame format to EISL for all frames transmitted by the interface in SD port mode. If you sent the frame encapsulation to EISL, all outgoing frames are transmitted in the EISL frame format, regardless of the SPAN sources. See the Cisco MDS 9000 Family NX-OS System Management Configuration Guide. Beacon LEDs Figure 2-2 displays the status, link, and speed LEDs in a 16-port switching module.
Chapter 2 Configuring Interfaces Information About Interfaces The bit errors can occur for the following reasons: • Faulty or bad cable. • Faulty or bad GBIC or SFP. • GBIC or SFP is specified to operate at 1 Gbps but is used at 2 Gbps. • GBIC or SFP is specified to operate at 2 Gbps but is used at 4 Gbps. • Short haul cable is used for long haul or long haul cable is used for short haul. • Momentary sync loss. • Loose cable connection at one or both ends.
Chapter 2 Configuring Interfaces Information About Interfaces Table 2-5 SFP Transmitter Acronym Definitions (continued) Definition Acronym Standard transmitters defined in the GBIC specifications CWDM-1590 c1590 CWDM-1610 c1610 TL Ports Private loop devices refer to legacy devices that reside on arbitrated loops. These devices are not aware of a switch fabric because they only communicate with devices on the same physical loop.
Chapter 2 Configuring Interfaces Information About Interfaces Figure 2-3 shows examples of TL port translation support.
Chapter 2 Configuring Interfaces Information About Interfaces The port guard feature gives the SAN administrator the ability to prevent this issue from occurring in environments that are vulnerable to these problems. The port can be configured to stay down after the first failure or after a specified number of failures in a specified time period. This allows the SAN administrator to intervene and control the recovery, avoiding any problems caused by the cycling.
Chapter 2 Configuring Interfaces Information About Interfaces Counter Threshold Type Interval (Seconds) % Rising Threshold % Falling Threshold Event RX Performance Delta 60 2147483648 4 524288000 4 TX Performance Delta 60 2147483648 4 524288000 4 Event Port Monitor Port Guard Port monitor port guard is a feature that disables or shuts down a port when an event occurs. Depending on the configuration, when an event occurs the port is either error-disabled or flapped.
Chapter 2 Configuring Interfaces Information About Interfaces Slow Drain Device Detection and Congestion Avoidance All data traffic between end devices in a SAN fabric is carried by Fibre Channel Class 3. In some cases, the traffic is carried by Class 2 services that use link-level, per-hop-based, and buffer-to-buffer flow control. These classes of service do not support end-to-end flow control.
Chapter 2 Configuring Interfaces Prerequisites for Interfaces VSAN Interfaces VSANs apply to Fibre Channel fabrics and enable you to configure multiple isolated SAN topologies within the same physical infrastructure. You can create an IP interface on top of a VSAN and then use this interface to send frames to this VSAN. To use this feature, you must configure the IP address for this VSAN. VSAN interfaces cannot be created for nonexisting VSANs.
Chapter 2 Configuring Interfaces Guidelines and Limitations • The auto mode is not allowed in a 32-port switching module or the host-optimized ports in the Cisco 9100 Series (16 host-optimized ports in the Cisco MDS 9120 switch and 32 host-optimized ports in the Cisco MDS 9140 switch). • The default port mode is Fx (Fx negotiates to F or FL) for 32-port switching modules. • The 32-port switching module does not support FICON.
Chapter 2 Configuring Interfaces Default Settings Default Settings Table 2-7 lists the default settings for interface parameters. Table 2-7 Default Interface Parameters Parameters Default Interface mode Auto Interface speed Auto Administrative state Shutdown (unless changed during initial setup) Trunk mode On (unless changed during initial setup) on non-NPV and NPIV core switches. Off on NPV switches.
Chapter 2 Configuring Interfaces Configuring Interfaces • Configuring Management Interfaces, page 2-37 • Creating VSAN Interfaces, page 2-38 • Configuring Slow Drain Device Detection and Congestion Avoidance, page 2-38 For more information on configuring mgmt0 interfaces, refer to the Cisco MDS 9000 Family NX-OS Fundamentals Configuration Guide and Cisco MDS 9000 Family NX-OS IP Services Configuration Guide.
Chapter 2 Configuring Interfaces Configuring Interfaces Setting the Interface Administrative State Detailed Steps To gracefully shut down an interface, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# interface fc1/1 Selects a Fibre Channel interface and enters interface configuration submode. Step 3 switch(config-if)# shutdown Gracefully shuts down the interface and administratively disables traffic flow (default).
Chapter 2 Configuring Interfaces Configuring Interfaces Configuring System Default Port Mode F The system default switchport mode F command sets the administrative mode of all Fibre Channel ports to mode F, while avoiding traffic disruption caused by the formation of unwanted Inter-Switch Links (ISLs). This command is part of the setup utility that runs during bootup after a write erase or reload. It can also be executed from the command line in configuration mode.
Chapter 2 Configuring Interfaces Configuring Interfaces Configuring ISL between Two Switches Note Ensure the FC cable is connected between the ports and do a no-shut on each port. E-port mode is used when the port functions as one end of an Inter-Switch Link (ISL) setting. When you set the port mode to E, you restrict the port coming up as an E port (trunking or non-trunking, depending on the trunking port mode). switch# conf t --------> Enter configuration commands, one per line.
Chapter 2 Configuring Interfaces Configuring Interfaces Detailed Steps To configure the interface mode, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# interface fc1/1-12 switch(config-if)# Selects a Fibre Channel interface and enters interface configuration submode. Note Ensure that one full ASIC range of ports are selected before executing this command. For example, /1-12 for a 48-port module or fcY/1-8 for a 32-port module.
Chapter 2 Configuring Interfaces Configuring Interfaces Step 3 Command Purpose switch(config-if)# switchport speed 1000 Configures the port speed of the interface to 1000 Mbps. The number indicates the speed in megabits per second (Mbps). switch(config-if)# switchport speed 10000 Configures the port speed of the interface to 10000 Mbps (for 10-Gbps).
Chapter 2 Configuring Interfaces Configuring Interfaces Step 3 Command Purpose switch(config-if)# switchport description cisco-HBA2 Configures the description of the interface. The string can be up to 80 characters long. switch(config-if)# no switchport description Clears the description of the interface. Specifying a Port Owner Using the port owner feature, you can specify the owner of a port and the purpose for which a port is used so that the other administrators are informed.
Chapter 2 Configuring Interfaces Configuring Interfaces Step 3 Command Purpose switch(config-if)# switchport beacon Enables the beacon mode for the interface. switch(config-if)# no switchport beacon Disables the beacon mode for the interface. Troubleshooting Tips • The flashing green light turns on automatically when an external loopback is detected that causes the interfaces to be isolated. The flashing green light overrides the beacon mode configuration.
Chapter 2 Configuring Interfaces Configuring Interfaces Detailed Steps To configure switch port attributes, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# no system default switchport shutdown switch(config)# Configures the default setting for administrative state of an interface as Up. (The factory default setting is Down).
Chapter 2 Configuring Interfaces Configuring Interfaces Configuring Port Guard Detailed Steps To enable or disable the port guard for a port, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# interface fc1/1 Selects the port interface. Step 3 switch(config)# errdisable detect cause link-down Brings the port to down state if the link flaps once.
Chapter 2 Configuring Interfaces Configuring Interfaces Switch# show port internal info interface fc8/3 fc8/3 is down (Error disabled - port down due to trustsec violation) Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN) Port WWN is 21:c3:00:0d:ec:10:57:80 Admin port mode is E, trunk mode is on snmp link state traps are enabled Port vsan is 1 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec 5 minutes output rate 0 bits/sec, 0 b
Chapter 2 Configuring Interfaces Configuring Interfaces Configuring a Port Monitor Policy Detailed Steps To configure a port monitor policy, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# port-monitor name policyname Specifies the policy name and enters the port monitoring policy configuration mode. switch(config)# no port-monitor name Removes the policy. policyname switch# config t Enters configuration mode.
Chapter 2 Configuring Interfaces Configuring Interfaces Command Purpose Step 10 switch(config-port-monitor)# counter link-loss poll-interval seconds delta rising-threshold percentage1 event event-id falling-threshold2 percentage2 event event-id Specifies the delta link failure counter, poll interval in seconds, the thresholds in percentage, and the event IDs of events to be triggered.
Chapter 2 Configuring Interfaces Configuring Interfaces Activating a Port Monitor Policy Detailed Steps To activate a port monitor policy, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# port-monitor activate Activates the specified port monitor policy. policyname switch(config)# port-monitor activate Activates the default port monitor policy. switch(config)# no port-monitor activate Deactivates the specified port monitoring policy.
Chapter 2 Configuring Interfaces Configuring Interfaces • Reverting to the Default Policy for a Specific Counter, page 2-36 • Turning Off the Monitoring of Specific Counter, page 2-36 • Activating a Port Group Monitor Policy, page 2-37 Enabling Port Group Monitor Detailed Steps To enable port group monitor, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# port-group-monitor enable Enables (default) port group monitoring.
Chapter 2 Configuring Interfaces Configuring Interfaces Reverting to the Default Policy for a Specific Counter When the no counter command is used in the config-port-group-monitor mode, the specified counter polling values will revert to the default values as seen in the following example: switch(config)# port-group-monitor name PGMON_policy switch(config-port-group-monitor)# counter tx-performance poll-interval 100 delta rising-threshold 65 falling-threshold 25 switch(config)# show port-group-monitor PG
Chapter 2 Configuring Interfaces Configuring Interfaces Port type : All Port Groups -----------------------------------------------------------------------------------------Counter Threshold Interval %ge Rising Threshold %ge Falling Threshold --------------- -------- -------------------- ---------------------RX Performance Delta 26 450 250 TX Performance Delta 60 100 80 ------------------------------------------------------------------------------------------ Activating a Port Group Monitor Policy To act
Chapter 2 Configuring Interfaces Configuring Interfaces To configure the mgmt0 Ethernet interface to connect over IPv6, follow these steps: Command Purpose Step 1 switch# config terminal switch(config)# Enters configuration mode. Step 2 switch(config)# interface mgmt0 switch(config-if)# Selects the management Ethernet interface on the switch and enters interface configuration submode. Step 3 switch(config-if)# ipv6 enable Enables IPv6 and assigns a link-local address on the interface.
Chapter 2 Configuring Interfaces Configuring Interfaces Configuring Congestion Frame Timeout Value The default congestion frame timeout value is 500 milliseconds. We recommend that you retain the default configuration for the ISLs and configure a value that does not exceed the default value for the edge ports. If the frame is in the switch for a longer time than the configured congestion frame timeout, it gets dropped, which empties the buffer space in the ISL and alleviates the congestion.
Chapter 2 Configuring Interfaces Configuring Interfaces Note The no-credit timeout value and stuck frame timeout value are interlinked. The no-credit timeout value must always be greater than the stuck frame timeout value. To configure the no-credit timeout value, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# system timeout no-credit-drop seconds mode E|F Specifies the no-credit timeout value and port mode for the switch.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration LR RX Delta 60 5 4 1 4 Not enabled LR TX Delta 60 5 4 1 4 Not enabled Timeout Discards Delta 60 200 4 10 4 Not enabled Credit Loss Reco Delta 60 1 4 0 4 Not enabled TX Credit Not Available Delta 60 10 4 0 4 Not enabled ------------------------------------------------------------------------------------------ The following default port monitor policy will be active when the switch comes up: Policy Name : slowdrain Admin status : Not Activ
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration • Displaying Port Monitor Status and Policies, page 2-51 • Displaying Port Group Monitor Status and Policies, page 2-53 • Displaying Management Interface Configuration, page 2-55 • Displaying VSAN Interface Information, page 2-55 Displaying Interface Information The show interface command is invoked from the EXEC mode and displays the interface configurations.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration 0 CRC, 0 unknown class 0 too long, 0 too short 4638491 frames output, 7264731728 bytes 0 discards, 0 errors 3 input OLS, 9 LRR, 1 NOS, 0 loop inits 9 output OLS, 7 LRR, 1 NOS, 0 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining. . . .
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Receive B2B Credit is 12 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec 5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec 1856 frames input, 116632 bytes 0 discards, 0 errors 0 CRC, 0 unknown class 0 too long, 0 too short 1886 frames output, 887712 bytes 0 discards, 0 errors 0 input OLS, 0 LRR, 0 NOS, 1 loop inits 1 output OLS, 1 LRR, 0 NOS, 1 loop inits 16 receiv
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration 2214834 frames input, 98673588 bytes 0 discards, 0 errors 0 CRC, 0 unknown class 0 too long, 0 too short 2262415 frames output, 343158368 bytes 0 discards, 0 errors 1 input OLS, 1 LRR, 1 NOS, 0 loop inits 2 output OLS, 1 LRR, 0 NOS, 0 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration ... GigabitEthernet4/6 GigabitEthernet4/7 GigabitEthernet4/8 down down down 10.1.1.2/8 10.1.1.27/24 -- auto auto auto 3000 1500 1500 ------------------------------------------------------------------------------Interface Status Oper Mode Oper Speed (Gbps) ------------------------------------------------------------------------------iscsi4/1 down -...
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration 0 frames output, 0 bytes 0 class-2 frames, 0 bytes 0 class-3 frames, 0 bytes 0 class-f frames, 0 bytes 0 discards 0 input OLS, 0 LRR, 0 NOS, 0 loop inits 0 output OLS, 0 LRR, 0 NOS, 0 loop inits 0 link failures, 0 sync losses, 0 signal losses 16 receive B2B credit remaining 3 transmit B2B credit remaining. . . .
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Mbits/s Frames Mbits/s Frames ------------------------------------------------------------------------------fc3/1 0 3871 0 3874 fc3/2 0 3902 0 4232 fc3/3 0 3901 0 4138 fc3/4 0 3895 0 3894 fc3/5 0 3890 0 3897 fc9/8 0 0 0 0 fc9/9 0 5 0 4 fc9/10 0 4186 0 4182 fc9/11 0 4331 0 4315 ------------------------------------------------------------------------------Interface Input (rate is 5 min avg) Output (rate is 5 min avg) ----------------------
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration channel-group 11 force no shutdown Example 2-12 displays the running configuration information for a specified interface. The interface configuration commands are grouped together.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Example 2-15 Displays Interface Information in a Brief Format After the System Default Switchport Mode F Command is Executed switch# show interface brief ------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode (Gbps) ------------------------------------------------------------------------------fc4/1 1 F -notConnected swl --fc4/2
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration In addition to these proxied devices, other virtual devices (local or remote domain controller addresses) are also allocated ALPAs on the loop. A switch reserves the ALPA for its own communication with private devices, and the switch acts as a SCSI initiator. The first column in the output of the show tlport interface command is the ALPA identity of the device on the loop.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration switch# show port-monitor status Port Monitor : Enabled Active Policies : sample Last 10 logs : switch# show port-monitor -----------------------------------------------------------------------------------------Port Monitor : enabled -----------------------------------------------------------------------------------------Policy Name : sample Admin status : Not Active Oper status : Not Active Port type : All Ports ------------------------
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Oper status : Active Port type : All Access Ports -----------------------------------------------------------------------------------------Counter Threshold Interval Rising Threshold event Falling Threshold event portguard --------------- -------- ---------------- ----- ------------------ ----- ----Link Loss Delta 60 5 4 1 4 enabled Sync Loss Delta 60 5 4 1 4 enabled Protocol Error Delta 60 1 4 0 4 enabled Signal Loss Delta 60 5 4 1 4 ena
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration switch# show port-group-monitor -----------------------------------------------------------------------------------------Port Group Monitor : enabled -----------------------------------------------------------------------------------------Policy Name : pgm1 Admin status : Not Active Oper status : Not Active Port type : All Port Groups -----------------------------------------------------------------------------------------Counter Thresho
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Displaying Management Interface Configuration To display the management interface configuration, use the show interface mgmt 0 command. switch# show interface mgmt 0 mgmt0 is up Hardware is FastEthernet Address is 000c.30d9.fdbc Internet address is 10.16.1.
Chapter 2 Configuring Interfaces Verifying Interfaces Configuration Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 2-56 OL-29284-01, Release 6.
CH A P T E R 3 Configuring Fibre Channel Interfaces • Information About Fibre Channel Interfaces, page 3-1 • Guidelines and Limitations, page 3-18 • Default Settings, page 3-25 • Configuring Fibre Channel Interfaces, page 3-26 • Verifying Fibre Channel Interfaces Configuration, page 3-43 • Configuration Examples for Fibre Channel Interfaces, page 3-45 Information About Fibre Channel Interfaces This section includes the following topics: • Generations of Modules and Switches, page 3-1 • Port
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces The Cisco MDS 9500 Series switches, Cisco MDS 9222i, Cisco MDS 9216A, and Cisco MDS 9216i switches support the Generation 2 modules. Each module or switch can have one or more ports in port groups that share common resources such as bandwidth and buffer credits. In addition to supporting Generation 2 modules, the Cisco MDS 9500 Series switches and the Cisco MDS 9222i switch support the Generation 3 modules.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-1 Fibre Channel Modules and Fabric Switches (continued) Part Number Product Name and Description DS-X9112 12-port 4-Gbps Fibre Channel switching module. DS-X9704 4-port 10-Gbps Fibre Channel switching module. DS-X9530-SF2-K9 Supervisor-2 module for Cisco MDS 9500 Series switches. Generation 2 Fabric Switches DS-C9134-K9 Cisco MDS 9134 Fabric switch.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-2 Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches (continued) Bandwidth Per Number of Ports Port Group Per Port Group (Gbps) Maximum Bandwidth Per Port (Gbps) Part Number Product Name/ Description 4 32.41 or 12.82 DS-X9232-256K9 32-port 8-Gbps Advanced Fibre Channel switching module.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-2 Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches (continued) Bandwidth Per Number of Ports Port Group Per Port Group (Gbps) Maximum Bandwidth Per Port (Gbps) 32-port 4-Gbps Fabric switch 4 16 4 Gbps 2-port 10-Gbps Fabric switch 1 10 10 Gbps DS-C9124K9 (Cisco MDS 9124 Fabric switch) 24-port 4-Gbps Fabric switch 4 16 4 Gbps DS-C9222i-K9 18-port 4-Gbps, 4 6 Gigabit
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-3 shows the modules that support dedicated, shared, and the default rate modes.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-3 Port Rate Mode Support on Generation 2, Generation 3, and Generation 4 Modules and Switches (continued) Supports Supports Default Speed Dedicated Rate Shared Rate Mode and Rate Mode Mode Mode on All Ports Product Name/ Description Part Number DS-C9134-K9 (Cisco MDS 9134 Fabric switch) DS-C9124-K9 (Cisco MDS 9124 Fabric switch) DS-C9222i-K9 (Cisco MDS 9222i Multiservice Modular switch) 32-port 4-Gbps Fa
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-4 Bandwidth Reserved for the Port Speeds on Generation 4 Switching Modules Configured Speed Reserved Bandwidth Auto with 2-Gbps maximum 2 Gbps 2-Gbps 1-Gbps 1 Gbps Table 3-5 shows the bandwidth provided by the various port speed configurations on the 8-Gbps Fibre Channel switching modules.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces All ports in 8-Gbps Advanced Fibre Channel switching modules where bandwidth is shared support 1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps traffic. However, it is possible to configure one or more ports in a port group to operate in dedicated rate mode with 1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps operating speed. All ports in 4-Gbps Fibre Channel switching modules where bandwidth is shared support 1-Gbps, 2-Gbps, or 4-Gbps traffic.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-8 Dedicated Rate Mode Bandwidth Reservation for Generation 3 Fibre Channel Modules Part Number DS-X9224-96K9 DS-X9248-96K9 DS-X9248-48K9 Dedicated Bandwidth per Port Maximum Allowed Ports That Can Come Up 24-port 8-Gbps Fibre Channel switching module 8 Gbps 8 Ports 4 Gbps 24 Ports 48-port 8-Gbps Fibre Channel switching module 8 Gbps 8 Ports 4 Gbps 24 Ports 2 Gbps 48 Ports 4/44-port 8-Gbps
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Ports configured in dedicated rate mode are allocated the required bandwidth to sustain a line rate of traffic at the maximum configured operating speed, and ports configured in shared mode share the available remaining bandwidth within the port group. Bandwidth allocation among the shared mode ports is based on the operational speed of the ports.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3-9 Bandwidth Allocation for Oversubscribed Interfaces Reserved Bandwidth (Gbps) Configured Speed Ratios enabled Maximum Bandwidth Ratios disabled (Gbps) 24-Port 8-Gbps Fibre Channel Module Auto 8 Gbps 0.8 0.8 8 Auto Max 4 Gbps 0.4 0.4 4 Auto Max 2 Gbps 0.2 0.2 2 4/44-Port 8-Gbps Host-Optimized Fibre Channel Module 8 Gbps 0.87 0.16 8 Auto Max 4 Gbps 0.436 0.08 4 Auto Max 2 Gbps 0.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces fc8/1 fc8/2 fc8/3 fc8/4 (out-of-service) fc8/5 (out-of-service) fc8/6 (out-of-service) 16 16 16 4.0 4.0 4.0 dedicated dedicated dedicated For dedicated ports, oversubscription ratio restrictions do not apply to the shared pool in port groups.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces fc5/12 32 8.0 shared Port-Group 4 Total bandwidth is 32.4 Gbps Total shared bandwidth is 32.4 Gbps Allocated dedicated bandwidth is 0.0 Gbps -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc5/13 32 8.0 shared fc5/14 32 8.0 shared fc5/15 32 8.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces fc5/31 fc5/32 Isola-13# show port-resources Module 13 Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers 32 32 8.0 8.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc13/19 250 8.0 dedicated fc13/20 250 8.0 dedicated fc13/21 250 8.0 dedicated fc13/22 250 8.0 dedicated fc13/23 32 8.0 shared fc13/24 32 8.0 shared Port-Group 5 Total bandwidth is 32.
Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces fc13/43 fc13/44 fc13/45 fc13/46 fc13/47 fc13/48 32 32 32 32 32 32 8.0 8.0 8.0 8.0 8.0 8.0 shared shared shared shared shared shared ... When disabling restrictions on oversubscription ratios, all ports in shared mode on 48-port and 24-port 4-Gbps or any 8-Gbps Fibre Channel switching modules must be shut down. When applying restrictions on oversubscription ratios, you must take shared ports out of service.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations When you are downgrading to a release earlier than Cisco SAN-OS Release 3.1(2), all modules keep operating in the same bandwidth fairness configuration prior to the downgrade. After the downgrade, any new module that is inserted has bandwidth fairness disabled. Note After the downgrade, any insertion of a module or module reload will have bandwidth fairness disabled.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Note When a Cisco or another vendor switch port is connected to a Generation 1 module port (ISL connection), the receive buffer-to-buffer credits of the port connected to the Generation 1 module port must not exceed 255. Local Switching Limitations • All ports in the module must be in shared mode. Use the switchport ratemode shared command to ensure that all the ports in the module are in shared mode.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Generation 1 switching modules have specific numbering requirements. If these requirements are not met, the module does not power up. The port index numbering requirements include the following: Note • If port indexes in the range of 256 to 1020 are assigned to operational ports, Generation 1 switching modules do not power up. • A block of contiguous port indexes is available.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations 13 68-ef-bd-a8-40-00 to 68-ef-bd-a8-40-04 Xbar --1 2 Ports ----0 0 Xbar --1 2 Sw -------------NA NA Xbar --1 2 MAC-Address(es) -------------------------------------NA NA JAF1450BMBP Module-Type ----------------------------------Fabric Module 3 Fabric Module 3 Hw -----0.4 0.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Note • The maximum number of PortChannels allowed is 256 if all switching modules are Generation 2 or Generation 3, or both. • The maximum number of PortChannels allowed is 128 whenever there is a Generation 1 switching module in use with a Generation 2 or Generation 3 switching module.
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Table 3-11 PortChannel Configuration and Addition Results PortChannel Members Configured Speed PortChannel New Member No members Any Any Generation 1 or Force Generation 2 or Generation 3 or Generation 4 Pass Auto Auto Generation 1 or Normal or force Generation 2 or Generation 3 or Generation 4 Pass Auto Auto max 2000 Generation 2 or Normal Generation 3 or Force Generation 4 Generation 1 interfaces New Member Type
Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Table 3-11 PortChannel Configuration and Addition Results (continued) PortChannel Members Configured Speed PortChannel New Member New Member Type Addition Type Result Generation 2 interfaces Auto Auto Generation 1 Normal or force Fail Auto max 2000 Auto Generation 1 Normal or force Pass Auto max 2000 Auto Generation 2 or Normal Generation 3 Force Fail Auto max 2000 Generation 2 or Normal Generation 3 Force F
Chapter 3 Configuring Fibre Channel Interfaces Default Settings Default Settings Table 3-12 lists the default settings for Generation 2 interface parameters.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Table 3-14 Default Generation 4 Interface Parameters (continued) Default Parameter 48-Port 8-Gbps Advanced Fibre Channel Switching Module 32-Port 8-Gbps Advanced Fibre Channel Switching Module Port mode Fx Fx BB_credit buffers 32 32 Configuring Fibre Channel Interfaces This section includes the following topics: • Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode, page 3-26 • Task Flow fo
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Note If you change the port bandwidth reservation parameters on a 48-port or 24-port 4-Gbps module, the change affects only the changed port. No other ports in the port group are affected.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Step 5 Configure the traffic speed (1 Gbps, 2 Gbps, 4 Gbps, or autosensing with a maximum of 2 Gbps or 4 Gbps) to use. See the “Configuring Port Speed” section on page 3-29.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces See the “Configuring Buffer-to-Buffer Credits” section on page 4-21, and the “Configuring Extended BB_credits” section on page 4-23. Configuring Port Speed Restrictions • Changing port speed and rate mode disrupts traffic on the port. Traffic on other ports in the port group is not affected.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Use the show interface command to verify the port speed configuration for an interface on a 4-Gbps or 8-Gbps Fibre Channel switching module.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Step 3 Command Purpose switch(config-if)# switchport rate-mode dedicated Reserves dedicated bandwidth for the interface. Note If you cannot reserve dedicated bandwidth on an interface, you might have exceeded the port group maximum bandwidth. Use the show port-resources command to determine what resources are already allocated.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Total shared bandwidth is 12.8 Gbps Allocated dedicated bandwidth is 0.0 Gbps -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc9/13 16 4.0 shared fc9/14 16 4.0 shared fc9/15 16 4.0 shared fc9/16 16 4.0 shared fc9/17 16 4.0 shared fc9/18 16 4.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces fc4/12 32 8.0 shared Port-Group 3 Total bandwidth is 12.8 Gbps Total shared bandwidth is 12.8 Gbps Allocated dedicated bandwidth is 0.0 Gbps -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc4/13 32 8.0 shared fc4/14 32 8.0 shared fc4/15 32 8.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Total shared bandwidth is 12.8 Gbps Allocated dedicated bandwidth is 0.0 Gbps -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc4/37 32 8.0 shared fc4/38 32 8.0 shared fc4/39 32 8.0 shared fc4/40 32 8.0 shared fc4/41 32 8.0 shared fc4/42 32 8.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces fc7/13 fc7/14 fc7/15 fc7/16 fc7/17 fc7/18 fc7/19 fc7/20 fc7/21 fc7/22 fc7/23 fc7/24 32 32 32 32 32 32 32 32 32 32 32 32 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 shared shared shared shared shared shared shared shared shared shared shared shared Port-Group 3 Total bandwidth is 12.8 Gbps Total shared bandwidth is 12.8 Gbps Allocated dedicated bandwidth is 0.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Configuring Local Switching Note We recommend that you shut down all of the ports on the module before you execute the local switching command. If local switching is enabled, then ports cannot be configured in dedicated mode. If there are dedicated ports and you enter the local switching command, a warning is displayed and the operation is prevented.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Disabling Restrictions on Oversubscription Ratios Prerequisites • Before disabling restrictions on oversubscription ratios, ensure that you have explicitly shut down shared ports.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Port-Group 1 Total bandwidth is 12.8 Gbps Total shared bandwidth is 12.8 Gbps Allocated dedicated bandwidth is 0.0 Gbps -------------------------------------------------------------------Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode Buffers (Gbps) -------------------------------------------------------------------fc2/1 16 4.0 shared fc2/2 16 4.0 shared fc2/3 16 4.0 dedicated fc2/4 16 4.0 shared fc2/5 16 4.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces fc2/7 fc2/8 ... fc2/48 Step 4 1 1 TE FX --- up down swl swl --- --- 1 FX -- down sw1 -- -- Disable restrictions on oversubscription ratios for the ports. switch# config t Enter configuration commands, one per line. End with CNTL/Z.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Detailed Steps To enable restrictions on oversubscription ratios on a 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching module, follow these steps: Command Purpose Step 1 switch# config t switch(config)# Enters configuration mode. Step 2 switch(config)# interface fc2/1-2, fc2/4-5, fc2/8-38, fc2/43-48 Specifies the port interfaces for which you want to enable restrictions on oversubscription ratios.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Disabling Bandwidth Fairness Restrictions • If you disable bandwidth fairness, up to a 20 percent increase in internal bandwidth allocation is possible for each port group; however, bandwidth fairness is not guaranteed when there is a mix of shared and full-rate ports in the same port group.
Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces Command Purpose Step 3 switch(config-if)# no channel-group Removes the interface from a PortChannel. Step 4 switch(config-if)# shutdown Disables the interface. Step 5 switch(config-if)# out-of-service Putting an interface into out-of-service will cause its shared resource configuration to revert to default Do you wish to continue (y/n)? [n] y Takes the interface out of service.
Chapter 3 Configuring Fibre Channel Interfaces Verifying Fibre Channel Interfaces Configuration Detailed Steps To release the shared resources for a port group, follow these steps: Command Purpose Step 1 switch# config t switch(config)# Enters configuration mode. Step 2 switch(config)# interface fc 1/1 switch(config-if)# Selects the interface and enters interface configuration submode. Tip You can use an interface range to release the resources for all interfaces in a port group.
Chapter 3 Configuring Fibre Channel Interfaces Verifying Fibre Channel Interfaces Configuration Command Purpose show interface fc slot/port Displays the slot or port information. show interface brief Displays the interface. show port index-allocation Displays the port in the index allocation. show port index-allocation startup Displays the startup port in the index allocation. show port-channel compatibility parameters Displays the PortChannel compatibility parameters.
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces BB state change notification Maximum BB state change notifications Rate Mode change Rate Mode Capabilities Receive BB Credit modification supported FX mode Receive BB Credit (min/max/default) ISL mode Receive BB Credit (min/max/default) Performance buffer modification supported Out of Service capable Beacon mode configurable Extended B2B credit capable supported 14 supported Shared yes (1/32/32) -no Dedica
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces Configuration Example for 48-Port 8-Gbps Module Interfaces These steps describe how to configure the 48-port 8-Gbps module interfaces: Step 1 Select the interfaces fc 4/1 through fc 4/2. switch# config t switch(config)# interface fc 4/1 - 2 Step 2 Configure the port speed, rate mode, and port mode on the interfaces.
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces Step 1 Select interfaces fc 3/1. switch# config t switch(config)# interface fc 3/1 Step 2 Configure the port speed, rate mode, and port mode on the interfaces. switch(config-if)# switchport speed 8000 switch(config-if)# switchport rate-mode dedicated switch(config-if)# switchport mode f Step 3 Enable the interfaces and return to configuration mode.
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces switch(config-if)# switchport rate-mode shared switch(config-if)# switchport mode f Step 6 Enable the interfaces and return to configuration mode. switch(config-if)# no shutdown switch(config-if)# exit switch# Step 7 Select the interfaces fc 4/2 through fc 4/10. switch# config t switch(config)# interface fc 4/2 - 10 Step 8 Configure the port speed, rate mode, and port mode on the interfaces.
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces Step 7 Select the interfaces fc 4/7 through fc 4/10. switch# config t switch(config)# interface fc 4/7 - 10 Step 8 Configure the port speed, rate mode, and port mode on the interfaces. switch(config-if)# switchport speed 1000 switch(config-if)# switchport rate-mode shared switch(config-if)# switchport mode f Step 9 Enable the interfaces and return to configuration mode.
Chapter 3 Configuring Fibre Channel Interfaces Configuration Examples for Fibre Channel Interfaces Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 3-50 OL-29284-01, Release 6.
CH A P T E R 4 Configuring Interface Buffers • Information About Interface Buffers, page 4-1 • Configuring Interface Buffers, page 4-21 • Verifying BB_Credit Configuration, page 4-25 Information About Interface Buffers Fibre Channel interfaces use buffer credits to ensure all packets are delivered to their destination.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers • For Generation 2, Generation 3, and Generation 4 switching modules, see the “Buffer Pools” section on page 4-2. Note In the Cisco MDS 9100 Series switches, the groups of ports on the left outlined in white are in dedicated rate mode. The other ports are host-optimized. Each group of 4 host-optimized ports have the same features as for the 32-port switching module.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Note The 48-port and 24-port 8-Gbps modules have dual global buffer pools. Each buffer pool in the 48-port modules support 24 ports and in the 24-port modules each buffer pool supports 12 ports. Figure 4-1 shows the allocation of BB_credit buffers on line cards (24-port and 48-port 4-Gbps line cards).
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Figure 4-3 shows the default BB_credit buffer allocation model for 24-port 8-Gbps switching modules. The minimum BB_credits required to bring up a port is two buffers.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Figure 4-5 shows the default BB_credit buffer allocation model for 24-port 4-Gbps switching modules. The minimum BB_credits required to bring up a port is two buffers.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers When you configure port mode to auto or E on a Generation 3 module, one or two of the ports will not come up for the following configuration: • Port Mode: auto or E for the first half of the ports, the second half of the ports or for all of the ports • Rate Mode: dedicated • Buffer Credits: default value When you configure port mode to auto or E for all ports in the global buffer pool, you need to reconfigure buffer credits
Chapter 4 Configuring Interface Buffers Information About Interface Buffers • Two ports with dedicated rate mode and 8-Gbps speed plus four ports with shared rate mode and 8-Gbps speed (2:1 oversubscription) . • One port with dedicated rate mode and 8-Gbps speed plus three ports with dedicated rate mode and 4-Gbps speed plus two ports with shared rate mode and 8-Gbps speed (1.33:1 oversubscription). • Six ports with dedicated rate mode and 8-Gbps speed.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers • One port with dedicated rate mode and 4-Gbps speed plus three ports with dedicated rate mode and 2-Gbps speed plus two ports with shared rate mode and 4-Gbps speed (4:1 oversubscription) • Six ports with dedicated rate mode and 2-Gbps speed 24-Port 8-Gbps Fibre Channel Module BB_Credit Buffers Table 4-3 lists the BB_credit buffer allocation for the 24-port 8-Gbps Fibre Channel switching module.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers • Three ports with dedicated rate mode and 4-Gbps speed 4/44-Port 8-Gbps Host-Optimized Fibre Channel Module BB_Credit Buffers Table 4-4 lists the BB_credit buffer allocation for the 4/44-port 8-Gbps Fibre Channel switching module.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers 48-Port 4-Gbps Fibre Channel Module BB_Credit Buffers Table 4-5 lists the BB_credit buffer allocation for 48-port 4-Gbps Fibre Channel switching modules.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers • 1 3 5 7 9 11 4-Gbps 4-Gbps 1-Gbps 1-Gbps Dedicated Dedicated Shared Shared Out of Service Out of Service 2 4 6 8 10 12 4-Gbps 1-Gbps 1-Gbps Dedicated Shared Shared Out of Service Out of Service Out of Service Six ports with dedicated rate mode and 2-Gbps speed plus four ports with shared rate mode and 1-Gbps speed plus two ports put out-of-service (see Figure 4-7) For detailed configuration steps
Chapter 4 Configuring Interface Buffers Information About Interface Buffers 24-Port 4-Gbps Fibre Channel Module BB_Credit Buffers Table 4-6 lists the BB_credit buffer allocation for 24-port 4-Gbps Fibre Channel switching modules.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Example Speed and Rate Configuration on a 24-Port 4-Gbps Switching Module 1 2 3 4 5 6 4-Gbps 4-Gbps 4-Gbps 1-Gbps 1-Gbps 1-Gbps Dedicated Dedicated Dedicated Shared Shared Shared 144857 Figure 4-8 18-Port Fibre Channel/4-Port Gigabit Ethernet Multiservice Module BB_Credit Buffers Table 4-7 lists the BB_credit buffer allocation for 18-port 4-Gbps multiservice modules.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Table 4-8 12-Port 4-Gbps Switching Module BB_Credit Buffer Allocation (continued) BB_Credit Buffers Per Port Dedicated Rate Mode 4-Gbps Speed BB_Credit Buffer Allocation Type ISL 1 Total number of BB_credit buffers per module 5488 Total number of performance buffers per module 512 (shared) Fx Port 1. ISL = E port or TE port.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers 4-Port 10-Gbps Switching Module BB_Credit Buffers Table 4-9 lists the BB_credit buffer allocation for 4-port 10-Gbps switching modules.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers BB_Credit Buffers for Fabric Switches This section describes how buffer credits are allocated to Cisco MDS 9000 Fabric switches, and includes the following topics: • Cisco MDS 9148 Fabric Switch BB_Credit Buffers, page 4-16 • Cisco MDS 9148 Fabric Switch BB_Credit Buffers, page 4-16 • Cisco MDS 9124 Fabric Switch BB_Credit Buffers, page 4-17 • Cisco MDS 9222i Multiservice Modular Switch BB_Credit Buffers, page 4-17 Cisco
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Cisco MDS 9124 Fabric Switch BB_Credit Buffers Table 4-12 lists the BB_credit buffer allocation for MDS 9124 Fabric Switches.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers All ports on the Generation 2 and Generation 3 switching modules support extended BB_credits. There are no limitations for how many extended BB_credits you can assign to a port (except for the maximum and minimum limits). If necessary, you can take interfaces out of service to make more extended BB_credits available to other ports.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Note The receive BB_credit value for the remaining three ports depends on the port mode. The default value is 16 for the Fx mode and 255 for E or TE modes. The maximum value is 255 in all modes. This value can be changed as required without exceeding the maximum value of 255 BB_credits.
Chapter 4 Configuring Interface Buffers Information About Interface Buffers Buffer-to-Buffer Credit Recovery Although the Fibre Channel standards require low bit error rates, bit errors do occur. Over time, the corruption of receiver-ready messages, known as R_RDY primitives, can lead to a loss of credits, which can eventually cause a link to stop transmitting in one direction. The Fibre Channel standards provide a feature for two attached ports to detect and correct this situation.
Chapter 4 Configuring Interface Buffers Configuring Interface Buffers Note If you use distance extension (buffer-to-buffer credit spoofing) on ISLs between switches, the BB_SCN parameter on both sides of the ISL needs to be disabled. Receive Data Field Size You can also configure the receive data field size for Fibre Channel interfaces. If the default data field size is 2112 bytes, the frame length will be 2148 bytes.
Chapter 4 Configuring Interface Buffers Configuring Interface Buffers Step 3 Step 4 Command Purpose switch(config-if)# switchport fcrxbbcredit default Applies the default operational value to the selected interface. The operational value depends on the port mode. The default values are assigned based on the port capabilities. switch(config-if)# switchport fcrxbbcredit 5 Assigns a BB_credit of 5 to the selected interface. The range to assign BB_credits is between 1 and 255.
Chapter 4 Configuring Interface Buffers Configuring Interface Buffers Note Use the show interface bbcredit command to display performance buffer values and other BB_credit information.
Chapter 4 Configuring Interface Buffers Configuring Interface Buffers Enabling Buffer-to-Buffer Credit Recovery Buffer-to-buffer credit recovery on ISLs (E or TE ports) is enabled by default. Detailed Steps To use buffer-to-buffer credit recovery on a port, follow these steps: Command Purpose Step 1 switch# config t switch(config)# Enters configuration mode. Step 2 switch(config)# interface fc 1/1 switch(config-if)# Selects the interface and enters interface configuration submode.
Chapter 4 Configuring Interface Buffers Verifying BB_Credit Configuration Command Purpose Step 2 switch(config)# interface fc1/1 switch(config-if)# Selects a Fibre Channel interface and enters interface configuration submode. Step 3 switch(config-if)# switchport fcrxbufsize 2000 Reduces the data field size for the selected interface to 2000 bytes. The default is 2112 bytes and the range is from 256 to 2112 bytes.
Chapter 4 Configuring Interface Buffers Verifying BB_Credit Configuration Transmit B2B Credit is 0 Receive B2B Credit is 12 Receive B2B Credit performance buffers is 48 12 receive B2B credit remaining 0 transmit B2B credit remaining Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 4-26 OL-29284-01, Release 6.
CH A P T E R 5 Configuring Trunking • Information About Trunking, page 5-1 • Guidelines and Limitations, page 5-7 • Default Settings, page 5-11 • Configuring Trunking, page 5-11 • Verifying Trunking Configuration, page 5-13 • Configuration Example for F Port Trunking, page 5-14 Information About Trunking Trunking, also known as VSAN trunking, is a feature specific to switches in the Cisco MDS 9000 Family.
Chapter 5 Configuring Trunking Information About Trunking Trunking E Ports Any other switch Switch 1 ISL E port Switch 1 Switch 2 EISL E port TE port TE port 79938 Figure 5-1 Trunking Note Trunking is not supported by internal ports on both the Cisco Fabric Switch for HP c_Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter. Trunking F Ports Trunking F ports allows interconnected ports to transmit and receive tagged frames in more than one VSAN, over the same physical link.
Chapter 5 Configuring Trunking Information About Trunking Link Number Link Description 3 F PortChannnel with NP port. 4 Trunked F PortChannel with NP port. 5 Trunking NP port with third-party core switch F port.1 1. These features are not supported currently. Key Concepts The trunking feature includes the following key concepts: • TE port—If trunk mode is enabled in an E port and that port becomes operational as a trunking E port, it is referred to as a TE port.
Chapter 5 Configuring Trunking Information About Trunking Table 5-1 Supported Trunking Protocols Trunk Link Default TE-TE port link TF-TN port link Cisco EPP (PTP) 1 FC-LS Rev 1.62 EVFP TF-TNP port link Cisco EPP (PTP) E or F PortChannel Cisco EPP (PCP) TF Port Channel Cisco EPP (PTP and PCP) Third-party TF-TNP port link 1 FC-LS Rev 1.62 EVFP 1. These features are not currently supported. By default, the trunking protocol is enabled on E ports and disabled on F ports.
Chapter 5 Configuring Trunking Information About Trunking Tip The preferred configuration on the Cisco MDS 9000 Family switches is one side of the trunk set to auto and the other side set to on. Note When connected to a third-party switch, the trunk mode configuration on E ports has no effect. The ISL is always in a trunking disabled state. In the case of F ports, if the third-party core switch ACC's physical FLOGI with the EVFP bit is configured, then EVFP protocol enables trunking on the link.
Chapter 5 Configuring Trunking Information About Trunking Default Allowed-Active VSAN Configuration 2 1, 2 , 4, re 3a 5a re o per atio nal . Switch 3 VSAN1 VSAN2 VSAN4 VSAN5 79945 s1 , AN Switch 1 VS VSAN1 VSAN2 VSAN3 VSAN4 VSAN5 VS AN s d , an al. tion a r ope Switch 2 VSAN1 VSAN2 VSAN3 VSANs 1 and 2 are operational. Figure 5-3 You can configure a select set of VSANs (from the allowed-active list) to control access to the VSANs specified in a trunking ISL.
Chapter 5 Configuring Trunking Guidelines and Limitations Operational and Allowed VSAN Configuration Switch 2 VSAN1 VSAN2 VSAN3 Switch 1 VSAN1 VSAN2 VSAN3 VSAN4 VSAN5 VS VS AN ANs 1 s1 , 2, , 2, 5 a 5a re o re ope ra n th e a tional llow . ed list. Switch 3 VSAN1 VSAN2 VSAN4 VSAN5 79946 list. ed w llo . ea nal n th eratio o re op 3a are nd 3 a s1 and AN Ns 1 VS A VS VSANs 1 and 2 are operational. VSANs 1 and 2 are on the allowed list.
Chapter 5 Configuring Trunking Guidelines and Limitations Note • No FC-SP authentication is supported by the NPV switch on the server F ports. • MDS does not enforce the uniqueness of logical pWWNs across VSANs. • DPVM is not supported on trunked F port logins. • The DPVM feature is limited to the control of the port VSAN, since the EVFP protocol does not allow changing the VSAN on which a logical pWWN has done FLOGI.
Chapter 5 Configuring Trunking Guidelines and Limitations Port WWN is 20:4f:00:0d:ec:6d:2b:40 Peer port WWN is 20:0a:00:0d:ec:3f:ab:80 Admin port mode is auto, trunk mode is on snmp link state traps are enabled Port mode is TE Port vsan is 1 Speed is 2 Gbps Rate mode is dedicated Transmit B2B Credit is 16 Receive B2B Credit is 250 B2B State Change Number is 14 Receive data field Size is 2112 Beacon is turned off Trunk vsans (admin allowed and active) (1,100-101,1101,1163-1166,1216,2172,2182-2183) Trunk vs
Chapter 5 Configuring Trunking Guidelines and Limitations Receive B2B Credit is 32 Receive data field Size is 2112 Beacon is turned off Trunk vsans (admin allowed and active) (1,100-101,1101,1163-1166,1216,2172,2 182-2183) Trunk vsans (up) (1,2183) Trunk vsans (isolated) () Trunk vsans (initializing) (1101,1163-1166,1216,2172,2182) Trunking Misconfiguration Examples If you do not configure the VSANs correctly, issues with the connection may occur.
Chapter 5 Configuring Trunking Default Settings Default Settings Table 5-4 lists the default settings for trunking parameters. Table 5-4 Default Trunk Configuration Parameters Parameters Default Switch port trunk mode ON on non-NPV and MDS core switches. OFF on NPV switches. Allowed VSAN list 1 to 4093 user-defined VSAN IDs. Allowed VF-ID list 1 to 4093 user-defined VF-IDs. Trunking protocol on E ports Enabled. Trunking protocol on F ports Disabled.
Chapter 5 Configuring Trunking Configuring Trunking Enabling the F Port Trunking and Channeling Protocol This section describes how to enable the F port trunking and channeling protocol. Prerequisites • To avoid inconsistent configurations, shut all ports before enabling or disabling the trunking protocols. Detailed Steps To enable or disable the F port trunking and channeling protocol, follow these steps: Command Purpose Step 1 switch# config tasf Enters configuration mode.
Chapter 5 Configuring Trunking Verifying Trunking Configuration Step 3 Command Purpose switch(config-if)# switchport trunk allowed vsan 2-4 Changes the allowed list for the specified VSANs. switch(config-if)# switchport trunk allowed vsan add 5 updated trunking membership Expands the specified VSAN (5) to the new allowed list. switch(config-if)# no switchport trunk allowed vsan 2-4 Deletes VSANs 2, 3, and 4.
Chapter 5 Configuring Trunking Configuration Example for F Port Trunking 34 output OLS, 19 LRR, 17 NOS, 12 loop inits Example 5-2 Displays the Trunking Protocol switch# show trunk protocol Trunk protocol is enabled Example 5-3 Displays Per VSAN Information on Trunk Ports switch# show interface trunk vsan 1-1000 fc3/1 is not trunking ... fc3/7 is trunking Vsan 1000 is down (Isolation due to vsan not configured on peer) ...
Chapter 5 Configuring Trunking Configuration Example for F Port Trunking Step 7 Set the port administrative state on NPIV and NPV switches to ON: switch(config)# interface fc1/2 switch(config-if)# shut switch(config-if)# no shut Step 8 Save the configuration. switch(config)# copy running-config startup-config Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide OL-29284-01, Release 6.
Chapter 5 Configuring Trunking Configuration Example for F Port Trunking Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 5-16 OL-29284-01, Release 6.
CH A P T E R 6 Configuring PortChannels • Information About PortChannels, page 6-1 • Prerequisites for PortChannels, page 6-12 • Guidelines and Limitations, page 6-13 • Default Settings, page 6-16 • Configuring PortChannels, page 6-17 • Verifying PortChannel Configuration, page 6-20 • Configuration Examples for F and TF PortChannels, page 6-24 Information About PortChannels This section includes the following topics: • PortChannels Overview, page 6-2 • E PortChannels, page 6-2 • F and T
Chapter 6 Configuring PortChannels Information About PortChannels PortChannels Overview PortChannels refer to the aggregation of multiple physical interfaces into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy (See Figure 6-1). PortChannels can connect to interfaces across switching modules, so a failure of a switching module cannot bring down the PortChannel link.
Chapter 6 Configuring PortChannels Information About PortChannels • Note Provides high availability on an ISL. If one link fails, traffic previously carried on this link is switched to the remaining links. If a link goes down in a PortChannel, the upper protocol is not aware of it. To the upper protocol, the link is still there, although the bandwidth is diminished. The routing tables are not affected by link failure.
Chapter 6 Configuring PortChannels Information About PortChannels PortChanneling and Trunking Switch 1 Switch 2 Switch 1 ISL 1 ISL 2 ISL 3 Port channel Switch 2 EISL 1 EISL 2 EISL 3 Port channel and trunking 79939 Figure 6-3 PortChanneling and trunking are used separately across an ISL.
Chapter 6 Configuring PortChannels Information About PortChannels Figure 6-4 SID1 and DID1-Based Load Balancing Frame 1 Frame 2 SID1, DID1, Exchange 1 Link 1 Link 2 Frame 3 Frame n Frame 1 Frame 2 SID1, DID1, Exchange 2 Link 1 Link 2 Frame 3 Frame n Frame 1 Link 1 Frame 2 Frame 3 Link 2 Frame n 79530 SID2, DID2 Exchange 1 Figure 6-5 illustrates how exchange-based load balancing works.
Chapter 6 Configuring PortChannels Information About PortChannels Figure 6-5 SID1, DID1, and Exchange-Based Load Balancing Frame 1 Frame 2 SID1, DID1, Exchange 1 Link 1 Link 2 Frame 3 Frame n Frame 1 Link 1 Frame 2 Frame 3 Link 2 Frame n 79531 SID1, DID1, Exchange 2 For more information on configuring load balancing and in-order delivery features, see the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide.
Chapter 6 Configuring PortChannels Information About PortChannels Table 6-1 compares ON and ACTIVE modes. Table 6-1 Channel Group Configuration Differences ON Mode ACTIVE Mode No protocol is exchanged. A PortChannel protocol negotiation is performed with the peer ports. Moves interfaces to the suspended state if its operational values are incompatible with the PortChannel. Moves interfaces to the isolated state if its operational values are incompatible with the PortChannel.
Chapter 6 Configuring PortChannels Information About PortChannels Note • Forcing an Interface Addition, page 6-9 • Interface Deletion from a PortChannel, page 6-9 For information about PortChannel support on Generation 2 switching modules, see the “PortChannel Limitations” section on page 3-21. Interface Addition to a PortChannel You can add a physical interface (or a range of interfaces) to an existing PortChannel. The compatible parameters on the configuration are mapped to the PortChannel.
Chapter 6 Configuring PortChannels Information About PortChannels Suspended and Isolated States If the operational parameters are incompatible, the compatibility check fails and the interface is placed in a suspended or isolated state based on the configured mode: • An interface enters the suspended state if the interface is configured in the ON mode. • An interface enters the isolated state if the interface is configured in the ACTIVE mode.
Chapter 6 Configuring PortChannels Information About PortChannels The PortChannel protocol is enabled by default. The PortChannel protocol expands the PortChannel functional model in Cisco MDS switches. It uses the exchange peer parameters (EPP) services to communicate across peer ports in an ISL. Each switch uses the information received from the peer ports along with its local configuration and operational values to decide if it should be part of a PortChannel.
Chapter 6 Configuring PortChannels Information About PortChannels The channel group numbers are selected dynamically, and as such, the administrative configuration of the ports forming the channel group at either end are applicable to the newly created channel group. The channel group number being chosen dynamically may be different across reboots for the same set of PortChannels based on the order of ports that are initialized in the switch.
Chapter 6 Configuring PortChannels Prerequisites for PortChannels Tip • Newly created PortChannels are allocated from the maximum possible PortChannel (128 for Generation 1 or a combination of Generation 1 and Generation 2 switches, or 256 for Generation 2 switches) in a decreasing order based on availability. If all 128 (or 256) numbers are used up, aggregation is not allowed. • You cannot change the membership or delete an autocreated PortChannel.
Chapter 6 Configuring PortChannels Guidelines and Limitations Note On switches with Generation 1 switching modules, or a combination of Generation 1 and Generation 2 switching modules, you can configure a maximum of 128 PortChannels. On switches with only Generation 2 switching modules, or Generation 2 and Generation 3 switching modules, you can configure a maximum of 256 PortChannels. If you misconfigure PortChannels, you may receive a misconfiguration message.
Chapter 6 Configuring PortChannels Guidelines and Limitations Generation 1 PortChannel Limitations This section includes the restrictions on creation and addition of PortChannel members to a PortChannel on Generation 1 hardware: • The 32-port 2-Gbps or 1-Gbps switching module. • The MDS 9140 and 9120 switches.
Chapter 6 Configuring PortChannels Guidelines and Limitations • The Dynamic Port VSAN Management (DPVM) database will be queried only for the first physical FLOGI of each member, so that the port VSAN can be configured automatically. • DPVM does not bind FC_IDs to VSANs, but pWWNs to VSANs. It will be queried only for the physical FLOGI. Valid and Invalid PortChannel Examples PortChannels are created with default values. You can change the default configuration just like any other physical interface.
Chapter 6 Configuring PortChannels Default Settings Figure 6-8 provides examples of invalid configurations. Assuming that the links are brought up in the 1, 2, 3, 4 sequence, links 3 and 4 will be operationally down as the fabric is misconfigured.
Chapter 6 Configuring PortChannels Configuring PortChannels Configuring PortChannels This section includes the following topics: • Configuring PortChannels Using the WizardCreating a PortChannel, page 6-17Configuring PortChannels Using the WizardCreating a PortChannel, page 6-17 • Configuring the PortChannel Mode, page 6-17 • Deleting PortChannels, page 6-18 • Adding an Interface to a PortChannel, page 6-18 • Forcing an Interface Addition, page 6-19 • Deleting an Interface from a PortChannel, p
Chapter 6 Configuring PortChannels Configuring PortChannels Step 3 Command Purpose switch(config-if)# channel mode active Configures the ACTIVE mode. switch(config-if)# no channel mode active Reverts to the default ON mode. Deleting PortChannels Detailed Steps To delete a PortChannel, follow these steps: Command Purpose Step 1 switch# config t switch(config)# Enters configuration mode.
Chapter 6 Configuring PortChannels Configuring PortChannels Command Purpose Step 2 switch(config)# interface fc1/1 - 5 switch(config-if)# Configures the specified range of interfaces. In this example, interfaces from 1/1 to 1/5 are configured. Step 3 switch(config-if)# channel-group 2 Adds physical interfaces 1/1, 1/2, 1/3, 1/4, and 1/5 to channel group 2. If channel group 2 does not exist, it is created.
Chapter 6 Configuring PortChannels Verifying PortChannel Configuration Enabling and Configuring Autocreation Detailed Steps To configure automatic channel groups, follow these steps: Command Purpose Step 1 switch# config t switch(config)# Enters configuration mode. Step 2 switch(config)# interface fc8/13 switch(config- if)# Enters the configuration mode for the selected interface(s).
Chapter 6 Configuring PortChannels Verifying PortChannel Configuration Command Purpose show interface fc slot/port Displays autocreated PortChannels. show port-channel database interface port-channel number Displays the specified PortChannel interface. For detailed information about the fields in the output from these commands, refer to the Cisco MDS NX-OS Command Reference. You can view specific information about existing PortChannels at any time from EXEC mode.
Chapter 6 Configuring PortChannels Verifying PortChannel Configuration fcip2 [down] port-channel 78 Administrative channel mode is active Operational channel mode is active Last membership update succeeded 2 ports in total, 0 ports up Ports: fc2/1 [down] fc2/5 [down] port-channel 79 Administrative channel mode is active Operational channel mode is active Last membership update succeeded First operational port is fcip200 2 ports in total, 2 ports up Ports: fcip101 [up] fcip200 [up] * The show port-channe
Chapter 6 Configuring PortChannels Verifying PortChannel Configuration port-channel 2 ports, fcip1 fcip2 port-channel 2 ports, fc2/1 fc2/5 port-channel 2 ports, fcip101 fcip200 ... 77: first operational port is none [down] [down] 78: first operational port is none [down] [down] 79: first operational port is fcip200 [up] [up] The show port-channel usage command displays details of the used and unused PortChannel numbers.
Chapter 6 Configuring PortChannels Configuration Examples for F and TF PortChannels 1 ports in total, 1 ports up Ports: fc1/1 [up] * Example 6-10 Displays the PortChannel Summary switch# show port-channel summary -----------------------------------------------------------------------------Interface Total Ports Oper Ports First Oper Port -----------------------------------------------------------------------------port-channel 1 1 0 -port-channel 2 1 1 fc8/13 port-channel 3 0 0 -port-channel 4 0 0 -port-c
Chapter 6 Configuring PortChannels Configuration Examples for F and TF PortChannels switch(config)# interface fc2/1-3 switch(config-if)# shut switch(config-if)# switchport mode NP switch(config-if)# switchport speed 4000 switch(config-if)# switchport rate-mode shared switch(config-if)# switchport trunk mode off switch(config-if)# channel-group 1 switch(config-if)# no shut switch(config-if)# exit Step 7 Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the
Chapter 6 Configuring PortChannels Configuration Examples for F and TF PortChannels switch(config-if)# switchport mode NP switch(config-if)# no shut switch(config-if)# exit Step 6 Configure the PortChannel member interfaces on the NPV switch in dedicated mode: switch(config)# interface fc3/1-3 switch(config-if)# shut switch(config-if)# switchport mode NP switch(config-if)# switchport speed 4000 switch(config-if)# switchport rate-mode dedicated switch(config-if)# switchport trunk mode on switch(config-i
CH A P T E R 7 Configuring N Port Virtualization • Information About N Port Virtualization, page 7-1 • Guidelines and Limitations, page 7-8 • Configuring N Port Virtualization, page 7-10 • Verifying NPV Configuration, page 7-13 Information About N Port Virtualization This section includes the following topics: • NPV Overview, page 7-1 • N Port Identifier Virtualization, page 7-2 • N Port Virtualization, page 7-2 • NPV Mode, page 7-4 • NP Ports, page 7-5 • NP Links, page 7-5 • Default
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization Note • Cisco Fabric Switch for HP c-Class BladeSystem • Cisco Fabric Switch for IBM BladeCenter • Cisco Nexus 5000 Series switches NPV is available on these switches only while in NPV mode; if in switch mode, NPV is not available. N Port Identifier Virtualization N port identifier virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port.
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization Figure 7-2 Cisco NPV Fabric Configuration NPV-Core Switch (MDS or 3rd party switch with NPIV support) FC FC F-port V NP-port N SA 20.2.1 VS 5 AN Can have multiple uplinks on different VSANs NPV Device uses the same domains as the NPV-core switches VSAN 10 10.1.
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization NPV Mode A switch is in NPV mode after a user has enabled NPV and the switch has successfully rebooted. NPV mode applies to an entire switch. All end devices connected to a switch that is in NPV mode must log in as an N port to use this feature (loop-attached devices are not supported).
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization NP Ports An NP port (proxy N port) is a port on a device that is in NPV mode and connected to the NPV core switch using an F port. NP ports behave like N ports except that in addition to providing N port behavior, they also function as proxies for multiple, physical N ports. NP Links An NP link is basically an NPIV uplink to a specific end device.
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization Figure 7-4 shows the internal FLOGI flows between an NPV core switch and an NPV device. Figure 7-4 Internal FLOGI Flows NPV Core Switch fc 5/10 fwwn fc 1/5 pwwn 184572 nwwn NPV Device Table 7-1 identifies the internal FLOGI parameters that appear in Figure 7-4. Table 7-1 Internal FLOGI Parameters Parameter Derived From pWWN The fWWN of the NP port. nWWN The VSAN-based sWWN of the NPV device.
Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization NPV CFS Distribution over IP NPV devices use only IP as the transport medium. CFS uses multicast forwarding for CFS distribution. NPV devices do not have ISL connectivity and FC domain. To use CFS over IP, multicast forwarding has to be enabled on the Ethernet IP switches all along the network that physically connects the NPV switch.
Chapter 7 Configuring N Port Virtualization Guidelines and Limitations Disruptive Disruptive load balance works independent of automatic selection of interfaces and a configured traffic map of external interfaces. This feature forces reinitialization of the server interfaces to achieve load balance when this feature is enabled and whenever a new external interface comes up.
Chapter 7 Configuring N Port Virtualization Guidelines and Limitations • NPV devices can connect to multiple NPV core switches. In other words, different NP ports can be connected to different NPV core switches. • NPV supports NPIV-capable module servers (nested NPIV). • Only F, NP, and SD ports are supported in NPV mode. • In the case of servers that are booted over the SAN with NPV, if an NPV link failover occurs, servers will lose access to their boot LUN temporarily.
Chapter 7 Configuring N Port Virtualization Configuring N Port Virtualization NPV and Port Security Configuration Guidelines Port security is enabled on the NPV core switch on a per interface basis. To enable port security on the NPV core switch for devices logging in via NPV, you must adhere to the following requirements: • The internal FLOGI must be in the port security database so that, the port on the NPV core switch will allow communications and links.
Chapter 7 Configuring N Port Virtualization Configuring N Port Virtualization switch# copy running bootflash:filename The configuration can be reapplied later using the following command: switch# copy bootflash:filename running-config Detailed Steps To configure NPV using the CLI, perform the following tasks: Command Purpose Step 1 switch# config t switch(config)# On the NPV core switch, enters configuration mode.
Chapter 7 Configuring N Port Virtualization Configuring N Port Virtualization Command Purpose Step 10 switch(config)# vsan database switch(config-vsan-db)# vsan 12 interface fc 1/1 - 6 switch(config-vsan-db)# vsan 13 interface fc 1/1 - 6 Configures the port VSANs for the F ports on the NPV device. Step 11 switch(config-npv)# no npv enable switch(config)# Terminates session and disables NPV mode, which results in a reload of the NPV device.
Chapter 7 Configuring N Port Virtualization Verifying NPV Configuration Enabling the Global Policy for Disruptive Load Balancing Disruptive load balancing allows you to review the load on all the external interfaces and balance the load disruptively. Disruptive load balancing is done by moving the servers using heavily loaded external interfaces, to the external interfaces running with fewer loads.
Chapter 7 Configuring N Port Virtualization Verifying NPV Configuration Verifying NPV To view all the NPV devices in all the VSANs that the aggregator switch belongs to, enter the show fcns database command.
Chapter 7 Configuring N Port Virtualization Verifying NPV Configuration switch# show npv flogi-table -------------------------------------------------------------------------------SERVER EXTERNAL INTERFACE VSAN FCID PORT NAME NODE NAME INTERFACE -------------------------------------------------------------------------------fc1/19 1 0xee0008 10:00:00:00:c9:60:e4:9a 20:00:00:00:c9:60:e4:9a fc1/9 fc1/19 1 0xee0009 20:00:00:00:0a:00:00:01 20:00:00:00:c9:60:e4:9a fc1/1 fc1/19 1 0xee000a 20:00:00:00:0a:00:00:02
Chapter 7 Configuring N Port Virtualization Verifying NPV Configuration Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 7-16 OL-29284-01, Release 6.
CH A P T E R 8 Configuring FlexAttach Virtual pWWN • Information About FlexAttach Virtual pWWN, page 8-1 • Guidelines and Limitations, page 8-3 • Configuring FlexAttach Virtual pWWN, page 8-3 • Verifying FlexAttach Virtual pWWN Configuration, page 8-5 • Monitoring FlexAttach Virtual pWWN, page 8-6 Information About FlexAttach Virtual pWWN This section includes the following topics: • FlexAttach Virtual pWWN, page 8-1 • Difference Between San Device Virtualization and FlexAttach Port Virtualiz
Chapter 8 Configuring FlexAttach Virtual pWWN Information About FlexAttach Virtual pWWN • Replacement to the same port—A failed server can be replaced onto the same port without changing the SAN. The new server gets a same pWWN as the failed server because the virtual pWWN is assigned to the port. • Replacement to (spare)—A spare server, which is on the same NPV device or a different NPV device) can be brought online without changes to the SAN.
Chapter 8 Configuring FlexAttach Virtual pWWN Guidelines and Limitations Security Settings for FlexAttach Virtual pWWN Security settings for the FlexAttach virtual pWWN feature are done by port security at the NPV core. Node WWN of the end device is used to provide physical security. For more details on enabling port security, refer to the Cisco MDS 9000 Family NX-OS Security Configuration Guide.
Chapter 8 Configuring FlexAttach Virtual pWWN Configuring FlexAttach Virtual pWWN Troubleshooting Tips • When the interface-list value is not included in the command, virtual pWWN is assigned globally. • All the interfaces mentioned in the interface-list value must be in a shut state. Manually Assigning FlexAttach Virtual pWWN You can manually assign a WWN to the interface, without generating it through the switch.
Chapter 8 Configuring FlexAttach Virtual pWWN Verifying FlexAttach Virtual pWWN Configuration Restrictions • The specified virtual pWWN and the real pWWN must not be logged in. Detailed Steps To map pWWN to virtual pWWN, perform this task: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch# (config)# flex-attach virtual-pwwn vpwwn pwwn pwwn Maps the pWWN to the virtual pWWN. Step 3 switch# (config)# flex-attach commit Commits the configuration.
Chapter 8 Configuring FlexAttach Virtual pWWN Monitoring FlexAttach Virtual pWWN Verifying the End Device To verify that the end device is logged with the correct virtual WWNs, use the show fcns database command on the NPV core. (See Example 8-2.
Chapter 8 Configuring FlexAttach Virtual pWWN Monitoring FlexAttach Virtual pWWN Error Description Workaround real port WWN and virtual WWN cannot be same This occurs when you try to configure FlexAttach with a similar value for pWWN and virtual pWWN. Use different values for pWWN and virtual pWWN, as similar values for pWWN and virtual pWWn are not allowed. Virtual port WWN already exists This occurs when you try to configure an already defined pWWN to a different interface.
Chapter 8 Configuring FlexAttach Virtual pWWN Monitoring FlexAttach Virtual pWWN Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide 8-8 OL-29284-01, Release 6.
CH A P T E R 9 Configuring Port Tracking The port tracking feature is unique to the Cisco MDS 9000 Family of switches. This feature uses information about the operational state of the link to initiate a failure in the link that connects the edge device. This process of converting the indirect failure to a direct failure triggers a faster recovery process towards redundant links.
Chapter 9 Configuring Port Tracking Guidelines and Limitations Figure 9-1 Traffic Recovery Using Port Tracking ISL2 X Direct link 1 WAN or MAN X FC FC 120490 WAN or MAN The port tracking feature monitors and detects failures that cause topology changes and brings down the links connecting the attached devices.
Chapter 9 Configuring Port Tracking Configuring Port Tracking Configuring Port Tracking Port tracking has the following features: • The application brings the linked port down when the tracked port goes down. When the tracked port recovers from the failure and comes back up again, the tracked port is also brought up automatically (unless otherwise configured). • You can forcefully continue to keep the linked port down, even though the tracked port comes back up.
Chapter 9 Configuring Port Tracking Configuring Port Tracking Binding a Tracked Port Operationally When you configure the first tracked port, operational binding is automatically in effect. When you use this method, you have the option to monitor multiple ports or monitor ports in one VSAN. Detailed Steps To operationally bind a tracked port, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode.
Chapter 9 Configuring Port Tracking Configuring Port Tracking Tracking Multiple Ports Detailed Steps To track multiple ports, follow these steps: Command Purpose Step 1 switch# config t Enters configuration mode. Step 2 switch(config)# interface fc8/6 Configures the specified interface and enters the interface configuration submode. You can now configure tracked ports. Note Step 3 Tracks interface fc8/6 with interface port-channel 1.
Chapter 9 Configuring Port Tracking Displaying Port Tracking Information Information AboutForceful Shutdown If a tracked port flaps frequently, then tracking ports using the operational binding feature may cause frequent topology change. In this case, you may choose to keep the port in the down state until you are able to resolve the reason for these frequent flaps.
Chapter 9 Configuring Port Tracking Displaying Port Tracking Information 269946 frames input, 22335204 bytes 0 discards, 0 errors 0 CRC, 0 unknown class 0 too long, 0 too short 205007 frames output, 10250904 bytes 0 discards, 0 errors 0 input OLS, 0 LRR, 0 NOS, 0 loop inits 2 output OLS, 2 LRR, 0 NOS, 1 loop inits 0 receive B2B credit remaining 0 transmit B2B credit remaining ...
Chapter 9 Example 9-4 Configuring Port Tracking Displays a Forced Shutdown Configuration switch# show interface fc 1/5 fc1/5 is up Hardware is Fibre Channel, FCOT is short wave laser Port WWN is 20:05:00:05:30:00:47:9e Admin port mode is F Port mode is F, FCID is 0x710005 Port vsan is 1 Speed is 1 Gbps Transmit B2B Credit is 64 Receive B2B Credit is 16 Receive data field Size is 2112 Beacon is turned off Port track mode is force_shut <--this port remains shut even if the tracked port is back up Cisco
INDEX bandwidth fairness Symbols default settings * (asterisk) 3-25 example configurations first operational port 6-20 3-11 shared resources 3-11 48-port 8-Gbps switching modules default settings 12-port 4-Gbps switching modules BB_credit buffers 3-25 example configurations 4-13 configuration guidelines default settings oversubscription 3-47, 3-48, 4-10 See also switching modules Numerics 3-46, 4-7 See also switching modules 3-28 4-port 10-Gbps switching modules 3-25 BB_credit buff
Index auto port mode description 2-5 interface configuration configuring 2-11, 2-28 description 2-11, 2-28 B port mode 2-2 autosensing speed description Generation 2 switching modules 2-5 interface modes 2-10 2-5 bridge port mode. See B port mode buffer pools B Generation 2 switching modules buffer-to-buffer credits. See BB_credits bandwidth fairness disabling 3-41 enabling 3-40 4-2 buffer-to-buffer start change.
Index PortChannel links configuring speeds 6-2 2-10, 2-25 enhanced ISLs. See EISLs default settings E port mode deleting from PortChannels classes of service description disabling 2-3 E ports 32-port guidelines 2-22 displaying information 2-18 enabling 32-port switching module configuration guidelines 6-14 isolation modes 2-22 2-9 2-2 to 2-6 operational states 2-6 exchange link parameter. See ELP performance buffers expansion port mode.
Index interface modes displaying information 2-5 See also F ports; FL ports 2-42 to 2-49 displaying SFP information 2-5 2-48 forced addition to PortChannels isolated states G 6-9 suspended states configuring mgmt0 interfaces 4-18 port index allocations PortChannel links 3-29 configuring rate modes 3-30 default settings L LEDs 3-25 beacon mode states 3-1 to 3-3 disabling ACL adjacency sharing displaying port resources 4-17, 4-19 interface capabilities 3-44 out-of-service interfac
Index configuring autocreation N converting autocreated groups to manually configured 6-12 NL ports interface modes creating channel group 2-6 description nonparticipating codes description 6-9 enabling adding interfaces 7-2 6-8, 6-18 administratively down 7-10 2-9 comparison with trunking 7-5 N port identifier virtualization.
Index port speed group SD ports configuring configuring 2-26 port speeds 2-22 SFPs configuring displaying transmitter types 2-10, 2-25 configuring on Generation 2 switching module interfaces 3-29 displaying configuration transmitter types 2-48 2-12 shared rate mode 3-30 description port tracking 3-8 migrating from dedicated rate mode default settings description 9-2 oversubscription displaying information enabling migrating to dedicated rate mode 9-1 9-6 exchange based 9-2 flow
Index TL ports allowed list ALPA caches multiplexing traffic 2-14 configuring 2-22, 2-29 description 2-13 displaying information 2-50 virtual devices 2-51 2-4 VSAN interfaces creating private devices 5-11 2-38 description 2-50 2-18 displaying information 2-55 VSANs tracked ports allowed-active binding operationally 5-7 configuring allowed-active lists 9-4 translative loop port mode.
Index Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide IN-8 OL-29284-01, Release 6.
