McDATA® Products in a SAN Environment Planning Manual P/N 620-000124-510 REV A
Record of Revisions and Updates Revision Date Description 620-000124-000 5/2002 Initial release of the manual. 620-000124-100 9/2002 Revision of the manual to describe the Intrepid 6140 Director, Sphereon 4500 Fabric Switch, and Release 6.3 of the Enterprise Fabric Connectivity Manager application. 620-000124-200 2/2003 Revision of the manual to include additional information and describe Release 7.1 of the Enterprise Fabric Connectivity Manager application.
Contents Preface .......................................................................................................................... xiii Chapter 1 Introduction to McDATA Multi-Protocol Products Product Overview.............................................................................1-2 Multi-Protocol Hardware .........................................................1-2 SAN Management Applications..............................................1-5 Directors .......................................
Contents Chapter 2 Product Management Product Management.......................................................................2-2 Out-of-Band Management .......................................................2-2 Inband Management .................................................................2-5 Management Interface Summary............................................2-6 Management Server Support ..........................................................2-7 Management Server Specifications .............
Contents Fabric Scalability ..................................................................... 3-39 Obtaining Professional Services............................................ 3-40 Mixed Fabric Design Considerations .......................................... 3-40 FCP and FICON in a Single Fabric ....................................... 3-41 Multiple Data Transmission Speeds in a Single Fabric ..... 3-51 FICON Cascading ..........................................................................
Contents SANtegrity Binding.................................................................5-19 PDCM Arrays...........................................................................5-20 Preferred Path ..........................................................................5-23 Zoning .......................................................................................5-25 Server and Storage-Level Access Control ............................5-29 Security Best Practices .................................
Contents Appendix A Product Specifications Director, Fabric Switch, and SAN Router Specifications .......... A-1 Dimensions ............................................................................... A-1 Power Requirements ............................................................... A-3 Heat Dissipation....................................................................... A-5 Clearances .................................................................................
Contents viii McDATA Products in a SAN Environment - Planning Manual
Tables 2-1 Out-of-Band and Inband Product Support Summary ............................ 2-6 4-1 4-2 4-3 4-4 mSAN Routing Domain ............................................................................ mSAN Supported Limits ........................................................................... mFCP Versus iFCP ..................................................................................... Transport Technology Comparison .........................................................
Tables x McDATA Products in a SAN Environment - Planning Manual
Figures 1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 1-10 1-11 Cabinet-Mount McDATA Products ........................................................... 1-4 Intrepid 6064 Director .................................................................................. 1-9 Intrepid 6140 Director ................................................................................ 1-11 Intrepid 10000 Director .............................................................................. 1-12 Sphereon 3232 Fabric Switch .......
Figures xii 3-9 3-10 3-11 3-12 3-13 3-14 3-15 3-16 3-17 3-18 Full Mesh Fabric .......................................................................................... 2-by-14 Core-to-Edge Fabric ...................................................................... Example Multiswitch Fabric ...................................................................... ISL Oversubscription .................................................................................. Device Locality .......................
Preface This publication is part of a documentation suite that supports McDATA® multi-protocol switching and routing products, including the: • Intrepid® 6064 Director. • Intrepid 6140 Director. • Intrepid 10000 Director. • Sphereon™ 3232 Fabric Switch. • Sphereon 4300 Fabric Switch. • Sphereon 4400 Fabric Switch. • Sphereon 4500 Fabric Switch. • Sphereon 4700 Fabric Switch. • Eclipse™ 1620 SAN Router. • Eclipse 2640 SAN Router.
Preface Chapter 1, Introduction to McDATA Multi-Protocol Products This chapter provides an overview of McDATA multi-protocol products, and describes product performance and connectivity, security, and serviceability features. Chapter 2, Product Management - This chapter describes out-of-band and inband product management; the management server; product firmware; backup and restore features; and software. Overviews of the graphical user interfaces (GUIs) and command line interface (CLI) are included.
Preface Related Publications Other publications that provide additional information about McDATA products include: • Intrepid 6064 and 6140 Directors: — McDATA Intrepid 6064 and 6140 Directors Element Manager User Manual (620-000172). — McDATA Intrepid 6064 Director Installation and Service Manual (620-000108). — McDATA Intrepid 6140 Director Installation and Service Manual (620-000157). • Intrepid 10000 Director: — McDATA Intrepid 10000 Director Element Manager User Manual (620-000227).
Preface • Sphereon 4700 Fabric Switch: — McDATA Sphereon 4700 Fabric Switch Element Manager User Manual (620-000242). — McDATA Sphereon 4700 Fabric Switch Installation and Service Manual (620-000239). • Eclipse 1620 SAN Router: — McDATA Eclipse 1620 SAN Router Administration and Configuration Manual (620-000206). — McDATA Eclipse 1620 SAN Router Installation and Service Manual (620-000205).
Preface — McDATA EFCM Lite Installation Instructions (958-000171). — McDATA FC-512 Fabricenter Equipment Cabinet Installation and Service Manual (620-000100). Ordering Printed Manuals Where to Get Help To order a printed copy of this publication, submit a purchase order as described in Ordering McDATA Documentation Instructions at http://www.mcdata.com. To obtain documentation CD-ROMs, contact your McDATA sales representative. For technical support, contact the McDATA Solution Center.
Preface All other trademarked terms, indicated by a registered trademark symbol (®) or trademark symbol (™) on first use in this publication, are trademarks of their respective owners in the United States or other countries or both. xviii Forwarding Publication Comments Please send comments to the McDATA Solution Center by telephone, fax, or e-mail. The numbers and e-mail address are listed above. Please identify the page numbers and details.
Preface United States and Canada UL Certification International Safety Conformity Declaration (CB Scheme) The C-UL-US mark on a product indicates compliance with American National Standards Institute (ANSI) and Standards Council of Canada (SCC) safety requirements as tested, evaluated, and certified by Underwriters Laboratories Inc. (UL) and Underwriters Laboratories of Canada (ULC).
Preface In addition, the European Union (EU) Council has implemented a series of directives that define product safety standards for member countries. The following directives apply: • Products conform with all protection requirements of EU directive 89/336/EEC (Electromagnetic Compatibility Directive) in accordance with the laws of the member countries relating to EMC emissions and immunity.
Preface Australia and New Zealand C-Tick Mark The Australia and New Zealand regulatory compliance mark (C-tick mark) on a product indicates compliance with regulatory requirements for safety and EMC (for information technology equipment) as set forth by the Australian Communications Authority (ACA) and the Radio Spectrum Management Group (RSM) of New Zealand.
Preface Japanese VCCI Statement Korean MIC Mark Mexican NOM Mark NOM Russian GOST Certification xxii The Voluntary Control Council for Interference (VCCI) statement below applies to information technology equipment, and indicates product compliance with Japanese regulatory requirements. The statement indicates a product is a Class A or Class B product, and in a domestic environment may cause radio interference, in which case the user is required to take corrective actions.
Preface Danger and Attention Statements The following DANGER statement appears in this publication and describes safety practices that must be observed while installing or servicing a product. A DANGER statement provides essential information or instructions for which disregard or noncompliance may result in death or severe personal injury. The statement appears in English, followed by translations to: • Chinese (simplified - People’s Republic of China). • Chinese (traditional - Taiwan).
Preface DANGER Utiliser les câbles d’alimentation fournis. S’assurer que la prise de courant du local est du type correct, délivre la tension requise et est correctement raccordée à la terre. GEFAHR Die mitgelieferten Netzkabel verwenden. Sicherstellen, dass die verwendete Netzsteckdose dem vorgeschriebenen Typ entspricht, die erforderliche Spannung liefert und einwandfrei geerdet ist. PERICOLO Usare il cavo di alimentazione in dotazione.
Preface PELIGRO Utilice los cables de alimentación proporcionados. Asegúrese que el receptáculo tomacorriente para la instalación sea el tipo correcto, suministre el voltaje necesario, y que esté apropiadamente puesto a tierra. PELIGRO Utilice los cables de alimentación proporcionados. Asegúrese que el receptáculo tomacorriente para la instalación sea del tipo correcto, suministre el voltaje necesario, y que esté apropiadamente conectado a tierra.
Preface xxvi McDATA Products in a SAN Environment - Planning Manual
1 Introduction to McDATA Multi-Protocol Products The enterprise-level storage area network (SAN) of today is typically complex and managed at the device layer. These problems result in SANs that use storage assets inefficiently, and are complex, error prone, expensive, and time-consuming to manage.
Introduction to McDATA Multi-Protocol Products 1 Product Overview McDATA provides storage network solutions that are integrated across a variety of platforms, original equipment manufacturers (OEMs), and locations. Solutions are modular and support multiple technologies (current and future), protocols, and data transmission speeds. These solutions include: Multi-Protocol Hardware • Multi-protocol hardware. • SAN management applications.
Introduction to McDATA Multi-Protocol Products 1 — 32-port Sphereon 3232 Fabric Switch. — 12-port Sphereon 4300 Fabric Switch. The switch provides both switched fabric and Fibre Channel arbitrated loop (FC-AL) connectivity. — 16-port Sphereon 4400 Fabric Switch. The switch provides both switched fabric and FC-AL connectivity. — 24-port Sphereon 4500 Fabric Switch. The switch provides both switched fabric and FC-AL connectivity. — 32-port Sphereon 4700 Fabric Switch.
Introduction to McDATA Multi-Protocol Products 1 3. Intrepid 6064 Director. 4. Rack-mount management server. 5. Sphereon 4300 Fabric Switch. 6. Sphereon 4500 Fabric Switch. 7. Intrepid 6140 Director. 8. Eclipse 1620 SAN Router. 9. Eclipse 2640 SAN Router. 10. Intrepid 10000 Director.
Introduction to McDATA Multi-Protocol Products 1 SAN Management Applications McDATA offers the following SAN management applications installed on the rack-mount management server: • SANavigator® application - The SANavigator application (Version 4.2 or later) is an integrated software package that provides management of an enterprise-wide, heterogeneous SAN (with multiple vendor applications) from a single console.
Introduction to McDATA Multi-Protocol Products 1 Refer to Chapter 2, Product Management for information about SAN management applications and the management server. Chapter 2 also describes switch management through: • The Internet using a product’s EFCM Basic Edition interface. • Inband (Fibre Channel) application clients. • Simple network management protocol (SNMP) workstations. • A command line interface (CLI) or PC-attached Telnet session.
Introduction to McDATA Multi-Protocol Products 1 Director Performance Directors provide the following general performance features: • High bandwidth - Ports on Intrepid-series directors provide full-duplex serial data transfer at a rate of 1.0625, 2.1250, or 10.2000 gigabits per second (Gbps).
Introduction to McDATA Multi-Protocol Products 1 — A multiswitch fabric topology provides the ability to connect directors (and other McDATA switch elements) through expansion ports (E_Ports) and interswitch links (ISLs) to form a Fibre Channel fabric. Director elements receive data from a device and, based on the destination N_Port address, route the data through the fabric (and possibly through multiple switch elements) to the destination device.
Introduction to McDATA Multi-Protocol Products 1 Figure 1-2 Intrepid 6064 Director The director supports McDATA’s non-blocking extendable open network (EON™) architecture and concurrent firmware downloads through hot code activation (HotCAT®) technology. The director also provides a modular design that enables quick removal and replacement of FRUs, including: • Cable management assembly and front bezel with power (green) and system error (amber) light-emitting diodes (LEDs).
Introduction to McDATA Multi-Protocol Products 1 — Universal port module (UPM) cards. Each UPM card provides four 2.1250 Gbps Fibre Channel port connections through duplex SFP fiber-optic transceivers. — Ten-gigabit port module (XPM) cards. Each XPM card provides one 10.2000 Gbps Fibre Channel port connection through a duplex ten-gigabit small form factor pluggable (XFP) fiber-optic transceiver.
Introduction to McDATA Multi-Protocol Products 1 Figure 1-3 Intrepid 6140 Director • A minimum of 16 to a maximum of 35 Fibre Channel port cards as follows: — UPM cards. Each UPM card provides four 2.1250 Gbps Fibre Channel port connections through duplex SFP fiber-optic transceivers. — XPM cards. Each XPM card provides one 10.2000 Gbps Fibre Channel port connection through a duplex XFP fiber-optic transceiver.
Introduction to McDATA Multi-Protocol Products 1 Intrepid 10000 Director Figure 1-4 The Intrepid 10000 Director is a fourth-generation, enterprise-class product that provides switched fabric connectivity for up to 256 Fibre Channel devices operating at 1.0625 or 2.1250 Gbps, or up to 64 devices operating at 10.2000 Gbps.
Introduction to McDATA Multi-Protocol Products 1 concurrently upgraded. The combination of high port count and the FlexPar feature enables an enterprise to use the director at the core of both small and large SAN fabrics. For example: • Large fabrics built around the director require fewer fabric elements (directors and switches) and ISLs. Large fabrics benefit from deterministic non-blocking performance, less ISL congestion, and better cable management.
Introduction to McDATA Multi-Protocol Products 1 — Optical paddles that operate at 1.0625 or 2.1250 Gbps provide eight Fibre Channel port connections through duplex SFP fiber-optic transceivers. A fully-populated director supports up to 256 port connections at 1.0625 or 2.1250 Gbps data rates. — Optical paddles that operate at 10.2000 Gbps provide two Fibre Channel port connections through duplex XFP fiber-optic transceivers. A fully-populated director supports up to 64 port connections at the 10.
Introduction to McDATA Multi-Protocol Products 1 Fabric Switch Performance Fabric switches provide an availability of 99.9% through a redundant configuration of power supplies and cooling fans. When an active FRU (power supply or fan) fails, the backup takes over operation automatically to maintain switch and Fibre Channel link operation. Availability is also provided through concurrent firmware upgrades and spare or unused Fibre Channel ports. Excluding an availability of 99.
Introduction to McDATA Multi-Protocol Products 1 • Up to 32 duplex SFP fiber-optic port transceivers. Shortwave laser transceivers are available for transferring data over multimode fiber-optic cable. Longwave laser transceivers are available for transferring data over singlemode fiber-optic cable. Fiber-optic cables attach to switch port transceivers with duplex LC connectors. NOTE: The Sphereon 3232 Fabric Switch can be purchased at a discount price with the Flexport Technology feature.
Introduction to McDATA Multi-Protocol Products 1 Switch ports can be configured as: • Fabric ports (F_Ports) to provide direct connectivity for switched fabric devices. • Fabric loop ports (FL_Ports) to provide switched arbitrated loop connectivity and fabric attachment for FC-AL devices. The switch supports: — Connectivity of public loop devices and private loop devices. Refer to Public Versus Private Devices for information. — Configuration of public arbitrated loops and private arbitrated loops.
Introduction to McDATA Multi-Protocol Products 1 Sphereon 4400 Fabric Switch Figure 1-7 The Sphereon 4400 Fabric Switch operates at 1.0625, 2.1250, or 4.2500 Gbps, provides connectivity through 16 Fibre Channel generic mixed ports (GX_Ports), and supports EON architecture and HotCAT technology. Figure 1-7 illustrates the switch. Sphereon 4400 Fabric Switch Switch ports can be configured as: • F_Ports to provide direct connectivity for switched fabric devices.
Introduction to McDATA Multi-Protocol Products 1 • Up to 16 duplex SFP fiber-optic port transceivers. Tri-rate shortwave laser (1.0625, 2.1250, or 4.2500 Gbps) transceivers are available for transferring data over multimode fiber-optic cable. Tri-rate longwave laser transceivers are available for transferring data over singlemode fiber-optic cable. Fiber-optic cables attach to switch port transceivers with duplex LC connectors.
Introduction to McDATA Multi-Protocol Products 1 Switch ports can be configured as: • F_Ports to provide direct connectivity for switched fabric devices. • FL_Ports to provide switched arbitrated loop connectivity and fabric attachment for FC-AL devices. The switch supports: — Connectivity of public loop devices and private loop devices. Refer to Public Versus Private Devices for information. — Configuration of public arbitrated loops and private arbitrated loops.
Introduction to McDATA Multi-Protocol Products 1 Sphereon 4700 Fabric Switch Figure 1-9 The Sphereon 4700 Fabric Switch operates at 1.0625, 2.1250, or 4.2500 Gbps, provides connectivity through 32 Fibre Channel generic mixed ports (GX_Ports), and supports FICON, EON architecture, and HotCAT technology. Figure 1-9 illustrates the switch. Sphereon 4700 Fabric Switch Switch ports can be configured as: • F_Ports to provide direct connectivity for switched fabric devices.
Introduction to McDATA Multi-Protocol Products 1 • Up to 32 duplex SFP fiber-optic port transceivers. Tri-rate shortwave laser (1.0625, 2.1250, or 4.2500 Gbps) transceivers are available for transferring data over multimode fiber-optic cable. Tri-rate longwave laser transceivers are available for transferring data over singlemode fiber-optic cable. Fiber-optic cables attach to switch port transceivers with duplex LC connectors.
Introduction to McDATA Multi-Protocol Products 1 SAN Router Performance • Perform SAN routing functions - SAN routers provide FCPprotocol, router port (R_Port) connectivity between local Fibre Channel fabrics (SAN routing). A SAN routing solution provides interoperable FCP connectivity and consolidates IT resources but ensures a disruption in one fabric remains isolated and does not propagate to other fabrics. Refer to SAN Routing for detailed information.
Introduction to McDATA Multi-Protocol Products 1 — Data compression - SAN router software identifies repetitive information in an output data stream and applies a compression algorithm to ensure the data is more compact and efficiently transmitted. — FastWrite technology - FastWrite software improves write performance over WANs by responding to initiator write commands with local transfer ready (XFR_RDY) commands, and buffering output data at the SAN router closest to the corresponding target device.
Introduction to McDATA Multi-Protocol Products 1 Figure 1-10 Eclipse 1620 SAN Router SAN router ports operate as follows: • Two user-configured FCP (FIBRE CHANNEL 1 and 2) ports provide 1.0625 Gbps Fibre Channel storage connectivity using SFP port connectors. FCP ports can be configured for: — Auto negotiation (FC-Auto) operation. This is the default selection.
Introduction to McDATA Multi-Protocol Products 1 — IP network connectivity (iFCP or iSCSI protocol) at up to full-duplex GbE (1,000 Mbps) port transmission speed, using only the SFP port connector. Refer to Intelligent Port Speed for detailed information. For SFP port connectors, shortwave laser transceivers are available for transferring data over multimode fiber-optic cable. Longwave laser transceivers are available for transferring data over singlemode fiber-optic cable.
Introduction to McDATA Multi-Protocol Products 1 SAN router ports operate as follows: • Twelve user-configured FCP ports (1 through 12) provide 1.0625 or 2.1250 Gbps FCP storage connectivity and UDP-based network connectivity using SFP port connectors. FCP ports can be configured for: — UDP-based network connectivity (mFCP protocol) at full-duplex GbE (1,000 Mbps) port transmission speed. — Fibre Channel auto negotiation (FC-Auto) operation. This is the default selection.
Introduction to McDATA Multi-Protocol Products 1 Product Features In addition to the characteristics and performance features described in this chapter, McDATA managed products also provide a variety of: Connectivity Features • Connectivity features. • Security features. • Serviceability features. McDATA directors, fabric switches, SAN routers, and their associated Element Manager applications support the following connectivity features.
Introduction to McDATA Multi-Protocol Products 1 • State change notification - Directors and the Sphereon 3232 Fabric Switch support a state change notification function that allows attached N_Ports to request notification when other N_Ports change operational state. Sphereon 4000-series fabric switches and FCP ports on SAN routers support a state change notification function that allows attached N_Ports and NL_Ports to request notification when fabric or loop-attached devices change operational state.
Introduction to McDATA Multi-Protocol Products 1 — Sphereon 4300 Fabric Switch - All switch ports are preset to a BB_Credit value of 5. By enabling the full fabric PFE key, the switch provides a port buffer pool of 144 receive BB_Credits and the per-port value is increased to 12, providing a data transmission distance of up to 24 km at 1.0625 Gbps and 12 km at 2.1250 Gbps. Refer to Full Fabric for information.
Introduction to McDATA Multi-Protocol Products 1 • Port binding - Directors and fabric switches support an optional feature that binds an attached Fibre Channel device to a specified product port through the device’s WWN. NOTE: SAN routers support port binding only for R_Ports. Security Features SAN management and Element Manager applications offer the following security features for McDATA switching products. Products or product classes that do not support a security feature are noted.
Introduction to McDATA Multi-Protocol Products 1 • Zoning - System administrators can create zones that provide product access control to increase network security, differentiate between operating systems, and prevent data loss or corruption. Zoning can be implemented in conjunction with server-level access control and storage device access control.
Introduction to McDATA Multi-Protocol Products 1 — Ethernet link status. — Fibre Channel link status. In addition, threshold alerts and a threshold alert log notify users when the transmit (Tx) or receive (Rx) throughput reaches a specified value for configured ports. • System alerts, system logs, message logs, SAN reports, and product reports display the following SAN router information at the management server or remote workstations: — SAN router status. — Port connectivity status.
Introduction to McDATA Multi-Protocol Products 1 • Directors and fabric switches provide an internal modem for use by support personnel to dial in to the management server for event notification and to perform remote diagnostics. NOTE: SAN routers do not provide modem support.
Introduction to McDATA Multi-Protocol Products 1 • SNMP management for directors and fabric switches using the following MIBs as defined by Internet Engineering Task Force (IETF) working documents, request for comment (RFC) memorandums, and McDATA: — Fibre Channel Management Framework Integration MIB (FC-MGMT-MIB) - This MIB (also called the Fibre Alliance MIB) defines an integrated management environment for Fibre Channel-attached devices. The MIB runs on the management server.
Introduction to McDATA Multi-Protocol Products 1 — VLAN Bridge MIB Module (Q-BRIDGE-MIB) - This MIB defines objects to manage virtual local area network (VLAN) bridging enhancements defined by IEEE 802.1Q-1998. — RFC 1213 - MIB-II - This MIB defines managed objects for the Internet community. — RFC 1354 - IP Forwarding Table - This MIB defines objects that manage IP-based Internet routing.
2 Product Management This chapter describes the management of McDATA multi-protocol products, including Intrepid-series directors, Sphereon-series fabric switches, and Eclipse-series SAN routers. The chapter specifically describes: • Product management, including out-of-band (non-Fibre Channel) methods, inband (fibre connection (FICON) or Fibre Channel) methods, and a management interface summary.
Product Management 2 Product Management Out-of-band (non-Fibre Channel) management server access to all McDATA products is provided through an Ethernet local area network (LAN) connection on a director control processor (CTP) card, fabric switch front panel, or SAN router front panel. As an optional feature, inband (Fibre Channel or FICON) management access to selected McDATA products is provided through a Fibre Channel port connection.
Product Management 2 • Management using simple network management protocol (SNMP). An SNMP agent is implemented through the Element Manager application that allows administrators on SNMP management workstations to access product management information using any standard network management tool.
Product Management 2 In contrast to the applications installed on the management server, EFCM Lite does not include support for the: — Call-home feature. — Ability to download remote clients from the server. Install clients on remote workstations from the software distribution disk provided with this management option. NOTE: The Sphereon 4300 Fabric Switch and SAN routers do not support product management through the EFCM Lite application. Figure 2-1 illustrates out-of-band product management.
Product Management 2 Inband Management The following inband management access methods are provided for directors and fabric switches as options: • Management through the product’s open-system management server (OSMS) that communicates with an application client. The application resides on an open-systems interconnection (OSI) device attached to a director or switch port, and communicates using Fibre Channel common transport (FC-CT) protocol.
Product Management 2 S/390 or zSeries 900 Parallel Enterprise Server Host-Attached Console FICON Channel OSI Server TM Fibre Channel Connection Intrepid 6064 Director Figure 2-2 Management Interface Summary Table 2-1 2-6 Inband Product Management Table 2-1 summarizes McDATA products and the out-of-band or inband management interfaces available to support the products.
Product Management 2 Table 2-1 Out-of-Band and Inband Product Support Summary (continued) Product SANavigator EFCM SANvergence Manager SNMP CLI EFCM Basic Edition EFCM Lite OSMS FMS 4300 Fabric Switch NO NO NO YES YES YES NO YES NO 4500 Fabric Switch YES YES NO YES YES YES YES YES NO 1620 SAN Router NO NO YES YES YES NO NO NO NO 2640 SAN Router NO NO YES YES YES NO NO NO NO Management Server Support The management server is a one rack unit (1U) high, LAN-
Product Management 2 The server is dedicated to operation of the SAN management and associated Element Manager applications. These applications provide a GUI and implement web and other server functions. Refer to SAN Management Applications for additional information. NOTE: The server and SAN management application provide a GUI to monitor and manage products and are a dedicated hardware and software solution that should not be used for other tasks.
Product Management 2 Recommended Specifications • 24X read speed slim-type compact disk-rewritable (CD-RW) and 8X read speed digital video disk (DVD) combination drive, data only. • 56K peripheral component interconnect (PCI) internal data and fax modem, using the V .92 dial-up specification. • 16 MB graphics card. • Network interface card (NIC) with two 10/100 Mbps Ethernet adapters using RJ-45 connectors.
Product Management 2 Ethernet Hub The management server and managed directors, fabric switches, and SAN routers connect through a 10/100 Base-T Ethernet hub. Figure 2-4 illustrates the 24-port hub. .
Product Management 2 • Unix workstation with color monitor, keyboard, and mouse, using a: — Linux-based system using an Intel Pentium III processor with one GHz or greater clock speed, using the Red Hat® 7.3 or higher operating system. — Hewlett-Packard® PA-RISC® processor with 400 MHz or greater clock speed, using the HP-UX® 11 or higher operating system. — Sun® Microsystems UltraSPARC™ IIi or later processor, using Solaris™ Version 7.0 or higher operating system.
Product Management 2 • Firmware Services 2-12 E/OSi - The Enterprise Operating System (internetworking) performs system configuration, management, and Fibre Channel and IP-based routing functions for Eclipse-series SAN routers. Director and fabric switch firmware (E/OSc and E/OSn) provides services that manage and maintain Fibre Channel connections between ports.
Product Management 2 • Fabric services - This function supports the fabric controller (login server) and name server. For redundant directors, fabric services also implement a replication manager that synchronizes node port (N_Port) registration databases between redundant CTP cards and allows CTP failover.
Product Management 2 — Operating parameters, such as buffer-to-buffer credit (BB_credit), error detect timeout value (E_D_TOV), resource allocation timeout value (R_A_TOV), switch priority, switch speed (1.0625 or 2.1250 Gbps), and preferred Domain_ID. — Active zoning configuration. — SNMP configuration parameters, such as trap recipients, community names, and write authorizations.
Product Management 2 SAN Management Applications This section describes SAN management applications that provide a GUI to monitor, manage, and control directors, fabric switches, and SAN routers. SAN management applications include SANavigator , EFCM, and SANvergence Manager. An associated Element Manager application is provided for each managed product. NOTE: The Element Manager application for a director or fabric switch resides on the associated management server.
Product Management 2 • Centralized configuration - Vendor-specific device management applications can be launched from the SAN management application, including McDATA Element Manager applications. The application also provides management of director and switch zoning across multiple vendors and product models. • Monitoring and notification - The application provides realtime monitoring and event notification for devices in the SAN. Informational, warning, and fatal events are recorded.
Product Management 2 Figure 2-5 Hardware View A status table appears at the top of the window, and a graphical representation of the hardware (front and rear) appears in the center of the window. The graphical representation of the product emulates the hardware configuration and operational status of the corresponding real product. For example, if a director or switch is fully redundant and fully populated, this configuration is reflected in the Hardware View.
Product Management 2 A status bar at the bottom left corner of the view window displays colored icons (green circle, yellow triangle, red and yellow diamond, or grey square) that indicate the status of the selected managed product. Messages display as required to the right of the icons.
Product Management 2 For additional information about the application, refer to the McDATA SANvergence Manager User Manual (620-000189). Element Manager Application An Element Manager application is provided for each managed SAN router. The application works in conjunction with the SANvergence Manager application and is a router-resident, Java-based applet for managing and monitoring the product.
Product Management 2 A menu bar at the top of the device window provides File, Configuration, Statistics/Info, Window, Options, and Help selections (with associated pop-up menus) that allow users to perform Element Manager tasks. EFCM Basic Edition Interface With E/OSc firmware installed, administrators or operators with a browser-capable PC and Internet connection can monitor and manage a product through the EFCM Basic Edition interface.
Product Management 2 The interface provides a GUI similar to the Element Manager application and supports product configuration, statistics monitoring, and basic operation. The EFCM Basic Edition interface manages only a single product (but has hyperlink access to other switches in a fabric). For additional information about the application, refer to the McDATA EFCM Basic Edition User Manual (620-000240).
Product Management 2 2-22 McDATA Products in a SAN Environment - Planning Manual
3 Planning Considerations for Fibre Channel Topologies A storage area network (SAN) is typically defined as a network of shared storage resources that can be allocated throughout a heterogeneous environment. This chapter describes planning considerations for incorporating McDATA products into Fibre Channel SAN topologies. The chapter specifically describes: • Fibre Channel topologies (arbitrated loop and multiswitch fabric). • Characteristics of arbitrated loop operation.
Planning Considerations for Fibre Channel Topologies 3 • Arbitrated loop - This topology uses a Sphereon 4000-series fabric switch to connect multiple device node loop ports (NL_Ports) in a Fibre Channel arbitrated loop (FC-AL) or hub configuration without benefit of a multiswitch fabric. Both switches support a switched mode topology that provides a single, logical connection between two device NL_Ports.
Planning Considerations for Fibre Channel Topologies 3 This section focuses on loop operation for Sphereon 4000-series fabric switches that operate at 1.0625, 2.1250, or 4.2500 gigabits per second (Gbps) and support FC-AL operation using FL_Ports and public and private device connectivity. Shared Mode Versus Switched Mode Legacy arbitrated loop switches (such as the McDATA ES-1000 Switch) are configured to operate in user-selectable shared or switched mode.
Planning Considerations for Fibre Channel Topologies 3 Part (A) of Figure 3-1 shows device D1 connected to server S1 through a pair of H_Ports. Although the remaining switch H_Ports (six ports) and device D2 are unavailable for connection, frame traffic between device D1 and server S1 travels through a loop that consists of all eight H_Ports, device D1, device D2, and server S1.
Planning Considerations for Fibre Channel Topologies 3 Part (A) of Figure 3-2 shows server S1 connected to device D1 through a switched pair of FL_Ports communicating at 1.0625 Gbps. Server S2 is connected to device D2 through a second switched pair of ports, also communicating at 1.0625 Gbps. Because of opportunistic bandwidth sharing, the two switched pairs effectively increase the switch bandwidth to 2.1250 Gbps.
Planning Considerations for Fibre Channel Topologies 3 Public Versus Private Devices Sphereon 4000-series fabric switches support connection of public and private arbitrated loop devices as follows: • Public device - A loop device that can transmit a fabric login (FLOGI) command to the switch, receive acknowledgement from the switch’s login server, register with the switch’s name server, and communicate with fabric-attached devices is a public device.
Planning Considerations for Fibre Channel Topologies 3 • Private device - A loop device that cannot transmit an FLOGI command to the switch nor communicate with fabric-attached devices is a private device. As shown in Figure 3-4, device D2 is a private loop device connected to a Sphereon 4500 Fabric Switch and cannot communicate with any fabric-attached device. However, device D2 can communicate with switch-attached server S2 (using private addressing mode).
Planning Considerations for Fibre Channel Topologies 3 Private devices only communicate with other devices on the same arbitrated loop, and interconnected public and private devices can communicate with each other. Such intermixed devices establish operating parameters and loop topology configuration through a port login (PLOGI) command exchange, rather than through the switch’s name server. Be aware that public device-to-private device communication may cause problems.
Planning Considerations for Fibre Channel Topologies 3 Figure 3-5 Public Loop Connectivity • Figure 3-6 Private loop - A private loop is not connected to a switched fabric and the switch’s embedded FL_Port is inactive. All devices attached to the loop can only communicate with each other. Private loop connectivity for a Sphereon 4500 Fabric Switch is illustrated in Figure 3-6.
Planning Considerations for Fibre Channel Topologies 3 FL_Port Connectivity Sphereon 4000-series fabric switches provide loop connectivity through GX ports that are active as FL_Ports. The ports provide port addressing, physical connectivity, and Fibre Channel frame routing. Each FL_Port (and the embedded FL_Port) has a 24-bit address identifier. The address identifier is expressed in hexadecimal format as DD AA PP, where: • DD is the domain identifier.
Planning Considerations for Fibre Channel Topologies 3 Planning for Fabric-Attached Loop Connectivity Public arbitrated loop topology supports the connection of workgroup or departmental FC-AL devices to a switched fabric through any Sphereon 4000-series fabric switch port active as an E_Port.
Planning Considerations for Fibre Channel Topologies 3 • Improved resource manageability. Distributed resources are consolidated and managed through Fibre Channel connectivity instead of physical relocation. One management server manages the operation and connectivity of multiple fabric directors, fabricattached devices, arbitrated loop switches, and FC-AL devices. • Improved security of business applications and data.
Planning Considerations for Fibre Channel Topologies 3 Figure 3-7 Tape Device Consolidation Server Consolidation Providing fabric connectivity for multiple FC-AL tape drives by attaching them individually to a Fibre Channel director is likewise not a cost-effective solution. A practical solution is to consolidate the tape drives on an inexpensive loop switch, then connect the switch to a director E_Port.
Planning Considerations for Fibre Channel Topologies 3 Figure 3-8 Tape Drive Consolidation Fabric Topologies Several topologies exist from which to build a Fibre Channel fabric infrastructure. This section describes the most effective fabric topologies and provides guidance on when to deploy each topology. The topologies are effective for a wide variety of applications, are extensively tested by McDATA, and are deployed in several customer environments.
Planning Considerations for Fibre Channel Topologies 3 TM TM TM TM Interswitch Link Fabric Connection Figure 3-9 Full Mesh Fabric Full-mesh fabrics provide increased resiliency over cascaded or ring fabrics and are well suited for applications that require any-to-any connectivity. If a single ISL fails, traffic is automatically routed through an alternate path. Mesh fabrics also form effective backbones to which other SAN islands can be connected.
Planning Considerations for Fibre Channel Topologies 3 A modified or partial-mesh fabric is similar to a full-mesh fabric, but each switch does not have to be directly connected to every other switch in the fabric. The fabric is still resilient to failure but does not carry a cost premium for unused or redundant ISLs. In addition, partial-mesh fabrics scale easier than full-mesh fabrics.
Planning Considerations for Fibre Channel Topologies 3 Tier 2 Devices TM TM TM Edge Switches 10/100 RST 10/100 RST 31 29 27 25 10/100 23 TM 30 PWR ERR Tier 1 Device Core Director 28 26 24 22 31 29 27 25 23 TM 21 RST 20 TM 21 10/100 19 18 17 16 RST 15 14 13 12 30 TM 11 10 9 8 7 5 3 28 26 24 1 PWR ERR 6 4 2 22 20 19 18 17 16 15 14 13 12 11 10 9 8 7 5 3 1 PWR ERR 6 4 2 0 PWR 0 ERR Tier 1 Device Core Director TM TM TM TM T
Planning Considerations for Fibre Channel Topologies 3 SAN Islands • Tier 2 - A Tier 2 device connects to an edge switch and Fibre Channel traffic from the device must traverse only one ISL (hop) to reach a device attached to a core director or switch. • Tier 3 - A Tier 3 device connects to an edge switch and Fibre Channel traffic from the device can traverse two ISLs (hops) to reach a device attached to a core director or switch.
Planning Considerations for Fibre Channel Topologies 3 D3 D4 D5 D6 D7 N N N N N F F F F F F N D8 F N D9 F N D10 ISLS D2 N N D1 F E E E E E E E E TM TM F Di2 S1 E E S2 ISL ISL TM E E F F N D12 Figure 3-11 S3 Multiswitch Fabric N D11 D = Device ISL = Interswitch Link S = Fabric Switch (Director) Example Multiswitch Fabric A multiswitch fabric is typically complex and provides the facilities to maintain routing to all device N_Ports attached to the fab
Planning Considerations for Fibre Channel Topologies 3 • Heterogeneous fabric - Vendor interoperability in the fabric environment is supported; therefore, fabric elements can include directors, fabric switches, and open-fabric compliant products supplied by original equipment manufacturers (OEMs). To determine if interoperability is supported for a product or if communication restrictions apply, refer to the supporting publications for the product or contact McDATA.
Planning Considerations for Fibre Channel Topologies 3 • Distance requirements - The distance between elements in a fabric affects the type of optical port transceiver and cabling required. In addition, variables such as the number of connections, grade of fiber-optic cable, device restrictions, application restrictions, buffer-to-buffer credit limits, and performance requirements can affect distance requirements.
Planning Considerations for Fibre Channel Topologies 3 • Bandwidth - ISL connections can be used to increase the total bandwidth available for data transfer between two directors or switches in a fabric. Increasing the number of ISLs between elements increases the corresponding total ISL bandwidth but decreases the number of port connections available to devices. • Load balancing - Planning consideration must be given to the amount of data traffic expected through the fabric or through a fabric element.
Planning Considerations for Fibre Channel Topologies 3 When balancing a load across multiple ISLs, a director or switch attempts to avoid assigning multiple ports attached to a device to the same ISL. This minimizes the probability that failure of a single ISL will affect all paths to the device. However, because port assignments are made incrementally as devices log into the fabric and ISLs become available, optimal results are not guaranteed.
Planning Considerations for Fibre Channel Topologies 3 — If you have three fabric elements and set two to Principal and one to Default, the element with the Principal setting that has the lowest WWN becomes the principal switch. — If you have three fabric elements and set two to Default and one to Never Principal, the element with the Default setting and the lowest WWN becomes the principal switch. Note that at least one director or switch in a multiswitch fabric needs to be set as Principal or Default.
Planning Considerations for Fibre Channel Topologies 3 If two operational fabrics join, they determine if any Domain_ID conflicts exist between the fabrics. If one or more conflicts exist, the interconnecting ISL E_Ports segment to prevent the fabrics from joining. To prevent this problem, it is recommended that all directors and switches be assigned a unique preferred Domain_ID. This is important if zoning is implemented through port number (and by default Domain_ID) rather that WWN.
Planning Considerations for Fibre Channel Topologies 3 Channel frames between devices attached to the fabric and enable operation of the fabric services firmware on each director or switch. Paths are determined when the fabric topology is determined and remain static as long as the fabric does not change. If the fabric topology changes (elements are added or removed or ISLs are added or removed), directors and switches detect the change and define new data transfer paths as required.
Planning Considerations for Fibre Channel Topologies 3 ATTENTION ! Activating a preferred path can result in receipt of out-oforder frames if the preferred path differs from the current path, if input and output (I/O) is active from the source port, and if congestion is present on the current path. A rerouting delay parameter can be enabled at the Element Manager application to ensure a director or switch provides correct frame order delivery.
Planning Considerations for Fibre Channel Topologies 3 — No response from attached switch - After a fabric is created, each element in the fabric periodically verifies operation of all attached switches and directors. An ISL segments if a switch or director does not respond to a verification request. — ELP retransmission failure timeout - A director or switch that exhibits a hardware failure or connectivity problem cannot transmit or receive Class F frames.
Planning Considerations for Fibre Channel Topologies 3 — Fabric A zoned and Fabric B zoned - The fabrics join successfully only if the zone sets can be merged. If the fabrics cannot join, the connecting E_Ports segment and the fabrics remain independent. Zone sets for two directors or switches are compatible (the fabrics can join) only if the zone names for each fabric element are unique.
Planning Considerations for Fibre Channel Topologies 3 If the fabric consists of a large number of elements (and therefore ISLs), Class F traffic may not be processed within the E_D_TOV, resulting in error recovery operations, timeouts, segmented links, or fabric failure. Because of these problems, a fabric with a high ISL count is more difficult to build.
Planning Considerations for Fibre Channel Topologies 3 • I/O Requirements Software limits, including the maximum number of fabric elements managed by the SAN management application and the maximum number of zones and zone members. For additional information, refer to SAN Management Applications and Configuring Zones. McDATA directors and fabric switches are designed with nonblocking architecture; therefore any two switch ports can communicate at the full Fibre Channel bandwidth of 1.0625, 2.1250, 4.
Planning Considerations for Fibre Channel Topologies 3 • I/O block size - The third characteristic of application I/O is data block size, which typically ranges from two kilobytes (KB) to over one megabyte (MB). Applications that generate large blocks of data require high bandwidth to the device. Prior to fabric design, application I/O profiles should be estimated or established that classify the application bandwidth requirements. Bandwidth consumption is classified as light, medium, or heavy.
Planning Considerations for Fibre Channel Topologies 3 Two NT servers, each with maximum I/O of 100 MBps, are contending for the bandwidth of a single ISL operating at 1.0625 Gbps. In addition to data, the ISL must also transmit Class F traffic internal to the fabric. When operating at peak load, each NT server receives less than half the available ISL bandwidth.
Planning Considerations for Fibre Channel Topologies 3 High Device Locality A Low Device Locality High Traffic Low Traffic TM TM TM ISL Figure 3-13 B TM ISL Device Locality Although it is possible to design a SAN that delivers sufficient ISL bandwidth in a zero-locality environment, it is preferable to design local, one-to-one connectivity for heavy-bandwidth applications such as video server, seismic data processing, or medical 3D imaging.
Planning Considerations for Fibre Channel Topologies 3 Device Fan-Out Ratio: 10 to 1 1,000 IOPS 1,000 IOPS 1,000 IOPS 1,000 IOPS 10,000 IOPS TM TM 1,000 IOPS 1,000 IOPS 1,000 IOPS 1,000 IOPS Interswitch Link Fabric Connection Figure 3-14 Performance Tuning 1,000 IOPS 1,000 IOPS Device Fan-Out Ratio When designing or tuning a fabric for performance, it is critical to understand application I/O characteristics so that: • Device output in Gbps does not oversubscribe ISLs, leading to fabric c
Planning Considerations for Fibre Channel Topologies 3 Tier 2 Storage 11 to 1 Fan-Out Region 6 to 1 Fan-Out Region 3 to 1 Fan-Out Region 11,000 IOPS 9,000 IOPS 6,000 IOPS 10/100 RST 10/100 RST 10/100 RST TM TM TM PWR ERR PWR PWR ERR ERR Local Tier 1 Devices 2,000 IOPS TM 6,000 IOPS 40 MBps 2,000 IOPS 10/100 RST 31 30 29 28 27 26 10/100 25 24 23 22 RST TM 21 20 19 18 10/100 TM 17 16 15 14 RST 13 12 11 10 TM 9 8 7 5 3 1 PWR ERR 6 4 PWR ERR 2 0
Planning Considerations for Fibre Channel Topologies 3 Fabric Availability • 11 to 1 fan-out region - Eleven NT servers with I/O capabilities of 10 MBps and 1,000 IOPS are fabric-attached through a 32-port edge switch. The primary applications are e-mail and online transaction processing (OLTP). Because bandwidth use is light and noncritical, the servers are connected to the core director with a single ISL that is intentionally oversubscribed (1.1 Gbps plus Class F traffic).
Planning Considerations for Fibre Channel Topologies 3 • Resilient single fabric - Directors and switches are connected to form a single fabric, but no single point of failure can cause the fabric to fail and segment into two or more smaller fabrics. • Nonresilient dual fabric - Half of the directors and switches are connected to form one fabric, and the remaining half are connected to form an identical but separate fabric. Servers and storage devices are connected to both fabrics.
Planning Considerations for Fibre Channel Topologies 3 10/100 10/100 RST 10/100 RST 10/100 RST TM RST TM TM TM PWR PWR PWR PWR ERR ERR ERR ERR Fabric “A” Fabric “B” TM TM 10/100 10/100 RST 10/100 RST RST 10/100 RST TM TM TM TM PWR ERR PWR ERR PWR PWR ERR ERR Interswitch Link Fabric Connection Figure 3-16 Redundant Fabrics When deploying redundant fabrics, it is not required that the fabrics be symmetrical.
Planning Considerations for Fibre Channel Topologies 3 • High-port count directors - Installing high-port count directors as SAN building blocks provides a larger number of non-blocking Fibre Channel ports per fabric element and reduces the need for ISLs. Newer products support high-bandwidth (10.2000 Gbps) ISLs that also reduce the fabric ISL count. In addition, fabrics and sub-fabrics can be merged or maintained as separate entities through dynamic partitioning.
Planning Considerations for Fibre Channel Topologies 3 FCP and FICON in a Single Fabric Fibre Channel Layer 4 (FC-4) describes the interface between Fibre Channel and various upper-level protocols. FCP and FICON are the major FC-4 protocols. FCP is the Fibre Channel protocol that supports the small computer system interface (SCSI) upper-level transport protocol.
Planning Considerations for Fibre Channel Topologies 3 • When a director or fabric switch is set to open systems management style, FCP connectivity is defined within a Fibre Channel fabric using WWNs of devices that are allowed to form connections. When connecting to the fabric, an FCP device queries the name server for a list of devices to which connectivity is allowed.
Planning Considerations for Fibre Channel Topologies 3 • The Domain_ID and physical port number of the director or fabric switch port to which a device is attached. FICON configuration attributes are implemented through logical port addressing. This concept is consistent with the address-centric nature of FICON and allows ports to be swapped for maintenance operations without regenerating a host configuration.
Planning Considerations for Fibre Channel Topologies 3 Figure 3-18 illustrates port numbering and logical port addressing for Intrepid 6140 Director ports accessed from the rear. UPM Cards UPM Cards 34 33 143 8F 142 8E 141 8D 140 8C 136 88 137 89 138 8A 139 8B 93 92 91 90 8C 8D 8E 8F 135 87 134 86 133 85 132 84 8B 8A 89 88 32 SBAR - 1 Module SBAR - 0 Module Figure 3-18 Intrepid 6140 Port Numbers and Logical Port Addresses (Rear) The figure shows: • SBAR positions (0 and 1).
Planning Considerations for Fibre Channel Topologies 3 • FICON port-to-port connectivity is hardware enforced, while FCP port-to-port connectivity is software or hardware enforced (depending on the director or switch firmware release level). — FICON architecture controls connectivity through a host-based HCD program, the CUP, and a director or switch-resident PDCM array.
Planning Considerations for Fibre Channel Topologies 3 • When employing inband (Fibre Channel) director or switch management, the open-systems management server (OSMS) is associated with the FCP protocol, and the FICON management server (FMS) is associated with the FICON protocol. Management server differences tend to complicate security and control issues. NOTE: The Intrepid 10000 Director and Sphereon 4300, 4400, and 4500 Fabric Switches do not support out-of-band management through FMS.
Planning Considerations for Fibre Channel Topologies 3 SANtegrity Binding McDATA offers a SANtegrity Binding feature (including both fabric binding and switch binding) that allows the creation of reliable SAN configurations and provides a mechanism for attached devices to query the user-configured security level employed in a SAN. The feature significantly reduces the impacts of accidental or operatorinduced errors.
Planning Considerations for Fibre Channel Topologies 3 However, the firmware and SAN management applications do not prevent FCP and FICON device configurations that may interfere with each other. A successful intermix environment requires a set of best practice conventions as follows: 1. Upgrade fabric element firmware to a common version - Ensure fabric elements are operating at a common firmware level. This reduces errors due to director or switch incompatibility. E/OSc Version 4.
Planning Considerations for Fibre Channel Topologies 3 — When using inband director or switch management, either (or both) of the FMS or OSMS features can be enabled. When either (or both) features are enabled, the director or switch can be set to open systems or FICON management style. 3. Upgrade fabric elements to a common feature set - Ensure a common set of PFE-keyed optional features (refer to Optional Feature Keys) is installed on each fabric element.
Planning Considerations for Fibre Channel Topologies 3 6. Configure PDCM arrays - For each director or switch managed by the FICON management style, define the allow and prohibit settings for FICON device connectivity. Use the Element Manager application’s Configure Allow/Prohibit Matrix - Active dialog box. Port connectivity assignment (step 4) should be reflected in PDCM arrays for FICON connectivity management.
Planning Considerations for Fibre Channel Topologies 3 — All FICON devices must be included in the same zone to facilitate proper state change notification. This is achieved by creating a unique FICON zone or using the default zone. Disable the default zone and explicitly create a unique zone for all FICON devices. Regardless of the director or switch operating mode, FCP devices must be zoned in the traditional fashion, and FICON devices must be zoned to provide isolation from the FCP devices.
Planning Considerations for Fibre Channel Topologies 3 • Better fabric performance - As a connection between fabric switches, a 10.2000 Gbps ISL delivers significantly greater bandwidth. Fibre Channel devices that are not 10.2000 Gbpscapable benefit from a higher-speed ISL, because slower traffic is multiplexed and transmitted through the 10.2000 Gbps ISL. • Additional port count - If additional ISL bandwidth is not required for fabric performance, 10.
Planning Considerations for Fibre Channel Topologies 3 High-Integrity Fabrics Cascaded FICON directors and switches must support high-integrity fabrics. McDATA fabric elements must have the SANtegrity Binding feature installed and operational with Enterprise Fabric Mode enabled. High-integrity fabric architecture support includes: • Fabric binding - Only directors or switches with fabric binding installed are allowed to attach to specified fabrics in a SAN.
Planning Considerations for Fibre Channel Topologies 3 — Intrepid 6064 or 6140 Director. — Sphereon 3232 or 4500 Fabric Switch. • E/OSc Version 4.0 (or later) must be installed on all directors or switches. E/OS firmware Version 6.0c (or later) is recommended. All fabric elements must be at the same firmware version level. • The SANtegrity Binding feature key must be installed and enabled on all directors and switches. Enterprise Fabric Mode must also be enabled on all fabric elements.
Planning Considerations for Fibre Channel Topologies 3 c. Ensure the R_A_TOV and E_D_TOV values for fabric elements are identical. d. Route multimode or singlemode fiber-optic cables (depending on the type of transceiver installed) between customer-specified E_Ports at each fabric element. 2. Verify operation of local FICON applications - Ensure the ISL connection(s) do not disrupt fabric element operation nor disrupt local FICON (non-cascaded) traffic. Perform this step at each director or switch. a.
Planning Considerations for Fibre Channel Topologies 3 b. At the SAN management application, configure fabric binding. Refer to installation instructions in the SANavigator Software Release 4.2 User Manual (621-000013) or the EFC Manager Software Release 8.7 User Manual (620-000170). c. At the Element Manager application, configure switch binding. Refer to installation instructions in the product-specific Installation and Service Manual. 5.
Planning Considerations for Fibre Channel Topologies 3 b. Ensure the Enable Switch Binding check box is checked (enabled). c. Select (click) the Restrict E_Ports radio button to restrict connections from specific fabric elements to E_Ports. WWNs can be added to the membership list to allow fabric element connection and removed from the list to prohibit fabric element connection. Devices are allowed to connect to any F_Port or FL_Port without restriction. d.
Planning Considerations for Fibre Channel Topologies 3 11. Verify cascaded FICON operation - Have the customer verify operation of established logical FICON paths between channels and control units, and verify that cascaded FICON traffic is transmitted through the fabric as expected.
4 Implementing SAN Internetworking Solutions Enterprise-level information technology (IT) departments often deploy storage configurations that include direct-attached storage, network-attached storage (NAS), and small, isolated storage area networks (SANs). These configurations often result in: • Isolated and inefficiently-used applications, data storage, and computing resources. • Costly, decentralized, and complex asset management. • Slow transactions and data access.
Implementing SAN Internetworking Solutions 4 SAN Island Consolidation SAN islands tend to be constructed along application (such as product test, finance, or engineering), operating system (OS), protocol, or geographical (site-based) boundaries. Because of application and OS segmentation, large data centers at single sites often consist of SAN islands constructed with relatively small Fibre Channel switches.
Implementing SAN Internetworking Solutions 4 Large Fabric Problems • Inability to consistently schedule maintenance downtime for each SAN. • Stranded resources. Unused ports in one SAN cannot be used by applications in another (port limited) SAN, and expensive resources (such as tape backup elements) cannot be easily shared across SAN boundaries. Fibre Channel fabrics configure and manage themselves, and require operator intervention only upon failure.
Implementing SAN Internetworking Solutions 4 Another persistent problem associated with large Fibre Channel fabrics is multi-vendor incompatibility. Due to lack of common communications standards and fabric shortest path first (FSPF) protocol, switch vendors may have to support multiple (standardscompliant and proprietary) interoperability modes. In addition, protocol enhancements may force vendors to support multiple firmware versions for the same product.
Implementing SAN Internetworking Solutions 4 This feature reduces unused ports and resources and consolidates the enterprise into a single infrastructure, while maintaining multiple independent application and fault isolation domains. Up to four partitions can be enabled for each director (0 though 3), where a partition consists of one or more line modules (LIMs). User access and Fibre Channel traffic (Class 2, Class 3, and Class F) are isolated within each partition.
Implementing SAN Internetworking Solutions 4 • Create up to three additional FlexPars and assign resources to those FlexPars. • Perform director firmware upgrades to all Flexpars. • Enable or disable the protocol subsystem for any Flexpar. • Enable or disable switch modules and control processor (CTP) cards. • Perform director shutdowns, restarts, and field-replaceable unit (FRU) switchovers. • Set the director Internet protocol (IP) address, gateway address, and subnet mask.
Implementing SAN Internetworking Solutions 4 Zone FlexPars implement an RSCN zone isolation feature that prevents fabric-format RSCNs from propagating to devices in zones not impacted by the RSCN. With zone FlexPars enabled, zoning change RSCNs are handled like device availability change RSCNs. Because the feature is device centric, zone FlexPars work in loop environments and with node port ID virtualization (NPIV) enabled.
Implementing SAN Internetworking Solutions 4 SAN Routing Connecting isolated, department-level, and application-specific Fibre Channel SANs is a requirement for most enterprises. Consolidating SAN islands: • Provides campus storage connectivity and interoperability between formerly-incompatible Fibre Channel fabrics (from the same or different vendors).
Implementing SAN Internetworking Solutions 4 Figure 4-2 • Tier 2 - To connect SAN islands without physically merging the fabrics, the second tier consists of metropolitan storage area networks (mSANs). SAN routers connect fabrics within a data center or campus to form an mSAN and transmit data between fabrics through router ports (R_Ports). Refer to mSAN Routing for additional information.
Implementing SAN Internetworking Solutions 4 Figure 4-3 SAN Routing Concepts The following sections discuss SAN routing concepts, including: 4-10 • R_Port operation. • Routed SAN zoning. • mSAN routing. • iFCP operation. • iSAN routing. • Inter-FlexPar routing. • Best practices.
Implementing SAN Internetworking Solutions 4 R_Port Operation To avoid building a large Fibre Channel fabric with its inherent reconfiguration issues, SAN Routing provides any-to-any connectivity (to maximize use of common assets across SAN islands), while retaining the fault isolation characteristics of smaller SANs.
Implementing SAN Internetworking Solutions 4 Instead of Class F frame transmission, routing communication is provided by Fibre Channel network address translation (FC_NAT) technology. This is similar to the technology used by IP networks to convert private addresses to public addresses. The principal switch in each router-connected fabric assigns the Domain_ID to the associated R_Port acting as an edge switch.
Implementing SAN Internetworking Solutions 4 NOTE: Proxy Domain_IDs 30 and 31 are reserved for routing domains and cannot be assigned to directors or switches in any router-attached fabric. The routing domain with proxy Domain_ID 30 represents Fibre Channel devices that are part of a router-attached fabric (part of a local mSAN). The routing domain with proxy Domain_ID 31 represents devices that are directly attached to the router’s Fibre Channel ports or connected through an iFCP link.
Implementing SAN Internetworking Solutions 4 For attached fabrics in which participating element’s Interop Mode is set to McDATA Fabric 1.0, Domain_IDs of 30 and 31 are recognized by SAN management applications and all attached devices. For attached fabrics in which participating element’s Interop Mode is set to Open Fabric 1.0, Domain_IDs of 30 and 31 are recognized by SAN management applications. Domain_IDs of 7E and 7F are recognized by all attached devices.
Implementing SAN Internetworking Solutions 4 R_Port Domain_ID Assignment The default preferred Domain_ID for each SAN router R_Port is 1. However, each port should be assigned a preferred Domain_ID (set at the R_Ports tab of the Fabric Configuration dialog box) that is unique within the attached fabric. The principal switch in the attached fabric then attempts to allocate this requested (preferred) Domain_ID to the R_Port.
Implementing SAN Internetworking Solutions 4 There is only one router fabric manager per fabric. If more than one R_Port (from the same or multiple routers) connects to a fabric, then the port with the lowest worldwide name (WWN) is elected router fabric manager for that fabric. Other R_Ports become subordinate ports. All R_Ports (router fabric manager or not) participate in FSPF routing protocol and traffic forwarding.
Implementing SAN Internetworking Solutions 4 A No Zone Synchronization policy is typically not suitable for larger SAN routing environments where many devices must be visible to numerous fabrics. Append IPS Zones When the zone policy is set to Append IPS Zones, Internet protocol storage (IPS) zone set information from the router is appended to the active zone set for every router-attached fabric in the mSAN.
Implementing SAN Internetworking Solutions 4 mSAN Routing An mSAN consists of one or two SAN routers that interconnect up to six Fibre Channel fabrics. These fabrics are typically dispersed within a data center or metropolitan campus. If two SAN routers are used, they are connected with multiple (one to four) Gigabit Ethernet (GbE) bandwidth IRLs. These GbE connections (using mFCP as the transport protocol) are characterized by low latency, high bandwidth, and negligible packet loss.
Implementing SAN Internetworking Solutions 4 During SAN router configuration, each R_Port is assigned (through the SANvergence manager application) a unique Fabric_ID between 1 and 12. Although the theoretical limit is 12 Fabric_IDs per mSAN, the supported limit is six. As shown in Table 4-1, four Area_IDs are available to each Fabric_ID. Therefore, the combination of domain, area, and fabric IDs creates a theoretical limit of 1,024 devices per fabric (although the supported number is far less).
Implementing SAN Internetworking Solutions 4 Router Connectivity through mFCP mFCP provides connectivity (through a GbE-bandwidth IRL) between two Eclipse 2640 SAN Routers. mFCP is similar to Fibre Channel protocol (FCP) but implements user datagram protocol (UDP) for open systems interconnection (OSI) Layer 4 transport. mFCP links are used for path failover in high-availability mSANs. NOTE: The Eclipse 1620 SAN Router does not support mFCP and must be deployed in mSANs as a single-router configuration.
Implementing SAN Internetworking Solutions 4 If a direct Fibre Channel connection exists between routed fabrics, storage traffic traverses the Fibre Channel ISL and not the routerto-router mFCP link. Only SNS traffic traverses the mFCP link. However, if a router-to-router mFCP link is the only path between two Fibre Channel devices, the link is traversed by storage traffic.
Implementing SAN Internetworking Solutions 4 iFCP Operation There are three protocols competing to transmit storage-related I/O traffic over long-distance transmission control protocol/Internet protocol (TCP/IP) links: • iSCSI is a TCP/IP-based protocol for establishing and managing connections between IP-based storage devices, hosts, and clients. iSCSI operates on top of TCP, moving block data (iSCSI packets) over an IP Ethernet network. Refer to iSCSI Protocol for additional information.
Implementing SAN Internetworking Solutions 4 From the standpoint of fabric build events, the only difference between a local and stretched E_Port connection is the latency introduced by the TCP/IP link and associated WDM or FCIP hardware. A disruptive build fabric event at one local site propagates to the connected site. Likewise, a disruption in the TCP/IP link may cause the extended SAN to segment into separate SAN islands.
Implementing SAN Internetworking Solutions 4 Figure 4-6 iSAN Routing iFCP WAN Extension An internetworked SAN (iSAN) is a network composed of multiple Fibre Channel fabrics or mSANs, connected by one or more SAN routers, where at least one fabric is remotely located and connected through a WAN. The WAN connection is an iFCP IRL, characterized by high latency and ranging in bandwidth from Digital Service 1 (DS1) at 1.544 Mbps to GbE at 1000 Mbps.
Implementing SAN Internetworking Solutions 4 SAN Routing also streamlines SAN connectivity by eliminating network address issues associated with duplicate Domain_IDs. Because fabric elements and devices at either end of an iFCP connection remain in separate mSANs, address conflicts between the mSANs do not occur. SAN Routing provides address translation (through FC_NAT) for zoned devices with authorization to communicate across the network.
Implementing SAN Internetworking Solutions 4 • Rate limiting - If ingress traffic enters the SAN router faster than egress traffic leaves, port buffers fill and cause dropped data packets. Dropped packets cause TCP to resort to internal (and inefficient) flow control, causing dramatic link throughput decrease. Rate limiting prevents this problem. Refer to Intelligent Port Speed for detailed information about rate limiting.
Implementing SAN Internetworking Solutions 4 and results in a lower compression bandwidth. The algorithm has an average compression ratio increase of approximately 30% over LZO. The algorithm is recommended when up to 8 TCP sessions are used and the available bandwidth is between 10 Mbps (thin Ethernet) and 45 Mbps (DS3 transport level). — Deflate - This algorithm incorporates a history cache with Huffman encoding. In addition, a hash table (saved in the compressed data) is used to perform string searches.
Implementing SAN Internetworking Solutions 4 mFCP to iFCP Comparison Table 4-3 compares mFCP to iFCP and summarizes the features of each protocol.
Implementing SAN Internetworking Solutions 4 Figure 4-7 illustrates inter-FlexPar routing. Flexpar B (tape backup fabric) is isolated from Flexpar C (product development fabric) as normally desired. However, development personnel occasionally perform tape backups that require access to Flexpar B devices. An E_Port from each FlexPar is physically connected to a SAN router R_Port, and Flexpar C servers are zoned to communicate with Flexpar B tape devices.
Implementing SAN Internetworking Solutions 4 2. Domain_ID assignment - Manually assign unique Domain_IDs to all Fibre Channel directors, fabric switches, and SAN router R_Ports. Ensure the Insistent Domain_ID option is enabled at the SAN management or SANvergence Manager application. Do not assign Domain_ID 30 or 31 to any fabric elements. These proxy Domain_IDs are reserved for routing domains within the SAN router. 3.
Implementing SAN Internetworking Solutions 4 7. Use redundant mFCP connections - For high availability (not increased bandwidth), use multiple mFCP connections between SAN routers to ensure the mSAN does not partition and connectivity to routing domains 30 and 31 remains intact. 8.
Implementing SAN Internetworking Solutions 4 Local mSAN Name: Boston Local mSAN_ID: 20 Date: 1/12/05 Exported Zone_ID Range: 1 to 100 Local Zone_ID Range: 101 to 512 Local mSAN Zone Summary Exported (Y/N) mSAN ID Y 2 DB_Replication_2 Remote site for disaster recovery. Y 3 Tape_Library_3 Remote site for data center tape library access. N 101 Nightly_Backup Local initiator for nightly tape backups. N 501 Test_Fabric Test fabric for the local mSAN.
Implementing SAN Internetworking Solutions 4 Local mSAN Name: Chicago Local mSAN_ID: 30 Date: 1/12/05 Exported Zone_ID Range: 1 to 100 Local Zone_ID Range: 101 to 512 Local mSAN Zone Summary Exported (Y/N) mSAN ID Y 2 N N mSAN Name Description DB_Replication_2 Replication site for disaster recovery. 101 Web_Server Local web server and associated hardware. 501 Local_Test Local test fabric.
Implementing SAN Internetworking Solutions 4 12. Configure R_Ports - For all configured SAN router R_Ports in the same fabric: a. The R_Port interconnect modes (McDATA Fabric 1.0 or Open Fabric 1.0) must be identical. This parameter is set at the SANvergence Manager application. The corresponding E_Port interoperability mode must also be identical. This parameter is set at the director or fabric switch Element Manager application. b.
Implementing SAN Internetworking Solutions 4 c. Port zoning can be confusing in a multi-vendor environment because OEMs implement zoning in different ways. When zoning with vendor-specific SAN management applications, zone through port WWNs, not node (device) WWNs. d. Ensure devices are physically connected before importing their node (device) WWNs to a router.
Implementing SAN Internetworking Solutions 4 Implementing BC/DR Solutions The post-9/11 business environment requires corporations to protect critical data by implementing cost-effective business continuity and disaster recovery (BC/DR) solutions. These BC/DR solutions drive the requirement to extend local data center SANs to geographically distant locations.
Implementing SAN Internetworking Solutions 4 • Reliability - Local storage traffic requires high-reliability communication and is intolerant of data loss, out-of-order packet receipt, or data retransmission. WANs typically provide besteffort communication service and rely on upper-level protocols for end-to-end transport. Because of these differences, a protocol conversion approach is usually required to integrate Fibre Channel SAN traffic over a geographically-dispersed network.
Implementing SAN Internetworking Solutions 4 • Synchronous remote data replication (RDR/S) - This operational mode ensures a remote data copy (identical to the primary copy) is created at the time the primary data is created. An update operation does not complete until confirmed at both the primary and mirrored sites. An incomplete operation rolls back at both locations, ensuring the remote copy is a mirror image of the primary copy. RDR/S is synonymous with disk mirroring.
Implementing SAN Internetworking Solutions 4 SAN Extension Transport Technologies Dark Fiber There are several extension transport technologies available to connect geographically-dispersed SAN islands, all of which differ in performance, latency, and implementation cost. The primary technologies include: • Dark fiber (repeated or unrepeated). • Wavelength division multiplexing (WDM). • Synchronous optical network (SONET) and synchronous digital hierarchy (SDH). • Internet protocol (IP).
Implementing SAN Internetworking Solutions 4 • Figure 4-8 Creates one logical Fibre Channel fabric through a stretched E_Port connection. The connection is vulnerable to disruptions caused by events at each site or to disruptions caused by problems with the extended-distance dark fiber link. Dark Fiber Extended-Distance Connectivity Due to the high cost of burying cables, dark fiber has limited physical availability relative to other WAN link options.
Implementing SAN Internetworking Solutions 4 Light wavelengths used are typically around 1,550 nanometers (nm). Optical fiber performs well in this wavelength region, with very little attenuation. For CWDM, differing wavelengths are separated by multiples of 20.0 nm. For DWDM, differing wavelengths are separated by multiples of 0.8 nm. The lower wavelength numbers provided by CWDM are due to lower accuracy (and price) of lasers.
Implementing SAN Internetworking Solutions 4 • Requires sufficient BB_Credits assigned to the link (such as credits available through the Intrepid 10000 Director buffer pool). Because WDM is a method to transmit multiple signals over the same fiber-optic cable, there is no BB_Credit limitation difference between WDM and dark fiber. Refer to Distance Extension Through BB_Credit for information about requirements. • Creates one logical Fibre Channel fabric through a stretched E_Port connection.
Implementing SAN Internetworking Solutions 4 SONET and SDH are globally standardized technologies, more widely deployed than dark fiber or WDM, and provide a protected connection between two locations. SONET and SDH rings are also self-healing. This means a link is usually restored within 50 ms of break detection without user intervention. This makes SONET and SDH highly-available services.
Implementing SAN Internetworking Solutions 4 • Does not require Fibre Channel BB_Credits assigned to the link because buffering is built in to the GFP function to enable long-distance transmission of storage traffic. • Creates a routed iSAN or one logical Fibre Channel fabric through a stretched E_Port connection, depending upon the protocol and if one or more SAN routers are deployed in the link.
Implementing SAN Internetworking Solutions 4 SONET or SDH service can be purchased on a monthly basis in accordance with a negotiated SLA. However, the transport links may require sufficient BB _Credits to use the purchased bandwidth. Because of BB_Credit limitations, GFP equipment must provide buffering and flow control for native FCP or FICON storage data. Without GFP equipment, iFCP must be used to transmit the data.
Implementing SAN Internetworking Solutions 4 routed SAN connection ensures disruptions at one site are isolated and not allowed to propagate to other locations. This connection does not support native FCP or FICON operation. Figure 4-11 SoIP Extended-Distance Connectivity Several network service providers provide long-distance IP or GbE network transport services. Long-distance circuits are common.
Implementing SAN Internetworking Solutions 4 Figure 4-12 SAN Extension Technology Comparison • WDM - This technology supports high-bandwidth, low-latency applications with short RTO and RPO requirements. Applications include peer-to-peer computer clustering (grid computing) and real-time disk mirroring (RDR/S or RDR/A operational mode) over short to medium metropolitan distances. WDM scales to higher bandwidths at a lower relative cost than SONET or SDH.
Implementing SAN Internetworking Solutions 4 • IP - This technology supports low-bandwidth, high-latency applications with long RTO and RPO requirements. Applications include asynchronous disk backup or tape vaulting over metropolitan to extended (intercity) distances. SAN routers are included in the extended-distance link (iFCP only), so the technology isolates the connected SANS and prevents disruptions caused by fabric or link problems. Table 4-4 compares and contrasts the transport technologies.
Implementing SAN Internetworking Solutions 4 • Distance Extension Through BB_Credit SAN routing requirements - If a single logical Fibre Channel fabric (created through a stretched E_Port connection) is unacceptable because of the potential for disruptive fabric rebuilds, include one or more SAN routers in the extendeddistance link. Token-based buffer-to-buffer flow control governs transmission of data and link control frames in a Fibre Channel switched fabric.
Implementing SAN Internetworking Solutions 4 To support greater Fibre Channel transmission rates (long-link ports), the Intrepid 10000 Director provides a buffer pool that allocates user-defined BB_Credits to each port. This buffer pool is increased if the remote fabric PFE key is enabled (refer to Remote Fabric for information). Each director line module (LIM) contains two scalable packet processors, each supporting two optical paddles. A paddle pair provides 16 ports (1.0625 or 2.
Implementing SAN Internetworking Solutions 4 • Intelligent Port Speed Figure 4-13 10.2000 Gbps long link (mixed paddles) - A paddle pair with one 1.0625 or 2.1250 Gbps paddle and one 10.2000 Gbps paddle provides ten connections. 16 BB_Credits are assigned to the slowest eight short-link ports, and 96 BB_Credits are assigned to one 10.2000 Gbps short-link port (224 BB_Credits total). The remaining 1149 BB_Credits are assigned to the long-link port, supporting a repeated transmission distance of 190 km.
Implementing SAN Internetworking Solutions 4 Figure 4-14 WAN Link Performance (Rate Limiting Enabled) When configuring a SAN router for extended-distance operation over an IP WAN link, the peak available bandwidth must be determined or obtained from the network service provider, and storage traffic over the link must be rate-limited accordingly. If the IP WAN link is dedicated, the peak available bandwidth equals the total link bandwidth.
Implementing SAN Internetworking Solutions 4 • Digital Signal 1 (DS1) - A framing and formatting specification that transmits 24 digital data channels on a T1 synchronous line. Each channel transmits at 64 Kbps (full-duplex), providing an aggregate bandwidth of 1.544 Mbps. Typical T1 lines are long-distance, point-to-point connections used for private networks and corporate Internet communication. The iFCP protocol overhead for a clear-channel (private) T1 link is 4.4%, which provides a bandwidth of 1.
Implementing SAN Internetworking Solutions 4 • Fast Ethernet - A transmission medium specified by IEEE 802.3 that carries information at 100 Mbps (full-duplex) in baseband form using Category-5 copper cable or fiber-optic cable. The specification was developed to enable faster communication of LAN-connected computers. Fast Ethernet is also called 100 Base-T. The iFCP protocol overhead for a fast Ethernet link is 6.3%, which provides a bandwidth of 93.70 Mbps (11.71 MBps) for storage traffic.
Implementing SAN Internetworking Solutions 4 Distance Extension Best Practices To implement a successful extended-distance BC/DR solution, follow a set of best practice conventions as follows: 1. Use dedicated bandwidth and rate limiting - If possible, negotiate dedicated bandwidth as part of the SLA with the network service provider. Enable intelligent port rate limiting to ensure the ingress data rate does not exceed the negotiated bandwidth.
Implementing SAN Internetworking Solutions 4 — Jumbo frames - To prevent fragmentation of Fibre Channel frames into multiple IP datagrams, enable jumbo frames to increase the data packet size from 1,500 bytes to approximately 9,000 bytes. Ensure the technology is supported by all IP network equipment in the data link. — FastWrite technology - Enable FastWrite technology to reduce protocol overhead for extended-distance write transactions.
Implementing SAN Internetworking Solutions 4 6. Do not implement IP network failover - Implement extendeddistance link failover through the remote data replication software, not the IP network. Many RDR/S and RDR/A software OEMs do not support IP network link failover. 7. Zone controller port pairs - When implementing load sharing, create a separate zone for each pair of communicating ports (one initiator and one target per zone). Assign the zones to different intelligent (iFCP) ports on the SAN router.
Implementing SAN Internetworking Solutions 4 13. Configure out-of-band product management - SAN routers can be managed through inband or out-of-band connectivity. Inband management is provided through the same GbE connections used for iFCP storage traffic. Out-of-band management is provided through a data center LAN that connects servers, workstations, and other network-related equipment. Out-of-band product management is recommended. 14.
Implementing SAN Internetworking Solutions 4 The following sections describe: iSCSI Protocol • iSCSI protocol. • iSCSI server consolidation. • SCSI storage consolidation. iSCSI is based on SCSI protocol that enables hosts to perform block data I/O operations with a variety of target peripherals. Targets include disk drives, tape devices, optical storage devices, printers, and scanners.
Implementing SAN Internetworking Solutions 4 iSCSI Server Consolidation Many enterprise-level IT departments have deployed decentralized computing configurations that include low-end, iSCSI-enabled servers directly attached to storage. While server acquisition costs are typically low, licensing and maintenance costs are often very high in terms of dollars and personnel time. The decentralized infrastructure also causes availability and reliability problems.
Implementing SAN Internetworking Solutions 4 As shown in the figure, server consolidation is enabled by installing an Eclipse 2640 SAN Router that provides iSCSI-to-native FCP connectivity. Stranded servers subject to consolidation do not require installation at one physical location. Servers may be located in a data center or remotely; however, consolidation provides logical connectivity and access as though the servers were co-located.
Implementing SAN Internetworking Solutions 4 4-62 McDATA Products in a SAN Environment - Planning Manual
5 Physical Planning Considerations This chapter describes physical planning considerations for incorporating McDATA directors and switches into storage area networks (SANs) and Fibre Channel fabric topologies. The chapter provides planning considerations and recommendations for: • Port connectivity and fiber-optic cabling. • Rack-mount management server, Ethernet local area network (LAN), and remote access support.
Physical Planning Considerations 5 Port Requirements Plan for sufficient shortwave laser, longwave laser, and 1.0625, 2.1250, 4.2500, and 10.2000 gigabit per second (Gbps) Fibre Channel ports to meet the needs of the SAN configuration. The number of ports required is equal to the number of device connections (including redundant connections), plus the number of interswitch links (ISLs) between fabric elements, plus the total number of spare port connections.
Physical Planning Considerations 5 — Optical paddles that operate at 10.2000 Gbps provide two Fibre Channel port connections. A fully-populated director supports up to 64 connections and can be configured with a combination of shortwave or longwave transceivers. • Sphereon 3232 Fabric Switch - The switch provides up to 32 duplex SFP fiber-optic port transceivers (1.0625 or 2.1250 Gbps operation). Shortwave laser and longwave laser transceivers are available.
Physical Planning Considerations 5 SFP Optical Transceivers Shortwave laser SFP optical transceivers (1.0625, 2.1250, 4.2500, or 10.2000 Gbps) provide a connection for multimode cable with a core diameter of 50 microns and a cladding diameter of 125 microns (50/125 micron), or multimode cable with a core diameter of 62.50 microns and a cladding diameter of 125 microns (62.5/125 micron). Longwave laser SFP optical transceivers (1.0625, 2.1250, 4.2500, or 10.
Physical Planning Considerations 5 When using multimode cable, note the decrease in performance as the cable core diameter or data transmission rate increases. When using singlemode cable, performance is a function of transceiver type. Data transmission distance and link budget are not affected by data transmission rate. Table 5-1 Cable Type and Transmission Rate versus Distance and Link Budget Cable Type and Data Transmission Rate Unrepeated Distance Link Budget 62.5/125 micron multimode at 1.
Physical Planning Considerations 5 • Extended-Distance Ports Existing cable restrictions - The enterprise may contain only one type of fiber-optic cable (multimode or singlemode), and the customer may be required to use the existing cables. Customers may also be required to use existing copper cables for some arbitrated loop devices.
Physical Planning Considerations 5 Fibre Channel Cables and Connectors Cables LC Connectors This section provides Fibre Channel cable and connector planning information as follows: • Cables for directors, fabric switches, and SAN routers. • Intrepid-series director, Sphereon-series fabric switch, and Eclipse-series SAN router optical connectors.
Physical Planning Considerations 5 Figure 5-1 Routing Fiber-Optic Cables SFP Transceiver and LC Duplex Connector Follow a logical plan for routing fiber-optic cables to avoid confusing connections during installation and operation. Route cables from the access holes at the bottom of the Fabricenter equipment cabinet to fabric element ports. When routing cables to ports be aware: • In a Fibre Channel Protocol (FCP) environment, ports are numbered by physical port number.
Physical Planning Considerations 5 • Cable routing outside the equipment cabinet. Plan for 1.5 meters (5 feet) of cable outside the cabinet to provide slack for service clearance, limited cabinet movement, and inadvertent cable pulls. • Cabling distance to servers, storage devices, and directors (for multiswitch fabric support). The need for additional fiber-optic cabling could grow rapidly.
Physical Planning Considerations 5 Management Server The management server is rack-mounted in a Fabricenter equipment cabinet. The server supports up to 48 McDATA directors, fabric switches, or SAN routers (managed products). The server is used to configure the product and associated SAN management and Element Manager applications, monitor product operation, change configurations, download firmware updates, and initiate diagnostics.
Physical Planning Considerations 5 Connectivity Planning Considerations Remote User Workstations Directors, fabric switches, SAN routers, and the management server are delivered in a cabinet-mount configuration in accordance with customer specifications. Because Ethernet cables that connect the managed products, hub, and management server are factoryinstalled, connectivity planning is not required for a stand-alone cabinet installation.
Physical Planning Considerations 5 NOTE: Remote workstation access to Eclipse-series SAN routers is not supported. Remote workstations must have access to the LAN segment on which the management server is installed. Product administrative functions are accessed through the LAN and management server.
Physical Planning Considerations 5 Connection to this LAN segment is optional and depends on customer requirements. This type of network configuration using both Ethernet connections is shown in Figure 5-3. Intrepid 6064 Directors are used as an example.
Physical Planning Considerations 5 • Up to 12 authorized management workstations can be configured through the director or fabric switch SAN management application to receive unsolicited SNMP trap messages that indicate product operational state changes and failure conditions. • Up to eight authorized management workstations can be configured through a SAN router Element Manager application to send SNMP trap messages that indicate product operational state changes and failure conditions.
Physical Planning Considerations 5 Security Provisions Security provisions are available to restrict unauthorized access to a director, switch, or attached Fibre Channel devices. Access to the director or switch (through the SAN management application, Element Manager application, or EFCM Basic Edition interface) is restricted by implementing password protection.
Physical Planning Considerations 5 System administrators can use the SAN management application to assign remote workstation access to directors and switches. Remote sessions are allowed for anyone on a customer intranet, disallowed completely, or restricted to specific workstations. Remote users must log into the SAN management application with a user name and password, just as when logging in to the local management server. Passwords are encrypted when sent across the network.
Physical Planning Considerations 5 The fabric element transmits a random value (used only once), an ID value (incremented at each login), and a shared CHAP secret (16-byte random value) to the server. The server concatenates the random value, ID value, and CHAP secret, and calculates a oneway message digest (also called a hash value). The hash value is transmitted to the authenticator (fabric element).
Physical Planning Considerations 5 • RADIUS server support - Remote authentication dial-in user service (RADIUS) is a client-server, UDP-based protocol that supports storage and authentication of passwords and CHAP secrets. Directors, fabric switches, and SAN routers support a RADIUS client (LAN-connected to a primary or secondary RADIUS server) that authenticates CHAP responses and login passwords. The RADIUS server stores: — Management server-to-fabric element (director or fabric switch) CHAP secrets.
Physical Planning Considerations 5 — Authorization errors. — Authentication errors. — Management application user connections. Use of the SANtegrity Authentication feature in conjunction with other security provisions must be carefully planned and coordinated. For additional information, refer to Security Best Practices. Obtain planning assistance from McDATA’s professional services organization before implementing the feature.
Physical Planning Considerations 5 SANtegrity Binding Planning Considerations • Rerouting delay - If a fabric topology changes, directors and fabric switches calculate a new least-cost data transfer path through a fabric, and routing tables immediately implement that path. This may result in Fibre Channel frames being delivered to a destination device out of order, because frames transmitted over the new (shorter) path may arrive ahead of previouslytransmitted frames that traverse the old (longer) path.
Physical Planning Considerations 5 Figure 5-4 Configure Allow/Prohibit Matrix - Active Dialog Box To access the dialog box, ensure the FICON management style is enabled for the director or switch, then select the Allow/Prohibit and Active options from the Element Manager application’s Configure menu. Figure 5-4 shows that port 1 (logical port address 05) is prohibited from communicating with port 6 (logical port address 0A), port 7 (logical port address 0B), and port 8 (logical port address 0C).
Physical Planning Considerations 5 Figure 5-5 PDCM Array - Example Problem A PDCM array configured for Director A prohibits logical port address 05 from communicating with logical port addresses 0A, 0B, and 0C. No PDCM array is configured for Director B. The PDCM array configured for Director A prohibits the source server from transmitting or receiving data across ISL 2. However, internal route tables at both directors indicate a valid server-to-destination device path across ISL 1.
Physical Planning Considerations 5 Preferred Path The preferred path option allows a user to specify and configure one or more ISL data paths between multiple directors or fabric switches in a fabric. At each fabric element, a preferred path consists of a source port on the director or switch being configured, an exit port on the director or switch, and the Domain_ID of the destination director or switch. Each participating director or switch must be configured as part of a desired path.
Physical Planning Considerations 5 A preferred path is configured between a source server and destination device (A or B), traversing Director 1, Director 2, and Director 3. To configure the preferred path through the first director: 1. Select the Preferred Path option from the Element Manager application’s Configure menu. The Configure Preferred Paths dialog box displays. 2. Click Add. The Add Preferred Path dialog box displays (bottom of Figure 5-6). 3.
Physical Planning Considerations 5 Zoning Directors and fabric switches support a user configuration that partitions attached devices into restricted-access groups called zones. Devices in the same zone can recognize and communicate with each other through switched port-to-port connections. Devices in separate zones cannot recognize name server or route table information and therefore cannot communicate with each other.
Physical Planning Considerations 5 Zones are configured through the SAN management application by authorizing or restricting access to name server or route table information (depending on the firmware release level) associated with device N_Ports that attach to director or switch fabric ports (F_Ports).
Physical Planning Considerations 5 ATTENTION ! If zoning is implemented by WWN, removal and replacement of a device HBA or Fibre Channel interface (thereby changing the device WWN) disrupts zone operation and may incorrectly exclude a device from a zone. • The domain identification (Domain_ID) and physical port number of the director or fabric switch port to which the device is attached.
Physical Planning Considerations 5 — The default zone is enabled or disabled separately from the active zone set. — If the default zone is enabled, then all devices not in a specified zone are included in the default zone and can communicate with each other. — If the default zone is disabled and there is no active zone set, then the zoning feature is completely disabled for the fabric and no devices can communicate with each other.
Physical Planning Considerations 5 • Reasons for zone implementation - Determine if zoning is to be implemented for the enterprise. If so, evaluate if the purpose of zoning is to differentiate between operating systems, data sets, user groups, devices, processes, or some combination thereof. Plan the use of zone members, zones, and zone sets accordingly. • Zone members specified by port number or WWN - Determine if zoning is to be implemented by port number or WWN.
Physical Planning Considerations 5 • Each server HBA is explicitly bound to a storage volume or LUN, and access is explicitly authorized (access is blocked by default). • The process is compatible with OSI standards. The following are transparently supported: — Different operating systems and applications. — Different storage volume managers and file systems. — Different fabric devices, including disk drives, tape drives, and tape libraries.
Physical Planning Considerations 5 2. SANtegrity Binding - The SANtegrity Binding feature is recommended for large and complex SANs with fabrics and devices provided by multiple OEMs or that intermix FCP and FICON protocols. The feature is required for FICON-cascaded high-integrity SANs. SANtegrity Binding includes: — Fabric binding (configured and enabled through the SAN management application) that allows only user-specified directors or switches to attach to specified fabrics in a SAN.
Physical Planning Considerations 5 5. Preferred path - A preferred path provides soft control of fabric routing decisions on a switch-by-switch or port-by-port basis. The path instructs a fabric to use a preferred exit port out of a director or fabric switch for a specified receive port and target domain. If a preferred path is prohibited by SANtegrity Binding, PDCM arrays, or hard zoning, the path is not programmed.
Physical Planning Considerations 5 Follow best practices listed here in order of precedence. Logically work in sequence from the most restrictive method to the least restrictive method, ensuring the most restrictive connectivity or routing paths override all other paths. Optional Feature Keys McDATA offers several operating features that are available for the switch as customer-specified options.
Physical Planning Considerations 5 • Full volatility - Enablement of this feature ensures that no Fibre Channel frames are stored after a director or Fabric switch is powered off or fails, and a memory dump file (that possibly includes classified frames) is not included as part of the data collection procedure. NOTE: The Intrepid 10000 Director and SAN routers do not support full volatility. • Full fabric - This feature is provided only for the Sphereon 4300 Fabric Switch.
Physical Planning Considerations 5 Inband Management Access Inband management console access (through a Fibre Channel port) is provided by enabling user-specified features that allow OSMS or FICON (FMS) host control of a director or fabric switch. The features can be simultaneously installed and enabled.
Physical Planning Considerations 5 Flexport Technology • Connectivity to an IBM eServer zSeries 800 (z800), zSeries 900 (z900), or zSeries 990 (z990) processor, with one or more FICON or FICON Express channel adapter cards installed, using the z/OS operating system, Version 1.1 or later. • A host-attached Hardware Management Console. The console runs the Hardware Management Console application (HWMCA) and is the operations and management PC platform for S/390 or zSeries servers.
Physical Planning Considerations 5 • Switch binding - This portion of the feature allows only specified devices and fabric elements to connect to specified director or fabric switch ports. • Enterprise Fabric Mode - Although Enterprise Fabric Mode is not a keyed feature, it is required for SANtegrity Binding operation. Enterprise Fabric Mode also enables the following parameters: — Rerouting delay. — Domain RSCNs. — Insistent Domain_ID.
Physical Planning Considerations 5 The figure illustrates two Intrepid 6064 Directors connected by two ISLs. Three servers use the ISLs to communicate with two storage devices. Without trunking, servers 1 through 3 route Fibre Channel traffic from to director B without regard to any data rates.
Physical Planning Considerations 5 Remote Fabric Intrepid 10000 Director LIMs contain two scalable packet processors, each supporting an optical paddle pair. Each paddle pair provides 16 ports (1.0625 or 2.12500 Gbps operation), four ports (10.2000 Gbps operation), or ten ports (mixed data rate operation). A minimal BB_Credit buffer pool is allocated among all paddle-pair ports that allows a 1.0625 or 2.12500 Gbps port to be set to 60 BB_Credits and a 10.2000 Gbps port to be set to 360 BB_Credits.
Physical Planning Considerations 5 5-40 McDATA Products in a SAN Environment - Planning Manual
6 Configuration Planning Tasks This chapter describes configuration planning best-practices tasks to be performed before installing one or more McDATA Fibre Channel switching products in a storage area network (SAN) configuration. Table 6-1 summarizes planning tasks described in the chapter.
Configuration Planning Tasks 6 Table 6-1 Configuration Planning Tasks (continued) Task Page Task 13: Complete the Planning Worksheet 6-14 Task 14: Plan AC Power 6-28 Task 15: Plan a Multiswitch Fabric (Optional) 6-29 Task 16: Plan Zone Sets for Multiple Products (Optional) 6-30 Task 17: Plan SAN Routing (Optional) 6-31 Task 18: Complete Planning Checklists 6-34 Task 1: Prepare a Site Plan For each director, fabric switch, SAN router, or FC-512 Fabricenter equipment cabinet installed, design
Configuration Planning Tasks 6 • Power requirements, including an optional uninterruptable power supply (UPS). • Lengths of power cables and location of electrical outlets (for directors, switches, and the management server) having the proper phase, voltage, amperage, and ground connection. DANGER Use the supplied power cords. Ensure the facility power receptacle is the correct type, supplies the required voltage, and is properly grounded.
Configuration Planning Tasks 6 In addition, consider the following when planning cable routing: • The need for additional fiber-optic cables could grow rapidly. Consider installing cable with extra fibers, especially in hard to reach places like underground trenches. Consider locating the equipment cabinet near a fiber-optic patch panel. • Follow proper procedures when moving an installed equipment cabinet to prevent cable or connector damage.
Configuration Planning Tasks 6 NOTE: If the FICON management server feature is enabled, the default operating style is FICON. The open systems management style cannot be enabled. Consider purchasing and enabling the SANtegrity Authentication and SANtegrity Binding features to provide additional data security in a complex and multi-OEM environment. Contact your McDATA representative for information about the features.
Configuration Planning Tasks 6 • Remote user workstations - If remote access to the management server is required, plan to install user workstations with the SAN management and Element Manager applications configured. Administrators can use these remote workstations to configure and monitor directors and fabric switches. Up to 25 sessions can be simultaneously active. Sessions from remote user workstations are disabled if the management server is powered off.
Configuration Planning Tasks 6 Task 5: Plan Ethernet Access The management server and one or more products are configured on a dedicated Ethernet LAN segment and delivered in the Fabricenter equipment cabinet. No Ethernet access planning is required for a stand-alone cabinet. This task is required to: • Connect equipment cabinets - Ethernet hubs in multiple equipment cabinets can be connected to provide management server access to up to 48 managed McDATA products.
Configuration Planning Tasks 6 • If installing products and the management server on a dedicated (private) LAN segment, there is no requirement to change any default network addresses. However, if multiple equipment cabinets are connected, ensure all managed products and servers have unique IP addresses. If new IP addresses are required, consult with the customer’s network administrator.
Configuration Planning Tasks 6 • Intrepid-series directors and Sphereon-series fabric switches: — MAC address is unique for each product. — Default IP address is 10.1.1.10. — Subnet mask is 255.0.0.0. — Gateway address is 0.0.0.0. • Eclipse 1620 SAN Router: — System addresses: • MAC address is unique for each product. • Default IP address is 192.168.111.100. • Subnet mask is 255.255.255.0. • Gateway address is 0.0.0.0. — 10/100 Base-T Ethernet management port addresses: • Default IP address is 192.168.
Configuration Planning Tasks 6 • Subnet mask is 0.0.0.0. • Gateway address is 0.0.0.0. — 10/100 Base-T Ethernet management port addresses: • Default IP address is 192.168.100.100. • Subnet mask is 255.255.255.0. • Gateway address is 0.0.0.0. — Intelligent port (13 through 16) addresses: • Default IP address is 0.0.0.0. • Subnet mask is 0.0.0.0. • External IP address is 0.0.0.0. • Internal IP address is 0.0.0.0.
Configuration Planning Tasks 6 • Obtain IP addresses and SNMP community names for management workstations that have access to products. • Determine which (if any) management workstations can have write permission for SNMP variables. • Obtain product-specific trap information from McDATA to load onto SNMP management workstations. Task 8: Plan E-Mail Notification (Optional) As an option, network administrators can configure director and fabric switch e-mail support.
Configuration Planning Tasks 6 • Remote access to products is possible through the maintenance port or an internal modem connection to the management server. These connections are for use by authorized service personnel only and should be carefully monitored. • The number of remote workstations with access to the management server and managed products can and should be restricted. Obtain IP addresses for workstations that should have exclusive access.
Configuration Planning Tasks 6 Part of this task may have been performed when the configuration was determined. It might be helpful to draw the configuration diagram. Indicate distances in the diagram if necessary. Transfer information from the configuration diagram to the product planning worksheet provided as part of Task 13: Complete the Planning Worksheet.
Configuration Planning Tasks 6 Rules for Nicknames Nicknames can be up to 32 alphanumeric characters in length. Spaces, hyphens, and underscores are allowed within the nickname. Each nickname must be unique (corresponding to a unique WWN). Example nicknames include: Fabric-1. Host system-1. DASD_001. Task 13: Complete the Planning Worksheet The planning worksheet included in this task is an eight-page form that depicts port assignments for a director, switch, or SAN router.
Configuration Planning Tasks 6 Product Planning Worksheet (Page 1 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 Configuration Planning Tasks 6-15
Configuration Planning Tasks 6 Product Planning Worksheet (Page 2 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 6-16 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 3 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 Configuration Planning Tasks 6-17
Configuration Planning Tasks 6 Product Planning Worksheet (Page 4 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 6-18 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 5 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 Configuration Planning Tasks 6-19
Configuration Planning Tasks 6 Product Planning Worksheet (Page 6 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 6-20 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 7 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 Configuration Planning Tasks 6-21
Configuration Planning Tasks 6 Product Planning Worksheet (Page 8 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 6-22 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 9 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 Configuration Planning Tasks 6-23
Configuration Planning Tasks 6 Product Planning Worksheet (Page 10 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 6-24 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 11 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 Configuration Planning Tasks 6-25
Configuration Planning Tasks 6 Product Planning Worksheet (Page 12 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 6-26 McDATA Products in a SAN Environment - Planning Manual Model IP Address Zone
Configuration Planning Tasks 6 Product Planning Worksheet (Page 13 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 Configuration Planning Tasks 6-27
Configuration Planning Tasks 6 Task 14: Plan AC Power Plan for facility power sources for each Fabricenter equipment cabinet, director, fabric switch, or SAN router as follows: • The Fabricenter equipment cabinet operates at 47 to 63 Hertz (Hz), 200 to 240 volts alternating current (VAC), and requires a minimum dedicated 30-ampere service. • The Intrepid 6140 Director operates at 47 to 63 Hz, 200 to 240 VAC, and requires a minimum dedicated 15-ampere service.
Configuration Planning Tasks 6 Task 15: Plan a Multiswitch Fabric (Optional) If a multiswitch fabric topology is to be implemented, carefully plan the physical characteristics and performance objectives of the topology, including the proposed number of fabric elements, characteristics of attached devices, cost, nondisruptive growth requirements, and service requirements. Refer to Fabric Topologies, Planning for Multiswitch Fabric Support, and General Fabric Design Considerations for detailed information.
Configuration Planning Tasks 6 Task 16: Plan Zone Sets for Multiple Products (Optional) If name server zoning is to be implemented, carefully plan the characteristics and security objectives (separation of operating systems, data sets user groups, devices, or processes) of zone members, zones and zone sets. If a fabric topology or routed SAN is implemented, zoning is configured on a fabric-wide or SAN-wide basis. Planning for zoned configurations must be carefully coordinated with planning the topology.
Configuration Planning Tasks 6 Task 17: Plan SAN Routing (Optional) If a routed SAN is to be implemented, carefully plan metropolitan area network (MAN) or wide area network (WAN) connectivity and the integration of SAN routers with standard fibre channel fabric elements. Ensure basic requirements for physical SAN routers are incorporated in the site plan(s) developed while performing Task 1: Prepare a Site Plan.
Configuration Planning Tasks 6 • IP network connections - At each location, cables with appropriate connectors must be routed between the IP transport network and the SAN router. The Eclipse 1620 SAN Router supports Ethernet RJ-45 connectors or SFP optical transceivers with LC duplex connectors. The Eclipse 2640 SAN Router supports SFP optical transceivers with LC duplex connectors.
Configuration Planning Tasks 6 • Negotiate SLA - Network service providers provide IP WAN transport services in accordance with a negotiated service level agreement (SLA). Ensure the SLA specifies the link availability, peak available bandwidth, latency, security level, monitoring level, packet loss, and mean time to repair (MTTR). • Intelligent port addresses - Each intelligent port that supports Internet Fibre Channel protocol (iFCP) requires an IP address and subnet mask.
Configuration Planning Tasks 6 Task 18: Complete Planning Checklists As a guide for planning tasks, complete the planning checklists under this task. Checklists provide detailed planning activities and provide space for a planned completion date for each activity. The customer’s management information system (MIS) project manager should examine the checklists and determine the personnel and resources required for completing planning and installation tasks.
Configuration Planning Tasks 6 Table 6-2 Physical Planning and Hardware Installation Tasks Activity Task Owner Due Date Comments Locate the physical facilities. Connect the facility alternating current (AC) power circuits. If more than one managed product, consider separate power circuits for availability. Obtain an uninterruptable power supply (optional). Recommended. Obtain two outside-access phone lines. One for the modem and the second for a telephone.
Configuration Planning Tasks 6 Table 6-2 Physical Planning and Hardware Installation Tasks (continued) Activity Task Owner Due Date Comments Set up LAN connections to corporate intranet for remote workstation access (optional). Remote workstation access is supported for directors and fabric switches only. Determine peak available bandwidth of the available IP WAN network (optional). If SAN routing is supported, rate limiting must be configured and enabled based on peak available bandwidth.
Configuration Planning Tasks 6 Table 6-3 Operational Setup Tasks (continued) Activity Determine if inband management of the director or switch is to be used, and if so, the type (FICON or open-systems). Task Owner Due Date Comments Management server and Fibre-Channel-attached server peripheral (optional). Determine if the call-home feature is to be used. Determine call-home telephone numbers to be used. Determine if the e-mail notification feature is to be used.
Configuration Planning Tasks 6 Table 6-3 Operational Setup Tasks (continued) Activity Configure extended distance ports. Task Owner Due Date Comments If SAN routing is supported, configure extended distance ports in accordance with IP WAN requirements. Enable and configure optional feature keys. Configure link incident alerts. Configure Ethernet events.
A Product Specifications This appendix lists specifications for McDATA directors, fabric switches, storage area network (SAN) Routers, and the FC-512 Fabricenter equipment cabinet.
Product Specifications Intrepid 10000 Director: Height: 62.2 centimeters (24.5 inches) or 14 rack units. Width: 44.3 centimeters (17.5 inches). Depth: 86.4 centimeters. (34.0 inches). Weight: 152.0 kilograms (335.0 pounds). Sphereon 3232 Fabric Switch: Height: 6.5 centimeters (2.6 inches) or 1.5 rack units. Width: 44.5 centimeters (17.5 inches). Depth: 64.1 centimeters (25.2 inches). Weight: 16.8 kilograms (37.0 pounds). Sphereon 4300 Fabric Switch: Height: 4.1 centimeters (1.6 inches) or 1 rack unit.
Product Specifications Eclipse 1620 SAN Router: Height: 4.1 centimeters (1.6 inches) or 1 rack unit. Width: 43.7 centimeters (17.2 inches). Depth: 45.7 centimeters (18.0 inches). Weight: 5.9 kilograms (13.0 pounds). Eclipse 2640 SAN Router: Height: 4.1 centimeters (1.6 inches) or 1 rack unit. Width: 43.7 centimeters (17.2 inches). Depth: 68.6.7 centimeters (27.0 inches). Weight: 10.9 kilograms (24.0 pounds).
Product Specifications Sphereon 3232 Fabric Switch: Input voltage: 100 to 240 VAC. Input current: 1.3 amps at 208 VAC. Input frequency: 47 to 63 Hz. Sphereon 4300 Fabric Switch: Input voltage: 100 to 240 VAC. Input current: 0.4 amps at 208 VAC. Input frequency: 47 to 63 Hz. Sphereon 4400 Fabric Switch: Input voltage: 90 to 264 VAC Input current: 0.18 amps at 208 VAC Input frequency: 47 to 63 Hz Sphereon 4500 Fabric Switch: Input voltage: 100 to 240 VAC. Input current: 0.5 amps at 208 VAC.
Product Specifications Heat Dissipation McDATA products have the following maximum heat dissipation characteristics: Intrepid 6064 Director: 490 watts (1,672 BTU/hr). Intrepid 6140 Director: 841 watts (2,873 BTU/hr). Intrepid 10000 Director: 2,496 watts (8,517 BTU/hr). Sphereon 3232 Fabric Switch: 245 watts (836 BTU/hr). Sphereon 4300 Fabric Switch: 37 watts (127 BTU/hr). Sphereon 4400 Fabric Switch: 36 watts (123 BTU/hr). Sphereon 4500 Fabric Switch: 49 watts (167 BTU/hr).
Product Specifications Sphereon 3232 Fabric Switch: Right and left side: No clearance required. Front and rear: 7.6 centimeters (3.0 inches). Top and bottom: No clearance required. Sphereon 4000-Series Switches: Right and left side: 1.3 centimeters (0.5 inches). Front and rear: 7.6 centimeters (3.0 inches). Top and bottom: No clearance required. Eclipse 1620 and 2640 SAN Routers: Right and left side: No clearance required. Front and rear: 7.6 centimeters (3.0 inches). Top and bottom: No clearance required.
Product Specifications Shipping temperature: -400 F to 1400 F (-400 C to 600 C). Storage temperature: 340 F to 1400 F (10 C to 600 C). Shipping relative humidity: 5% to 100%. Storage relative humidity: 5% to 80%. Maximum wet-bulb temperature: 840 F (290 C). Altitude: 40,000 feet (12,192 meters). Operating Environment This section specifies environmental requirements for operating McDATA products. Temperature: 400 F to 1040 F (40 C to 400 C). Relative humidity: 8% to 80%.
Product Specifications Dimensions The Fabricenter cabinet has the following physical dimensions: Height: 186.1 centimeters (73.2 inches). A total of 39 rack units (39 U) are available internal to the cabinet for product installation. Width: 61.0 centimeters (24.0 inches). Depth: 105.4 centimeters (41.5 inches). Weight (no installed products): 153.2 kilograms (337.0 pounds). Weight (shipping container): 50.9 kilograms (112.0 pounds).
Product Specifications Figure A-1 Fabricenter Cabinet Footprint Product Specifications A-9
Product Specifications A-10 McDATA Products in a SAN Environment - Planning Manual
B Firmware Summary This appendix summarizes differences and similarities between the Enterprise Operating System, classic (E/OSc) for Intrepid 6000-series directors and Sphereon-series fabric switches; Enterprise Operating System, nScale (E/OSn) for the Intrepid 10000 Director; and Enterprise Operating System, internetworking (E/OSi) for Eclipse-series SAN routers. The appendix includes tables that list: • System-related differences. • Fibre Channel protocol-related differences.
Firmware Summary Table B-1 B-2 E/OSc versus E/OSn and E/OSi - System-Related Differences (Continued) Feature E/OSc 8.0 E/OSn 6.0 Nondisruptive hot code activation (HotCAT) Upgrade nondisruptive to Fibre Channel traffic for single and dual CTP card directors and for fabric switches. Upgrade does not require director CTP card switchover. E/OSc allows upgrade or downgrade by greater than one functional release. Director must be set offline to upgrade a single CTP card.
Firmware Summary Table B-1 E/OSc versus E/OSn and E/OSi - System-Related Differences (Continued) Feature E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 Misaligned word generation Misaligned words not generated. Misaligned word is generated when the port state machine (PSM) transitions from inactive to active state or from active to inactive state. This anomaly has no effect on Fibre Channel device behavior. Misaligned words not generated. Flexible partition (FlexPar) feature support Flexpar feature not supported.
Firmware Summary Table B-2 E/OSc versus E/OSn and E/OSi - Fibre Channel Protocol-Related Differences Feature B-4 E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 Expansion port (E_Port) staging E_Port staging not supported. The Intrepid 10000 Director supports E_Port staging. The director allows only one E_Port connection to a neighbor switch to take part in a fabric build process. Upon process completion, subsequent E_Ports to neighbor switches are brought up (staged).
Firmware Summary Table B-2 E/OSc versus E/OSn and E/OSi - Fibre Channel Protocol-Related Differences Feature E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 Reroute delay behavior With reroute delay enabled, the destination route point is cleared and a delay equal to the error detect time-out value (ED_TOV) is applied before a new route is programmed. During the delay, Class 3 frames are dropped.
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature B-6 E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 General SNMP support SNMP interface supports Version 1, Version 2c, and a configuration-related subset of product features. SNMP interface supports Version 1, Version 2c, and Version 3. Read-only use recommended with Versions 1 and 2c. Read-write use recommended with secure Version 3.
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 FICON Management Server (FMS) PFE key install and enable behavior FMS is auto-enabled upon PFE installation. Management style automatically changes to FICON. When PFE is installed, the user must explicitly enable FMS. Management style does not automatically change to FICON. FMS PFE key not supported.
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature B-8 E/OSc 8.0 E/OSn 6.0 Port group online diagnostics (directors) Diagnostic errors for any single port in a group (port card) cause a group failure indication. Diagnostic errors for any single port in a group (LIM) are indicated individually and do not cause a group failure indication. Port grouping (port cards or LIMs) not supported.
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 Special port states and reason codes Port state Inactive with special reason codes Reserved and InvalidOTPConfig not supported. Logical port addresses FE and FF forced offline and placed in special state Inactive with reason code Reserved if FICON CUP is enabled. FICON states are Internal Port (address FE) and Unimplemented (address FF).
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature B-10 E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 Diagnostics (port failure state) Port set to Failed state upon failing a user- invoked port diagnostic test. If external port diagnostics are performed without a loopback plug the diagnostic test fails, but port is not set to Failed.
Firmware Summary Table B-3 E/OSc versus E/OSn and E/OSi - Management-Related Differences (Continued) Feature E/OSc 8.0 E/OSn 6.0 E/OSi 4.6 FICON management style support User-selectable option on a per product basis. Setting is stored on the product. User-selectable option on a per user (different for each login ID) or per product basis. Setting is stored on the management server, not the product. Setting is backed up by the server backup process.
Firmware Summary B-12 McDATA Products in a SAN Environment - Planning Manual
Index A access control list inband 5-18 out-of-band 5-18 role-based access 4-7 any-to-any connectivity 1-28 application I/O profiles 3-31 arbitrated loop switch connectivity features 1-28 default network address 6-9 security features 1-31 serviceability features 1-32 arbitrated loop topology description 3-2 fabric-attached planning considerations 3-11 FL_Port connectivity 3-10 operating characteristics 3-2 private device connectivity 3-7 private loop 3-9 private loop planning considerations 3-10 public dev
Index Sphereon 4300 Fabric Switch 1-30 Sphereon 4400 Fabric Switch 1-30 Sphereon 4500 Fabric Switch 1-30 Sphereon 4700 Fabric Switch 1-30 best practices cabling 3-20 configuration planning 6-1 connectivity 3-20 distance extension 4-55 FCP and FICON intermix 3-47 FICON cascading 3-54 multiswitch fabric topology 3-20 preventing ISL oversubscription 3-33 SAN routing 4-29 security 5-30 binding fabric 5-19 persistent 5-29 SANtegrity 5-19 switch 5-19 business continuance IP versus storage traffic 4-36 operationa
Index Tier 1 connections 3-17 Tier 2 connections 3-18 Tier 3 connections 3-18 CT authentication 5-17 D dark fiber distance extension bandwidth 4-46 description 4-39 illustration 4-40 latency 4-46 recovery point objective 4-46 recovery time objective 4-46 data compression algorithm selections 4-26 description 1-24 optimizing WAN use 4-55 set compression level 4-57 data transmission distance cable type 5-4 multiswitch fabric requirements 3-21 transceiver type 5-4 default network addresses director or fabric
Index proxy Domain_ID 30 4-13, 4-18 proxy Domain_ID 31 4-13, 4-25 R_Port 4-15 SAN router 4-15 E E/OSc description 2-11 management-related properties B-5 operating system differences B-1 operating system similarities B-1 protocol-related properties B-3 system-related properties B-1 E/OSi description 2-12 management-related properties B-5 operating system differences B-1 operating system similarities B-1 protocol-related properties B-3 system-related properties B-1 E/OSn description 2-11 management-related
Index port configuration 5-6 remote fabric feature 5-39 support 1-29 transport technology dark fiber 4-39 IP 4-45 SONET/SDH 4-42 WDM 4-40 transport technology comparison 4-48 F fabric availability nonresilient dual fabric 3-38 nonresilient single fabric 3-37 redundant fabrics 3-38 resilient dual fabric 3-38 resilient single fabric 3-38 fabric binding 5-19 fabric element FCP and FICON intermix environment 3-41 limitations in a fabric 3-19 fabric performance device fan-out ratio 3-34 fabric initialization 3
Index FICON management server description 5-35 introduction 2-5 plan console support 6-6 firmware application services 2-12 E/OSc description 2-11 E/OSi description 2-12 E/OSn description 2-11 fabric services 2-13 Fibre Channel protocol services 2-12 loop services 2-13 network services 2-12 operating system differences B-1 operating system services 2-12 operating system similarities B-1 port services 2-12, 2-13 system management services 2-12 FL_Port connectivity 3-10 FlexPar technology description 4-4 dir
Index I I/O requirements application I/O profiles 3-31 device locality 3-33 ISL oversubscription 3-32 iFCP protocol build fabric events 4-23 comparison to mFCP 4-28 description 4-22 inband product management feature keys 5-35 FMS feature 2-5 OSMS feature 2-5 plan console support 6-6 intelligent port Eclipse 1620 SAN Router 1-25 Eclipse 2640 SAN Router 1-27 implement rate limiting 4-51 set port speed 4-52 inter-FlexPar routing 4-28 interoperability planning 6-4 vendor limitations 3-20 Intrepid 10000 Directo
Index L large fabric fabric initialization 3-29 fabric scalability 3-39 high-bandwidth ISLs 3-30 high-port count directors 3-30 problems 4-3 laser transceiver description 5-4 restrictions 5-5 SFP transceiver 5-8 transmission distance 5-4 latency dark fiber transport 4-46 directors 1-7 IP transport 4-48 SONET/SDH transport 4-47 WDM transport 4-47 LC duplex connector 5-7 LIM assigning BB_Credits 4-50 Intrepid 10000 Director 5-2 load balancing 3-22 local area network comparison to WAN 4-36 latency 4-36 protoc
Index N N_Port DHCHAP authentication 5-17 N_Port ID virtualization feature 5-39 name conventions, ports 6-13 name server zoning introduction 1-28 planning requirements 6-30 network addresses default settings 6-8 planning 6-7 nickname conventions, ports 6-14 nonresilient fabric dual 3-38 single 3-37 nScale architecture 1-12 O open-system management server description 5-35 introduction 2-5 plan console support 6-6 OpenTrunking feature description 5-37 planning considerations 3-22 support planning 6-29 opera
Index plan SAN routing 6-31 plan SNMP support 6-10 plan zone sets 6-30 prepare a site plan 6-2 planning worksheet 6-14 port binding 1-31 blocking 1-28 fiber-optic cabling 5-1 logical port addressing 3-42 name conventions 6-13 nickname conventions 6-14 numbering 3-42 port card Intrepid 6064 Director 5-2 Intrepid 6140 Director 5-2 map 3-43 port connections any-to-any connectivity 1-28 Eclipse 1620 SAN Router 1-25 Eclipse 2640 SAN Router 1-27 Intrepid 10000 Director 1-13 Intrepid 6064 Director 1-9 Intrepid 61
Index public arbitrated loop 3-8 loop device connectivity 3-6 R R_Port configuring 4-34 Domain_ID assignment 4-15 operation 4-11 RADIUS server support 5-18 rate limiting description 4-26 implementation 4-51 intelligent port speed selections 4-52 recovery point objective dark fiber transport 4-46 description 4-37 IP transport 4-48 SONET/SDH transport 4-47 WDM transport 4-47 recovery time objective dark fiber transport 4-46 description 4-37 IP transport 4-48 SONET/SDH transport 4-47 WDM transport 4-47 redun
Index mSAN routing 4-18 planning requirements 6-31 R_Port operation 4-11 routing domain (iSAN) 4-25 routing domain (mSAN) 4-18 SAN island consolidation 4-8 Tier 1 (fabrics) 4-8 Tier 2 (mSANs) 4-9 Tier 3 (iSANs) 4-9 zone policy 4-16 SANavigator application description 2-15 GUI description 2-15 product overview 1-5 SANtegrity Authentication feature CHAP authentication 5-16 CT authentication 5-17 description 5-16 DHCHAP authentication 5-17 feature key description 5-36 inband access control list 5-18 out-of-ba
Index SONET/SDH distance extension bandwidth 4-47 description 4-42 illustration 4-43 latency 4-47 recovery point objective 4-47 recovery time objective 4-47 specifications director clearances A-5 director dimensions A-1 director heat dissipation A-5 fabric switch clearances A-5 fabric switch dimensions A-1 fabric switch heat dissipation A-5 fabric switch power requirements A-3 Fabricenter cabinet clearances A-8 Fabricenter cabinet dimensions A-8 Fabricenter cabinet footprint A-8 Fabricenter cabinet power r
Index V vendor interoperability limitations 3-20 W WDM distance extension bandwidth 4-47 description 4-40 illustration 4-41 latency 4-47 recovery point objective 4-47 recovery time objective 4-47 weight directors A-1 fabric switches A-1 Fabricenter cabinet A-8 SAN routers A-1 wide area network comparison to LAN 4-36 dedicated bandwidth 4-55 latency 4-36 optimize use buffering 4-55 data compression 4-55 FastWrite technology 4-56 flow control 4-55 jumbo frames 4-56 protocol stack 4-36 rate limiting 4-51 rel
