An Oracle Deployment Guide September 2011 Deploying a Converged Network Using Oracle CNAs and a Cisco FCoE Switch
Deploying a Converged Storage Network Using a Cisco FCoE Switch Disclaimer Information furnished in this manual is believed to be accurate and reliable. However, Oracle assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. Oracle reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Documentation Conventions................................................................................ 1 Introduction ......................................................................................................... 2 Intended Audience .............................................................................................. 3 Planning ..............................................................................................
Deploying a Converged Storage Network Using a Cisco FCoE Switch Sun Storage 10GbE FCoE PCIe Converged Network Adapter ...................... 21 Sun ZFS Storage Appliance .......................................................................... 21 Cisco Unified Fabric Switch ........................................................................... 21 Appendix C: Converged Network Adapter Overview.......................................... 23 Kit Contents ...................................................
Deploying a Converged Storage Network Using a Cisco FCoE Switch Documentation Conventions This guide uses the following documentation conventions: CAUTION! indicates the presence of a hazard that has the potential of causing damage to data or equipment. WARNING!! indicates the presence of a hazard that has the potential of causing personal injury. Text in blue font indicates a hyperlink (jump) to a figure, table, or section in this guide, and links to Websites are shown in underlined blue.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Introduction In the Unified Data Center Fabric Primer: FCoE and Data Center Bridging publication, a converged network is defined as a unified data center fabric: "A unified data center fabric is a networking fabric that combines traditional LAN and storage area network (SAN) traffic on the same physical network with the aim of reducing architecture complexity and enhancing data flow and access.
Deploying a Converged Storage Network Using a Cisco FCoE Switch FIGURE 1 ILLUSTRATION OF EFFICIENCY OF CONVERGING Intended Audience This guide is intended for system engineers and planners who want to provide converged networking products, solutions, and services to their customers. It is also intended for network planners and administrators who are implementing a converged network for their company. This guide describes how to install an Oracle converged network in preparation for production deployment.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Planning Selecting a Test Architecture When planning to install a converged network, it is important to choose both Fibre Channel and traditional Ethernet-based traffic flows. Combining a test SAN infrastructure and a test LAN infrastructure is often the easiest and most available option for a pilot project. Alternatively, a critical business application test system can closely simulate a production environment.
Deploying a Converged Storage Network Using a Cisco FCoE Switch There are three ways to connect Fibre Channel storage to a unified fabric: Converged network adapter > FCoE switch > Fibre Channel switch > Fibre Channel storage: The adapter connects to the FCoE switch with Ethernet infrastructure, and the FCoE switch connects to storage through a Fibre Channel switch.
Deploying a Converged Storage Network Using a Cisco FCoE Switch FIGURE 2 REFERENCE ARCHITECTURE DIAGRAM Equipment Details Table 1 lists the referenced architecture equipment. Two Sun servers from Oracle were installed with Oracle Solaris 10 9/10. All servers used Oracle’s Sun Storage 10GbE FCoE PCIe Converged Network Adapter. One Sun ZFS Storage Appliance from Oracle and one Sun Storage 6000 Series Storage Array from Oracle provided the FC SAN storage.
Deploying a Converged Storage Network Using a Cisco FCoE Switch SG-(X)EMFCOE2-Q-SR, (short-range optics) SG-(X)EMFCOE2-Q-TA (twin-axial copper) 1 Cisco Nexus™ 5000 Series FCoE switch 1 Sun ZFS Storage Appliance Sun ZFS Storage 7000 Series 1 Sun Storage 6000 Series Array Sun Storage 6180 Array Installation This section describes how to set up an FCoE environment. It assumes a general understanding of SAN administration concepts. The installation process consists of the following steps: 1.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Determining the Configuration Sun 10 GbE FCoE PCIe Converged Network Adapters are supported on multiple hardware platforms and operating systems. Generally, the following specifications apply, but you should always check the Oracle Website for current information.
Deploying a Converged Storage Network Using a Cisco FCoE Switch 9. Plug the appropriate Ethernet cable (either copper or optical) into the adapter. Optical models ship with optical transceivers installed. Go to the Oracle compatibility Website for a listing of approved copper cables. 10. Plug in the power cable, and turn on the computer. For detailed installation instructions, see the Sun Storage 10GbE FCoE PCIe Converged Network Adapter Installation Guide.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Cabling To connect the Fibre Channel and Ethernet cables: 1. Connect the Fibre Channel cables from the servers to the Cisco FCoE Nexus switch. 2. Connect the Fibre Channel cables from the storage to the Cisco FCoE Nexus switch. 3. Connect any necessary Ethernet cables for device management and iSCSI storage.
Deploying a Converged Storage Network Using a Cisco FCoE Switch 4. Create a virtual Fibre Channel interface. switch(config)# interface vfc101 switch(config-if)# bind interface ethernet 1/1 5. Map VFC to VSAN. switch(config)# vsan database switch(config-vsan)# vsan 3 interface vfc101 6. Configure VLAN on the physical Ethernet port.
Deploying a Converged Storage Network Using a Cisco FCoE Switch 12. Verify that the CNA is logged in to the switch. To verify that all equipment is logged in and operating properly: a. Verify LAN management capability on all devices through the associated device management application. b. Verify that servers and converged network adapters are logged in to an FCoE switch under both Ethernet (eth1/16) and Fibre Channel (vfc116) protocols.
Deploying a Converged Storage Network Using a Cisco FCoE Switch 13
Deploying a Converged Storage Network Using a Cisco FCoE Switch FIGURE 4 LUN ASSIGNMENTS 14
Deploying a Converged Storage Network Using a Cisco FCoE Switch 2. Reboot the servers to discover the assigned LUNs. On the SPARC Enterprise M5000 Server, the following command is an intermediate check to ensure that the hardware is seeing the LUNs in a preboot environment. The following information is the LUN portion obtained from the SPARC Enterprise M5000 Server NVRAM display. SPARC Enterprise M5000 Server, using Domain console Copyright (c) 1998, 2010, Oracle and/or its affiliates. All rights reserved.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Driver Version: 20110321-3.05 Type: N-port State: online Supported Speeds: 10Gb Current Speed: 10Gb Node WWN: 200000c0dd1460c9 HBA Port WWN: 210000c0dd1460cb OS Device Name: /dev/cfg/c3 Manufacturer: QLogic Corp. Model: 375-3681-01 Firmware Version: 05.04.03 FCode/BIOS Version: BIOS: 2.14; fcode: 3.10; EFI: 3.20; Serial Number: 0402A00-1006527597 Driver Name: qlc Driver Version: 20110321-3.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Target List Menu HBA Model QLE8142-S 1: Port 1 (HBA 0 OS 0): WWPN: 21-00-00-C0-DD-14-60-C9 Online 2: Port 2 (HBA 1 OS 1): WWPN: 21-00-00-C0-DD-14-60-CB Link Down 3: All HBAs 4: Return to Previous Menu Note: 0 to return to Main Menu Enter Selection: 1 SANsurfer FC/CNA HBA CLI v1.7.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Port ID 4: All Target(s) 5: Return to Previous Menu : 01-03-00 4. Verify that you can access the LUNs with the format command. SCX4450-BRM-03# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 /pci@0,0/pci8086,3607@4/pci108e,286@0/disk@0,0 1. c0t1d0 /pci@0,0/pci8086,3607@4/pci108e,286@0/disk@1,0 2.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Conclusion This document summarized how to plan for, manage, and implement an FCoE pilot. This step-by-step guide provided guidance on the implementation of a unified fabric. The reference architecture described in this guide was assembled from equipment that was available in the Oracle Solution Center located in Broomfield, Colorado.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Appendix A: FCoE and Enhanced Ethernet Related Materials The following links provide more detailed information, and connect to the IEEE documents that define the Enhanced Ethernet functions: P802.1Qbb (Draft 2.3): Priority-based Flow Control http://www.ieee802.org/1/files/private/bb-drafts/d2/802-1bb-d2-3.pdf P802.1Qaz(Draft 2.5): Enhanced Transmission Selection http://www.ieee802.org/1/files/private/az-drafts/d2/802-1az-d2-5.pdf P802.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Appendix B: Hardware and Software Sun Storage 10GbE FCoE PCIe Converged Network Adapter The Sun Storage 10GbE FCoE PCIe Converged Network Adapter is a single-chip, fully offloaded FCoE initiator, operating in both virtual and nonvirtual environments, running over an Enhanced Ethernet fabric. This converged network adapter initiator boosts system performance with 10 Gbps speed and full hardware offload for FCoE protocol processing.
Deploying a Converged Storage Network Using a Cisco FCoE Switch For more information about the Cisco Nexus 5000 Series switch, visit http://www.cisco.com/en/US/products/ps9670/index.html.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Appendix C: Converged Network Adapter Overview This appendix provides a basic overview of the Sun Storage 10GbE FCoE PCIe Converged Network Adapter, which uses QLogic technology. This appendix also describes the various operating systems, storage, and infrastructure configurations that support the converged network adapter, and it lists the adapter’s environmental requirements.
Deploying a Converged Storage Network Using a Cisco FCoE Switch System Interoperability Environmental Requirements Kit Contents Sun Storage 10GbE FCoE PCIe Converged Network Adapter Standard bracket Accessing Documentation document Converged Network Adapter Features and Specifications The Sun Storage 10GbE FCoE PCIe Converged Network Adapter is a standard low-profile, dual-port converged network adapter that is available in two models: SG-(X)PCIEFCOE2-Q-SR—This model ships with two small form
Deploying a Converged Storage Network Using a Cisco FCoE Switch CAUTION!—Do not replace any of the SFP+ connectors. Doing so will void the warranty or serviceability of the converged network adapter. Table 2 lists the features and specifications for this adapter.
Deploying a Converged Storage Network Using a Cisco FCoE Switch FEATURE DESCRIPTION Ethernet and NIC support Standard Ethernet and Enhanced Ethernet support for: IEEE 802.1Q VLAN IEEE 802.1p IEEE 802.3x IEEE 802.1Qbb IEEE 802.
Deploying a Converged Storage Network Using a Cisco FCoE Switch System Interoperability This section provides information about platforms, storage systems, switches, and software that are compatible with the heterogeneous Fibre Channel and Ethernet network design of the converged network adapter.
Deploying a Converged Storage Network Using a Cisco FCoE Switch PLATFORM SUPPORTED OS/TECHNOLOGY Oracle x86 Servers: Oracle Solaris, Linux, VMware, Virtual Machine, and Windows Sun Fire X2200 M2 Sun Fire X2250 Sun Fire X2270 Sun Fire X4140 Sun Fire X4150 Sun Fire X4170 Sun Fire X4240 Sun Fire X4250 Sun Fire X4270 Sun Fire X4275 Sun Fire X4440 Sun Fire X4450 Sun Fire X4640 Storage Support This section lists the arrays, disk systems, and tape storage devices supported by the converged network adapter.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Tape Storage Support The converged network adapter supports connecting to, using a supported switch, the following tape storage devices: Sun StorageTek SL24 tape autoloader Sun StorageTek SL48 tape library Sun StorageTek SL500 modular library Sun StorageTek SL3000 modular library Sun StorageTek SL8500 modular library Sun StorageTek L1400 tape library Sun StorageTek T10000A and T10000B tape drives Sun StorageTek 9840C and 9840D
Deploying a Converged Storage Network Using a Cisco FCoE Switch TABLE 6 OTHER SUPPORTED SOFTWARE APPLICATIONS SOFTWARE SUPPORTED OS Veritas Storage Foundation (VxSF) 5.0 Oracle Solaris Veritas NetBackup 6.5 Oracle Solaris Sun StorageTek Enterprise Backup Software (EBS) Oracle Solaris, Linux, and Windows 7.2/7.3/7.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Appendix D: Data Center Bridging Technology The following descriptions of Enhanced Ethernet were taken from Ethernet: The Converged Network Ethernet Alliance Demonstration, which was presented at the Super Computing 2009 (SC09) conference and published by the Ethernet Alliance in November 2009.
Deploying a Converged Storage Network Using a Cisco FCoE Switch ETS and DCBX simplify the management of DCB nodes significantly, especially when deployed end-to-end in a converged data center. The DCBX protocol uses Link Layer Discovery Protocol (LLDP) defined by IEEE 802.1AB to exchange and discover DCB capabilities. Priority Flow Control A fundamental requirement for a high-performance storage network is guaranteed data delivery.
Deploying a Converged Storage Network Using a Cisco FCoE Switch FIGURE 6 FCOE MAPPING ILLUSTRATION (SOURCE FC-BB-5 REV 2.0) iSCSI The Internet Small Computer Systems Interface (iSCSI) is a SCSI mass storage transport that operates between the Transport Control Protocol (TCP) and the SCSI Protocol Layers. The iSCSI protocol is defined in RFC 3720 [iSCSI], which was finalized by the Internet Engineering Task Force (IETF) in April 2004.
Deploying a Converged Storage Network Using a Cisco FCoE Switch Appendix E: References Unified Data Center Fabric Primer: FCoE and Data Center Bridging, Martin, D. (2010). SearchNetworking.com, retrieved from http://searchnetworking.techtarget.com/tip/0,289483,sid7_gci1378613,00.html Ethernet: The Converged Network Ethernet Alliance Demonstration at SC'09, Ethernet Alliance. (2009), retrieved from http://www.ethernetalliance.
Deploying a Converged Network Using Oracle Copyright © 2011, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the CNAs and Cisco FCoE Switch contents hereof are subject to change without notice.
