Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
CONTENTS Preface v Audience v Organization v Conventions v Related Documentation vi Contacting Technical Support vii IBM Support Contact Information vii Before You Call viii Determining the Software Version viii Downloading the Latest Software Release viii Cisco Systems Support Contact Information viii Cisco Technical Support & Documentation Website Submitting a Service Request ix Definitions of Service Request Severity ix IBM Statement of Limited Warranty Z125-4753-08 04/2004 ix xi Part 1 - Ge
Contents EQUADOR xv MEXICO xv PARAGUAY xv PERU xv URUGUAY xvi VENEZUELA xvi NORTH AMERICA xvi CANADA xvi UNITED STATES xvi ASIA PACIFIC xvi AUSTRALIA xvi CAMBODIA AND LAOS xvii CAMBODIA, INDONESIA, AND LAOS xvii HONG KONG S.A.R. OF CHINA AND MACAU S.A.R.
Contents Types of Warranty Service EU Country Telephone List Notices xxvii xxviii xxix Edition Notice Trademarks xxx xxx Important Notes xxxi Product Recycling and Disposal Battery Return Program xxxi xxxi Electronic Emission Notices xxxii Federal Communications Commission (FCC) Statement xxxii Industry Canada Class A emission compliance statement xxxiii Australia and New Zealand Class A statement xxxiii United Kingdom telecommunications safety requirement xxxiii European Union EMC Directive conf
Contents Changing the Default Password 2-3 Configuring the System Clock 2-3 Connecting Beyond BladeCenter 2-5 Connecting InfiniBand Cables 2-5 CHAPTER 3 Specifications 3-1 Electrical Requirements 3-1 Environmental Requirements 3-1 INDEX Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide iv 78-17838-01
Preface Audience The intended audience is the administrator responsible for installing, configuring, and managing Server Switch equipment. This administrator should have experience administering similar networking or storage equipment. Organization This publication is organized as follows: Chapter Title Description Chapter 1 About the Server Switch Module Describes the server switch module and HCA expansion cards, including topology, management, the LEDs, and external InfiniBand ports.
Preface Related Documentation Convention Description Menu1 > Menu2 > Item… Series indicate a pop-up menu sequence to open a form or execute a desired function. [ ] Elements in square brackets are optional. {x|y|z} Alternative keywords are grouped in braces and separated by vertical bars. Braces can also be used to group keywords and/or arguments; for example, {interface interface type}. [x|y|z] Optional alternative keywords are grouped in brackets and separated by vertical bars.
Preface Contacting Technical Support • Cisco Command Line Interface Reference Guide • Cisco Chassis Manager User Guide (Web GUI) • Cisco Element Manager User Guide Contacting Technical Support Please read this section before contacting IBM or Cisco technical support. IBM Support Contact Information For technical support, see the following Web site: http://www.ibm.com/support To contact technical support, send E-mail to support@ibm.com or see one of the telephone numbers below.
Preface Contacting Technical Support Before You Call Be prepared to provide the following information to support personnel.
Preface Contacting Technical Support Cisco Technical Support & Documentation Website The Cisco Technical Support & Documentation website provides online documents and tools for troubleshooting and resolving technical issues with Cisco products and technologies. The website is available 24 hours a day, at this URL: http://www.cisco.com/techsupport Access to all tools on the Cisco Technical Support & Documentation website requires a Cisco.com user ID and password.
Preface Contacting Technical Support Severity 2 (S2)—Operation of an existing network is severely degraded, or significant aspects of your business operation are negatively affected by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation. Severity 3 (S3)—Operational performance of your network is impaired, but most business operations remain functional.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms Part 1 - General Terms This Statement of Limited Warranty includes Part 1 - General Terms, Part 2 - Country-unique Terms, and Part 3 - Warranty Information. The terms of Part 2 replace or modify those of Part 1. The warranties provided by IBM in this Statement of Limited Warranty apply only to Machines you purchase for your use, and not for resale.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms What This Warranty Does Not Cover This warranty does not cover the following: • any software programs, whether pre-loaded or shipped with the Machine, or installed subsequently; • failure resulting from misuse (including but not limited to use of any Machine capacity or capability, other than that authorized by IBM in writing), accident, modification, unsuitable physical or operating environment, or improper maintenance by you
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms Exchange of a Machine or Part When the warranty service involves the exchange of a Machine or part, the item IBM or your reseller replaces becomes its property and the replacement becomes yours. You represent that all removed items are genuine and unaltered. The replacement may not be new, but will be in good working order and at least functionally equivalent to the item replaced.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms 2. the amount of any other actual direct damages, up to the charges (if recurring, 12 months' charges apply) for the Machine that is subject of the claim. For purposes of this item, the term “Machine” includes Machine Code and Licensed Internal Code (“LIC”). This limit also applies to IBM's suppliers and your reseller. It is the maximum for which IBM, its suppliers, and your reseller are collectively responsible.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the courts of the city of La Paz. BRAZIL Jurisdiction: The following is added after the first sentence: Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the court of Rio de Janeiro, RJ.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms URUGUAY Jurisdiction: The following is added after the first sentence: Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the City of Montevideo Court’s Jurisdiction.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms Limitation of Liability: The following is added to this section: Where IBM is in breach of a condition or warranty implied by the Trade Practices Act 1974 or other similar legislation, IBM's liability is limited to the repair or replacement of the goods or the supply of equivalent goods.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms INDIA Limitation of Liability: The following replaces items 1 and 2 of this section: 1. liability for bodily injury (including death) or damage to real property and tangible personal property will be limited to that caused by IBM's negligence; and 2.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms PEOPLE'S REPUBLIC OF CHINA (PRC) Governing Law: The following replaces “laws of the country in which you acquired the Machine” in the first sentence: laws of the State of New York, United States of America (except when local law requires otherwise).
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms If you purchase a Machine in one of the Western European countries, as defined above, you may obtain warranty service for that Machine in any of those countries from either (1) an IBM reseller approved to perform warranty service or (2) from IBM, provided the Machine has been announced and made available by IBM in the country in which you wish to obtain service.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms execution of this Statement of Limited Warranty shall be settled by Arbitration Court of Moscow; 6) in South Africa, Namibia, Lesotho and Swaziland, both of us agree to submit all disputes relating to this Statement of Limited Warranty to the jurisdiction of the High Court in Johannesburg; 7) in Turkey all disputes arising out of or in connection with this Statement of Limited Warranty shall be resolved by the Istanbul Cen
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms CONSUMERS Consumers have legal rights under applicable national legislation governing the sale of consumer goods. Such rights are not affected by the warranties provided in this Statement of Limited Warranty.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms THE FOLLOWING TERMS APPLY TO THE COUNTRY SPECIFIED: AUSTRIA The provisions of this Statement of Limited Warranty replace any applicable statutory warranties. What this Warranty Covers: The following replaces the first sentence of the first paragraph of this section: The warranty for an IBM Machine covers the functionality of the Machine for its normal use and the Machine's conformity to its Specifications.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms GERMANY What this Warranty Covers: The following replaces the first sentence of the first paragraph of this section: The warranty for an IBM Machine covers the functionality of the Machine for its normal use and the Machine's conformity to its Specifications. The following paragraphs are added to this section: The minimum warranty period for Machines is twelve months.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms This section sets out the extent of IBM's liability and your sole remedy. 1. IBM will accept unlimited liability for death or personal injury caused by the negligence of IBM. 2. Subject always to the Items for Which IBM is Not Liable below, IBM will accept unlimited liability for physical damage to your tangible property resulting from the negligence of IBM. 3.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 3 - Warranty Information 3. IBM's entire liability for actual damages for any one Default will not in any event, except as provided in items 1 and 2 above, exceed the greater of 1) Pounds Sterling 75,000, or 2) 125% of the total purchase price payable or the charges for the Machine directly relating to the Default. These limits also apply to IBM's suppliers and resellers.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 3 - Warranty Information Types of Warranty Service If required, IBM provides repair or exchange service depending on the type of warranty service specified for your Machine in the above table and as described below. Warranty service may be provided by your reseller if approved by IBM to perform warranty service. Scheduling of service will depend upon the time of your call and is subject to parts availability.
IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 3 - Warranty Information When a 5, 6 or 7 type of warranty service is listed, IBM will determine which type of warranty service is appropriate for the repair. * This type of service is called ThinkPad® EasyServ or EasyServ in some countries. The IBM Machine Warranty World Wide Web site at http://www.ibm.
Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used.
Notices Edition Notice Edition Notice © Copyright International Business Machines Corporation 2004. All rights reserved. U.S. Government Users Restricted Rights — Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Notices Important Notes Other company, product, or service names may be trademarks or service marks of others. Important Notes Processor speeds indicate the internal clock speed of the microprocessor; other factors also affect application performance. CD-ROM drive speeds list the variable read rate. Actual speeds vary and are often less than the maximum possible.
Notices Electronic Emission Notices In the United States, IBM has established a collection process for reuse, recycling, or proper disposal of used IBM sealed lead acid, nickel cadmium, nickel metal hydride, and battery packs from IBM equipment. For information on proper disposal of these batteries, contact IBM at 1-800-426-4333. Have the IBM part number listed on the battery available prior to your call. In the Netherlands, the following applies.
Notices Electronic Emission Notices Industry Canada Class A emission compliance statement This Class A digital apparatus complies with Canadian ICES-003. Avis de conformité à la réglementation d'Industrie Canada Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada. Australia and New Zealand Class A statement Attention: This is a Class A product. In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures.
Notices Electronic Emission Notices Chinese Class A warning statement Japanese Voluntary Control Council for Interference (VCCI) statement Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxiv 78-17838-01
WEEE Notice Note This mark applies only to countries within the European Union (EU) and Norway. This appliance is labelled in accordance with European Directive 2002/96/EC concerning waste electrical and electronic equipment (WEEE). The Directive determines the framework for the return and recycling of used appliances as applicable throughout the European Union.
WEEE Notice Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxvi 78-17838-01
Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 xxxvii
Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxviii 78-17838-01
Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 xxxix
Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xl 78-17838-01
C H A P T E R 1 About the Server Switch Module This chapter describes the Cisco 4x InfiniBand Switch module for IBM BladeCenter. This chapter contains the following sections: • Introduction, page 1-1 • Topologies, page 1-3 • Module Management, page 1-3 • LEDs, page 1-4 • External InfiniBand Ports, page 1-5 Introduction The Cisco 4x InfiniBand Switch module for IBM BladeCenter (hereafter “Server Switch Module”) adds InfiniBand (IB) switching capability to hosts in your IBM BladeCenter H chassis.
Chapter 1 About the Server Switch Module Introduction Cisco 4x InfiniBand Switch Module for IBM BladeCenter 154589 Figure 1-1 Each Server Switch Module includes 16 4x ports to the backplane and 8 4x ports (in the form of 2 4x connectors and 2 12x connectors) on the front panel. The Server Switch Module provides full non-blocking switching for all 24 ports. On the backplane, 14 of the internal 4x ports provide 10 Gbps connections to the HCA expansion cards on server blades.
Chapter 1 About the Server Switch Module Topologies Topologies With the Server Switch Module and HCA expansion cards, you can create a non-redundant, single-switch topology or a redundant, dual-switch topology. Note There are four I/O module bays in the switch, labelled 7-10. Bay 7 is for the first InfiniBand interface, and bay 9 is for the second InfiniBand interface. Bays 8 and 10 are not supported.
Chapter 1 About the Server Switch Module LEDs LEDs The Server Switch Module provides module status LEDs and port status LEDs. Table 1-1provides more information about the LEDs. Note External port numbers begin at 17 because ports 1 through 16 are internal.
Chapter 1 About the Server Switch Module External InfiniBand Ports Port Status LEDs The port LEDs on the Server Switch Module indicate connection and transmission status (see tables below). Link LEDs indicate connection status. Traffic LEDs indicate transmission status. Table 1-2 IB 4x Port LED Status Indicators LED Label Color Function Green LED Indication Decode Off 4x port has not established logical link. On 4x port is logically connected.
Chapter 1 About the Server Switch Module External InfiniBand Ports Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 1-6 78-17838-01
C H A P T E R 2 Installing and Configuring the Server Switch This chapter describes installing and configuring the Server Switch Module and configuring the password and system clock, Additional configuring includes synchronizing with NTP and extending an InfiniBand fabric This chapter contains the following sections: • Installing the Server Switch Module, page 2-1 • Downloading Operating System Software from Cisco.
Chapter 2 Installing and Configuring the Server Switch Configuring the Server Switch Module Step 5 Click Cisco InfiniBand Switch Module for IBM BladeCenter to obtain the operating system TopspinOS v2.6.0. The software is an ISO image. Step 6 Burn the ISO image onto a CD. The ISO image contains the latest switch operating system software, release notes and other documentation.
Chapter 2 Installing and Configuring the Server Switch Configuring the Server Switch Module Step 9 Open a Telnet session to the Server Switch Module with the IP address that you configured in Step 6. Log in to your Server Switch Module, and then proceed to “Changing the Default Password” to secure your Server Switch Module. Log in with the default user name of USERID and password of PASSW0RD, where the 0 in PASSW0RD is a zero.
Chapter 2 Installing and Configuring the Server Switch Configuring the Server Switch Module Step 2 Set the on-board clock by entering the clock set command with the following variables: • the time of day in hh:mm:ss format • the day in dd format • the month in mm format • the year in yy format The example that follows sets the date to 1:45 am on June 3rd, 2004: Cisco BladeCenterH# clock set 01:45:00 03 06 04 To configure primary, secondary, and tertiary NTP servers to synchronize the Server Swi
Chapter 2 Installing and Configuring the Server Switch Connecting Beyond BladeCenter Connecting Beyond BladeCenter To extend your InfiniBand fabric beyond your BladeCenter cluster, you must configure the priority of the subnet manager on your Server Switch Module, then connect one or more of the external interfaces on your Server Switch Module to an outside InfiniBand fabric. The device with the highest subnet manager priority becomes the master subnet manager in the InfiniBand network.
Chapter 2 Installing and Configuring the Server Switch Connecting Beyond BladeCenter Removing a Pull Connector 154590 Figure 2-1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 2-6 78-17838-01
C H A P T E R 3 Specifications This chapter details the Server Switch Module specifications. Electrical Requirements Table 3-1 Note DC Supply Requirements Area Requirement DC Supply Voltage 12.2 Volts nominal DC Supply Current 6.0 Amps maximum DC Supply Power without External Port Powered Devices 39 watts maximum DC Supply Power with External Port Powered Devices 60 watts maximum External port powered devices are limited to 21 watts total.
Chapter 3 Specifications Environmental Requirements Note Area Requirement Operational Vibration 0.27G at 5Hz to 500Hz for 30 minutes Storage Shock 50G with a velocity change of 3454 mm/sec squared Storage Vibration 0.5G at 2Hz to 200Hz for 15 minutes In the case of non-uniform air flow, the velocity at any point must meet or exceed 150 LFM.
INDEX A L about the server switch module LEDs 1-1 1-4 port status 1-5 switch module status C cable connection 2-5 change the default password clock, setting M 2-3 management interfaces 2-3 configuration, initial 1-3 2-2 configuring the server switch module connect IB cables 1-4 2-2 2-5 connect to Topspin IB networks 2-5 N new static IP configuration D P documentation password, changing 2-3 conventions v ports, internal and external organization v port status LEDs H 1-2
Index T topology dual-switch single-switch 1-3 1-3 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IN-2 78-17838-01
