AOC-CIBF-M1 User's Guide Revision 1.
The information in this User's Guide has been carefully reviewed and is believed to be accurate. The vendor assumes no responsibility for any inaccuracies that may be contained in this document, and makes no commitment to update or to keep current the information in this user's guide, or to notify any person or organization of the updates. Please Note: For the most up-to-date version of this user's guide, please see our Website at www.supermicro.com. Super Micro Computer, Inc.
Preface Preface About this User's Guide This user's guide is written for system integrators, PC technicians and knowledgeable PC users. It provides information for the installation and use of the AOC-CIBF-M1 add-on card. About this Add-on Card The AOC-CIBF-m1 is the most compact, powerful InfiniBand adapter on the market.
AOC-CIBF-M1 Add-on Card User's Guide Conventions Used in the User's Guide Pay special attention to the following symbols for proper system installation and to prevent damage to the system or injury to yourself: Warning: Important information given to ensure proper system installation or to prevent damage to the components or injury to yourself. Note: Additional information given to differentiate between various models or provides information for correct system setup.
Preface Contacting Supermicro Headquarters Address: Super Micro Computer, Inc. 980 Rock Ave. San Jose, CA 95131 U.S.A. Tel: +1 (408) 503-8000 Fax: +1 (408) 503-8008 Email: marketing@supermicro.com (General Information) support@supermicro.com (Technical Support) Website: www.supermicro.com Europe Address: Super Micro Computer B.V. Het Sterrenbeeld 28, 5215 ML 's-Hertogenbosch, The Netherlands Tel: +31 (0) 73-6400390 Fax: +31 (0) 73-6416525 Email: sales@supermicro.
AOC-CIBF-M1 Add-on Card User's Guide Table of Contents Preface Chapter 1 Overview 1-1 Overview.......................................................................................................... 1-1 1-2 Key Features.................................................................................................... 1-1 1-3 Specifications...................................................................................................
Chapter 1: Overview Chapter 1 Overview 1-1 Overview Congratulations on purchasing your add-on card from an acknowledged leader in the industry. Supermicro products are designed with the utmost attention to detail to provide you with the highest standards in quality and performance. For product support and updates, please refer to our website at http://www.supermicro.com/ products/nfo/networking.cfm#adapter.
AOC-CIBF-m1 Add-on Card User's Guide • Connectivity Interoperable with InfiniBand or 10/40GbE switches • Passive copper cable with ESD protection • Powered connectors for optical and active cable support • InfiniBand IBTA Specification 1.2.
Chapter 1: Overview • IEEE Std 802.1Qau Congestion Notification • IEEE P802.1Qaz D0.2 ETS • IEEE P802.1Qbb D1.0 Priority-based Flow Control • Jumbo frame support (9.
AOC-CIBF-m1 Add-on Card User's Guide • Protocol Support Open MPI, OSU MVAPICH, Intel MPE, MS MPI, PLATFORM MPI • TCP,UDP, EoIB, IPoIB, SDP, RDS • SRP, iSER, NFS RDMA • uDAPL • Operating Systems/Distributions Novell SLES, Red Hat Enterprise Linux (RHEL), and other Linux distributions • Microsoft Windows Server 2008/CCS 2003, HPC Server 2008 • OpenFabrics Enterprise Distribution (OFED) • OpenFabrics Windows Distribution (WinOF) • VMware ESX Server 3.5, vSphere 4.0/4.
Chapter 1: Overview • Supported Platforms Supermicro Twin Server Systems with microLP expansion slot (see table below) Notes: (1) = 1U Bracket. (2) = 2U Bracket. This product is only available as an integrated solution with Supermicro server systems. For the most current product information, visit: www. supermicro.com.
AOC-CIBF-m1 Add-on Card User's Guide Notes 1-6
Chapter 2: Hardware Components Chapter 2 Hardware Components 2-1 Add-On Card Image and Layout The AOC-CIBF-m1 Image 1 3 2 The AOC-CIBF-m1 Layout Mellanox ConnectX-3 FDR Controller 2 1 3 2-2 Major Components The following major components are installed on the AOC-CIBF-m1: 1. Mellanox ConnectX-3 FDR Controller 2. QSFP (Quad Small Form Factor Pluggable) Port 3. Dual USB 2.
AOC-CIBF-m1 Add-on Card User's Guide 2-3 Connectors and PCB LEDs USB Ports Two USB 2.0 ports are located on the add-on card. Refer to the add-on card layout on page 2-1 for the the location of the USB ports. QSFP Port A single QSFP port is located on the add-on card. Connect a QSFP cable to the port to provide Infiniband (56Gb/s) and Ethernet (40Gbp/s) communication. Refer to the add-on card layout on Page 2-1 for the location of the QSPF port.
Chapter 3: Installation Chapter 3 Installation 3-1 Static-Sensitive Devices Electrostatic Discharge (ESD) can damage electronic components. To avoid damaging your add-on card, it is important to handle it very carefully. The following measures are generally sufficient to protect your equipment from ESD. • • • • • Precautions Use a grounded wrist strap designed to prevent static discharge. Touch a grounded metal object before removing the add-on card from the antistatic bag.
AOC-CIBF-m1 Add-on Card User's Guide 3-2 Add-On Card Brackets The add-on card ships with a PCB bracket that is pre-installed at the factory. For 1U systems, an I/O bracket is also pre-installed on the card. For 2U systems, the I\O bracket is bundled with the card and must be installed by the customer. See the drawings below for bracket details.
Chapter 3: Installation 3-4 Installing the Add-on Card Follow the steps below to install the add-on card into your system. 1. Remove the server cover and, if necessary, set aside any screws for later use. 2. Remove the add-on card slot cover. If the case requires a screw, place the screw aside for later use. 3. Position the add-on card in the slot directly over the connector, and gently push down on both sides of the card until it slides into the PCI connector. 4. Secure the add-on card to the chassis.
AOC-CIBF-m1 Add-on Card User's Guide 3. Click on INSTALL DRIVERS AND SOFTWARE. 4. Follow the prompts to complete the installation. 3-6 Installing Drivers and Firmware Use the procedures below to install both drivers and firmware for the AOC-CIBF-m1 add-on card for both Linux and Windows. Linux Drivers Use the following procedures for installing AOC-CIBF-m1 drivers for the Linux operating system. Installing InfiniBand Drivers for the Linux Operating System 1.
Chapter 3: Installation 3. Install the driver by entering the following commands: tar xzvf mlnx_en–.tgz cd mlnx_en– ./install.sh This installs the 10G drivers to your system. Windows Drivers Use the following procedures for installing AOC-CIBF-m1 add-on card drivers for the Windows operating system. Installing InfiniBand Drivers for the Windows Operating System 1. From the CDR-NIC LAN driver CD or FTP site, go to the following directory: Mellanox > InfiniBand > Windows. 2.
AOC-CIBF-m1 Add-on Card User's Guide Firmware Update Procedures Use the following procedures for updating the AOC-CIBF-m1 add-on card firmware for both the Linux and Windows operating systems. Updating Firmware for the Linux Operating System 1. From the CDR-NIC LAN driver CD or FTP site, go to the following directory: Mellanox > Firmware_Tool > Linux. 2. Choose the MFT Linux package file and untar the package file. 3. Install the package by entering the following command: cd mft– ./install.sh 4.
(Disclaimer Continued) The products sold by Supermicro are not intended for and will not be used in life support systems, medical equipment, nuclear facilities or systems, aircraft, aircraft devices, aircraft/emergency communication devices or other critical systems whose failure to perform be reasonably expected to result in significant injury or loss of life or catastrophic property damage.
