SGI® 10-Gigabit Ethernet Adapter II User’s Guide 007-5476-001
COPYRIGHT © 2008, SGI. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No permission is granted to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole or in part, without the prior written permission of SGI.
Canadian Department of Communications Statement This digital apparatus does not exceed the Class A limits for radio noise emissions from digital apparatus as set out in the Radio Interference Regulations of the Canadian Department of Communications.
Record of Revision Version Description 001 May 2008 Initial release.
Contents Figures . . . . . . . . . . . . . . . . . . . . . . . . . . ix Tables . . . . . . . . . . . . . . . . . . . . . . . . . . xi About This Book . . . . . . . Important Information . . . . . Chapter Descriptions . . . . . . Related Publications . . . . . . SGI Altix Hardware Documentation Conventions . . . . . . . . Product Support . . . . . . . Reader Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Connecting to a Network . . . . . Connecting with a Fiber-Optic Cable Connecting with a Copper Cable . . . . . . . . . . . . . . . . . . . . . . . . . . 10 . 11 . 12 3. Operating the Adapter . . . . . . . . . . . . . Verifying Functionality . . . . . . . . . . . . . Using LEDs to Determine Adapter Functionality . . . . . Verifying That the Adapter Driver Is Loaded . . . . . . Enabling the Adapter . . . . . . . . . . . . .
Figures Figure 2-1 SGI 10-Gigabit Ethernet Adapter II . Connecting the Fiber-Optic Cable . . . . . . . . . . . . . . . . . . 2 . 12 Figure 2-2 Connecting the Copper Cable . . . . . . . . . . 13 Figure 1-1 007-5476-001 . .
Tables Table 1-1 Table 1-2 Table 1-3 Table 1-4 Table 1-5 Table 1-6 Table 3-1 007-5476-001 Two Versions of the SGI 10-Gigabit Ethernet Adapter II . Adapter Features . . . . . . . . . . . . . . . . . . LC-to-LC Optical Cables . . . SGI 4X Infiniband Optical Cables SGI CX4 Copper Cables . . . Configuration Limits . . . . Adapter LED . . . . . . . . . . . . . . . . . 4 . 5 . 6 . 6 . 16 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
About This Book This guide describes the two versions of the SGI 10-Gigabit Ethernet Adapter II, shows you how to connect the adapter to an Ethernet network, and explains how to operate the adapter. You can use the adapter to replace the built-in Ethernet network adapter in your system or use it in addition to your current adapter. This guide is written for users of the SGI 10-Gigabit Ethernet Adapter II.
About This Book Chapter Descriptions This guide contains the following chapters: • Chapter 1, “Features and Capabilities,” summarizes the adapter features, lists the protocols and interfaces with which the adapter is compatible, and gives adapter configuration limits for various systems. • Chapter 2, “Connecting the Adapter to a Network,” shows you how to connect the adapter to your network.
About This Book • SGI Altix 450 System User’s Guide Provides an overview of the Altix 450 system components, and it describes how to set up and operate this system • SGI Altix 4700 User’s Guide This guide provides an overview of the architecture and descriptions of the major components that compose the SGI Altix 4700 family of servers.
About This Book • You can also view man pages by typing man
on a command line. For example, to display the man page for the apropos command, type the following on a command line: man apropos Important system configuration files and commands are documented on man pages. References in the documentation to these pages include the name of the command and the section number in which the command is found.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-17.html>PAGE 17</a></h6><p>About This Book Product Support SGI provides a comprehensive product support and maintenance program for its products: • If you are in North America, contact the Technical Assistance Center at +1 800 800 4SGI or contact your authorized service provider. • If you are outside North America, contact the SGI subsidiary or authorized distributor in your country. Reader Comments If you have comments about the technical accuracy, content, or organization of this document, contact SGI.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-18.html>PAGE 18</a></h6><p></p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-19.html>PAGE 19</a></h6><p>Chapter 1 1. Features and Capabilities Gigabit Ethernet is technology that allows computer systems to communicate at speeds up to 10 gigabits per second (Gbps).</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-20.html>PAGE 20</a></h6><p>1: Features and Capabilities Adapter Features The product features/specifications for the adapter are from the product specification for the 10GbE Storage Accelerator from Chelsio Communications, Inc. See the following webpage: http://www.chelsio.com Figure 1-1 shows the SGI 10-Gigabit Ethernet Adapter II.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-21.html>PAGE 21</a></h6><p>Adapter Features There are two versions of the SGI 10-Gigabit Ethernet Adapter II, as described in Table 1-1. Table 1-1 Two Versions of the SGI 10-Gigabit Ethernet Adapter II Part Number Marketing Code Description 030-2313-001 PCIE-10G-C-RDMA The CX4 version of the adapter 030-2314-001 PCIE-10G-OR-RDMA Standard version Table 1-2 lists the product features of the adapter. Table 1-2 007-5476-001 Adapter Features Feature Description/Specifications Host Interface + PCI Express 1.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-22.html>PAGE 22</a></h6><p>1: Features and Capabilities Table 1-2 Adapter Features (continued) Feature Description/Specifications High Performance RDMA + Ultra-Low latency, line rate bandwidth + IETF RDDP and RDMAC iWARP compliance + APIs: RNIC-PI, kDAPL and OpenFabrics 1.2 Physical and Environmental + Dimensions without bracket: 6.6 in. x 2.5 in. or 16.8 cm x 6.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-23.html>PAGE 23</a></h6><p>Cabling Table 1-3 LC-to-LC Optical Cables (continued) Marketing Code Length X-F44-OPT-30M 30 meters X-F44-OPT-50M 50 meter X-F44-OPT-100M 100 meters X-F44-OPT-300M 300 meters Table 1-4 lists the SGI 4X InfiniBand fiber-optic cables for the PCIE-10G-C-RDMA version of the adapter.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-24.html>PAGE 24</a></h6><p>1: Features and Capabilities Copper Cables Table 1-5 lists the SGI CX4 copper cables for the PCIE-10G-C-RDMA version of the adapter. Table 1-5 SGI CX4 Copper Cables Marketing Code Length IB-CABLE-01M-Z 1 meter IB-CABLE-02M-Z 2 meters IB-CABLE-03M-Z 3 meters IB-CABLE-04M-Z 4 meters IB-CABLE-05M-Z 5 meters IB-CABLE-06M-Z 6 meters IB-CABLE-07M-Z 7 meters IB-CABLE-06M-Z 1 meters IB-CABLE-10M-Z 10 meters Configuration Limits Table 1-6 summarizes the configuration limits for the adapter .</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-25.html>PAGE 25</a></h6><p>Configuration Limits Table 1-6 Configuration Limits (continued) System Maximum Per Bus Maximum Number of Adapters Altix XE310 1 1 Altix XE320 1 1 Altix 450 1 16 Altix ICE 8200 1 4 Altix 4700 1 16 Note: You may see a performance degradation if you configure two adapters to the same 12E or 14E blade on Altix 450 or Altix 4700 servers.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-26.html>PAGE 26</a></h6><p></p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-27.html>PAGE 27</a></h6><p>Chapter 2 2.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-28.html>PAGE 28</a></h6><p>2: Connecting the Adapter to a Network Procedure 2 1. Remove one or two adjacent DIMMs. 2. Remove the chassis-mounted bracket latch of the adapter. The bracket latch is the small assembly attached at the top of the slot with two screws. Procedure 3 (Most Complex) Note: This procedure is required for the PCIE-10G-C-RDMA version (part number 030-2313-001) of the adapter. 1. Remove one or two adjacent DIMMs. 2. Remove the short PCIe riser assembly from the chassis (one screw). 3.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-29.html>PAGE 29</a></h6><p>Connecting to a Network Connecting with a Fiber-Optic Cable To connect your adapter to a network with a fiber-optic cable, do the following: 1. Remove the protective end caps and save them. Caution: 10-Gigabit optics are sensitive. If you plan to leave them disconnected for any significant length of time, you must replace the end caps. The optics on the adapter cannot be cleaned. 2. Insert the LC connector on one end of the fiber-optic cable into the adapter, as shown in Figure 2-1.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-30.html>PAGE 30</a></h6><p>2: Connecting the Adapter to a Network Fiber-optic connector Figure 2-1 Connecting the Fiber-Optic Cable Connecting with a Copper Cable To connect your adapter to a network with a copper cable, do the following: 1. Insert the connector on one end of the copper cable into the adapter, as shown in Figure 2-2. Ensure that the connector is inserted completely into the jack. 2.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-31.html>PAGE 31</a></h6><p>Connecting to a Network Copper connector Figure 2-2 007-5476-001 Connecting the Copper Cable 13</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-32.html>PAGE 32</a></h6><p>2: Connecting the Adapter to a Network 14 007-5476-001</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-33.html>PAGE 33</a></h6><p>Chapter 3 3. Operating the Adapter This chapter describes several operational tasks as well as troubleshooting.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-34.html>PAGE 34</a></h6><p>3: Operating the Adapter Using LEDs to Determine Adapter Functionality The adapter has a single light-emitting diode (LED) display that indicates if the adapter is configured correctly and connected to an active Ethernet. Table 3-1 describes the LED. Table 3-1 Adapter LED Label Color Meaning LINK/ACT Green Link established Yellow Link active Red Power-on indicator (ON by default during power-on but OFF after the driver is loaded.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-35.html>PAGE 35</a></h6><p>Resetting the Adapter Verifying That the Adapter Is Properly Configured and Enabled Use the installation tool that comes with your operating system to install and configure the adapter. You can use the ifconfig -a command to verify the configuration information, as follows: systemA:~ # ifconfig -a eth2 Link encap:Ethernet HWaddr 00:60:DD:47:81:24 inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-36.html>PAGE 36</a></h6><p>3: Operating the Adapter Changing the Configuration Configuration changes for Ethernet devices drivers are made by means of the ethtool(8) command. The ethtool command works with all Linux Ethernet drivers. In general, each feature has a query and a modify variant.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-37.html>PAGE 37</a></h6><p>Chapter 4 4.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-38.html>PAGE 38</a></h6><p>4: Performance Tuning Load Balancing the Receive Activity Among CPUs The adapter can be configured to use a maximum of eight receive queues. The adapter will be spread across all configured Rx queues. Multiple Rx queues are available when the driver is loaded in MSI-X mode (default mode). In this mode, once the driver is loaded, the number of queues must be specified before the port is configured.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-39.html>PAGE 39</a></h6><p>NIC Acceleration Features The following entry dumps the current settings: cxgbtool intf qset The following entry updates the holdoff timer value for queue X to Y usecs: cxgbtool intf qset X lat Y NIC Acceleration Features The adapter supports all widespread acceleration features: • Checksum offload • TCP segmentation offload (TSO) • Large receive offload (LRO) Checksum offload and TSO are controlled through ethtool(8).</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-40.html>PAGE 40</a></h6><p>4: Performance Tuning Further Tuning with a Sample Script Chelsio Communications, Inc. provides a sample script, perftune.sh.txt, to use for further tuning. You can access the script by contacting Chelsio Communications, Inc.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-41.html>PAGE 41</a></h6><p>Glossary acknowledge (Ack) packet The Ack packet informs the PE that initiated a message that the destination PE accepted the message. autonegotiation The process by which two computers (or a computer and a switch) connected by Gigabit Ethernet determine the speed and other parameters with which they will communicate. CD-ROM (CD) A flat metallic disk that contains information that you can view and copy onto your own hard disk; you cannot change or add to the disk.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-42.html>PAGE 42</a></h6><p>Glossary MAC Medium access control, also called the physical layer. MAC address The physical address of the Gigabit Ethernet board, which is distinct from the IP address. MTU Maximum Transmission Unit is a configuration parameter that controls the size of the Ethernet frames that the Gigabit Ethernet board can transmit and receive. man (manual) page An online document that describes how to use a particular Linux command. Also called reference page.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-43.html>PAGE 43</a></h6><p>Index C F cabling copper, 6, 12 fiber-optic, 4, 5, 11 checksum offload, 21 configuration changes, 18 configuration file /etc/modprobe.conf.</p></li><li><h6><a class="text-decoration-none text-link fw-bold" href=/manual/sgi/altix-xe210/user-s-guide-english/page-44.html>PAGE 44</a></h6><p>Index LC connector, 11 LEDs troubleshooting, 16 load balancing, receive queues, 20 LRO, large receive offload, 21 O optical cables, 4, 5, 11 P PCI Express, 1, 3, 24 PCIe, 1, 3, 24 performance tuning, 19 perftune.sh.</p></li></ul></nav></div></section></section></main><footer class=page-footer><div class="container flex-column py-3 py-sm-4"><div class="row mb-1 mb-sm-2 mb-md-5"><a href=/ class="brand offset-sm-0 col-sm-4 col-md-3 col-lg-2 offset-2 col-8 text-center text-md-start"><img class=logo-small src=/assets/img/brand-large.png alt=Manualshelf></a></div><div class=row><dl class="text-center text-sm-start col-sm-3"><dt class=fw-bold>Who We Are</dt><dd><a class=text-decoration-none href=/about>About Us</a></dd><dd><a class=text-decoration-none href=/company>Company</a></dd><dd><a class=text-decoration-none href=/careers>Careers</a></dd><dd><a class=text-decoration-none href=/terms>Terms & Privacy</a></dd></dl><dl class="text-center text-sm-start col-sm-3"><dt class=fw-bold>Resources</dt><dd><a class=text-decoration-none href=/brands>List of Manufacturers</a></dd><dd><a href=/ class=text-decoration-none>Support</a></dd><dd><a class=text-decoration-none href=/press>For the Press</a></dd><dd><a class=text-decoration-none href=/press#assets>Media assets</a></dd><dd><a class=text-decoration-none href=/faq>FAQ</a></dd></dl><dl class="text-center text-sm-start col-sm-3"><dt class=fw-bold>Get in touch</dt><dd><a class=text-decoration-none href=#>Blog</a></dd><dd><a class=text-decoration-none href=#>Email</a></dd><dd><a class=text-decoration-none href=/dmca>DMCA</a></dd></dl></div><div class=row><div class="text-center text-sm-start col-sm-4 pe-sm-0"><a href=https://www.facebook.com/pages/ManualShelf/1488826334681423 class="fs-5 text-center d-inline-block social rounded-2 align-middle text-decoration-none me-2"><i class="fab fa-facebook-f"></i></a><a href=https://www.twitter.com/ManualShelf class="fs-5 text-center d-inline-block social rounded-2 align-middle text-decoration-none me-2"><i class="fab fa-twitter"></i></a><a href=https://plus.google.com/+ManualShelf class="fs-5 text-center d-inline-block social rounded-2 align-middle text-decoration-none me-2"><i class="fab fa-google-plus-g"></i></a><a href=http://www.pinterest.com/ManualShelf class="fs-5 text-center d-inline-block social rounded-2 align-middle text-decoration-none"><i class="fab fa-pinterest-p"></i></a></div><div class="col col-sm-8 d-flex flex-column align-items-center flex-sm-row flex-wrap flex-sm-nowrap justify-content-center justify-content-sm-end"><div class="copy w-auto d-flex align-items-center justify-content-center justify-content-sm-end mt-2 mt-sm-0 me-2">ManualShelf © 2013-2025</div><select class="form-select form-select-sm w-auto mt-2 mt-sm-0"><option value=usa>USA</option></select><div></div></div><div id=pixel><script>
googletag.cmd.push(function() { googletag.display('pixel'); });
</script></div></footer><script src=/assets/js/jquery-3.3.1.min.js></script><script src=/assets/js/bootstrap.bundle.min.js></script><script src=/assets/js/jquery.autocomplete.min.js></script><script type=text/javascript>
$.extend({
redirectPost: function(location, args) {
var form = '';
$.each(args, function(key, value) {
form += '<input type="hidden" name="' + key + '" value="' + value + '">';
});
form = '<form id="s1" action="' + location + '" method="POST">' + form + '</form>';
$(document.body).append(form);
$("#s1").submit();
}
});
$(document).ready(function(){
$("#query").autocomplete({
minChars: 3,
groupBy: 'type',
paramName: 'q',
dataType: 'json',
serviceUrl: '/autocomplete',
noCache: true,
showNoSuggestionNotice: true,
noSuggestionNotice: 'Zarro results found',
onSelect: function (suggestion) {
$.redirectPost(suggestion.data.url, {highlight: encodeURIComponent(JSON.stringify(suggestion.data.hls))});
}
});
var toc = $('.toc-selector');
if (toc.length) {
var slideParams = { duration: 800, easing: 'linear' };
toc.click(function() {
$(this).toggleClass('open');
var section = $('.toc-content');
var isCollapsed = section.attr('data-collapsed') === 'true';
if (isCollapsed) {
section.slideDown(slideParams)
section.attr('data-collapsed', 'false')
} else {
section.slideUp(slideParams);
section.attr('data-collapsed', 'true')
}
});
}
});
var lastSmall = $(window).width() < 576;
$(window).resize(function() {
var newSmall = $(window).width() < 576;
if (lastSmall != newSmall) {
lastSmall = newSmall;
window.location.href = window.location.href;
}
});
</script></body></html>
