Specifications

42
High Performance Trading/Algo Speed with Wombat Design and Implementation Guide
OL-15617-01
Appendix B—Building and Configuring Switches
Appendix B—Building and Configuring Switches
Definitions
The Basics
When you begin to design and implement a Cisco high-performance computing cluster over InfiniBand
with Cisco Server Fabric Switches, it helps to understand the conventional wisdom in this section.
• Contact Cisco’s partner, Infrastructure Development Corporation (IDC), to prepare an installation
plan. IDC has experience with this process and it will make your job much easier. This plan will
include physical factors, such as air flow and cooling. Contact information is as follows:
– [e-mail]
– [e-mail]
Ta b l e 10 Definition of Key Terms
Term Description
Blocking Blocking topologies do not provide a 1:1 ratio of paths in and paths out. In a blocking topology,
traffic may potentially contend for paths.
Non-blocking Non-blocking topologies provide, for each path into a switch or network, an equal path out.
Non-blocking topologies avoid oversubscription.
Cluster Node Number A number that uniquely identifies every node in a cluster. For example, a 500-node cluster
would have Cluster Node Numbers from 1 to 500.
Core Switch Core switches form the second tier of an InfiniBand fabric and are used to interconnect Leaf
Switches and Edge Switches. They form the backbone of the InfiniBand fabric. A typical Core
Switch would be a 96-port Cisco SFS 7008.
Edge Switch Edge switches provide support for InfiniBand I/O Gateways and are used to connect an
InfiniBand fabric to external network and storage subsystems. A typical Edge Switch would be
a 24-port Cisco SFS 3504 with support for up to 12 Gigabit Ethernet or Fibre Channel Gateways.
Host Channel Adapter
(HCA)
An InfiniBand Host Channel Adapter (HCA) connects a server to the InfiniBand fabric. HCAs
are either PCI-X or PCI-Express based. They are typically dual ported to allow for redundant
InfiniBand connections, though single-port HCAs are available. Each port runs at 10 Gb/s, 20
Gb/s for double data rate (DDR).
Leaf Switch Leaf switches form the first tier of an InfiniBand Fabric and are used to connect hosts to the
Fabric. A typical Leaf Switch would be a 24-port Cisco SFS 7000.
Management Network Also referred to as “Administration Network”. A 100 Base T or Gigabit Ethernet network used
for out-of-band administration and management of the nodes and switches.
Pod A leaf switch and all attached hosts.
Host A single compute element of the cluster, for example a 1U server with one or more CPUs.
Rack Node Number A number that uniquely identifies a node within a rack or frame (the terms “rack” and “frame”
are frequently used interchangeably, but both refer to the structure that supports the nodes and
switches in the cluster). Typically a Rack Node Number will range from 1 (lowest slot in a rack)
to 42 (for a 42 RU Rack).