Data Center Architecture with Panduit, Intel, and Cisco

6
AN-02
ToR Architecture
Top-of-Rack is a distributed modular system in which a switch or a fabric extender such as the Cisco Nexus 2232TM resides
at the top of the server rack. Although the EoR and MoR designs are most common today, ToR is becoming popular
because it allows the use of higher density server racks. As the number of cables exiting the rack increases with the
number of servers the cost and space constraints become harder to maintain. When used in ToR architecture the Nexus
2232TM fabric extender aggregates network connections in the rack, reducing the number of intra-enclosure cable runs
back to the Cisco Nexus 5000 Series switches.
Each Nexus 2232TM has (32) 10 GbE downlink ports, allowing it to support up to 32 servers. A common configuration has
twelve cabinets per row, each containing up to sixteen 2U servers, or 32 1U servers each with a dual-port Intel Ethernet
Server Adapter X520-T2. One or two Nexus 2232TM fabric extenders are placed at the top of each cabinet. The second
fabric extender can be used to provide either redundancy or the capability to add more servers per cabinet.
In the above scenario, the Nexus 2232TM downlinks to the Intel Ethernet Server Adapter X520-T2 in the server cabinet via
Panduit
®
TX6A
™
10Gig
™
or TX6A-SD
™
10Gig
™
Patch Cords. The Nexus 2232TM fabric extender acts as a remote line card for
a Cisco Nexus 5000 Series switch located in a switch cabinet either at the end of the row or in the middle of the row.
For redundancy the Nexus 5000 Series Switch are generally located in two different cabinets. The Nexus 2232TM is up-
linked to the Nexus 5000 switch using Panduit OM3 structured fiber or possibly SFP+ DAC, depending on the location and
distance of the switch. Locating the Nexus 5000 in the middle of the row reduces cable lengths to the Nexus 2232TM and
allows the use of SFP+ DAC cabling, which has a maximum reach of 7 meters (23 feet). When fiber is used, the OM3 cable
is factory terminated with duplex LC connectors and is routed in overhead pathways such as the Panduit
®
FiberRunner
®
Cable Routing System.
1
2
1
Cisco Nexus 5548UP
Intel Ethernet Server
Adapter X520-T2
QuickNet
™
OM3
Fiber or SFP+DAC
Panduit
®
QuickNet
™
OM3/OM4 Fiber Uplinks
to Cisco Nexus 7000
Cisco Nexus 2232TM
Panduit Category 6A
patch cords
Cisco Nexus 2232TM fabric extenders located at the top of each server cabinet
Intel Ethernet Server Adapter X520-T2 located in each server
Panduit
®
TX6A
™
or TX6A-SD
™
10Gig
™
Patch Cords connect Intel Ethernet Server Adapter X520-T2 and Nexus 2232TM
Panduit
®
QuickNet
™
OM3 fiber or SFP+ Direct Attach Copper cable assemblies connect Nexus 2232TM to Nexus 5000
Cisco Nexus 5000 Series switches located in switch cabinets at end of the row
ToR Architecture
1
3
2
1
3
2
1
3
2
1
3
2
1 1
3
2
3
2
1
3
2
1
3
2
1
3
2
3
2
4
5
1
3
2
4
5
5
5
5
4 4 4
4
1