HP StorageWorks Scalable File Share System Installation and Upgrade Guide Version 2.2

Preparing for the installation2–12
2.4.2.1 Gigabit Ethernet interconnect considerations
The NIC 2 port on a server is used to connect the server to the Gigabit Ethernet switch. While performance
or logistics concerns may dictate specific Gigabit Ethernet port assignments, how the ports are assigned is
of no consequence to the HP SFS system. Provided that all server and client ports are on the same physical
or virtual LAN, the HP SFS system will logically behave in the same way.
The Gigabit Ethernet interconnect backbone must have sufficient connectivity to ensure that the full
bandwidth of the Object Storage Servers can be delivered to all networks equally. Examine the following
issues:
• If the servers are connected to a Gigabit Ethernet switch, there must be sufficient interconnectivity
between the server switch and the client nodes.
• Port-to-port bandwidth within the switch must be adequate.
• For certain switch types, the ports must be configured as non-blocking.
Consult the reference manual for your switch type to determine whether this is necessary.
• The ports must be configured at Gigabit Ethernet speeds and not at a slower speed, either by
misconfiguration or by an incorrect autonegotiation. Autonegotiation resulting in slower speeds can
occur if there is a poor quality cable connecting a node to a switch.
The following example discusses a network design issue that may help to explain how Lustre performance
can be impacted by network design.
Example configuration
• HP SFS configuration:
Top level ProCurve switch, with eight Object Storage Servers, and trunks down to compute level
switches
• Client configuration:
Three compute-level 2848 switches, each with 32 client nodes, and four-port trunk group to top level
switch.
The driving consideration is to ensure that all of the available bandwidth from the Object Storage Servers
to the clients can be used. Given that there are eight Object Storage Servers, if 100MB/sec is budgeted for
each Object Storage Server, there is 800MB/sec bandwidth to be made available to the client nodes. In
many test cases, eight or more client nodes are enough to extract full bandwidth from an Object Storage
Server. If the client nodes in question lie on a different switch to the switch the servers are on, the limiting
factor is the inter-switch bandwidth. If you assume that the interlink rates are 100MB/sec also, a four-port
trunk carries only 400MB/sec, and the bandwidth available to clients on the leaf switches is also limited to
this value.
2.4.2.2 Dual Gigabit Ethernet network considerations
The two ports on the Dual Port PCI-X Gigabit Ethernet adapter card can be connected either to the same
switch or to different switches. While performance or logistics concerns may dictate specific Gigabit Ethernet
port assignments, how the ports are assigned is of no consequence to the HP SFS system; provided that the
same port on each server is connected to the same physical or virtual LAN the HP SFS system will logically
behave in the same way
To realize the performance increases that dual Gigabit Ethernet can provide, the network must be set up
correctly, both to provide connectivity and also to make use of all available links to provide maximum
throughput.