HP StorageWorks Scalable File Share System Installation and Upgrade Guide Version 2.2

Interconnect topology 1–17
1.5.1.1 Voltaire InfiniBand interconnects
In the general case where no special configuration is applied, all nodes connected to an InfiniBand
interconnect can communicate with each other. At least one InfiniBand IP address (IPoIB address) must be
configured on each server and on each client node. HP SFS and Lustre use the IPoIB address to form the
Network ID (NID) and to establish connections with other nodes. The IPoIB addresses are used only to
establish connections between the client nodes and the servers; once the communication is established using
the ARP protocol, the file data traffic uses the dedicated InfiniBand native protocol, and is not layered on
top of the IPoIB connection.
Voltaire InfiniBand partitioning
Partitioning is an optional configuration that enforces isolation among systems that share an InfiniBand
interconnect. One partition key (Pkey) is assigned to each partition. Pkeys are configured on the InfiniBand
interconnect switch. Up to 16 keys (Pkey0 through Pkey15) can be configured per port. A collection of nodes
with the same Pkey are referred to as being members of a partition. At the host (server or client) side, you
can configure as many IP interfaces as there are Pkeys configured for the associated port.
When partitioning is used, HP SFS can serve multiple client clusters that are isolated from each other. There
are several ways to configure such environments; see Section 2.4.3 for more information.
1.5.1.2 Dual or bonded Gigabit Ethernet interconnects
An HP SFS system can be configured with two Gigabit Ethernet connections to achieve extra bandwidth from
the servers. There are two ways of configuring a system with two Gigabit Ethernet connections: dual links
and bonded links.
Dual and bonded Gigabit Ethernet connections on HP SFS servers always use two ports on a dual Gigabit
Ethernet adapter card.
Dual Gigabit Ethernet links are appropriate in systems where:
• There are client populations in two disconnected subnets or networks, but the HP SFS system can
configure a link on each subnet or network
• It has been decided not to configure bonded links at the HP SFS server or the network switch (or for
network infrastructure reasons, it is not possible to configure such links), but the client population is to
be partitioned into separate subnets.
Bonded links are appropriate where:
• All client nodes can access the subnet of the bonded network.
• The physical infrastructure allows bonded links.
• The extra availability provided by a bonded link at the server is required. (With bonded links, the
logical link continues to function even if there is a failure in one component.)
Figure 1-13 shows a bonded network configuration.