User guide

3–InfiniBand
®
Cluster Setup and Administration
QLogic Distributed Subnet Administration
3-14 IB0054606-02 A
If you are using the QLogic Fabric Manager in its default configuration, and you
are using the standard QLogic PSM SIDs, this arrangement will work fine and you
will not need to modify the Distributed SA's configuration file - but notice that the
Distributed SA has restricted the range of SIDs it cares about to those that were
defined in its configuration file. Attempts to get path records using other SIDs will
not work, even if those other SIDs are valid for the fabric. When using this default
configuration it is necessary that MPI applications only be run using one of these
32 SIDs.
Multiple Virtual Fabrics Example
A person configuring the physical IB fabric may want to limit how much IB
bandwidth MPI applications are permitted to consume. In that case, they may
re-configure the QLogic Fabric Manager, turning off the “Default” Virtual Fabric
and replacing it with several other Virtual Fabrics.
In Figure 3-3, the administrator has divided the physical fabric into four virtual
fabrics: “Admin” (used to communicate with the Fabric Manager), “Storage” (used
by SRP), “PSM_MPI” (used by regular MPI jobs) and a special “Reserved” fabric
for special high-priority jobs.
Figure 3-3. Distributed SA Multiple Virtual Fabrics Example
Due to the fact that the Distributed SA was not configured to include the SID
Range 0x10 through 0x1f, it has simply ignored the “Reserved” VF. Adding those
SIDs to the qlogic_sa.conf file solves the problem as shown in Figure 3-4.
9LUWXDO)DEULF³$GPLQ´
3NH\[III
,QILQLEDQG)DEULF
9LUWXDO)DEULF
³360B03,´
3NH\[
6,'5DQJH[[I
6,'5DQJH
[
[I
9LUWXDO)DEULF³6WRUDJH´
3NH\[
6,'
[
9LUWXDO)DEULF
³5HVHUYHG´
3NH\[
6,'5DQJH[[I
'LVWULEXWHG6$
9LUWXDO)DEULF³360B03,´
3NH\[
6,'5DQJH[[I
6,'5DQJH[
[I