User Manual

OpenSM – Subnet ManagerRev 2.1-1.0.6
Mellanox Technologies
156
1. A valid guid file specifies one guid in each line. Lines with an invalid format will be dis-
carded.
2. The user should specify the root switch guids. However, it is also possible to specify CA
guids; OpenSM will use the guid of the switch (if it exists) that connects the CA to the subnet
as a root node.
8.5.4 Fat-tree Routing Algorithm
The fat-tree algorithm optimizes routing for "shift" communication pattern. It should be chosen if
a subnet is a symmetrical or almost symmetrical fat-tree of various types. It supports not just
K-ary-N-Trees, by handling for non-constant K, cases where not all leafs (CAs) are present, any
Constant Bisectional Ratio (CBB )ratio. As in UPDN, fat-tree also prevents credit-loop-dead-
locks.
If the root guid file is not provided ('
-a' or '--root_guid_file' options), the topology has to
be pure fat-tree that complies with the following rules:
• Tree rank should be between two and eight (inclusively)
• Switches of the same rank should have the same number of UP-going port groups
1
,
unless they are root switches, in which case the shouldn't have UP-going ports at all.
• Switches of the same rank should have the same number of DOWN-going port groups,
unless they are leaf switches.
• Switches of the same rank should have the same number of ports in each UP-going port
group.
• Switches of the same rank should have the same number of ports in each DOWN-going
port group.
• All the CAs have to be at the same tree level (rank).
If the root guid file is provided, the topology does not have to be pure fat-tree, and it should only
comply with the following rules:
• Tree rank should be between two and eight (inclusively)
• All the Compute Nodes
2
have to be at the same tree level (rank). Note that non-compute
node CAs are allowed here to be at different tree ranks.
Topologies that do not comply cause a fallback to min hop routing. Note that this can also
occur on link failures which cause the topology to no longer be a “pure” fat-tree.
Note that although fat-tree algorithm supports trees with non-integer CBB ratio, the routing
will not be as balanced as in case of integer CBB ratio. In addition to this, although the algo-
rithm allows leaf switches to have any number of CAs, the closer the tree is to be fully popu-
lated, the more effective the "shift" communication pattern will be. In general, even if the
root list is provided, the closer the topology to a pure and symmetrical fat-tree, the more optimal
the routing will be.
The algorithm also dumps compute node ordering file (
opensm-ftree-ca-order.dump) in
the same directory where the OpenSM log resides. This ordering file provides the CN order
that may be used to create efficient communication pattern, that will match the routing tables.
1. Ports that are connected to the same remote switch are referenced as ‘port group’
2. List of compute nodes (CNs) can be specified by ‘-u’ or ‘--cn_guid_file’ OpenSM options.