User Manual

Rev 2.1-1.0.6
Mellanox Technologies
157
8.5.4.1 Routing between non-CN Nodes
The use of the cn_guid_file option allows non-CN nodes to be located on different levels in the
fat tree. In such case, it is not guaranteed that the Fat Tree algorithm will route between two non-
CN nodes. In the scheme below, N1, N2 and N3 are non-CN nodes. Although all the CN have
routes to and from them, there will not necessarily be a route between N1,N2 and N3. Such
routes would require to use at least one of the switches the wrong way around.
Spine1 Spine2 Spine 3
/ \ / | \ / \
/ \ / | \ / \
N1 Switch N2 Switch N3
/|\ /|\
/ | \ / | \
Going down to compute nodes
To solve this problem, a list of non-CN nodes can be specified by \'-G\' or \'--io_guid_file\'
option. These nodes will be allowed to use switches the wrong way around a specific number of
times (specified by \'-H\' or \'--max_reverse_hops\'. With the proper max_reverse_hops and
io_guid_file values, you can ensure full connectivity in the Fat Tree. In the scheme above, with a
max_reverse_hop of 1, routes will be instanciated between N1<->N2 and N2<->N3. With a
max_reverse_hops value of 2, N1,N2 and N3 will all have routes between them.
8.5.4.2 Activation through OpenSM
• Use ‘-R ftree’ option to activate the fat-tree algorithm
8.5.5 LASH Routing Algorithm
LASH is an acronym for LAyered SHortest Path Routing. It is a deterministic shortest path rout-
ing algorithm that enables topology agnostic deadlock-free routing within communication net-
works.
When computing the routing function, LASH analyzes the network topology for the shortest-path
routes between all pairs of sources / destinations and groups these paths into virtual layers in
such a way as to avoid deadlock.
Using max_reverse_hops creates routes that use the switch in a counter-stream way.
This option should never be used to connect nodes with high bandwidth traffic
between them! It should only be used to allow connectivity for HA purposes or similar.
Also having routes the other way around can cause credit loops.
LMC > 0 is not supported by fat-tree routing. If this is specified, the default routing
algorithm is invoked instead.