User Manual

Rev 2.3-1.0.1

Mellanox Technologies

tion, if none of the above spanning tree branches crosses a dateline used for unicast credit loop

avoidance on a torus, and if multicast traffic is confined to SL 0 or SL 8 (recall that torus-2QoS

uses SL bit 3 to differentiate QoS level), then multicast traffic also cannot contribute to the "ring"

credit loops that are otherwise possible in a torus. Torus-2QoS uses these ideas to create a master

spanning tree. Every multicast group spanning tree will be constructed as a subset of the master

tree, with the same root as the master tree. Such multicast group spanning trees will in general

not be optimal for groups which are a subset of the full fabric. However, this compromise must

be made to enable support for two QoS levels on a torus while preventing credit loops. In the

presence of link or switch failures that result in a fabric for which torus-2QoS can generate

credit-loop-free unicast routes, it is also possible to generate a master spanning tree for multicast

that retains the required properties. For example, consider that same 2D 6x5 torus, with the link

from (2,2) to (3,2) failed. Torus-2QoS will generate the following master spanning tree:

Two things are notable about this master spanning tree. First, assuming the x dateline was

between x=5 and x=0, this spanning tree has a branch that crosses the dateline. However, just as

for unicast, crossing a dateline on a 1D ring (here, the ring for y=2) that is broken by a failure

cannot contribute to a torus credit loop. Second, this spanning tree is no longer optimal even for

multicast groups that encompass the entire fabric. That, unfortunately, is a compromise that must

be made to retain the other desirable properties of torus-2QoS routing. In the event that a single

switch fails, torus-2QoS will generate a master spanning tree that has no "extra" turns by appro-

priately selecting a root switch. In the 2D 6x5 torus example, assume now that the switch at

(3,2), i.e. the root for a pristine fabric, fails.

Torus-2QoS will generate the following master

spanning tree for that case: