User Manual

Rev 2.3-1.0.1
Mellanox Technologies
121
Figure 5: I/O Consolidation Over InfiniBand
The basic need is to differentiate the service levels provided to different traffic flows, such that a
policy can be enforced and can control each flow utilization of fabric resources.
The InfiniBand Architecture Specification defines several hardware features and management
interfaces for supporting QoS:
• Up to 15 Virtual Lanes (VL) carry traffic in a non-blocking manner
• Arbitration between traffic of different VLs is performed by a two-priority-level
weighted round robin arbiter
. The arbiter is programmable with a sequence of (VL,
weight) pairs and a maximal number of high priority credits to be processed before low
priority is served
• Packets carry class of service marking in the range 0 to 15 in their header SL field
• Each switch can map the incoming packet by its SL to a particular output VL, based on a
programmable table
VL=SL-to-VL-MAP(in-port, out-port, SL)
• The Subnet Administrator controls the parameters of each communication flow by pro-
viding them as a response to Path Record (PR) or MultiPathRecord (MPR) queries
DiffServ architecture (IETF RFC 2474 & 2475) is widely used in highly dynamic fabrics. The
following subsections provide the functional definition of the various software elements that
enable a Dif
fServ-like architecture over the Mellanox OFED software stack.
3.2.3.1 QoS Architecture
QoS functionality is split between the SM/SA, CMA and the various ULPs. We take the “chro-
nology approach” to describe how the overall system works.
1. The network manager (human) provides a set of rules (policy) that define how the network is
being configured and how its resources are split to dif
ferent QoS-Levels. The policy also
define how to decide which QoS-Level each application or ULP or service use.
2. The SM analyzes the provided policy to see if it is realizable and performs the necessary fab-
ric setup. Part of this policy defines the default QoS-Level of each partition. The SA is
enhanced to match the requested Source, Destination, QoS-Class, Service-ID, PKey against