User guide

32 Implementing Cisco InfiniBand on IBM BladeCenter
3.1 InfiniBand Network Layer Model
InfiniBand uses a multi-layer architecture (similar to the seven layer OSI model) to transfer
data between nodes, as shown in Figure 3-1. Each layer is responsible for separate tasks in
passing messages.
Figure 3-1 InfiniBand Network Layers
3.1.1 Upper layer protocols
The upper layer protocols are closest to the operating system and dictate the user
applications that can use the InfiniBand host node. They also define how much overhead is
needed for data transfer. We define the more common protocols in 3.2, “InfiniBand protocols”
on page 35.
3.1.2 Transport layer
The transport layer handles transaction data segmentation when sending, and reassembly
when receiving. When sending, the transport layer splits messages into packets of up to 4k
bytes. It encapsulates the 64-bit Globally Unique ID (GUID) of the destination node with the
data of each packet and passes it down to the network layer.
The transport layer is responsible for in-order packet delivery, partitioning, channel
multiplexing, and transport services (reliable connection, reliable datagram, unreliable
connection, unreliable datagram, raw datagram). Based on the maximum transfer unit (MTU)
of the path, the transport layer divides the data into packets of the proper size. The receiver
reassembles the packets based on a base transport header (BTH) that contains the
destination queue pair and packet sequence number. The receiver acknowledges the
packets and the sender receives these acknowledgements and updates the completion
queue with the status of the operation. InfiniBand Architecture offers a significant
improvement for the transport layer: all functions are implemented in hardware.
Transpor
t
Laye
r
Networ
k
Laye
r
Lin
k
Laye
r
Physica
l
Laye
r
Uppe
r
Layers
End Node End Node Switch Router
Subnet Routing (LID)
Flow
Control
Messages (QP)
Transactions
Inter Subnet Routing (IPv6)
Host
Client
IBA
Operation
SAR
Network
Link
Encoding
MAC
Remote
Client
IBA
Operation
SAR
Network
Link
Encoding
MAC
Packet
Relay
MAC
Packet
Relay
Link
MAC