User Manual

Mellanox OFED OverviewRev 2.2-1.0.1

Mellanox Technologies

• GID format can be of 2 types, IPv4 and IPv6. IPv4 GID is a IPv4-mapped IPv6 address

while IPv6 GID is the IPv6 address itself

• VLAN tagged Ethernet frames carry a 3-bit priority field. The value of this field is

derived from the IB SL field by taking the 3 least significant bits of the SL field

• RoCE traffic is not shown in the associated Ethernet device's counters since it is

floaded by the hardware and does not go through Ethernet network driver. RoCE traf-

fic is counted in the same place where InfiniBand traffic is counted; /sys/class/infini-

band/<device>/ports/<port number>/counters/

1.5.1 Configuring DAPL over RoCE

The default dat.conf file which contains entries for the DAPL devices, does not contain entries

for the DAPL over RDMA_CM over RoCE devices.

 To add the missing entries:

Step 1. Run the ibdev2netdev utility to see all the associations between the Ethernet devices and the

IB devices/ports.

Step 2. Add a new entry line according to the format below to the dat.conf file for each output line

of the ibdev2netdev utility.

<IA Name> u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "<ethX> <port>" ""

Parameter Description Example

<IA Name> The device's IA name. The name must be unique. ofa-v2-ethx

<ethX> The associated Ethernet device used by RoCE. eth3

<port> The port number. 1

The following is an example of the ibdev2netdev utility's output and the entries added per

each output line:

sw419:~ # ibdev2netdev mlx4_0 port 2 <===> eth2 mlx4_0 port 1 <===> eth3

ofa-v2-eth2 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 2" ""

ofa-v2-eth3 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth3 1" ""

1. For the IPv4 address A.B.C.D the corresponding IPv4-mapped IPv6 address is ::ffff.A.B.C.D