User Manual

Rev 2.3-1.0.1

Mellanox Technologies

3.1.6.2.6 Associating InfiniBand Ports to Ethernet Ports

Since both RoCE and mlx4_en use the Ethernet port of the adapter, one of the drivers must con-

trol the port state. In the example above, the mlx4_en driver controls the port’s state. The

mlx4_ib driver holds a reference to the mlx4_en net device for getting notifications about the

state of the port, as well as using the mlx4_en driver to resolve IP addresses to MAC that are

required for address vector creation. However

, RoCE traffic does not go through the mlx4_en

driver, it is completely offloaded by the hardware.

# ibdev2netdev

mlx4_0 port 2 <===> eth2

mlx4_0 port 1 <===> ib0

3.1.6.2.7 Configuring an IP Address to the mlx4_en Interface

 To configure an IP address to the mlx4_en interface:

Step 1. Configure an IP address to the mlx4_en interface on both sides of the link.

# ifconfig eth2 20.4.3.220

# ifconfig eth2

eth2 Link encap:Ethernet HWaddr 00:02:C9:08:E8:11

inet addr:20.4.3.220 Bcast:20.255.255.255 Mask:255.0.0.0

UP BROADCAST MULTICAST MTU:1500 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

The fw_ver parameter shows that the firm-

ware version is 2.31.5050.

The firmware version can also be obtained by run-

ning the following commands:

# cat /sys/class/infiniband/mlx4_0/fw_ver

2.31.5050

Although the InfiniBand over Ethernet's

Port MTU is 2K byte at maximum, the

actual MTU cannot exceed the mlx4_en

interface's MTU. Since the mlx4_en inter-

face's MTU is typically 1560, port 2 will

run with MTU of 1K.

Please note that RoCE's MTU are subject

to InfiniBand MTU restrictions.

The

RoCE's MTU values are, 256 byte, 512

byte, 1024 byte and 2K. In general RoCE

MTU is the largest power of 2 that is still

lower than mlx4_en interface MTU.