User Manual

Rev 2.1-1.0.6

Mellanox Technologies

4.8.2 Configuring the Ethernet Tunneling Over IPoIB Driver

When eth_ipoib is loaded, number of eIPoIB interfaces are created, with the following default

naming scheme:

ethX, where X represents the ETH port available on the system.

To check which eIPoIB interfaces were created:

For example, on a system with dual port HCA, the following two interfaces might be created;

eth4 and eth5.

These interfaces can be used to configure the network for the guest. For example, if the guest has

a VIF that is connected to the Virtual Bridge

br0, then enslave the eIPoIB interface to br0 by

running:

The IPoIB daemon (ipoibd) detects the new virtual interface that is attached to the same bridge as

the eIPoIB interface and creates a new IPoIB instances for it in order to send/receive data. As a

result, number of IPoIB interfaces (ibX.Y) are shown as being created/destroyed, and are being

enslaved to the corresponding

ethX interface to serve any active VIF in the system according to

the set configuration, This process is done automatically by the

ipoibd service.

 To see the list of IPoIB interfaces enslaved under eth_ipoib interface.

For example:

Each ethX interface has at lease one ibX.Y slave to serve the PIF itself. In the VIFs list of ethX

you will notice that ibX.1 is always created to serve applications running from the Hypervisor on

top of the ethX interface directly.

For InfiniBand applications that require native IPoIB interfaces (e.g. CMA), the original IPoIB

interfaces ibX can still be used. For example, CMA and ethX drivers can co-exist and make use

of IPoIB ports; CMA can use ib0, while eth0.ipoib interface will use ibX.Y interfaces.

 To see the list of eIPoIB interfaces.

For example:

cat /sys/class/net/eth_ipoib_interfaces

eth4 over IB port: ib0

eth5 over IB port: ib1

# brctl addif br0 ethX

In RHEL KVM environment, there are other methods to create/configure your virtual net-

work (e.g. macvtap). For additional information, please refer to the Red Hat User Manual.

# cat /sys/class/net/ethX/eth/vifs

# cat /sys/class/net/eth5/eth/vifs

SLAVE=ib0.1 MAC=9a:c2:1f:d7:3b:63 VLAN=N/A

SLAVE=ib0.2 MAC=52:54:00:60:55:88 VLAN=N/A

SLAVE=ib0.3 MAC=52:54:00:60:55:89 VLAN=N/A

# cat /sys/class/net/eth_ipoib_interfaces

eth4 over IB port: ib0

eth5 over IB port: ib1