User Manual

Rev 2.2-1.0.1
Mellanox Technologies
103
For example, on a system with dual port HCA, the following two interfaces might be created;
eth4 and eth5.
cat /sys/class/net/eth_ipoib_interfaces
eth4 over IB port: ib0
eth5 over IB port: ib1
These interfaces can be used to configure the network for the guest. For example, if the guest has
a VIF that is connected to the Virtual Bridge
br0, then enslave the eIPoIB interface to br0 by
running:
# brctl addif br0 ethX
In RHEL KVM environment, there are other methods to create/configure your virtual net-
work (e.g. macvtap). For additional information, please refer to the Red Hat User Manual.
The IPoIB daemon (ipoibd) detects the new virtual interface that is attached to the same bridge as
the eIPoIB interface and creates a new IPoIB instances for it in order to send/receive data. As a
result, number of IPoIB interfaces (ibX.Y) are shown as being created/destroyed, and are being
enslaved to the corresponding
ethX interface to serve any active VIF in the system according to
the set configuration, This process is done automatically by the
ipoibd service.
 To see the list of IPoIB interfaces enslaved under eth_ipoib interface.
# cat /sys/class/net/ethX/eth/vifs
For example:
# cat /sys/class/net/eth5/eth/vifs
SLAVE=ib0.1 MAC=9a:c2:1f:d7:3b:63 VLAN=N/A
SLAVE=ib0.2 MAC=52:54:00:60:55:88 VLAN=N/A
SLAVE=ib0.3 MAC=52:54:00:60:55:89 VLAN=N/A
Each ethX interface has at lease one ibX.Y slave to serve the PIF itself. In the VIFs list of ethX
you will notice that ibX.1 is always created to serve applications running from the Hypervisor on
top of the ethX interface directly.
For InfiniBand applications that require native IPoIB interfaces (e.g. CMA), the original IPoIB
interfaces ibX can still be used. For example, CMA and ethX drivers can co-exist and make use
of IPoIB ports; CMA can use ib0, while eth0.ipoib interface will use ibX.Y interfaces.
 To see the list of eIPoIB interfaces.
# cat /sys/class/net/eth_ipoib_interfaces
For example:
# cat /sys/class/net/eth_ipoib_interfaces
eth4 over IB port: ib0
eth5 over IB port: ib1
The example above shows, two eIPoIB interfaces, where eth4 runs traffic over ib0, and eth5 runs
traffic over ib1.