User Manual

Driver FeaturesRev 2.2-1.0.1

Mellanox Technologies

service. The InfiniBand UD datagrams encapsulates the entire Ethernet L2 datagram and its pay-

load.

To perform this operation the module performs an address translation from Ethernet layer 2

MAC addresses (48 bits long) to InfiniBand layer 2 addresses made of LID/GID and QPN. This

translation is totally invisible to the OS and user. Thus, differentiating EoIB from IPoIB which

exposes a 20 Bytes HW address to the OS. The mlx4_vnic module is designed for Mellanox's

ConnectX® family of HCAs and intended to be used with Mellanox's BridgeX® gateway family.

Having a BridgeX gateway is a requirement for using EoIB. It performs the following operations:

• Enables the layer 2 address translation required by the mlx4_vnic module.

• Enables routing of packets from the InfiniBand fabric to a 1 or 10 GigE Ethernet subnet.

4.3.1 Ethernet over IB Topology

EoIB is designed to work over an InfiniBand fabric and requires the presence of two entities:

• Subnet Manager (SM)

The required subnet manager configuration is not unique to EoIB but rather similar to other Infini-

Band applications and ULPs.

• BridgeX gateway

The BridgeX gateway is at the heart of EoIB. On one side, usually referred to as the "internal" side, it

is connected to the InfiniBand fabric by one or more links. On the other side, usually referred to as the

"external" side, it is connected to the Ethernet subnet by one or more ports. The Ethernet connections

on the BridgeX's external side are called external ports or eports. Every BridgeX that is in use with

EoIB needs to have one or more eports connected.

4.3.1.1 External Ports (eports) and Gateway

The combination of a specific BridgeX box and a specific eport is referred to as a gateway. The

gateway is an entity that is visible to the EoIB host driver and is used in the configuration of the

network interfaces on the host side. For example, in the host administered vNics the user will

request to open an interface on a specific gateway identifying it by the BridgeX box and eport

name.

Distinguishing between gateways is essential because they determine the network topology and

affect the path that a packet traverses between hosts. A packet that is sent from the host on a spe

cific EoIB interface will be routed to the Ethernet subnet through a specific external port connec-

tion on the BridgeX box.

4.3.1.2 Virtual Hubs (vHubs)

Virtual hubs connect zero or more EoIB interfaces (on internal hosts) and an eport through a vir-

tual hub. Each vHub has a unique virtual LAN (VLAN) ID. Virtual hub participants can send

packets to one another directly without the assistance of the Ethernet subnet (external side) rout

ing. This means that two EoIB interfaces on the same vHub will communicate solely using the

InfiniBand fabric. EoIB interfaces residing on two different vHubs (whether on the same gate

way or not) cannot communicate directly.

There are two types of vHubs:

• a default vHub (one per gateway) without a VLAN ID

• vHubs with unique different VLAN IDs