User Manual

Rev 2.3-1.0.1
Mellanox Technologies
137
request to open an interface on a specific gateway identifying it by the BridgeX box and eport
name.
Distinguishing between gateways is essential because they determine the network topology and
affect the path that a packet traverses between hosts. A packet that is sent from the host on a spe
-
cific EoIB interface will be routed to the Ethernet subnet through a specific external port connec-
tion on the BridgeX box.
3.2.5.3.1.2 Virtual Hubs (vHubs)
Virtual hubs connect zero or more EoIB interfaces (on internal hosts) and an eport through a vir-
tual hub. Each vHub has a unique virtual LAN (VLAN) ID. Virtual hub participants can send
packets to one another directly without the assistance of the Ethernet subnet (external side) rout
-
ing. This means that two EoIB interfaces on the same vHub will communicate solely using the
InfiniBand fabric. EoIB interfaces residing on two different vHubs (whether on the same gate
-
way or not) cannot communicate directly.
There are two types of vHubs:
• a default vHub (one per gateway) without a VLAN ID
• vHubs with unique different VLAN IDs
Each vHub belongs to a specific gateway (BridgeX® + eport), and each gateway has one default
vHub, and zero or more VLAN-associated vHubs. A specific gateway can have multiple vHubs
distinguishable by their unique VLAN ID. Traffic coming from the Ethernet side on a specific
eport will be routed to the relevant vHub group based on its VLAN tag (or to the default vHub for
that GW if no vLan ID is present).
3.2.5.3.1.3 Virtual NIC (vNic)
A virtual NIC is a network interface instance on the host side which belongs to a single vHub on
a specific GW. The vNic behaves similar to any regular hardware network interface. The host can
have multiple interfaces that belong to the same vHub.
3.2.5.3.2 EoIB Configuration
mlx4_vnic module supports two different modes of configuration which is passed to the host
mlx4_vnic driver using the EoIB protocol:
• host administration where the vNic is configured on the host side
• network administration where the configuration is done by the BridgeX
Both modes of operation require the presence of a BridgeX gateway in order to work properly.
The EoIB driver supports a mixture of host and network administered vNics.
3.2.5.3.2.1 EoIB Host Administered vNic
In the host administered mode, vNics are configured using static configuration files located on
the host side. These configuration files define the number of vNics, and the vHub that each host
administered vNic will belong to (i.e., the vNic's BridgeX box, eport and VLAN id properties).
The mlx4_vnic_confd service is used to read these configuration files and pass the relevant data
to the mlx4_vnic module. EoIB Host Administered vNic supports two forms of configuration
files:
• “Central Configuration File - /etc/infiniband/mlx4_vnic.conf”
• “vNic Specific Configuration Files - ifcfg-ethX”