User Manual

Table Of Contents

Rev 1.4.1-2.0.000

Mellanox Techologies

Ethernet VLANs are implemented on InfiniBand using Partition Keys (See RFC 4392 for informa-

tion). Thus, the InfiniBand network must be configured first. This can be done by configuring the

Subnet Manager (SM) on your subnet. Note that this configuration is needed for both VLAN config-

uration modes, VGT and VST.

For further information on the InfiniBand Partition Keys configuration for IPoIB, see the Subnet

Manager manual installed in your subnet.

The maximum number of Partition Keys available on Mellanox HCAs is:

 64 for the InfiniHost™ III family

 128 for ConnectX™ IB family

 Check with IB switch documentation for the number of supported partition keys

Guest Operating System Configuration

For VGT mode, VLANs need to be configured in the installed

guest operating system. This proce-

dure may vary for different operating systems. See your guest operating system manual on VLAN

configuration.

In addition, for each new interface created within the virtual machine, at least one packet should be

transmitted. For example:

Create a new interface (e.g., <eth1>) with IP address <ip1>.

To guarantee that a packet is transmitted from the new interface, run:

arping -I <eth1> <ip1> -c 1

y Virtual Switch Configuration

For VST mode, the virtual switch using an InfiniBand

uplink needs to be configured. For further

information, see the ESX Server 3 Configuration Guide and ESX Server 3 802.1Q VLAN Solutions

documents.

4.2.5 Maximum Transmit Unit (MTU) Configuration

On VMware ESX Server machines, the MTU is set to 1500 bytes by default. IPoIB supports larger

values and allows Jumbo Frames (JF) traffic up to 4052 bytes on VI3 and 4092 bytes on vSphere

4. The maximum value of JF supported by the InfiniBand device is:

y 2044 bytes for the InfiniHost III family

y 4052 / 4092 bytes for ConnectX IB family (vSphere 4)

It is the administrator's responsibility to make

sure that all the machines in the network support and

work with the same MTU value. For operating systems other than VMware ESX Server, the

default value is set to 2044 bytes.

The procedure for changing the MTU may vary, depending

on the OS. For example, to change it to

1500 bytes:

y On Linux - if the IPoIB interface is named ib0, run:

ifconfig ib0 mtu 1500

y On Microsoft® Windows - execute the following steps:

Mellanox Technologies Confidential