User Manual

Rev 2.3-1.0.1
Mellanox Technologies
127
3.2.5.1.1.1 Port Configuration
The physical port MTU in Datagram mode (indicates the port capability) default value is 4k,
whereas the IPoIB port MTU ("logical" MTU ) default value is 2k as it is set by the OpenSM.
To change the IPoIB MTU to 4k, edit the OpenSM partition file in the section of IPoIB setting as
follow:
Default=0xffff, ipoib, mtu=5 : ALL=full;
*Where "mtu=5" indicates that all IPoIB ports in the fabric are using 4k MTU, ("mtu=4" indi-
cates 2k MTU)
3.2.5.1.2 IPoIB Configuration
Unless you have run the installation script mlnxofedinstall with the flag ‘-n’, then IPoIB has
not been configured by the installation. The configuration of IPoIB requires assigning an IP
address and a subnet mask to each HCA port, like any other network adapter card (i.e., you need
to prepare a file called ifcfg-ib<n> for each port). The first port on the first HCA in the host is
called interface ib0, the second port is called ib1, and so on.
An IPoIB configuration can be based on DHCP (Section 3.2.5.1.2.1) or on a static configuration
(Section 3.2.5.1.2.4) that you need to supply. You can also apply a manual configuration that per-
sists only until the next reboot or driver restart (Section 3.2.5.1.2.5).
3.2.5.1.2.1 IPoIB Configuration Based on DHCP
Setting an IPoIB interface configuration based on DHCP is performed similarly to the configura-
tion of Ethernet interfaces. In other words, you need to make sure that IPoIB configuration files
include the following line:
For RedHat:
BOOTPROTO=dhcp
For IPoIB interface configuration on RHEL7 please refer to:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/
Networking_Guide/sec-Configuring_IPoIB.html
For SLES:
BOOTPROTO=’dchp’
If IPoIB configuration files are included, ifcfg-ib<n> files will be installed under:
/etc/sysconfig/network-scripts/ on a RedHat machine
/etc/sysconfig/network/ on a SuSE machine.
A patch for DHCP may be required for supporting IPoIB. For further information,
please see the REAME which is available under the docs/dhcp/ directory.
Standard DHCP fields holding MAC addresses are not large enough to contain an IPoIB hard-
ware address. To overcome this problem, DHCP over InfiniBand messages convey a client iden-
tifier field used to identify the DHCP session. This client identifier field can be used to associate
an IP address with a client identifier value, such that the DHCP server will grant the same IP
address to any client that conveys this client identifier
.