User Manual

Driver FeaturesRev 2.2-1.0.1
Mellanox Technologies
100
4.8.3 Important Notes
• VXLAN tunneling adds 50 bytes (14-eth + 20-ip + 8-udp + 8-vxlan) to the VM Ethernet
frame. Please verify that either the MTU of the NIC who sends the packets, e.g. the VM
virtio-net NIC or the host side veth device or the uplink takes into account the tunneling
overhead. Meaning, the MTU of the sending NIC has to be decremented by 50 bytes
(e.g 1450 instead of 1500), or the uplink NIC MTU has to be incremented by 50 bytes
(e.g 1550 instead of 1500)
• From upstream 3.15-rc1 and onward, it is possible to use arbitrary UDP port for
VXLAN. Note that this requires firmware version 2.31.2800 or higher
. Additionally,
you need to enable this kernel configuration option
CONFIG_MLX4_EN_VXLAN=y.
• On upstream kernels 3.12/3.13 GRO with VXLAN is not supported
4.9 Atomic Operations
4.9.1 Atomic Operations in mlx5 Driver
Atomic Operations in Connect-IB® (mlx5 driver) are fully supported on big-endian machines
(e.g. PPC). Their support is limited on little-endian machines (e.g. x86)
When using
ibv_exp_query_device on little-endian machines with Connect-IB® the
attr.exp_atomic_cap is set to IBV_EXP_ATOMIC_HCA_REPLY_BE which indicates that if enabled,
the atomic operation replied value is big-endian and contradicts the host endianness.
To enable atomic operation with this endianness contradiction use the
ibv_exp_create_qp to
create the QP and set the
IBV_EXP_QP_CREATE_ATOMIC_BE_REPLY flag on exp_create_flags.
4.9.2 Enhanced Atomic Operations
ConnectX® implements a set of Extended Atomic Operations beyond those defined by the IB
spec. Atomicity guarantees, Atomic Ack generation, ordering rules and error behavior for this set
of extended Atomic operations is the same as that for IB standard Atomic operations (as defined
in section 9.4.5 of the IB spec).
4.9.2.1 Masked Compare and Swap (MskCmpSwap)
The MskCmpSwap atomic operation is an extension to the CmpSwap operation defined in the IB
spec. MskCmpSwap allows the user to select a portion of the 64 bit tar
get data for the "compare"
check as well as to restrict the swap to a (possibly different) portion. The pseudocode below
describes the operation:
| atomic_response = *va
| if (!((compare_add ^ *va) & compare_add_mask)) then
| *va = (*va & ~(swap_mask)) | (swap & swap_mask)
|
| return atomic_response
The additional operands are carried in the Extended Transport Header. Atomic response genera-
tion and packet format for MskCmpSwap is as for standard IB Atomic operations.