User Manual

Driver FeaturesRev 2.0-3.0.0

Mellanox Technologies

4.8.4 Setting Performance Tuning

• Use larger IPoIB RX/TX rings (dom0)

Reload the IPoIB driver with the larger

send_queue_size/recv_queue_size values, set the follow-

ing

ib_ipoib module parameters:

• Use Jumbo Frames (JF) up to 64K (domu/domu):

• In UD mode, the maximum MTU value is 4092 Bytes

• In CM mode, the maximum MTU value is 65520 Bytes

Make sure that all interfaces (including the guest interface and its virtual bridge) have the same MTU

value. For further information of MTU and JF settings, please refer to the Hypervisor User Manual.

• Tune the TCP/IP stack using sysctl (dom0/domu)

• Enable irqbalancer (dom0/domu)

• Other performance tuning for KVM environment such as vCPU pinning and NUMA

tuning may apply. For further information, please refer to the Hypervisor User Manual.

4.9 Contiguous Pages

Contiguous Pages improves performance by allocating user memory regions over physical con-

tiguous pages. It enables a user application to ask low level drivers to allocate contiguous mem-

ory for it as part of

ibv_reg_mr.

Additional performance improvements can be reached by allocating Queue Pair (QP) and Com-

pletion Queue (CQ|) buffers to the Contiguous Pages.

To activate set the below environment variables with values of PREFER_CONTIG or CONTIG.

• For QP: MLX_QP_ALLOC_TYPE

• For CQ: MLX_CQ_ALLOC_TYPE

The following are all the possible values that can be allocated to the buffer:

send_queue_size=1024 recv_queue_size=1024

# /sbin/sysctl_perf_tuning

# /etc/init.d/irqbalance start

Table 3 - Buffer Values

Possible Value

Description

ANON Use current pages ANON small ones.

Default value.

HUGE Force huge pages.

CONTIG Force contiguous pages.

PREFER_CONTIG Try contiguous fallback to ANON small pages.

PREFER_HUGE Try huge fallback to ANON small pages.

ALL Try huge fallback to contiguous if failed fallback to ANON

small pages.