User Manual

Rev 2.1-1.0.6

Mellanox Technologies

121

5.3.4 Configuring Multi-Rail Support

Multi-Rail support enables the user to use more than one of the active ports on the card, by mak-

ing a better use of the resources. It provides a combined throughput among the used ports.

 To configure dual rail support:

• Specify the list of ports you would like to use to enable multi rail support.

5.3.5 Configuring MXM over the Ethernet Fabric

 To configure MXM over the Ethernet fabric:

Step 1. Make sure the Ethernet port is active.

Step 2. Specify the ports you would like to use, if there is a non Ethernet active port in the card.

5.4 Fabric Collective Accelerator

The Mellanox Fabric Collective Accelerator (FCA) is a unique solution for offloading collective

operations from the Message Passing Interface (MPI) process to the server CPUs. As a system-

wide solution, FCA does not require any additional hardware. The FCA manager creates a topol-

ogy based collective tree, and orchestrates an efficient collective operation using the CPUs in the

servers that are part of the collective operation. FCA accelerates MPI collective operation perfor-

mance by up to 100 times providing a reduction in the overall job runtime. Implementation is

simple and transparent during the job runtime.

MLNX_OFED v2.0 or later comes with a pre-installed version of FCA v2.x

FCA is built on the following main principles:

• Topology-aware Orchestration

The MPI collective logical tree is matched to the physical topology. The collective logical

tree is constructed to assure:

• Maximum utilization of fast inter-core communication

• Distribution of the results.

• Communication Isolation

-x MXM_RDMA_PORTS=cardName:portNum

-x MXM_IB_PORTS=cardName:portNum

ibv_devinfo

ibv_devinfo displays the list of cards and ports in the system. Please make sure (in the

ibv_devinfo output) that the desired port has Ethernet at the link_layer field and that

its state is PORT_ACTIVE.

-x MXM_RDMA_PORTS=mlx4_0:1

-x MXM_IB_PORTS=mlx4_0:1