User Manual

Rev 4.70
Mellanox Technologies
169
Step 3. [Recommended] Direct ALL TCP/UDP traffic to a lossy priority by using the “IPProtocol-
MatchCondition”.
Arista switches forwards the pcp bits (e.g. 802.1p priority within the vlan tag) from ingress to
egress to enable any two End-Nodes in the fabric as to maintain the priority along the route.
In this case the packet from the sender goes out with priority X and reaches the far end-node with
the same priority X.
 To force MSMPI to work over ND and not over sockets, add the following in mpiexec com-
mand:
A.5 Configuring MPI
Step 1. Configure all the hosts in the cluster with identical PFC (see the PFC example below).
Step 2. Run the WHCK ND based traffic tests to Check PFC (ndrping, ndping, ndrpingpong, ndping-
pong).
Step 3. Validate PFC counters, during the run-time of ND tests, with “Mellanox Adapter QoS Coun-
ters” in the perfmon.
Step 4. Install the same version of HPC Pack in the entire cluster.
NOTE: Version mismatch in HPC Pack 2012 can cause MPI to hung.
Step 5. Validate the MPI base infrastructure with simple commands, such as “hostname”.
A.5.1 PFC Example
In the example below, ND and NDK go to priority 3 that configures no-drop in the switches. The
TCP/UDP traffic directs ALL traffic to priority 1.
• Install dcbx, and remove any previous settings
• Install-WindowsFeature Data-Center-Bridging
• Remove-NetQosTrafficClass
• Remove-NetQosPolicy -Confirm:$False
• Set-NetQosDcbxSetting -Willing 0
• New-NetQosPolicy “SMB" -NetDirectPortMatchCondition 445 -
PriorityValue8021Action 3
• New-NetQosPolicy “DEFAULT" -Default -PriorityValue8021Action 3
• New-NetQosPolicy “TCP" -IPProtocolMatchCondition TCP -
PriorityValue8021Action1
TCP is being used for MPI control channel (smpd), while UDP is being used for other
services such as remote-desktop.
The priority should be losslessin the switches
-env MPICH_DISABLE_ND 0 -env MPICH_DISABLE_SOCK 1