User Manual

Rev 4.60
Mellanox Technologies
129
Appendix A: Windows MPI (MS-MPI)
A.1 Overview
Message Passing Interface (MPI) is meant to provide virtual topology, synchronization, and com-
munication functionality between a set of processes.
With MPI you can run one process on several hosts.
• Windows MPI run over the following protocols:
• Sockets (Ethernet)
• Network Direct (ND)
A.1.1 Prerequisites
• Install HPC (Build: 4.0.3906.0).
• Validate traffic (ping) between the whole MPI Hosts.
• Every MPI client need to run smpd process which open the mpi channel.
• MPI Initiator Server need to run: mpiexec. If the initiator is also client it should also run
smpd.
A.2 Running MPI
Step 1. Run the following command on each mpi client.
Step 2. Install ND provider on each MPI client in MPI ND.
Step 3. Run the following command on MPI server.
A.3 Directing MSMPI Traffic
Directing MPI traffic to a specific QoS priority may delayed due to:
• Except for NetDirectPortMatchCondition, the QoS powershell CmdLet for NetworkDi-
rect traffic does not support port range. Therefore, NetwrokDirect traffic cannot be
directed to ports 1-65536.
• The MSMPI directive to control the port range (namely: MPICH_PORT_RANGE
3000,3030) is not working for ND, and MSMPI chose a random port.
A.4 Running MSMPI on the Desired Priority
Step 1. Set the default QoS policy to be the desired priority (Note: this prio should be lossless all the
way in the switches*)
Step 2. Set SMB policy to a desired priority only if SMD Traffic running.
start smpd -d -p <port>
mpiexec.exe -p <smpd_port> -hosts <num_of_hosts> <hosts_ip_list>
-env MPICH_NETMASK <network_ip/subnet> -env
MPICH_ND_ZCOPY_THRESHOLD -1 -env MPICH_DISABLE_ND <0/1> -env
MPICH_DISABLE_SOCK <0/1> -affinity <process>