HP-MPI Version 2.2 for Linux Release Note

HP-MPI V2.2 for Linux Release Note
What’s in This Version
20
New Environment Variables
The following section provides brief descriptions of the new environment variables included in
this release.
When using InfiniBand, a certain amount of memory is pinned. This means it is locked to
physical memory and cannot be paged out. The amount of pre-pinned memory HP-MPI uses
can be adjusted using several tunables. These are described below.
By default when the number of ranks is less than or equal to 512, each rank will pre-pin
256KB per remote rank; thus making each rank pin up to 128MB. If the number of ranks is
above 512 but less than or equal to 1024, then each rank will only pre-pin 96KB per remote
rank; thus making each rank pin up to 96MB. If the number of ranks is over 1024, then the
‘shared receiving queue’ option is used which reduces the amount of pre-pinned memory used
for each rank to a fixed 64MB regardless of how many ranks are used.
MPI_RDMA_MSGSIZE=a,b,c
Specifies message protocol length where:
a Short message protocol threshold. If the message length is bigger than this
value, middle or long message protocol is used. The default is 16384 bytes.
b Middle message protocol threshold. If the message length is less than or
equal to b, consecutive short messages are used to send the whole message.
By default, we set b to 16384 bytes, the same as a, to effectively turn off
middle message protocol.
c Long message fragment size. If the message is greater than b, the message
is fragmented into pieces up to c in length (or actual length if less than c)
and the corresponding piece of the user’s buffer is pinned directly. For VAPI
and uDAPL, the default is 4194304 bytes. For Myrinet, the default is
1048576 bytes.
MPI_RDMA_NENVELOPE=N
Specifies the number of short message envelope pairs for each connection if RDMA protocol is
used, where N is the number of envelope pairs. The default is 8.
MPI_RDMA_NSRQRECV=K
Specifies the number of receiving buffers used when the shared receiving queue is used,
where K is the number of receiving buffers. If N is the number of offhost connections from a
rank, then the default value can be calculated as:
((N x 8) <= 2048) ? (N x 8) : 2048