User Manual

Rev 2.2-1.0.1
Mellanox Technologies
143
5 HPC Features
5.1 HPC-X
Mellanox HPC-X Scalable Software Toolkit provides various acceleration packages to improve
both the performance and scalability of popular MPI and SHMEM/PGAS libraries. These pack
-
ages, including MXM (Mellanox Messaging) which accelerates the underlying send/receive (or
put/get) messages, and FCA (Fabric Collectives Accelerations) that accelerates the underlying
collective operations used by the MPI/PGAS languages. This full-featured, tested and packaged
version of HPC software enables MPI, SHMEM and PGAS programming languages to scale to
extremely large clusters by improving on memory and latency related efficiencies, and to assure
that the communication libraries are fully optimized of the Mellanox interconnect solutions.
The FCA and MXM software packages are also readily integrated into popular commercial and
open-source MPI languages including OpenSHMEM / PGAS and the Berkeley UPC program
-
ming languages.
Mellanox OFED (MLNX_OFED) is a Mellanox tested and packaged version of the OpenFabrics
Enterprise Distribution (OFED™) and supports RDMA (Remote DMA) and kernel bypass APIs
called OFED verbs over InfiniBand and Ethernet.
Mellanox OFED and HPC-X allow OEM's and System Integrators to meet the needs of their
endusers by deploying the latest available software that takes advantage of the features and capa
-
bilities available in the most recent hardware and firmware changes.
For further information, please refer to the HPC-X README file. (www.mellanox.com -->
Products --> ScalableHPC --> HPC-X)
5.2 Shared Memory Access
The Shared Memory Access (SHMEM) routines provide low-latency, high-bandwidth communi-
cation for use in highly parallel scalable programs. The routines in the SHMEM Application Pro-
gramming Interface (API) provide a programming model for exchanging data between
cooperating parallel processes. The SHMEM API can be used either alone or in combination
with MPI routines in the same parallel program.
The SHMEM parallel programming library is an easy-to-use programming model which uses
highly efficient one-sided communication APIs to provide an intuitive global-view interface to
shared or distributed memory systems. SHMEM's capabilities provide an excellent low level
interface for PGAS applications.
A SHMEM program is of a single program, multiple data (SPMD) style. All the SHMEM pro-
cesses, referred as processing elements (PEs), start simultaneously and run the same program.
Commonly, the PEs perform computation on their own sub-domains of the larger problem, and
periodically communicate with other PEs to exchange information on which the next communi
-
cation phase depends.
The SHMEM routines minimize the overhead associated with data transfer requests, maximize
bandwidth, and minimize data latency (the period of time that starts when a PE initiates a transfer
of data and ends when a PE can use the data).
SHMEM routines support remote data transfer through:
• “put” operations - data transfer to a different PE