Reference Guide

Solution overview
11 Reference Architecture of Dell EMC Ready Solution for HPC Life Sciences | Document 309
- CPU: 2x Intel Xeon™ Gold 6136 CPU with 12 cores at 3.0 GHz
- Memory: 24x 16 GB at 2666 MHz
- MDS SAS controller: 2 x Dell 12 Gbps SAS HBAs
- MDS Storage Array: 1 x Dell EMC PowerVault ME4024
> Disks: 12 x 960 GB SAS SSD (active/passive configuration, no DNE) or 24 x 960 GB SAS
SSD (active/Aactive Configuration, DNE)
2.4 Network components
2.4.1 Dell networking H1048-OPF
Intel Omni-Path Architecture (OPA) is an evolution of the Intel True Scale Fabric Cray Aries interconnect and
internal Intel IP (12). In contrast to Intel True Scale Fabric edge switches that support 36 ports of InfiniBand
QDR-40Gbps performance, the new Intel Omni-Path fabric edge switches support 48 ports of 100 Gbps
performance. The switching latency for True Scale edge switches is 165 ns-175 ns. The switching latency for
the 48-port Omni-Path edge switch has been reduced to around 100 ns-110 ns. The Omni-Path host fabric
interface (HFI) MPI messaging rate is expected to be around 160 million messages per second (MMPS) with
a link bandwidth of 100 Gbps.
2.4.2 Mellanox SB7700
This 36-port Non-blocking Managed InfiniBand EDR 100 Gb/s Switch System provides the highest performing
fabric solution in a 1U form factor by delivering up to 7.2 Tb/s of non-blocking bandwidth with 90 ns port-to-
port latency.
2.4.3 Dell networking S3048-ON
Management traffic typically communicates with the baseboard management controller (BMC) on the
compute nodes using IPMI. The management network is used to push images or packages to the compute
nodes from the master nodes and for reporting data from client to the master node. Dell EMC Networking
S3048-ON is recommended for management network.
2.5 Reference architecture
This reference architecture is configured with Intel OPA fabric, Dell EMC Ready Solution for HPC NFS
Storage and Dell EMC Ready Solution for HPC Lustre Storage. Dell EMC Ready Solution for HPC NFS
Storage, referred to as Dell NFS Storage Solution- HA configuration is configured for a general usage such as
a home directory, while Dell EMC Ready Solution for HPC Lustre Storage servers as a high-performance
scratch storage system. An example solution is illustrated in Figure 3. Although benchmark testing was
performed with 64 C6420s, throughput test results showed that 56 compute nodes are optimal with medium
base configuration of Dell EMC Ready Solution for HPC Lustre Storage (9).