Concept Guide

Performance Characterization
15 Dell EMC Ready Solutions for HPC BeeGFS High Performance Storage | ID 460
Client Configuration
Component
Details
Clients
32x Dell EMC PowerEdge C6420 Compute Nodes
BIOS
2.2.9
Processor
2x Intel Xeon Gold 6148 CPU @ 2.40GHz, 20 cores
Memory
12x 16GB DDR4 2666 MT/s DIMMs - 192GB
BOSS Card
2x 120GB M.2 boot drives in RAID 1 for OS
Operating System
Red Hat Enterprise Linux Server release 7.6
Kernel Version
3.10.0-957.el7.x86_64
Interconnect
1x Mellanox ConnectX-4 EDR card
OFED Version
4.5-1.0.1.0
The transparent huge pages were disabled, and the following tuning options are in place on the metadata and
storage servers:
vm.dirty_background_ratio = 5
vm.dirty_ratio = 20
vm.min_free_kbytes = 262144
vm.vfs_cache_pressure = 50
vm.zone_reclaim_mode = 2
kernel.numa_balancing = 0
In addition to the above, the following BeeGFS tuning options were used:
• tuneTargetChooser parameter was set to "roundrobin" in the metadata configuration file
• tuneNumWorkers parameter was set to 24 for metadata and 32 for storage
• connMaxInternodeNum parameter was set to 32 for metadata and 12 for storage and 24 for clients
4.1 Sequential writes and reads IOzone N-N
To evaluate sequential reads and writes, the IOzone benchmark v3.487 was used in the sequential read and
write mode. These tests were conducted on multiple thread counts starting at 1 thread and increasing in
powers of 2, up to 1024 threads. At each thread count, an equal number of files were generated since this
test works on one file per thread or the N clients to N file (N-N) case. The processes were distributed across
32 physical client nodes in a round robin or cyclical fashion so that the requests are equally distributed and
there is load balancing. An aggregate file size of 8TB was selected which was equally divided among the
number of threads within any given test. The aggregate file size was chosen large enough to minimize the
effects of caching from the servers as well as from BeeGFS clients. IOzone was run in a combined mode of
write then read (-i 0, -i 1) to allow it to coordinate the boundaries between the operations. For this testing and
results, we used a 1MiB record size for every run. The commands used for Sequential N-N tests are as
follows:
iozone -i 0 -i 1 -c -e -w -r 1m -I -s $Size -t $Thread -+n -+m
/path/to/threadlist