Administrator Guide

Performance characterization
30 Dell EMC Ready Solution for HPC PixStor Storage | Document ID
Here it is important to remember that GPFS preferred mode of operation is scattered, and the solution was
formatted to use such mode. In this mode, blocks are allocated from the very beginning of operation in a
pseudo-random fashion, spreading data across the whole surface of each HDD. While the obvious
disadvantage is a smaller initial maximum performance, that performance is maintained fairly constant
regardless of how much space is used on the file system. That in contrast to other parallel file systems that
initially use the outer tracks that can hold more data (sectors) per disk revolution, and therefore have the
highest possible performance the HDDs can provide, but as the system uses more space, inner tracks with
less data per revolution are used, with the consequent reduction of performance.
Sequential IOR Performance N clients to 1 file
Sequential N clients to a single shared file performance was measured with IOR version 3.3.0, assisted by
OpenMPI v4.0.1 to run the benchmark over the 16 compute nodes. Tests executed varied from a one thread
up to 512 threads (since there were not enough cores for 1024 threads), and results are contrasted to the
solution without the capacity expansion.
Caching effects were minimized by setting the GPFS page pool tunable to 16GiB and using files bigger that
two times that size. This benchmark tests used 8 MiB blocks for optimal performance. The previous
performance test section has a more complete explanation for those matters.
The following commands were used to execute the benchmark for writes and reads, where Threads was the
variable with the number of threads used (1 to 1024 incremented in powers of two), and my_hosts.$Threads
is the corresponding file that allocated each thread on a different node, using round robin to spread them
homogeneously across the 16 compute nodes.
mpirun --allow-run-as-root -np $Threads --hostfile my_hosts.$Threads --mca
btl_openib_allow_ib 1 --mca pml ^ucx --oversubscribe --prefix
/mmfs1/perftest/ompi /mmfs1/perftest/lanl_ior/bin/ior -a POSIX -v -i 1 -d 3 -e -
k -o /mmfs1/perftest/tst.file -w -s 1 -t 8m -b 128G
mpirun --allow-run-as-root -np $Threads --hostfile my_hosts.$Threads --mca
btl_openib_allow_ib 1 --mca pml ^ucx --oversubscribe --prefix
/mmfs1/perftest/ompi /mmfs1/perftest/lanl_ior/bin/ior -a POSIX -v -i 1 -d 3 -e -
k -o /mmfs1/perftest/tst.file -r -s 1 -t 8m -b 128G