Administrator Guide

Performance characterization
41 Dell EMC Ready Solution for HPC PixStor Storage | Document ID
Since 4KiB files cannot fit into an inode along with the metadata information, NVMe nodes will be used to
store data for each file. Therefore, MDtest can give a rough idea of small files performance for reads and the
rest of the metadata operations.
The following command was used to execute the benchmark, where $Threads was the variable with the
number of threads used (1 to 512 incremented in powers of two), and my_hosts.$Threads is the
corresponding file that allocated each thread on a different node, using round robin to spread them
homogeneously across the 16 compute nodes. Similar to the Random IO benchmark, the maximum number
of threads was limited to 512, since there are not enough cores for 1024 threads and context switching would
affect the results, reporting a number lower than the real performance of the solution.
mpirun --allow-run-as-root -np $Threads --hostfile my_hosts.$Threads --prefix
/mmfs1/perftest/ompi --mca btl_openib_allow_ib 1
/mmfs1/perftest/lanl_ior/bin/mdtest -v -d /mmfs1/perftest/ -i 1 -b $Directories
-z 1 -L -I 1024 -y -u -t -F -w 4K -e 4K
Since performance results can be affected by the total number of IOPS, the number of files per directory and
the number of threads, it was decided to keep fixed the total number of files to 2 MiB files (2^21 = 2097152),
the number of files per directory fixed at 1024, and the number of directories varied as the number of threads
changed as shown in Table 4.
Figure 23 Metadata Performance – 4 KiB Files
First, notice that the scale chosen was logarithmic with base 10, to allow comparing operations that have
differences with several orders of magnitude; otherwise some of the operations would look like a flat line close
to 0 on a linear scale. A log graph with base 2 could be more appropriate, since the number of threads are
increased in powers of 2, but the graph would look very similar and people tend to handle and remember
better numbers based on powers of 10.