Administrator Guide

Performance characterization
43 Dell EMC Ready Solution for HPC PixStor Storage | Document ID
server had either 128 GiB, or 256 GiB with a total of 3 TiB. However, to simplify testing and avoid caching, all
clients were counted as having 256GiB.
Regarding network connectivity, all clients have a Mellanox CX4 VPI adapter configured for 100 Gb Ethernet
and connected using a Dell EMC Z9100 switch, so that the limited number of clients could provide the highest
possible load for the Gateways. The two gateways tested are also connected to the Z9100 switch using a
single 100 GbE link each.
Since the number of compute nodes available for testing was only 16, when a higher number of threads was
required, those threads were equally distributed on the compute nodes (i.e. 32 threads = 2 threads per node,
64 threads = 4 threads per node, 128 threads = 8 threads per node, 256 threads =16 threads per node, 512
threads = 32 threads per node, 1024 threads = 64 threads per node). The intention was to simulate a higher
number of concurrent clients with the limited number of compute nodes available. Since some benchmarks
support a high number of threads, a maximum value up to 1024 was used (specified for each test), while
avoiding excessive context switching and other related side effects from affecting performance results.
Table 3 has the software versions used on the clients. The software versions used on the servers during the
characterization are listed in Table 10.
Table 10 Software Components versions during characterization
Solution Component
Version at Characterization
Operating System
CentOS 7.7
Kernel version
3.10.0-1062.12.1.el7.x86_64
PixStor Software
5.1.3.1
Spectrum Scale (GPFS)
5.0.4-3
OFED Version
Mellanox OFED-5.0-2.1.8.0
NFS testing
For this testing, two PixStor Gateways were used exporting the PixStor file system via native OS NFS server.
Clients were mounted the FS via NFSv3 using the IP of each gateway in a round robin fashion (odd
numbered clients to first gateway and even numbered clients to the second gateway). This manual
configuration was used to deterministically have half of the clients mounted from each Gateway. While NFSv4
is supported, NFSv3 is still widely used and it was selected as the protocol version to use.
Sequential IOzone Performance N clients to N files
Sequential N clients to N files performance was measured with IOzone version 3.487. Tests executed varied
from single thread up to 512 threads in increments of powers of two (since there are not enough cores on the
clients for executing efficiently 1024 threads).
Caching effects on the servers were minimized by setting the GPFS page pool tunable to 16GiB and using
files bigger than two times that size. It is important to notice that for GPFS that tunable sets the maximum
amount of memory used for caching data, regardless the amount of RAM installed and free.
To avoid any possible data caching effects from the clients, the total data size of the files was twice the total
amount of RAM in the clients used. That is, since each client was considered as having 256 GiB of RAM, for
threads counts equal or above 16 threads the file size was 8192 GiB divided by the number of threads (the
variable $Size below was used to manage that value). For those cases with less than 16 threads (which