White Papers

Dell HPC Lustre Storage solution with Mellanox Infiniband EDR
Figure 12: File Metadata Operations
Figure 12 illustrates the file metadata results using MDtest.As shown in this graph, file create metadata
operations start with a little more than 13.8k OPS at 12 threads and scale to slightly over 29K OPS with
240 concurrent threads. This may be due to the Lustre locks needed on the MDT, but also those on the
OSTs, since using a stripe count of 24 had a significant decrease in performance. At 240 threads, we
had 2 directories (-b 2) and 2.2 million files were created.
File stat metadata operations is overwhelmingly the lightest metadata operation of the three observed.
At 12 threads, test yield over 34K OPS and scale to more than 544K OPS with 240 concurrent threads.
The increase in performance could be due to improvements made in Lustre version 2.7 metadata
operations. Also of note is the use of 15K RPM drives in the MDT volumes.
Removal of files is also limited by accesses to OSTs, similar to the create operation. However, the
remove operation has an advantage over the create operation when increasing in total threads with
over 18K OPS at 12 thread and scaling to over 160K OPS with increasing concurrent threads.
5. Performance Tuning Parameters
There are a number of performance tuning parameters that can be configured for an optimal system
depending on intended workload patterns to be used. This section will detail the tuning parameters
that we configured on the Lustre testbed system in the Dell HPC Engineering Innovations lab.
5.1 Lustre specific tunings
[root@node057 ~]# lctl set_param osc.*.max_pages_per_rpc=1024
0
100000
200000
300000
400000
500000
600000
12 16 24 32 48 64 72 96 120 128 144 168 192 216 240
OPS
Threads
MDtest Files Metadata - Dell HPC Lustre Storage with
Infiniband EDR
File Create File Stat File Remove