White Papers
Deep Learning Performance: Scale-up vs Scale-out
Architectures & Technologies Dell EMC | Infrastructure Solutions Group
3
7.2 Throughput images/s – Multi Node ............................................................................................ 31
7.2.1 PowerEdge C4130-P100 16GB PCIe- Multi Node ................................................................ 31
7.2.2 PowerEdge C4140-K-V100-16GB and V100-32GB: SXM2 Multi Node ................................ 33
7.2.3 PowerEdge C4140-M-V100-16GB-SXM2 Multi Node ......................................................... 35
7.2.4 PowerEdge C4140-K Multi Node Training vs Non-Dell EMC 8x V100-16GB-SXM2 ............ 36
7.2.5 PowerEdge C4140-M Multi Node Training vs Non-Dell EMC 8x V100-16GB-SXM2 ........... 38
7.2.6 PowerEdge C4140 Multi Node Training with Different CPU Models vs 8x V100-16GB-SXM2
39
7.3 Results Showing Training Time Vs Accuracy ............................................................................... 40
7.3.1 Elapsed Training Time for Several Models .......................................................................... 41
7.4 Other Explored Aspects .............................................................................................................. 45
7.4.1 Hyper-parameters tuning.................................................................................................... 46
7.4.2 Learning Rate Effect in Distributed Mode ........................................................................... 47
7.4.3 Communication and Neural Networks Primitives............................................................... 49
8 Conclusion and Future Work .............................................................................................................. 50
9 Citation ................................................................................................................................................ 51
10 References ...................................................................................................................................... 51