Administrator Guide

14 Deep Learning Performance Scale-Out
Figure 12: Multi Node PowerEdge C4140-M - ResNet-50’s Configuration for Best Performance
ResNet-50’s Scale-out
PowerEdge C4140 using Nvidia 4x NVLink architecture scales relatively well when using Uber
Horovod distributed training library and Mellanox InfiniBand as the high-speed link between
nodes. It scales ~3.9x times within the node and ~6.9x using scale-out for ResNet-50 with batch
size 256. See Figure 13.