White Papers

31 CheXNet – Inference with Nvidia T4 on Dell EMC PowerEdge R7425
Where: --native: Benchmark model with it's native precision FP32 and without TensorRT™.
Script Output sample:
==========================
network: native_frozen_graph.pb, batchsize 1, steps 100
fps median: 9.2, mean: 9.1, uncertainty: 0.1, jitter: 0.3
latency median: 0.10912, mean: 0.11459, 99th_p: 0.23157, 99th_uncertainty: 0.18079
==========================
• Throughput (images/sec): ~9
• Latency (sec): 0.11459*1000 = ~115
5.2 CheXNet Inference - Native TensorFlow fp32 with GPU
Benchmarks ran with batch sizes 1-32 using native TensorFlow FP32 GPU without
TensorRT™. We ran the benchmarks within the docker image
nvcr.io/nvidia/tensorflow:18.10-py3, which supports TensorFlow with GPU support.
Figure 11. CheXNet Inference - Native TensorFlow FP32 with GPU