White Papers

22 CheXNet Inference with Nvidia T4 on Dell EMC PowerEdge R7425 | Document ID
max_batch_size=batch_size,
max_workspace_size_bytes=workspace_size<<20,
precision_mode=precision_mode)
write_graph_to_file(graph_name, trt_graph, output_dir)
return trt_graph
Create and save GraphDef for the TensorRT™ inference using TensorRT™ library
(optional INT8 ):
Convert a TensorRT™ graph used for calibration to an inference graph “
def get_trt_graph_from_calib(graph_name, calib_graph_def, output_dir):
trt_graph = trt.calib_graph_to_infer_graph(calib_graph_def)
write_graph_to_file(graph_name, trt_graph, output_dir)
return trt_graph
Import the TensorRT™ graph into a new graph:
output_node = tf.import_graph_def(
trt_graph,
return_elements=[“chexnet_sigmoid_tensor”])
Run the Optimized Inference in all desired modes:
output = return_tensors[0].outputs[0]
with tf.Session(graph=g, config=get_gpu_config()) as sess:
result = sess.run([output])
Command line example to execute the tensorrt_chexnet.py file
To evaluate the inference with TF-TRT integration using the trained CheXNet model:
python3 tensorrt_chexnet.py \
--savedmodel_dir=/home/chest-x-ray/chexnet_saved_model/1541777429/ \