Users Guide

Broadcom NetXtreme-E-UG304-2CS
120
NetXtreme-E User Guide User Guide for Dell Platforms
14.2.5.10 Applications
14.2.5.10.1 OpenMPI
If OpenMPI is used, the OpenMPI source and configuration must be patched using the following commands:
$ cd openmpi-x.y.z
$ cat > openmpi.patch << EOF
diff --git a/opal/mca/common/verbs/common_verbs_port.c b/opal/mca/common/verbs/common_verbs_port.c
index 831ba3f..7ebeb30 100644
--- a/opal/mca/common/verbs/common_verbs_port.c
+++ b/opal/mca/common/verbs/common_verbs_port.c
@@ -68,6 +68,10 @@ int opal_common_verbs_port_bw(struct ibv_port_attr *port_attr,
/* EDR: 25.78125 Gb/s * 64/66, in megabits */
*bandwidth = 25000;
break;
+ case 64:
+ /* ODR: 50 Gb/s * 64/66, in megabits */
+ *bandwidth = 50000;
+ break;
+ case 128:
+ /* ODR: 100 Gb/s * 64/66, in megabits */
+ *bandwidth = 100000;
+ break;
default:
/* Who knows? */
return OPAL_ERR_NOT_FOUND;
EOF
$ patch -p1 < openmpi.patch
$ make
$ sudo make install
A device configuration section must also be added using the following commands:
$ sudo cat >> /usr/local/share/openmpi/mca-btl-openib-device-params.ini << EOF
[Broadcom Netxtreme]
vendor_id = 0x14e4
vendor_part_id =
5637,5638,5652,5824,5825,5838,5839,5846,5847,5848,5849,5855,5858,5859,5861,5867,5869,5871,5872,5873
use_eager_rdma = 1
mtu = 1024
receive_queues = P,128,256,192,128:S,65536,256,192,128
max_inline_data = 96
EOF
14.2.5.11 Performance
NCC is a comprehensive solution used for congestion management that can help reduce packet losses and congestion
spreading as well as improve latency by keeping switch queue levels low.
Table 42: PCI Errors
Counters Description
res_tx_pci_err Number of responder transmit PCI errors detected.
res_rx_pci_err Number of responder receive PCI errors detected.