User Manual

Rev 2.3-1.0.1
Mellanox Technologies
199
• Bandwidth of links can be reduced if cable performance degrades and LLR retransmis-
sions become too numerous. Traditional IB bandwidth performance utilities can be used
to monitor any bandwidth impact.
Due to these factors, an LLR retransmission rate counter has been added to the ibdiagnet utility
that can give end users an indication of the link health.
 To monitor LLR retransmission rate:
1. Run ibdiagnet, no special flags required.
2. If the LLR retransmission rate limit is exceeded it will print to the screen.
3. The default limit is set to 500 and requires further investigation if exceeded.
4. The LLR retransmission rate is reflected in the results file /var/tmp/ibdiagnet2/ibdiagnet2.pm.
The default value of 500 retransmissions/sec has been determined by Mellanox based on the
extensive simulations and testing. Links exhibiting a lower LLR retransmission rate should not
raise special concern.
4.5 Performance Utilities
The performance utilities described in this chapter are intended to be used as a performance
micro-benchmark.
Table 8 - Diagnostic Utilities (Sheet 1 of 3)
Utility Description
ib_atomic_bw Calculates the BW of RDMA Atomic transactions between a pair of
machines. One acts as a server and the other as a client. The client RDMA
sends atomic operation to the server and calculate the BW by sampling the
CPU each time it receive a successful completion. The test supports features
such as Bidirectional, in which they both RDMA atomic to each other at the
same time, change of MTU size, tx size, number of iteration, message size
and more. Using the "-a" flag provides results for all message sizes.
For further information, please refer to the tool’s man page.
ib_atomic_lat Calculates the latency of RDMA Atomic transaction of message_size
between a pair of machines. One acts as a server and the other as a client.
The client sends RDMA atomic operation and sample the CPU clock when
it receives a successful completion, in order to calculate latency.
For further information, please refer to the tool’s man page.
ib_read_bw Calculates the BW of RDMA read between a pair of machines. One acts as
a server and the other as a client. The client RDMA reads the server mem
-
ory and calculate the BW by sampling the CPU each time it receive a suc-
cessful completion. The test supports features such as Bidirectional, in
which they both RDMA read from each other memory's at the same time,
change of mtu size, tx size, number of iteration, message size and more.
Read is available only in RC connection mode (as specified in IB spec).
For further information, please refer to the tool’s man page.