User Manual

Features Overview and ConfigurationRev 2.3-1.0.1
Mellanox Technologies
54
Figure 2: RoCE and v2 Protocol Stack
3.1.6.2 RoCE Configuration
In order to function reliably, RoCE requires a form of flow control. While it is possible to use
global flow control, this is normally undesirable, for performance reasons.
The normal and optimal way to use RoCE is to use Priority Flow Control (PFC). To use PFC, it
must be enabled on all endpoints and switches in the flow path.
For further information, please refer to RoCE Over L2 Network Enabled with PFC User Guide:
http://www.mellanox.com/related-docs/prod_software/
RoCE_with_Priority_Flow_Control_Application_Guide.pdf
3.1.6.2.1 Prerequisites
The following are the driver’s prerequisites in order to set or configure RoCE:
• ConnectX®-3 firmware version 2.32.5000 or higher
• ConnectX®-3 Pro firmware version 2.32.5000 or higher
• All InfiniBand verbs applications which run over InfiniBand verbs should work on
RoCE links if they use GRH headers.
• Set HCA to use Ethernet protocol:
Display the Device Manager and expand “System Devices”. Please refer to
Section 3.1.1.1, “Port Type Management”, on page 34.
3.1.6.2.2 Configuring SwitchX® Based Switch System
 To enable RoCE, the SwitchX should be configured as follows:
• Ports facing the host should be configured as access ports, and either use global pause or
Port Control Protocol (PCP) for priority flow control