Users Guide

Broadcom NetXtreme-E-UG304-2CS
122
NetXtreme-E User Guide User Guide for Dell Platforms

bnxt_en.conf to override to use updated version. Users can also erase the current NetXtreme-E Linux kernel driver
using the rpm -e kmod-bnxt_en command. RHEL 7.3/SLES 12 Sp2 has bnxt_en inbox driver (older than v1.7.x). This
driver must be removed and the latest bnxt_en be added before applying the bnxt_re (RoCE drivers).
14.3 Windows and Use Case Examples
14.3.1 SMB Direct
Windows Server 2012 includes a new feature called SMB Multichannel, part of the SMB 3.0 protocol, which increases the
network performance and availability for file servers.
SMB Multichannel allows file servers to use multiple network connections simultaneously and provides the following
capabilities:
 Increased Throughput – The file server can simultaneously transmit more data using multiple connections for high
speed network adapters or multiple network adapters.
 Network Fault Tolerance – When using multiple network connections at the same time, the clients can continue to work
uninterrupted despite the loss of a network connection.
 Automatic Configuration – SMB Multichannel automatically discovers the existence of multiple available network paths
and dynamically adds connections as required.
SMB Multichannel is the feature responsible for detecting the RDMA capabilities of NICs to enable the SMB Direct feature
(SMB over RDMA). Without SMB Multichannel, SMB uses regular TCP/IP with these RDMA-capable NICs ( all provide a
TCP/IP stack side-by-side with the new RDMA stack).
With SMB Multichannel, SMB detects the RDMA capability and create multiple RDMA connections for that single session
(two per interface). This allows SMB to use the high throughput, low latency, and low CPU utilization offered by these RDMA
NICs. It also offers fault tolerance if multiple RDMA interfaces are used.
14.3.1.1 Kernel Mode
Windows Server 2012 and beyond invokes the RDMA capability in the NIC for SMB file traffic if both ends are enabled for
RDMA. Broadcom NDIS miniport bnxtnd.sys v20.6.2 and beyond support RoCEv1 and RoCEv2 via the NDKPI interface.
The default setting is RoCEv1.
To enable RDMA:
1. The goal is to exercise as many RDMA connections per network interface as possible using a simple server to client
relationship. In order to achieve that, following registry key needs to be added and set its value to 16:
a. HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\ConnectionCountPerRdmaNetworkInterf
ace
b. The default is 2; with a valid range from 1 to 16.
c. This allows 16 RDMA connections per Network share. In order to scale up from here with a single interface, additional
alias IP address must be created with different subnets.
2. Upgrade the NIC NVRAM using the appropriate board packages. In CCM or in UEFI HII, enable support for RDMA.
3. Go to the adapter Advanced Properties page and set NetworkDirect Functionality to Enabled for each NetXtreme-E
miniport, or using PowerShell window, run the following command:
Set-NetAdapterAdvancedProperty -RegistryKeyword *NetworkDirect -RegistryValue 1
4. The following Powershell commands return true if NetworkDirect is enabled.