User Manual
Table Of Contents
- Mellanox WinOF VPI User Manual
- Table of Contents
- List of Tables
- Document Revision History
- About this Manual
- 1 Introduction
- 2 Firmware Upgrade
- 3 Driver Features
- 3.1 Hyper-V with VMQ
- 3.2 Header Data Split
- 3.3 Receive Side Scaling (RSS)
- 3.4 Port Configuration
- 3.5 Load Balancing, Fail-Over (LBFO) and VLAN
- 3.6 Ports TX Arbitration
- 3.7 RDMA over Converged Ethernet (RoCE)
- 3.8 Network Virtualization using Generic Routing Encapsulation
- 3.9 Differentiated Services Code Point (DSCP)
- 4 Deploying Windows Server 2012 and Above with SMB Direct
- 5 Driver Configuration
- 6 Performance Tuning
- 7 OpenSM - Subnet Manager
- 8 InfiniBand Fabric
- 8.1 Network Direct Interface
- 8.2 part_man - Virtual IPoIB Port Creation Utility
- 8.3 InfiniBand Fabric Diagnostic Utilities
- 8.3.1 Utilities Usage
- 8.3.2 ibdiagnet
- 8.3.3 ibportstate
- 8.3.4 ibroute
- 8.3.5 ibdump
- 8.3.6 smpquery
- 8.3.7 perfquery
- 8.3.8 ibping
- 8.3.9 ibnetdiscover
- 8.3.10 ibtracert
- 8.3.11 sminfo
- 8.3.12 ibclearerrors
- 8.3.13 ibstat
- 8.3.14 vstat
- 8.3.15 osmtest
- 8.3.16 ibaddr
- 8.3.17 ibcacheedit
- 8.3.18 iblinkinfo
- 8.3.19 ibqueryerrors
- 8.3.20 ibsysstat
- 8.3.21 saquery
- 8.3.22 smpdump
- 8.4 InfiniBand Fabric Performance Utilities
- 8.4.1 ib_read_bw
- 8.4.2 ib_read_lat
- 8.4.3 ib_send_bw
- 8.4.4 ib_send_lat
- 8.4.5 ib_write_bw
- 8.4.6 ib_write_lat
- 8.4.7 ibv_read_bw
- 8.4.8 ibv_read_lat
- 8.4.9 ibv_send_bw
- 8.4.10 ibv_send_lat
- 8.4.11 ibv_write_bw
- 8.4.12 ibv_write_lat
- 8.4.13 nd_write_bw
- 8.4.14 nd_write_lat
- 8.4.15 nd_read_bw
- 8.4.16 nd_read_lat
- 8.4.17 nd_send_bw
- 8.4.18 nd_send_lat
- 8.4.19 NTttcp
- 9 Software Development Kit
- 10 Troubleshooting
- 11 Documentation
- Appendix A: Windows MPI (MS-MPI)
- Appendix B: NVGRE Configuration Scrips Examples
Rev 4.60
Mellanox Technologies
32
3.7.4 Configuring Router (PFC only)
The router uses L3's DSCP value to mark the egress traffic of L2 PCP. The required mapping,
maps the three most significant bits of the DSCP into the PCP. This is the default behavior, and
no additional configuration is required.
3.7.4.1 Copying Port Control Protocol (PCP) between Subnets
The captured PCP option from the Ethernet header of the incoming packet can be used to set the
PCP bits on the outgoing Ethernet header.
3.7.5 Configuring the RoCE Mode
Configuring the RoCE mode requires the following:
• RoCE mode is configured per-driver and is enforced on all the devices in the system
RoCE mode can be enabled and disabled via PowerShell.
•
To enable RoCE using the PowerShell:
• Open the PowerShell and run:
To disable RoCE using the PowerShell:
• Open the PowerShell and run:
3.8 Network Virtualization using Generic Routing Encapsulation
Network Virtualization using Generic Routing Encapsulation (NVGRE) is a network virtualiza-
tion technology that attempts to alleviate the scalability problems associated with large cloud
computing deployments. It uses Generic Routing Encapsulation (GRE) to tunnel layer 2 packets
across an IP fabric, and uses 24 bits of the GRE key as a logical network discriminator (which
they call a tenant network ID).
Configuring the Hyper-V Network Virtualization, requires two types of IP addresses:
• Provider Addresses (PA) - unique IP addresses assigned to each Hyper-V host that are
routable across the physical network infrastructure. Each Hyper-V host requires at least
one PA to be assigned.
• Customer Addresses (CA) - unique IP addresses assigned to each Virtual Machine that
participate on a virtualized network. Using NVGRE, multiple CAs for VMs running on
The supported RoCE modes depend on the firmware installed. If the firmware does not
support the needed mode, the fallback mode would be the maximum supported RoCE
mode of the installed NIC.
Set-MlnxDriverCoreSetting –RoceMode 1
Set-MlnxDriverCoreSetting –RoceMode 0
Network Virtualization using Generic Routing Encapsulation (NVGRE) off-load is cur-
rently supported in Windows Server 2012 R2 only.