Reference Guide

RDMA is a technology that a virtual machine (VM) uses to directly transfer information to the memory of
another VM, thus enabling VMs to be connected to storage networks. With RoCE, RDMA enables data to
be forwarded without passing through the CPU and the main memory path of TCP/IP. In a deployment
that contains both the RoCE network and the normal IP network on two different networks, RRoCE
combines the RoCE and the IP networks and sends the RoCE frames over the IP network. This method of
transmission, called RRoCE, results in the encapsulation of RoCE packets to IP packets.
When a storage area network (SAN) is connected over an IP network, the following conditions must be
satisfied:
• Faster Connectivity: QoS for RRoCE enables faster and lossless nature of disk input and output
services.
• Lossless connectivity: VMs require the connectivity to the storage network to be lossless always.
When a planned upgrade of the network nodes happens, especially with top-of-rack (ToR) nodes
where there is a single point of failure for the VMs, disk I/O operations are expected to occur in 20
seconds. If disk in not accessible in 20 seconds, unexpected and undefined behavior of the VMs
occurs. You can optimize the booting time of the ToR nodes that experience a single point of failure
to reduce the outage in traffic-handling operations.
RoCE over a routed system is called RRoCE. RRoCE has IP headers. RRoCE is bursty and uses the entire
10-Gigabit Ethernet interface. Although RRoCE and normal data traffic are propagated in separate
network portions, it may be necessary in certain topologies to combine both the RRoCE and the data
traffic in a single network structure. RRoCE traffic is marked with dot1p priorities 3 and 4 (code points 011
and 100, respectively) and these queues are strict and lossless. DSCP code points are not tagged for
RRoCE. Both ECN and PFC are enabled for RRoCE traffic. For normal IP or data traffic that is not RRoCE-
enabled, the packets comprise TCP and UDP packets and they can be marked with DSCP code points.
Multicast is not supported in that network.
Preserving 802.1Q VLAN Tag Value for Lite Subinterfaces
This functionality is supported on the S6000 platform.
All the frames in a Layer 2 VLAN are identified using a tag defined in the IEEE 802.1Q standard to
determine the VLAN to which the frames or traffic are relevant or associated. Such frames are
encapsulated with the 802.1Q tags. If a single VLAN is configured in a network topology, all the traffic
packets contain the same do1q tag, which is the tag value of the 802.1Q header. If a VLAN is split into
multiple, different sub-VLANs, each VLAN is denoted by a unique 8021.Q tag to enable the nodes that
receive the traffic frames determine the VLAN for which the frames are destined.
Typically, a Layer 3 physical interface processes only untagged or priority-tagged packets. Tagged
packets that are received on Layer 3 physical interfaces are dropped. To enable the routing of tagged
packets, the port that receives such tagged packets needs to be configured as a switchport and must be
bound to a VLAN as a tagged member port.
A lite subinterface is similar to a normal Layer 3 physical interface, except that additional provisioning is
performed to set the VLAN ID for encapsulation.
A physical interface or a Layer 3 Port channel interface can be configured as a lite subinterface. Once a
lite subinterface is configured, only tagged IP packets with encapsulation VLAN ID are processed and
routed. All other data packets are discarded except the Layer 2 and Layer 3 control frames. It is not
required for a VLAN ID to be preserved (in the hardware or the OS application) when a VLAN ID, used for
Flex Hash and Optimized Boot-Up
341