Addendum

1. The ‘wait-while’ timer is not started in the ‘waiting’ state of the MUX state machine. The standard
recommends waiting for some time for additional ports to try to join the aggregator as that may
potentially cause the original port to be unselected or be placed in a ‘standby’ state. However, FTOS
does not support the concept of a ‘standby’ state and all ports that are operational can be attached
to the aggregator. Therefore, this timer is not started and instead the port moves directly to the
‘attached’ state.
2. Optimization to the MUX state machine provides coupled control instead of independent control.
Because of this phenomenon, the state machine does not wait for the partner to signal that
collection has started before enabling both collection and distribution. This process optimizes a PDU
exchange.
Operation of FIB
For the FIB application, the CAM index generation for NextHops and FirstHops is enhanced to use a
bitmap, which speeds up the FIB initialization process. This method of using bitmaps is the same as the
technique used for Neighbor Discovery and Prefix Delegation entries for IPv6 prefixes. Also, design
enhancements in the queueing for packets directed to the CPU. Incoming ARP requests and packets to
directly-connected destinations with unresolved ARP and packets to unknown destinations (that matches
the catch-all entry and is forwarded to the CPU) are sent to queue Q0, instead being sent to the CPU on
the same queue (Q5). Such a method of operation caused a delay in resolution of the address using ARP.
Because the S6000 platform has more number of queues than the other S-Series platforms, such as
S4810, packets triggering ARP resolution and packets destined or sent to the catch-all entry are
forwarded to queue Q0, and ARP request packets are transmitted to queue Q5. This method of servicing
packets also applies for IPv6 traffic.
RDMA Over Converged Ethernet (RoCE) Overview
This functionality is supported on the S6000 platform.
Remote direct memory access (RDMA) is a mechanism that reduces both CPU cycles and latency. RDMA
over converged Ethernet (RoCE) implements IB over Ethernet. RRoCE sends InfiniBand (IB) packets over
IP. IB supports input and output connectivity for the Internet infrastructure. InfiniBand is supported to
enable the expansion of network topologies over large geographical boundaries and creation of next-
generation I/O interconnect standard in servers. Although the endpoints or the destination servers
generate such RRoCE packets, from the perspective of the switch, RRoCE is considered and processed as
an IP packet.
RRoCE packets are received and transmitted on specific interfaces called lite-subinterfaces. These
interfaces are similar the normal L3 physical interfaces with the exception of additional provisioning that
they offer to enable the VLAN ID for encapsulation.
You can configure a physical interface or a L3 Port Channel interface as a lite-subinterface. When you
configure a lite subinterface, only tagged IP packets with VLAN encapsulation are processed and routed.
All other data packets are discarded.
To provide lossless service for RRoCE, Qos service policy must be configured in the ingress direction,
such as dot1p and PFC and in the egress direction, such as strict priority for queues mapped to the VLAN
dot1p values, on lite-subinterfaces.
Normal L3 physical interface processes only untagged packets and makes routing decisions based on the
default L3 VLAN ID(4095), while the routed packets are transmitted as untagged.
To enable routing of RRoCE packets, the VLAN ID is mapped to the default VLAN ID of 4095 and this
mapping is performed using VLAN translation. After VLAN translation, the RRoCE packets are considered
in the same manner as normal IP packets that received on L3 interface and routed in the egress direction.
Flex Hash and Optimized Boot-Up
143