Specifications
DATA CENTER BEST PRACTICES
SAN Design and Best Practices 44 of 84
DSCP and L2CoS are congured on a per-FCIP circuit basis. It is recommended that you not alter the QoS
markings for F-Class trafc unless it is required to differentiate and expedite F-Class trafc across the IP network
between the sites. Failure of F-class trafc to arrive in a timely manner will cause instability in the FC fabric. This
is less of an issue with directly connected separate RDR networks. FCIP networks that have to be connected to
the production FC fabrics can use FCR (IR license) to protect the edge fabrics from instability.
FCIP Design Best Practices
Bandwidth Allocation
For RDR, best practice is to use a separate and dedicated IP connection between the production data center
and the backup site. Often a dedicated IP connection between data centers is not practical. In this case,
bandwidth must at least be logically dedicated. There are a few ways this can be done. First, use QoS, and
give FCIP a high priority. This logically dedicates enough bandwidth to FCIP over other trafc. Second, use
Committed Access Rate (CAR) to identify and rate-limit certain trafc types. Use CAR on the non-FCIP trafc
to apportion and limit that trafc to a maximum amount of bandwidth, leaving the remainder of the bandwidth
to FCIP. Set the aggregate FCIP rate limit on the Brocade 7800 switch or FX8-24 blade to use the remaining
portion of the bandwidth. This results in logically dedicating bandwidth to FCIP. Last, it is possible, with massive
overprovisioning of bandwidth, for various trafc types to coexist over the same IP link. Brocade FCIP uses an
aggressive TCP stack called Storage Optimized TCP (SO-TCP), which dominates other TCP ows within the IP link,
causing them to back off dramatically. If the other ows are UDP-based the result is considerable congestion
and excessive dropped packets for all trafc.
Best practice is to always rate limit the FCIP trafc on the Brocade 7800 or FX8-24 blade and never rate limit
FCIP trafc in the IP network, which often leads to problems that are difcult to troubleshoot. The rate limiting
technology on the Brocade 7800/FX8-24 is advanced, accurate, and consistent, so there is no need to double
rate limit. If policy required you to double rate limit, then the IP network should set its rate limiting above that of
the Brocade 7800/FX8-24 with plenty of headroom.
To determine the amount of network bandwidth needed, it is recommended that a month’s worth of data is
gathered using various tools that are host-, fabric-, and storage-based. It is important to understand the host–to-
disk trafc, since that is the amount of trafc to be replicated, or mirrored, to the remote disk.
If you are going to be doing synchronous RDR (RDR/S), then record peak values. If you are going to be using
asynchronous RDR (RDR/A) then record the average value over the hour. RDR/S must have enough bandwidth to
send the write I/O immediately; therefore, there must be enough bandwidth to accommodate the entire demand,
which is peak value. RDR/A needs only enough bandwidth to accommodate the high average discovered over
an adequate recording period, because RDR/A essentially performs trafc shaping, moving the peaks into the
troughs, which works out to the average. It cannot be the average over a very long period of time, because those
troughs may not occur soon enough to relieve the array of the peaks. This causes excessive journaling of data,
which is difcult to recover from.
Plot the values into a histogram. More than likely, you will get a Gaussian curve (see Figure 32). Most of
the averages will fall within the rst standard deviation of the curve, which is 68.2% of the obtained values.
The second standard deviation will include 95.4% of the obtained values, which are enough samples to
determine the bandwidth you will need. Outside of this, the values are corner cases, which most likely can be
accommodated by the FCIP network due to their infrequency. Use a bandwidth utilization value that you are
comfortable with between
σ
and 2
σ
. You can plan for a certain amount of compression, such as 2:1. However,
best practice is to use compression as a way to address future bandwidth needs. It is probably best not to push
the limit right at the start, because then you will have nowhere to go in the near future.