White Papers

23 PS Series Asynchronous Replication Best Practices and Sizing Guide | BP1012
Effect of latency on replication time
Figure 10 shows the time it took to replicate 10 GB of data across an OC3 (155 Mbps) WAN link for three
different simulated link latencies. The results clearly show a significant impact on the performance of
replication across the WAN link. When 20 ms of latency was added in each direction (40 ms round trip), the
replication time increased by a factor of 12. When 50 ms of latency was added (100ms round trip), the
replication time increased by a factor of over 30.
When a device sends TCP data packets over a network, it will attempt to send as many as possible within its
TCP window size. Once it reaches its TCP window size limit, it will stop transmitting packets and wait until it
receives an acknowledgement back from the receiving device. After it receives acknowledgement, it can then
send more packets if necessary. The maximum receive window size for standard TCP is 65,535 bytes. TCP
also allows the devices to scale (increase) the size of this window as high as 1 GB (see RFC 1323, TCP
Extensions for High Performance). With larger window sizes, it is possible to have more packets in flight at a
given point in time. This window size scaling feature allows TCP to optimize data transmission over a variety
of link conditions and possibly improve its throughput.
As latency grows, it becomes more likely that devices will have to wait while data is in flight. Since they are
waiting, and not actually sending or receiving data (the packets are still traveling somewhere in the middle),
their overall throughput efficiency can decrease. Increasing the size of the TCP window allows more data to
be placed on the wire at time, thus keeping the packet exchange going closer to 100 percent of what is
theoretically possible. Of course, this also represents a greater risk—if more data is outstanding and an
acknowledgement is never received (a packet is lost somewhere or times out), then the sending device will
have to retransmit all of the data that was not acknowledged. If this scenario occurs too often, then
performance may be degraded.
11.6
141.52
346.13
0 50 100 150 200 250 300 350 400
0
20
50
Replication time (minutes)
One-way latency (ms)
Effects of latency at OC3
10GB data