Specifications

16
High Performance Trading/Algo Speed with Wombat Design and Implementation Guide
OL-15617-01
Testing
the mcpub instances were instructed to begin playback at 1x recorded rate, and increase to 4x recorded
rate at around 20 seconds after market-open. This resulted in four times as much data being played back
over the remainder of the run.
Data Observations
The Wombat software provides for in-line time-stamping at several points through the data path. In this
case, we focused on transport latency—roughly, the time it takes from the moment the feed handler
publishes data to the network stack to the time the application receives parsed data.
In terms of Wombat-defined timestamps, this meant the mamaperf timestamp minus mamaSendTime.
The mamaSendTime is a conservative approximation of the publication time, since it occurs just before
the feed handler hands an update off to LBM, not when LBM hands off to the network stack.
The mamaperf calculates time deltas for each update it receives. At the end of each ten-second interval,
mamaperf calculates latency statistics (mean, standard deviation, minimum, maximum) for the last 10
seconds, as well as the number of messages received in the interval and the CPU and memory at the end
of that interval. It writes these 10-second statistics to a file.
Time Synchronization
Timestamp accuracy was managed according to standard Wombat procedures. A server that does not
participate in data traffic acts as an NTP server. Each machine runs ntpdate once per second, which
resets its clock to that of the NTP server. NTP daemons are not run. Ntpdate communicates with the
master clock over a quiet network that is not carrying OPRA data.
Limitations
A few aspects of the test procedure have known limitations:
• Time synchronization—NTP using CPU clocks has limited accuracy in the sub-millisecond range,
and using ntupdate rather than NTP daemons does not take advantage of the corrective algorithms
in NTP. An explicit assumption in this report is that while some of the sub-millisecond jitter may
be due to clocks, the error is unbiased.
• Data granularity—The Wombat capture tool (mamaperf) does not preserve underlying data points
after it calculates latency statistics for an interval. This limits our ability to obtain statistics over
multiple intervals such as standard deviation, percentiles, etc. It also limits our ability to understand
behavior within an interval, such as 1-second or 1-millisecond spikes.
• Market timing replication—As described above, the intent of papareplay is to play data back with
its original spikiness. However, our observations of the 10-second-interval update rate data from
mamaperf showed that at 4x recorded rate, much of the spikiness seemed to get smoothed out. We
could not ascertain the spikiness of the data within the 10-second intervals (see previous bullet).
Some of the smoothness is no doubt due to the fact that the update rate is reported by the mamaperf
consumer, which means that updates may have been buffered at one or more points before arrival.
Nevertheless, the overall update rate was indeed four times the original update rate, and mcpub
output was considerably spikier than it is when papareplay is not used.
• Source data—The recorded OPRA data was from April 2, 2007. This has two drawbacks:
a. It is now well past April, so data rates have increased substantially since then.
b. This day may not have been as busy as other days. Therefore, a 4x recorded rate playback does
not necessarily mean four times April’s maximum rates.