Kevin Formby
VP, Finance and Capital Markets
Blog

Real World Exchange Feed Latency

March 26, 2020 by Kevin Formby

Knowing the latency of market data received from exchanges is often critical to trading profitability or ensuring compliance with regulatory requirements for best execution. Although clearly an interest to algorithmic traders, others such as brokers and resellers of market data also need to understand how delayed the data is that they receive from trading venues. Historically the measurement of the one way latency from exchanges has been difficult. Solutions have often involved the deployment of two measuring devices - one close to the exchange and one close to where the data is being received. These devices have often been difficult to deploy and suffer from the fundamental issue of not measuring the latency from the actual exchange matching engine or ticker plant - they measure the latency from the first device to the second, and often the first device cannot be physically adjacent to the matching engine.

With the advent of precision time stamps in most major exchanges its now possible to implement with an alternative approach. Most exchanges now timestamp their trades with timestamps that are accurately synchronized to the global work clock (UTC). This is typically provided via a time source derived from the global GNSS systems such as Galileo or the US GPS systems, though some exchanges have direct access to their national time reference laboratories. If a single FPGA based appliance is used to measure the arrival of a market data packet at a given point (synchronized to UTC), it is then possible to calculate the one way latency from the exchange matching engine by subtracting the original exchange time stamp contained with the packet from the time of arrival.

This technique has been used in the past by high frequency traders, but Keysight/Ixia have recently introduced the same capability on the TradeVision market feed monitoring tool. Using this technique it's possible to set up one way latency (we call it OWL) measurements of market data feeds simply  in under a minute. We believe we are the first  vendor to provide a standard FPGA based feed monitoring appliance with this capability. The appliance is called TradeVision. I thought it would be useful if you could see some real life examples of this technique at work.

In conjunction with one of our customers, we have deployed a TradeVision in a major colo data center facility in the Chicago area and are using this as a test bed to refine this technique.

Let's now look at some real life examples.

1. NYSE ARCA from Mahwah NJ to Chicago IL.

The first example shows a days worth of latency measurements for the NYSE ARCA feed received at the Chicago data center received from the NYSE in Mahwah. Every packet received is examined and maximum, minimum and average latencies are calculated every second. The following graphs shows the average latency over a 12 hour period, including market open and close. As this is a view of a real customers network I have hidden the most significant digit of the latency, but it's single digit milliseconds. So lets call it X,493,000 nSec. As you can see, we have very spread of latency variation 'jitter' and are only seeing variations of +/- 1 micro seconds across the trading day. As the network being used is very deterministic and not heavily loaded we believe that the variation is primarily being caused by in-accuracies in the GPS timing signals at both locations, the accuracy in which TradeVision measures the arrival of packets, and the performance of the ticker plant at the NYSE ARCA data center in Mahwah.

On thing to note is that the latency 'jitter' seems to get worse after market close. Perhaps a counter intuitive result? We suspect the main reason is that after market close we have much less market data packets and hence there is less 'averaging' going on.

NYSE ARCA Feed Latency Graph - New York Mahwah to Chicago

2. OPRA from Mahwah NJ to Chicago IL.

The OPRA feed is produced in the same data center as the NYSE ARCA feed, so we would expect the latency from this data center to Chicago to be the same. The following graph shows the latency on the same day as the above traffic, but with just two hours of traffic. The network path is exactly the same.

Again the latency 'jitter' is usually around +/- 1 micro second. Three points to note though:

i) At the end of the trading day, the latency increases from an average of around X,588,000 to X,592,000 nSec. An increase of latency of 4 micro-seconds. We believe this to be due to the increased traffic within the OPRA ticker plant. If this was a network congestion effect, it would have shown up on the results for the NYSE ARCA feed shown above.

ii) Notice that the average latency is X,588,000 nSec vs X,493,000 nSec on the NYSE ARCA feed shown above. This is a difference of 95 micro seconds. The NYSE ARCA feed latency is using the timestamp for the "Send Time" contained in the XDP packet header, whilst the OPRA feed is using the timestamp contained in the Block Time field within the Block Header. Not easy to determine why there should be this fixed offset in latency measurement. I can think of two possible reasons - 1. There are some IT infrastructure limitations within the OPRA market data feed limiting the publishing of the messages onto the wire, or 2. Is it possible that the OPRA 'clock' has not been fully calibrated and is offset by 95 microseconds?

iii) The OPRA timestamp within the Block Header is rounded to the nearest micro-second, so in this case the maximum accuracy is going to be at least equal to this.

OPRA Mahwah to Chicago

3. NASDAQ from Carteret to Chicago IL.

The final graph shows the one way latency for the NASAQ ITCH feed from Carteret NJ to Chicago Il. This time the the X axis is a time period from 10:10 AM to 11:20AM EST. Latency jitter is similar to the above two cases. In this case the latency is around 0.35mSec smaller than the above cases due to network topology issues.

NASDAQ PSX feed

The above demonstrates how TradeVision's One Way Latency can be used to measure the latency of market data feeds from Exchanges.

If you have any questions or comments on the above, please get in touch.

Kevin