Packet Sizes for Market Data Feeds and Their Impact on Latency
Many network switches and routers operate on a ‘store and forward’ basis. This means that in an IP network the full packet has to be clocked into the device, before it can be forwarded to the next networking device in the path to the end destination. This delay is often referred to as a serialization delay and these delays are often tested using 64 byte packets. Typically, a 64 byte packet will have serialization delays of 58 nSec when clocked in using a 10 Gbps switch/router port. If only a 1Gbps port was used this would be 580 nSec. However, serialization delays are usually proportional to the size of the packet. If the size of the packet was 640 bytes, the serialization delays would be ten times bigger – 580nSec at 10Gbps and 5.8 microseconds at 1Gbps. These delays are typically the same order of magnitude as the core switching latency through standard switches.
In financial trading applications, the one way latency of market data feeds can be critical to the performance of algorithmic trading. Testing of the latency across a network is often done by sending either ICMP pings or special test packets across the network. These tests are often done using 64 byte packet sizes. However, as the above paragraph explains, this can often give the wrong result if the market data feed is actually made up of packets with sizes significantly larger than 64bytes. So the question is – what are typical packet sizes for real time market data feeds?
Fortunately Ixia knows the answer! Ixia sells a high performance network packet broker called TradeVision which, as well as with conventional packet broker functionality, can also measure multicast gaps and the one way latency of market data feeds. To test TradeVision, Ixia has built up a large collection of packet captures (pcaps) of market data feeds. Ixia has analyzed these and come up with statistics on average packet sizes. The results for a selection of market data feeds from major exchanges is shown below.
From the above you can see that the average packet size is relatively small. This is due to the fact that exchanges are sensitive to the need to publish market data on their feeds as quickly as possible and so tend to keep packet sizes small. It is also interesting that if you look at the major equity exchanges the average packet size is under half of this – 108 bytes.
However, as the old adage goes, "There are lies, damn lies and statistics.” Averages can be very misleading. It’s important to know not just the average packet size, but to understand the distribution of packet sizes as this will tell you the maximum and minimum latencies across a network. The following graph shows the distribution of the packet sizes based on market data feeds. The data has been extracted from pcaps that Ixia has acquired over the last couple of years. It is possible that updates to the technologies in use at an exchange or even changes in the trading methodology of market participants may have changed the packet size distributions over time, but we believe this is a reasonable snapshot.
From the above you can see that the packet size distribution changes by execution venue. Some exchanges tend to have very narrow ranges of packet size and some larger.
So what does this mean for network design or operations staff who need to understand and measure the latency of their trading networks?
- At a minimum you should test and model the performance of your trading network using average packet sizes based on an up to date analysis of packet size in use on that exchange. If that is not available, go with a 256 byte average, as that is close to the average across a number of exchange market data feeds. However, if you are only interested in equities, it’s probably best to go with a 128 byte average.
- If you have up-to-date pcaps of your market data feeds, use the minimum and maximum packet sizes to test the extremes of packet size.
- If you are using ICMP “pings” to test your network, remember you can set the size of the test packets by using the -s packetsize command
Many thanks to the Ixia TradeVision QA team in Romania who helped pull together the above data.
If you want to know more about how Ixia monitors market data feeds with TradeVision then click on the link below or drop me an email firstname.lastname@example.org