Kevin Formby
VP, Finance and Capital Markets
Blog

Micro-burst Traffic Level Correlation in Market Data Feeds

April 17, 2020 by Kevin Formby

In financial market networks, Micro-bursts are spikes in traffic over relatively short time periods. The time intervals are usually measured in fractions of a second or even fractions of a milli-second. Real time market data feeds are often based on multicast and use multicast channels to segment the total traffic. This allows multicast traffic to be easily load balanced across trading infrastructure such as network switches or servers. The traffic that is sent across a given multicast channel is often defined by the symbol ranges of traded instruments. For example in the Arcabook feed from ICE, the first multicast channel includes all equities traded that start with the symbols A - D, channel 2 with symbols E - L, channel 3 with M - S and channel 4 with T - Z.

As network engineers have to design trading infrastructure to combine or dis-aggregate multicast traffic to and from multiple channels, a key point of interest is whether the volume of traffic between the multicast groups is correlated. Normally in telecommunication networks, traffic is often "noise" like. Adding traffic together will usually increase the average traffic levels, but peaks in traffic will often not be additive due to the noise like traffic profiles. Peaks will coincide with troughs etc. However, is this the case with multicast traffic from market data feeds? Will a peak in traffic on say channel 1 always coincide with a traffic on another multicast channel from the same feed? Are micro-bursts random across the multicast channels?

Using TradeVision, which is "multicast channel aware", its quick and simple to plot micro-burst traffic on a graph and export it for analysis. The following is a graph taken this afternoon (17th April 2020) and shows the effective data rate (in kbps) for the four production channels that make up Line A of the Arcabook market data feed over 30 seconds (the sampling period was 0.1 Secs, so there are 300 data points on the graph for each channel):-

Arca Correlation of Traffic Across Channels

Well on the surface it certainly looks correlated! There is close correlation between the peaks in traffic. A peak in one channels traffic seems to also coincide with traffic on the other channels. Its possible to export this data and then perform statistical analysis to confirm this conclusion. Exporting and then using this data to calculate the cross-correlation functions for the four channels we get:

Correlation Matrix Results

This shows clearly the traffic spikes/micro-bursts are closely correlated. Network and trading infrastructure design engineers should therefore always assume that a peak on any one given multicast channel is likely to coincide with peaks on all the other multicast channels.