sarah headshot
Product Marketer
Blog

Why You Can’t Trust Your Market Data Feed- Current Approaches to Feed Monitoring

September 20, 2018 by Sarah Gross

As I discussed in my last blog post, multicast gaps are a major problem that affect market data feeds and those making trades using the data. Due to the nature of UDP broadcasting (or IP multicasting), the issue is not if a gap will occur in the data sequence, but when. This type of multicasting is great when transmitting high volumes of data, but it isn’t perfect. It has no built-in error detection or correction so gaps in transmissions may occur without you even knowing! Currently, there are four main approaches to monitoring market data feeds and detecting gaps, each with their own benefits and limitations. This blog will discuss the most popular methods for multicast gap detection in market data today and how it affects your trading.

man looking at a paper with a magnifying glass

Approach #1- Wireshark. Wireshark is a free protocol analyzer that can be used to capture and view traffic on a network. You can set up filters in Wireshark that dissect each packet of data and reverse engineer the sequence numbers to find gaps. These gaps, however, are only flagged by the software and must then be located by someone manually combing through the data packets. With some market feeds generating a million packets per minute, this is not a feasible means for gap detection as it can take days or even weeks to locate a point at which a gap occurred. Since gap detection is time sensitive, this method is not ideal.

Approach #2- Capture + Homegrown Solution. Traffic may be captured and processed using the above method or by deploying high performance hardware. Once packets are captured, a Linux-based decoder can be written in-house to detect gaps. This method is feasible; however, it is not sustainable as IP addresses, market data, and message formats are constantly changing. Additionally, software engineers could potentially spend hours and hours building decoders, inefficiently spending their time and energy for the business. This method can turn costly, quickly.

man looking in wallet with money flying away

Approach #3- Use Trade Analytics Tools. Trade analytics tools offer real-time traffic statistics, advanced calculations, latency monitoring, report generation, and much more for the data on your network. They also have the capability to detect multicast gaps. This method of gap detection is also feasible, but at a price. Analytics tools are specialized and are built for real-time trade analytics. Why overwhelm and slowdown your analytics tools with gap detection when they are already doing significant work in other areas? Why would you crack a nut with a sledgehammer? You wouldn’t!

Approach #4- Hope and pray. This is self-explanatory… just hope that data isn’t dropped or that prices don’t freeze. In a perfect world, this method would work great. Sadly, it’s not that easy.

screen with code on it that reads error

Luckily, Ixia has created a gap detection solution that can revamp your current feed monitoring. Ixia’s TradeVision is the first market data monitoring platform to deliver a unique combination of market feed health monitoring and advanced network visibility with pre-programmed support for hundreds of trading venues. TradeVision can detect multicast gaps, microbursts, and feed failures in real-time using a dedicated FPGA so that resolution times are minimized.

Hardly a month goes by without a ‘glitch’ occurring in IT infrastructure that impacts major stock exchanges around the world. Market data feed quality has become a focus for all in the industry as IT glitches can take weeks to resolve and have significant financial impacts on a company. Learn more about the benefits TradeVision can bring to your business.

My next blog will cover microbursts, feed health, and misleading data sheets and how they affect your trust in your market data feed.