COVID-19 Late Testing a Lesson for Every Network Equipment Manufacturer
I was watching the TED talk by Bill Gates recorded in March 2020 about how humanity should have responded to the coronavirus pandemic. He said, “Timely testing is super, super urgent” and probably the lack of it is costing humanity its biggest premium now in April 2020 and time to come. The one lost month between January and February 2020 could have changed the situation to a better one if we reacted and started testing sooner. It struck me how we make similar mistakes around testing in other fields as well. This blog is about one such instance, where early testing of protocol conformance would have resulted in a much better outcome for interoperability in the field
A few months back, one of Keysight’s customers for Layer 2-3 network infrastructure validation asked for a conformance test solution for segment routing (SR) protocols. I was a bit surprised since SR adoption became popular around 2017 and almost all the large equipment vendors were implementing it in their devices. The validation cycle continued throughout 2018 and 2019 and all the top vendors were using Ixia’s IxNetwork for functional and scale testing of SR protocol implementations. Then why is there a renewed demand for conformance testing at this later stage? Digging a bit deeper revealed that one of the challenges the network equipment manufacturers (NEMs) are facing is ensuring interoperability with other implementations that service providers may be using.
Experiment with a well-known implementation
To understand the nature of the problems, we picked up the segment routing implementation from one of the well-known device manufacturers and, in our lab, we used the IxANVL segment routing conformance suite to qualify it. For this experiment, we used the ISIS-SR as a protocol. To our surprise, the result showed 20% failure. Out of those 15 failures, we identified 2 of them were because the device under test (DUT) didn’t support certain features. The other 13 failures were due to non-conformance of the implementation. The table below gives an overview of the experiment results.
When we ran the test, it was very easy to isolate the problem using IxANVL debugging features. The IxANVL ladder diagram showed the packet exchange and clicking on the steps showed the detailed packet view. Here is an example of a test failure due to the DUT sending incorrect values of SID in the packet.
It’s clear that these types of issues in the routers will result in wrong configuration of the flows in the network and eventually result in data traffic drop. This type of error should be caught very early in the test cycle. Imagine the loss of goodwill for the NEM if this is found during a bake-off at a service provider’s campus.
With Ixia’s IxANVL (Automated Network Validation Library), you can quickly and effortlessly access a vast array of protocol libraries and utilities to validate protocol compliance and interoperability. By emulating large, multi-node networks, it not only reduces costs—it also leads to more efficient testing and faster product-release times. And extensive and thorough automated testing increases your confidence in product quality. Use the IxANVL Segment Routing conformance test to validate your SR stack when it’s still “asymptomatic” and avoid the cost of failure later.