Social Media

Ixia on Facebook Ixia on LinkedIn Ixia on Twitter
Sales 1.877.367.4942 INTL 1.818.871.1800


White Papers

Router Testing with Real-World Emulation

Introduction: The Real World Requires a Real Tester

The explosive growth of the Internet has necessarily been matched by a growth of the underlying network infrastructure. Bandwidth growth has been fueled by the big bandwidth consumers: web sites, Email - esp. spam, file sharing and recently IPTV.

Complexity has been driven by the increased usage, but also by the convergence of services. The bundle of Triple-Play voice, video and data traffic carried over a common network has engendered a raft of protocols designed to initiate and manage long-term connections while balancing the quality and delivery needs of each type of traffic.

Internet routing technology has matured in waves designed to provide solutions to growth and complexity issues. Existing technologies gave way to newer ones with more capabilities, more flexibility and more interoperability.



Protocol interoperability, allowing complexity issues to be addressed in a 'divide and conquer' manner gave rise to significant new network features:
  • OSPF + LDP gave rise to Virtual Private Line Services (VPLS)
  • OSPF + LDP + PIM + BGP enabled multicast VPNs (mVPNs)
  • ISIS + RSVP + BGP were used to implement Layer 3 VPNs (L3VPN)

During this process, scalability grew in many dimensions: total bandwidth, number and speed of interfaces, number of connections, number of routes, number of neighbors, etc.

The result: The Next Generation Network (NGN) router with increased complexity and speed - Extreme Scalability!

Both routers and switches have become ever more complex. Edge routers, which previously only acted as aggregation devices have morphed into super-routers. They now need to act as aggregators from end-users using Ethernet, Frame Relay, ATM, PPP, MPLS and other technologies and encapsulations, packet classifiers, and QoS/SLA enforcers. In their effort to complete with each other, router manufacturers have differentiated themselves by providing their edge routers with a set of rich services and features. Understanding that heavy growth will continue, they need to demonstrate that today's deployments can meet tomorrow's capacity and scale requirements. They need to prove that they can handle large numbers of protocols - each with their own routing peers, large routing tables, large numbers of VPN site, and most importantly all of these at the same time.

Although core network routers need to support a different set of services, the requirements for capacity and scale are amplified. With Internet traffic doubling every year or two, core routers must provide significant expansion of their ability to handle routing peers, routing tables, LSP tunnels, and combinations of these.



As routers become more complex, and increase in scale and speed, so does the test hardware used to characterize and test them. In fact, test equipment must have more capacity than the equipment that it tests in order to ensure their reliability and correct throttling operation. This is true both in the engineering lab, where the item being characterized is called a DUT (Device under Test), and in pre-deployment testing, where often a system or sub-system is tested and is referred to as a SUT (System under Test). Network Equipment Manufacturers (NEMs), Service Providers (SPs) and Enterprises are continually on the lookout for the right test equipment for current and future needs.

Extreme Scalability comes with a high price. In Figure 1, the area under the growth arrow at each stage represents the amount of effort required to develop products and ultimately the time to test for the product. Time to test is a substantial part of time to market; it is essential that NEMs and their customers recognize the issues involved and understand the required test technologies. With more protocols to test and more scale to characterize, test tools must exhibit substantial ease-of-use and productivity.

Ixia has always provided highly efficient tools to test switches and routers. As these DUTs have matured, so have Ixia's solutions. Platforms, load modules and applications have continually stayed ahead of the curve of technology. Ixia's latest generation: Optixia XM chassis, load modules and IxNetwork test application are all up to the task.

In this white paper, we will example a few real world scenarios that demand highly scalable and innovative test tools. Each scenario is introduced with a technology brief, the test challenges for the scenario, and realistic test targets for complexity and scale - from the NEMs that manufacturer the devices and Service Providers and Enterprises that use them. Finally, we will discuss Ixia's approach to testing the scenario.

Real World Test Case Studies

Testing IPv4/IPv6 Performance for Next-Gen-Networks (NGN)
The well documented benefits of IPv6, along with it's uptake by U.S. government agencies has prompted network equipment vendors to offer IPv6 support on devices that previously only supported IPv4. Support for IPv6 goes well beyond handling a different addressing technique; IPv6 comes with its own versions of routing protocols: BGP+, OSPFv3, ISISv6, MLD, PIM-SMv6, and 6PE/6VPE.

Devices that support both addressing modes are said to have 'dual-stack' implementations, because they have network stacks for each. Dual-stack devices must handle full mixes of IPv4/IPv6 switching and routing protocols on each of their ports. The ultimate success of IPv6 will be based on its ability to perform as well, or better than, equivalent IPv4 services. NEMs must ensure that their devices behave their consistently across a full mix of address type usage.

There are a number of challenges that must be addressed by test vendors. First and foremost, each test port must be able to emulate routers that handle mixed IPv4 and IPv6. In order for equipment vendors to properly select a test tool, test equipment vendors must publish their maximums for protocols: neighbors, sessions, tunnels, routes, etc. Testing of a single protocol at a time is unrealistic; routers see mixes of all types and test vendors must indicate maximums that can be applied in parallel. Since VLANs are often used to partition emulated routers, these limits must also be expressed with respect to VLAN usage - optimally independent of VLAN counts.

Test vendors have developed test ports that are powerful traffic generators and include high-powered CPU on every port, or sometimes shared by two or more ports. These ports are always coordinated by a test application; the application must be able to handle tens, if not hundreds of test ports. The test application should increase its capacity linearly with the number of test ports.

The real world scenario pictured below was solicited from a well-established service provider. They wished to validate that their mixed IPv4/IPv6 performance was at least as good as their IPv4-only performance in a pre-deployment scenario. The test scenario involved use of IPv4 and their equivalent IPv6 versions: OSPFv2/OSPFv3 and BGP4/BGP+ along with a volume of IPv6 multicast traffic.


Figure 1: Testing IPv4/IPv6 Dual-Stack Performance


Specifically, the service provider requested the target numbers shown in Table 1 using the smallest number of test ports.
IP Type Protocol Metric 1 Metric 2
IPv6 MLDv2 800,000 Joins
IPv4 BGP4 4,000 sessions 2,000,000 routes
IPv6 BGP+ 4,000 sessions 2,000,000 routes
IPv4 OSPFv2 800 sessions 800,000 routes
IPv6 OSVPv3 800 sessions 800,000 routes
Table 1: IPv4/IPv6 Dual-Stack Performance Requirements


Due to it's unique per-port CPU hardware architecture, Ixia's numbers scale linearly with the number of test ports. Ixia's per-port numbers are shown in Table 2.
IP Type Protocol Metric 1 Metric 2
IPv6 MLDv2 1,000 hosts 50 groups
IPv4 BGP4 250 peers 500 routes/peer
IPv6 BGP+ 250 peers 500 routes/peer
IPv4 OSPFv2 50 neighbors 1,000 routes/neighbor
IPv6 OSVPv3 50 neighbors 1,000 routes/neighbor
Table 2: Ixia Per-Port Performance Numbers


Ixia selected its Gigabit Ethernet XMV LAN Service Module (P/N LM1000XMV16), with 16 ports as candidate for this test. It provides the necessary scale, with margin, to perform the requested tests. Using a single XMV16 load module in an Optixia XM12 chassis, Ixia tested the service provider's router with the required scale - easily and effectively.

Multicast VPN Scalability
Multicast traffic is used to distribute the same content to multiple destinations. Used only occasionally until recently, it is now experiencing explosive growth. This is due to the expanding use of video conferencing, IPTV and Video on Demand (VoD). Service providers, especially Telcos looking to compete with cable and satellite operators, will seek to expand their video offerings in new and unique ways, combining video with voice and high-speed Internet (HSI) services.

The technological challenges for delivery of video to distinct customers are substantial. Virtual Private Networks (VPNs) have been very effective in segregating geographic regions and customers from each other. Originally used solely for unicast traffic, multicast traffic is now an essential requirement. Multicast versions of VPNs are referred to as multicast VPNs (mVPNs).

The implementation of VPNs internally involves the use of a Virtual Routing and Forwarding (VRF), also known as a VPN routing table. The same VRF instance is used to handle both unicast and multicast traffic for IPv4 and IPv6 connections. An additional optimization, known as Data MDT, is used to limit the destinations to which multicast traffic is sent. Together, these make for a complex implementation scenario, both in terms of functional features - not to mention scalability issues.

Testing mVPNs is equally complex. There are substantial challenges involved mVPNs at scale. In order to understand the scale, we need to explore the protocols used and how connections multiply to very large numbers.

Figure 2 is a test scenario used to test a planned configuration from a major carrier's network. Test ports are used to emulate both CE (Customer Edge) and PE (Provider Edge) routers; each supports many multicast VRFs (mVRFs). All sites use the same VPN, yielding the use of non-unique mVRFs.


Figure 2: mVPN Scalability Test Changeless with Multiple Test Ports


Although Figure 2 specifies the use of 16 Ixia ports, we'll calculate the scale quantities needed in the general case. Let's define the following three quantities:
  • X = the number of test ports used to emulate PE routers. One additional port is used to emulate the CE routers.
  • Y = the number of PE routers emulated per test port.
  • Z = the total number of mVRFs required for each emulated PE router.
  • T = the total number of PEs in the network. T = (X * Y + 1)
In an mVPN environment each mVRF requires an emulated PIM-SM/SSM router. The PIM-SM/SSM router communicates with the other PIM-SM/SSM routers that belong to the same mVRF in order to form a multicast neighborhood. Let's assume that each PE has exactly one customer site for each mVPN in the network. For each mVRF of each PE, its PIM-SM/SSM router faces a total number of neighbors.
  • Total # of neighbors = T-1 = X*Y
Within a test environment in which a test port emulates multiple PE routers, it is not necessary to emulate neighbors on the same port. This reduces the total number of PIM-SM/SSM neighbors.
  • N = total number of PIM-SM/SSM neighbors per PE router = X*Y - (Y-1) = (X-1)*Y + 1
This applies to each mVRF. Since the total number of mVRFs emulated on a test port is Y*Z, the total number of PIM-SM/SSM routers that a test port must form a neighborhood with is:
  • Total emulated PIM-SM/SSM neighbors per port = N*Y*Z = ((X-1)*Y + 1)*Y*Z
An additional neighbor is required for the provider core DUT. The final total number of neighbors:
  • Total PIM-SM/SSM neighbors per port = N*Y*Z + 1 = ((X-1)*Y + 1)*Y*Z + 1
If we use the following test equipment quantities:
  • Number of test ports for PE router emulation = 15
  • Number of PE routers emulated per test port = 4
  • Number of mVRFs per PE router
  • Total emulated PIM-SM/SSM neighbors per port = ((15-1) * 4 + 1) * 4 * 200 + 1 = 57*4*200 + 1 = 45,601
This is a truly massive number of PIM-SM/SSM neighbors for the test port to maintain. Ixia test ports handled this and the complete test case with a single XMV16 load module./

Carrier Class Router Scalability with High Availability
Carrier Class routers need to deliver a highly scalable routing implementation that also highly reliable. The routers must support a wide range of applications, while at the same time reducing CAPEX and OPEX.

The challenge in testing these carrier class routers is to test them at their full scale, both in terms of throughput and routing limits. In order to perform realistic testing, it's particularly important is to be able to test combinations of routing protocols. Simultaneous simulation of large numbers of OSPF, BGP, LDP and RSVP-TE sessions is required, along with multicast protocols.

Reliability is tested by route flapping, simulating common failures such as fiber cuts, routing process failures and power failures. The time that the DUT takes to reroute and recover from the flap must be carefully measured.

A major NEM's system integration lab wished to validate the performance and service availability of one of their carrier class routers, under conditions of heavy stress. The goal was to simulate a real world network using Ixia hardware and test applications. The desired results were a measurement of router performance and reliability/repeatability.


Figure 3: Carrier Class Router Scalability with High Availability


The requested quantities were:
  • 3200 BGP sessions with 4,500 routes/session
  • 800 OSPF sessions with 250 routes/session
  • IGMP - 153,600 joins
  • MLDv2 - 40,960 joins
  • 20 LDP sessions
  • Total # of routes = (3,200 * 4,500) + (800 * 250) = 14,600,000 routes

IxNetwork, along with an Optixia XM12 and a single XMV16 load module was used to achieve these targeted numbers. IxNetwork's build-in event scheduler was used to introduce network flaps. Overall performance, per-flow frame loss and route convergence times were all measured and validated.

Application Traffic over Emulated Topology
As we have seen, modern Internet devices are very complex devices. In order to properly make QoS and traffic admission decisions, they must perform Deep Packet Inspection (DPI). Stateless traffic, although sufficient for measuring gross throughput, is not appropriate for intelligent network devices. It is necessary to use true stateful traffic for such devices. This is referred to as application traffic, or Layer 4-7 traffic.

A number of test tools perform stateful testing, among them Ixia's IxLoad. These tools, however, uniformly perform their tests without a routing plane in place. Tester ports are directly hooked up to device ports and stateful traffic is sent from device input to output ports. In order to properly perform Layer 4-7 tests on complex routers that perform DPI, it is necessary to set up a full complement of routes, establish an anticipated number of sessions and then send traffic to the limit of the device.


Figure 4: Running Application Traffic over Emulated Topology


Figure 4 is a typical test scenario in which a Provider Edge (PE) acting as a DUT is part of a Layer 3 VPN (L3VPN) that connects clients and servers. The clients and servers could be any Internet service, including web, Email, FTP or video. The PE router, performing DPI, requires stateful traffic. Routes are necessary in order to establish Layer 4-7 performance under realistic conditions. Full testing will require the ability to scale the number of VPNs created, and the number of sites for each VPN. Application traffic, of course, will need to run across all VPNs simultaneously.

The scenario depicted in Figure 4 is in response to a request from a major service provider seeking a proof-of-concept termination for a configuration that would run triple-play traffic over an MPLS L3VPN network. They specifically requested:
  • 60 PEs in a full mesh configuration
  • 100 VPNs
  • 30,000 VPN sites
  • 100 VPN routes per site
  • Triple-play traffic over all VPN routes
  • Measurements of MOS scores for voice, MDI value for Video, and FTP throughput for data

Using only a few Ixia XMV test ports and IxNetwork with its application traffic module, Ixia was able to deliver exactly what the service provider was looking for.

Ixia's Test Solutions

Optixia XM12 chassis and XMV Series Line Card
Ixia test systems deliver the industry's most comprehensive solutions for the performance, functional, and conformance testing of networks and networked applications. The 12-slot Optixia XM12 modular chassis provides an ultra-high density, highly flexible platform on which an Ixia test system can be built. Operating in conjunction with the Aptixia family of test applications, the Optixia XM12 provides the foundation for a complete, high performance test environment.

A wide array of interface modules is available for the Optixia XM12. The chassis supports up to 192 Gigabit Ethernet ports, 36 - 10 Gigabit Ethernet ports, and 24 Packet over SONET (POS) or Asynchronous Transfer Mode (ATM) ports. These modules provide the network interfaces and distributed processing resources needed for executing a broad range of data, signaling, voice, video, and application testing from Layers 2-7.

Each chassis supports an integrated test controller that manages all system and testing resources. Resource ownership down to a per-port level coupled with hot-swappable interface modules ensures a highly flexible, multi-user testing environment. Backward compatibility is maintained with existing Ixia interface modules and test applications to provide seamless migration from and integration with existing Ixia test installations.


Figure 5: Ixia Optixia XM12 Main Frame


Ixia's Gigabit Ethernet XMV LAN Services Modules (LSMs) offer complete Layer 2-7 network and application testing functionality in a single Optixia XM test system. Each test port supports wire-speed Layer 2-3 traffic generation and analysis, high performance routing/bridging protocol emulation, and true Layer 4-7 application traffic generation and subscriber emulation. With 16 ports per module, ultra-high density test environments can be created for auto-negotiable 10/100/1000 Mbps Ethernet over copper as well as Gigabit/100FX Ethernet over fiber. With 12 slots per Optixia XM12 chassis, up to 192 Gigabit Ethernet test ports are supported in a single test system. Each port on a Gigabit Ethernet XMV module contains a powerful RISC processor running Linux and a full, testing-optimized TCP/IP stack. This architecture provides unprecedented performance and flexibility for testing routers, switches, broadband and wireless access devices, web servers, video servers, secure gateways, firewalls, and many other networking and application-aware devices.


Figure 6: Ixia Next Generation Gigabit Ethernet XMV LAN Services Module


Aptixia IxNetwork
Ixia's Aptixia IxNetwork is designed for the performance and functionality testing of high-speed, high-capacity routers, switches and application servers. It supports multiple powerful routing/bridging protocol emulations that are flexible, highly scalable, and easy to use. In addition, the analysis systems is able to generate wire-rate traffic and automatically analyze thousands of traffic flows with comprehensive QoS analysis

Specifically, IxNetwork offers users the flexibility to customize the application to meet a wide range of requirements for testing complex network topologies consisting of thousands of network devices. Millions of routes and reachable hosts can be emulated within the topology. IxNetwork also provides users with the ability to customize millions of traffic flows to stress the data plane performance. Sophisticated configurations can be created using powerful wizards and grid controls in the graphical user interface. With its enhanced real-time analysis and statistics, IxNetwork is capable of reporting comprehensive protocol status and detailed per-flow traffic performance metrics.

As network functions continue to be aggregated into devices, it becomes increasingly important to consider security and encapsulation protocols, such as NAC, PPP and L2TP. IxNetwork provides the ability to authenticate emulated clients and to establish broadband sessions. Traffic can then be encapsulated over the tunneling protocols.

Conclusion

The demand for highly scalable test solution to meet realistic test needs is real and strong. Ixia's next generation hardware and software have proved that it can face the real-world challenges and deliver the scalability and performance that can meet or exceed your expectations.

back | top of page | back to white papers ]