Library: Test Plans

Edge Router Test Plan

• Introduction to Edge Router Testing
• 1. BGP Conformance Test
• 2. BGP Route Dampening Test
• 3. BGP Multi-Homing Test
• 4. BGP Graceful Restart Test
• 5. OSPF Route Capacity Test
• 6. OSPF Route Convergence Test
• 7. OSPF Control and Data Plane Integration Performance Test
• 8. L2VPN Label Advertisement and Withdrawal Test
• 9. L3VPN VRF Isolation and Scalability Test
• 10. L3VPN Ingress/Egress Forwarding Performance Test
• 11. VPLS MAC Address Cache Capacity Test
• 12. Multicast Join/Leave Latency Test
• 13. Multicast No Drop Throughput Test
• 14. Multicast Group Capacity Test
• 15. PIM-SM Rendezvous Point (RP) Functionality Test
• 16. PIM-SM Last Hop Router (LHR) Functionality Test
• 17. PIM-SM Reverse Path Forwarding (RPF) Functionality Test
• 18. PIM-SM Rendezvous Point Tree (RPT) to Shortest Path Tree (SPT) Switchover Functionality Test
• Conclusion

Introduction to Edge Router Testing

Over time, the edge router has evolved from a simple device focused on routing to a multifunctional, multi-service all-in-one box. The functions and features packed into this single box are unprecedented and the trend to complexity is likely to continue. The technology that makes such sophistication possible is Multiple Protocol Label Switching (MPLS).

MPLS was initially developed for the core network with the intention of switching instead of routing packets. It has since become the unifying technology that forms the foundation of a converged network (See Figure 1). As Ethernet and MPLS continue to gain ground in both the access and metro space, the demand for more functions integrated into the same edge router becomes even greater.



Testing an edge router to its fullest extent, as you can imagine, is quite a challenge. Network Equipment Manufacturers (NEMs) want to make sure the edge router can work as designed and function flawlessly under all circumstances. Service Providers (SPs) want to make sure the edge router can successfully aggregate and convert multi-service multi-technologies to the converged network. Enterprise users want to make sure the edge routers deployed in their network can support Triple Play services seamlessly and can also scale for future growth. This test plan is written to provide basic reference test cases for users in all categories to test such a multifunctional and multi-service all-in-one box.

Testing methodologies for a given Device Under Test (DUT) generally include conformance tests, functional tests, interoperability tests, performances and stress tests. Conformance is the starting point for validating basic functionality in both positive and negative tests. It is an important tool to verify how a DUT complies with specific protocol standards. Conformance test tools perform their tests as a dialog: they send stimulus packets to the router being tested, receive the packets sent in response, and then analyze the response to determine the next action to take. This methodology allows conformance test tools to test complicated scenarios much more intelligently and flexibly than achievable by simple packet generation and capture devices.

Functional and interoperability tests are more focused on specific DUT features in a more realistic setup. While conformance provides assurance of protocol conformity to certain standards or RFCs, the setup or test topology is usually limited and less realistic. Functional and interoperability tests, on the other hand, allow users to expand the test coverage from simple lab setup to more realistic real world configuration. Performance and stress tests take the DUT to next level -- given that a DUT is working flawlessly in a reasonable environment under various test scenarios including invalid and inopportune events, the next question obviously is how well the DUT will work under different scenarios with increasing traffic load. There are many performance metrics one would like to collect and stress scenarios that one would want to try out before the device is deployed in the field and required to support revenue generating traffic.

The rest of this document follows the test methodologies outlined above with detailed coverage of various technologies that may be implemented in the DUT. Conformance testing is illustrated first, with sample configuration and test results. In a strict sense, every single protocol implemented on the DUT should have gone through a conformance test first before beginning other testing.

The remainder of this document is structured around testing technologies from a functional and interoperability perspective, as well as assessing performance in typical and stressed circumstances. Keep in mind that these samples tests are provided to illustrate a testing methodology, not as complete or final testing plan for a DUT.

Section I: Protocol Conformance Testing

Technically, every single protocol implemented in the DUT should go through rigid conformance testing first. The need for protocol conformity can never be over emphasized. Your device may function just fine without conformance tests in most circumstances, but you will never be 100 percent sure that it will work as expected at all times with every device in all topologies. There are always corner cases where interpretation of the protocol varies with vendors. Conformance testing provides a rigid standard by which to interpret the protocols in the right way.

As stated earlier, since there are one or more conformance suites for every standards or RFCs published, it's impossible to cover every single one of them here. Instead, we will use BGP conformance to illustrate the methodology and test results. All other conformance test suites work exactly the same way: from configuration setup to test results report (verdict and explanation).

1. BGP Conformance Test

Objective
Verify the DUT's compliance with the following capabilities defined in various BGP specifications: RFC 1771, RFC 1772, draft-ietf-idr-bgp4-12, draft-ietf-idr-bgp4-17

Setup
A minimum of two test ports are required from the test tool to the DUT - one for stimulus packets and one for response packets. Ixia's IxANVL conformance test suite is run from a Linux workstation either connected directly to the DUT, or via Ixia test hardware (sees Figure 2). IxANVL emulates various BGP topologies, depending on the configuration of each test case.



Input Parameters
Two sets of parameters are required prior to running conformance test, one for test tool configuration and one for DUT configuration. The test tool configuration describes the interface and protocol configuration of the tester, while the DUT configuration describes the BGP commands sent to the DUT using Expect scripts (see Table 1).

Parameters	Description
Test Tool Configuration	Tester Test IP Addresses, DUT IP Address, BGP protocol parameters (AS number, authentication, and timer values)
DUT Configuration	BGP features (TOS Routing, timers, AS number, peer configuration, etc.) via Expect scripts

Methodology
For BGP conformance testing, a number of test cases are run against the DUT based on the direct interpretation of various BGP RFCs.

1. Enter parameters to describe both the Conformance Tester and DUT configuration.
2. Select all or a set of test cases to run (see Figure 3).
3. Run the conformance tests from the user interface or in batch mode via command scripts, reconfiguring the DUT as required between test cases to match the test setup.

Results
A listed number of tests passed/failed, including reasons for failed cases. IxANVL also keeps the history of each pass or fail test case in the Test Journal (Figure 4).



Recommended Conformance Test Coverage

	IxANVL Test Suites	Target Protocols	Reference Specification	Test Case Count	Required Test Interfaces
IP Test Suites	IPv6 Core	IPv6 IPv6CP	Interfaces, RFC 2460, 2464 RFC 2472	99 17	2 1
	IPv6 Advanced	ICMPv6 NDP AutoConfig V6oV4 PMTU IP Router Alert	RFC 2463 RFC 2461 RFC 2462 RFC 2893, 2529, 3056 RFC 1981 RFC 2711	38 203 28 92 10 13	2 2 2 2 2 1
	IPv4	IPv4	RFC 791, parts of 1122, 1812	70	2
		ICMP	RFC 792	32	2
		DHCP Client	RFC 2131	92	2
		DHCP Server	RFC 2131	74	2
Routing	IP RIP	RIP	RFC 1058, 2453	50	2
		IPGW	RFC 1812, 1122	18	2
	RIPng	RIPng	RFC 2080	63	2
	OSPF Core	OSPF	RFC 1583, 2328	312	3
	OSPF Extensions	Opaque LSA, NSSA, DB Overflow	RFC 2370, 3101, 1765	53	3
	OSPFv3	OSPFv3	RFC 2740	325	3
	BGP4 Core	BGP	draft-ietf-idr-bgp4-26	187	3
	BGP4 Extensions	BGP-OSPF, Communities, Route Flap Damping, Route Reflection, Route Refresh, Confederations	RFC 1403, 1997, 2439, 2796, 2918, 3065	118	3
	BGP Plus	BGP+ with IPv6	draft-ietf-idr-bgp4-26, draft-ietf-idr-rfc2858-bis-06, RFC 2545	199	3
	ISIS	ISIS	RFC 1195, ISO/IEC 10589: 1992(E)	213	2
	ISISv6	ISIS-v6	draft-ietf-isis-ipv6-05	205	2
MPLS	MPLS	Label Encapsulation	RFC 3032	59	2
	RSVP-TE	RSVP-TE	RFC 3209	90	3
	LDP	LDP	RFC 3036	335	3
	L2VPN (PWE3)	PWE3-Control	draft-ietf-pwe3-control-protocol-17	67	2
		PWE3-Encapsulation	draft-ietf-pwe3-hdlc-ppp-encap-mpls-05, draft-ietf-pwe3-atm-encap-09, draft-ietf-pwe3-ethernet-encap-10	70	2
	VPLS	VPLS	draft-ietf-l2vpn-vpls-ldp-08	58	4
	L3 VPN	L3 VPN	draft-ietf-l3vpn-rfc2547bis-03	101	3
Multicasting Test Suites	IGMP	IGMPv2	RFC 2236	49	2
		IGMPv3	RFC 3376	155	2
	DVMRP	DVMRP	draft-ietf-idmr-dvmrp-v3-07	66	3
	PIM	Dense Mode	draft-ietf-pim-v2-dm-04	162	3
		Sparse Mode	draft-ietf-pim-sm-v2-new-07	268	3
	MLD	MLDv1	RFC 2710	98	2
		MLDv2	RFC 3810	203	2
TCP Test Suites	TCP Core	TCP	RFC 793, 1122, 2460	179	2
	TCP Advanced	Slow Start, Congestion Control, PMTU Disc, MD5	RFC 2001, 2581, 1191, 2385, 2463, 1981	48	1

Note: The draft RFCs listed in this table should be updated regularly to reflect the ongoing development of standardization process.

Section II: Underlying Routing Protocol Testing

The underlying routing protocols in an edge router provide the foundation for all other protocols and technologies, including MPLS and applications built on top of the MPLS. There are several families of protocols in this area. First is the distance vector family of protocol, most notably RIP and Cisco proprietary EIGRP, which utilize hop counts as cost of routing metrics. Simplicity is the most salient feature of this type of protocol. Due to its severe limitation in convergence time, however, these protocols are more often seen in the LAN area or remote VPN sites that have limited access to the POP.

The second family of protocols is known as link-state protocols with OSPF and ISIS as the outstanding representatives. As the name implies, link state protocols exchange and propagate each and every link status to the neighbors, which include link type and link metrics that are directly associated with the link speed. After topology convergence, all routers have an identical view of their own area. Optimality, in route selection is possible due to the granular link state information with this type of protocol. The network as a whole is more responsive to local changes and convergence is also much faster compared to their distance vector counterpart. To improve scalability, the protocol has built-in areas or levels to allow hierarchical design. In more recent years, OSPF and ISIS have been extended to include Traffic Engineering (TE) parameters for advertisement and propagation. Indeed, support of TE forms the foundation of high layer applications that require strict QoS and SLA guarantees.

The third family of routing protocols is the inter-autonomous area routing protocol represented by BGP. Contrary to the intra-autonomous routing protocols such as RIP and OSPF, BGP runs across different autonomous areas to glue many otherwise disconnected or isolated islands into an internet (hence BPG is also known as the Internet protocol). BGP is based on a distance vector algorithm for simplicity to allow the least routing dependency between ASes. To improve scalability, router reflector and confederation exist in the protocol to reduce mesh peering requirement among routers. Note also, BGP has been augmented in recent years with Multiple Protocol (MP) extensions to allow MPLS applications to piggyback label request and assignment with the route advertisement.

Additionally, all the three families of protocols have their counterparts in IPv6. While there are not many new requirements in these protocols when dealing with IPv6 other than the fact the address field is four times larger, the performance in both the control plane and the data plane is noticeably degraded. Most of the edge routers today are capable of handling both IPv4 and IPv6 simultaneously with different operation modes.

This section provides example test cases to test basic routing functions and performance. By no means are they enumerative and complete; they serve as reference for designing your own test cases. The section begins with a few BGP function tests, followed by some performance tests for OSPF, which can be easily adapted to cover similar tests for BGP and ISIS. The tests can also be expanded to cover IPv6 protocols such as to test BGP4+, OSPFv3, and ISIS for IPv6.

2. BGP Route Dampening Test

Objective
This test verifies a router's policy for BGP dampening of unstable routes that have experienced periodic withdraw and re-advertisement. This test confirms the proper operation of BGP dampening timers.

Setup
This test requires two test ports – one to transmit traffic to both the stable and the unstable routes and the other one to emulate BGP routes with flapping capabilities. At least two prefixes are advertised, one of which is stable and the other which is unstable. The device's BGP dampening policy is tested for proper suppression and reuse.

Input Parameters

Parameter	Description
Penalty	Increasing number assigned to a route every time it flaps.
Half Time	Amount of time that must pass to reduce the Penalty by half. It determines how fast the penalty timer decays.
Suppress Limit	Numeric number compared to the Penalty whenever a route is received. If the Penalty is larger than the Suppress Limit, the route is suppressed (i.e., no longer advertised to BGP peers).
Reuse Limit	Numeric number compared to the Penalty whenever a route is received. If the Penalty is less than the Reuse Limit, the route will be unsuppressed or re-used (i.e., re-advertised to BGP peers).

Methodology

1. Configure a single BGP session on Test Port 2 (BGP port) and advertise two different prefixes, one that is stable and the other to represent unstable (flapping) behavior.
2. Configure the dampening/damping policy on the DUT. The objective is to test timed parameters, such as Penalty and Half Life for route damping.
3. Start a BGP session on Test Port 2, and ensure both the stable and the unstable routes are advertised successfully to the DUT.
4. Configure Test Port 1 (transmit port) to send traffic to both advertised routes. Set the tester to track by destination IP address. Two flows will be created and per-flow stats will be tracked and reported in real time. Send the traffic at a low frame rate (e.g., 5000 fps for each flow) for easy observation of any route dampening effect.
5. Carefully design a flap scheduler to start the dampening process. For easy prediction of route suppression time, flap the route quickly (e.g., withdraw in 2 seconds and re-advertise in another 2 seconds). Start the traffic and activate real-time graphing tools to monitor the port receiving rate. Flap the route just enough times to see the route being suppressed and reused in a reasonably short time. See Figure 6 for an example of a flapping scheduler using Ixia's IxNetwork Test Scheduler. The unstable route is flapped three times; the second time it flaps caused the DUT to suppress the route.

Results
This test results display the traffic received on the BGP test port where BGP route flap occurred. Figure 7 shows sample test results of the dampening policy being applied correctly only after the second route flap. Note that the exact behavior of route dampening is dependent upon the input parameters. In this particular example, half-time is set as 1 minute; suppress limit is set as 2000; and reuse limit is set as 1000. The penalty is the default value (1000). As indicated in the following diagram, route suppression starts immediately after the second flap. The third flap will raise the Penalty to approximately 3000. The route will remain being suppressed until approximately two times half-time (120 seconds) when the Penalty time finally decays to a value lower than reuse limit.

3. BGP Multi-Homing Test

Objective
This test verifies the influence of the BGP optional non-transitive MED (Multi-Exit Discriminator) attribute on several paths established with EBGP. Traffic is sent from multiple peers (established via single test port or different test ports), each containing a route with a pre-configured MED metric, and the impact on IP forwarding is observed.

Setup
Even though two test ports are sufficient for proof of concept testing for this test case, we recommend you use three test ports so the traffic forwarding impact can be better observed when manipulating the MED value. The first test port is to transmit frames destined to the same prefix that have been advertised by all BGP peers (via a single test port or different test ports). The other two (or more) ports will be EBGP peers to the DUT that advertise exactly the same prefix but with different MED value. These ports also act as traffic receiver for verification. As traffic is sent from Test Port 1, the DUT will select the best path corresponding to lowest MED value and forward the traffic to the right BGP peer/port. Alter the MED value on the fly and observe traffic being switched to a new port that now has the lowest MED value. Ixia's IxNetwork can be used to execute this test.



Input Parameters

Parameter	Description
MED Metric	A 32-bit integer representing the non-transitive BGP metric.
Optional Non-transitive	An optional capability in BGP that is commonly used to influence route decisions from EBGP into a network.

Methodology

1. Configure an EBGP emulation on Test Port 2, and 3 as applicable. Each port is configured with one or more external BGP peers and each peer advertising the same prefix but a different MED. Figure 9 shows a sample configuration of the MED metric using Ixia's IxNetwork BGP emulation module.
2. Verify on the control plane that all routes are in the BGP table of the DUT with the configured MEDs. Verify also the best path shown in DUT corresponds to the lowest MED value. It should also be the default data forwarding path.
3. Configure Test Port 1 with a traffic destined to the one prefix advertised by all peers.
4. Begin transmitting traffic on the data plane to obtain results.
5. Change MED value on the fly and observe new traffic results.

Results
Verify traffic flows on Test Port 2 or 3 as applicable, and ensure traffic is only received on the port that has the lowest MED value. Change the advertised route to a different MED value and observe traffic being switched over to another receiver port that now has the lowest MED value. Reset the MED back to the original value and ensure traffic again flows along the original path. You can repeat the process a few times and ensure the results are repeatable and consistent.

4. BGP Graceful Restart Test

Objective
This test is used for verifying BGP graceful restart capabilities. The test validates graceful restart functionality using traffic flows. For maximum effect, two separate BGP sessions are used, each with one distinct prefix, one should be configured with graceful restart and the other one should not. By flapping both BGP peers and observing the traffic impact on both flows, one can easily conclude if BGP graceful restart works as expected.

Setup
This test requires a minimum of two ports — one to transmit and the other to receive and represent a BGP neighbor adjacencies. Two EBGP sessions will be established between the Test Port 2 and the DUT: one session will advertise graceful restart capabilities which include restart and stale timers, and the other does not. Each EBGP peer will contain a single prefix to represent an IP destination. When no BGP session flapping occurs, traffic will enter the DUT from Test Port 1 and will be received without loss by Test Port 2. The test then introduces the session flapping on both the graceful restart session and the non-graceful restart session. You can then view the traffic continuity using the graceful restart session while seeing continuous drop on the non-graceful restart session. Ixia's IxNetwork can be used for flap scheduler and BGP session capabilities with graceful restart.

Input Parameters

Parameter	Description
Restart Time	The estimated time, following a restart operation, allowed re-establishing a BGP session.
Stale Path Time	The maximum time to maintain stale paths of a gracefully restarted peer. All stale paths are deleted after the expiration of this value.

Methodology

1. Test Port 2 emulates the control plane for BGP and establishes two separate BGP sessions with the DUT. The BGP graceful restart function is configured on both the first BGP session and the DUT. See Figure 12 for a sample setup of a graceful restart on the Ixia test port.
   
   Figure 12 - Ixia graceful restart configuration
2. Confirm that the DUT forwarding table has learned both prefixes via BGP sessions. Program Test Port 1 to send continuous traffic at given rate (e.g., 10000 frames/second) to both prefixes.
3. Use the Ixia graphing tool to monitor the receiving rate of Test Port 2. No frame loss should be observed before the flap.
4. Design a test scheduler to flap both BGP sessions. The session flap timer should be less than the graceful restart timer to observe traffic continuity.
5. Manipulate event flap on different sessions with a different time and observe the traffic rate change. Make sure you can explain the results based on graceful restart timers set on both the DUT and the first BGP session.
   
   Figure 13 - Ixia Test Scheduler allows complex flap scheduling

Results
The primary goal of this test is to verify traffic flows in BGP even though adjacencies flap. When both sessions are under flap, only the session that has had the graceful restart enabled will continue to receive traffic. The other session will have no traffic when the session is down. This is evident in the following result diagram. A total of 10,000 frames per second were sent from Test Port 1 to both prefixes that were advertised through two separate sessions. When flap occurs, only 5,000 frames-per second were received. When both sessions resume normal, so does the traffic throughput.

5. OSPF Route Capacity Test

Objective
This test determines the maximum number of routes that an OSPF DUT can sustain at a single time. This scalability test is designed to help network and test engineers to test capacity and understand network limitations before actual deployment of new network elements and services.

Setup
The test requires two Test Ports – one to transmit traffic and one to receive. The transmit direction of traffic is unidirectional. Test Port 2 is used to advertise the OSPF network topology and routes, while Test Port 1 sends traffic to verify the advertised topology (Figure 15). During the test, Test Port 2 gradually increases the number of advertised routes until the maximum sustainable route capacity can be determined. Ixia's IxScriptMate application can be used to configure, control, and execute this test. IxScriptMate also provides comprehensive test results showing frame loss percentages based on the ability to forward under maximum route capacity.



Input Parameters

Parameter	Description
Max Rate	Rate at which frames will be sent to advertised routes
Tolerance	Percentage of traffic loss tolerance
Route Step	Number of routes to increase per iteration
Number of Routes	The number of prefixes to generate at the beginning of the test.
Advertise Delay Per Route	The maximum time in seconds the router is allowed to absorb the advertised route. This number is multiplied by the number of routes to calculate the "Max Wait Time."

Methodology

1. Test Port 2 advertises the initial number of routes defined by the parameter "Number of Routes".
2. After passing the "Max Wait Time" (determined by "Advertised Delay Per Route"), Test Port 1 sends traffic targeting each advertised route behind Test Port 2. The traffic throughput rate is set by the parameter "Max Rate".
3. Test Port 2 verifies packets received within the defined loss "Tolerance".
4. Test Port 2 advertises more routes increased by the amount defined by "Route Step".
5. Repeat step 2 through step 4 until port 2 receives no packets or packet loss is above the "Tolerance" level.
6. Repeat the same test with different packet size.

Results
When the test completes and the tolerance has been exceeded, the test results will show the maximum number of routes learned by the DUT. Figure 17 shows the results page created by IxScriptMate. The results are broken down per frame size and identify the resulting numbers for "max routes verified," "total loss percentage," and "tolerance." The "Max Routes Verified" value shows the maximum number of routes that could be sustained at that particular traffic rate and frame size. This test can be executed manually as well but automation with IxScriptMate helps to simplify and speed the testing process.

6. OSPF Route Convergence Test

Objective
This test verifies the ability of a router to switch between preferred and less-preferred routes when the preferred routes are withdrawn and re-advertised. The test calculates convergence by taking an average convergence latency of multiple topological changes.

Setup
This test uses three test ports – one to transmit and two to receive (see Figure 18). Both receive ports emulate OSPF networks. The transmit direction of traffic is unidirectional. T he DUT must have three ports utilized with two enabled for OSPF. All three ports should be configured for IP and have unique subnets in which to communicate with the Test Ports. Ixia's IxScriptMate application can be used to configure, control, and execute this test.



Input Parameters

Parameter	Description
Max Rate	The rate at which frames are transmitted. This is the percentage of the maximum theoretical frame rate.
Number of Routes	The number of prefixes to generate at the start of the test.
Number of Withdrawals	The number of prefixes to withdraw. The test can be executed with partial withdrawal or complete withdrawal.
Advertise Delay Per Route	The maximum time, in seconds, to allow the router to absorb each route. This time is multiplied by the number of routes to calculate the "Max Wait Time" – the amount of time the test will wait for the entire topology to stabilize.

Methodology
This methodology can be executed manually or by script. The key to determining an accurate convergence time is to understand the DUT capabilities and properly manipulate the test parameters.

1. Test Ports 1 and 2 advertise the same OSPF topology and routes with different metrics. The path via Test Port 1 will be used as the preferred route, while the path via Test Port 2 will be used as the alternate route.
2. After the "Max Wait Time," the Tx port sends traffic to target all advertised routes. The DUT should route the traffic via the preferred routes to Test Port 1.
3. Routes are withdrawn from Test Port 1 (the preferred path). Traffic should reroute to arrive at Test Port 2 (the alternate path).
4. Measure the timestamp T1 of the last packet targeting a specific route delivered on the preferred path. Measure the timestamp T2 of the first packet targeting the same route arriving via the alternate path.
5. Calculate the convergence time for one specific route = T2 – T1.
6. Repeat step 4 and 5 to obtain convergence time for all withdrawn routes. Calculate average convergence for all routes.
7. Repeat the test with different frame sizes

.

Results
The test results provide an average convergence time for all routes. Figure 20 displays sample results for the automated OSPF convergence test in IxScriptMate. In addition to convergence time, this test also indicates the amount of lost packets caused by the convergence.

7. OSPF Control and Data Plane Integration Performance Test

Objective
This test assesses the DUT's data plane performance (throughput and latency) in forwarding bidirectional traffic with respect to different frame sizes, when the control plane is pre-defined with fixed number of peering routers each with a number of prefixes.

Setup
The test requires at least two Test Ports – each to emulate pre-defined number OSPF routers and advertise a given number of prefixes. The test then generates bidirectional traffic with binary search for no-drop throughput and calculates the packet latency. Ixia's IxScriptMate can be used to run this test.

Input Parameters

Parameter	Description
Max Rate	Maximum traffic rate to the advertised routes. Used by binary search for no-drop throughput
Number of Routers	The number of OSPF routers emulated by each port
Number of Routes	The number of routes to be advertised by each emulated routers
Loss Tolerance	The percentage of packets loss that is acceptable before proceeding with next iteration

Methodology

1. Configure the number of test ports used for this test. A minimum of two test ports is required for this test.
2. Input the total number of routers emulated by each port and the total number of prefixes to be advertised by each emulated routers.
3. Specify the frame sizes to be used for traffic generation and verification. Input the loss tolerance for acceptable packet loss ratio.
4. Start the test and monitor the progress of the test. The test should proceed in sequence to resolve MAC addresses first on all ports under test. It should then build peering adjacency with all routers emulated, and finally, the test should automatically perform a binary search for throughput on all advertised routes. The test provides results of no-drop throughput and latency as performance matrix.
5. If more than one packet size is specified, the test will proceed with the next packet size on the list.
6. Specify a separate control plane setup with a different number of routers being emulated and different number of routes to be advertised, and then repeat the same test. Notice possible performance degradation when both the number of peering routers and the number of advertised routers increase.

Results
This test should auto discover the no-drop throughput and the associated latency stats under a given control plane setup. The results should be easily viewable in graph or a raw packet stats. Figure 23 shows an example of test results using Ixia's IxScriptMate. It displays both the aggregated per port stats, as well as the per emulated router stats. The detail stats page also shows a per iteration number when the program attempts to auto discover the no-drop throughput.

Section III: MPLS and Applications Test

The MPLS technology was initially designed for the core network with the intention to switch packets instead of routing every single packet across the core. Packets entering the network are classified at the edge for assigning proper labels with the help of signaling protocols such as LDP or RSVP-TE. Once labels are assigned, packets that have similar properties, such as prefix length and value or TOS value on the packet header, are directed towards exactly the same Label Switch Path (LSP). Never before was this possible with traditional routing protocols; in fact, every packet was examined and paths identified in all routers, core or edge routers alike, in a completely connectionless and datagram delivery fashion. Essentially, MPLS allows packets of certain characteristics to flow to a pre-determined path with negotiated QoS guarantee. This strategy makes it possible to provide QoS or SLA on a traditionally best-effort based IP network in a way that was only achievable through connection oriented technologies such as Frame Relay and ATM in the past. With MPLS, it's possible to deploy Ethernet everywhere, including access, metro and even core networks. In fact, MPLS has become the de facto technology for a converged network that is capable of providing triple-play services.

The primary applications for MPLS include VPN and traffic engineering. There are various flavors of VPNs, and, in general, they are known as L2VPN and L3VPN. The L2VPN was created to provide point-to-point (P2P) connection across the MPLS core in much the same way as an IP connection was established across an ATM core network as documented in RFC1483. This P2P connection simulates a pseudowire that connect two isolated VPN sites (hence the term, "pseudowire emulation"). The connection was built from Layer 2 standpoint and thus supports many dissimilar technologies such as PPP, HDLC, FR and ATM; in addition to the standard Ethernet and VLAN.

One of the variant of L2VPN application is known as Virtual Private LAN Service (VPLS). This type of VPN binds multiple L2VPN pseudowires (Ethernet and VLAN only) to form a virtual Ethernet switch. The VPN application technology has made it possible to expand the concept and power of Ethernet connectivity that has been traditionally limited to the LAN area to across the metro and core network. To improve the scalability of the VPLS, a Hierarchical VPLS (HVPLS) was proposed and has gained tremendous success over the past few years.

In contrast to the L2VPN, there is L3VPN application, which is based on RFC 2547bis. L3VPN works in quite differently than L2VPNs and are one of the first MPLS applications to enjoy successful deployment in large scale Service Provider networks. Since these VPNS are Layer 3-based, packets are routed through the MPLS core, with the help of MPLS LSP. Customer VPN sites form routing peers with the Service Provider PE routers and expose routing information to the Service Provider. Before packets are delivered over the MPLS tunnel (or LSP), they are pre-pended with L3VPN information and another label to uniquely identify the VPN sites. The provider PE router generates and stores a separate routing table of each VPN (known as Virtual Routing Forwarding instance (VRF). Typical application of L3VPN include for example a wholesale Service Provider that provides connections for two or more retail Service Providers; or a large enterprise customer that wants connectivity among all sites in geographically separated locations.

The traffic engineering piece of the MPLS technology was crucial for making it possible for Ethernet to extend beyond the LAN. A dedicated LSP with certain QoS requirement can be negotiated and signaled through the MPLS core beforehand so traffic following through the tunnel can enjoy the QoS guarantee. In addition, Fast Reroute (FRR) enabled protection and traffic restoration in sub 50ms, a target that has only been delivered by SONET/SDH-based devices to date. Both link and node protections are possible with MPLS.

This section of the test plan focuses primarily on major MPLS applications, including L2VPN, VPLS and L3VPN. A few test cases are used to illustrate test methodology with test results. By no means are the test cases listed here enumerative and complete, we encourage you to expand the test to test areas that are not covered here.

8. L2VPN Label Advertisement and Withdrawal Test

Objective
This test verifies the proper operation of a PE router that supports L2VPN PWE to forward and stop forwarding traffic over Martini pseudowires that have been advertised and subsequently withdrawn from its forwarding tables.

Setup
This test requires two ports at a minimum - one to simulate a PE router and another to simulate a CE router. Multiple pseudowires are established between a single CE and one or more emulated PEs. Ixia's IxNetwork with routing protocol emulations can be used to construct the topology and fulfill the control and data plane traffic requirements for this test.

Input Parameters

Parameters	Description
Peer Address	Target IP address on the DUT for the Martini LDP session - usually the device's loopback address
MTU	The maximum frame size allowed over the established pseudowires; must be identical at both sides of the pseudowire
VC ID	Virtual Circuit ID providing a unique identifier for the L2 VPN
VLAN ID	Unique traffic identifier for CE traffic at the entry point to the MPLS domain

Methodology

1. Test Port 1 establishes IGP (OSPF or ISIS), LDP Basic, and LDP Extended Martini sessions with the DUT. At least two PW VCs must be established to execute this test. A VC type of VLAN or Ethernet (per VLAN or per port bridging) should be used if Ethernet is the physical interface type.
2. Start all protocols involved and ensure they all reach stable states. PW VCs under test should show a state of "up" as seen below when all parameters are configured correctly (including MTU size).
   
   
3. Test Port 2 emulates CEs and MAC hosts behind each CE.
4. Build bidirectional traffic flows and ensure proper forwarding of traffic in both CE->PE and PE->CE directions. Use proper the tracking mechanism, e.g., label on the PE->CE direction and VLAN ID for CE->PE direction to ensure each and ever PW VCs are working as desired.
5. Take down one of the VCs by inducing a withdrawal of the Martini label. With the Ixia test tool, this can be done by simply deselecting the VC Range checkbox in the GUI as shown in the result figure (Figure 25).
6. Pseudowires can be withdrawn and re-advertised dynamically to ensure traffic is either being stopped or is successfully passing through depending whether or not there is a label for it.
7. Use a test scheduler to automate the flap of the VC range repeatedly and test the DUT's robustness in handling VC withdrawal and re-advertisement as demonstrated below:
   
   
8. Further test can be conducted by adding more PEs each with increased number of VCs.

Results
The expected result is decreased or zero traffic throughput as a result of PW flapping. By monitoring the per-flow stats, including the received traffic rate on Test Port 2, the result of label withdrawal can be seen in terms of traffic loss. In Figure 25, it is seen that the received traffic rate for both PE->CE and CE->PE directions decreases to zero when a withdrawal occurs and frame delta keeps increasing, indicating a total packet loss. This test can be scaled simply by adding more emulated PEs and/or increased number of VCs for each PE to characterize the behavior of large numbers of pseudowires in a dynamic environment.

9. L3VPN VRF Isolation and Scalability Test

Objective
This test verifies a DUT's capabilities for L3VPN VRF isolation with multiple CEs and PEs, each with similar (duplicated) or different prefix advertisements. The test utilizes EBGP on the CE side and MP-iBPG on the PE side to advertise a fixed number of prefixes. The test then creates multiple traffic streams on both CE->PE direction and PE->CE direction. Each traffic stream will send packets destined to hosts on the same VRF, as well as hosts on different VRFs with intention to generate cross-talk traffic. The DUT should only deliver traffic to the right VRF that belongs to the same VPN and drop packets destined to the different VRFs. Scalability can be achieved by increasing the number of CEs or PEs and/or the number of VRFs. Ixia's IxNetwork can be used to perform this test.



Setup
A minimum of two test ports is required to perform this test. One port will emulate multiple CEs with each CE part of a different VRF. The second port will emulate multiple PEs, each with a number of different VRFs. The DUT is configured with a trunk 802.1q VLAN interface with a separate VLAN for each VRF instance facing the CE port.

Input Parameters

Parameters	Description
Route Distinguisher (RD)	Route Distinguishers are used primarily to distinguish routes especially in overlapping VPN scenarios. The Route Distinguisher is made up of two parts, the "Admin part" and "AS number."
Route Targets Import/Export	Route targets are appended to prefixes to establish destination VRFs and for import/export policies. This is applied to VPN prefixes to allow for import to other receiving VPN instances.
IGP/MPLS protocol	IGP is the interior gateway protocol used to distribute the loopback address, and the MPLS protocol is used for signaling the LSP to destination loopback.
Number of PEs	The total number of PE routers emulated by a single physical test port
Number of VRFs	The total number of VRF (or VPN sites) emulated by each emulated PE router

Methodology

1. Configure Test Port 1 to emulate P and PEs. Select the IGP and MPLS signaling protocol (see the example below).
   
   
2. Configure both the emulated PE and VRF information per example below:
   
   
   
3. Configure Test Port 2 to emulate multiple CEs with EBGP as the peering protocol as shown below:
   
   
4. Start protocols, and ensure all protocols, including MPLS IGP and signaling protocols for both CE peering EGBP and PE peering MP-iBGP, are up and enter stable state.
5. Construct traffic streams on both CE->PE and PE->CE direction so that traffic sourcing form one VRF are destined to both the same VRF and different VRFs on the other port. Set up proper tracking for traffic going to both directions. We recommend that you track by MPLS label on the PE->CE direction while tracking by VLAN ID on the CE->PE direction.
   Note that labels for cross talk traffic from PE->CE direction can't be resolved and typically tester won't be able to generate traffic but should have proper warning as indicated below. Cross talk traffic from CE->PE side should be sent as normal but should be dropped by the DUT.
   
   
6. Scale the test by increasing the number of CE/PEs emulated and/or the number of VRFs being tested.

Results
The success of the result depends on the desired size of the network, i.e., number of PEs and number of VRFs. A positive result will be made up of received flows for traffic at specified rates destined to the right VRFs that belong to the same VPN while no flows are received for the cross-talk traffic. Figure 27 shows two CEs that are sending traffic successfully for both VRFs of the same VPN at the PE side but no traffic for the other two VRFs of a different VPN. For robustness testing, you can simply increase the number of PEs and/or the number of VRFs at each emulated PEs.

10. L3VPN Ingress/Egress Forwarding Performance Test

Objective
Determines the maximum rate at which a Label Switched Router (LSR) configured as a L3 VPN Provider Edge (PE) node can strip/pop or apply/push MPLS labels to and from incoming IP packets. The resulting throughput of this operation is then measured to the point of no loss.

Setup
This test requires two ports -- one to simulate the PE router and another to simulate the customer edge (CE) router. Ixia's IxScriptMate can be used to execute this test.

Input Parameters

Parameters	Description
OSPF Parameters	Area ID and network type, Prefix/Mask information
BGP parameters	AS number, Peer address, Prefix/Mask information
# Routes per VRF	Defined amount of routes per VRF to advertise. The number of routes will be split between PE->CE and CE->PE directions.
Traffic rate	Desired rate to send traffic to be popped or pushed

Methodology

1. The port simulating the PE/P router establishes an IGP session with the DUT, and advertises the loopback address of the simulated PE router.
2. Bi-directional LSPs are established using RSVP-TE or LDP.
3. A multi-protocol internal BGP (MP-iBGP) session is established between the DUT and every emulated Pes, and VRF or VPN labels are assigned for each advertised routes.
4. The port simulating the PE router transmits MPLS traffic at the specified rate; if frame loss occurs, the transmission rate is alternately reduced/increased using a binary search algorithm to determine the maximum rate at which the DUT can forward traffic without loss. Latency at the maximum no-loss rate is always measured.
5. Repeat the test with another frame size.
6. Scale the test up by adding more PEs and/or number of VRFs.

Results
The test results given by Ixia's IxScriptMate will reflect and produce successful no-drop throughput, latency of the MPLS traffic popped/pushed, data errors in payload (Figure 30). Detailed stats also give per iteration results for both PE->CE and CE->PE directions (Figure 31).

11. VPLS MAC Address Cache Capacity Test

Objective
This test determines the maximum number of VPLS endpoint MAC addresses a VPLS-enabled DUT can retain. The test allows for emulation of multiple PE's/port as well as VC's/PE, and utilizes a binary search algorithm to find the maximum number of MAC addresses the DUT can store in the VPLS FIB or (Forwarding Information Base).

Setup
A minimum of two Ixia Test Ports are required connected to the DUT. One emulates P/PE routes and the other emulates CE routers (Figure 32). Traffic will be sent from CE ports targeting different MAC hosts behind the emulated PE router. The maximum number of MAC addresses beyond which the DUT will stop learning further will be identified.





Input Parameters

LDP Parameters	Description
Number of PEs	Number of PE routers emulated per physical port
Number of VC's	Number of per PE router
MTU Size	Setting for MTU sizes
First PE Loopback IP	First IP address used to establish LDP session with the DUT
First VC ID	ID of the first VC established with the DUT.
Peer IP Address	Internal loopback address on the simulated CE router
OSPF Parameters	Description
Interface Network Type	OSPF interface network types (Broadcast or Point-to-Point)
Area ID	OSPF area of the simulated CE router

Methodology

1. When the test starts, the CE and PE ports send ARP frames to the DUT so that it can populate its ARP table. It then starts OSPF and LDP on the PE ports. OSPF advertises one route range for each of the simulated PE/VPLS routers.
2. The test waits for the OSPF neighbors to reach Full State and for the LDP Basic Sessions and Targeted Sessions to come up. The CE ports then transmit a series of frames, each bearing the MAC addresses of one of the simulated PE/VPLS routers.
3. The DUT then looks up each MAC address in its VPLS Forwarding Information Base (FIB), which records the associations between MAC addresses and VC LSPs.
4. If it finds the MAC address in the VPLS FIB, the DUT encapsulates the frame as an MPLS packet, pushes an inner VC label and outer tunnel label onto the packet, and sends the packet out on the VC LSP associated with the MAC address. The packet travels over a tunnel LSP to the VC peer (the simulated destination PE).
   
   Note that if the DUT does not find the MAC address in its VPLS FIB, it floods the frame to all the simulated PE devices and other CE ports (except for the CE port that originated the frame) in the VPLS configuration.
5. When it receives a response, it adds an entry to its VPLS FIB recording the MAC address and the VC it arrived on. Any future frames that arrive with the same MAC address will be sent out the corresponding VC.
6. After the test has sent the frames to enable the DUT to populate its VPLS FIB, it begins transmitting load-test traffic, the frames that simulate actual customer traffic, in the CE --> PE direction. It transmits this traffic at the rate specified by Max Rate or Frame /Second, whichever one is selected.
7. After the first iteration is complete, the test checks to see if the DUT flooded any frames. If it did, the test assumes the DUT failed to learn those frames' addresses. The test measures the number of unlearned addresses and compares it with the Loss Tolerance value.
   1. If the number of unlearned addresses was within the Loss Tolerance, the test increases the rate at which it transmits MAC addresses to the DUT (MAC Learn Frames Rate parameter) and runs another iteration.
   2. If the number of unlearned addresses exceeded the Loss Tolerance, the test reduces the MAC address transmit rate and runs another iteration.
8. While the test checks for flooding, it also waits for the DUT to age-out its VPLS FIB. After the age-out time has elapsed, the test again transmits MAC addresses to the DUT, followed by the load-test traffic.
9. After the second iteration, the test again checks for unlearned addresses. It applies a binary search algorithm to calculate a new transmit rate based on the previous rates and the number of unlearned addresses. It goes through this same process for every subsequent iteration until it finds the highest rate at which the DUT can learn addresses within the Loss Tolerance.

Results

1. The results for the applied VPLS Address Cache capabilities for the DUT are shown using three methods, Log files, Graphs, and PDF reports obtained during the test execution.
2. PDF reports show the Aggregate of Tx Frames and Rx Frames for various frame sizes.
   
   Figure 34 - IxScriptMate test graphed results
3. Log File information showing that the DUT was capable of supporting 65,000 Mac addresses in the VPLS FIB.
   
   Figure 35 - IxScriptMate test logged results

Section IV: Multicast Testing

Multicast is an efficient way to deliver traffic from one sender to many potential receivers. There are multiple multicast applications that operate today over the Internet, and Service Providers are searching for solutions to support ever increasing multicast traffic. Thorough and rigorous testing of multicast devices is essential to guarantee appropriate functioning of multicast applications.

The varied multicast protocols in use today range from host protocols (IGMP, MLD) to routing protocols (MOSPF, DVMRP, PIM-DM, and PIM-SM/SSM). Earlier versions of multicast routing protocols suffered from limitations and scalability issues of one type or another. For these and other reasons, the PIM-SM/SSM has emerged as the most popular multicast routing protocol for most Service Providers today.

The multicast test plan below is divided into two sections. Section 1 deals with multicast host protocol testing. IGMPv2 is used in all sample tests, and you can easily modify the test to include other flavors of host protocols. Section 2 deals with multicast routing protocol testing. PIM-SM/SSM is used in the sample tests.

1: Multicast Host Protocol Testing

12. Multicast Join/Leave Latency Test

Objective
This test characterizes the multicast group join/leave latency with respect to frame size and the number of groups in test. The group join latency is defined as the elapsed time between the receipt by the DUT of a group of IGMP/MLD Join requests and the time when the multicast clients begin receiving traffic for the groups they joined. The group leave latency is defined as the elapsed time between the receipt by the DUT of a group of IGMP/MLD Leave requests and the time when the multicast clients stop receiving traffic for the groups they left.

Setup
A minimum of three test ports is required for this test: one to transmit, two to receive. IxScriptMate supports a suite of powerful scripts that can be used to provide RFC 2432 multicast benchmark testing in addition to performance testing for specific topology setups. In particular, IxScriptMate's Multicast Group Join and Group Leave latency tests are used to execute this test.

Input Parameters

Parameter	Description
Frame Size	Multicast traffic frame size
Host Protocol	IGMPv1, IGMPv2, IGMPv3, MLDv1, or MLDv2
Groups to Join/Leave	Total number of groups to join or leave
Group Type	Accumulated or Distributed – the receivers are either joining/leaving the same groups or different groups
Frame Rate	Multicast traffic rate

Methodology

1. Configure the DUT to act as FHR, RP and LHR routers.
2. Set up the Test Ports to simulate IGMP clients and multicast sources only (no involvement in routing).
   
   Test Port 1 will be used as the multicast source and Test Port 2 and Test Port 3 will be used as receivers.
3. Select a lower frame rate (e.g., 10 frames/second) to start the test. Gradually increase the frame rate if needed.
   
   A higher frame rate may cause DUT to congest when the number of multicast groups becomes big, and this may negatively impact latency results.
4. Alter frame size, and the number of groups to join and leave and observe latency result changes.
5. In addition to average latency results, the min and max value should also be examined.

Results
Both multicast group join and leave latency are measured using different frame sizes with a different number of groups under test (10 and 100 as an example). As shown below, frame size has little influence on group join and leave latency while the number of multicast groups under test has dramatic impact on the latency results.

13. Multicast No Drop Throughput Test

Objective
This test is designed to discover the maximum rate at which the DUT can forward multicast traffic with varying frame size.

Setup
A minimum of three test ports is required for this test: one to transmit and two to receive. Further, scalability tests can be executed simply by increasing number of receiver ports. The IxScriptMate Multicast Throughput No Drop Rate test can be used to execute this test.

Input Parameters

Parameter	Description
Frame Size	Multicast traffic frame size
Host Protocol	IGMPv1, IGMPv2, IGMPv3, MLDv1, or MLDv2
# of Multicast Groups	Total number of multicast groups
Throughput Search Method	Binary or linear
Loss Tolerance	Loss Tolerance for the specified search method

Methodology

1. Configure the DUT to act as FHR, RP, and LHR routers.
2. Set up the Test Ports to simulate IGMP clients and multicast sources only (no involvement in routing).
   
   Test Port 1 will be used as the multicast source, and Test Port 2 and Test Port 3 will be used as receivers.
3. Select frame size of interest and start the test.
4. Adjust initial throughput rate for quicker convergence if needed.

Results
Figure 42 illustrates no-drop throughput for various frame sizes, in addition to data latency and errors at the measured throughput.

14. Multicast Group Capacity Test

Objective
This test discovers the maximum number of multicast groups supported by a given DUT.

Setup
A minimum of three test ports is required for this test. Test Port 1 will simulate multicast sources, and Test Port 2 and Test Port 3 will simulate multicast receivers. Further scalability tests can be executed simply by increasing the number of test ports. The IxScriptMate Multicast Group Capacity test can be used to execute this test.

Input Parameters

Parameter	Description
Frame Size	Multicast traffic frame size
Host Protocol	IGMPv1, IGMPv2, IGMPv3, MLDv1, or MLDv2
Step Size of Group Increase	Number of new groups added after each successful iteration
Group Type	Accumulated or Distributed – the receivers are either joining/leaving the same groups or different groups

Methodology

1. Configure the DUT to act as FHR, RP and LHR routers.
2. Set up the Test Ports to simulate IGMP clients and multicast sources only (no involvement in routing).
   
   Test Port 1 will be uses as the multicast source and Test Port 2, and Test Port 3 will be used as receivers.
3. Select frame sizes of interest and start the test.
4. Adjust initial number of groups and the step size for quicker convergence if needed.
   
   Some DUTs may have a port-level capacity as well as chassis-level capacity.
5. Change group type to observe any difference in terms of total number of groups supported by the DUT (per port and per chassis).

Results
Figure 44 and 45 illustrate that the DUT has a group capacity on per port basis. When two test ports or three test ports are used as receiver ports, per port group capacity doesn't change (approximately 175 groups). You can easily verify this behavior by adding more receiver ports. The total group capacity of the DUT would be simply a multiplication of number of ports times per port group capacity.

2: Multicast Routing Protocol Testing

15. PIM-SM Rendezvous Point (RP) Functionality Test

Objective
To ensure that when the DUT operates as a PIM-SM RP router it can register a number of new receivers to one or more specified groups and deliver the traffic from the source to the receivers successfully.

Setup
A minimum of two test ports is required to perform the test. Test Port 1 simulates the First Hop Router(s) (FHR) communicating to the RP router (DUT), as well as simulating multicast source hosts. Test Port 2 simulates the Last Hop Router(s) (LHR) and all the IGMP clients that are requesting to join to different multicast groups. Ixia's IxNetwork can be used to execute this test.

Input Parameters

Parameter	Description
IGP Protocol	IGP protocols used by the FHR to communicate reachability of all multicast sources to the RP router (OSPF or ISIS)
RP address	DUT RP address
# of Emulated Routers at Source port	Number of FHR emulated by the source port
Source Addr count	Number of sources that are generating multicast traffic
# of Emulated routers at Receiver port	Number of LHR emulated by the receiver port when PIM-SM/SSM is used as receiver protocol
# of Groups and Starting Group Addr	Number of groups requested by all receivers and the starting group address
# of Receivers	Number of receivers per emulated LHR

Methodology

1. Configure Test Port 1 to emulate a few FHRs and multiple sources behind each FHR.
2. Select the right IGP for the FHR (OSPF or ISIS) to communicate the location of each source to the DUT.
4. Enable multicast routing on the DUT and configure the DUT as RP router. Make sure the RP address is referenced correctly in the setup of Test Port 1.
5. Configure Test Port 2 as the receiver.
6. Select the right receiver protocols (IGMP or PIM-SM/SSM). Ensure multicast group addresses entered match the group addresses set on Test Port 1.
7. Start protocol emulation on all interfaces involved. Ensure all protocols (IGMP, PIM-SM/SSM, OSPF) are entering stable state.
8. Configure traffic on Test Port 1 to send multicast traffic with predefined source addresses and multicast group addresses.
9. Track the traffic either by source address (per source flow stats) or the multicast group addresses (per receiver flow stats). Ensure traffic can pass thru the DUT and be received successfully by Test Port 2. This can be done by checking Test Port 2's receive rate with respect to the sending rate on Test Port 1.
10. Increase the sending rate on Test Port 1 and observe that the receiving rate on Test Port 2 is increased accordingly.
11. Examine DUT's multicast routing table to ensure all multicast groups have their corresponding multicast routing entries.

Results
Traffic sent from Test Port 1 should all pass through the DUT and be received by Test Port 2. You can increase the traffic rate on Test Port 1 and observe that receiving rate on Test Port 2 increases accordingly. Examine the multicast routing table on the DUT to ensure that all multicast groups have corresponding multicast routing entries.

16. PIM-SM Last Hop Router (LHR) Functionality Test

Objective
To ensure that when the DUT is acting as a Last Hop Router(LHR) it will initiate new PIM Join requests to the RP router when new clients are registering either to a new group or to an existing group via an IGMP message. Ensure multicast traffic can reach the clients successfully.

Setup
A minimum of two test ports is required for the test. One port emulates receivers that are requesting to join new groups and the other port emulates the FHR router, as well as the multicast sources behind the FHR. In addition to acting as the LHR, the DUT can function as an RP in this configuration. Ixia's IxNetwork can be used to execute this test.

Input Parameters Number of groups requested by all receivers and the starting group address

Parameter	Description
IGP Protocol	IGP protocols used by the FHR to communicate reachability of all multicast sources to the RP router (OSPF or ISIS)
RP address	DUT RP address
# of Emulated Routers at Source port	Number of FHR emulated by the source port
Source Addr count	Number of sources that are generating multicast traffic
# of Receivers	Number of receivers
# of Groups and Starting Group Addr

Methodology

1. Configure Test Port 1 to emulate a few FHRs and multiple sources behind each emulated FHR.
2. Select the right IGP for the FHR (OSPF or ISIS) to communicate the location of each source to the DUT.
3. Enable multicast routing on the DUT and configure the DUT as RP router. Make sure the RP address is referenced correctly in the setup of Test Port 1.
4. Configure Test Port 2 as the receiver. Select IGMP as the receiver protocol so no LHRs are emulated. The DUT will function as both the RP and LHR in this case. Ensure that multicast group addresses entered match the group addresses set on Test Port 1.
5. Start protocol emulation on all interfaces involved. Ensure all protocols (IGMP, PIM-SM/SSM, OSPF) are entering stable state.
6. Configure the traffic source to send multicast traffic with predefined sources. Ensure traffic from all sources on Test Port 1 can successfully pass through the DUT and be received by Test Port 2. This can be done by checking Test Port 2's receiving rate with respect to the sending rate on Test Port 1.
7. Increase the sending rate on Test Port 1 to observe that the receiving rate on Test Port 2 is increased accordingly.
8. Examine DUT's multicast routing table to ensure all multicast groups have corresponding multicast routing entries.

Results
Traffic sent from Test Port 1 should all pass through the DUT and be received by Test Port 2. Increase the traffic rate on Test Port 1 and observe that the receiving rate on Test Port 2 increases accordingly. Examine the multicast routing table on the DUT to ensure that all multicast groups have corresponding multicast routing entries.

17. PIM-SM Reverse Path Forwarding (RPF) Functionality Test

Objective
This test is designed to ensure that the RPF is functioning correctly in delivering multicast traffic. Multicast traffic coming from a port that is NOT used to deliver unicast traffic back to the source address as seen by IGP (OSPF or ISIS), and should not be forwarded to the receiver.

Setup
A minimum of 3 test ports is required to perform this test. Test Port 3 emulates IGMP clients that will join a number of multicast groups. Both Test Port 1 and Test Port 2 are used to emulate FHRs and various distinct multicast sources behind them. All sources will generate traffic towards all multicast groups in a full-mesh manner. Ixia's IxNetwork can be used to perform this test.

Input Parameters Number of groups requested by all receivers and the starting group address

Parameter	Description
Receiver Multicast Protocol	Either PIM-SM or IGMP to emulate clients with or without LHRs. In the case of no LHR, the DUT will act as RP as well as LHR.
IGP Protocol	IGP protocols used by the FHR to communicate reachability of all multicast sources to the RP router (OSPF or ISIS)
RP address	DUT RP address
# of Emulated Routers at Source port	Number of FHR emulated by the source port
Source Addr count	Number of sources that are generating multicast traffic
# of Emulated routers at Receiver port	Number of LHR emulated by the receiver port when PIM-SM/SSM is used as receiver protocol
# of Groups and Starting Group Addr

Methodology

1. Configure Test Port 1 and Test Port 2 so that each port emulates a few FHRs and multiple sources behind them.
2. Select the right IGP for the FHR (OSPF or ISIS) to communicate the location of each source to the DUT.
3. Enable multicast routing on the DUT and configure the DUT as RP router. Make sure the RP address is referenced correctly in the setup of both Test Port 1 and Test Port 2.
4. Configure Test Port 3 as the receiver. Select the right receiver protocols (e.g., IGMP or PIM-SM/SSM). Ensure that the multicast group addresses entered match the group addresses set on both Test Port 1 and Test Port 2.
5. Start protocol emulation on all interfaces involved. Ensure all protocols (e.g., IGMP, PIM-SM/SSM, OSPF) are entering stable state.
6. Configure traffic sources to send multicast traffic with predefined sources. Ensure traffic from all sources on both Test Port 1 and Test Port 2 can successfully pass through the DUT and be received by Test Port 3. This can be done by using multicast source address as "tracking by" option and observe the per-flow stats clearly indicate equal amount of traffic from all sources.
7. Swap traffic sources on Test Port 1 and Test Port 2. This can be easily achieved by modifying OSPF advertised routes on the emulated routers.
   
   No traffic should be passed through the DUT, and Test Port 3 should report count of zero. These packets are considered as coming from the wrong port and should be discarded by DUT per the RPF algorithm.
8. Now only swap partial source addresses on Test Port 1 and Test Port 2. Again, simply modify OSPF advertised routes.
   
   The DUT should forward the traffic with the correct source address and discard traffic that has wrong addresses. Traffic received on Test Port 3 should be proportional to the ratio of correct sources in Test Port 1 and Test Port 2.

Results
The DUT should pass 100 percent, 0 percent, or partial traffic when the multicast sources are correct (i.e., not swapped), totally swapped, or partially swapped respectively per the RPF algorithm. In this example, 1000 frames-per-second (fps) is generated on both Test Port 1 and Test Port 2. Test Port 3 received 500 fps per multicast source when sources are correct, i.e., not swapped as shown in Figure 53; 0 fps are received when multicast sources are totally swapped as shown in Figure 54; and 500 and 0 fps respectively when multicast sources are partially swapped in Test Port 1 and Test Port 2 as shown in Figure 55 and Figure 56.

18. PIM-SM Rendezvous Point Tree (RPT) to Shortest Path Tree (SPT) Switchover Functionality Test

Objective
This test is designed to ensure that the DUT can construct both RPT and SPT based on source and receiver location, and that the DUT can switch traffic over from RPT to SPT once the SPT is constructed.

Setup
A minimum of three test ports is required for this test. Test Port 1 will simulate a few FHRs with a better path to the multicast source while Test Port 2 will simulate a RP router and a few FHRs with a less-than-optimal path to the same multicast source. Test Port 3 will simulate LHRs and multicast receivers. The DUT acts as a regular multicast router that participates as both an RPT and SPT in delivering multicast traffic. IxNetwork can be used to execute this test.

Input Parameters

Parameter	Description
IGP Protocol	IGP protocols used by the FHR to communicate reachability of multicast sources to the RP router (OSPF or ISIS)
IGP cost metric	OSPF or ISIS metric to determine optimal vs. less-than-optimal path
RP address	RP address – emulated by Test Port 2
# of Emulated Routers at Source port	Number of FHR emulated by the source port
Source Addr count	Number of sources that are generating multicast traffic
LHR Join/Prune Options	Sending (S,G) vs. (*,G) – by sending (S,G) join request, it will force DUT to switch from RPT to SPT
# of Groups and Starting Group Addr	Number of groups requested by all receivers and the starting group address

Methodology

1. Configure both Test Port 1 and Test Port 2 as OSPF and PIM-SM/SSM capable.
2. Set both ports to generate traffic from exactly the same multicast source(s) to exactly the same multicast groups. Test Port 1 will be generating for example 100 frames per second while Port 2 is generating at 200 frames per second.
3. Configure the OSPF route(s) to the multicast source(s) on Test Port 1 to have less metric than the same routes on Test Port 2 as shown below.
   
   
   
4. Configure the DUT to reference Test Port 2 as the RP router.
   
   Multicast traffic will initially flow from source to the DUT via Test Port 2 (RPT) and reach the receivers. On switching from RPT to SPT, multicast traffic should flow instead from Test Port 1 to reach the receivers.
5. Set Test Port 3 to emulate LHR and a few receivers behind it.
6. Configure the group Join/Prune option to send (*,G) initially.
   
   Make sure multicast traffic can initially flow from source to receiver successfully. Then force the LHR to send a (*,G) -> (S,G) request and ensure the S value matches the multicast source. This should trigger the DUT to switch from RPT to SPT and multicast traffic should flow from Test Port 1 at this point, as evidenced by its lower sending rate.
7. Optionally, you can configure the LHR (emulated by Test Port 3) to send source specific join request at all time.
   
   Traffic will initially flow via RPT for a short period of time but should quickly switch to SPT as soon as the DUT finished the construction of both RPT and SPT. Again, the way to verify that traffic is passing through SPT or RPT to reach the receivers is by observing the traffic rate.
8. Scale the test by adding more multicast sources, or by increasing number of multicast groups/receivers.
   
   To benchmark switchover performance, measurement of the variation of latency over time is required. This type of measurement provides historical timestamps per packet group during the sampling interval. Care must be taken to assign the sampling interval so that you have sufficient time to react and cause RPT to SPT switchover. Ensure the switchover is occurring within the measuring duration. A latency-based measurement provides much more accurate results than packet loss based methods. To get an accurate measurement of switchover time, locate the first timestamp where a mixture of packets start to appear (i.e., the receiver sees traffic from both sources - an indication of start of switchover) and find the last timestamp where a single packet group reappears (i.e., completion of switchover).

Results
When PIM-SM and OSPF are first started, multicast traffic immediately follows the RPT path (e.g., 200 frames per second). After manually triggering the LHR to send source specific join requests to the DUT, traffic will follow the SPT path as evidenced by the lower rate (100 frames per second). A brief disruption of traffic is due to temporary disabling of (*,G) join requests and the re-enabling of (S,G) join requests on the LHR.

When the LHR (emulated by Test Port 3) is constantly sending source specific join requests from the beginning, the DUT switches to SPT as soon as it is constructed. Traffic initially follows the RPT path until the switchover to the SPT.

When configured with 1000 multicast groups, the DUT shows latency in switchover from RPT to SPT. During this time, traffic from both sources is passing through the DUT and being received by the receiver.

When history timestamps show a mixture of Packet Group ID (PGID) (a unique identifier of traffic source), it's an indication of the start of a switchover. Record the timestamp in hex value with 20ns resolution.

When history timestamps show a single PGID, it's an indication of the completion of a switchover. Record the timestamp again. In this example, the switchover time is simply the difference between the two timestamps in hex value and multiplied by 20ns resolution, which yields 989 ms.

Conclusion

As this test plan has illustrated, sophisticated testing methodologies are required to assess the functionality and performance of today's complex edge routers. As stated earlier, the intention of the test cases presented here was to provide sample, or typical, testing strategies to guide your own testing in a variety of test areas. There are other, relevant and important test areas that have not been addressed in this plan such as the bundling of PPPoX functions within an edge router, allowing MPLS to backhaul access data. Testing MPLS advanced applications, such as L2VPN and L3VPN with FastReroute, to benchmark their performance are also significant testing opportunities. In the multicast arena, mVPN applications have become more and more popular and testing these technologies will provide extremely useful pre-deployment data. Similarly, Metro Ethernet applications, including Ethernet OAM and MPLS troubleshooting utilities, will likely become key elements of your testing strategy in comprehensively assessing edge router performance.

[ back | top of page | back to test plans ]