Benchmarking Open Source SDN Controllers – Are They Ready for Carrier-Grade Services?
SDN Controllers - Are They Ready for Carrier-Grade Services?
That was the title of my presentation at MPLS+SDN+NFV World Congress 2016 in Paris last week. The response from the congress delegates was overwhelming. Obviously this has been a key question in everyone's mind. I’m happy to say that SDN is ready for prime time deployment. However, further benchmark testing is critical to determine if individual SDN controllers are ready to match carrier-grade routers.
How does the centralized control brain live up to the expectation of carrier-grade services and abuse? What are the performance hits and other challenges when you put together all the open source components on a commercially available server?
We picked three widely popular open-source controllers - RYU, ODL, and ONOS and ran them through with the benchmark test methodology defined by ONF and IETF BMWG. We used Ixia's IxNetwork SDN Emulation to simulate hundreds of SDN switches to fully stress each controller under test. I presented the test results and preliminary assessment at the congress.
In summary, we ran into challenges and were not able to complete all of the benchmark tests. All three open source controllers do come with applications in addition to a south-bound API, but were not sufficient to match the typical functions of distributed routers. Without writing the necessary networking applications or purchasing the all integrated packages from the vendors, we were able to validate three test cases with limited results.
We also discovered that the different implementations could drastically influence the test results. For example, ODL configures bi-directional flows by default while RYU and ONOS only work with unidirectional flows. The default applications that come with ONOS can only respond to IP/ARP learning, while the other two controllers only come with L2 learning. In spite of the lack of needed applications, we did observe a few key points:
- All three controllers do provide reasonable performance and scale. They all sustained well with more than 500 switches.
- Benchmark testing is the critical tool to determine if the SDN controller is ready to match carrier-grade routers.
- Applications and implementation greatly impact performance.