Load Balancing Your Security Solution for Fun and Profit!
Maximizing the Value and Resiliency of Your Deployed Enterprise Security Solution with Intelligent Load Balancing
Author: Phil Trainor
Correctly implementing your security solution in the presence of complex, high-volume user traffic has always been a difficult challenge for network architects. The data in transit on your network originates from many places and fluctuates with respect to data rates, complexity, and the occurrence of malicious events. Internal users create vastly different network traffic than external users using your publically available resources. Synthetic network traffic from bots has very recently ousted real users as the most prevalent creators of network traffic on the internet (http://www.dmnews.com/privacy/bots-overtake-people-on-the-web/article/414822). How do you maximize your investment in a security solution while gaining the most value from the deployed solution? The answer is intelligent deployment through realistic preparation.
Let’s say that you have more than one point of ingress and egress into your network, and predicting traffic loads it is very difficult (since your employees and customers are global). Do you simply throw money at the problem by purchasing multiple instances of expensive network security infrastructure that could sit idle at times and then get saturated during others? A massive influx of user traffic could overwhelm your security solution in one rack, causing security policies to not be enforced, while the solution at the other point of ingress has resources to spare.
High speed inline security devices are not just expensive—the more features you enable on them the less network traffic they can successfully parse. If you start turning on features like sandboxing (which spawns virtual machines to deeply analyze potential new security events) you can really feel the pain.
I solved this dilemma by load balancing multiple inline Next Generation Firewalls (NGFW) into a single logical solution with an Ixia xStream40, and then attacked them with Ixia BreakingPoint to prove they work even during peak network traffic.
I took two high end NGFWs, enabled nearly every feature (including scanning traffic for attacks, identifying user applications, and classifying network security risk based on the geolocation of the client) and load balanced the two devices with my Net Optics xStream40 solution. The xStream 40 has 48x 10GbE ports and 4x 40GbE ports. With this solution I can load balance up to 24 inline security devices at speeds of over 500Gbps. Then I took an Ixia BreakingPoint traffic generator and created all of my real users and a deluge of different attack scenarios.
At the end of a week of testing, I proved that my combined security devices maximized the dollars spent while maintaining the security posture they advertised. Below are a few of the more interesting tests that I ran and their results.
Scenario One: Traffic Spikes
Your 10GbE NGFW will experience inconsistent amounts of network traffic. It is crucial to be able effectively inforce security policies during such events. In the first test I created a baseline of 8Gbps of real user traffic, then introduced a large influx of traffic that pushed the overall volume to 14Gbps. The xStream 40 load balancer ensured that the traffic was split between the two NGFWs evenly, and all of my security policies were enforced.
Figure 1: Network traffic spike
Scenario Two: Endurance Testing
Handling an isolated event is interesting, but maintaining security effectiveness over long periods of time is crucial for a deployed security solution. In the next scenario, I ran all of the applications I anticipated on my network at 11Gbps for 60 hours. The xStream 40 gave each of my NGFWs just over 5Gbps of traffic, allowing all of my policies to be enforced. Of the 625 million application transactions attempted throughout the duration of the test, users enjoyed a 99.979% success rate.
Figure 2: Applications executed during 60 hour endurance test
Scenario Three: Attack Traffic
Where the rubber meets the road for a security solution is during an attack. Security solutions are insurance policies against network failure, data exfiltration, misuse of your resources, and loss of reputation. I created a 10Gbps baseline of the user traffic (described in Figure 2) and added a curveball by launching 7261 remote exploits from one zone to another. Had these events not been load balanced with my xStream 40, a single NGFW might have experienced the entire brunt of this attack. The NGFW could have been overwhelmed and failed to inforce policies. The NGFW might have been under such duress mitigating the attacks that legitimate users would have been collateral damage of the NGFW attempting to inforce policies. The deployed solution performed excellently, mitigating all but 152 of my attacks.
Concerning the missed 152 attacks: the Ixia BreakingPoint attack library contains a comprehensive amount of undisclosed exploits. That being said, as with the 99.979% application success rate experienced during the endurance test, nothing is infallible. If my test worked with 100% success, I wouldn’t believe it and neither should you.
Figure 3: Attack success rate
Scenario Four: The Kitchen Sink
Life would indeed be rosy if the totality of a content aware security solution was simply making decisions between legitimate users and known exploits. For my final test I added another wrinkle. The solution also had to deal with large volume of fuzzing to my existing deluge of real users and attacks. Fuzzing is the concept of sending intentionally flawed network traffic through a device or at an endpoint with the hopes of uncovering a bug that could lead to a successful exploitation. Fuzzed traffic can be as simple as incorrectly advertised packet lengths, to erroneously crafted application transactions. My test included those two scenarios and everything in between. The goal of this test was stability. I achieved this by mixing 400Mbps of pure chaos via Ixia BreakingPoint’s fuzzing engine, with Scenario Three’s 10Gbps of real user traffic and exploits. I wanted to make certain that my load-balanced pair of NGFWs were not going to topple over when the unexpected took place.
The results were also exceptionally good. Of the 804 million application transactions my users attempted, I only had 4.5 million go awry—leaving me with a 99.436% success rate. This extra measure of maliciousness only changed the user experience by increasing the failures by about ½ of a percent. Nothing crashed and burned.
Figure 4: Application Success rates during the “Kitchen Sink” test
All four of the above scenarios illustrate how you can enhance the effectiveness of a security solution while maximizing your budget. However, we are only scratching the surface. What if you needed your security solution to be deployed in a High Availability environment? What if the traffic your network services expand? Setting up the xStream 40 to operate in HA or adding additional inline security solutions to be load balanced is probably the most effective and affordable way of addressing these issues.
Interested in seeing a live demonstration of the xStream 40 load balancing attacks from Ixia BreakingPoint over multiple inline security solutions? We would be happy to show you how it is done.