RSA 2014 Panel Session: Selecting the Right Security Device
I attended the panel session Analyst Quadrants, Third-Party Tests, Vendor Data Sheets and YOU. There was an interesting mix of panelists…a research company, a security gear vendor, a research and testing company, and a financial enterprise. Each provided a different perspective on how companies can get the information they need to select the right security gear.
Ixia’s Fred Kost moderated the panel, opening with questions about the difference between vendor data sheets and technical specifications. The panel, consisting of Forrester Research’s John Kindervag, Fortinet’s John Maddison, NSS Labs’ Ryan Liles, and Bank of America’s Paul Yancey, agreed that these materials valuable for narrowing the list of devices down to those that have the features and functions your implementation requires. However, they should not be trusted as a crucial source for network security purchasing decisions as they most likely represent the best-case scenario for performance and functionality.
The real world is very different from data sheet hype. Organizations must get as many data points as possible the make optimal purchasing decisions.
Independent industry reports, like those from NSS Labs, use standardized testing so that companies can get an apples-to-apples comparison among the security solutions. To make these tests truly “independent”, vendors are not able to pay for these tests, nor are they allowed to opt out. NSS Labs uses Ixia BreakingPoint as a testbed that provides consistent, reliable tests. However, the limitation in this reporting is that the tests probably don’t reflect the use of the security products in your particular network. Organizations need a system approach to device purchases, understanding how it will impact the entire system and overall security.
Proof of concepts (POC) and bake-offs. Some labs will create simulations of a customer’s actual network and run it against short-listed security gear, giving a good indication of how it will perform in real-world use. Ixia’s BreakingPoint is a good tool for generating these real-world simulations. Another suggestion was to get loaner equipment to see how it actually performs in your lab.
Industry analyst reports and general knowledge, like those from John Kindervag of Forrester Research, are another valuable point of reference. It’s not just the value of industry reports and analyst quadrants, but John hears from the actual end users of the security gear and what they like and don’t like about it in very specific terms…information that isn’t “up-leveled” by marketing.
Finally, sharing of information among your industry peers is a great way to find out what security gear is working best for them. Sources like Bank of America’s Paul Yancey, who work with large implementations and see the performance of this equipment on a day-to-day basis, know the real value of various gear.