Penetration Testing: Money Spent, Still Vulnerable

Penetration testing is a real-life test of your security defenses. It is a simulated attack on your computer systems, performed by an external security expert, or “white hat hacker.”

The penetration tester might try different types of attacks to identify key vulnerabilities in your system and provide some evidence of the type and magnitude of damage that a real attack could cause.

Penetration tests, or “pen tests” as they are typically called, are sometimes thought of as insurance policies for your network security. It is assumed that if you have undergone a penetration test, received a report, and fixed the issues discovered, you are secure and immune to devastating attacks.

But data shows that this view is dangerously wrong—organizations that pay big bucks for penetration testing services may find they are still vulnerable.

Why is this the case? In our experience, while a penetration test can expose important security issues, it doesn’t help an organization build up the knowledge and expertise of its internal security team. On the day of an attack, it is that internal team that will have to detect, contain, and remediate a security breach.

White Hat Hacker

Fact 1: Penetration tests leave security holes wide open

A 2015 study by WhiteHat Security examined security activities at 118 organizations. Of the organizations in the study, 92% had done penetration testing at least once as part of their security programs. The study revealed 21% of the organizations surveyed carry out a penetration test every year.

Despite this significant expense, this was the security aftermath:

Penetration Tests Leave Security  Holes Wide Open

The average organization that carried out regular penetration testing had as many as 10 security vulnerabilities, and only 50% of them were eventually fixed.

The situation for organizations that never performed a penetration test was even worse—an average of 32 vulnerabilities, vulnerabilities were left open 431 days on average, and there was a remediation rate of just 22%.

However, it is clear that penetration testing, even carried out regularly on an annual basis, left organizations wide open in their defenses with a huge time gap before security holes were remediated.

Fact 2: External consultant use in a data breach meant bigger financial losses

Ponemon Institute’s 2015 Cost of Data Breach Study surveyed 350 companies from 11 countries that had experienced data breaches. Almost half of those breaches (47%) were due to a malicious attack, and the rest were due to human error and system glitches. These companies reported on factors present in their security organization or computing systems that affected their ability to defend themselves.

One of the findings of the study was that bringing in external consultants to assist with security operations correlated with bigger financial damage from an attack, as opposed to ongoing, in-house security activities, which correlated with reduced financial damage.

Data Breach Cost Analysis Table

While these statistics are not directly related to penetration testing, they reflect what organizations can expect from penetration testing. A security intervention performed by an external consultant, while beneficial on its own, typically does not help the company build proper security practices and the knowledge needed inside the organization, giving teams a false sense of security. On the contrary, in-house initiatives like the creation of incident response teams and employee training not only expose problems and apply fixes, they actually improve an organization’s ability to defend itself.

Penetration testing cannot replace a continuous security process

Penetration testing, while costly, can expose important vulnerabilities and provide valuable security information. However, the data shows that it is not enough on its own. While getting a penetration test through an external security consultant may be sufficient for compliance purposes, it cannot replace the continuous in-house security testing. In our experience, here are a few reasons why:

  • Systems change: Penetration tests provide a “snapshot” of vulnerabilities at a certain point in time—but systems and configurations change frequently, opening up new vulnerabilities. In addition, hackers discover new vulnerabilities on a daily basis.
  • Attacks change: Hackers are constantly devising new more powerful and more sophisticated forms of attack. Your system may have passed the penetration test, but will it stand up to a new type of attack or a similar attack that is a hundred times more powerful?
  • People change: The most important part of your security defense is your security team. You need to ensure that your team is battle trained and ready on an ongoing basis.
  • Application traffic changes: Cyber attacks don’t happen in isolation. While they are happening, there are legitimate users using an organization’s network who may be affected by the attack. This legitimate application traffic is dynamic in nature—loads can change dramatically, as well as the types of traffic (e.g., web browsers, uploads and downloads, video streaming, universal communications)—and this can impact the effectiveness of network security defenses. For example, a web application firewall might miss a vulnerability attack if it is experiencing traffic loads at 99% of its maximum capacity versus only 50%.

Meet Ixia

Meet Ixia, a leader in security testing and network visibility. We work with 77 of the Fortune 100 and 47 of the world’s top 50 telecom carriers. We emulate real attacks, helping our customers practice their defenses through testing and validation of their security architecture in the face of known attacker techniques. Ixia can conduct testing and validation of security components and architecture, obfuscating attacks to determine whether your devices will detect an issue. Our tests are not just centered on the technology, but also the people and processes, involved in citing security threats.

Applications do great things, but they all have bugs and blind spots. Making them stronger means better testing, better security resilience, and better monitoring ability. Ixia takes a three-pronged approach to making applications stronger with IxTest, IxSecure, and IxVision architecture capabilities.

We put together a quick guide that will teach you how to set up continuous and holistic security practices in your organization to complement a one-off penetration test. After the pen test is over and you have implemented the recommendations, you need to battle test your security infrastructure, policies and staff repeatedly to make sure your systems and defenses are up to the challenge of tomorrow’s attacks.