Yong Zhou
Security Solutions Architect

Assess the Effectiveness of Dynamic NGFW Updates: Palo Alto Security Audit

June 15, 2020 by Yong Zhou

One benefit of breach and attack simulation is continuous assessment, and I set Keysight Threat Simulator to automatically, each day,  assess the effectiveness of my network security controls that includes a Palo Alto NGFW/IPS, which offers protection for our remote access VPN users.

When I logged onto Threat Simulator this morning, I noticed a significant change in my protection score (thankfully it was positive) on one of my scenarios – Web Browser Assessment. This assessment is a highly relevant security validation under our current work from home (WFH) environment.


As shown in the screen above, my web browser assessment protection score improved to 80% (4 out of 5 passed audits) from 20% (1 out of 5 passed audits) in the previous daily assessment runs.

Here is a more detailed comparison of the results in the last two days:


As you can see, the Detection Result, based on Threat Simulator’s SIEM log correlation, remains the same, the only change is the Prevention Result. So, what happened? Given that there is no security policy setting change on the Palo Alto NGFW/IPS that I am aware of, why did my network security control change its behavior?

To get some answers, I logged onto the Palo Alto NGFW/IP, and found the following:

  1. There was no security policy change.
  2. On the threat monitoring page, I could see threat action changes for Mozilla Firefox browser vulnerabilities. This exactly matches the audit result from the Threat Simulation assessment (#3 in the prior screengrab).
  3. Next, I checked the dynamic updates page and the system logs and found that indeed, the latest application and threat content package was installed on June 4th!

  4. To conclude my investigation, I clicked Threat Simulator’s recommendation guide link to confirm Palo Alto’s ThreatVault information for this specific vulnerability:


What a pleasant surprise! My network security control vendor (in this case, it was Palo Alto) provided improved protection via dynamic updates. Also, my continuous security assessments with Threat Simulator ensured I had an immediate measurement of the effectiveness of my network security controls using the dynamic update—rather than waiting days, weeks, or even months for an attack attempt to know the effectiveness. Examples like this are great to show the boss as you look to validate all the work you do to improve cybersecurity!

Security is never static. New cyberattacks, misconfigurations, and security products are rampant. How do you take control of this ever-changing threat landscape? The only way to know is to assess your defenses before hackers do. Give Threat simulator a (free) try today.