Shiny New Pen Test Report! So What…

Today we’re really pleased to publish a guest post by @_tonygee_. Tony is a CISSP certified Information Security Analyst, a mad motorsport fan, and a man who doesn’t mince his words. Enjoy.

So you have managed to secure budget, have completed a test and have a shiny pen test report to show for it… Yay! Now what? Erm…
This is often what happens with pen testing, it’s usually quite easy for an organisation to secure funding for a pen test. Maybe the auditors have raised a non-compliance issue during the annual audit, so the Board react quickly to get it carried out. Maybe it’s required because you process credit cards and have an urgent need to tick the PCI box before your acquirer has a strop at you.

So you know what you want to test and why you need to test it, you will have a clear idea of how much it will cost given you have a quote from the testers. It’s easy to explain and justify the expense to those who hold the purse strings. But what about remediation?

When you are planning your test it is critical to plan for remediation; but how much time should you plan and crucially how much will it cost? So often we see organisations have the test, yet do sod all about the issues raised, because they didn’t think about the cost of fixing them when budgeting for it.

The answer to this question is unfortunately similar to the one about a piece of string… However, you can implement techniques to help you define what is more important to fix and what is not. That is the key point to this article, what vulnerabilities am I happy to not fix?

Enter ‘Risk Management’.

Almost every security manager will be familiar with risk assessment and even pen testers will be familiar the terminology and so will helpfully include a rating, either a CVSS or just a plain old High/Medium/Low on the report (maybe both!). However, they invariably do not know your organisation as well as you do and they certainly don’t know your risk appetite.

When examining your pen test report you need to consider the pen tester may have had only a few days to look at the system and report any findings, so can only provide you a risk rating based on what they can see. Even with extended testing in your business, they are unlikely to know the intimate detail of subtle compensating controls you might have in place. This is especially common with web applications being used by trusted 3rd party organisations. You may have limited the access to the website to a handful of companies and the pen test firm, but the tester may not know that and even if they do, they will probably still rate it as if it was on the internet open to the world, just in case you don’t implement the firewall rules correctly.

Even widely used scoring systems such as CVSS can’t account for every variety of access restrictions that may or may not be in place. I have seen some pen test firms that try to indicate the complexity of a particular fix, yet miss the point by miles. Yes, the fix might be trivial, but it could be on a live trading system that has scheduled downtime once per month overnight on a Saturday, during which every other upgrade and bugfix needs to be applied too. It needs functional testing, acceptance testing, staging and finally deployed in to live. That ‘trivial fix’ might in fact cost tens of thousands to implement, because the pen test co simply doesn’t understand your business in the way that you do.

It is of utmost importance that you review the report and apply your own risk assessment to each finding. Many organisations have proven processes to do this; it is critical to ensure you work within the risk methodology that your business works with. You don’t want to stand in front of a CEO and have to explain how your risk methodology works before you can ask for money to fix the issues.

A simple heat map that can be used to visually illustrate risk.
Green is lower risk and likely within appetite.
Red is higher risk and is likely outside of appetite and will need remediation.

An approach I have taken in the past is to start with a position of fix everything and then work back from that. Starting in this way you can begin discussions with the support teams and get a real feel for how much effort and cost each finding will take to remediate and if there are any other compensating controls you may not be aware of.

When preparing your paper for the CIO you should be able to justify why you think something should be fixed (or not), a rough idea of how much it will cost and the business risk rating for each finding. This may take you time and you may feel pressured to deliver this quickly especially if you have high risk findings, but once you build a repeatable process it will get easier.

Good luck!

For help on some IT risk methodologies try:
http://en.wikipedia.org/wiki/IT_risk_management
ISO27000 series have multiple references to risk management, the important standard is 27005.