There are “pen tests” and then there are “PEN TESTS”…

I’m on Twitter, in fact, who isn’t these days? I follow some of the more interesting and illustrious characters in the security industry, as they normally have something interesting to say or a paper or tool that would be useful.

The tweet below popped up in my feed:

At first glance, this seems to be a somewhat contentious statement. In fact, when I posted it to my colleagues to see what they thought of it some reactions were, shall we say, unprintable whilst others were more reasoned. Digging deeper into the background and subsequent tweets from others, it seems that the intent of the statement is to show that, as pen testers, we really need to up our game. But, and it’s a big one, only for certain types of engagement.

What is a pen test?

For me, it raised the question of whether or not clients understand what a pen test is and what they can gain from the exercise. Don’t get me wrong, I appreciate that a small web dev outfit isn’t going to want a full, no holds barred pen test. They don’t need custom malware being created specifically to attack them. But, what about bigger clients?

As a colleague pointed out, a pen test is designed to find as many issues as possible. Being stealthy isn’t part of the deal as they know we are there. But, what does this prove? OK, so they now have a list of things that need fixing. Have they learnt anything from the exercise or not? That depends. If we go back in a year and find either the same issues or ones caused by similar failures (configuration issues, lack of security awareness etc), then you could argue that the test has been a waste of time.

Let’s turn the test on its head. What if the whole idea of the test was to just “get in”? After all, that’s what a “real world malicious attacker” would be looking to do. As Bruce Schneier pointed out in 2000: “Security is a process, not a product”. Where does a pen test fit into this? Results from a pen test are highly technical and, for the most part, boring to anyone who doesn’t deal with the technical running of a business. Which means that when the results get passed up the metaphorical food chain, they lose their meaning and it then tends to boil down to someone blaming the IT department for not going a good job. But, to my mind at least, pen testing should itself be a process. Not just a “hit ‘em hard, hit ‘em fast” exercise that results in an epic report that Tolkien would be proud of….

Stealth

What should be done, then? Really, the clue is in the name: “penetration test”. The idea is to break into a client’s environment. But, and this is really important, it must be done stealthily. Reading further into the Tweets that followed on from the original, it’s becoming clear that the philosophy behind it is that we pen testers use the same tools, and that those tools are easy to identify. This, obviously, leads to being caught quicker by AV, IDS, IPS and so on. Writing payloads for Metasploit isn’t something that can be done in 10 minutes, and especially when the time is tight for a test.

If the idea is to test a client’s defences and responses to an attack, then getting caught by the new guy on the helpdesk has to be considered a fail…. It takes a lot of time and effort to avoid defences, especially if creating custom attacks is required. How many clients would be willing to pay for an extended engagement? We’re talking months here. Not a few days in a cold data centre. The big question here, especially for the accountants and shareholders, is: what value does this stealthy approach bring? I’d, of course, argue that it is of great value. Just look at the attacks that have hit the headlines recently. They’ve only been noticed because they were successful, not because the attackers were caught during the attack. Once the data is out of the client controlled perimeter then it’s all over. The only things to do then are forensics and damage limitation.

The real world

Testing that emulates attacks like those is important in that it has several major benefits:

• It exposes weaknesses not only in, for argument’s sake, the patching process, but in all processes.
• It helps to train staff in incident response, triage and follow up forensics. Working with the staff and explaining tactics and approach is invaluable in getting them to understand what tactics attackers use and what to look out for.
• It can help focus, or distribute, security resources. Getting into the environment by that poorly defended ADSL router or the security door with the “1234” PIN code helps to highlight that security is needed everywhere, not just what is perceived to be a high value target.
• New tools are developed that can help expose weaknesses that may not have been considered before.

Playing Devil’s Advocate, I’d argue that no pen test is a waste of money so long as the client gets value from the exercise and the results.

Don’t get me wrong, I’m not saying that “bread & butter” testing should go the way of the Dodo. That would be foolish. However, I definitely think that companies need to expand their thinking when it comes to scoping a pen test…