3yrs of CAA ASSURE assessments. What we’ve learned

Introduction

We’re now in our third year of CREST CAA ASSURE auditing and we’ve learned a lot. The Cyber Assessment Framework (CAF) is big, there’s no denying that. It’s not something that you can complete overnight, it’s not something that requires minimal effort and can just be thrown at an auditor to interpret.

It demands context, justification, and understanding of the requirements and how to meet them.

TL;DR

• This is not a cheat sheet for CAA ASSURE compliance
• It is a guide on how to ensure a smooth journey to CAA ASSURE compliance
• We’ve included handy hints and tips we’ve found along the way
• How to prepare for an audit
• Not ready? We can help

Overview

The Civil Aviation Authority (CAA) is the statutory body that governs all facets of civil aviation in the UK. Its duties cover the supervision of pilot and aircraft engineer license issuance, equipment testing, NAVAID calibration, and various other inspection activities including the cyber security requirements of airports and air carriers and air navigation providers.

The CAA ASSURE Scheme launched in January 2020 and was developed in partnership with CREST. Its purpose is to help the aviation industry manage cyber security risks without compromising aviation safety, security, or resilience. Aviation organisations will have to complete a cyber security self-assessment using the CAF for aviation and have this audited by an ASSURE accredited supplier.

We regularly undertake consultancy and audit activities for aviation organisations to assist in ASSURE compliance. Because of that we can share some dos and don’ts that’ll make the process easier for you.

Part of the ASSURE audit process is to evidence justifications. Evidencing comes in many forms:

• Interviewing key stakeholders
• Documentation, policy, and procedure
• Observation

We understand that not every organisation has a full ISMS documentation suite and a cyber maturity level of 5 (optimised). There are going to be questions / indicators of good Practice (IGPs as they’re known in the CAF) that you just can’t achieve or only partially meet.

To help you deal with that here’s some things that’ll help your audit team prior to submitting your CAF and supporting documentation to make that process easier.

Don’t know where to start?

You’re not alone and you’re not the first. Book some consultancy time with a professional who can help you answer the CAF questions. It’s important to note that if a company assists you with this they can’t perform the audit. You’ll need to engage another organisation for that.

Understand that the scope and scale of what’s ahead

As mentioned earlier read the Cyber Assessment Framework (CAF). These audits are mandated by the CAA. That’s not to say that they expect everything to be fully achieved across the board. The purpose of these assessments is to understand your current maturity and what can be done to help you in remediating any gaps identified. The emphasis of this scheme is partnership. The CAA want better for the industry and that starts with you, the aviation organisation!

Preparation is everything

The time you invest in ensuring your answers are comprehensive and accurate pays dividends. The more you invest  here the less time you’ll spend trying to justify and evidence it to an auditor.

Whether you need to allocate specific resources to the project or engage a third party to guide you through answering the contributing outcomes, you should not have to struggle and feel alone in the process.

Use a collaborative platform

Many clients use Microsoft products like Teams or SharePoint, but other option like Slack, Trello, Confluence etc. can work too. The point here is that documentation is shared live between you and your auditor, allowing for timely additions and edits.

Create and use an evidence tracker

The evidence tracker does not need to be complex, it should be simple and clear. A spreadsheet will cover this perfectly. It should highlight the relevant IGP, what’s missing, who needs to take action, and a status. When shared on a collaborative platform it allows you and the auditor to document a list of outstanding actions to tick off as you go along.

Order your evidence in folders for each critical system

A folder per system allows the auditor to find the relevant evidence relating to the justification and IGP.

Have key stakeholders ready to interview

This ensures the audit isn’t unnecessarily dragged out.

Book a milestone check in with your auditor before the audit

We would far rather see you succeed than fail. We regularly ask our CAA clients to provide a copy of their CAF and evidence a few weeks prior to audit to ensure that everything looks good. This is a small check we undertake to ensure you’re in the best position for a smooth ride.

Your consultant will review your answers within the CAF and that the evidence that has been provided is sufficient to audit. Of course, there will be elements that rely on interviewing key personnel, so we leave these for the audit itself. If we feel that you are not quite ready for the audit, we will say.

We are more than happy to defer the work for a week or two giving you more time to provide more detailed answers or supporting evidence.

Understand that IGPs are not mix and match

You either meet the requirement and all achieved IGPs are ticked, or you don’t.

You need to decide whether you meet all the requirements of an achieved status. If you think you do, you need justification and evidence for each point. The CAF is designed so that you have to clearly justify every IGP to meet a fully achieved status on a contributing outcome.

Answer everything as honestly and completely as possible

Don’t leave any section blank. If you don’t have a complete answer, and / or it’s in the pipeline, say that.

If you think you address the control in a different manner, with different controls, say that, with examples.

Leaving a section blank means the whole contributing outcome will be deemed ‘not achieved’. If you explain that there is potential for partial achievement, with a rationale and justification of your current position, you’ll be in a far better place.

Get your third parties on board

Answering a question with ‘a third party does that for us’ is not acceptable. You need to understand how they do whatever it is that they do, and what their contractual obligations are. Get a copy of your MSA or contract that outlines their responsibilities versus yours and scrutinise it.

Conclusion

We understand how long and difficult the CAF is to complete. It is a task in itself and a labour and resource intensive one at that. The more you can offload to the auditor to complete offline such as documentation review, the less time your team needs to spend in meetings and answering questions.

Being prepared is key here, so the more detail in your justifications the better.