Top 10 things to improve OT security

Where do you start with making your industrial control systems more secure? Many of our team come from an OT background, including gas, water and electricity control rooms, running ship control systems and even turbine-based power plant management on board military vessels.

Here are some of the most common security issues they keep finding when working on customer systems. Start by addressing these and you’ll make the attackers life harder and our job more interesting!

1. Segregation

By far the most common issue we keep finding is a lack of appropriate segregation between IT and OT networks, or between OT and telemetry systems.

Gathering data and managing OT performance is a critical benefit to businesses, particularly around efficiency. That’s one reason why OT systems are networked with others.

However, mistakes in the segregation is a common problem that we exploit when testing security. Excessive permissions, misconfigurations or even vulnerabilities in switches and firewalls themselves expose OT networks.

We’ve also found issues in the telemetry communications systems used to transmit data from remote outstations. Common problems include assumptions made about private APN security, default or crackable credentials on data modems and communications over completely unencrypted RF protocols.

It’s not always unintentional mistakes either; we’ve found modems fitted by third parties suppliers without authorisation. We’ve seen remote access software such as Team Viewer installed by staff who wanted to review data without leaving their chair. We’ve seen staff install their own wireless access points in order to stream music. In all cases, these bridged important network segments that were otherwise isolated.

2. Passwords

As part of any good commissioning and build checklist, credentials MUST be changed from the defaults. Consider carefully how you manage passwords in your OT estate. Your risk assessment might conclude that having the same password for all of a particular type of PLC is acceptable, if additional controls are in place, but don’t forget how you would recover in the event that the password was compromised.

Back door accounts are remarkably common in OT devices. Ensure that your OT vendor discloses these to you, or preferably signs contractual terms to confirm that no back door accounts exist.

ICS environments are a hotbed for default passwords, passwords stuck to systems, passwords files left on accessible shares and plenty of  daft password choices and password reminders. My favourite being “who do you work for” where the password is the company name.

3. Updates

Perhaps the most significant challenge in OT is maintaining and updating firmware in a distributed, safety critical environment.

The ‘it ain’t broke’ view is simply not acceptable any more: even if one chooses to accept the risk of not updating, that risk assessment process has to actually happen!

In many organisations, updating of software is simply overlooked.

You need to ensure you’re getting alerts from vendors in a timely fashion, then assessing the impact of those to your estate.

Bear in mind many OT vendors are still cagey about security related issues, so why not help your procurement team include contractual terms about security and updates in to your purchasing contracts?

4. Misconfiguration

Failing to ‘harden’ an OT environment is another common problem: services are left running that don’t need to be, or mistakes are made.

“It’s working” isn’t enough. We need to move to “It’s working securely”

Demand evidence from your suppliers of OT services and systems that they follow recognised hardening guides. Ensure these meet with the security standards that you have defined and ensure these are included in your procurement contracts.

5. Engineer/contractor/vendor laptops for PLC maintenance

When OT systems were still totally physically isolated, troubleshooting and maintenance required an engineer, a laptop and a serial cable. This still happens, though remote diagnostics and management is emerging.

That engineers laptop was often a minefield of local admin, unsigned code, various W32 executables for PLC administration, connections to numerous dirty networks and a firewall that had probably been disabled at some point as it got in the way of an unusual ICS protocol.

Vendor and contractor laptops were often even worse, as they had been hooked up to numerous different customers networks.

Discipline and controls need to be brought in to ensure that the laptop isn’t the back door in to your OT networks. Strict separation between clean and dirty systems needs to be in place.

6. 3^rd party support/commissioning links

Who has access to your systems? We’ve found remote engine management systems running on huge marine diesel engines, yet none of the crew knew about them.

In one case, the support contract had expired years ago, yet the connection was still in place and the support organisation still had access to the vessel remotely!

Do you allow ‘always on’ connectivity to your OT by 3^rd party support organisations, or do you have to enable it when necessary? 3^rd party breaches are increasingly common.

Commissioning links are often put in place when installing a new system, but who checks that those links are disabled once the system is up and running?

7. RF

Telemetry is increasingly critical for OT security. If your comms go down, how do you recover to a safe state?

Radio links are fairly reliable, but are rarely suitably encrypted. Historically, there was no requirement to encrypt – the risk simply wasn’t great enough or properly understood.

Have you examined what control now happens over those radio connections?

Mobile data telemetry is more common now, though we often find issues with the platforms and devices used to enable this. It’s often assumed that using a private APN is enough, yet we’ve shown in the past how APN keys are often weak and are very easy to crack.

Who actually checked that the APN keys were suitably secure? It’s often just assumed that the mobile platform provider ‘does’ the security.

Bluetooth is increasingly found in switchgear, allowing an engineer to be some distance away from high-voltage equipment when switching. This reduces the risk to health from a local ‘flashover’.

Did anyone check that the Bluetooth connections to the engineer’s mobile device were secure? If not, you’re exposing your switchgear to attackers located outside the fences of your outstations.

8. Physical security

Far too many assumptions are made that OT is physically secure. Key management for remote outstations is painful, so good intentions of having strong locks with secure keys often fall down after a few keys have been lost.

Are alarms in place? Is CCTV being used? Great, but did anyone actually check the escalation process works? So often, something is triggered, but no-one actually takes ownership of the alert and takes action.

It’s a great idea to actually send someone to a remote outstation ‘unauthorised’ and see what happens.

Remote, unmanned sites can take a long time for law enforcement or security personnel to get to. How would you know if any of your systems in that outstation had been modified? Everything is working just fine, but how do you know if the ladder logic on a PLC had been edited?

Tamper-evident controls are one way to at least know if something has been connected to. We’re used to safety-critical tamper controls, so why not extend similar processes to OT security?

9. Press releases by vendors & social media

It strikes me as a massive ‘facepalm’ moment when a customer allows an OT vendor to put out a press release about a product sale.

Why on earth would one advertise which systems one is using internally? If a vulnerability is found subsequently, it’s a trivial matter to work out who has that system.

These press releases often quote senior OT personnel at the client too, disclosing perfect targets for phishing. Yes, that data is often available for other sources, but why make it easy?

I strongly recommend that you mandate that vendors are not permitted to announce product sales to your organisation. If this cannot be stopped, at least ensure that no exact product details are released.

10. Disclosure programme

This is an easy win for any utility, manufacturer or other organisation that uses OT. Start a vulnerability disclosure programme.

At the very least, just make it easy for security researchers to contact you. Someone notices one of your systems exposed on the internet with configuration issues? How do they report it to you fast?

It’s up to you if you want to go further and establish a bug bounty programme, but a VDP is a no-brainer.

Most responsible researchers just want to help make a difference, receive a ‘thank you’ and to feel listened to. Send them some swag too; be a super-cool organisation.