What does the Basecamp project PLC disclosures mean to your†operations?
We have been asked by several of our industrial control clients about the Basecamp PLC vulnerability disclosures, the new exploits coming to a Metasploit /SCADA folder near you, and how this might impact their operations. First, if you haven’t read or watched the Basecamp PLC vulnerability presentation provided at the S4 conference this year, I would suggest that you watch it now at the Digital Bond web site. It is worth the 90 minutes, and if you were not already aware of how fragile your embedded PLC and RTU devices really are, this will be an eye-opener. (You can go watch it now and come back here…)
Now that you have watched the video, we can speak on the same page. The next knee-jerk reaction that some asset owners have is to look at the list of devices that were involved in the project, then compare those devices to their inventory, and then say… “Man… I’m glad we are not running any of those PLCs”. Avoid that reaction. We have worked with a large number of embedded automation controllers and RTUs, and we can safely say that all have major cyber security issues.
The security vulnerabilities that are explained in this disclosure are systemic across almost all embedded devices used in SCADA and Industrial Control Systems. The next negative reaction that most have is to blame Digital Bond or blame the control system vendors. Neither of these two groups can really do anything today or tomorrow that can protect your system. Even if you nag your control system vendor for them to patch the problems discovered in the basecamp project, and they listen to you and repair these issues, more vulnerabilities will just crop up later. Yes, the long term strategy is to get the control system vendors to create hardened and secure devices from the factory, but that is a very long term strategy that will not address all of the millions of vulnerable devices already installed today or being installed tomorrow.
If your field controller is connected to an ethernet network using TCP/IP based protocols, just go ahead and assume that any of the vulnerabilities mentioned in the S4 video are potential security issues with your devices as well. We have seen PLC devices from major control system vendors that freeze or reboot just by sending them a flood of ping requests or by simply scanning them with NMAP or NESSUS. There are several documented examples of PLCs failing because a local printer ran out of ink and flooded the local control network with SNMP packets. So if these devices fail under simple network broadcast storms, it is easy to see how much mischief you can cause to these devices if you actually spend some time to send specifc attacks to these systems.
The point of this discussion is that with or without the recent Basecamp project disclosures, if your field devices are connected over an ethernet network, you are and have already been exposed to cyber risks to your automation or control system performance. Attackers do not need to be physically present on your control network for these vulnerabilities to work, since they can be routed into the control system from other networks, or become activated from a script or batch file on a USB stick. Air-gapped networks are not safe either.
So what do you do about this? Knowing is half the battle, and some will say that getting funding to fix the problems once you know about them is the other half. That is an entirely different topic, but assuming that you have the backing and support to actually make changes to your system and secure it, here are some steps to think about.
6 Steps to Securing Access to Ethernet-Connected PLCs and Field Devices
1. Flush out the logical data flow and architecture – Dive into the weeds and understand your control system in-depth. Build an accurate depiction of how the your PLC and field devices are connected back through the telemetry, LAN/WAN infrastructure, and back up through to your control room HMI and I/O servers. Ensure that everything is documented well, including the IP addresses of all major system components. If you need to, setup a laptop, run Wireshark or TCPDUMP, and capture actual SCADA protocol commands on the network to understand the full route from the control room out to the field devices. Try running tracroute to document all of the hops along the way. Knowing the logical data flows from the plant floor or field systems back to the control room is a great first step.
2. Determine what TCP or UDP ports are required to be open on your PLCs and field devices – This is not only a requirement for utilities that must map out their ports and services for NERC CIP compliance, but is also equally important for all critical infrastructure sectors. You can not adequately secure something when you do not completely understand its attack surface. For instance, if you are running a Rockwell Automation controller, and it uses ports 23, 80, 443, and 2222, you probably do not need the lower ports to be open to the controller, since the protocol operates over 2222. You can block all of the other non-essential ports. If you do not have this information, take a spare PLC off the shelf and scan all TCP and UDP ports to determine what ones answer back. You can also sniff the network while it is in use. Knowing how the devices are connected together (obtained from step 1 above), and then armed with what ports need to be open are the first two steps.
3. Next determine what source and destination IP addresses are used for PLC communications – This step may be a bit easier, since there should be only a finite number of computers that are required to poll the PLC network. In most SCADA systems, this number is less than 10. Do your homework to determine what computers really need to communicate with the controller network, and then conversely, what field devices are required to receive and answer back to these commands. Build a model in Excel using one column to show the list of computers that are communicating, then the next column to show the TCP or UDP ports that are used, then lastly a third column showing the devices that are the destination for the SCADA commands. This is not only an excellent troubleshooting tool for how the ethernet-based SCADA system functions at the PLC level, but can also be used to help secure or lock down the network.
4. Enable Access Control Lists (ACLs) between the computers in the SCADA system and the embedded devices in the SCADA system.† The computers in the SCADA system, such as operator consoles, engineering workstations, data historians, and special-use application computers all typically run on modern fat operating systems like windows or *nix systems. These need a large number of TCP and UDP ports to be open for them to function on the network. The embedded devices do not require the same ports to be open, so you can innoculate your field devices from being vulnerable to viruses, worms, and malware that affect your computers by simply blocking all non-essential ports. This way a virus would have to be written specifically for your field device in order for it to affect your control system. Create additional access controls that limit what source addresses can send packets into the control network. This can also help greatly reduce the attack surface by reducing the potential computers that have digital access to the ethernet-connected field devices. After this step, there should only be a limited number of computers that can access your field devices. If you can not introduce ACLs between your SCADA operations computers and the field devices, then at a minimum, back up to the firewall or DMZ that sits between the SCADA environment and the corporate IT or other networks. At a minimum, access controls should be placed at this level.
5. Enable Application Control, HIPS, or Application Whitelisting in those computers that can communicate with the field devices. If you remember, the Stuxnet attack did not directly infect the Siemens PLCs, but rather first infected a computer workstation that already had ethernet access to the PLC network. The engineering workstations were used as a weapon to deliver the Stuxnet malicious PLC logic to the PLCs. Now, because of the above 4 steps, you are aware of the finite number of computers that have the ability to communicate with the field devices using TCP/IP SCADA protocols. These computers should have end-point security on them, and we STRONGLY suggest application whitelisting. Application whitelisting can essentially freeze the computer systems that have access to the PLCs so that only those approved applications can run and execute on those computers. Any new code that is introduced to the system that is not already whitelisted will be blocked and will not execute.
6. Detection compliments active defense techniques – Now if you have gone through all of the above 5 steps, you should feel fairly good about how the system is locked down. The work is not done just yet. You have to assume that active defenses will eventually fail, and there should be a way to detect when you may be under an attack. Implementing both network and host-based IDS products can help alert you when packets are dropped at the perimeter that was intended for your field devices. Now that you know the TCP or UDP ports that your controllers are using, and their destination IP addresses, you can pull in the switch or firewall logs into a SEM product, and have it alert you when any network traffic was intended for any of your field devices or attempted to utilize their protocol ports, especially if it came from a non-approved source address. Parsing logs from the network devices is a great way to obtain situational awareness. To compliment that further, you can also add an IDS solution to the infrastructure and either write or obtain IDS siguatures for known control system vulnerabilities.
There are additional measures for protecting access to ethernet-connected PLCs and field devices, so this is in no way a complete list. It does provide some helpful tips and actionable things that most control system admins can do with their system to prevent accidental or intentional abuse of ethernet-connected embedded devices.
The main things that I took away from the basecamp PLC disclosures are:
– Almost all ethernet-connected field devices are vulnerable
– You can not secure your environment unless you understand how it works down to the device level
– Place as many access controls in front of the PLC devices as possible to reduce the attack surface
– Block all unnecessary devices and TCP/UDP ports so that only essential traffic is allowed
– Implement strong end-point security on the remaining computers that are allowed to communicate with the PLC devices
– Ensure to extract logs and meaningful security context off of network and host devices to detect abnormal or malicious behavior on the PLC network
Hope my comments in this briefing help illuminate the real issues surrounding the Basecamp PLC disclosures, and provide practical tips for PLC administrators to consider when securing or locking down the PLC network.