Unpacking the Challenges of Network Security in IT and OT Environments

Chris: Thanks John it's great to be on your podcast today. I'm Chris Bihary, the CEO co-founder of Garland Technology, we've been working with EPS Global for over five years.

The ‘IT’ and ‘OT’ are distinctions we talk a lot about. The simple way I like to describe the differences is that IT is generally very sophisticated, clean data centers. You walk into a banking customer and see a pristine data center: power, cooling, all your racks. It's all about the data so obviously that's the most important thing. You could have a theft of data and lose money occurring on the IT side, but it's totally different than OT. With OT, we're talking about critical environments that can be life or death situations. Somes examples: a breach where air conditioning units in facilities are shut down; when we're dealing with cranes at a port; a breach of a water system where toxins are put in. Obviously, IT and OT are very important to secure, but in the IT world, it's a financial risk. You may lose IP, but OT is a different scenario because it can be life and death.

Chris: From my past experience, I've done some work in 911 centers and securing OT environments, doing a lot in IT from financials to SaaS. I really look at the basics. Garland Technology as a company is in the infrastructure side, we're in the foundation. What happens on the network, you have two devices communicating, you have packets flowing on your network. If we wanted to analyze our conversation, we would want to repeat everything I say and everything you say and get a copy of that. So regardless of IT or OT, I always like to go back to the simplest thing - the packet, meaning that if you have access to the traffic that flows on your network, there's a lot that you can do. If companies had unlimited budget and they had the ability to copy every single packet that ever flowed on their network and stored it and had access to that, think about the power. You can literally see every single thing that occurred. If there was a breach, you could look back and see what happened. If you have the security tools you can review the data and immediately find out why it occurred. So to me, in order to be able to go back and analyze effectively, you really want to have access to the packets and the traffic on your network. Our company's tagline is "see every bit byte and packet." We always say that the truth is in the packet.

Chris: In the OT sector, there's a tremendous need for security. This could be manufacturing, utilities, in Europe, in the US. A lot of regulations are coming into effect and timeframes are being set for when certain security specifications must be met.

One of the things that's moving really fast is the need for these OT industries to quickly deploy asset management, security, and intrusion detection systems. However, many of these environments are challenging because the IT and network security isn’t necessarily the priority when a new manufacturing facility is being set up. The focus tends to be on productivity, margins, and safety. Network and security are often afterthoughts.

As a network engineer on the IT side, you're usually given a detailed network diagram and have the racks laid out. But in many manufacturing and utility sectors, that documentation isn't generally available. For example, we were working with a manufacturing company that had a couple of hundred sites, and every site had different routers, switches, and equipment. So, when there's a need to do something quickly, it becomes a real challenge to deploy.

The current situation is intriguing as it unfolds rapidly; however, it presents various challenges when individuals venture into the field to commence installations. They realize that each site might require different equipment, making the process more difficult. While it's acceptable to use 100BASE-FX or 100BASE-LX if you're making money and the production lines are operational, connecting to these environments can be particularly challenging. I see very quick growth in these segments to deploy network security, but I also see that once they make a commitment, they find that there's a lot of challenges with the deployments.

Chris: Yes. Often, the urgency lies in the substantial time and effort companies invest in selecting vendors for their security solutions. After purchasing the equipment, deployment becomes a significant challenge. This is particularly true on the OT side.

For instance, I worked with an insurance company a few years ago that purchased new intrusion prevention systems. They wanted to deploy them in-line but didn't realize that they were migrating from 1G to 10G. They lacked the appropriate tapping infrastructure - bypass tap. Consequently, the equipment they invested in started collecting dust because they couldn't physically deploy it.

This issue is increasingly common in the OT side. We've worked with utilities where security tools are left unused in boxes, collecting dust. This is often because they didn't realize that some sites have unmanaged switches, or there's no way to connect it to the network. In many environments, the equipment is managed by providers like ABB, Honeywell, Schneider Electric. The Director of Security’s job is to ensure the security of their locations, but if their equipment is managed by a third party it can become a significant problem.

This is where we introduce wiretapping technology. We bring in a non-intrusive device that you plug between a PLC (Programmable Logic Controllers) and a switch, or a switch and a router. We disconnect the cable and connect to the wiretap, then take another cable. If you have a PLC, you connect it to the tap, and then the TAP to the switch. This creates a passive connection where traffic flows, but the wiretap, if powered, starts copying all the packets from that segment. This allows you to access packet-level data and deploy your security and monitoring tools.

The urgency is often, "I spent all this money, and now I have a problem." Budget issues often arise because Network TAPs, data diodes, or aggregators weren't considered in the initial budget. This can become messy, especially if you have numerous sites with different equipment and poor documentation. The urgency is there, but you have to figure out ways to properly instrument your network to connect them.

Chris: I would say OT provides more challenges because IT is a cleaner environment, generally, you're dealing with data centers. When you move into utilities or manufacturing, you're dealing with utilities at substations, facing environmental challenges. You're also dealing with a lot of older technology.

For example, several years ago, I was working with a company, CyberX, which was later bought by Microsoft. They were doing a deployment for the Super Bowl to protect the water intake and outtake. They were deploying their sensors on several connections. When they arrived a week before, they realized that many of the water treatment facilities were in a ring topology. Two issues arose. One, they weren't sure how to access the packets to connect their sensor. The other challenge was dealing with 100BaseFX.

Many people might not even know what that is. Try to find a network interface card for your computer that will accept traffic coming in 100BaseFX. You're not going to find it. Several years ago, we designed a passive tab that can go into 100BaseFX LX, and it'll convert out to standard one gate copper. We received a panic call, overnighted some TAPs to them, and they got everything connected. Everything ended up working out fine.

But again, all these unique challenges of needing DIN rails, DC power, dealing with higher ranges of temperature. We were working with a manufacturer that was doing cabinets and wanted to do some monitoring near some heated machines. We recently did some work in some nuclear plants and the regulation you have to go through is much more challenging. People had to come to our manufacturing facility and certify the process and see how products are built. There's just a lot more going on. The rigor sometimes that you have to go through is much more challenging than in IT, where everyone knows the standards. Generally, it's going to be in a regulated environment. Everything's going to be in place in terms of your generators, your UPSs, your cooling.

There are just a lot of things that are different, like wiring challenges. If you go on an oil rig, just trying to get wiring installed is very expensive. Often in IT, we can centralize monitoring. However, in some OT environments, it may be necessary to deploy sensors in multiple locations where wiring everything up is not feasible. This field is constantly evolving and presents unique challenges.