Industrial IoT is the ability to connect manufacturing machines, equipment, and interfaces together so that data can be acquired from each component and used for industrial automation, predictive analysis, remote data collection, and many other related activities. Effectively, IoT allows you to make smarter business decisions faster and prevents downtime.
Recently at a conference on Industrial Internet of Things, I was discussing the need to connect machines to collect data and heard an absolute horror story. It is worth taking time to read it as it may save you and your company significant downtime, frustration, and costs.
The Operations Manager for the company had just acquired a used CNC machine from what they believed was a reputable company. The machine, while ‘reconditioned’ was 15+ years old and had the original PLC controls as well as an Industrial PC. During the installation, the Operations Manager wanted to be sure this machine had direct download capability from their CAD system and tool path generation systems, so the machine was set up on their engineering network by the engineers. During the startup and provisioning of the CNC machine is when the saga really began. Shortly after setup, the CAD systems, which were up to date on anti-virus and malware software, as IT maintained these machines, began to indicate intrusions and viruses. Other anomalies on the network began to happen as well. The IT team was confounded as to the cause of the virus intrusion as they had not added new machines to the engineering network. They also were unaware of the addition of this new CNC machine as the Operations technicians installing the CNC machine failed to coordinate with IT. Other issues began to happen as well, as other existing CNC machines were going offline, and engineering was unable to download tool path code to them. Significant downtime and loss of production were now occurring. IT was able to use tools such as Wireshark to perform detailed network investigation which lead them back to the infected mystery machine on the network. More time was taken to locate the source which turned out to be the ‘new’ CNC machine which had an industrial PC running Windows XP. The Windows XP machine had never been updated or even patched. It was the source of the intrusion and infection of the network. Apparently, the machine supplier had checked out the PLC and hardware operation of the machine but failed to check into the industrial PC and its health.
The CNC machine was pulled off the network and cleanup of the entire engineering network had to take place, which was not easy nor cheap. Frustration, downtime, and cost were high. The IT team was consulted by Operations and the need to update the Windows XP industrial server was undertaken with the machine supplier. The next roadblock came in when it was determined that the code running on the XP system was not able to be upgraded. The machine supplier was of little help and it almost looked as if the ‘new’ CNC machine was to become the ‘oldest’ and most useless machine, as little could be done to get it back on the network. The low cost ‘great deal’ which Operations thought they had gotten on the machine, now appeared to be a very costly investment they might never even be able to run. Add to the mix that Operations now had a virus, and selling the machine was not an option without full disclosure.
Fortunately, the IT team was proficient in migrating old software and systems to more current operating systems and was able to work with the code and get it running in XP compatibility mode. However, this took them off other pressing projects and was costly. The company was finally able to get the machine online without virus and running properly. IT then began to set up processes to modernize and keep the operations side of the technology systems up to date and properly maintained. Operations now consults with IT before purchasing new equipment and new rules and guidelines have been put in place to get machine vendors to sign off on the state of all new technology purchases of equipment and devices.
The bottom line…. had Operations been consulting and involving IT and the networking team, costs, downtime, and headaches could have been avoided. The great deal on the used machine and equipment also needed to be fully evaluated not just from a manufacturing capability, but also from a technology functionality aspect. Welcome to Industry 4.0, where IT and OT must work together for success.
To learn more about what Prophecy IoT can do to help prevent a situation like this from occurring, click here.
