Over the past 20 years, it has become an essential business requirement for process manufacturing plants to use digital automation systems to compete in a global economy. During this time period, the reliability of electronic equipment has also significantly improved. As a result, management has often redirected limited plant resources to perform higher value-added continuous improvement projects, with less time being allocated to manage the automation system. Today’s lean manufacturing environment provides limited technical expertise and reduced staffing levels. In turn, some plant site’s awareness and processes are not fully optimized for reacting to disruptive events or unexpected failure(s) in their automation system. Even when a plant is in full regulatory compliance, any loss in monitoring and control functionality while diagnosing a problem increases risk to the plant’s overall site health, safety, product and the environment (HSE).
To address these challenges, many plant sites are investing in an affordable system health monitoring (SHM) solution that continuously monitors the health and fitness of the automation system, detects trends and uses diagnostic best practices to proactively notify plant staff to take corrective action and avoid unexpected failure(s). As a result, scarce plant resources can be directed to work on higher priority projects without having to worry about handling an unexpected system failure.
WHY IS MONITORING A CHALLENGE?
Retaining experienced automation professionals to work at plant sites is often challenging for many reasons. Consistently finding the appropriate level of expertise in all plant locations around the world can be equally difficult. The task of managing an automation system is often delegated to a system administrator, who is responsible for the operation and maintenance of the system.
With continuous improvements in automation technology, it’s often technically difficult and administratively challenging for a systems administrator to keep the hardware and software revision levels up to date and within allowable budget and production schedules. Obtaining around-the-clock, continuous coverage to monitor the automation system is not only tough, it’s often cost prohibitive. Thus, with limited expertise and available time, the amount of resources available for monitoring the automation system are scarce and must be used wisely.
All automation systems offer maintenance and diagnostic displays to indicate operational status of the system and generate alarms in case of failures. However, the type of information provided by the automation system is typically after the fact, that is, after the failure event. Visibility to information leading up to the event(s) that caused the system failure is often not readily available. The response time to react to the system failure may be longer than desired since engineering staff must conduct an investigation to determine root cause and implement the necessary corrective action.
INCREASING SYSTEM RELIABILITY, RELIABLY
Plant staff at Pfizer’s Biopharma facility located in Sanford, North Carolina, faced a challenge of increasing automation system reliability, availability and production uptime, while reducing infrastructure support costs. To address these issues, the plant staff explored industry best practices for shifting monitoring activities from after-the-fact or reactive to implementing proactive processes that improve automation system availability and avoid unscheduled downtime. Secondly, the plant staff explored options for an affordable 24x7x365 system health monitoring solution that would centralize the data collection while providing maximum value for the investment.
DEVELOPING SHM REQUIREMENTS
While there are many commercially available IT-related network health monitoring systems on the market, the SHM solution Pfizer adopted was tailored to specifically monitor a DeltaV distributed control system infrastructure. Requirements for the SHM solution included automatically checking health information of all automation system components including controllers, servers, workstations and safety controllers. Firewalls, cyber security protection devices, uninterruptible power systems and other non-automation system servers and workstations connected to the automation network were also included. The SHM solution had to provide trending and automated diagnostics to help leverage best practices and shift maintenance practices from a reactive to proactive stance. Pfizer expected the SHM solution to deliver a centralized, consolidated monitoring service for the entire plant. The SHM service also needed to integrate and be complementary to existing remote monitoring services.
Within a year of addressing the challenge and exploring options, the plant staff partnered with the digital automation systems supplier and its local service provider to implement the SHM solution at the site. Figure 1 reveals an architecture diagram of the SHM system. The SHM monitoring device is coupled to the control network and to the plant-wide information network behind a corporate firewall. An SHM monitoring device automatically checks health information of any network device connected to the control network.