Managing Automation Project Risk

Oct. 21, 2008
Getting a "MAC" involved, early in the process, can save headaches and costs

The upgrade project looked good. All the vendors were lined up, everyone knew the schedule, and all the milestones were posted and agreed to. But one thing nagged at the chief engineer: what about the documentation?

Sure, all the vendors swore they had their ducks in a row, but with so many involved, how could he be sure that all of them had the same documentation standards, and that they adhered to them? Had the prime contractor or end user reviewed all the skid vendor design submittals and compared them to the automation device specifications/standards? How could he be sure that all the vendors had adequate internal testing standards? And what would this mean for validation?

The upgrade was built around skids, which can speed a project to completion but can cause difficulties because of the increased number of vendors involved and the need for coordination and consistency in the supply of devices. Validation! The worry just wouldn’t go away. No validation, no product; just tanks full of useless material that couldn’t be sold. And the company president was starting to get curious. As it turned out, there were good reasons to worry.

Documentation standards and internal testing requirements varied from vendor to vendor, which led to longer validation document development and execution times than expected. It also led to widely varying validation results. Some protocols took longer to execute and resolve than others. It was also discovered that some vendors’ internal testing was inadequate in terms of content and rigor. One vendor did not provide any evidence of internal testing that was assumed to be a “standard” requirement and the customer believed should have been conducted by the vendor.

On the plus side, the OQ validation protocol was written to include the required tests. But on the minus side, there were many test errors that required vendor remediation. This caused several iterations of re-testing and protocol amendments to be executed, putting a strain on the company’s validation resources. As a result, other validation tasks slipped and system water batching had to be delayed several weeks.

What should have been done? The use of skids — standard plant modules, complete with documents and drawings, based on known pre-tested process cells — can help avoid long lead times on new plant builds. The skids are produced in parallel by separate skid vendors, while the site is prepared to receive them and is fitted out with utilities and buildings. The skids are then integrated at the site. Done well, this method is a good way to handle the project and time pressures brought about by market shifts, delayed investment decisions, regulatory requirements and patent expiration deadlines.

Skids are supposed to be plug-and-play, but for that to work all vendors must be on the same page. The main control system (DCS, or distributed control system) must contain accurate information on each skid; if skid vendors veer off track, or use outdated standards or information, the success of the whole project can be endangered, and the schedule pushed back. The chief engineer in our story should have made the decision to use a more programmatical approach with activity diagrams, and to require the vendors to take responsibility for their scope of work and how it related to the overall project, so he could track everything and prevent the project from “skidding” out of bounds.

An activity diagram is essentially a network/block diagram representation of a Gantt chart. Its value is to show a knowledgeable project engineer or project manager that planning has taken place to indicate what work has to occur and in what sequence to mitigate the risk. potential cost and schedule impacts What could the scenario above do to your project? Let’s try to put it into numbers. Assume that 25 percent of the vendor skid testing documentation does not align with or is inadequate to meet validation requirements, and that a typical vendor skid carries 200 devices. Further assume that this occurs with four skid vendors. Table 1 summarizes what can happen.

The scenario played out above is one of several that may face a company doing new installations or upgrades.

Others could include:

  • Lack of oversight over contractors
  • Late process changes resulting from safety design reviews
  • Software delivery issues
  • Commissioning responsibility disputes.

Any one of these can derail or seriously delay a project and add considerably to costs. contrAct A mAc The common thread running through all the risk scenarios above is the lack of someone to take overall responsibility for automation: a main automation contractor (MAC). The MAC approach involves picking the strategic suppliers before final design. This allows for the inclusion of ideas and application expertise to improve the project’s financials and the long-term operating cost of the plant.

It makes sense to contract a MAC as partner for front-end planning to include development of the risk management plan — to do risk identification and response, to act as an application software supplier, to manage I/O databases including skid vendor supplied devices, and to provide validation services, including review and comment to skid vendor validation protocols.

Figure 1 shows typical project organization with a MAC involved. A project without a MAC involved would include many more steps, the possibility of additional complications, and consume many more days. Benefits of the MAC approach The MAC approach involves earlier involvement by the automation contractor in the project, which allows for greater influence over the costs and benefits of a project.

This benefit was validated by the Construction Industry Institute (CII) in a 1998 report, “Reforming Owner, Contractor, Supplier Relationships: A Project Delivery System to Optimize Supplier Roles in EPC (engineering, procurement, construction) Projects.” The CII report stated that, in both theoretical and field implementations, results indicated that PEpC (procurement, engineering, procurement and construction) could produce savings in excess of 10 to 15 percent of the time and 4 to 8 percent of the cost of the traditional EPC process.

Early involvement of a MAC can reduce construction costs and help ensure a faster and more seamless startup. It can help reduce capital expenditure (CAPEX) and operating expenditure (OPEX) spends by reducing project costs, bringing the system on line faster, and providing the mechanism and services to optimize performance and maintain high reliability of the asset.

Advantages of the MAC approach are summarized below:

  • Early involvement leads to accurate estimates and cost control.
  • Equipment selection, which is made later in the process, minimizes risks and costs of change orders.
  • Seamless integration of all control packages is ensured by the MAC.
  • The lump sum turnkey (LSTK) portion of the contract relieves the customer of cost control concerns.
  • Contract definition provides defined add/delete pricing for goods and services.

A well-qualified MAC with global capability can provide local resourcing and support infrastructure that delivers additional benefi ts in operations and maintenance. In the scenario where end user customers require one or more of their skid suppliers to also include the automation system and measurement/control devices, a MAC can provide the service of reviewing and commenting on consistency of design and supply of devices and providing that feedback to the end user and/or the EPC team.

Validation documentation should be developed from an end user customer’s validation policy. An EPC or MAC — if contracted to do so — will develop test documentation that is consistent with and meets the overall objectives of the end user validation policy/strategy. If these requirements are not passed along to major equipment suppliers (such as those for bioreactors, etc.), schedule delays are possible.

If engaged early enough, the MAC, through consistency in device specifi cations, calibration certs, etc. will be able to minimize the end user customer’s efforts to produce remedial test documentation or, depending on the customer’s strategy for validation, would have already provided device documents that could be leveraged/referenced from one plant area to another and/or one equipment supplier to another.

The MAC approach creates a close relationship and improved cooperation between contractor and customer in setting business and process operating objectives. This trust relationship is imperative to the successful execution of MAC projects.

About the Author

Bill Robertson | P.E.