Extending the Validated Workplace

Jan. 1, 2004
Today's Building Automation Systems Resemble Process Automation Systems---and Pose Comparable Validation Issues
Not so long ago, the utilities and environmental controls of a pharmaceutical manufacturing facility were a given--you needed to have a way to turn the lights on and off, or to adjust the temperature, but beyond that, you'd "set it and forget it." True, specialized areas such as cleanrooms needed extra attention, but the main focus was always on the process control systems to run actual production lines.

Nowadays, building automation systems (BAS) can be nearly as complex as the process control system--a network of controllers, data centers, sensors and video cameras. The reason: 21 CFR Part 11, the electronic-recordkeeping requirements that FDA has imposed on the pharmaceutical industry. On the surface, Part 11 rules simply require that pharmaceutical manufacturers ensure that their electronic production records are stored securely and signed off appropriately.

However, the need to ensure compliance has generated a near-revolution in BAS technology. BASs now feature comprehensive data centers, networking tools, levels of security and redundancy, along with a long list of specification requirements "The pharmaceutical industry, along with semiconductor manufacturing, is leading the way in [adopting] integrated systems," says David Clayton, an analyst with ARC Advisory Group, a Dedham, Mass.-based market-research company.

"We prefer to work with both pharmaceutical clients and BAS vendors at an early stage," says Nejat Babur, mechanical department manager at the engineering-construction company Lockwood-Greene, Somerset, N.J., and now part of CH2M Hill. "We like to have a specific location guide in order to implement the design, and to produce airflow designs that can be as complex as the piping & instrumentation designs used for production systems."

Validation can be inordinately time-consuming. "Speed to market has emerged as the key issue in how BASs are specified and installed," says Simon James, buildings sector manager at Honeywell Building Solutions, Minneapolis. "We have to bring solutions to the market that do not slow down the validation process." While customers remain concerned with traditional performance parameters of BASs, such as energy management and productivity enhancement, getting through the validation tangle is paramount, James adds.

Devices and Interfaces

Most of the leading BAS vendors serve the full range of commercial and industrial clients. The pharmaceutical industry is one of many specialized markets, which, ARC estimates, add up to around $12 billion globally each year. However, those vendors competing in the fast-growing pharmaceutical field offer increasingly comprehensive and tailored systems.

In determining which hardware and software actually goes into pharmaceutical plants, the first decisions are made based on what parts of the BAS can directly affect production (and therefore need full-blown validation) and which parts do not. A dependable starting point for this analysis is the "Baseline Guide on Commissioning," a document produced by ISPE, with advisement from FDA. This Guide (there are several others that have been published or are in the works for other aspects of pharmaceutical manufacturing) defines "direct impact," "indirect impact" and "non-impact" systems with regard to building environmental controls.

If a plant's environmental control system fails and product is adulterated, that is a direct impact. If the system is simply connected to a direct-impact system--say, a chiller that provides cooling to an air-handling unit, with the air-handling unit determined to be a direct-impact system, is an indirect-impact system. And if it doesn't affect product quality at all--for example, the heating and cooling to cafeterias or offices at a facility, it is a non-impact system.

Determining whether a system is direct, indirect or non-impact is "the million-dollar question" for facility designers, says Agostino Renna, director of the life sciences group at Johnson Controls, Inc., Milwaukee, Wis. "An FDA inspector will come into your plant and ask to see how your direct-impact systems are documented. Then that inspector might ask to look at the documentation of your non-impact systems, and ask you to justify that assessment."

However the impacts have been defined, at some point, a validated component will interface with a non-validated part. Here, there is considerable debate about how to handle such interfaces, on both the hardware and software requirements, ranging from using the same networking and supervisory software to installing two completely separate BASs for each "side" of the facility. "When designing a unified BAS, you have to be careful that the unregulated part doesn't affect the regulated part," says Renna. "Otherwise, it's like the sweater with one loose thread--once you start pulling on that thread, the whole sweater could come apart."

Johnson Controls recommends that either "logical" (software) or "physical" (hardware) separation be made between the two components, Renna says. Logical separation is created by employing strict access-control to BAS setpoints and operating parameters, and by ensuring that a change in, say, the lobby airflow will not affect airflow in a temperature-controlled warehouse. Physical separation, while more expensive in terms of installed hardware, might be justified to ensure complete independence of the regulated space.

The distinction between regulated and nonregulated components can also carry over into the BAS's hardware, notes Lockwood Greene's Babur. Typically, the BAS encompasses sensors for temperature, relative humidity and other environmental factors, controllers or other field devices for executing instructions, and centralized data centers or control rooms for supervising overall operations.

"Commercial-grade devices usually don't have the same accuracy or performance as industrial-grade devices," Babur explains. "Industrial sensors might have a +/-0.5-1.0% accuracy, while commercial units might be 5%." In these cases, Lockwood Greene might produce a design that blends the two types of devices, with the industrial-grade units going into the regulated, direct-impact areas.

Because of validation concerns and differences in equipment quality, Lockwood Greene generally favors physically separated systems. "Validation is definitely a factor in the design process," says Faroukh Butt, department manager for instrumentation and automation. "If you have to validate the entire BAS, clients may benefit from having two smaller systems rather than one united BAS. We've had clients of both separated and united systems," he says.

Siemens Building Technologies, Buffalo Grove, Ill., has tried to structure its BAS offerings to the pharmaceutical industry by bundling solutions for four distinct areas: laboratories, animal-research facilities, manufacturing, and warehousing. "There's a specific problem with many facilities whose workspace is continually changing," notes Diane Welpe, marketing manager. "We've had instances where office space is converted to production, and we need to certify that area for production."

More than HVAC

Traditionally, "building controls" have mostly encompassed heating, ventilation and air conditioning (HVAC), especially in the context of commercial buildings. But the new need to secure facilities, as well as the steady growth in automation of other building functions, means that fire safety, access control and other types of systems are being incorporated into an overall BAS.

Most of the leading BAS vendors offer these systems, either themselves or through alliances with other vendors. Here, too, though, questions about whether activities are regulated or non regulated have crept in. Many validated facilities put strict controls on access to clean rooms, for example, and thus records on who entered such rooms can become a part of the documentation a manufacturer needs for an audit trail.

Most pharmaceutical companies are holding back on an overall integration of these systems, however. "I might be an old-timer, but I would be nervous about having all my automated systems on one platform," says James Agalloco, president of Agalloco & Associates, a consulting firm in Belle Mead, N.J. "You need to have some redundancy."

Siemens' Welpe, whose company offers a variety of BAS types, notes that special care must be taken when optimizing alarm system design. "Some responses might be properly handled by an operator onsite, but others might require telephone or pager response," she says.

BAS Meets Process Control

The most intriguing connections are occurring between the BAS and the process automation system. After all, if the BAS--or parts of it--have to meet the same regulatory requirements as the production line, and improved communication technology makes it possible to integrate the two, why not do it?

ARC, in an informal online survey of its pharmaceutical clients this summer, found that more than half of respondents expected some degree of integration between their BASs and their process control systems over the next five years. "In past years, this would not have been cost-justifiable, but now the hardware costs are not as significant," says ARC's Clayton. "Both process control and building automation are more IT-dependent. It might be time to break down the conception of these being two separate worlds."

Most industry experts, however, believe that such integration is still a long way off. "Today, the basic unit for much production control is the programmable logic controller [PLC]," notes Johnson Control's Renna. "That's a good platform for control, but it lacks the embedded logic that is used for environmental control." He says that installation engineers can go through simple yes/no lists in programming an environmental control unit, but a PLC's programming would be a difficult exercise in ladder logic. He concedes, however, that "industry is headed in this direction."

Siemens and Honeywell, both of which have extensive process-control operations, might be considered well positioned for such integration, but both companies say that there hasn't been much demand. "It's technically feasible, but the customer perception of the value of such a hybrid system isn' there," says Phil Chou, a manager in the pharmaceutical business unit of Honeywell. "We've been involved in linking the PLCs on our process side with our BAS," adds Siemens' Welpe, "but typically we do not provide process automation from within our division."

Indeed, current technology makes it relatively simple to communicate a desired value from a BAS over to the process control system, so that the value can become part of the permanent electronic record of a batch or production step. That's one of the capabilities touted by Wonderware, a division of Invensys PLC in Lake Forest, Calif., whose InTouch and InBatch human-machine interface (HMI) software is widely used for supervisory control. "Our software doesn't run the BAS, but it can extract a data point and include that in batch reports," says Yves DuFort, pharmaceutical industry manager. Wonderware makes use of Microsoft software standards, including its SQL database tools, and DuFort says that it's simply a matter of associating a particular sensor or instrument tag to the Wonderware database.

In some pharmaceutical installations, the BAS system can be as critical as the actual production control system. That's the case at B. Braun, an Irvine, Calif.-based producer of intravenous solutions and packaging. "Because of the sterility and cleanliness needs of our products, we have interlocks on the production line. If, say, the air-handling system to a clean room is not operating properly, production will shut down until it's fixed," says Gary Collins, manager of control engineering. The facility uses environmental controllers from Johnson Controls and a variety of process control units run by InTouch.

Production typically involves mixing active pharmaceutical ingredients into solution, while another part of the plant produces IV bags and closures, Collins says. The two come together in filling stations in clean rooms, and the finished products are then steam-sterilized. Regulatory compliance issues are handled from the perspective of the production line, recognizing that there is a need for the environmental controls to transmit data--usually using TCP/IP over Ethernet protocols--to the production line where they become part of the batch record. The company has a 21 CFR Part 11 compliance program under way, but Collins says it is not affecting operations very significantly because of the high level of regulatory compliance that is already necessary.

BACnet protocol

Whenever there are major interfacing issues, there's usually an attempt to create a standard communication protocol, and that is very much the case in BAS. In this case, the standardization effort, called BACnet, comes from ASHRAE (Am. Soc. of Heating, Refrigeration and Air-Conditioning Engineers), with the support of the U.S.'s National Institute for Standards and Technology (NIST). The group has 23 corporate members, a website of its own (bacnetassociation.org) and links to many other automation associations in the U.S. and abroad.

BACnet, technically known as ANSI/ASHRAE Standard 135-2001, has been under development since the mid-1990s, and, as befits an ASHRAE-originated effort, reflects mostly the needs of commercial building systems suppliers and users. It is not tailored to the needs of the pharmaceutical industry. Nevertheless, it is generating interest in the pharmaceutical sector as a way to provide interoperability among BAS products of different vendors.

Andover Controls (Andover, Mass.), for one, is making a big bet on BACnet, offering a range of controllers and other field devices that make use of the protocol. "BACnet is a maturing protocol, but it needs additional work to be fully capable for regulated industries," says John Williams, a product marketing manager at Andover. He says that it lacks the security requirements necessary for electronic signatures, and has limited definition for recordkeeping necessary for audit trails.

The standard became "Internet-aware," and able to handle IP addresses, in 2001, but there are still details to work out. "It's an ongoing development, and there needs to be more education in the pharmaceutical industry before it becomes an option," says Clive Smith, an industry manger at Andover.

Paradoxically, another potential industry standard, LonWorks, which reaches back to the early 1990s, has been put to use in some pharmaceutical installations, especially in Europe, but most industry observers are betting on BACnet. LonWorks, originated by Echelon Corp. (San Jose, Calif.), has over 300 members, and is strong in the electrical systems used in transportation, housing and commercial buildings. It, too, has a members'website (lonmark.org). "It's a bit of a turf battle," sums up Andover's Smith.

Will BAS technology gradually blend with process control, with everything--the HVAC, the access control, the process control and other systems, all talking to each other over industry-standard protocols? It might happen, but don't hold your breath. "Im thinking of the biblical injunction to 'render unto Caesar that which is Caesar's'," says James Agalloco, who figures that process control engineers need their system, and building engineers their own.

"I question whether we'll ever get the entire industry to sit down at the same table," says Johnson Controls' Renna, citing the normal competitive pressures that keep companies from being fully open technically to each other. He notes that his company has a "black box" interface called the Integrator, which can be specified to provide intercommunication between Johnson products and some 400 other vendors' protocols. "Ultimately, if the pharmaceutical industry pushes hard enough, it will get what it wants."

 

Match Your Needs with BAS Supplier Capabilities

Once you've assessed your 21 CFR Part 11 needs, the following basic checklist of vendor criteria can help you select an appropriate building automation system (BAS) supplier fit:

System/Software

Appropriate document-encryption systems

Appropriate digital-signature standards

Custom programming of files

Trending/historical recordkeeping

Alarm events recording/management

Customizable database records

Validation per cGMP requirements

Validation protocols

Functional specifications

Standard operating procedures

Human read and electronic forms

Records protection

Access restriction

 Independent operator actions capture

Data hold for safekeeping during changes

Technology-based auditing

Data-retention integrity

Policy of permitted sequencing

Authority checks incorporated into processes

Remote access to testing and service

Device checking

Maintenance help screens

Signature Requirements

Ability to print e-signatures

Date/time and origin capture

Accuracy verification

Generation of authorized copies

Falsification prevention

Signature/electronic record-linking

Signatures based on biometrics

Multiple ID components

Duplicate ID code and password

Automatic log-off after defined period of inactivity

Customer Support

Training and tutorial programs

Update and defect notification

Operations supervision

Written requirements and procedures

Documented training and qualified trainers

E-business support

Overall/Organizational

Management organization chart

Management succession plan

Documented quality system and procedures

Periodic self-audits

Problem resolution processes

Established testing procedures

Source: Siemens Building Technologies Inc.