Seems simple enough, but have you ever found yourself wondering why a FAT was taking so much longer than originally expected, or why a simple system FAT is requiring so long to execute? Incomplete execution plans usually are to blame. Successful FATs require that plans be developed as early as possible, and that goals and responsibilities be defined at the outset, so that they can be more easily incorporated into vendor purchase-order requirements, vendor's required FAT capabilities, and the destination facility construction schedule.
The keys to effective FAT plan development vary, depending on the extent to which the design is driven by the ultimate system owner:
Vendor-standard, standalone units are pre-engineered systems designed and automated by the vendor with minimal customization and plant interaction requirements. Examples include autoclaves, clean/pure steam generators, purified water and water for injection (WFI) generation equipment, glassware/cabinet washers, air handling units, air compressors and plant steam boilers
Vendor-custom designed units are designed by a vendor that specializes in a particular type of equipment. Automation for these units may be provided either by the vendor or the owner. Examples of vendor-custom designed units include bioreactors/fermentors, centrifuges, clean-in-place (CIP) skids, filtration skids and chromatography skids
Owner-designed units are designed by the owner (or a third party employed by the owner, such as a design firm) and built by a fabrication vendor. Automation for these units is typically provided by the owner. Owner-designed units can include the same types of equipment as vendor-custom designed units. However, the fabrication vendor may only be able to contribute to layout issues and may not be able to provide design guidance for process and operating specific requirements.
Work Backward From Goals, Criteria
To maximize plan effectiveness, start by identifying goals, including any acceptance criteria, and then work backwards towards defining the FAT content, assigning responsibilities and determining the required vendor involvement. Acceptance critera should be developed early on in the project and documented within the equipment purchase order. The plan should address the following questions:
When determining goals, validation requirements may aid in defining the protocol's contents. Documentation sections can then be structured to support validation requirements. Included would be the FAT's purpose and scope, a system description, a mechanical inspection, mechanical tests, functional tests, and calibration reports. Typically, only some limited tests, such as system walk-down against the design documents or CIP spray-coverage tests, will be used as support documentation for validation.
However, if validation will use other documentation to replace, or augment, documentation for Installation Qualification (IQ) or Operation Qualification (OQ) tests, then the validation group must first review and approve those sections during protocol development and in the final documentation package. Validation team members may be required to witness the execution of the FAT protocol.
For vendor-standard, stand-alone units, the goal is to have a fully functional unit by the time the FAT is executed, so that any testing at the destination facility merely replicates the FAT tests utilizing the destination facility's utility support systems. Frequently, these vendor-standard units are tied into the destination facility automation system for monitoring and data recording. Communication between the vendor-standard unit and the destination facilities' automation system must then be tested.However, the software on the unit itself should have been thoroughly tested during the FAT at the vendor facility.
Similarly, for vendor-custom designed units, the ultimate goal would be to have a fully functional unit by the time the FAT is executed, and to have testing at the destination facility merely replicate tests with the utility support systems of the destination facility. If extensive disassembly and re-assembly will be required, and if the vendor was not responsible for software development, then testing software at the vendor facility during FAT won't provide any real advantage. Even the basic steps of I/O check-out may have to be repeated at the destination facility due to the disassembly and the vendor's involvement would not be required during the software testing. However, some software testing may be required at the FAT to determine if the units meet the performance specifications.
For owner-designed units, the FAT's goal can vary widely, from a purely mechanical check of system components to a fully functional test of the unit including the software. Systems requiring a greater reassurance of the design will require more extensive functional testing at the FAT. For example, if the owner-designed unit is similar to an existing unit that the owner has experience with, limited functional testing may be performed during the FAT, since the owner is confident in the performance of the system and software testing may be completely omitted at the FAT. If the owner-designed unit, however, is a scaled-up version of a complex system or a newly developed design, the owner may want to perform a more comprehensive functional test with at least partial software utilization for control loop testing, such as temperature control. Additionally, the system may be analyzed at the FAT for potential design problems, such as the ability to hold pressure within the sterile boundary of a bioreactor or the presence of cold spots during temperature control or SIP of a bioreactor.
When multiple units are bought from one vendor, travel can be minimized by testing several of the units during the same trip. However, a vendor-specific execution plan will be required for each unit. This plan must:
Activities should be scheduled to allow designated support personnel to work from one unit to the next as efficiently as possible. If the vendor utilities cannot support simultaneous testing of multiple units, utility availability will help determine the schedule. When limited testing space is available, to avoid owner idle time, the vendor may need to move a unit that has completed FAT from the test bay and set up the next unit overnight, utilizing a second work shift.
Automation Requirements Vary
Integrating automation into the FAT poses several potential issues that must be addressed in early plan development stages. Ask the following questions:
For vendor-standard, stand-alone units, software is typically provided by the vendor and full functional testing of the unit should occur at the FAT. Software for the communication between the vendor-standard unit and the destination facility's automation system may not be integrated until the unit is installed at the destination facility. For these vendor-standard units, the owner may chose to only witness the testing of representative units at the vendor shop or even leave it to the vendor to test the unit and only review the completed FAT documentation.
For vendor-custom designed units, the software may be developed by the vendor or the owner, depending upon the software platform to be used and respective experience on that platform. If the vendor develops the software, it is recommended to perform software testing at the vendor shop during FAT. If the owner develops the software, software testing may still occur at the vendor shop during FAT if the required hardware and software is available for the FAT. If all of the hardware or software required for full software testing is not available for the FAT, some minimum hardware (such as a laptop) and software may be required at the FAT to determine if the unit meets the specifications.
For owner-designed units, the owner would most likely perform the software development. As with vendor-custom designed units, it may be advantageous to test the software at the vendor shop during FAT if the required hardware and software is available. The extent of hardware and software required at the vendor shop will depend on the owner's comfort level with the design. For system analysis such as the ability of a bioreactor to hold pressure within the sterile boundary or checking for the presence of cold spots during sterilization of a bioreactor, it may be desirable to have software available to perform these functions. If software is not available to perform this testing, the appropriate control devices could be manipulated manually.
Assign Responsibilities Up Front
Assigning responsibilities and tracking progress ensure that everything is ready for the FAT. The type of unit will greatly influence who has responsibility for which items (Table I). The plan must define clearly what documentation, testing and equipment the vendor must provide. Documentation required prior to FAT execution may include the equipment purchase order and associated specifications, the bill of materials, design drawings such as piping and instrumentation diagrams (P&IDs), general arrangements, mechanical and electrical drawings, and welding certifications, hydrostatic testing and passivation reports and calibration documentation.
First, it must be determined which group should develop the system design, develop the system software, develop the FAT protocol, perform any check-out prior to FAT, perform the FAT, trouble-shoot and resolve any problems that may arise during the FAT, and generate the final FAT report. The owner, end user, validation, and automation requirements will help determine who should review, approve or participate in the FAT.
For vendor-standard, stand-alone units, the vendor has the expertise not only for the design and software development, but also for the FAT protocol development and execution, and any trouble-shooting and problem resolution. Any owner requirements beyond the vendor's standard scope of work should be discussed and resolved as soon as possible. Examples of these additional requirements include the use of specific suppliers for any pieces of equipment or any instrumentation. These additional requirements can include programming changes to allow for monitoring of the vendor-standard, stand-alone system from the owner's automation system. The owner may chose to defer the primary responsibility of all aspects related to these units to the vendor. The owner involvement between the placement of the purchase order and the receipt of the unit at the destination facility may be limited to review and approval of design documents.
For vendor-custom designed units, the vendor would also have the expertise for the design development. However, depending on the software platform, the owner may have the expertise for the software development. If the software is developed by the owner and it is desired to utilize some or all of the software at the FAT, the owner would probably be better suited to develop that portion of the FAT protocol and would be required to take an active role during the FAT execution. If the software is developed by the vendor or not involved at the FAT, the vendor could retain the primary responsibility for the FAT protocol development and execution.
For most owner-designed units, the owner has the expertise for the design and software development and also for FAT protocol development and execution. The vendor may be employed for the software development or FAT protocol development under the direction of the owner. The primary responsibility for all aspects relating to the design of these units lies with the owner.
Troubleshooting and Problem Resolution
Discovering non-conformance, whether in the mechanical design, documentation or performance of a system is time-consuming. A non-conformance report should be generated during the execution of the FAT. Mechanical non-conformances will have to be resolved prior to completion of the FAT; documentation non-conformances may be resolved afterwards, but prior to shipment. Performance or functional non-conformances are the most difficult to predict and, depending on their impact to the overall system design, may require resolution prior to shipment or upon receipt at the destination facility.
For vendor-custom designed units as well as owner-designed units, a check-out should be performed once the system is mechanically completed, but roughly one week prior to the FAT. This check-out should involve minimal owner resources but include a walk-down of the system to resolve any discrepancies between the design documents and the actual system. If software is to be utilized during the FAT, the check-out should also include uploading the software onto the system and performing a dry-loop check. Performing the walk-down and the dry-loop checks before the FAT should help limit the number of non-conformances that will be discovered during the FAT, saving the time needed to make these corrections.
Performance or functional non-conformances are the most difficult to address and may require last-minute design change to the system. These types of non-conformances are most likely to happen in vendor-custom and owner-designed units, since both types of systems are usually customized to owner preference and have not been built and operated in a particular design before.
Anticipate Performance Issues During Design
To avoid lengthening the FAT to address performance issues, vendor and owner design teams need to think about potential problems during the design of the system. For example, if a pneumatically-fed rotary lobe pump is to be operated near its lower flow rate limit, it is not uncommon to experience blow-by. Space could be designed into the system downstream for the installation of a flow control valve to limit the fluid amount that is allowed to pass through the system at the low flow rates.
Other common examples of designing a system for potential performance issues include leaving space for a future piece of equipment, such as a filter or pre-filter or instrumentation that may be required depending on product properties.
The most common documentation non-conformance is the failure to use good documentation practices (GDPs) during execution. Even though these do not lengthen the FAT itself, resolving documentation non-conformances usually will delay shipment. Not resolving documentation non-conformances may ultimately result in not being able to use part or all of the FAT final report during validation of the system.
In general, plan early for successful FAT execution. But also heed the advice of noted architect and educator, Eliel Saarinen: "Always design a thing by considering it in its next-larger context."
Sonja Bryan is focused on the engineering design and execution of biotech facilities, with an emphasis on bioreactor system design, clean-in-place (CIP) circuit integration/optimization, and project delivery. She has a B.S. in Chemical Engineering from the University of Tennesse, and is a licensed P.E.
Table I
Matrix of FAT Responsibilities Based on Type of Unit
Vendor-Standard
Vendor-Custom Designed
Owner-Designed
Design Development
By Vendor with Review and Approval by Owner
By Owner
Software Development
By Vendor with Review and Approval by Owner
By Vendor with Review and Approval by Owner, or
By Owner
By Owner
FAT Protocol Development
By Vendor with Review and Approval by Owner
By Vendor with Review and Approval by Owner, or
By Owner
By Owner
Pre-FAT Execution
By Vendor with Optional Participation by Owner
By Vendor with Optional Participation by Owner
By Owner
FAT Execution
By Vendor with Participation by Owner
By Vendor with Participation by Owner
By Owner with Participation by Vendor
Trouble-Shooting and Problem Resolution
By Vendor with Participation by Owner
By Vendor with Participation by Owner
By Owner with Participation by Vendor
Final Report
By Vendor with Sign-off by Owner
By Vendor with Review and Approval by Owner, or
By Owner
By Owner
