Statistics are critical to the pharmaceutical industry, from clinical operations through manufacturing. However, clinical and manufacturing statistics represent entirely different worlds. Where they might be well staffed on the clinical side, some pharmaceutical companies today aren’t hiring qualified staff to analyze operations data, resulting in misapplied tools, inadequate CAPAs and superficial root cause analysis, all of which lead to financial loss and noncompliance.
Few people have analyzed these trends more closely than staffers at FDA’s Office of Compliance, which must examine problem cases where inspectors have found inadequate compliance with CGMP.
In a recent interview, Rick Friedman, Associate Director, and Karthik Iyer, statistician and Senior Policy Advisor at CDER’s Office of Manufacturing and Product Quality, now part of FDA’s Compliance super office, discussed problem areas and areas where pharma might learn from the way other industries use statistics.
Representative of a new breed of regulatory professional at FDA, Iyer, who has been with the Agency for two years, has a B.S. in chemical engineering and an MBA, is certified by the American Society of Quality as a Quality Engineer and Six Sigma Black Belt, and spent over 11 years in the chemicals, refining and consumer products industries.
These industries, he says, use standardized methods to analyze root cause and have an understanding of the cost of poor quality. Below is an excerpt from that interview.
PhM: Looking at 483s and inspection notes of the past few years, we continue to see inadequate CAPA and root cause analysis. How can this situation be improved?
R.F.: Companies should assess the effectiveness of their CAPA systems. The C in CAPA (Corrective and Preventative Action) means that the company is reacting to a problem, the P means that it has seen signals of an emerging problem and has acted in a preventive way to avoid the risk.
In pharmaceutical manufacturing today, there is still a lot more C going on than P. For a CAPA program to be truly mature, the company involved must have implemented a system that really does look at the root causes of problems, and does not assume that the root cause is restricted to the product that may have failed on the line that day. Other products may share the common root cause problem.
Even though industry professionals know what CAPA is, generally its practitioners within individual companies are not always the more experienced quality and production people, and in some cases, they’re not always scientifically qualified to get to the true root cause of failures. As a result, some investigations end up being superficial and problems remain there, latently, until they show up later as causes of excessive variability, further batch failures, unexpected delays or shortages, and all these can lead to financial losses.
[Philip B.] Crosby is famous for saying that quality is free. This is because a QA-oriented system allows an organization to prevent problems that are often very costly. If you’re merely reacting, you can’t assure the two essential objectives of any manufacturing operation: consistent product quality and customer supply.
PhM: Are there any lessons in determining root cause that pharma could learn from the petroleum and chemicals industries?
K.I.: Basic unit operations in the refining and chemicals industries have remained virtually unchanged since the 1920s and 1930s. In these industries, which use and produce dangerous chemicals, improvements have focused primarily on worker safety, with OSHA putting a microscope on chemical and refinery workplace safety practices, and EPA looking closely at environmental safety practices. Because they are also highly competitive commodity markets, product quality also became a key differentiator.
In petrochemicals and chemicals, safety concepts combined with the quality concepts advanced by Deming and Juran and merged into one, so the concept of CAPA is more robust in those industries. When chemical and petrochemical companies do an investigation, they use the same approach to determine the root causes of safety and quality problems, resulting in fairly robust processes. No matter what type of incident is reported to EPA and OSHA, regulators look to see whether a formal process has been set up or whether a proven methodology is being used. For example, two companies that I used to work for used a detailed investigative methodology that a third party private company created but a lot of chemical and petroleum companies use. The software package included a system to analyze root cause, offering structured methodology that is consistent, standardized, and calibrated across industries. So proven standardized root cause analysis methods do exist for manufacturing.
R.F.: In this case, regulatory attention from EPA and OSHA helped accelerate the realization by petrochemical and chemical companies that improving root cause analysis was important for both safety and business reasons. For the drug industry today, a credible surveillance and enforcement presence that focuses on the effectiveness of a company’s systems to analyze and resolve manufacturing problems has never been more critical to effectively regulate in a complex global environment.
PhM: Why does it seem that established practice in pharma has veered so far from the code of regulations? For instance, the GMP code requires statistically relevant sampling and never indicates that validation is somehow a three batch closed-end exercise.
K.I.: There are indications in our review of cases and FDA 483 trends that cause us some concern, although we (CDER Office of Compliance) do tend to review issues when there are adverse findings, and those practices are not necessarily representative of the whole industry.
R.F.: We have found that some firms have not sufficiently incorporated the basic staples of monitoring manufacturing operations that have been standard procedure for decades across the industries. This includes SPC and monitoring suitability of incoming raw materials.
The GMP regulations, as you mention, also reflect these basic expectations. But less regulated industries know they must do SPC and they must have reliable materials to maintain a consistent process. That’s how they ensure manufacturing dependability and thrive as a business. They always want to improve because they want a competitive edge. In any industry, quality and compliance can be significantly impacted by the organization’s commitment to robust product and process design, continual learning and improvement, and sound lifecycle decision-making by including the needed subject matter experts.
K.I.: In the cases that come to us, we often see situations where qualified personnel are not there to analyze manufacturing statistics as the regulations require. There may be qualified people in the company, but they are not supporting that particular plant, or analyzing manufacturing data using the correct statistical tools.
R.F.: You need qualified people to perform this evaluation. How can you determine what a statistically relevant sample is if you’re not trained in that area? These kind of specialized staff also are needed when evaluating the state of process control. So bringing the right people to bear in interpreting process trends and making relevant decisions is extremely important.
PhM: In the interests of process understanding, and better root cause analyses, wouldn’t it be better to have QC and manufacturing more closely connected? What’s FDA’s position current on the practice of integrating some manufacturing and quality functions?
R.F.: In the device and drug industry, FDA expects an independence of quality from manufacturing units. It’s a venerable concept that is rooted in many years of manufacturing experience that also goes across industries. For example, there must be a final authority who determines whether procedures are adequate and if a batch is acceptable for distribution. These are the responsibilities of the quality department. The EU and Japan have similar requirements.
When I’d go on plant inspections, at times the company’s quality department would report to the operations vice president or plant manager. When quality is subordinate to operations, even if it is given final decision-making authority on paper, in reality I found it had to carefully pick and choose which of the significant product quality issues it pursued and corrections were frequently either slow or not instituted.
However, at some companies today, there appears to be a wall up between Operations and Quality Assurance. This creates a situation that is almost as problematic as having the two departments fully merged.
At recent ICH Q10 conferences that FDA co-chaired with the EU, we talked about whether quality and production work mostly as partners on a day-to-day basis. Fundamentally, you can’t achieve quality simply because the QA unit is there at the plant every day, but instead because the production unit builds quality into everyday operations, and the engineers, product development and R&D people build it into the design to begin with. But while that is a major part of the picture of how a state of control is achieved, there is much more.
The other part is where we see deviations or atypical events. And there must be strong commitment from the company’s top management—irrespective of department —to document and investigate these. There’s always something unexpected that can come up, there can be a lot of undetected variability in materials and operations.
For instance, when equipment gets older, the equipment may actually fall out of control before the recalibration date. Or sometimes a significant, environmental or operator anomaly impacts the product but is not detected during the operation—so you hopefully catch the problem in the final QC test.
A production department is charged with supplying products to customers in a timely fashion and annual performance reviews are frequently based on the volume they get out the door. Perhaps because of this tension between timeliness and quality, I have seen cases where problems are, at least temporarily, swept under the rug. In contrast, an independent and empowered quality department should be able to put on the brakes, say “Don’t ship this lot yet,” and prevent substandard product from being released. The Quality department has the authority to make these final decisions and oversee the investigation. But production must nonetheless have a significant degree of ownership in the investigation, ideally including playing a critical role in identifying the root cause and leading implementation of an effective solution.
So we hear from industry that a strong and routine partnership has been shown to be the most effective model for identifying and resolving problems. This partnership is characteristic of a healthy quality system. In the less integrated and proactive organizations, operations staff are not given incentives to document deviations, and some managers thus want to minimize documentation that QA would require. If you don’t document deviations on an early basis, you can’t address the problems that are starting to snowball, before you have the failure. So, ultimately, it’s good medicine but I have heard that operations leaders may not always understand that at some companies.
PhM: But then QA might be missing operations data that’s critical to process understanding. What should they do?
R.F.: Production management must believe in quality first, and quality departments should understand that they cannot do their jobs optimally, or properly investigate the root cause of quality failures, unless they have a good relationship with production. This organizational culture starts at the top of the company. Issues need to be surfaced and collaborative problem solving is critical. This is where the industry has started to mature. Today, there is more of a realization that integration between the two functions is needed, but Quality has to make the final call.
PhM: Is the pharma industry underutilizing statistical tools or using them incorrectly?
K.I.: From our perspective, we cannot make a statement that the entire industry is underutilizing statistical tools, but based on what we see at the Office of Compliance, there are four key areas where we find problems: product sampling, process capability, statistical process control, and analysis of variance.
PhM: Can analysis of out of trend data in Annual Product Reports aid process understanding and control?
R.F.: It’s all about the P in CAPA. If a firm doesn’t have a program in place to look for out of trend conditions, then they don’t have an effective prevention program in place. ICH Q10 calls this the “process performance and product quality monitoring system.” This guideline was written by EU, US and Japanese industry and regulators, and included this term to underscore the importance of monitoring daily operations and batch to batch performance to detect when processes start to drift. It’s not only APR data that tell whether a process is about to fall out of control, but day to day monitoring of near term trends.
K.I.: Philosophically, firms agree that it is important to manage trends, and they have the right intent but in many of the cases we’ve seen, the wrong tools have been applied. For instance, we’ve seen cases where companies talk about an SPC chart. These charts can be set up for two different types of activities: a variable chart, for example, might be where you’re measuring something continuous like tablet weight, so you might go from 2 micrograms to 2.1, to 2.3. The alternative is an attribute chart, in which you monitor the number of defects on a tablet. In this case, you are counting, for example, 1 or 2 blemishes per tablet or other defects. These are two different types of charts, and you can’t use them interchangeably or compare one to the other. If you set up a chart that’s counting and compare it to a continuous chart, you’ll be comparing apples to oranges and you won’t detect whether defect count has gone up or down. This goes back to the importance of having qualified personnel doing this work.
We’ve also seen cases where firms have applied specification limits as control limits on a control chart. That’s a big no-no. The whole point of a control chart is to give you a signal of where you’re heading toward a point where there’s some probability of making offspec products.
R.F.: That’s why there are usually inner control and outer control limits that are still within the specification. The cumulative variability of each operation in a process can, if it’s not controlled tightly enough, exceed specification limit.
K.I.: Again, this doesn’t reflect the actions of all drug companies. We don’t want to indict the whole industry, but misunderstanding of cGMP-related statistics is a significant enough issue, suggesting that there is a lot of work to be done. The firm may be making good product, but if they’re using the wrong tool they might change parameters and then start manufacturing bad product. In the end, any tool must be chosen with fitness of purpose in mind.
For more information, click here.