Jackson Nickerson
In October, data from the first Pharmaceutical Research Manufacturing Project, over 450 pages of it (to download, click here), were finally released. Led by Georgetown University (Washington, D.C.) professor Jeffrey Macher and Washington University (St. Louis, Mo.) professor Jackson Nickerson, the project took four years.
“I thought we’d be done in six months,” says Macher, but it took time —“handholding, interviews and meetings,“ he says — to build the level of trust required to get FDA and manufacturers to participate. “The enormity of data we were asking for probably scared some manufacturers off,” he quips. The fact that the research was completely independent, and funded entirely by Georgetown and Washington Universities may have convinced more pharmaceutical firms to participate, while some stiff nondisclosure agreements didn’t hurt.
Jeffrey Macher
The study was released with the attention-getting press release headline “Drug manufacturing wastes $50 billion a year.” If that number isn’t believable, Nickerson and Macher say it’s extremely conservative.
Early in the survey process, back in 2001-2002, both professors surveyed more than 50 senior pharma executives, FDA officials, and senior managers at biopharma and vendor companies. “To the one, they agreed,” Nickerson says, “that if FDA could change the way it regulated, and if managers could respond to a change in the way it regulated, the industry could save anywhere from 10 to 50% of the cost of goods sold,” he says. Some interviewees suggested more than 50%, but the professors took 15% to come up with the figure (MIT had pegged it earlier at 25%).
The study’s first phase, completed last year, focused on FDA (see "Benchmarking FDA Site Selection and Inspectors" below), cGMP inspection data from 1990 to 2003; results indicate that the Agency’s risk management approach, articulated in 21st Century GMPs, is exactly what the Agency and the industry need right now. Their research also concludes that the Agency’s Inspectorate Training program will soon show results, if it hasn’t already. “We suggest that FDA move more rapidly toward risk-based site selection for its inspections,” Nickerson says. The professors are now moving on to Phase II of the FDA study.
But, if FDA is a secretive organization, it was even more challenging to gain access to the inner sanctum of biopharma and pharma facilities. Information gathered from 19 manufacturers representing 42 facilities in the U.S. and Europe, from 1999 to 2003, a mix of large and small firms, some vertically integrated as well as a few contract manufacturers.
Information contained in hundreds of pages of graphs and analyses can be distilled to the following messages:
- The extent and use of information technology (IT) at any facility has a direct impact on the frequency of lot failures and other problems. In general, Nickerson says, having a centralized database accessible to all, and people trained to use it, improves performance.
- Degree of manufacturing complexity tends to have a negative impact on overall performance, thus contract manufacturing operations tend to show inferior manufacturing performance because they often deal with more complex scale and scope issues, Nickerson says.
- Operational decisions are best made by those doing the work, so the lower the point of decisionmaking, the better the results; facilities where the plant floor operations staff make the day-to-day manufacturing choices generally face fewer ruined batches and other manufacturing problems than facilities where such decisions come from on high.
- Bringing people from outside any given functional area improves performance, particularly when managing deviation.
- New approaches, such as applying process analytical technologies (PAT), are initially disruptive but are likely to accelerate the learning curve. Techniques such as PAT can increase accelerate learning, but the data show that they do not improve performance, at least not at first.
The FDA portion of the study also offers many insights for manufacturers, and suggests that the way downsizings, mergers and acquisitions are handled today may hurt regulatory performance. Consider the impact of mergers and acquisitions, an almost monthly occurrence in this industry. After any merger, Nickerson says, nothing happens to the manufacturing facility at first. But then, after two or three years the “grey hairs” retire or are fired, and new operating procedures, roles and responsibilities, and suppliers are brought in. When these changes finally do occur, the company’s likelihood of being in violation of FDA regulations (or receiving an ordered action issue (OAI) increases significantly.
Anyone looking for information on pharma’s implementation of Six Sigma or Lean programs will be disappointed. Instead, the study examines managerial, organizational and technical practices associated with drug manufacturing. “We were more interested in vertical reporting relationships, and who has responsibility for problem-solving within a manufacturing facility,” Nickerson says.
However, by analyzing performance trends (each company and facility is assigned its own identifier), extremely useful data emerge that can help with any Op Ex or PAT program, and can benefit any department, from HR to management. (For a two-part audio interview with both professors, click here).
Caveat: Don’t Connect the Wrong Dots
Similarly, those looking for direct cause and effect analysis won’t find it in this tome. “These data come with a big caveat,” says Nickerson. “What we’ve done here are simple statistical regressions; we haven’t yet touched causality.”
For example, results showed a drop off in performance when older and more experienced workers were brought in for API manufacturing. They also documented a decrease in performance when companies implemented process analytical technologies (PAT). Drawing direct conclusions in both cases, and applying them to any manufacturing operation, would be erroneous, if not dangerous.
While data haven’t been interpreted yet, it is likely that API manufacturers bring in the most experienced people to solve the most demanding manufacturing problems, and yield isn’t good because of the degree of difficulty involved, Nickerson says. In such situations, bringing in less experienced people would likely worsen the results.
Similarly, companies with complex manufacturing operations and performance issues are more likely to try tools such as PAT to help them solve these problems. “You’re going to use the best tools to solve the most problematic and complex problems, so you’ll see the worst performance associated with PAT, but that doesn’t imply causality,” Nickerson says. “If this study continues, I would expect to see PAT results turn around and have a positive impact on performance in the same facility.” In fact, Macher says, this is often seen with introducing “disruptive” managerial, organizational and technical practices, a topic that he has written extensively on. (Click here to read his most recent paper on this topic, analyzing the semiconductor industry.)
Both Macher and Nickerson show the good working rapport that can only stem from a long friendship. The two worked together as graduate students at the University of California (Berkeley), collaborating on a similar study of manufacturing processes in the semiconductor industry.
But their latest survey is a model for anyone attempting to gather and analyze a massive amount of data in the most user-friendly way. “At Berkeley, we had developed a questionnaire-based survey but too many Ph.D.’s were involved in the process, and that survey ballooned to 250 pages,” Nickerson recalls.
“The smartest thing we did, this time around, was making the survey Internet based,” says Macher. It’s broken down into 11 semi-independent sections, so that each responder could pass it along to resident experts in areas outside his or her scope.
It took each respondent, on average, between one to 1.5 months to gather data for the survey, and they had to fit it in with their day jobs.
Benchmarking charts compare data from oral and topical manufacturing, active pharmaceutical ingredients (API) and parenterals facilities, and include 27 statistical analyses that look into the impact of managerial, organizational and technical practices on manufacturing and deviation management performance metrics.
“We take the big picture view,” Macher says. “There are lots of opportunities for manufacturers to innovate. Innovation would drive down cycle times, improve reliability and decrease deviations, but it’s costly to do, because you have to file with FDA to make changes. FDA’s sending out different inspectors creates a disincentive for manufacturers to innovate.”
Next, Nickerson and Macher will dig down deeper into cause and effect, which will likely take two years of intense work, fit in with their existing research and teaching schedules. Their study, if extended, could bring more enlightenment to pharmaceutical manufacturing managers and professionals. So far, there have been no plans announced to fund more research.
But clearly, even this phase of the study is a mother lode of data just waiting to be mined.
|
Benchmarking FDA Site Selection and Inspectors Macher and Nickerson developed risk-based statistical models to determine the likelihood of any drug manufacturing facility being chosen for inspection; they also looked at the impact that investigator training and experience had on inspection outcomes, and which characteristics defined those facilities and firms most likely to run afoul of the Agency. “It’s no secret that FDA is trying to come out with more robust and sensible internal site selection processes,” says Nickerson, “and we’re trying to add to that effort.” Those facilities that have never been in violation, he and Macher say, deserve a certain amount of regulatory relief, while violators should face greater scrutiny. “This approach benefits everyone, frees up resources and assigns inspectors to those facilities that could have the most detrimental impact on society,” Macher adds. Recently, Washington lawmakers, notably Rep. Henry Waxman, have noted a drop in enforcement activities. Neither professor would comment on that report, except to say that the number of inspections is clearly the wrong metric for gauging Agency performance. “The issue is, not how many times we punish manufacturers, but rather how can we make sure that bad things don’t get out into the public,” says Nickerson. The focus must be on investing and continuously improving the process, on the quality systems that, in turn, encourage innovation. The study didn’t look into the impact of constraints on resources, but some points were clear:
If fewer Agency resources mean that the same inspector is sent out more frequently, Nickerson says, or that some inspectors are focused on pharmaceuticals and others on food, the results could actually be very positive, for both Agency and industry. Clearly, the professors agree, if a site has an ordered action (OAI), it’s likely to be inspected quite frequently for a long time, even if it’s a reliable, quality-focused facility. At this point, they suggest, some other facilities that don’t perform as well often don’t receive the same level of scrutiny, but should. A risk-based approach will be critical to resolving this problem, they agree. Click here to view and listen to a webcast summarizing results of the FDA portion of the research. |
