Process Capability in Focus in FDA Device Inspections

Process capability—the measure of how well a manufacturing process consistently produces products that meet specifications—is solidly in focus in FDA inspections of medical device manufacturers.

Issues with process capability and the manufacture of defective products are proving not to be ends unto themselves in agency inspections, but clues that point investigators to other areas where potential issues may exist.

At the April virtual FDA/Xavier MedCon conference, FDA Office of Regulatory Affairs (ORA) Office of Medical Device and Radiological Health Operations (OMDRHO) Division 3 Medical Device Specialist and Senior Operations Officer Janet Pulver from the agency’s Los Angeles duty station shared insights regarding what field investigators with FDA expect to see during medical device inspections in the area of process capability.

During her presentation, Pulver gave a general overview of process capability with some insights as to how agency investigators review process capability during inspections as well as the main shortcomings she has seen and how inspection observations can lead an investigator to other areas.

Regulatory Requirements and Definitions

Pulver began by providing the key regulations and definitions that serve as a foundation for the discussion of process capability.

21 CFR section 820.250(a) states, “where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling and verifying the acceptability of process capability and product characteristics.”

In addition to the regulations, Pulver also referenced Juran’s Quality Handbook—a quality “bible” that manufacturing industries have been using for 60 years. It states that process capability is “the measured inherent reproducibility of a product turned out by a process.”

Process capability tells you how well a process produces products that meet expected specifications, Pulver explained. “A process is said to be capable if most of all the measurements fall within the specification limits.”

The two primary indices that the agency calculates are Cp (an indicator of process capability) and Cpk (an indicator of process capability and stability).  Pulver also introduced the concept of “Ppk”—another index of process performance.

“I am only going to focus on capability studies. But you should also be conducting performance studies by calculating the Ppk to determine actual performance of the process in the long term. Cpk is calculated using short-term data or subgroup data, and only tells you part of the story,” Pulver emphasized.

“From an investigator perspective, if I see a more mature process, I will also ask for Ppk studies, even if the firm shows me an acceptable Cpk, because, the Cpk and the Ppk could be different and you could have a nice high Cpk, but a low Ppk, which would tell me that the process is not in statistical control.”

Process Capability Tools

Process capability information can be used for:

• Predicting the variability in a process
• Providing the basis for scheduling process control checks
• Testing theories of causes of defects—for example, to demonstrate that a particular corrective action improved a process, and
• As the basis for quality requirements with suppliers.

Process capability can be better understood by examining visual patterns—for example, the shape of a histogram. Agency investigators expect a normal bell-shaped curve, which gives an indication of the distribution of the process output. The width of the histogram is an indication of the variability or the spread of the process.

A histogram is a graphical representation of the distribution of the numerical data. The horizontal scale shows the bars, which are called bins or classes. The vertical scale represents the frequencies or counts of data points that fall within each bin.

A histogram can be described by the density curve, which is a solid line that you see superimposed over the histogram in programs like Minitab.

The density curve represents the area under the curve and gives the probability of values falling along the curve. For a process with a normal distribution, about 68% of the data points will fall within one standard deviation, about 95% will fall within two standard deviations, and 99.9% will fall within three standard deviations.

The normal distribution refers to the bell-shaped distribution that is described by the mean and the standard deviation. These two charts (Figure 1) show that by looking at the density curves, investigators can get an idea of what is going on with the process.

Assume that Figure 1 is an example of data collected before and after a change was implemented. The blue curve represents the process before the change and the red line is the process after. In the graph on the left, both curves have the same mean, but the red curve is shorter and wider. This means that there is a lot more variation. The tighter the curve, the less the variability.

“When we see before and after data like this, we are certainly going to dig a little bit deeper because obviously the ‘after’ curve shows that a higher number of defective products are being manufactured— the tails are now extending more outside the upper and lower spec limits,” Pulver explained.

The graph on the right shows two sets of data with the same standard deviation, but the mean has now shifted. The curve is now biased to the right. Again, it demonstrates that more defective products are being produced at the upper end. Both examples show a process that is no longer in a validated state and would trigger an investigator to examine other areas.

Interpreting Process Capability Results

“How do we interpret process capability results?” Pulver asked. Before looking at the process spread, investigators first confirm that the process is stable and that the data is normal. Otherwise, the capability results may not be reliable. If the data is not normal, that does not automatically mean that it cannot be used. “But we will make sure that it was handled appropriately, for example, by transforming the data or performing a non-normal capability analysis such as with Minitab.”

“We expect the spread to be narrower than the upper and lower limits because the data falling outside the limits represents nonconforming items. Next, we look at the centering of the process. The center of the peak of the curve should be the center of the target value. If it is not, the process is said to be biased, like in this example where the process is biased to the right (Figure 2).

What do Cpk values tell investigators about a process? Cpk values less than one indicate the mean is biased. Here is an example where it is biased to the left (Figure 3). The process is not centered and the process is drifting outside the limit. This is not a validated process. A significant number of defective products are being manufactured that could go undetected and be released for distribution.

When the Cpk equals one, the mean is centered and the process is just barely meeting specifications. Some defective products are being manufactured. About 0.3% can still fall outside of the limits.

“Based on the risk of this process or the product, the investigator may ask you for justification as to why this would be acceptable,” the ORA investigator said. When the Cpk is greater than one, then the mean is centered and all the data is inside the limits.

Inspection Findings

Pulver provided examples of process issues she has seen during inspections of medical device manufacturers.

The first is lack of process characterization. “You must understand your process and the common and special sources of variation in order to be able to control it,” she stressed.

The classic example that Pulver uses by way of explanation is injection molding, in which the variation of parameters such as temperature, pressure, speed, and cooling time can affect the parts and must be understood to control the process. Also important is the variation between the molds that appear to be the same or variation between the resins that are used. The bottom line is that the process must be understood to enable control.

Also found is a process not being centered and variability not minimized. Both are factors that contribute to non-conforming products being manufactured.

A process is said to be capable if most of all the measurements fall within the specification limits

Also seen is operating at a Cpk that is not commensurate with the risk.

“What is the appropriate Cpk for a process?” Pulver asked. “Typically, we see firms aim for 1.33 or 1.67. Is this appropriate for a particular process? The process yield and the product risk need to be considered. We do understand that to reduce variability, more controls need to be implemented. This is more expensive and takes more resources. But ultimately, it is about protection to consumers and the level of control has to be adequate to make sure that safe products are being distributed.”

Other issues investigators see include a failure to investigate and implement corrective actions to address special cause variation and a lack of feedback to the CAPA system to identify potential sources of nonconforming product or other quality problems.

Sometimes critical specifications are not evaluated. The specifications that are identified in the design outputs as essential or critical to quality should be monitored.

Downstream Acceptance Activities

Investigators have also seen a lack of downstream acceptance activities. Process capability can be used to help determine the frequency of inspections and testing. “But if this is used as a reason for not inspecting a critical characteristic, we will be asking you for justification,” Pulver said.

If it is a characteristic that is critical to product safety, downstream acceptance activities would still need to be implemented—for example, at final release. Depending on the Cpk, there still will be a probability of non-conforming products being manufactured. It might be a small percentage, but still, that possibility exists. If there is no additional inspection step, nonconforming product could be released for distribution.

“On the other hand, I have seen firms use acceptance activity as a justification for not conducting process capability or process control. For example, some companies say they do a 100% inspection of a critical variable, and, therefore, do not need to do process capability or even validation.”

If the data is not normal, that does not automatically mean that it cannot be used

This goes back to the concept that quality must be built into a product rather than tested for. But consider the impact of the human element in the inspection process.

Juran’s handbook discusses studies that have demonstrated the human factor results in an 80% accuracy in finding defects because of inspection errors. Even a 100% inspection could miss 20% of defective products.

“The bottom line is that you should be prepared to provide justification to investigators, making sure that safety of users is a priority.”

[Related: Medical device manufacturers are invited to access a FREE version of the Redica Systems platform. Click here to get started.]

Inspectional Targets

Pulver explained how FDA investigators may use issues found during an inspection to serve as clues that lead to other areas to examine.

“When investigators identify potential concerns, we may look at other areas of the quality system that may reveal some systemic issues—for example, design controls. We make sure that essential outputs are defined and are appropriately identified. Was the risk analysis adequate? Was the design transferred properly into production?”

“Under the CAPA system, we could look at statistical techniques to make sure that analysis of quality data sources is adequate. When issues are identified, did you conduct a thorough investigation and implement effective corrective actions? And we look at complaints and MDRs and recalls to see if there are any issues in the field related to the same defects.”

Under production process controls, investigators typically will review the Device Master Record, the Device History Record, process validation, process monitoring, deviations, and acceptance activities.

Investigators also look for nonconformances for the same defect, equipment maintenance and calibration, and personnel training.

“The worst-case scenario for us to find would be an out-of-control process that results in the non-conforming products being released to the field and causing injury or death to the consumers.”