How to Identify Data Integrity (and Other) Trends

Why should firms review regulatory noncompliance data?  What useful information can be gained via analysis of such information?

Paul Smith, Global Compliance Specialist, Agilent Technologies, provided examples of why firms in regulated industries should take the time to review regulatory trends during the Aug. 26 Redica (at that time Govzilla) webinar, “Impact of the Global Pandemic on Recent Data Integrity and Regulatory Noncompliance Trends.”

[Related: Download the webinar, “Impact of the Global Pandemic on Recent Data Integrity and Regulatory Noncompliance Trends,” here.]

“One of the questions which I often get asked, is why would it be useful for us to review regulatory noncompliance data?  Why should we do it?  I think for me, the standout is that you have to understand the regulations,” he said, explaining that while global regulations provide a framework that must be satisfied, the regulations do not provide explicit instructions on how to accomplish this. 

This leads to companies applying their own, often varying interpretations.  Working for an instrument company, Smith sees this a lot. 

“I see lots of variation, sometimes small, sometimes large, but nonetheless variation in how companies interpret some of the basic requirements,” he said.  “One of the questions becomes, well, how are the regulators interpreting that requirement?”

A Wealth of FDA Data

Smith pointed to nonconformance data as a particularly rich source of information on how regulators interpret regulations.  He focused primarily on FDA data due to the high quality of information and the volume of data, which encompasses Warning Letters and 483 observations. 

“Fundamentally, there’s at least ten times more data that the FDA has made available than all of the regulators in the world combined,” he explained, also noting that this is changing and other regulatory agencies are releasing more information. 

Two Different Approaches 

In Smith’s experience, companies take two approaches to regulatory trend data. 

“Companies are either interested in data that they can analyze to understand and interpret, or may be looking for specific information.  One, we’ll give them regulatory trends, the data to analyze, or one is more looking for regulatory answers.  So, one is a longer term process, and one is a short-term process,” he said.

The first approach is a more proactive, risk reduction approach while the other is more reactive, typically in response to a problem. 

Figure 1 compares the two approaches, the proactive, long-term approach on top and the short-term, reactive one on the bottom.  Smith explained that while the short-term, reactive approach is “lowest cost,” it is more of an “apparent cost reduction” because “it is hard to get exact data on what is the real cost of noncompliance.”

FDA Trend Analysis in Action

Next, Smith showed how a company could conduct initial analysis of regulatory trend data using the FDA website.  A review of Warning Letters issued between November 2019 to March 2020 shows that the number of inspections decreased between February and March.  This could be due to the interpretation but Smith has seen other February-March timeframes with similar low inspections.  This is an example of why careful analysis and a complete review of available data is critical.

He then showed how to conduct basic searches using the FDA Warning Letters database.  Here, you can filter FDA Warning Letters by year, Center, and response and closeout letters, if available. Yet there are limitations.  For one, the search is limited to just company names. Previously, Warning Letter content could be searched but that is no longer the case. 

While the FDA Warning Letters database contains a wealth of information, it does have limitations. Often the FDA website can be slow.  You can export selected Warning Letters into an Excel file, however, this removes the hyperlinks. 

This process can yield useful information yet it is not suitable for truly identifying trends.  When Smith conducts compliance seminars within the industry, he always recommends customers sign up for the free version of Redica due to its amount of data.  Not only does Redica include information from Warning Letters but it also includes 483 observations and reports from other regulatory agencies.

Identifying Data Integrity Trends

Smith then showed how to effectively search for a regulatory trend.  Data integrity is a critical issue within the industry. His research started by accessing the FDA document, Data Integrity and Compliance With Drug CGMP Questions and Answers Guidance for Industry.

“I did a simple word answer of the questions the FDA asked in the draft and the final guidance,” he said. “For example, question 17 in the draft document had 41 words, in the final guidance, it had 110 words, which was a change of 168%. So quite a large change.”

Smith further showed that looking at the change in wording within the updated guidance shows that FDA is increasingly interested in audit trails.

“So, where a regulator updates that guidance and the change, in this case, focuses on these areas, that’s telling you that as part of their audit program, those are the areas they’re likely to focus on.”

Historical HPLC Trends

Smith regularly researches the role of high-performance liquid chromatography (HPLC) on instrumental techniques. He presented data on HPLC-related Warning Letter citations from 2005 to 2017 (Figure 2).

“You’ll notice that for a part of this period, there were up to 40% of the warning letters for HPLC cited, HPLC usage, or qualification/calibration, or method validation, or how the instrument was being used,” he said.  “But over time, as the regulator started to focus on data integrity, and just for the sake of showing a time point, I’m suggesting this occurs around about 2011, it really depends on the FDA data you look at, where that time point occurs.”

Smith continued, “Essentially, data integrity now dominates, in terms of the wording and content that’s in the warning letters…The risk is that companies only think the regulators are interested in data integrity and not interested in that sort of technical reasons around defending what you do, whether it’s in a lab, or in process, or whatever.  And in this case, it’s a lab.  So those are still things you have to defend, but because of the dominant focus on data integrity, some of that technical content has been missed in the warning letters.”

“One of the things that you have to consider is that you should not really only use warning letters as a source of regulatory data.  You should use quite a few other sources.  And if it’s the FDA, one of those sources is 483 data.  The FDA makes some examples available, but really in order to access 483 data in any reasonable volume, you’re going to have to consider subscribing to an organization that provides a data service such as the one that Redica provides.”

Concerns for Lab Equipment

Smith also discussed the challenges facing labs during the pandemic, notably that engineers often cannot get to sites due to travel restrictions or policies.  Options include having vendors come onsite (dependent on local policies) and swap equipment provided there is redundancy.  Those companies that are proactive about analysis regulatory trends will have an advantage, according to Smith.

“Fundamentally, when the pandemic is over, those decisions will need to be defended when face-to-face audits resume,” he said. “The laboratories that have been more proactive will generally have a much more robust system in place.”

[Related: Paul Smith provided additional examples of how to research regulatory trends effectively. To learn more, download the webinar here.]