Data Integrity and Your Clinical Investigator: What the Data Shows

Data integrity is a standout observation in 483s as revealed by the Redica Systems expert model for GCP from a key set of human clinical study inspections. This is the first in a series of articles on using artificial intelligence (AI) to identify trends in GCP inspections. Part II addresses “responsibilities of the investigator” 483 observation trends. Part III examines specific protocol violations.

Using the proprietary Redica Systems expert model for GCP, we showed that data integrity observations appear in more than half of the 483s issued to clinical investigators (CIs) of human clinical studies over the past 20 years in a key dataset, which includes only 483s that resulted from inspections of CIs during that period. 

In our analysis, the most common type of observation falls under the umbrella of “responsibilities of the investigator,” which will not come as a big surprise to most people in the industry. But our data integrity findings highlight a subject that does not get the attention it probably deserves.

This insight about the prominence of data integrity observations played a small part in Jerry Chapman’s April 6 presentation at the 2022 Society of Quality Assurance Annual Meeting, “Evaluating Clinical Study Deficiencies Found During Inspection Using AI.” In upcoming articles, we will introduce a few more insights from that talk, including our ability to subcategorize those “responsibilities of the investigator” observations and our findings from applying the Redica Systems GCP expert model to Warning Letters in the same 20-year dataset.

[Related: For more insights in this area, download a webinar recording featuring Jerry Chapman.]

Where Did Redica Systems Begin This Analysis?

So, why prioritize this dataset? It is a fairly representative place to begin—broad deficiency statistics from the FDA show that inspections of CIs from 2016–2021 account for about 75% of BIMO inspections, and adding inspections of institutional review boards (IRBs) brings that up to about 87%. This was our rationale for focusing our first GCP expert models on these two areas. The remainder of BIMO inspections were concentrated on sponsors/contract research organizations (CROs)/sponsor investigators. (Note: Our analysis omits GLP, BA/BE, PADE, and REMS inspections.)

Focusing on inspections of CIs, far more of these BIMO inspections (in data covering 2017–2021) resulted in 483s than in Warning Letters, of course, about one-third resulted in 483s, with Warning Letters at fewer than one in 100. And it should also come as no surprise that the number-one observation in 483s overall—beyond just CI inspections—is always 21 CFR 312.60, “protocol compliance.”

Our GCP expert model supplies the detail behind that broad “protocol compliance” category, showing which aspects of protocol compliance deserve your focused attention because they are often found deficient. What is truly important is the content of the observations, not the number of observations—even though industry lore often gets that backward.

How Redica Systems Extracts Crucial Detail

We have more publicly available enforcement documents than any other non-governmental entity, including more than 50,000 unique 483s, 483 Responses, and Establishment Inspection Reports (EIRs). With the help of dozens of data scientists and engineers, our GXP subject-matter experts—Jerry Chapman included — build proprietary expert models for each GXP area. And these allow a computer algorithm to analyze documents as a human expert would.

Our GCP expert model reveals an abundance of data integrity observations—and we can go even deeper

This involves making every document, including pdf documents, text-searchable using optical character recognition, followed by careful proofing. Before it is possible to analyze, we need to parse the text based on logical document sections, each of which serves specific functions and features specific language and other properties.

Extracting the Detail on “Data Integrity”

Unfortunately, data integrity issues are usually not specifically labeled as such. They rarely use the exact words “data integrity,” except under egregious circumstances. And CI inspections often list these observations as related to case histories or documentation, so they are not necessarily even recorded in consistent places.

But using our GCP expert model on our collection of 1,200-plus 483s issued to CIs over the past 20 years, we can uncover an abundance of data integrity observations—and we can go even deeper.

Revealing Subcategories of Data Integrity Observations

We were able to classify the data integrity observations in our 20-year set of CI 483s into several useful subcategories. As you can see in the figure below, more than three-quarters of observations related to data integrity fit into three of these.

• Original Data — whether the data were reliable, whether they were even available, whether they had been altered, and so forth. In our dataset, more than half of data integrity observations fit into this subcategory.
• Attributable — whether documents requiring authentication had been signed, whether they had been dated, whether it is possible to tell who signed and dated them, and related observations.
• Accuracy — verifiable representation of facts.

How Redica Systems Can Help

Our models show what the FDA is looking for, and the deficiencies it finds—not just at a high level of detail, but at an actionable level of detail. 

But our clients can go beyond seeing these observations clearly at a macro level. They can trace them back to specific Warning Letters and specific 483s, giving them a great deal of utility when it comes to due diligence. For example, this utility includes gaining insight into their own operational deficiencies and those of company partners and contract organizations.

Here is how Redica Systems expert GXP models can help you:

• Vendor Quality — Intelligence to identify and control risk with your critical GLP, GCP, GDP, and GMP vendors

Study startup and ongoing vendor monitoring that draws from full regulatory histories on GLP and GCP vendors.

• Inspection Preparation—Access deeper intelligence about inspection trends and inspector tendencies

Strategic inspection preparation based on current inspection findings at your sites and others. Find trends in the data to identify where to focus your attention.

Tactical inspection preparation using complete FDA investigator profiles. What do they tend to look for? What do they tend to find?

For more analysis of clinical investigator 483 observation findings, click here to read Part II.