In a six-year collection of FDA warning letters issued to clinical investigators (CIs) of human studies, the Redica Systems Expert Model for GCP confirms that “protocol compliance” is the most common category of observation involving “responsibilities of the investigator”—and “dosing issues” is the largest subcategory under “protocol compliance.”
[Related: For more analysis of FDA GCP warning letters, view a recording of the webinar, “FDA GCP Inspection Trends Identified Using AI.”]
Designed to unearth an actionable level of detail from Redica Systems’ vast collection of inspection documents, the Expert Model for GCP also reveals several other subtypes of warning letter citations, as well as their prevalence in the sample. These results differed in notable ways from a similar analysis of a 20-year collection of 483s issued to CIs of human studies.
Redica Systems Senior GMP Quality Expert Jerry Chapman presented these warning letter findings as part of his April 6 presentation at the 2022 Society of Quality Assurance Annual Meeting, “Evaluating Clinical Study Deficiencies Found During Inspection Using AI.”
This is the third article in a series based on that presentation, all of which focus on our findings from enforcement actions issued to CIs. The previous two articles in this series discussed our results from applying the GCP Expert Model to the 20-year collection of 483s:
- The first article in the series, “Data Integrity and Your Clinical Investigator: What the Data Shows,” illustrates the prevalence of “data integrity” observations
- The second article, “The Components of “Responsibilities of the Investigator” Observations,” presents the Expert Model’s division of observations in 483s involving “responsibilities of the investigator” into useful subcategories
Why Examine Warning Letters to CIs?
We gave this dataset priority because it is fairly representative. According to FDA deficiency statistics, inspections of CIs from 2016-2021 account for about 75% of BIMO inspections. Including inspections of institutional review boards (IRBs) in the dataset total accounts for about 87% (see Figure 1). This is why we focused our first GCP expert models on these two areas. The remainder of BIMO inspections were concentrated on sponsors/contract research organizations (CROs)/sponsor investigators. (Note: Our analysis omits GLP, BA/BE, PADE, and REMS inspections.)
Considering only inspections of CIs, far more of these BIMO inspections (in data covering 2017-2021) resulted in 483s than in warning letters; about one-third resulted in 483s, with warning letters at fewer than one in 100.
The Detail Behind “Protocol Compliance”
In warning letters and in 483s, the No. 1 finding is 21 CFR 312.60, “protocol compliance.” But that general category offers little guidance on how to safeguard against deficiencies.
Our GCP Expert Model supplies the detail behind that broad “protocol compliance” category, showing which aspects of protocol compliance deserve your focused attention because they are often found deficient.
What is truly important is the content of the findings, not the number of findings—though industry lore often gets it backward
How Redica Systems Extracts Crucial Detail
With a growing archive that includes every FDA warning letter issued since 2000, as well as 50,000 unique 483s, 483 responses, Establishment Inspection Reports (EIRs), and more, Redica Systems has a larger collection of publicly available enforcement documents than any other non-governmental entity.
With the help of our GXP Expert Models, a computer algorithm analyzes these documents as a human expert would, in order to extract actionable detail for our customers. Our GXP subject-matter experts, such as Jerry Chapman, build proprietary expert models for each GXP area with the help of dozens of data scientists and engineers.
To extract that valuable detail, we use optical character recognition to make every document—PDFs included—text-searchable, and we carefully proofread them all. Finally, we parse the text based on logical document sections—each section serves specific functions and features specific language and other properties. Along with properly directing the algorithm’s search, this careful parsing is crucial for avoiding misleading results. Only then are the documents available for analysis.
[Related: See how Redica Systems gives you access to FDA GCP inspections. Contact us today to test-drive our actionable intelligence.]
“Responsibilities of the Investigator” in Warning Letters
Our warning letter dataset includes 25 such documents issued to clinical investigators of human studies over the past six years. (This is a small number when compared to the dataset behind our similar analysis of 483s, which includes more than 1,200 documents.)
As we turn our model to this data set, what do we see?
Our Expert Model for GCP found a large majority of our warning letter documents—22 of them—contained citations it categorizes as “responsibilities of the investigator.” As in the dataset of 1,200 483s, it is the largest category by far (see Figure 2).
In contrast, our analysis of 483s issued to CIs reveals “Data Integrity” as a large—and possibly unexpected—second-largest category of observation after “Responsibilities of the Investigator” (see Figure 3).
Dividing “Responsibilities of the Investigator” Into Valuable Subcategories
We analyzed the 6-year collection of CI warning letters using the GCP Expert Model based on subcategories we developed for citations involving “responsibilities of the investigator.” The analysis tabulated how many warning letters could be classified into each subcategory—and what share of the total dataset each subcategory accounted for.
As shown in Figure 4, our model categorized warning letters into the following six neat categories of citations, with “protocol compliance” (otherwise known as “conducted according to the investigational plan”) as the top category:
- General Responsibilities Under §312.60 — Conducted according to investigational plan, otherwise known as protocol compliance
- General Responsibilities Under §312.60— Three small subcategories not overlapping with protocol compliance
- Assurance of IRB Review
- Investigator Record-keeping and Record Retention — Case histories
- Investigator Record-keeping and Record Retention — Disposition of drug during study
- Investigator Record-keeping and Record Retention — Miscellaneous
- Investigator Reports — Financial Disclosure Reports
In our 20-year set of 483s issued to CIs, the analogous pie chart looks slightly different, although the top category is still “protocol compliance” (see Figure 5).
Transforming Protocol Compliance Findings into Useful Information
Protocol compliance findings—agency shorthand for “conducted according to investigational plan”—are the most common type of finding in warning letters and 483s.
What is behind those protocol compliance findings?]
When the Redica Systems Expert Model for GCP divides this sprawling category into six valuable subcategories of §312.60 citations, recording their occurrence and their shares of the total dataset, “Dosing Issue” is revealed as the top subcategory in warning letters, followed by “Inclusion/Exclusion Criteria” (Figure 6).
- Dosing Issue — Incorrect dose or administration
- Inclusion/Exclusion Criteria — Not including or excluding subjects according to predetermined criteria
- PI Personal Attention — Problems related to oversight by the principal investigator
- Lab Issue — Failure to conduct labs on time or to read labs at the right time
- Assessment Issue — Inappropriately conducted assessments of different types
- Adverse Event Reporting
Because the dataset includes far fewer warning letters (25) than 483s (1,200), we cannot draw strong conclusions about the differences in our findings, although they may point to interesting subjects for further research.
In our analysis of 20 years of 483s, “Dosing Issue” was less prominent than among warning letters, assuming a share about equal to that of three other subcategories. Also, note that “PI Personal Attention” is far more prominent in our set of CI warning letters than in 483s (see Figure 7).
However, the two breakdowns show a certain similarity—and both point to operational areas where we recommend focusing your attention. They point to where the FDA is finding the most issues on inspection—the best places at your site to look for gaps.
But your partnership with Redica Systems is not limited to that.
How Redica Systems Can Help You
Our models show what the FDA is looking for, and the deficiencies it finds—not just at a high level of detail, but at an actionable level of detail.
And our clients can go beyond seeing citations or observations at a macro level. They can trace them back to specific warning letters and specific 483s, giving them a great deal of utility when it comes to due diligence. For example, this utility includes gaining insight into their own operational deficiencies and those of company partners and contract organizations.
Here is how Redica Systems Expert Models for GXP can help you:
- Vendor Quality — Intelligence to identify and control risk with your critical GLP, GCP, GDP, and GMP vendors
Study startup and ongoing vendor monitoring that draws from full regulatory histories on GLP and GCP vendors.
- Inspection Readiness — Access deeper intelligence about inspection trends and inspector tendencies
Strategic inspection preparation based on current inspection findings at your sites and others. Find trends in the data to identify where to focus your attention.
Tactical inspection preparation using complete FDA investigator profiles. What do they tend to look for? What do they tend to find?
Get started with Redica today!
Get a Demo
We can show you insights into any of your key suppliers, FDA investigators, inspection trends, and much more.