In a six-year collection of FDA warning letters issued to clinical investigators (CIs) of human studies, the Redica Systems Expert Model for GCP confirms that “protocol compliance” is the most common category of observation involving “responsibilities of the investigator”—and “dosing issues is the largest subcategory under “protocol compliance.”

[Related: For more analysis of FDA GCP warning letters, view a recording of the webinar, “FDA GCP Inspection Trends Identified Using AI.”]

FDA GCP Inspection Trends Identified Using AI - Ondemand Webinar

Designed to unearth an actionable level of detail from Redica Systems’ vast collection of inspection documents, the Expert Model for GCP also reveals several other subtypes of warning letter citations, as well as their prevalence in the sample. These results differed in notable ways from a similar analysis of a 20-year collection of 483s issued to CIs of human studies.

Redica Systems Senior GMP Quality Expert Jerry Chapman presented these warning letter findings as part of his April 6 presentation at the 2022 Society of Quality Assurance Annual Meeting, “Evaluating Clinical Study Deficiencies Found During Inspection Using AI.”

This is the third article in a series based on that presentation, all of which focus on our findings from enforcement actions issued to CIs. The previous two articles in this series discussed our results from applying the GCP Expert Model to the 20-year collection of 483s:

Why Examine Warning Letters to CIs?

We gave this dataset priority because it is fairly representative. According to FDA deficiency statistics, inspections of CIs from 2016-2021 account for about 75% of BIMO inspections. Including inspections of institutional review boards (IRBs) in the dataset total accounts for about 87% (see Figure 1). This is why we focused our first GCP expert models on these two areas. The remainder of BIMO inspections were concentrated on sponsors/contract research organizations (CROs)/sponsor investigators. (Note: Our analysis omits GLP, BA/BE, PADE, and REMS inspections.)

Figure 1 | The GCP Inspection Landscape: Prioritizing which data sets to examine
Figure 1 | The GCP Inspection Landscape: Prioritizing which data sets to examine

Considering only inspections of CIs, far more of these BIMO inspections (in data covering 2017-2021) resulted in 483s than in warning letters; about one-third resulted in 483s, with warning letters at fewer than one in 100. 

The Detail Behind “Protocol Compliance”

In warning letters and in 483s, the No. 1 finding is 21 CFR 312.60, “protocol compliance.” But that general category offers little guidance on how to safeguard against deficiencies.

Our GCP Expert Model supplies the detail behind that broad “protocol compliance” category, showing which aspects of protocol compliance deserve your focused attention because they are often found deficient.

What is truly important is the content of the findings, not the number of findings—though industry lore often gets it backward

How Redica Systems Extracts Crucial Detail

With a growing archive that includes every FDA warning letter issued since 2000, as well as 50,000 unique 483s, 483 responses, Establishment Inspection Reports (EIRs), and more, Redica Systems has a larger collection of publicly available enforcement documents than any other non-governmental entity.

With the help of our GXP Expert Models, a computer algorithm analyzes these documents as a human expert would, in order to extract actionable detail for our customers. Our GXP subject-matter experts, such as Jerry Chapman, build proprietary expert models for each GXP area with the help of dozens of data scientists and engineers.

To extract that valuable detail, we use optical character recognition to make every document—PDFs included—text-searchable, and we carefully proofread them all. Finally, we parse the text based on logical document sections—each section serves specific functions and features specific language and other properties. Along with properly directing the algorithm’s search, this careful parsing is crucial for avoiding misleading results. Only then are the documents available for analysis.

[Related: See how Redica Systems gives you access to FDA GCP inspections. Contact us today to test-drive our actionable intelligence.]

“Responsibilities of the Investigator” in Warning Letters

Our warning letter dataset includes 25 such documents issued to clinical investigators of human studies over the past six years. (This is a small number when compared to the dataset behind our similar analysis of 483s, which includes more than 1,200 documents.)

As we turn our model to this data set, what do we see?

Our Expert Model for GCP found a large majority of our warning letter documents—22 of them—contained citations it categorizes as “responsibilities of the investigator.” As in the dataset of 1,200 483s, it is the largest category by far (see Figure 2).

Figure 2 | FDA Warning Letter Analysis
Figure 2 | “Responsibilities of the Investigator” was the largest major category of citation in our warning letter dataset.

In contrast, our analysis of 483s issued to CIs reveals “Data Integrity” as a large—and possibly unexpected—second-largest category of observation after “Responsibilities of the Investigator” (see Figure 3).

Figure 3 | FDA 483 analysis
Figure 3 | In our dataset of 483s issued to CIs in the past 20 years, “Responsibilities of the Investigator” was the largest major category of observation—but unlike in our warning letter dataset, our GCP Expert Model revealed “Data Integrity” as a second major category.

Dividing “Responsibilities of the Investigator” Into Valuable Subcategories

We analyzed the 6-year collection of CI warning letters using the GCP Expert Model based on subcategories we developed for citations involving “responsibilities of the investigator.” The analysis tabulated how many warning letters could be classified into each subcategory—and what share of the total dataset each subcategory accounted for.

As shown in Figure 4, our model categorized warning letters into the following six neat categories of citations, with “protocol compliance” (otherwise known as “conducted according to the investigational plan”) as the top category:

  • General Responsibilities Under §312.60 — Conducted according to investigational plan, otherwise known as protocol compliance
  • General Responsibilities Under §312.60— Three small subcategories not overlapping with protocol compliance
  • Assurance of IRB Review
  • Investigator Record-keeping and Record Retention — Case histories
  • Investigator Record-keeping and Record Retention — Disposition of drug during study
  • Investigator Record-keeping and Record Retention — Miscellaneous
  • Investigator Reports — Financial Disclosure Reports
Figure 4 | Warning Letters: Responsibilities of Investigator Breakdown
Figure 4 | First-level categories of citations involving “responsibilities of the investigator” in our six-year CI warning letter dataset.

In our 20-year set of 483s issued to CIs, the analogous pie chart looks slightly different, although the top category is still “protocol compliance” (see Figure 5).

Figure 5 | FDA 483 Analysis: Responsibilities of Clinical Investigator
Figure 5 | In our 20-year dataset of 483s issued to CIs, “protocol compliance” (dark blue) took a smaller share of the total, compared to our analysis of the six-year warning letter dataset. “Investigator Recordkeeping” and “Assurance of IRB Review” consumed a larger share of the total. Large-font percentages represent findings in the 483 dataset, while small-font percentages represent the warning letter dataset.

Transforming Protocol Compliance Findings into Useful Information

Protocol compliance findings—agency shorthand for “conducted according to investigational plan”—are the most common type of finding in warning letters and 483s.

What is behind those protocol compliance findings?]

When the Redica Systems Expert Model for GCP divides this sprawling category into six valuable subcategories of §312.60 citations, recording their occurrence and their shares of the total dataset, “Dosing Issue” is revealed as the top subcategory in warning letters, followed by “Inclusion/Exclusion Criteria” (Figure 6).

Figure 6 | Warning Letters: Not Conducted According to Plan
Figure 6 | The larger pie chart represents specific protocol violations drawn from §312.60 findings (those involving protocol compliance, known as “not conducted according to investigational plan”) in a six-year data set of warning letters to CIs.
  • Dosing Issue — Incorrect dose or administration
  • Inclusion/Exclusion Criteria — Not including or excluding subjects according to predetermined criteria
  • PI Personal Attention — Problems related to oversight by the principal investigator
  • Lab Issue — Failure to conduct labs on time or to read labs at the right time
  • Assessment Issue — Inappropriately conducted assessments of different types
  • Adverse Event Reporting

Because the dataset includes far fewer warning letters (25) than 483s (1,200), we cannot draw strong conclusions about the differences in our findings, although they may point to interesting subjects for further research.

In our analysis of 20 years of 483s, “Dosing Issue” was less prominent than among warning letters, assuming a share about equal to that of three other subcategories. Also, note that “PI Personal Attention” is far more prominent in our set of CI warning letters than in 483s (see Figure 7).

Figure 7 | The breakdown of Protocol Compliance Observations in CI Warning Letters and 483s
Figure 7 | Actual, useful subcategories of §312.60 findings, as defined by our GCP Expert Model in a six-year dataset of CI warning letters and a 20-year dataset of CI 483s, respectively.

However, the two breakdowns show a certain similarity—and both point to operational areas where we recommend focusing your attention. They point to where the FDA is finding the most issues on inspection—the best places at your site to look for gaps.

But your partnership with Redica Systems is not limited to that.

How Redica Systems Can Help You

Our models show what the FDA is looking for, and the deficiencies it finds—not just at a high level of detail, but at an actionable level of detail.

And our clients can go beyond seeing citations or observations at a macro level. They can trace them back to specific warning letters and specific 483s, giving them a great deal of utility when it comes to due diligence. For example, this utility includes gaining insight into their own operational deficiencies and those of company partners and contract organizations.

Here is how Redica Systems Expert Models for GXP can help you:

  • Vendor Quality — Intelligence to identify and control risk with your critical GLP, GCP, GDP, and GMP vendors

Study startup and ongoing vendor monitoring that draws from full regulatory histories on GLP and GCP vendors.

  • Inspection Readiness — Access deeper intelligence about inspection trends and inspector tendencies

Strategic inspection preparation based on current inspection findings at your sites and others. Find trends in the data to identify where to focus your attention.

Tactical inspection preparation using complete FDA investigator profiles. What do they tend to look for? What do they tend to find?

Get started with Redica today!

GCP Expert Model

Comparing and Analyzing Russian, UK, and US Top Ten Pharma Inspection Findings FY 2019

Webinar: Comparing and Analyzing Russian, UK, and US Top Ten Pharma Inspection Findings FY 2019

Presented by Jerry Chapman on October 29, 2020

What you will learn:

  • A comparison of 483 observations in FY2019
  • Top 10 Russian Ministry of Health inspection findings
  • Top 10 FDA Warning Letter findings
  • Top MHRA inspection findings
Redica Systems Enforcement Analytics

Get help choosing the right CRO.

Enter your email address and someone will contact you shortly to customize your insights.

Experience the Power of Redica Systems

Complete the form and one of our experts will reach out to you to schedule a demo and answer questions about our subscription options.

INDEX INFORMATION  Redica consolidates regulatory data in real-time on the single largest database for quality, safety, and compliance intelligence.

STRENGTHEN DATASETS Redica machine intelligence algorithms identify and triage risk signals to derive meaning from vast amounts of private and public data.

ANALYZE & INTERPRET PATTERNS Redica instantly models the customer-specific regulatory risks, trends, and opportunities that stand to fundamentally change compliance operations.

Redica Systems Enforcement Analytics

Evaluate Your CDMOs

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Compare the inspection history of your CDMO sites.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Compare the enforcement history of your CDMOs.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Monitor compliance at your Clinical Investigator sites.

Enter your email address and someone will contact you shortly to get you started.

Redica Systems Enforcement Analytics

Prepare for inspections of Clinical Investigator sites.

Enter your email address and someone will contact you shortly to provide more Clinical Investigator data about your sites.

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to explore enforcement trends. 

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to explore observations and trends. 

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to run your custom report.

Redica Systems Enforcement Analytics

Compare two companies of your choice.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Upgrade your inspection strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need.

Download the Special Report

Download your copy of THE FDA INSPECTION LANDSCAPE IN MID-2020 AND A LOOK AT VIRTUAL INSPECTION

In this summary we identify:

  • Trend analysis of FDA inspections through mid-2020
  • Strategies for preparing and hosting virtual inspections
  • New technologies to support remote inspections

Download the Special Report

CDMO Comparison opt-in

In this summary we identify:

FY2018-FY2020 GMP data for Lonza & Catalent

Top GMP primary citations

Top FDA 483 observations

Panel Discussion – Remote Audits

Panel Discussion opt in image

Featuring Panelists:
—Barbara W. Unger, Unger Consulting, Inc.
—Jerry Chapman, Redica Systems
—Stephanie Gaulding, Pharmatech Associates

Who Will Benefit?
This session will be valuable to GMP quality, regulatory, compliance, and management personnel in FDA-regulated industries who want to have a conversation on remote audits and get to know what is going on in the industry.

Turning Failure into Success

Presented on Tuesday, April 27, 2021 by Steve Greer, Executive Coach, Genesis Assist, and former Quality Leader, Procter and Gamble

What you will learn:

  • How to use the Leadership SOS Model to transform quality culture
  • How to strengthen quality systems to eliminate human error
  • How to generate ideas on how to set your organization up for success for shareholders, FDA, and staff
Redica Systems Enforcement Analytics

Get Full Access to Redica Enforcement Analytics

Enter your email address and someone will contact you shortly to answer all of your questions and get you started.

Quality Management Systems for Small Pharma and Biotech Firms

Webinar: QMS for Small Pharma and Biotech Firms

Presented by Fenton Fong, Founder, Managing Director, & Principal at xCellarate

Get instant access to the webinar video and slides.

What you will learn:

  • Common challenges small pharma firms face when developing a QMS
    Recent enforcement trends involving the Quality Unit
    Impact of the COVID-19 pandemic

Download the Special Report

APAC ad

In this summary we identify:

GMP data from FY2017 to FY2020 for the Asia-Pacific region

Top 3 GMP primary and secondary citations

Top 3 483 observations

Download the Special Report

An Analysis Of MHRA’s Latest Annual GMP Inspection Deficiencies Report

In this summary we identify:

  • Data from 2019 UK MHRA Inspections
  • Trends from 2015, 2016, 2018, and 2019 inspections
  • Conclusions drawn from an analysis of drug inspection data

Experience the Power of Redica Systems

Redica Systems

SEARCH Find the inspection records you want by inspector’s name, company name, site, city, country, etc.

WARNING LETTERS Easily search the largest database of FDA Warning Letters by date, company, FDA office, or subject

SITES Quickly assess inspection records including dates and inspectors across all of your sites

INSPECTIONS Complete list of FDA inspections by date, company, category, and country

Get Complete Access to the Redica Systems Platform

Please submit your email address and one of our team members will contact you shortly to answer all of your questions and get you started with the new Enforcement Analytics platform.

Redica Systems Enforcement Analytics

Upgrade your inspection strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need to prepare for your next inspection.

Experience the Power of Redica Systems

Redica Systems Enforcement Analytics

Sign up for your FREE account today and get instant access to Enforcement Analytics.

Champion of Change - 3Cs to Success

Webinar: Champion of Change - 3Cs to Success

Presented by Steve Greer on June 23, 2020

What you will learn:

  • The 3C Model to become a champion of change
  • How to identify game-changing habits and the steps to implement them
  • Ways to develop greater purpose-centered leadership
  • How to generate awareness along with actions to create changes in how you think about challenges and change putting you on a path to lead change successfully in your organization
  •  

Download the Special Report

5 GMP Case Studies Report

What You Will Learn from This Report:

The first countrywide import alert issued by FDA

An inadequate deviation investigation

An inadequate product specifications and a product recall

Process validation issues at a CMO

A different perspective on process validation and the culpability of the quality unit

 

Download the Special Report

Quality Week report sidebar

What You Will Learn from This Report:

How a major pharmaceutical company designed a program to train future leaders in quality

An industry-led initiative to advance the state of quality in the pharma industry

A pharma GMP leader’s tips for supporting quality culture within an organization

An update on FDA’s Quality Maturity Model

Impact of the Global Pandemic on Recent Data Integrity and Regulatory Non-compliance Trends

Webinar: Impact of the Global Pandemic on Recent Data Integrity and Regulatory Non-compliance Trends

Presented by Paul Smith, Agilent Technologies on Aug. 26, 2020

What you will learn:

  • Examples of essential laboratory actions to remain compliant during the pandemic
  • Recent data integrity non-compliance findings and trends
  • Essential strategies to find, understand, and leverage regulatory non-compliance data

Past, Present, and Future of Quality Culture

Presented by Regulatory Compliance Associates Distinguished Fellow Susan Schniepp

What you will learn:

  • How the 2014 FDA quality metrics initiative led to greater focus on quality culture
  • New quality culture tools and standards under development by PDA and other organizations
  • Best practices for quality culture, including case studies with lessons learned

Changing Medical Device Regulations

Presented by Redica Systems 

What you will learn:

  • The latest developments regarding the EU MDR
  • Quality Systems requirements for medical devices
  • Regulatory updates affecting medical devices

Download the Special Report

Combination Product Report

What You Will Learn from This Report:

The latest U.S. and European regulatory developments

The impact of smart combination products

How the EU MDR impacts drug-device combination products

Download the Special Report

5 GMP Case Studies Report

What You Will Learn from This Report:

In-depth analysis of the findings

Lessons learned

How companies can avoid similar shortcomings.

Experience the Power of Redica Systems

Redica Systems Expert Jerry Chapman to Present at SQA Annual Meeting

Index Information Redica consolidates regulatory data in real-time on the single largest database for quality, safety, and compliance intelligence.

Strengthen Datasets Redica machine intelligence algorithms identify and triage risk signals to derive meaning from vast amounts of private and public data.

Analyze & Interpret Patterns Redica instantly models the customer-specific regulatory risks, trends, and opportunities that stand to fundamentally change compliance operations.

Download the 510(k) Sample

What You Will Learn from This Sample:

See how a 510(k) submission is structured

Find out what information to include

Review correspondence between sponsor and FDA

Redica Systems Enforcement Analytics

Unlimited Access to Reports and Data

Enter your email address and someone will contact you shortly.

A.I. for Quality and Compliance Teams

August 3 webinar

Presented by Redica Systems Senior GMP Quality Expert Jerry Chapman

What you will learn:

  • A basic understanding of data sources, machine learning, NLP, and A.I. tools
  • Building and applying Expert Models
  • Two case studies, one on sterile manufacturing/503B and one on data integrity
Redica Systems Enforcement Analytics

Upgrade your CDMO management strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need.

Get the scoop on recent data integrity 483 observations!

  • FY 2020 Data Integrity 483 Observations
  • Data Integrity Observations for Human Drugs
  • Observations Falling Under GMP
  •  

Human Factors – Compliance and Applications

June webinar

Presented by Alison Sathe, Founder, Regulatory Mark & Valerie Fenster, Director of Insights and  Human Factors, Kaleidoscope Innovation 

What you will learn:

  • Human factors strategies from simple drug products to complex surgical devices
  • Primary and secondary labeling development and evaluation for usability
  • Human factors activities from product development to post-market
  • International strategies: expectations for human factors in regulatory submissions in international markets

Experience the Power of Redica Systems

Redica Systems Enforcement Analytics

Let us know who you are and we’ll be in touch to answer all of your questions and get you started.

Redica Systems Enforcement Analytics

Keep up with the latest Regulatory Developments on Clinical Trials

Enter your email address and someone will contact you shortly. 

Redica Systems Enforcement Analytics

Save your team valuable time when using External Monitoring.

Enter your email address and someone will contact you shortly. 

Redica Systems Enforcement Analytics

Save time and Resources when using External Monitoring.

Enter your email address and someone will contact you shortly. 

Redica Systems Enforcement Analytics

Get immediate access to the news that matters to your sites.

Enter your email address and someone will contact you shortly. 

Redica Systems Enforcement Analytics

See how we can help you eliminate yet another spreadsheet.

Enter your email address and someone will contact you shortly. 

Redica Systems Enforcement Analytics

Stay ahead of changing regulations.

Enter your email address and someone will contact you shortly to get the data and analysis you need.

Redica Systems Enforcement Analytics

Upgrade your CDMO monitoring strategy.

Enter your email address and someone will contact you shortly to get the data and analysis you need.

Discover the power of the most comprehensive database of 483s in existence.

Our database of over 20,000 483s is retyped and tagged, making it easy to create an Observation Report in seconds.

Eliminate days of work and get your FREE personalized 483 Observation Report.