In recent years, significant data integrity lapses have occurred at global pharmaceutical, biotechnology, and medical device firms.  Repercussions have included new products not being approved, rescinded approval of products on the market, closure of manufacturing plants, massive product recalls, FDA import alerts, and drug shortages.

At the same time, data is proving crucial to advancing medical products that cure or prevent disease. Clean, solid data also underpins successful machine learning projects within life science industries. Artificial intelligence applications hold significant promise for developing new patient therapies.

Yet the medical device and pharmaceutical industries remain hesitant to expand their use of artificial intelligence.  Over the past century, healthcare-related industries have amassed immense amounts of data that remain untapped.  This data could potentially be used for unprecedented and groundbreaking analyses. The integrity of the data must be assured; otherwise, those efforts will be wasted.

Fortunately, for those who want to ensure the integrity of their data, there are tried and true methods for performing that activity. Recent developments even provide a more modern, holistic look at data integrity.

[Related: Learn more about trends in data integrity. Download the webinar, “Impact of the Global Pandemic on Recent Data Integrity and Regulatory Non-Compliance Trends” here.]

At the Xavier Artificial Intelligence (AI) Summit held virtually in August 2020, a session addressed data integrity and the challenges for artificial intelligence. Industry veteran Steve Niedelman discussed the importance and criticality of data, the consequences of data integrity failures, and how to find and eliminate data integrity issues.

Niedelman is the Lead Quality System and Compliance Consultant in the FDA and Life Science Practice at Washington, D.C. office of law firm King and Spalding. He completed a 34-year distinguished career with FDA and serves as a consultant to medical device, drug, biologics, tobacco, and other FDA-regulated industries. Although not an attorney, Niedelman is an FDA drug, biologics, and medical device expert.

Solid data integrity provides a foundation for the quality and safety of pharmaceuticals and medical devices in the product development and manufacturing phases of the product lifecycle,” Niedelman maintained. “It plays a fundamental role in ensuring that the decisions being made by a company have the supportive documentation behind them and that they can be defended.”

FDA: Under a Data Integrity Microscope

“It is important to understand that everything you do, everything FDA will look at is based upon the integrity of your data,” Niedelman emphasized.

Any discussion about data integrity requires first understanding the language FDA uses internally to define high-quality data. 

How does FDA define high-quality data?  Their definition: “Data are of high quality if they are fit for their intended uses in operations, decision-making, and planning . . . as data volume increases, the question of internal consistency within data becomes paramount….”

It is “extremely important” to understand this, he said, “and it is followed in everything that you do in support of development of data for clinical work, clinical data, clinical studies, machine learning, and artificial intelligence. You need to be able to stand behind the data that you are creating and the data that you are using to make your decisions. That will include the importance of following data integrity principles.”

“Today, I will be presenting information on a mnemonic that is currently in place that is used by many of the FDA regulated industries,” Niedelman said.

Solid data integrity provides a foundation for the quality and safety of pharmaceuticals and medical device

The mnemonic was first used by Stan Woollen of FDA’s Office of Enforcement in the 1990s, and during subsequent years it was expanded with input from EMA. Woollen coined “ALCOA” as a memory device to ensure data is Attributable, Legible, Contemporaneous, Original, and Accurate. Subsequently, EMA input broadened it with additional terms to “ALCOA Plus.”

“It is important that you can support the data, and it is important that you basically follow the premise of ALCOA, which has now grown to become ALCOA Plus.  It has expanded over time based upon experience in the industry and learning that there were additional elements to consider in this mnemonic that was created to assure accurate and timely and contemporaneous data development.”

Consequences of Data Integrity Failures

Failure to ensure data integrity, Niedelman pointed out, may result in undesirable results including bias, concept drift, and ineffectiveness, all of which can affect future decisions in product developmentfor example, whether to go forward with a product in development.  In addition, it may result in unvalidated outcomes that cannot be supported, only to result in post-market issues and problems down the road, including product recalls.

It may also affect a firm’s reputation in the eyes of regulators.  If data integrity issues are identified during inspection or product reviews, that could result in the firm receiving a warning letter or an untitled letter.  Data integrity remediation has been included in several negotiated consent decrees with firms that have been subject to injunction.

It is essential that data integrity serves as the fundamental underpinning to assuring that what your firm is doing can be substantiated.  In other words, you must be able to stand behind it and have consistent controls over the data.

Routine data integrity issues left unchecked can lead to frequent product recalls.  Firms that routinely recall product impact customer loyalty when the customer sees a pattern of continuing problems.

“It is essential that you control your data from the outset,” Niedelman stressed.  “Make sure you have consistent oversight, even for data that is being collected in other geographic areas where oversight may be lax.  Knowing that in advance, it is your responsibility to make sure that appropriate oversight is in place, procedures are being followed, and audit trails are in place.  Have 21 CFR Part 11 compliance systems in place and ensure that employees have been trained and are following those requirements.”

Too frequently employees have not been appropriately trained on Part 11.  Many of the data integrity problems that FDA has identified are associated with nonconformance to 21 CFR Part 11, where people are using shared passwords or people are signing for outputs when they were not even at work that day.

“It just indicates that somebody else is using those passwords, and it raises the question as to who is date stamping that if that person was on leave that day,” Niedelman pointed out.  “Those are the kinds of problems that in part come from ineffective employee training.  But it also emphasizes the importance of understanding Part 11 to assure that your data integrity is protected.”

Routine data integrity issues left unchecked can lead to frequent product recalls

The Importance and Criticality of Data

“Criticality of data serves as the underpinning of everything from concept, to feasibility studies, to pre-market, to manufacturing, to post-market, and unfortunately to recalls and subsequent regulatory actions that may be necessary,” Niedelman pointed out (Figure 1).

Figure 1 Importance and Criticality of Data

“It is all based upon the integrity of the data that you have in hand and are using to make decisions.  It is extremely important to ensure the integrity of the data and to be able to stand behind it and understand it and defend it.”

“When regulators perform an inspection or review drug or device applications, they need to be able to rely on the data they are given.  When they see patterns of data that are too perfect, it raises questions,” Nidelman explained.  “Similarly, they can pull apart data and identify some of the problems I just mentioned associated with the data integrity and employee practices that raise questions about the data that is being reviewed.”

The firm should be able to provide as many assurances of product safety, strength, purity, quality, as possible, all of which depend on the data being discussed.  Any data-driven decisions must be reproducible and based upon data that can support those outcomes.  Personnel need to be trained to understand the importance of Part 11 requirements and of following them.

“While integrity of data is not a new issue, it is one of increasing attention at FDA,” Neidelman said. “Investigators are being trained.  They are receiving specialized training to detect data integrity issues, data manipulation, and in the worst case, when it may be intentional and fraudulent, which then could become criminal in nature.”

As a result, it is important that you maintain vigilance over your data from the very outset.  You do not want to erode your public confidence or impugn your product quality.  That can have an impact on your bottom line and an impact on your firm’s integrity going forward.

When they see patterns of data that are too perfect, it raises questions

Some of the issues are not always straightforward.  It could be the geographical areas ranging from domestic to international, depending on level of oversight.  Different countries do not understand the same importance that FDA understands regarding the level of oversight that may be expected over the data. It does not matter whether you are a large firm or small firm.  It could be an individual employee, or there may be a conspiracy to hide or share data, skew data, or destroy data, which could be viewed as criminal.

Many times, FDA will walk into a firmgenerally a foreign firmand find a whole bunch of data being shredded or already dumped in a trashcan.  That raises issues as to the actual integrity of the data. After all, why isn’t it being stored?  How is it able to be shredded so easily? All of that contributes to the integrity of your data.

“You need to understand the data that is generated both pre-market and post-market can be critical to your firm,” Niedelman said.  “Acts of commission, as well as omission, are equally important.  If you eliminate data unnecessarily or hide data or do not report all the results, these all raise data integrity issues.  FDA has been on top of data integrity for a while. While it is certainly more prevalent in pharmaceuticals right now, it is growing in the medical device area as well.”

ALCOA Expands

Returning to ALCOA, over time additional elements were added to the mnemonic and it became ALCOA Plus to assure that that data was also complete, consistent, enduring, and available (Figure 2).

Figure 2 ALCOA Plus

Niedelman commented on what each of the various elements means for data:

  • Attributable: Specific to staff members via audit trails and e-signatures. No sharing of passwords. Requires compliance to Part 11.
  • Legible: Usable and readable during internal audits
  • Contemporaneous: Recorded during performance of a task, not before or later
  • Original: In the original format, not manipulated, stored elsewhere, or shared
  • Accurate: Maintained using a system that minimizes errors, raw data and analytical results correctly presented

The additional elements include ensuring that the data is:

  • Complete: Everything is included and nothing is missing
  • Consistent: Enforcing the use of approved data acquisition and analysis methods
  • Enduring: Use of a media that maintains and protects records and it does not deteriorate over time
  • Available: Records available as needed

And, it is “really important to understand that following the basic tenets of ALCOA Plus provides you the assurance that your data as building blocks will get you to the endpoint where your data will be useful in making sound decisions.”

[Related: Learn more about trends in data integrity. Download the webinar, “Impact of the Global Pandemic on Recent Data Integrity and Regulatory Non-Compliance Trends” here.]

Redica Systems Enforcement Analytics

Upgrade your CDMO monitoring strategy.

Enter your email address and someone will contact you shortly to get the data and analysis you need.

Redica Systems Enforcement Analytics

Get help choosing the right CRO.

Enter your email address and someone will contact you shortly to customize your insights.

Get Complete Access to the Redica Systems Platform

Please submit your email address and one of our team members will contact you shortly to answer all of your questions and get you started with the new Enforcement Analytics platform.

Experience the Power of Redica Systems

Redica Pro Free

Sign up for your FREE account today and get instant access to:

SEARCH Find the inspection records you want by inspector’s name, company name, site, city, country, etc.

WARNING LETTERS Easily search the largest database of FDA Warning Letters by date, company, FDA office, or subject

SITES Quickly assess inspection records including dates and inspectors across all of your sites

INSPECTIONS Complete list of FDA inspections by date, company, category, and country

Download the Special Report

An Analysis Of MHRA’s Latest Annual GMP Inspection Deficiencies Report

In this summary we identify:

  • Data from 2019 UK MHRA Inspections
  • Trends from 2015, 2016, 2018, and 2019 inspections
  • Conclusions drawn from an analysis of drug inspection data

Download the Special Report

APAC ad

In this summary we identify:

GMP data from FY2017 to FY2020 for the Asia-Pacific region

Top 3 GMP primary and secondary citations

Top 3 483 observations

Quality Management Systems for Small Pharma and Biotech Firms

Webinar: QMS for Small Pharma and Biotech Firms

Presented by Fenton Fong, Founder, Managing Director, & Principal at xCellarate

Get instant access to the webinar video and slides.

What you will learn:

  • Common challenges small pharma firms face when developing a QMS
    Recent enforcement trends involving the Quality Unit
    Impact of the COVID-19 pandemic
Redica Systems Enforcement Analytics

Get Full Access to Redica Enforcement Analytics

Enter your email address and someone will contact you shortly to answer all of your questions and get you started.

Turning Failure into Success

Presented on Tuesday, April 27, 2021 by Steve Greer, Executive Coach, Genesis Assist, and former Quality Leader, Procter and Gamble

What you will learn:

  • How to use the Leadership SOS Model to transform quality culture
  • How to strengthen quality systems to eliminate human error
  • How to generate ideas on how to set your organization up for success for shareholders, FDA, and staff

Panel Discussion – Remote Audits

Panel Discussion opt in image

Tuesday, May 18, 2021
10:30 AM PST / 1:30 PM EST

Secure your spot for this informative 1-hour Panel Discussion among experts sharing their thoughts and perspectives on Remote Audits.

Can’t make the discussion?Register and we will send you a post-show recording.

Panelists:
Barbara W. Unger, Unger Consulting, Inc.
Jerry Chapman, Redica Systems
Stephanie Gaulding, Pharmatech Associates

Who Will Benefit?
This session will be valuable to GMP quality, regulatory, compliance, and management personnel in FDA-regulated industries who want to have a conversation on remote audits and get to know what is going on in the industry.

Download the Special Report

CDMO Comparison opt-in

In this summary we identify:

FY2018-FY2020 GMP data for Lonza & Catalent

Top GMP primary citations

Top FDA 483 observations

Download the Special Report

Download your copy of THE FDA INSPECTION LANDSCAPE IN MID-2020 AND A LOOK AT VIRTUAL INSPECTION

In this summary we identify:

  • Trend analysis of FDA inspections through mid-2020
  • Strategies for preparing and hosting virtual inspections
  • New technologies to support remote inspections
Redica Systems Enforcement Analytics

Compare two companies of your choice.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to run your custom report.

Redica Systems Enforcement Analytics

Upgrade your CDMO management strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need.

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to explore observations and trends. 

Redica Systems Enforcement Analytics

Prepare for your next inspection.

Enter your email address and someone will contact you shortly to explore enforcement trends. 

Redica Systems Enforcement Analytics

Prepare for inspections of Clinical Investigator sites.

Enter your email address and someone will contact you shortly to provide more Clinical Investigator data about your sites.

Redica Systems Enforcement Analytics

Monitor compliance at your Clinical Investigator sites.

Enter your email address and someone will contact you shortly to get you started.

Redica Systems Enforcement Analytics

Compare the enforcement history of your CDMOs.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Compare the inspection history of your CDMO sites.

Enter your email address and someone will contact you shortly to customize your report.

Redica Systems Enforcement Analytics

Evaluate Your CDMOs

Enter your email address and someone will contact you shortly to customize your report.

Experience the Power of Redica Systems

Complete the form and one of our experts will reach out to you to schedule a demo and answer questions about our subscription options.

INDEX INFORMATION  Redica consolidates regulatory data in real-time on the single largest database for quality, safety, and compliance intelligence.

STRENGTHEN DATASETS Redica machine intelligence algorithms identify and triage risk signals to derive meaning from vast amounts of private and public data.

ANALYZE & INTERPRET PATTERNS Redica instantly models the customer-specific regulatory risks, trends, and opportunities that stand to fundamentally change compliance operations.

Redica Systems Enforcement Analytics

Upgrade your inspection strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need to prepare for your next inspection.

Redica Systems Enforcement Analytics

Upgrade your inspection strategy.

Enter your email address and someone will contact you shortly to get the reports and analysis you need.

Experience the Power of Redica Systems

Redica Pro Free

Sign up for your FREE account today and get instant access to:

SEARCH Find the inspection records you want by inspector’s name, company name, site, city, country, etc.

WARNING LETTERS Easily search the largest database of FDA Warning Letters by date, company, FDA office, or subject

SITES Quickly assess inspection records including dates and inspectors across all of your sites

INSPECTIONS Complete list of FDA inspections by date, company, category, and country

Discover the power of the most comprehensive database of 483s in existence.

Our database of over 20,000 483s is retyped and tagged, making it easy to create an Observation Report in seconds.

Eliminate days of work and get your FREE personalized 483 Observation Report.