An Alternative to GMP Quality System Auditing in the Pharma Industry

An Alternative to GMP Quality System Auditing in the Pharma Industry

Executive Summary:

• Historically, a quality systems approach to GMP auditing has become common practice. However, this systems approach does not routinely address the essential input into the system – the data.
• Data integrity issues have drastically increased in the last year – and not just in foreign warning letters.
• Why would you do business with a firm whose data you can’t trust?

A QUALITY SYSTEMS APPROACH IS ALWAYS THE BEST WAY TO GO, RIGHT? NOT NECESSARILY.

A Quality Systems approach to GMP auditing has become common practice for both regulatory authorities and internal GMP audit programs at pharmaceutical companies. The approach began with the CDRH QSIT Inspection paradigm and was adopted by CDER and CBER for pharmaceutical drug inspections. CDRH published Guide to Inspections of Quality Systems  in August 1999.  In September, 2006, CDER, CBER, ORA and CVM published Guidance for Industry, Quality Systems Approach to Pharmaceutical CGMP Regulations. The pharmaceutical inspection approach relies on the evaluation of several or all components of the Quality Systems:  quality, production, laboratory, materials, facilities & equipment, packaging and labeling.  FDA deems that if one ‘system’ is out of compliance then all may be deemed to be out of compliance.  This approach provides structure for internal audits, supplier / contractor qualification and monitoring, due diligence efforts and health authority inspections.  Application of this approach, however, has serious limitations in conduct of data management and data integrity assessments.

Routine quality system approach to auditing can frequently fail to identify data management and data integrity shortcomings.  As an example, consider the common questions asked and evaluated during a Quality Systems audit of Laboratory System Out-of-Specification events and how they are managed.  I chose this topic because it demonstrates linkages among laboratory activities, deviation investigations and corrective and preventive actions and thus provides a good example to evaluate the health and effectiveness of a pharmaceutical quality system.  It is also independent of whether the firm in question manufactures APIs, intermediates, or dosage forms. Nominally, an audit of this area will include review of OOS results within a particular time-period or for a particular product, their investigations and outcome, and any corrective and preventive actions and their effectiveness. This often results in determinations that investigations were not adequate for a number of reasons including but not limited to: staff failed to follow procedures, results were invalidated for reasons not scientifically supported, or the number of samples retested were not justified.

Consider though, the initiating action of OOS review is the identification and reporting of a suspected OOS event. What this approach fails to address are the instances where electronic data are not reviewed, and OOS events may not be identified. For example, data could be inappropriately manipulated by repeated manual reintegration until the desired result is obtained.  Alternatively, the failing data could be simply deleted and not addressed at all.  If the electronic system has the critical meta-data (audit trails) are enabled, data reviewers could be aware of these actions if and only if they review the electronic data.  If OOS events occur where staff share logins and passwords, it is impossible to assign responsibility to a unique individual and thus impossible to have documented evidence to know who to question during a suspected OOS investigation. Even in instances where the final test result is ‘meets specification’ it does not ensure that the procedures and processes used to generate those results are acceptable or in compliance with regulatory agency requirements.

Thus, a classical quality system audit is unlikely to evaluate potential OOS results that are found only in the electronic data where they can be obscured, ignored or deleted unless the reviewers evaluate these data.

IS IT TIME FOR A CHANGE?

To address this problem, it may be time to consider a dramatic change in approach to some types of GMP auditing driven by the premise that if we cannot trust the data generated by a firm, why would we want to do business with them?  It doesn’t much matter to me as an auditor how well written the governing procedures and processes are if the underlying data are not complete and trustworthy.  Like any evaluation, this approach must focus on serious deficiencies, those that have potential impact on product quality and patient safety and not just the occasional minor error in good documentation practices.

The proposed approach focuses on identifying the data management and data integrity status at the audit site before proceeding to an evaluation of any specific quality systems.  In this approach, auditors evaluate how data are controlled as they are collected, recorded, processed, reviewed, approved and archived throughout their lifecycle.  The auditor should determine whether the data are under sufficient control to determine whether they have been altered, modified or deleted.  Data considered for evaluation first would be that associated with product release, critical in-process determinations or with release of critical raw materials. Also, included in this evaluation would be the computer validation status of the various systems.  This evaluation would NOT require review of reams of validation documents but rather would focus on the application of general computer system validation principles and how requirements were developed, incorporated and tested as part of the validation process.  It would include a review of how the validation met the requirements of Part 11 and Annex 11.  This approach may result in an audit limited to data management and data integrity when serious deficiencies are identified early on.  When data are not trustworthy, evaluation of several quality systems is unnecessary to be able to make a conclusion regarding the firm’s GMP compliance status.

This approach will likely result in the need to provide additional training for auditors in requirements and expectations for data management and data integrity and how to perform these evaluations.  Among the best training materials are the publicly available forms 483 and warning letters from FDA.  Conduct of data management and data integrity focused audits are not “rocket science”, but they do require a knowledgeable audit group with deep expertise in GMPs, where to look for deficiencies, critical thinking ability to know which questions to ask and how to connect-the-dots.

Another reason to apply this new approach is that it addresses the challenge in all audits or assessments to quickly identify those areas of serious deficiencies with potential impact on patient safety and product quality.  This is particularly critical for visits that focus on vendor qualification, due-diligence efforts and contract manufacture and contract laboratory qualification. Time is generally limited for each of these activities. Between the conference room presentations by the hosting company and limited tour of the facility, that leaves precious little time to evaluate meaningful raw data and original records.  The alternative approach described herein could provide a more efficient use of limited resources and yet will more accurately establish the overall validity of data and reports generated at or by the firm in question.

THE DATA SAYS IT ALL: IT’S DATA INTEGRITY

Over the past four FDA fiscal years, the percent of warning letters that cite data integrity deficiencies has continued to increase as shown in Table 1 and Figure 1.  In FY2016, 80% of warning letters, excluding those issued to compounding pharmacies, cited deficiencies in data management / data integrity.  This was true for warning letters issued both to sites inside and outside the United States. Thus, it seems reasonable that if the FDA is placing such emphasis in this area, we in industry would be prudent to consider the same.

TABLE 1:  Data Integrity Deficiencies in Warning Letters, Non-Compounders

	FY2013	FY2014	FY2015	FY2016
Total WLs	38	22	19	46
US WL Sites citing data integrity	0 of 13 (0%)	0 of 4 (0%)	1 of 3 (33%)	8 of 11 (73%)
OUS Sites citing data integrity	10 of 25 (40%)	12 of 18 (67%)	13 of 16 (81%)	29 of 35 (81%)
TOTAL NUMBER of WARNING LETTERS CITING DATA INTEGRITY	10 (26%)	12 (55%)	14 (74%)	37 of 46 (79%)

Figure 1

CONCLUSION:

Quality system auditing does not generally include detailed review of the raw data, particularly electronic data, underlying the governing processes and procedures.  It thus can fall short in detecting instances of serious data integrity failures that may impact product quality and patient safety.  The failure becomes more pronounced when time for audits and assessments is limited to a day or two specifically for due-diligence evaluations, vendor qualification, and periodic auditing of critical suppliers and contract manufacturers and contract laboratories.

To make the best use of limited resources and time allocated to audits and assessments, I propose auditors focus first on data management and data integrity.  Audits and assessments should establish that the auditee produces trustworthy and valid data before pursuing more broad assessments of other aspects of the pharmaceutical Quality System.  An evaluation of the facility design and visual maintenance should be included based on facility walk-throughs.  Advantages of this approach include that it:

• Reflects the FDA identified seriousness of this issue with approximately 80% of drug GMP warning letters in FY2016, both domestic and outside the US, citing data governance and data integrity concerns. Other global authorities have identified deficiencies in this area but data are not available to establish percentages.
• Requires the auditors to take a new approach and not permit themselves to be isolated in conference rooms evaluating SOPs that cannot provide documented evidence regarding the trustworthiness of data generated by the auditee.
• Can be performed with a limited number of auditors in a limited period of time. Permits rapid determination of the validity and trustworthiness of data generated by the auditee, and thus their overall GMP compliance status.
• Provides a means of effectively qualifying and ensuring effective ongoing evaluations of key players in the drug supply chain based on documented evidence supported by trustworthy data.

The focused auditing approach described herein provides a means to more quickly determine the trustworthiness of GMP data that are necessary to ensure patient safety and product quality.   It also supports an efficient use of limited resources.  After the trustworthiness of the data are established it is appropriate and necessary to move onto a more detailed evaluation of the entire pharmaceutical quality systems.  If data are determined to not be trustworthy, however, it suggests that the additional effort of evaluating the quality system may not be appropriate or value adding until the data management issues are remediated.