It’s easier than you think!

Let’s…

Start with a paper record example
Explore how to apply ALCOA to electronic records
Point out 3 warning signs on the trail to data integrity

ALCOA in the Paper World

ALCOA defines the generally accepted standard for GLP and GCP data quality and data integrity. Let’s use paper CRFs at a clinical investigator’s site as a starting point for our example:

A is for Attributable. You can tell who created, modified, or deleted a record. AND you can judge if that person was appropriately authorized to do it! The study coordinator initialed the change he made to a subject’s body weight on the paper CRF. The primary investigator signed the Serious Adverse Event form.

L is for Legible. You can read all the entries on the paper CRF. If a change was made, the original value was crossed out with a single line, and the change was dated and initialed. If the change was made as the result of a query from the sponsor, there is a paper trail supporting the change.

C is for Contemporaneous. Creation, modification, and deletion of data happens at the right time in the process. The study coordinator transcribed data from the subject’s health record onto the paper CRF shortly after the subject visit. (Checking this can be a challenge for paper records.)

O is for Original. The study site retains a carbonless copy of the paper CRF. You can check for the impression of the pen used to complete the paper CRF and even compare it to the sponsor’s copy.

A is for Accurate. Auditors and monitors can compare the data written on the paper CRF to the subject’s medical record to verify accuracy. (At least of the transcription. Accuracy of the source is a different problem!)

A is also for Available. Investigators store the paper CRFs safely, protecting them from fire, water, and other hazards.

Application to E-records

Believe it or not, computerizing processes in R&D in many ways makes checking data quality and integrity easier than the paper world. How? By using the audit trail to your advantage. Never looked at an audit trail? Don’t get flustered – think about who owns the records and what kind of oversight they’d be providing if it were paper. Identify some records of interest and ask for demonstrations.

Depending on the records and the system, there are special things you could look for. You can always look for the following:

Attributable – ask for an Excel listing of all audit trail records of data changes and deletions. If the data set is very large, you might want to bound your request in time (say all records from 1st quarter this year). Can you tell who is making the changes? Are the actions performed by the appropriate people?

Legible – Sample a few data changes and deletions. Is it clear what the original value was? When it was created and who created it? How about the new value?

Contemporaneous – Computerized systems make checking this a dream. If you know when a record should be created in a process, then the audit trail should confirm that. If body weights are being entered before the visit date, you have a problem.

Original – Ask questions about the use of the system and look at the audit trail to understand if the records are first recording of the data.

Accurate – You can use the techniques you already use for paper records to check the accuracy of electronic records. In addition, if the records are transformed by the computer system during processing, examine the validation records for the transformation. Are the requirements well defined and tested?

Available – Electronic records also need protection to ensure their availability through the record retention period. Records should be backed up regularly and stored securely in a different location from the original records. IT and the business should work together to periodically test the ability to restore the records from backup and make sure they are still accessible.

3 Warning Signs

Here are 3 warning signs that you may have data integrity problems:

1. Audit trails aren’t just for IT anymore!

IT is only a custodian for GLP and GCP study data. IT is helpful when it comes to examining audit trails and system logs during a fraud investigation or a security incident.

BUT!

The Study Director owns GLP study data.

The Clinical Investigator owns their site’s EDC data.

If an auditee tells you only IT can see the audit trail, explain that it’s the data owner’s responsibility to be reviewing the audit trail. Not IT. IT can’t be expected to know what changes to your reproductive toxicology or Phase 3 trial data mean. Supervisors and QA routinely review paper lab notebooks. Clinical investigators (or their delegates), CRAs, and auditors routinely review paper CRFs. They do this, in part, to verify the ALCOA attributes. Why wouldn’t they look at electronic audit trails?

FDA inspections in the GMP world demonstrate they are looking at audit trails and expect to see SOPs for reviewing audit trails and those SOPs to be followed.

2. Data changed after it was signed.

A poorly designed system can allow a record to be changed after it was signed and not notify the signer the record needs to be resigned.

Imagine signing a check and having the recipient change the amount!

This is a great place to dig into computer system validation records. Ask to see the requirements, testing, and test results around making a change to a signed record.

Select a sample of signed records. As the auditee, provide a spreadsheet containing the most recent audit trail values for both the records and the associated signatures. If the change date on the record was after the signing date, someone has some explaining to do.

3. The auditee says the paper printouts are their raw data.

Why can’t you just use the paper printouts? Well, you could, if those printouts are the same in content and meaning as the electronic records. Just remember, paper records are rarely the same as electronic records. Read more here…

About the Author

Jamie Colgin is Govzilla’s GCP Product Manager and is the recipient of the prestigious Charles H. Butler Excellence in Teaching Award. She joins Govzilla from Colgin Consulting, Inc.