According to FDA 820.3(z), “Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled.”
Ironically, the actual provision of objective evidence is subjective. In this article, we’ll examine ways to overcome the subjectivity that can influence the quality of your objective evidence and jeopardize compliance.
First, let’s consider the difference between the words "subjective" and "objective." A subjective perspective is influenced by emotion or opinion and cannot be proved or disproved. Conversely, an objective perspective is factual, measurable, and quantifiable.
But what exactly is objective evidence? Clause 3.8.3 of ISO 9000:2015 defines objective evidence as “data supporting the existence or verity of something” and notes that “objective evidence can be collected by performing observations, measurements, tests, or using other suitable methods.” In other words, it’s physical evidence (e.g., records, documentation) relevant to the audit criteria that auditors can inspect and evaluate for themselves.
Objective Evidence in Computer System Validation
In the context of computer system validation (CSV), “provision of objective evidence” means software requirements must be identified, and all validation efforts and test results should be documented. “Consistently fulfilled” means the test must ensure the software functions in all possible scenarios.
Objective evidence should detail what actually happened (or didn’t happen), including what the tester actually observed during testing. Avoid subjective conclusions like “yes/no” and try to minimize the opportunity for confirmation bias. This is easier said than done.
The Dangers of Confirmation Bias in Test Protocols
As humans, we’re prone to confirmation bias, a cognitive error that occurs when we give more importance to information that aligns with our existing beliefs while disregarding information that contradicts them.
Our beliefs can unintentionally alter our observations. Natural bias can creep into our test protocols. This isn’t good because the quality of the objective data we receive is only as good as the quality of the protocols we write. Interpretation of information can also hold bias.
Will Your Objective Evidence Hold Up to Auditor Scrutiny?
When it comes to providing objective evidence to support that a validation activity is, in fact, valid, we are influenced by opinion as to what objective is required. We do not know if our objective evidence is sufficient until it is scrutinized during an audit or inspection. Then, the auditor's or inspector’s professional opinion is subjectively exercised to assess the objective evidence.
Here’s the dilemma: How do we deliver objective evidence with a high degree of certainty that it will withstand regulatory scrutiny?
Applying a scientific method that is risk-based, consistent, and reliable is one way — possibly the only way. But how do we deliver upon these criteria if we depend on manual, paper-based processes that are difficult to manage and subject to human error? The answer: leverage the power of technology.
Tackling Subjectivity with Technology
A validation lifecycle management system (VLMS) replaces paper-based manual processes with paperless electronic records and signatures, allowing users to enforce compliance through programmatic controls, minimizing or eliminating the unconscious human bias that compromises compliance.
The system can also apply risk-based processes consistently. For example, high-risk requirements can trigger actions that require more objective evidence, such as step-by-step electronic signatures, screenshots, or supporting file attachments at the test step level. This can be captured and maintained precisely where the requirement exists in the form of a hyperlink that, when clicked, will immediately reveal the supporting objective evidence.
Capturing and Documenting Objective Evidence Automatically
If you’ve manually captured screenshots and assembled them in paper documents, you know how tedious it can be. With a VLMS, objective evidence in the form of screenshots and file attachments can be captured and documented automatically, directly in the protocol, saving enormous time and effort.
The tool can be configured based on customer-specific requirements to consistently enforce an organization’s risk-based methodology. When authorized users perform validation, they must follow the rules; otherwise, they cannot continue.
Automation allows for objective evidence to be captured and documented at the push of a button. Also, automated trace matrices illustrate test coverage along with successes and failures. Any failures are required to be addressed successfully through to completion.
Objective Evidence and CSA
The FDA’s draft CSA guidance emphasizes the importance of testing, with different techniques (scripted vs. unscripted) requiring varying levels of objective evidence to demonstrate that the software satisfies the intended use. Unscripted does not mean undocumented. If anything, objective evidence takes on a more meaningful role when less formal testing methods (e.g., ad hoc) are used.
ValGenesis VLMS is the first-to-market CSA-ready solution that allows users to demonstrate 21 CFR Part 11 and Annex 11 compliance with strong objective evidence and secure audit trails that stand up to regulatory scrutiny. The system is not only aligned with FDA’s CSA initiative but also expedites an organization’s ability to adopt and deploy it. Learn more at [email protected].