CSA vs. CSV: How They Differ in Terms of Processes, Standards, and Outcome in the Medical Device Industry

Article Context:

1. Computer System Validation
2. Computer Software Assurance
3. Unscripted Testing

Introduction:

CSV (Computer System Validation)

The conventional method for validating computer systems, known as Computer System Validation (CSV), has been in practice since the inception of the United States Food and Drug Administration (FDA) 21 CFR Part 11 in 1997. However, technology since that time has changed significantly, with enhanced reliance on automation, broad adoption of 21 CFR Part 11-compliant solutions, the cloud, and abandonment of company data centers for managed server farms. These factors have resulted in ambiguity, confusion, and inconsistency in the practice of CSV across the industry. Furthermore, CSV has morphed into an activity done primarily to secure evidence for auditors, rather than to assure the quality of systems. Somewhere along the line, the thought process became “more is better.”

As the leading regulatory agency, the US FDA Center for Devices and Radiologic Health (CDRH) acknowledged the evolution in technology and the gap in guidance. They set out to change the paradigm through the “Case for Quality” industry collaboration initiative.

CSA (Computer Software Assurance)

The FDA recently released the article, "Computer Software Assurance for Manufacturing, Operations, and Quality System Software." This introduces a new approach called Computer Software Assurance (CSA), which shakes up the conventional approach of Computer System Validation (CSV). CSA encourages a more thoughtful and risk-oriented mindset, homing in on what truly matters: ensuring patient safety, maintaining product quality, and safeguarding data integrity. It's important to note that CSA applies specifically to non-product systems.

CSA guidance addresses the following pain points:

1. Paradigm shift in focus: CSV, as it stands today, is a documentation-heavy exercise. Documentation is done at the expense of critical thinking and testing. CSA brings about a paradigm shift by encouraging critical thinking over documentation. By leveraging the tenets of CSA, companies can execute more testing with less documentation.

2. Leverage trusted vendor data: Take credit for the work that is already done by a trusted / audited vendor—i.e., validation of the core software, vendor testing of software releases, etc. For cloud-based software solutions, the vendor assessment is accessible in the cloud, so that elements of the assessment can be referenced quickly and easily. This reduces time and effort spent on downstream validation.

3. Focus on intended use: Computer Software Assurance allows clearer focus, and limits validation of the software to the manufacturer’s intended use. With CSV, it is far too common to have a Software as a Service (SaaS) system’s out-of-the-box features validated only during client implementation. It is important for a manufacturer to clearly define the system’s intended focus to focus well on terms of validation.

4. Use a Risk-based testing approach: Failure Mode Effect Analysis (FMEA) risk assessments demand considerable effort and are better suited for assessing product and process risks. It often takes weeks to get cross-functional teams together to work through traditional “Severity, Occurrence, Detectability” ratings of system functionalities. It is also often done as an expected documentation deliverable, rather than as a real framework for driving testing against risk. CSA recommends the following streamlined risk assessment process, aimed at actively driving testing. This simple framework includes only two variables:

• A functionality’s potential impact on patient / user safety and product quality
• Implementation method of the functionality

5. Unscripted Testing: During a typical validation testing phase, ~80% of the defects stem from test script / tester issues. This is due to detailed and often rigid click-by-click level test scripts that focus more on documenting steps than on testing the system. Their prescriptive nature defines only one single way to interact with the software, which usually does not reflect the dynamic functionality of today’s technology. It is important to note that “Unscripted” does not mean “Undocumented.” Unscripted testing frees a tester from following a detailed, click-by-click level test script, which allows the tester to conduct free-form testing and documentation of the results. An unscripted test case defines the test objective, but does not include detailed test steps. Limiting required documentation significantly reduces test script / tester errors, while it increases detection of functional irregularities that may be encountered in the live environment.

Conclusion

The strategy that works best for you will depend on your company and how it now operates. Continuous improvement is something that every organisation should aim for, but how, where, and how quickly should be determined by an honest evaluation of where you are right now. In any case, you shouldn't consider yourself to be presented with a black-or-white option. CSA is not a different route, but rather an addition, codification, and improvement of CSV.