Software Validation: A Comprehensive Approach to Software Validation in the Medical Device Industry

Overview

Modern medical device regulations are putting more and more emphasis on the management of software tools.

These tools are software packages that are not part of the medical devices themselves, but support the device during its development lifecycle.

Nowadays companies use dozens of applications, ranging from accounting tools to email clients to software compilers. Of course not all of them have an impact on the product, so which ones should be validated and how?

What is “Validation”?

The term “validation” can have different meanings when applied to different items; for this application, the definitions below are the ones we have to consider:

• Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled. [FDA 21 CFR 820.3 (z)]
• Process validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications. [FDA 21 CFR 820.3 (z)(1)]

or in plain words “demonstration that the software reliably does what is supposed to do”.

Approach to Software Validation

The first step for any situation is to assess the software tool for its impact on the medical device. If it can have an impact on the “quality” of the device (in the broader sense), then software validation may be required. Consider also that the software tools used to manage your Quality System documents, including CAPAs, complaints, NCs, requirements, risks, etc., fall into this category.

The framework for the software validation mimics the well known steps used for process validation: plan, risk assessment, requirements, protocols, results, report.

FDA recently released a draft guidance document on Software Tools Validation that we have summarized here.

Example

I use Atlassian Confluence Cloud for my electronic Document Management System. How can I demonstrate that it is validated?

Scope, Requirements and Risk Assessment

It is often useful to break down the software package into “functions”. Software typically has a number of different features with different importance for your application:

1. What features will be used?
2. Of those, which ones have an impact on quality?
3. Of those, which ones are more critical?

This first screening will help you narrow down the scope of your software validation effort and identify features that need more thorough validation.

Example

What features of Confluence Cloud should I validate?

1. Define the features that you are using. Confluence has hundreds of features but only a handful of them will be used for regulated activities. E.g. Create, edit a page. Permission management. Data retention and integrity.
2. What are the most critical features? For this type of tool, data integrity is probably the most important, followed by permission management and electronic signatures.

Validation Plan and Protocols

This is all well and good if you have insight and knowledge about the object you are validating. But in most cases, especially for the off-the-shelf software tools, the user sees them only as black boxes. Setting up a comprehensive validation for a software tool without having an idea of its internal mechanism is a challenging tasks; not because of what you know, but for what you don’t. It is difficult to develop tests for unknown boundaries and unclear algorithms. And typically this results in significant gaps in the validation coverage.

Recently, more mature software tool development companies have started providing pre-validated software and validation packages aimed at the medical device market. This is a priceless product for a medical company of any size, as it allows to demonstrate compliance using the expertise and knowledge of the developer(s) of the tool; due to their knowledge of the internal processes of the tool, they can put together a relatively lean protocol that adequately challenges the product. It also shows that the software tool developer has an idea about the regulatory framework of the medical device market, which may also help them design software tools that capture the key requirements so dear to the medical regulations but little known to the outside world (e.g. electronic records, electronic signatures, etc.).

A word of caution: it is best practice (if not actually expected by regulatory bodies) to repeat at least part of the software validation protocol in-house, to confirm the results of the pre-validation provided by the developer. It is unlikely that you will be able to adequately control your software tool providers (read: audit them) to be able to solely rely on their own internal activities.

Example

What type of activities are required for software validation?

1. Testing is probably the bulk of the activities. Make sure you include appropriate corner cases.
2. Quality Assurance activities can also support the validation effort. E.g. periodic backups (and backup validation), procedures detailing access control, appropriate training of users, etc.
3. Documentation provided by Atlassian and App developers can also be leveraged.

Cloud & SaaS Tools

The situation is even “worse” when the software tool is not directly managed by the Company (read: you, the end-user), but resides on the cloud and is controlled by the tool developer. This is a scenario that will become more and more frequent.

Medical Device Manufacturers have always been skeptical about these tools as they feel that they are not in full control of them. And this is true. But at the same time, the tool developer is the best operator to keep it running smoothly, in particular for matters related to cybersecurity.

Once a Company adopts a software tool, this remains unchanged for a long period of time, as the re-validation effort can be substantial. This does not bode well with security patches that need to be implemented immediately; 0-day vulnerabilities are extremely dangerous and there is no day that goes by without news about known tools being attacked and new vulnerabilities exploited.

Repeating the whole validation every week is not realistic, not even part of it at periodic intervals.

That’s where tool developers can help with their insights. Features like “integrity checkers”, “self-validators”, “status reports”, etc. can provide supporting evidence that the initial validation is still valid and the tool is still running as expected.

Example

Atlassian makes changes to Confluence Cloud every day. Unfortunately, the users are not informed about the different types of changes made, which means that a regular validation of your Confluence Cloud instance is the best way to see if Confluence is working as expected. Manual validation of Confluence Cloud takes around 2 man-weeks.

Automated Validation of Confluence

SoftComply has just released the Validation app for Confluence Cloud that runs integrity checks of the Confluence user’s instance once every 7 days. As a result of each run, you will have the validation protocol and validation results with documented evidence. Test Results describe each test that was run recording the documented evidence (the screenshots) highlighting the result (pass/fail) of the test.

To learn more about the automated validation app for Confluence, you are welcome to join our Webinar on Validation of Cloud tools that take place once a month. Please REGISTER HERE.

---

References

1. ISO 13485:2016 Clause 4.1.6
2. CFR – Code of Federal Regulations Title 21 (Part 820.75 – Process Validation)
3. CFR – Code of Federal Regulations Title 21 (Part 11 – Electronic Records; Electronic Signatures)
4. General Principles of Software Validation – Final Guidance
5. Computer Software Assurance for Production and QS Software guidance (September 2022 Draft)
6. Process Validation: General Principles and Practices
7. What is Software Tool Validation? – SoftComply
8. On draft FDA Guidance on Software Assurance – SoftComply