SaaS System Validation, practical tips on getting validated for go-live and taking updates. Lennert Jansen, Serotonine
2 likes753 views
Validation is required for computerized systems used in regulated activities according to regulations like EMA Annex 11 and FDA 21 CFR Part 11. While traditional validation is project-based, agile development uses iterative sprints requiring a different validation approach. When using SaaS, the regulated company performs validation activities while leveraging documentation from the SaaS provider. Both parties should work together throughout product setup and testing to facilitate validation.
![Validation: Why Validate?
EMA’s Annex 11
‘Computerised Systems’
Principle
This annex applies to all forms of
computerised systems used as part
of a GMP regulated activities. […].
The application should be validated;
IT infrastructure should be qualified.
FDA’s 21 CFR Part 11
Subpart B—Electronic Records
§ 11.10 Controls for closed
systems.
Persons who use closed systems […]
shall employ procedures and
controls […]. Such procedures and
controls shall include the following:
(a) Validation of systems to ensure
accuracy, reliability, consistent
intended performance, and the
ability to discern invalid or altered
records.](https://image.slidesharecdn.com/lennertjansen-171009110637/85/SaaS-System-Validation-practical-tips-on-getting-validated-for-go-live-and-taking-updates-Lennert-Jansen-Serotonine-4-320.jpg)
![Validation: Why Validate?
Business Impact
(extract from Gamp 5 guideline, section 1.5 ‘Business Benefits):
[…] Specific benefits to both regulated companies and suppliers include:
• early defect identification and resolution leading to reduced impact on cost and schedule
• cost effective operation and maintenance
• effective change management and continuous improvement
• providing frameworks for user/supplier co-operation
• assisting suppliers to produce required documentation
• promotion of common system life cycle, language, and terminology
• promoting pragmatic interpretation of regulations
• […]
GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008](https://image.slidesharecdn.com/lennertjansen-171009110637/85/SaaS-System-Validation-practical-tips-on-getting-validated-for-go-live-and-taking-updates-Lennert-Jansen-Serotonine-5-320.jpg)
![Regulated Company versus SaaS Provider
Extract from ‘Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management ‘, ISPE, 2016
1 Introduction
In the evolving regulated IT environment there are many things to consider when thinking of turning to the
cloud for a solution. […] While it is not universally true, SaaS providers delivering specialized support to
regulatory business processes (e.g., Clinical Trails, Release Testing, AE reporting) tend to have a good
understanding of the needs of regulated companies.
Using a SaaS provider can be an excellent option for regulated companies, but doing appropriate research and
identifying the company’s specific support needs are critical to making the right choice of SaaS provider. […]](https://image.slidesharecdn.com/lennertjansen-171009110637/85/SaaS-System-Validation-practical-tips-on-getting-validated-for-go-live-and-taking-updates-Lennert-Jansen-Serotonine-15-320.jpg)
SaaS System Validation, practical tips on getting validated for go-live and taking updates. Lennert Jansen, Serotonine
- 1. practical tips on getting validated for go-live and taking updates SaaS System Validation
- 2. Introduction • Computerised System Validation • eSource system change control • eSource system selection • On-site audit
- 3. Structure • Validation • Agile • SaaS • Combine!
- 4. Validation: Why Validate? EMA’s Annex 11 ‘Computerised Systems’ Principle This annex applies to all forms of computerised systems used as part of a GMP regulated activities. […]. The application should be validated; IT infrastructure should be qualified. FDA’s 21 CFR Part 11 Subpart B—Electronic Records § 11.10 Controls for closed systems. Persons who use closed systems […] shall employ procedures and controls […]. Such procedures and controls shall include the following: (a) Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.
- 5. Validation: Why Validate? Business Impact (extract from Gamp 5 guideline, section 1.5 ‘Business Benefits): […] Specific benefits to both regulated companies and suppliers include: • early defect identification and resolution leading to reduced impact on cost and schedule • cost effective operation and maintenance • effective change management and continuous improvement • providing frameworks for user/supplier co-operation • assisting suppliers to produce required documentation • promotion of common system life cycle, language, and terminology • promoting pragmatic interpretation of regulations • […] GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008
- 6. Validation: What is System Validation ‘Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management ‘, ISPE, 2016
- 7. Validation: What is System Validation Traditional validation procedures describe • Project Phase or Initial system validation • An Assessment (Regulatory Applicability, System Risk, …) • Requirements & Specifications • Plan • Protocols which demonstrate ’fitness for intended use’ • Matrix, matching requirements to tests • Report • System Operation phase: • Change control • Periodic Review • Retirement phase: • System retirement
- 8. Agile System Software Development Product Backlog Sprint Backlog Sprint Daily Standup 24h 2-3 weeks Potentially shippable product Iteration Potential issue in the regulated industry
- 9. Agile System Software Development Fast Train: System Development Slow Train: System Validation, Training, .. Validation of version 1.0 V2.0 Sprint 1 V2.0 Sprint 2 V2.0 Sprint 3 Validation of version 2.0 V3.0 Sprint 1 V3.0 Sprint 2 V3.0 Sprint 3 Version 1.0 in production!
- 10. Agile Testing Quadrants • Testing is key in agile development • ‘Potentially shippable product’ • Product should be functionally tried and tested when sprint is finished Agile Testing = OQ?
- 11. SaaS – Software As A Service Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’ What is SaaS and why is it confusing? “Some of the following factors are still confusing the discussions in the industry: the term “cloud” is used as though it is one homogeneous thing, without consideration that SaaS, PaaS, and IaaS are very different, for instance, and that deployment models vary. Also, some things described informally as cloud are not really cloud, if you recognize generally accepted definitions of the essential characteristics of cloud, and are really just flavours of outsourcing.”
- 12. SaaS – Software As A Service Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’ Example: a consultancy firm resells software and hosts it on their own servers in a collocation data center (Infrastructure as a service, IAAS). An auditor enters the room, asks for the coverage on backup & recovery for their service, and the consultancy firm presents the ISO27001 or SOC certificate from the datacentre they’ve used.
- 13. SaaS – Software As A Service Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’ This certificate might be useful to mitigate risks related to: • Physical security to the datacentre • Business continuity aspects like internet lines This certificate is completely irrelevant for ISO27001 aspects like: • Backup and recovery: the datacentre doesn’t care if you backup your data • Disaster recovery aspects: if that datacentre goes down; your servers too. It’s your duty to get a contract with another data center & ensure sufficient failover mechanisms • Business continuity aspects like server failure: they’re not the datacenter’s servers • Validation aspects, quality of software & services, SLA, …
- 14. Working with SaaS providers which create Software using an Agile methodology in a Regulated industry
- 15. Regulated Company versus SaaS Provider Extract from ‘Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management ‘, ISPE, 2016 1 Introduction In the evolving regulated IT environment there are many things to consider when thinking of turning to the cloud for a solution. […] While it is not universally true, SaaS providers delivering specialized support to regulatory business processes (e.g., Clinical Trails, Release Testing, AE reporting) tend to have a good understanding of the needs of regulated companies. Using a SaaS provider can be an excellent option for regulated companies, but doing appropriate research and identifying the company’s specific support needs are critical to making the right choice of SaaS provider. […]
- 16. Regulated Company versus SaaS Provider Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’ Approaches to assessment and management of technology service providers must be flexible, practical, and pragmatic, and insisting on physical audits of all providers, regardless of type of service or level of risk, is unrealistic. The three key elements of regulated company management of technology service providers are: • Appropriate risk assessments (taking into account the nature of the process, the data, and in the case of cloud- based solutions, the service model and deployment model) • Supplier/provider assessments of the primary provider and the proposed solution (including their management of sub-suppliers) • Agreements/Contracts/SLAs in place to establish the controls that are managed by the service provider It is also unrealistic to insist that any service providers perform traditional and cumbersome paper-based qualification activities, rather than encouraging them to apply effective IT good practices supported by appropriate and modern tools and technologies.
- 17. Work together with the supplier • Maximize supplier involvement, as encouraged by GAMP5: 2.1.5 Leveraging Supplier Involvement Regulated companies should seek to maximize supplier involvement throughout the system life cycle in order to leverage knowledge, experience, and documentation, subject to satisfactory supplier assessment. For example, the supplier may assist with requirements gathering, risk assessments, the creation of functional and other specifications, system configuration, testing, support, and maintenance. Planning should determine how best to use supplier documentation, including existing test documentation, to avoid wasted effort and duplication. Justification for the use of supplier documentation should be provided by the satisfactory outcome of supplier assessments, which may include supplier audits. Documentation should be assessed for suitability, accuracy, and completeness. There should be flexibility regarding acceptable format, structure, and documentation practices. GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008
- 18. Work together with the supplier GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008
- 19. User Involvement We differentiate CSV Awareness & CSV participation: • CSV Awareness is the client becoming aware of all procedures on validation, to be able to: • Stand and defend the computerized system during client audits or inspections • Assess & even perform change control & periodic review • CSV Participation is the client having an active role during the validation: • Help Key Users, or Subject Matter Experts to understand applicable procedures • Guide Key Users, or Subject Matter Experts to write scenario’s & execute test scripts “Up-to-date software with up-to-date users”
- 20. Validation should not be seen as a big black box at the end of each software implementation process, but should be structured in such a way that parts of the validation can be performed during product setup, configuration or even proof of concept phase. CSV participation
- 21. CSV participation Speed up validation performing activities throughout the onboarding process!
- 22. eSource is audit sensitive! Auditors often have experience with eCRFs but not with eSource: source of confusion eCRF system: • Basic validation on system level • Extensive testing on study level, eCRF solutions are often building boxes, tools to create forms, edit checks, … eSource system: • Extensive system level validation. E.g. Volunteer database requiring overall validation towards HIPAA, European directive on data protection, interfaces to operational devices and external labs • Study setup verification according to a well-defined SOP A lot to explain, and as such a high need for proper validation documentation
- 23. Practical Tips • IQ: Limit to Installation Verification • OQ: leverage supplier activities for validation GAMP5 Agile Modules Epics User Requirements User Stories Functional Specifications Acceptance Criteria • CQ: Limit to configuration verification • PQ / UAT: Use key users to execute Scenario’s for each module to check business processes
- 24. Conclusions • Define internal validation SOP’s! Cfr. GAMP5’s defined business benefits: ‘promote pragmatic interpretation of rules’ • Validation effort helps to streamline your business processes • Hire validation consultants with care!
- 25. Conclusions • Leverage documentation of your SaaS vendor (without expecting it to be in the same form as the regulated company)
- 26. Conclusions • Perform on-site vendor audits to assess SDLC processes
- 27. Check applicable regulations! • 1996: Health Insurance Portability and Accountability Act, HIPAA • FDA: • 2003: FDA’s 21 CFR Part 11 on Electronic Records / Electronic Signatures for US • 2007: FDA: Guidance for Industry: Computerized Systems Used in Clinical Investigations. • 2010: FDA: Guidance for Industry: Electronic Source Documentation in Clinical Investigations. • 2011: FDA: Guidance for Industry: Oversight of Clinical Investigations – A Risk – Based Approach to Monitoring. • ISPE: • 2008: GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008 • 2016: Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management • 2006: CDISC: Leveraging the CDISC Standards to Facilitate the use of Electronic Source Data within Clinical Trials à includes a list of requirements for eSource systems. • 2011: EMA’s Annex 11 on computerized systems for EU • 2016: ICH / GCP for executing clinical studies: ICH HARMONISED TRIPARTITE GUIDELINE GUIDELINE FOR GOOD CLINICAL PRACTICE E6(R1), step 4 • 2018: EU wide ‘Data Protection Directive’, streamlining processes for all eu countries