IVT Session 15 April 27, 2017
Common Sense Practices for Agile
SDLC Compliance –The Auditable
OQ Verification Package
Dr. Teri Stokes, Director, QA Compliance
Cytel, Inc. – USA, Europe, India
What is Common Sense?
• Working Definition:
Common sense is knowledge based on “native good judgment” and personal experience. It is that combination of “book” smart, “street” smart, and “work” smart that a person can apply to scoping computer validation projects.
Slide 2
FDA Inspection View of GXP System
Operating environment (Clinical Research -GCP)
Computerized system (CDM System)
People
Software
Hardware
Computer system
SOPs
People
Equipment
Work Process
Infrastructure Platform Software application system - CDMSSlide 3
Defining Terms
Is SW Product GXP or Not?
Risk Analysis for Safety, Efficacy, and Quality (SEQ).
– SEQ Form to record a specific system risk assessment
– Impact of SW failure on People, Product, or Data in clinical studies or other regulated work activities (GCP/GLP/GMP = GXP)
– Priority #1 Safety Impact, #2 Efficacy Impact, #3 Other Quality
– If GXP yes = OQ/IQ/PQ If GXP no = general business practice
Slide 4
OQ Verification Package
Slide 5
GAMP 4 – The “V” Model
Functional
Specification
Design
Specification
System
Build
Installation
Qualification
IQ
Operational
Qualification
OQ
Performance
Qualification
PQ
User
Requirements
Specification
Specification & Qualification Basic Framework
Slide 6
Verifies
Verifies
Verifies
Ref. GAMP-4 Figure 6.2
Installation
Qualification
IQ&
SW Project Definition
SDLC SOP.DEV.001 The Agile Software Development Process for Custom Software
Slide 7
Product
Ideas PRD FRS
SRS OQ Trace
Matrix
SW Architect.
Document
SW Design
Description
SDLC SOPs,
Name/Code Stds
Developer Handbook
(Product Rqts. Doc.)
SAD SDD
(Functional Rqts.)
(Software Rqts.)
Developer Coding of SW
SDLC SOP.DEV.001 SDLC - Sand Box Sprints
CodeMaster OQ
Verification
Plan
Code Review
Form
SW Architect.
Description
SW Design
Description
Name & Code
Conventions
Developer Handbook
Unit Test
Smoke Test
Code ReviewSOP.DEV.003
Unit, Smoke &
Code Review
FormMaster OQ
Test Plan
SOP.DEV.002 SPRINT-N SAND BOX
Release
Testing
Sprint Plan
Sprint Report
SAD SDD
Issues are
tracked and
recycled until
resolved
OQ Testing of Release Candidate
QC Server OQ Testing
Code
Code Review
Form/Report
Unit Test
Smoke Test
Code ReviewSOP.DEV.003
Unit, Smoke &
Code Review
Form/ReportMaster OQ
Test Plan
SOP.DEV.002 SPRINT SAND BOX
OQ Release
Testing
OQ Test
Cases
OQ Test
Scripts
Test log,
Printout, P/F Report
OQ Test
Summary
Report
OQ
Trace
Matrix
QC Test
Server for
OQ Release
Release Candidate
Master OQ
Verification
Plan
OQ Verif.
Summary
Report
Sprint Plan
Sprint Report
SDLC Control Docs for SW Quality
• SDLC SOP.DEV.001 SDLC– Custom Software Project
Slide 10
Ideas MRS PRD Product
Rqts. Doc
FRS - SRS SW Rqts.
Spec.
Master OQ
Verification
Plan
Master
OQ Test
Plan
OQ Test
Sum. Rpt.
QC Server
IQ Report
Customer IT performs IQ to PRS
End Users perform PQ to URS
Customer
Audits
Authority
Inspections
OQ-VSR
Verification
Summary
Report
Developer Handbook
N Sprint Reports
Trace Matrix
User Acceptance at Go-Live
2.SYSTEM PLAN
What? URS
3.DESIGN
How? FRS &
SDD
4.BUILD
Program or
Configure
5.TEST
Verify to SDD
& Release
6.COMMISSION
Accept &
Validate to URS
9.RETIRE
Decommission
& Replace
Test Fit
to Design
Test Fit to
Work Process
8.MAINTAIN
Fix & Modify
7.OPERATE
Use&Monitor
1.SYSTEM IDEA
Needs Analysis,
RFP & Contract
Application
Life Cycle
SDLC
OQ – Operational
Qualification
IQ – Installation
Qualification
PQ – Performance
Qualification
Platform System
1.
Configure
2.
Install
3.
Test
Test Fit to Install Specs.
PQ
OQ
IQ
System User
Software
Supplier
SDLC - SW
Development
Life Cycle
Slide 11
Ongoing Testing &
Change Control
2.SYSTEM PLAN
What? URS
3.DESIGN
How? SDD
4.BUILD
Program or
Configure
5.TEST
Verify to SDD
& Release
6.COMMISSION
Accept &
Validate to URS
9.RETIRE
Decommission
& Replace
Test Fit
to Design
Test Fit to
Work Process
8.MAINTAIN
Fix & Modify
7.OPERATE
Use & Monitor
1.SYSTEM IDEA
Needs Analysis,
RFP & Contract
Application
Life Cycle
SDLC
Platform System
1.
Change
2.
Install
3.
Test
Test Fit to Design & Work
SDLC - Software
Development
Life Cycle
Software
Supplier
System User
Change
Symbol
Slide 12
Life Cycle Qualification at
New Version or Retirement
2.SYSTEM PLAN
What? URS
3.DESIGN
How? SDD
4.BUILD
Program or
Configure
5.TEST
Verify to SDD
& Release
6.COMMISSION
Accept &
Validate to URS
9.RETIRE
Decommission
& Replace
Test Fit
to Design
8.MAINTAIN
Fix & Modify
7.OPERATE
Use & Monitor
1.SYSTEM IDEA
Needs Analysis,
RFP & Contract
Application
Life Cycle
SDLC
RQ –
Retirement
Qualification &
Archive
Management
Platform System
1.
Back Up
2.
Archive
3.
Test
Test Media for RetrievalOQ
System User
Software
Supplier
SDLC - Software
Development
Life Cycle
Slide 13
Replacement
System
Replacement Plan
Data Migration Plan
Archive Plan
CSV Package Model
Control of Software &
Platform Environment
CSV Package Plan Verification/Validation Plan
Control of System & SW
Changes & Audit of
CSV Package, QMS
CSV Package Summary Report Verification/Validation Summary Report
Standard CSV
package
SOPs, Training &
Materials for Human
Work with System
Prepared and maintained by a CSV Package Team for IQ, OQ, or PQ
Requirements &
Spec.s for SW, System
& Services - SLA
Slide 14
1. System Control 2. Human Control 3. Testing Control
Test PlansStartup & Ongoing
Test Cases, Scripts,
Data & Result Logs
Test Summary
Reports
QMS = Quality Management System SLA = Service Level Agreement
Trace
IEEE Format for a Software
V&V Plan
IEEE Std. 1012-1986
1. Purpose
State plan identifier. Describe system. State assumptions and scope for this plan. What’s included, excluded, and limited?
2. Referenced Documents
Identify key documents associated with plan activities.
3. Definitions
Define terms and acronyms needed to understand this Plan.
4. Verification & Validation (V&V) Overview
Team structure, master task schedule, assigned responsibilities, tools, techniques, and methodologies to be used for tasks.
Slide 15
IEEE Format for a Software
V&V Plan
IEEE Std. 1012-1986
5. Life-Cycle Verification & Validation (V&V)
Management of V&V, concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase
6. Software Verification & Validation Reporting
Required reports, optional reports
7. Verification & Validation Administrative Procedures
Anomaly reporting and resolution, task iteration policy, deviation policy, control procedures, standards, practices, and conventions
Slide 16
XQ Validation Plan (VP)
Task Schedule
Slide 17
VP Task for OQ or IQ
or PQ (= XQ)
Role Responsible Status –Date
Write XQ Ver/Val Plan Team Leader with CSV
Team
Week 0
Review & Approve V.Plan System Sponsor & QA Week 1
Write XQ XRS* – Tables of
Work & System Tasks
CSV Team Week 0-1
Write XQ Test Plan for
Startup
Test Coordinator Week 1-2
*XRS = System Requirements = Functional (FRS) for OQ, Platform
(PRS) for IQ or User (URS) for PQ
1. Package Summary Report identifier - Unique ID traceable to associated Validation Plan.
2. Summary of all Life Cycle Tasks - Identifies CSV Package Team. Includes Task List from Validation Plan with updated status and roles responsible. References Audit Reports and/or Test or Package Summary Reports to cover Platform and Application Provider phases of Life Cycle.
3. Summary of Anomalies and Resolutions - Describes any deviations from Validation Plan and reasons why. Highlights any anomalies pending from the Test Summary Report.
CSV Package Summary Report
IEEE Std. 1012-1992 Adapted
Slide 18
4. Assessment of overall software quality - Discusses assumptions and limits to scope of CSV Package. Gives highlights from Test Summary Report. Describes how Package items are sufficient to assess system quality and user readiness for Go-Live in a regulated environment.
5. Recommendations - Recommends a Go-Live release status to the System Sponsor as Full Pass, Pass with Specified Conditions, or System Fail.
6. Approvals - Names, titles, signatures, dates and meaning of signatures.
7. Appendix - Table of Contents listing for Package items.
IEEE Std. 1012-1992 Adapted
Slide 19
CSV Package Summary Report
CSV Package Team Model –
Any Package - OQ/IQ/PQ
Package Sponsor
Test Coordinator
Package Manager
(Pkg. QC)
CSV Package Team
Quality
AssuranceTeam Leader
Ad hoc
Members
Trains Team and
Audits CSV Pkg.
Funds and Approves
Validation Work
Team to Develop and
Maintain a CSV Pkg.
Slide 20
Software Supplier’s (SDLC) OQ Package
Code & Tools Mgt. Logs,
SDLC, SAD, Controlled SW
Devel. Platform, Tools IQ
OQ Verification Plan
SW Upgrade Plan, Bug
Tracking, Audit Logs,
QMS with SQC & QA
OQ Verification Summary Report
Standard SW supplier’s
CSV package
SW Engr. SOPs, Pgmr.
Trng, Product Manuals,
User Help Desk
Prepared by internal or external software supplier team
MRS, FRS, SDD, Code,
SDLC Docs., Support
Contract (SLA), (Escrow)
Slide 21
1. System Control 2. Human Control
Software Supplier’s (SDLC) OQ Package
Code & Tools Mgt. Logs,
SDLC, SAD, Controlled SW
Devel. Platform, Tools IQ
OQ Verification Plan
Test Cases, Scripts,
Data & Result Logs
Master Test Plan &
Sub Test Plans –
U,F,S,M,S,I,R*
SW Upgrade Plan, Bug
Tracking, Audit Logs,
QMS with SQC & QA
OQ Verification Summary Report
Standard SW supplier’s
CSV package
SW Engr. SOPs, Pgmr.
Trng, Product Manuals,
User Help Desk
Prepared by internal or external software supplier team
MRS, FRS, SDD, Code,
SDLC Docs., Support
Contract (SLA), (Escrow)
Slide 22
Test Summary
Report (per Plan)
*U,F,S,M,S,I,R = Unit, Function, Smoke, Module, System, Interface, Regression
1. System Control 2. Human Control 3. Testing Control
Trace
Platform System IQ Package
Data Ctr. Config. Mgt.
Logs, Ops. SOPs, System
Manuals & Security WI
IQ Validation Plan
Test Cases, Scripts,
Data & Result Logs
IQ Test Plan(s)
Startup & Ongoing
Dis. Rec. Plan, Change
Control Log, Backup &
Archive Logs, Supplier
Records, IT Audit Log
IQ Test Summary
Report(s)
IQ Validation Summary Report
Standard platform
CSV package
IT/IS CVs, Training
Records, Dept. SOPs,
WIs, Check Lists
Prepared and maintained by IS/IT Department Team
SLA, SLA Monthly
Reports, Security
Logs, Help Desk Log
Platform URS
Slide 23
3. Testing Control 1. System Control 2. Human Control
Trace
Application User’s PQ Package
Applic. Admin. SOP &
Applic. Config. Mgt. Logs
PQ Validation Plan
Test Cases, Scripts,
Data & Result Logs
PQ Test PlansStartup & Ongoing
Change Control Log,
QA Audit Log, Supplier
Reports & BDG* Minutes
PQ Validation Summary Report
Standard user’s
CSV package
User Manuals, CVs,
Trng Records, Work
SOPs, WIs, Help Logs
Prepared and maintained by User Department(s) Team
Needs Analysis, RFP,
Contract, URS, User
Access Profiles, SLAs
Slide 24
PQ Test Summary
Reports
* BDG = Business Decision Group
1. System Control 2. Human Control 3. Testing Control
Trace
CSV Package XQ Definitions
• Operational Qualification (OQ) – Documented evidence
that a system operates as intended throughout its expected
range as per design/functional specifications. (Supplier)
• Installation Qualification (IQ) – Documented evidence that
all system components are installed to supplier instructions &
user requirements. (Infrastructure/IT)
• Performance Qualification (PQ) – Documented evidence
that a system operates as intended in the user’s work process.
(User – Start-up & Ongoing)
Slide 25
CSV Package Contents – GXP Systems
Slide 26
Agile SW OQ Package SW Application PQ Package
1. OQ Verification Plan (OQ VeP) 1. PQ Validation Plan (PQ VP)
2. Software Requirements Spec. (SRS) 2. User Requirements Spec. (URS)
3. OQ Master Test Plan (OQ MTP) 3. PQ Test Plan (PQ TP)
4. OQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle
5. OQ Trace Matrix (Scripts to SRS) 5. PQ Trace Matrix (Scripts to URS)
6. OQ Test Summary Report (OQ TSR) 6. PQ Test Summary Report (PQ TSR)
7. SOP/WIs (SDLC Sprint Tasks) 7. SOP/WIs (User Work Process Tasks)
8. Disaster Recovery Plan & DRP Test 8. Business Continuity Plan & BCP Test
9. QA Audit Certificate of SDLC Site 9. QA Audit Certificate of SW Vendor
10. OQ Change Management Plan 10. PQ Change Management Plan
11. OQ Verification Summary Report 11. PQ Validation Summary Report
12. QA Audit Certificate of OQ Pkg. 12. QA Audit Certificate of PQ Package
OQ Package
&
PQ Package
CSV* Regulatory Themes Matrix
Management Control
Auditable Quality
System Reliability
Data Integrity & Privacy
Slide 27 *CSV = Computerized System Validation
1. Verification/Validation Plan
2. Requirements Specifications
7. SOP/WIs
9. Vendor QA Audit
11. Validation Summary Report
12. QA Audit of CSV Package
3. Test Plan
4. Test Cases & Scripts
5. Trace Matrix
6. Test Summary Report
8. DRP/BCP – Test Report
10. Change Management Plan
CSV Package Contents – IT Platform IQ
Platform System IQ Package Document ID
1. IQ Validation Plan CTX.IQ.VP.v.1.0
2. Platform Requirements Spec. (PRS) CTX.IQ.PRS.MTX.v.2.1
3. IQ Test Plan CTX.IQ.TP.v.1.0
4. IQ Test Cases & Scripts & Fix Cycle 9 Test Cases & 24 Test Scripts
5. IQ Trace Matrix (Scripts to PRS) CTX.IQ.PRS.MTX.v.2.1
6. IQ Test Summary Report SOP/WIs CTX.IQ.TSR.v.1.0
7. SOP/WIs (Data Center Tasks) SOP IT_C002, CO23, C024 & C026
8. Disaster Recovery Plan & Test IT_C001, TBD in Standby Configuration
9. QA Audit Certificate of IT Facility TBD in Production Configuration
10. IQ Change Management Plan CTX.IQ.TC09 & SOP IT_C012
11. IQ Validation Summary Report CTX.IQ.VSR.v.1.0
12. QA Audit Certificate of IQ Package PQ Team Walk-through Report
IQ Package
Slide 28
Validation Focus for CSV* Packages
• Operational Qualification (OQ): Provision of documented evidence by system supplier that the software reliably operates as expected to its functional requirements and design specifications.
• Installation Qualification (IQ): Provision of documented evidence by IT that the network and infrastructure systems reliably operate to platform requirements for user’s application.
• Performance Qualification (PQ): Provision of documented evidence by end users that a software application reliably operates as intended to the user requirements of the regulated work process.
Slide 29 * CSV = Computerized System Validation
Computer Validation Training
Thank You!
Tak – Tack – Takk
Mahalo - Shukria
Bangalore – 2012
Any Questions or Comments?
??
OQ Group Exercise #1:
Inspection Themes for the 11’s –
FDA Part 11 & EMA Annex 11
Slide 32
Assignment: Prepare a regulation-based quick review chart for assessing
the status of your GXP systems.
1. Divide into paired teams for Annex 11 and 21 CFR Part 11.
2. Use the respective regulation to identify which CSV package item(s)
satisfy each stated document requirement.
3. Use the regulation’s paragraph numbering to plot the relationship on the
table of CSV Package Items.
(Example: Validation Plan = Annex 11 Principle & Sections 4.1-4.8)
(Example: Validation Plan = Part 11 Section 11.10 (a))
4. Are there any regulations not fulfilled by the package?
5. If so, how would you address the issue?
SDLC OQ/PQ Group Exercise:
Validation Inspection Themes for the 11’sAnnex 11 SDLC Software OQ Package SW Application User’s PQ Package 21 Part 11
Principle; 4.1-4.8 1. OQ Verification Plan (OQ VeP) 1. PQ Validation Plan (PQ VP) 11.10 a
2. Software Requirements Spec. (SRS) 2. User Requirements Spec. (URS)
3. OQ Master Test Plan (OQ MTP) 3. PQ Test Plan (PQ TP)
4. OQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle
5. OQ Trace Matrix (Scripts to SRS) 5. PQ Trace Matrix (Scripts to URS)
6. OQ Test Summary Report (OQ TSR) 6. PQ Test Summary Report (PQ TSR)
7. SOP/WIs (SDLC Sprint Tasks) 7. SOP/WIs (User Work Process Tasks)
8. Disaster Recovery Plan & DRP Test 8. Business Continuity Plan & BCP Test
9. QA Audit Certificate of SDLC Site 9. QA Audit Certificate of SW Vendor
10. OQ Change Management Plan 10. PQ Change Management Plan
11. OQ Verification Summary Report 11. PQ Validation Summary Report
12. QA Audit Certificate of OQ Pkg. 12. QA Audit Certificate of PQ Package
Name: _____________________ Date: ________
End Users’ IQ/PQ Group Exercise:
Validation Inspection Themes for the 11’sAnnex 11 Platform System IQ Package Software Application PQ Package 21 Part 11
Principle; 4.1-4.8 1. IQ Validation Plan 1. PQ Validation Plan 11.10 a
2. Platform Requirements Spec. (PRS) 2. User Requirements Spec. (URS)
3. IQ Test Plan (IT Infrastructure PRS) 3. PQ Test Plan (GXP Work URS)
4. IQ Test Cases, Scripts & Fix Cycle 4. PQ Test Cases, Scripts & Fix Cycle
5. IQ Trace Matrix (Scripts to PRS) 5. PQ Trace Matrix (Scripts to URS)
6. IQ Test Summary Report 6. PQ Test Summary Report
7. SOP/WIs (Data Center Tasks) 7. SOP/WIs (User Work Process Tasks)
8. Disaster Recovery Plan & DRP Test 8. Business Continuity Plan & BCP Test
9. QA Audit Certificate of IT Facility 9. QA Audit Certificate of SW Vendor
10. IQ Change Management Plan 10. PQ Change Management Plan
11. IQ Validation Summary Report 11. PQ Validation Summary Report
12. QA Audit Certificate of IQ Package 12. QA Audit Certificate of PQ Package
Name: _____________________ Date: ________
OQ Group Exercise #2 :
CSV Policy: GXP SEQ Risk Assessments
Slide 35
Assignment: Prepare an SEQ Risk Assessment for a known SW Product.
1. Divide into teams for performing GXP SEQ Risk Assessments.
2. Use the SEQ Attachments 1 and 2 to identify which Software products
are GXP applications that need OQ. What is their validation priority?
3. Pick a SW Product used in your company to analyze.
4. Are there any software products that are not GXP?
5. If so, how would you address SDLC quality for them?