Quality Control for Medical Device Software - It Arena Lviv Presentation

CONFIDENTIAL©2014 GlobalLogic Inc.

Quality Control for

Medical Device Software

by Roman Lavriv,

Senior Project Manager

©2014 GlobalLogic Inc. CONFIDENTIAL

Introduction

Regulations

Requirements & V-model

Verification & Validation

Going Agile

01

02

03

04

05

3 CONFIDENTIAL

Introduction01

4 CONFIDENTIAL

−Over 11 years in IT outsourcing as Project Manager

−Managing software quality for a medical device for last 4 years

−Team of 48 software testers

−Class C medical device – severe injury or death possible

About MeIntroduction

5 CONFIDENTIAL

Regulations02

6 CONFIDENTIAL

Regulations

IEC 60601-1-4 – Medical electrical equipment

• Harmonized standard for medical electrical equipment recognized by public health authorities in most countries

IEC 60601

• Part 1 - General requirements for basic safety and essential performance

IEC 60601-1

• Part 1-4 – Programmable electrical medical systems

IEC 60601-1-4

7 CONFIDENTIAL

Regulations

Software Development Process

IEC 62304

• Medical Device Software - software lifecycle process

AAMI TIR (SW1)

• Guidance on agile practices in the Medical Devise Software

• (based on IEC 62304)

8 CONFIDENTIAL

Regulations

Title 21 of the Code of Federal Regulations21 CFR

Title 21 is the portion of the Code of Federal Regulations that governs food and drugs within the US for the Food and Drug Administration

(FDA), the Drug Enforcement Administration (DEA), and the Office of National Drug Control

Policy (ONDCP)

21 CFR Part 11

Electronic Records; Electronic Signatures

Validation

21 CFR Part 820

Quality System Regulation

9 CONFIDENTIAL

Regulations

GPSV

General Principles of Software Validation – Guidance for Industry and FDA staff

− This guidance outlines general validation principles that the Food and Drug Administration (FDA) considers to be applicable to the validation of medical device software or the validation of software used to design, develop, or manufacture medical devices.

10 CONFIDENTIAL

Regulations

FDA Premarket Approval (PMA)

PMA (21 CFR Part 814)

Premarket approval (PMA) is the U.S. Food and Drug Administration (FDA) process of scientific and regulatory review to evaluate the safety and effectiveness of Class III medical devices

11 CONFIDENTIAL

Requirements & V-model03

12 CONFIDENTIAL


Requirements & Design Hierarchy

User Needs & Use Cases

System Requirements•Can be broken down into individual subsystems

•Software is one of subsystems

Software Requirements• Identify risk mitigations•Update software and system requirements, as appropriate

Software Architectural Design• Identify SW items• Identify internal / external interfaces

• Identify SOUP (including requirements, HW/SW, & acceptance criteria)

Software Detailed Design• Identify & design SW units

•Design interfaces

13 CONFIDENTIAL


V-modelSystem

Requirements

Software Requirements

Software Architecture

Design

Software Units

Code

Unit Testing

Integration Testing

Functional Testing

System TestingIs Tested By

Is Tested By

Is Tested By

Is Tested By

14 CONFIDENTIAL


Software Risk Identification− Assign software system safety classification

− Analysis of software contributing to the hazardous situation

− Risk Control Traceability

o Hazardous situation to software itemo Software item to software causeo Software cause to risk controlo Risk control to verification

− Software changes Additional causes to hazardous situations? Additional risk mitigations? Interference with existing mitigations?

15 CONFIDENTIAL


Software Safety ClassificationRisk Management Severity Software System Safety Classification

Nuisance Class A (No injury or damage to health is possible)

Minor, Moderate Class B (Non serious injury is possible)

Major, Severe Class C (Death or serious injury is possible)

16 CONFIDENTIAL


Change Control− Record every change request (defect-fix or

enhancement)

− Software Change Review Group (SCRG) Approve change requests, features, design changes Approve work products (new and changed) to be

delivered Notify affected parties of approved changes Evaluate residual risks Determine acceptability of software for its intended

use Review defect trend metrics Disposition problem reports

o Problem resolvedo Adverse trends reversedo Changes implemented appropriatelyo Additional problems introduced are addressed

− Configuration items may only be changed in response to a change request approved by SCRG

− Execute applicable design control activities

− Audit trail of change Problem reports List of changes History of change requests

17 CONFIDENTIAL

Verification & Validation04

18 CONFIDENTIAL


Software Unit Verification(Unit Testing, Code

review and Static code analysis)

Software Architecture Specification

Software Requirements Specifications

Software Detail Design

System Requirements

Software Unit Integration

Software Functional Testing

Intended Use testing

System testing

Product Specifications

System Specifications

SOFTWARE MAINTENANCE PROCESS – Post Production Software Release

SO

FT

WA

RE

D

ES

IGN

VE

RIF

ICA

TIO

N

SOFTWARE CHANGE MANAGEMENT (CHANGE CONTROL, PROBLEM RESOLUTION AND CONFIGURATION MANAGEMENT)

SO

FT

WA

RE

RIS

K M

AN

AG

EM

EN

T

SO

FT

WA

RE

DE

SIG

N

VA

LID

AT

ION

Software Detailed Design

Software Architecture Design


Product Requirements

Verification Complete Software Release

Production Software Release

Software Development Planning

Static ValidationRequirements

Reviews

Static ValidationRequirements

Reviews

Static VerificationRequirements

Reviews

Static VerificationSW Architecture

Reviews

Static VerificationDetailed Design

Reviews

Dynamic ValidationTest Cases / Results

Dynamic ValidationTest Cases / Results

Dynamic VerificationTest Cases / Results

Dynamic VerificationTest Cases / Results

Dynamic VerificationTest Cases / ResultsSoftware Unit

Implementation

Controlled Code

Traceability for Test Coverage



Requirements Traceability

Software Development Plan (SDP)

Software Development Report (SDR)

Requirements Traceability

Verification Start Software Release

Software Development Report (SDR)

19 CONFIDENTIAL

Verification

Verification vs ValidationVerification & Validation

…provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase. Software verification looks for consistency, completeness, and correctness of the software and its supporting documentation, as it is being developed, and provides support for a subsequent conclusion that software is validated. Software testing is one of many verification activities intended to confirm that software development output meets its input requirements. Other verification activities include various static and dynamic analyses, code and document inspections, walkthroughs, and other techniques. (GPSV, 3.1.2)

Validation

…confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled. (GPSV)

20 CONFIDENTIAL


Static vs. Dynamic• Walkthroughs / Technical

Reviews / Inspections

Static(IEC 62304:2006, 5.2.6, 5.3.6, 5.4.4, 5.6.5, 5.7.4)

• Cases / Scripts / Procedures / Protocols / Specs

• Black Box / White Box / Grey Box• Functional / Structural• Regression• Performance / Load / Stress• Security• Integration / Unit• Smoke• Alpha / Beta

Dynamic(IEC 62304:2006, 5.5, 5.6, 5.7)

21 CONFIDENTIAL


Static Verification ElementsSoftware Requirements• Implement system requirements• Traceable to system requirements• No contradictions• Not ambiguous• Uniquely identified• Testable

Software Architectural Design

• Implements software requirements• Supports interfaces• Supports SOUP


• Implements software architecture• No contradictions

Unit Tests

• Correct

Code Review

• Meets Design inputs• Implements coding standards

Static code analysis

• Detects defects prior to code execution

Integration Tests

• Correct

Functional Tests

• Appropriate• Traces to software requirements• Tests or ensures verification of all

requirements• Pass / fail criteria

22 CONFIDENTIAL

Two phases:

Integration TestingVerification & Validation

1. Integration of software units into software items and the software system

2. Integration of hardware / software items; support for manual operations

Components

Verifies architecture Static verification of test cases Controlled code Formal problem resolution Mandated test case components Impact analysis and regression

testing required

23 CONFIDENTIAL


Functional Testing

Static verification of test cases Trace to SRS Prototype software Formal problem resolution Mandated test case components Impact analysis and regression testing required

24 CONFIDENTIAL


Software Design Validation

Intended use testing and software system testing Static verification of test cases Trace to system specification and product specification Production or production-equivalent software Formal problem resolution Mandated test case components Impact analysis and regression testing required

25 CONFIDENTIAL

Questions to ask yourself:

Impact AnalysisVerification & Validation

1. Did the change work?2. Did I break something that

worked before?3. Did I compromise any aspect of

risk management?a) Introduce new hazardsb) Break existing mitigations

Ways to accomplish:

Traceability Analysis Expert Review Architectural Review Code Inspection

26 CONFIDENTIAL


Zero Defects?•3.2 ANOMALYany condition that deviates from the expected based on requirements specifications, design documents, standards, etc. or from someone’s perceptions or experiences. ANOMALIES may be found during, but not limited to, the review, test, analysis, compilation, or use of SOFTWARE PRODUCTS or applicable documentation

•5.8.2 Document known residual ANOMALIESThe MANUFACTURER shall document all known residual ANOMALIES. [Class B, C]

•5.8.3 EVALUATE known residual ANOMALIESThe MANUFACTURER shall ensure that all known residual ANOMALIES have been EVALUATED to ensure that they do not contribute to an unacceptable RISK. [Class B, C]

IEC 62304:2006

•Unresolved Anomalies (Bugs or Defects)For Moderate and Major Level of Concern Software Devices, the submission should include a list of all unresolved software anomalies. For each anomaly, we recommend that you indicate the:•problem• impact on device performance•any plans or timeframes for correcting the problem (where appropriate).

•We recommend that you annotate each item with an explanation of the impact of the anomaly on device safety or effectiveness, including operator usage and human factors issues. Typically, this list can be generated as an output of a change control board or similar mechanism for evaluation and disposition of unresolved software anomalies. We recommend that you communicate this list to the end user as appropriate to assist in the proper operation of the device. In all instances where it is practical to do so, you should include any mitigations or possible workarounds for unresolved anomalies; this recommendation applies to Blood Establishment Computer Software in particular.

FDA’s “Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices”

27 CONFIDENTIAL

Going Agile05

28 CONFIDENTIAL

Going Agile

Regulatory Requirements“But government agencies requires a waterfall method, doesn’t it?”

From IEC 62304

This standard does not prescribe a specific life cycle model.

The users of this standard are responsible for selecting a life cycle model for the software project and for mapping the processes, activities, and tasks in this standard onto that model.

Annex B (informative)

Guidance on the provisions of this standard:

The purpose of this standard is to provide a development process that will consistently produce high quality, safe medical device software. To accomplish this, the standard identifies the minimum activities and tasks that need to be accomplished to provide confidence that the software has been developed in a manner that is likely to produce highly reliable and safe software products.

29 CONFIDENTIAL

Going Agile

V-model EvolutionSystem

Requirements


Software Architecture

Design

Software Units

Code

Unit Testing

Integration Testing

Functional Testing

System TestingIs Tested By

Is Tested By

Is Tested By

Is Tested By

30 CONFIDENTIAL

Going Agile

V-model Evolution

31 CONFIDENTIAL

Going Agile

Software Development Planning – Story Level


Static Verification: Requirements Reviews

Software Architecture Design

Static Verification: Architecture Reviews



Unit Implementation and Verification(Unit Testing, Code review and Static

code analysis)

Software Integration testing

Dynamic Verification – Test Cases / Results


Dynamic Verification – Test Cases / Results


Integration testing

Dynamic Verification Test Cases / Results


StoryStory

StoryStory

Software Development Planning – Increment/Sprint Level

Increment

Increment

Increment

Increment

System Level Testing

Intended Use Testing

Dynamic Validation Test Cases / Results

Dynamic Validation Test Cases / Results


Integration testing



Software Development Planning – Release Level

ReleaseRelease

Release

Software Development Planning – Project Level (Software Development Plan and Software Development Report)


Static Verification: Architecture Reviews


SW Architecture Design

Static Validation: Requirements Reviews

Static Validation: Requirements Reviews

System Requirements

Product Requirements

SOFTWARE MAINTENANCE PROCESS – Post Production Software Release

SOFTWARE CHANGE MANAGEMENT (CHANGE CONTROL, PROBLEM RESOLUTION AND CONFIGURATION MANAGEMENT)

SOFTWARE RISK MANAGEMENT

SO

FTW

AR

E T

RA

CE

AB

ILIT

Y

Sys

tem

Req

uire

men

ts to

Pro

duct

Req

uire

men

ts a

nd S

yste

m te

stin

g,

Pro

duct

Req

uire

men

ts to

Sof

twar

e R

equi

rem

ents

and

Inte

nded

Use

test

ing

Sof

twar

e R

equi

rem

ents

to S

oftw

are

Func

tiona

l Tes

ting


Questions?


Thank you!

Date post:	08-Dec-2014
Category:	Devices & Hardware
Upload:	roman-lavriv
View:	147 times
Download:	0 times

Quality Control for Medical Device Software - It Arena Lviv Presentation

Devices & Hardware