Software validation do's and dont's may 2013

© Copyright John Cachat

Software Validation:

The Do’s and Don’ts

John M. [email protected]


Housekeeping

Phones are muted

Use the question

block for questions Copy of presentation

available upon request

2


About John M. Cachat

• Driving Business Performance

– Helping companies align their business and technology

– Focus on people, process, and then the technology

– Subject matter expert on business process management

– On-going research into next generation of technology for enterprise

systems

• 28 years experience in enterprise systems

– USAF Research Project (1985)

– Founder of enterprise quality software company (1988)

– Chair of ASQ technical committee on computerizing quality (1992)

• Trusted advisor to global organizations, government agencies,

and professional groups

http://www.linkedin.com/in/johncachat

3


Today’s Discussion

• Discusses how to solve several software validation issues, including: – Requirements

– Defect Prevention

– Time and Effort

– Validation Coverage

– Software Life Cycle

– Plans & Procedures

– Software Validation After a Change

– Independence of Review

– Flexibility and Responsibility

4


What is Driving You?

• Business Excellence

– Faster Product Launch

– Lower Costs (CMMI view)

• Industry Requirements (i.e., FDA)

– Compliance

• Can you have both?

5


Software Requirements Specification

(SRS)

• A complete description of the behavior of a system to be developed.

• It includes a set of use cases that describe all the interactions the users will have with the software.

• Use cases are also known as functional requirements. In addition to use cases, the SRS also contains non-functional (or supplementary) requirements.

• Non-functional requirements are requirements which impose constraints on the design or implementation (such as performance engineering requirements, standards, or design constraints

6


Let’s Talk Business

• Do I have to validate all software?

– NO

• What software do I have to validate?

– Based on Risk

• To end users of your product

• To your company

7


Manufacturing vs. Software

• Same Concepts? Yes

– Prevention vs. Detection

– Cannot inspect quality into the result

– Total cost of quality (prevention, inspection, failure)

• Same Thing? Not really, software is

– Very easy to change, quickly

– One change can impact a lot

– Hard to inspect everything, literally everything

8


Build, Buy, Use – Impacts Everyone!

9

www.fda.gov/downloads/.../Guidances/ucm126955.pdf


Major Sections

• Section 1. Purpose

• Section 2. Scope

• Section 3. Context for Software Validation

• Section 4. Principles of Software Validation

• Section 5. Activities and Tasks

• Section 6. Validation Of Automated Process

Equipment And Quality System Software

10


Section 1. Purpose

• Provide general validation principles that the

FDA considers to be applicable to the

validation of medical device software or the

validation of software used to design,

develop, or manufacture medical devices.

11

Or pharmaceuticals, or cars, or airplanes,

or anything that if it does not work,

people may die, or get sick, or ……


Section 2. Scope

• The scope of this guidance is somewhat

broader than the scope of validation in the

strictest definition of that term.

• Planning, verification, testing, traceability,

configuration management, and many other

aspects of good software engineering

discussed in this guidance are important

activities that together help to support a final

conclusion that software is validated.

12


Section 2. Scope

• Based on the intended use and the safety risk

associated with the software to be

developed, the software developer should

determine the specific approach, the

combination of techniques to be used, and

the level of effort to be applied.

• Recommends an integration of software life

cycle management and risk management

activities.

13


Section 2. Scope

• Validate these:– Software used as a component of a medical device;

– Software that is itself a medical device

– Software used in the production of a device and

– Software used in implementation of the device manufacturer's quality system

• What about these?– Accounting

– Plant floor scheduling

– Microsoft desktop software

– Plant floor automation

14


Section 2. Scope

• THE LEAST BURDENSOME APPROACH

• Clarification

– Software validation process should not be

confused with any other validation requirements,

such as process or product validation

– Does not cover any specific safety or efficacy

issues with respect to product being

manufactured

15


Section 3.

Context for Software Validation

• 3.1. Definitions and Terminology

– 3.1.1 Requirements and Specifications

– 3.1.2 Verification and Validation

– 3.1.3 IQ/OQ/PQ

• 3.2. Software Development as Part of System Design

• 3.3. Software is Different from Hardware

• 3.4. Benefits of Software Validation

• 3.5 Design Review

16


3.1. Definitions and Terminology

17

http://www.fda.gov/iceci/inspections/inspectionguides/ucm074875.htm


3.1.1 Requirements and Specifications

• A requirement can be any need or expectation for a system or for its software.

• Requirements reflect the stated or implied needs of the customer, and may be market-based, contractual, or statutory, as well as an organization's internal requirements.

• There can be many different kinds of requirements (e.g., design, functional, implementation, interface, performance, or physical requirements).

• A specification is defined as “a document that states requirements.”

18


3.1.2 Verification and Validation

• Treats “verification” and “validation” as separate and distinct terms

• Software verification provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase.

• Software validation to be “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”

19


3.1.2 Verification and Validation

• Software verification - a developer cannot test forever & testing does not guarantee no defects

• Software validation - it is hard to know how much evidence is enough.

• Software validation is a matter of developing a “level of confidence”

• How much “confidence?” How much risk?

20


3.1.3 IQ/OQ/PQ

• Installation qualification (IQ) - the system has been built and installed correctly and that all required supporting services are available and connected correctly.

• Operational qualification (OQ) - demonstrate that the newly acquired software functions as expected; that all parts and components operate correctly

• Performance qualification (PQ) - demonstrate compliance with all requirements given in the User Requirements Specification document

21


3.2. Software Development as Part of System Design

• Software validation must be considered within the context of the overall design validation for the system

• End user rarely cares about the software

• The user's needs and intended uses from which the product is developed

– Correct blood pressure reading

– Anti-lock brakes work

– Airplane controls work

22


3.3. Software is Different from Hardware

• Seemingly insignificant changes in software code can create unexpected and very significant problems elsewhere in the software program The vast majority of software problems are traceable to errors made during the design and development process.

• One of the most significant features of software is branching, i.e., the ability to execute alternative series of commands, based on differing inputs.

• Software also has the speed and ease with which it can be changed concern regarding change management

• Software failures often occur without advanced warning (no noise, vibration, etc)

23


3.3. Software is Different from Hardware

• Because of its complexity,

the software development

process should be

controlled more tightly

than hardware.

24


3.4. Benefits of Software Validation

• This is the business part

– decreased failure rates

– fewer recalls and corrective actions

– less risk to patients and users, and

– reduced liability to manufacturers

25

This Car Runs on Code

It takes dozens of microprocessors running

100 million lines of code to get a premium

car out of the driveway.

As Much Software Code as an Airbus A380


3.5 Design Review

• Design reviews are documented,

comprehensive, and systematic

examinations of a design to evaluate the

adequacy of the design requirements,

to evaluate the capability of the design

to meet these requirements, and to

identify problems.

• FMEA - Failure mode effect analysis

• SET – Success every time

26


FMEA

27


Section 4.

Principles of Software Validation 4.1. Requirements

4.2. Defect Prevention

4.3. Time and Effort

4.4. Software Life Cycle

4.5. Plans

4.6. Procedures

4.7. Software Validation after a Change

4.8. Validation Coverage

4.9. Independence of Review

4.10. Flexibility and Responsibility

28


Section 4.

Software Validation Summary

• Software testing is a necessary activity.

• In most cases software testing by itself is not sufficient to establish confidence that the software is fit for its intended use

• Validation coverage should be based on the software's complexity and safety risk - not on firm size or resource constraints.

• The final conclusion that the software is validated should be based on evidence collected from planned efforts conducted throughout the software lifecycle

• Whenever software is changed, a validation analysis should be conducted not just for validation of the individual change, but also to determine the extent and impact of that change on the entire software system.

29


Section 5. Activities and Tasks

5.1. Software Life Cycle Activities

5.2. Typical Tasks Supporting Validation

5.2.1. Quality Planning

5.2.2. Requirements

5.2.3. Design

5.2.4. Construction or Coding

5.2.5. Testing by the Software Developer

5.2.6. User Site Testing

5.2.7. Maintenance and Software Changes

30


Software Life Cycle

• Quality Planning

• System Requirements Definition

• Detailed Software Requirements Specification

• Software Design Specification

• Construction or Coding

• Testing

• Installation

• Operation and Support

• Maintenance - Change Control, Revision Control

• Retirement

31


Quality Planning Tasks

• Risk Management Plan

• Configuration Management Plan

• Software Quality Assurance Plan (Software Verification & Validation Plan)

• Verification and Validation Tasks, and Acceptance Criteria

• Schedule and Resource Allocation

• Reporting Requirements

– Formal Design Review Requirements

– Other Technical Review Requirements

• Problem Reporting and Resolution Procedures

• Other Support Activities

32


5.2.2 Requirements• All software system inputs;

• All software system outputs;

• All functions that the software system will perform;

• All performance requirements that the software will meet, (e.g., data throughput, reliability, and timing);

• Definition of all external and user interfaces, as well as any internal software-to-system interfaces;

• How users will interact with the system;

• What constitutes an error and how errors should be handled;

• Required response times;

• The intended operating environment for the software, if this is a design constraint

• All ranges, limits, defaults, and specific values that the software will accept; and

• All safety related requirements, specifications, features, or functions that will be implemented in software.

33


Testing Coverage

• Statement Coverage

• Decision (Branch) Coverage

• Condition Coverage

• Multi-Condition Coverage

• Loop Coverage

• Path Coverage

• Data Flow Coverage

34


Section 6. Validation Of Automated Process Equipment

And

Quality System Software

• 6.1. How Much Validation Evidence Is Needed?

• 6.2. Defined User Requirements

• 6.3. Validation of Off-the-Shelf Software and

Automated Equipment

35


6.1. How Much Validation Evidence Is

Needed?• The level of validation effort should be

commensurate with the risk posed by the automated operation

• The extent of validation evidence needed for such software depends on the device manufacturer's documented intended use of that software

• COTS - consider auditing the vendor's design and development methodologies used in the construction of the software and assess the development and validation documentation generated for the COTS software

36


IEEE Technical Resources

• SRS - Software Requirements Specification IEEE 830

• SQAP - Software Quality Assurance Plan IEEE 730

• SCMP- Software Configuration Management Plan IEEE 828

• STD - Software Test Documentation IEEE 829

• SVVP - Software Validation & Verification Plan IEEE 1012

• SDD - Software Design Description IEEE 1016

• SPMP - Software Project Management Plan IEEE 1058

37


Software Best Practices

• Develop software iteratively

• Manage requirements

• Use component-based architectures

• Visually model software

• Verify and validate

• Software change control process

• Document, document, document

38


Summary

• You DO NOT have to

validate all software

• You validate based on risk

• Software is not the same as

hardware

• Plan for the entire software

life cycle

39


About Us

John Cachat

[email protected]

Contact

Proven expertise in business

information systems

Rapid Solution

Development™ process

ServicesAssess Current Status

Develop Short and Long

Term Plans

Develop Specific Solutions

to Your Problems

Assist in ROI Analysis

40


Software Validation: The Do’s and Don’ts

&

Contact:

John Cachat

[email protected]

Copy of Presentation

&

Request a Demo

Visit:

http://peproso.com/webinars

Future Webinars

41

Date post:	20-Jan-2015
Category:	Business
Upload:	john-cachat
View:	397 times
Download:	1 times