2004 11 17 Test Process Improvement

7/29/2019 2004 11 17 Test Process Improvement

1/18

Test processimprovement

The Models and the Practice

Steven Deneir

November 17th 2004

Test process improvementNovember 17th 2004 2

Index

Introduction

Models

Models

Models

Conclusion & Closing


2/18


Goal of this presentation

Provide an introduction to models that might help during

test process improvement


What is Test Process Improvement?

Continuous: everything changes around us, so should testing

Improvement: where are you now? where do you want to be?

Quality: product quality, process quality (insight, coverage, predictability,etc.)

Efficiency: cost, lead time, effort

Testing: Techniques, Management, Environments, Tools, etc.

Process: Ad-hoc testing, Structured testing, Exploratory testing, etc.

Related to: the development process, the risks of the project, therequirements, the project planning, etc

Continuous Improvement of the quality, and the efficiency

of the testing process, related to the output of the total

software process


3/18


Why Test Process Improvement?

Changes are all around us: Business of your organisation might be changing

Higher quality demands of customers

Time to market

Cost cutting

Development process changes: CMMI programs

Development delivers faster thanks to more efficient tools

Etc., etc., etc.

The test process must keep up with these changes!


SW-CMM Capability Maturity Model

In 2000, the SW-CMM was upgraded to CMMI

The SEI no longer maintains the SW-CMM model, its

associated appraisal methods, or training materials, nor

does the SEI offer SW-CMM training.


4/18


SW-CMM Structure

Maturity Levels

Key Process Areas

Common Features

Key Practices

Process

Capability

Goal

Implementation or

Institutionalization

Infrastructure or

Activities

indicate contain

achieveorganized by

addresscontain

describe

ML 1: Initial

ML2: Repeatable

ML 3: Defined

ML 4: Managed

ML 5: Optimizing


SW-CMM and Testing

SW-CMM Level 3

Key process area Software Product Engineering

Activities:

5: Software Testing is performed according to the projects definedsoftware process

6: Integration testing of the software is planned and performed

according to the projects defined software process

7: System and acceptance testing of the software are planned and

performed to demonstrate that the software satisfies its requirements

A dedicated KPA exists for Peer Reviews


5/18


SW-CMM Summary

Not really helping us with test process improvement

Upgraded to CMMI in 2000

But reminds that some other things should better be in

place


CMMI CMM Integrated

Based on experience of previous models (SE, SW, )

More flexible: 2 representations Staged representation (similar to SW-CMM)

Continuous representation

Common terminology for the different models that are

integrated

Consistent with ISO 15504 (SPICE)


6/18


CMMI Staged Representation

ML 1: Initial

ML2:Managed

ML 3: Defined

ML 4: Quantitatively

Managed

ML 5: Optimizing

Maturity Levels

Process Areas

Specific Goals Generic Goals

Specific Practices Generic Practices

Commitment

to Perform

Ability

To Perform

Directing

Implementation

Verifying

Implementation

Common Features


CMMI Continuous Representation

Process Areas

Specific Goals Generic Goals

Specific Practices Generic PracticesCapability Levels

CL 0

CL 1

CL 2

CL 3

CL 4

CL 5

Process Areas


7/18


CMMI and Testing

Still at maturity level 3 of the staged representation

But continuous representation allows addressing a

process area depending the need

2 dedicated process areas Verification: you built it right

Validation: you built the right thing

After I have the presentation I learned that there was a third process

area strongly related to testing: Product Integration which strongly

relates to integration testing.


CMMI VerificationYou Built it Right

SG 1 Prepare for Verification SP 1.1-1 Select Work Products for Verification

SP 1.2-2 Establish the Verification Environment

SP 1.3-3 Establish Verification Procedures and Criteria

SG 2 Perform Peer Reviews SP 2.1-1 Prepare for Peer Reviews

SP 2.2-1 Conduct Peer Reviews

SP 2.3-2 Analyze Peer Review Data

SG 3 Verify Selected Work Products SP 3.1-1 Perform Verification

SP 3.2-2 Analyze Verification Results and Identify Corrective Action


8/18


CMMI ValidationYou Built the Right Thing

SG 1 Prepare for Validation SP 1.1-1 Select Products for Validation

SP 1.2-2 Establish the Validation Environment

SP 1.3-3 Establish Validation Procedures and Criteria

SG 2 Validate Product or Product Components SP 2.1-1 Perform Validation

SP 2.2-1 Analyze Validation Results


CMMI Summary

More on testing compared to SW-CMM (2 new process

areas: verification & validation)

But still very general

Keeps showing us that maybe we need first to focus on

some other process areas


9/18


Test Maturity Models

Testability Maturity Model (TMM)

Test Organization Maturity (TOM) Testing Assessment Program (TAP)

Proposition KPA for SW-CMM

Test Process Improvement (TPI)

Software Testing Maturity Model (SW-TMM)


(SW-)TMM Testing Maturity Model

Existing models did not adequately address testing

needs

CMM model widely used basis for the structure

Developed by Illinois Institute of Technology

A lot of research in different types of companies

Gathered the best practices and characteristics of

mature and immature organizations


10/18


TMM Structure Levels

Testing

CapabilityMaturity Goals

Maturity Sub-goals

Activities

/ Tasks /

Responsibilities

Implementation

and Organizational

AdaptationCritical

Views

User /

ClientDeveloper

/ Tester

Manager

indicatecontain

supported by

achieved by

address

organized by


TMM Maturity

Levels

Level 1: Initial

Level 2: Definition

Test Policy and Goals

Test Planning

Test Techniques and Methods

Test Environment

Level 3: Integration

Test Organisation

Test Training Program

Test Life Cycle and Integration

Control and Monitor

Level 4: Management and Measurement

Peer Reviews

Test Measurement

Software Quality Evaluation

Level 5: Optimisation

Defect Prevention

Quality Control

Test Process Optimisation


11/18


TMM & CMM(I)

Process Change Management, Technology Change

Management, Defect Prevention55

Software Quality Management and Quantitative Process

Management44

Intergroup Coordination; Peer Reviews34

Organizational Process Focus, Organizational Process

Definition, Organizational Process Training33

Project Tracking and Oversight, Software Quality

Assurance23

Requirements Management, Project Planning, Software

Configuration Management22

Supporting KPACMMTMM


TMM - Summary

An organization wide approach

Strong management commitment required (link with SPI)

Well founded model

(Strong) link with CMM(I)

Provides support all the way


12/18


TPI Test Process Improvement

Based on experience of a lot of consultants (Sogeti)

Closely related to TMap

Is an assessment model


TPI Structure

Test

Maturity

Matrix

Key Areas

Levels

Improvement SuggestionsCheckpoints


13/18


TPI Structure

Test

Maturity

Matrix

CheckpointsTest Domains:

Key Areas

L: Lifecycle

T: Techniques

I: Infrastructure

O: Organisation

MaturityLevels


TPI Summary

Highly practical, highly detailed

Focus is more on higher level testing

Project focus, instead of organizational focus

Limited maturity level and scope (some org. need more)

No link with SPI (such as CMMI)

Strongly TMap related


14/18


IDEAL

SEI: Software Engineering Institute / Carnegie Mellon

University

Originally a life-cycle model for software process

improvement based on the CMM

An organizational improvement model

How to introduce CMM improvements into an

organization?


IDEAL Structure

Source: SEI


15/18


IDEAL Summary

Remembers very important steps during process

improvement

Helps in planning

Looks at the next improvement cycle


Assessment

TMM: has an accompanying assessment model (TMM-

AM)

TPI: is an assessment model

CMMI has SCAMPI: Standard CMMI Appraisal Method

for Process Improvement

SCAMPI is also ISO 15504 (Spice) compliant


16/18


SCAMPI What to retain

Practice Implementation Indicators (PII) Direct Artefacts

Indirect Artefacts

Affirmations

Combination of Objective Evidence (OE) necessary to

corroborate multiple sources Instruments

Presentations

Documents

Interviews


ADKAR Model

Intended as a coaching tool to help employees through a

change

Published by Prosci in 1999

Research in more than 700 companies


17/18


ADKAR Structure

Incentives and rewards

Compensation changes

Celebrations

Personal recognition

Reinforcement to sustain the

changeR

Practice applying new skills or using new processes and tools

Coaching

Mentoring

Removal of barriers

Ability to implement required

skills and behaviorsA

Training and education

Information access

Examples

Role models

Knowledge on how to changeK

Fear of job loss

Discontent with the current state

Imminent negative consequence

Enhanced job security

Affiliation and sense of belonging

Career advancement

Status or change in social standing

Acquisition of power or position

Incentives or compensation

Trust and respect for leadership

Hope in the future state

Desire to participate and

support the changeD

Management communications

Customer input

Marketplace changes

Ready-access to information

Awareness of the need for

changeA Management communications

Customer input

Marketplace changes

Ready-access to information Fear of job loss Discontent with the current state

Imminent negative consequence



Career advancement







Information access

Examples

Role models Practice applying new skills or

using new processes and tools

Coaching Mentoring

Removal of barriers



Celebrations Personal recognition

E

mployeephasesofcha

nge

Enablers



Celebrations

Personal recognition

Reinforcement to sustain the

changeR

Practice applying new skills or using new processes and tools

Coaching

Mentoring

Removal of barriers

Ability to implement required

skills and behaviorsA


Information access

Examples

Role models

Knowledge on how to changeK

Fear of job loss

Discontent with the current state Imminent negative consequence



Career advancement






Desire to participate and

support the changeD

Management communications

Customer input

Marketplace changes

Ready-access to information

Awareness of the need for

changeA


ADKAR Summary

plan communication about a change

identify why changes are not working

helps individuals to see why they have difficulties with a

change


18/18


Conclusion & Closing

Test Process Improvement is not easy. The hardest part

are not the test techniques; but probably theimplementation of the required changes

Models are here to help, by providing a roadmap, not by

being the silver bullet

Questions? Now

or later at [email protected]

Thank you for your time and attention!


References

CMMIhttp://www.sei.cmu.edu/cmmi

TMM

http://www.asq.org/pub/sqp/past/vol1_issue4/burnstein.html

TPIhttp://www.sogeti.nl/tpi

IDEALhttp://www.sei.cmu.edu/ideal

ADKARhttp://www.change-management.com/

Date post:	04-Apr-2018
Category:	Documents
Upload:	phanindrakishore
View:	215 times
Download:	0 times