Statistically controlling the software process

8/14/2019 Statistically controlling the software process

1/25

1

Carnegie Mellon University

Software Engineering Institute

Statistically Controlling the

Software Process

Anita D. Carleton and William A. FloracThe 99 SEI Software Engineering Symposium

September 1, 1999

Software Engineering InstituteCarnegie Mellon University

Pittsburgh, PA 15213-3890

Sponsored by the U.S. Department of Defense

1999 by Carnegie Mellon University


2/25

2



Topics

Introduction

Statistical Thinking and Process Thinking

Understanding Variation

Control Charts

Software Example

10 Steps to Get Started and FAQs


3/25



Process Thinking

Focus on the processes to improve quality and

productivity

Managers must focus on fixing processes, not

blaming people

Management action uses data from the process

to guide decisions

Recognize that variation is present in all

processes and that it is an opportunity for

improvement


4/25



Statistical Thinking

Fundamental axioms

all work is a series of interconnected

processes

all processes are variable

understanding variation is the basis for

management by fact and systematicimprovement

Source: G. Britz, How to Teach Others to Apply Statistical Thinking, Quality Progress, June 1997.


5/25



Understanding Variation

While every process displays variation, some processes

display controlled variation, while others displayuncontrolled variation.

- Walter Shewhart

Common cause or controlled variation - due to normal orinherent activities among the process components

Assignable cause or uncontrolled variation - due to

anomalies in the process

total variation = common cause variation + assignable

cause variation


6/25

6



Process BehaviorVariation andStability

Variation = process noise + process anomalies

Stable process =

Stable process = Controlled process

process anomalies


7/25

7



Statistical Process Control

A phenomenon will be said to be

control led when, through the use ofpast experience, we can predict, atleast within l imits, how thephenomenon may be expected to

vary in the future.

Walter A. Shewhart, 1931


8/25



Why SPC for Software Development?

To understand the reliability of human processes

To establish bounds on management expectations

To understand patterns and causes of variations

To validate metric analysis used to forecast and plan

Source: T. Keller, Applying SPC Techniques to Software Development--A Management Approach, 1999 SEPG Conference.


9/25



Control Charts - 1

Lower Control Limit

(LCL)

Upper Control Limit

(UCL)

Specification

Limits

Limits

Control Limits Determined by Process Performance Measurements(Voice of the process)

Specification Limits Set by customer, engineer, etc.(Voice of the customer)


10/25



Control Charts - 2

UCL

LCL

DefectsPer

KLOC

Voice

of the

Process

Time

Inspection Process Performance


11/25

11



Control chartslet you know whatyour processes cando, so that you canset achievable goals.

They represent the voice of the process.

Control charts provide the evidence of stability that

justifies predicting process performance.

Control charts separate signal from noise, sothat you can recognize a process change when

it occurs.

Why Control Charts?

0 5 10 15 20 25 30

10

14

18

22

26

30

LCL=7.89

CL=20.4

UCL=32.9

Backlog of Unresolved Problem Reports

IndividualValues


12/25

12



Detecting Instabilities and Out-of-Control Situations - 1

To test for instabilities in processes, we examinecontrol charts for instances and patterns that

signal nonrandom behavior.

Values falling outside the control limits andunusual patterns within the running record

suggest that assignable causes exist.


13/25

13




Test 1: A single point falls outside the 3-sigma control limits.

Test 2: At least two of three successive values fall on

the same side of, and more than two sigma

units away from, the center line.

Test 3: At least four out of five successive values fall

on the same side of, and more than one sigma

unit away from, the center line.

Test 4: At least eight successive values fall on the

same side of the center line.

Source: Western Electric Handbook


14/25



0 5 10 15 20 25

-20

-10

0

10

20

30

40

50

60

.

LCL

UCL

Individuals

Subgroup No.


Test 3: 4 out of 5

signals in zone B

Test 1:Single point outside of zone CTest 2:

2 out of three beyond zone B

Test 4: 8 successive pointson same side of

centerline

Zone A

Zone A

Zone B

Zone C

Zone B

Zone C

CL


15/25



Component 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Totals

Defects 12 16 18 32 22 16 23 35 15 27 16 25 20 26 20 23 23 36 22 27 17 471

Defect Type Number of Defects per Type per Component

Function 3 5 4 4 4 3 3 20 4 11 2 3 3 5 3 7 4 5 5 15 2 115

Interface 2 2 4 4 3 4 2 3 3 4 2 3 5 3 3 3 2 16 6 2 4 80

Timing 1 1 0 1 1 0 2 1 0 0 2 0 1 1 1 1 1 0 1 0 0 15

Algorithm 0 0 1 14 2 0 0 0 0 0 0 1 5 2 7 6 5 1 2 0 1 47

Checking 1 1 5 1 7 1 1 2 0 1 6 3 1 12 1 0 2 4 3 5 2 59

Assignment 0 2 0 4 1 2 1 3 2 3 2 8 1 0 2 1 2 1 0 1 1 37

Build/Pkg. 3 1 1 2 1 0 0 4 3 6 1 0 2 1 1 1 3 2 2 2 1 37

Document 2 4 3 2 3 6 14 2 3 2 1 7 2 2 2 4 4 7 3 2 6 81

Example: Summary of Defect TypesFound During Component Inspections

Source: W. Florac and A. Carleton,Measuring the Software: Statistical Process Control for Software Process Improvement, Addison-Wesley,

June 1999.


16/25



0 5 10 15 20

25

05101520

303540

45

LCL = 0

CL = 22.4

UCL = 44.9

MovingRange

TotalDefects

Component Number

0 5 10 15 20

0

5

10

15

20

25

30

CL = 8.45

UCL = 27.6

XmR Charts for the Total Numberof Defects Found in Component Inspections

Source: W. Florac and A. Carleton,Measuring the Software: Statistical Process Control for Software Process Improvement, Addison-Wesley,

June 1999.


17/25



0

1

2

3

4

5

6

ControlCharts

forIndividualDefect

Types

0 5 10 15 20

0

2

4

6

8

10

12Defect Type = Checking

0 5 10 15 200

5

10

15

20

Defect Type = Function

0

2

4

6

8

10

12

14

1618

0 5 10 15 20

Defect Type = Interface

Defect Type = Algorithm

0 5 10 15 20

0

2

4

6

8

10

12

14

16

0

2

4

6

8

0 5 10 15 20

Defect Type = Assignment

0 5 10 15 20

Defect Type = Build/Pkg.

0

2

4

6

8

10

12

14

16

0 5 10 15 20

Defect Type = Documentation

Defect Type = Timing

0.0

0.5

1.0

1.5

2.0

2.5

0 5 10 15 20

Source: W. Florac and A. Carleton,Measuring the Software: Statistical Process Control for Software Process Improvement, Addison-Wesley, June 1999.


18/25



0 5 10 15 200

2

4

6

8

10

12

0

2

4

6

8

10

12

14

16

Defect Type = Timing

0.0

0.5

1.0

1.5

2.0

2.5

0 5 10 15 20

0 5 10 15 200

5

10

15

20Defect Type = Function

X

X

X

0

2

4

6

8

10

12

14

16

18

0 5 10 15 20

Defect Type = Interface X

0 5 10 15 200

2

4

6

8

10

12

14

16

X

Defect Type = Algorithm

X XX

X

0

1

2

3

4

5

6

0 5 10 15 20

Defect Type = Build/Pkg. X

Defect Type = Documentation

0 5 10 15 20

X

0 5 10 15 20

0

2

4

6

8Defect Type = Assignment

X

Defect Type = CheckingX

RevisedControl

Chartsfor Eachof the

DefectTypes


19/25



810121416182022

24

0

2

4

6

810

0 5 10 15 20LCL=6.634

CL=15.81

UCL=24.99

MovingRang

e

TotalDefec

ts

Component Inspection Sequence0 5 10 15 20

CL=3.45

UCL=11.27

Improved Process-Reduction in Defect Insertion

Source: W. Florac and A. Carleton,Measuring the Software: Statistical Process Control for Software Process Improvement, Addison-Wesley,June 1999.


20/25



EvaluatingProcessPerformance


21/25

21



10 Steps to Get Started - 11. Get familiar with SPC techniques; refer to the

SEI measurement guidebook and related

books and papers on SPC.

2. Obtain a SPC tool.

3. Identify process issues.

4. Identi fy process performance attributes.

5. Select and define measures.

6. Collect data.

7. Organize the data and ensure principles underlying SPC

hold (e.g., homogeniety)


22/25

22



10 Steps to Get Started - 28. Plot/graph data.

9. Examine each plot/graph for process stability,

process shifts, and assignable causes. (a) If process NOT STABLE, THEN:

- cannot determine the capabil ity of the process

- no basis to predict outcomes

- action: understand why the process is not

stable and determine what steps can be taken

to achieve stability

(b) If process is STABLE, THEN:

- capability of the process can be determined

and used to compare future process performance

- action: predict future performance and/or

examine other process constituents

10. Run additional analysis as situation requires.


23/25

23



FAQs - 1

1. There are many types of control charts--which ones are

relevant to software?

- XmR charts

- Average and Range Charts

- c and u charts

2. Can I construct control charts with limited software data? How

much data is enough?

- Set trial limits as soon as possible (be cautious in

interpreting initial results)

- Can get early indications of presence of

assignable causes

- Concluding a process is stable with < 20 or 30

subgroups is risky

- Update control limits as more data becomes available


24/25

24



FAQs - 2

3. Do I need to be a high maturity organization to apply SPC

techniques?

- it is advisable to have basic measurement practices

defined ini tially and then use SPC to manage and

improve your process performance

4. Where in the software process can I apply SPC?

- Useful for defining process stability for design, code, and

inspection processes

- Useful for determining i f processes are being conducted

as intended


25/25

25



Date post:	04-Jun-2018
Category:	Documents
Upload:	kosaraju
View:	228 times
Download:	0 times

Statistically controlling the software process

Documents