Software Quality Management Copyright, 1999 © Jerzy R. Nawrocki [email protected]...

Software Quality ManagementSoftware Quality Management

Copyright, 1999 © Jerzy R. Nawrocki

[email protected]@put.poznan.pl

www.cs.put.poznan.pl/jnawrocki/www.cs.put.poznan.pl/jnawrocki/mse/psp/mse/psp/

Personal Software Process Personal Software Process

Lecture 12Lecture 12

J. Nawrocki, PSP, Lecture 12

From the previous lectureFrom the previous lecture

1 Introduction1 Introduction

2 Management plan2 Management plan

3 Quality plan3 Quality plan

4 Purchasing plan4 Purchasing plan

5 Development plan5 Development plan

6 Test plan6 Test plan

7 Installation and delivery7 Installation and delivery

8 Product management8 Product management

9 Maintenance9 Maintenance

10 The project plan10 The project plan


From the previous lectureFrom the previous lecture

A good plan should be:A good plan should be:• complete (forms)complete (forms)• accessible (a proper place & accessible (a proper place &

format)format)• clear (readable)clear (readable)• specific (what, when, by whom, at specific (what, when, by whom, at

what costs)what costs)• precise (unit of time)precise (unit of time)• accurate (random errors - no accurate (random errors - no

problem)problem)


IntroductionIntroduction

What is “software quality” ?What is “software quality” ?

Is quality an important issue ?Is quality an important issue ?

How to improve quality ?How to improve quality ?


Software qualitySoftware quality

American Heritage American Heritage DictionaryDictionary: a : a characteristic or attribute characteristic or attribute of somethingof something

Słownik języka polskiegoSłownik języka polskiego: : właściwość, rodzaj, właściwość, rodzaj, gatunek, wartośćgatunek, wartość



Philip Crosby’83Philip Crosby’83: : conformance to conformance to requirementsrequirements



Quality of designQuality of design

Quality of conformance



Quality controlQuality control

• product-orientedproduct-oriented

• a part of the a part of the manufacturing manufacturing processprocess

• passivepassive

• inspections, inspections, reviews, testsreviews, tests



Quality assuranceQuality assurance

• process-orientedprocess-oriented

• a part of the a part of the management management processprocess

• activeactive

• audits and reportsaudits and reports



Quality managementQuality management

• What level of quality What level of quality do we need ?do we need ?

• How quality is How quality is important for our important for our business ?business ?

• How much is it How much is it going to cost ?going to cost ?



• Product quality Product quality Process Process qualityquality

• The user of the PSP is the The user of the PSP is the programmerprogrammer

• The main requirement: to The main requirement: to produce a quality softwareproduce a quality software

• Other requirements: Other requirements: efficiency, ease of efficiency, ease of learning, ..learning, ..

I like thatI like that

process ..process ..


Finding and fixing defectsFinding and fixing defects

The cost elements:The cost elements:

• Determining that there is a problemDetermining that there is a problem

• Isolating the source of the problemIsolating the source of the problem

• Determining exactly what is wrongDetermining exactly what is wrong

• Fixing the requirements as neededFixing the requirements as needed

• Fixing the design as neededFixing the design as needed

• Fixing the implementation as Fixing the implementation as neededneeded


Finding and fixing defectsFinding and fixing defects

The cost elements (contd.):The cost elements (contd.):

• Inspecting the fixInspecting the fix

• Testing the fix to ensure it fixes Testing the fix to ensure it fixes the identified problemthe identified problem

• Testing the fix to ensure it Testing the fix to ensure it doesn’t case other problemsdoesn’t case other problems

• Changing the documentation as Changing the documentation as need to reflect the fixneed to reflect the fix


Some fix time dataSome fix time data

The relative times to identify The relative times to identify the defects (the defects (TRWTRW):):

• during requirements: 1during requirements: 1• during design: 3 - 6during design: 3 - 6• during coding: 10during coding: 10• in development test: 15 - 40in development test: 15 - 40• in acceptance test: 30 -70in acceptance test: 30 -70• during operation: 40 - 1000during operation: 40 - 1000


Some fix time dataSome fix time data

The relative time to identify The relative time to identify defects (defects (IBMIBM):):

• during design reviews: 1during design reviews: 1

• during code inspections: 20during code inspections: 20

• during machine test: 82during machine test: 82


Some fix cost dataSome fix cost data

Relative Relative costs to identify software costs to identify software defects (defects (IBMIBM):):

• during design: 1.5during design: 1.5• prior to coding: 1prior to coding: 1• during coding: 1.5during coding: 1.5• prior to test: 10prior to test: 10• during test: 60during test: 60• in field use: 100in field use: 100


Some fix cost dataSome fix cost data

JPLJPL, average cost per defect:, average cost per defect:• inspections: $90 - $120inspections: $90 - $120• tests: $10 000tests: $10 000

Conclusions:Conclusions:

Remove most of the defects Remove most of the defects before testingbefore testing


A case study - problemA case study - problem

• 5-person team5-person team• Estimated size: 50 000 LOC in C++Estimated size: 50 000 LOC in C++• Requirements + prototyping: 10 monthsRequirements + prototyping: 10 months• High level design: 3 monthsHigh level design: 3 months• Estimated completion time: 21 monthsEstimated completion time: 21 months• Integration and system tests: to start in 5 monthsIntegration and system tests: to start in 5 months• Testing should take 3 monthsTesting should take 3 months• Inspections would cost another 3 monthsInspections would cost another 3 months• Can we afford to inspect specification before Can we afford to inspect specification before

starting implementation ?starting implementation ?


A case study - problemA case study - problem

How did we get the 3-month How did we get the 3-month testing estimate ?testing estimate ?

Brooks’72: Brooks’72: large software large software projects are about half done projects are about half done when integration starts.when integration starts.

More realistic schedule: More realistic schedule:

2 x 18 = 36 months2 x 18 = 36 months


A case study - testingA case study - testing

• An experienced software engineer injects ~ 100 An experienced software engineer injects ~ 100 defects per KLOCdefects per KLOC

• Half these defects are found by the compilerHalf these defects are found by the compiler• 50 KLOC x 50 defects / KLOC = 2 500 defects50 KLOC x 50 defects / KLOC = 2 500 defects• Hours to find a defect (industry data): 3 - 20Hours to find a defect (industry data): 3 - 20• 2 500 defects x 8 hours/defect = 2 500 defects x 8 hours/defect = 20 00020 000 hours hours• 360 days x 11 hours/day = ~ 4 000 hours360 days x 11 hours/day = ~ 4 000 hours• To test the system To test the system 5 programmers5 programmers would have to would have to

work work 360 days360 days, , 11 hours11 hours per day per day


A case study - inspectionsA case study - inspections

• The yield of the review or inspection: ~ 70%The yield of the review or inspection: ~ 70%• Average cost of inspections: 0.5 hours/defectAverage cost of inspections: 0.5 hours/defect• 70% of 2 500 defects = 1 750 defects70% of 2 500 defects = 1 750 defects• Inspection timeInspection time: 1 750 defects x 0.5 : 1 750 defects x 0.5

hours/defect = 875 hourshours/defect = 875 hours• Testing timeTesting time: 750 defects x 8 hours/defect = : 750 defects x 8 hours/defect =

6 000 hours6 000 hours• Total timeTotal time: 6 875 hours ~ : 6 875 hours ~ 1/31/3 x 20 000 hours x 20 000 hours• Reviews and inspections would save us 8 Reviews and inspections would save us 8

months out of 36 monthsmonths out of 36 months


Cost-of-quality componentsCost-of-quality components

• Failure costsFailure costs (diagnosing a failure, (diagnosing a failure, repairs)repairs)

• Appraisal costsAppraisal costs (evaluating the product (evaluating the product to determine its quality)to determine its quality)

• Prevention costsPrevention costs (identifying the causes (identifying the causes of defects; preventing them in the of defects; preventing them in the future)future)


PSP COQ componentsPSP COQ components

• Failure costsFailure costs: the total time spent : the total time spent in compile and testin compile and test

• Appraisal costsAppraisal costs: the time spent in : the time spent in design and code reviews + any design and code reviews + any inspection timesinspection times


PSP COQ measuresPSP COQ measures

• Failure COQFailure COQ = 100 * (compile time + test = 100 * (compile time + test time) / total development timetime) / total development time

• Appraisal COQAppraisal COQ = 100 * (design review time + = 100 * (design review time + code review time) / total development timecode review time) / total development time

• Total COQTotal COQ = Appraisal COQ + Failure COQ = Appraisal COQ + Failure COQ• Appraisal as a % of Total Quality CostsAppraisal as a % of Total Quality Costs = 100 = 100

* Appraisal COQ / Total COQ* Appraisal COQ / Total COQ• A/FR ratio = A/FR ratio = Appraisal to failure cost ratioAppraisal to failure cost ratio = =

Appraisal COQ / Failure COQAppraisal COQ / Failure COQ


Developing a Quality StrategyDeveloping a Quality Strategy

• Decide how to measure your processDecide how to measure your process

• Determine the quality methods that are Determine the quality methods that are most effective for youmost effective for you

• Periodically re-evaluate your strategy Periodically re-evaluate your strategy and set new goalsand set new goals


Measuring the processMeasuring the process

Yield (step Yield (step nn)) = 100 * r / (r + e) = 100 * r / (r + e)

r - defects removed in step r - defects removed in step nn

e - defects escaping from step e - defects escaping from step nn

Yield (overall)Yield (overall) = 100 * R / I = 100 * R / I

R - defects removed before compileR - defects removed before compile

I - defects injected before compile entryI - defects injected before compile entry


Basic defect causesBasic defect causes

• EducationEducation: don’t understand how to do : don’t understand how to do somethingsomething

• CommunicationCommunication: not properly informed : not properly informed about somethingabout something

• OversightOversight: omitted doing something: omitted doing something

• TranscriptionTranscription: knew what to do but : knew what to do but made a mistakemade a mistake

• ProcessProcess: the process somehow : the process somehow misdirected the actionsmisdirected the actions


Defect preventionDefect prevention

• Select a specific defect typeSelect a specific defect type• What are the causes of the defect ?What are the causes of the defect ?• Why it has been made ? (education, communication, Why it has been made ? (education, communication,

oversight, ..)oversight, ..)• How to prevent the problem in the future ?How to prevent the problem in the future ?• Repeat the above steps a few timesRepeat the above steps a few times• What are the trends or patterns in the data that might What are the trends or patterns in the data that might

suggest larger problems ?suggest larger problems ?• What are the actions that had worked in the past ?What are the actions that had worked in the past ?• Test your defect prevention ideasTest your defect prevention ideas


Defect preventionDefect prevention

Exemplary defect types:Exemplary defect types:

• most frequentmost frequent

• most difficult to find and fixmost difficult to find and fix

• easiest to find and fixeasiest to find and fix

• . . .. . .


Quality assurance techniquesQuality assurance techniques

• Inspections and reviewsInspections and reviews

• Software configuration Software configuration managementmanagement


SummarySummary

Software quality is not only a Software quality is not only a technical issue but also an technical issue but also an economicaleconomical one. one.

Cost-of-qualityCost-of-quality measures allow measures allow us to observe our process us to observe our process and to identify the and to identify the prospective improvementsprospective improvements


Further readingsFurther readings

W.S. Humphrey, A discipline for W.S. Humphrey, A discipline for software engineering, Addison-software engineering, Addison-Wesley, Reading MA, 1995.Wesley, Reading MA, 1995.


Quality assessmentQuality assessment

1. What is your general 1. What is your general impression ? (1 - 6)impression ? (1 - 6)

2. Was it too slow or too fast ?2. Was it too slow or too fast ?

3. Did you learn something 3. Did you learn something important to you ?important to you ?

4. What to improve and how ?4. What to improve and how ?

Date post:	28-Mar-2015
Category:	Documents
Upload:	marlon-melson
View:	212 times
Download:	0 times

Software Quality Management Copyright, 1999 © Jerzy R. Nawrocki [email protected]...

Documents