Personal Software ProcessSoftware Quality
CIS 376
Bruce R. Maxim
UM-Dearborn
These notes are based on:
Introduction to the
Personal Software Process
Watts S. Humphrey
Addison-Wesley Longman (1997)
Software Quality
• Product meets the customer’s needs (not necessarily the same as customer’s wants)
• Defects are the primary measure of quality in PSP
• Yield Management– defect removal– defect prevention
Hierarchy of Needs
• Software performs its required tasks
• Product meets its performance requirements
• Software is usable
• Development is economical and timely
• Product is dependable and reliable
Software Bugs
• Latent bugs must
– be operationally insignificant
– not be destructive
– not be observed often
• Bugs are not important to the customer if they do not
– affect operations
– cause inconvenience
– cost time or money
– cause loss of confidence in software results
PSP Quality Focus
• Low defect content is an essential prerequisite to quality software process
• Low defect rates can best be achieved at the PSP level (e.g. individual SE’s)
• SE’s are the ultimate source of defects are injected
• SE’s need to remove defects, determine their causes, and learn to prevent them
Testing and Inspections
• The sooner a defect is located the cheaper it is to repair and faster the repair
• Design inspections will– improve product quality– lead to a more predictable schedule– allow more time to focus on quality issues
• It is important for SE’s to review their own work before giving it to others to review
Some Fix Time Data
•Some typical fix time ratios–IBM rules of thumb - coding: 1.5; testing: 60; usage: 100
–Boehm - design: 1; development test: 15 to 40; acceptance test: 30 to 70; operation: 40 to 1000
–Remus - design: 1, code: 20, test: 82
–Ackerman - test: 2 - 10 times inspection time
–Russell - inspection: 1, test: 2 to 4, use: 33
–PSP research - unit test takes 12 times longer than code review to find and fix defects
Cost of Quality - part 1
• Failure costs– repair, rework, scrap– in PSP includes all compile and testing time
• Appraisal costs– cost of defect inspections– in PSP includes all design and code review time
Cost of Quality - part 2
• Prevention costs– finding and resolving defect causes– handle before project starts– should be a process (not product) activity– in PSP this includes
• formal specification and design work
• prototyping
• process analysis and improvement
PSP Quality Strategy - part 1
• Identify PSP quality objectives, e.g.– remove all defects before first compile
• Establish PSP process quality measures, e.g.– overall process yield– LOC reviewed per hour
• Examine products reviewed– determine their ratings on the measures– see which behaviors impacted these results
PSP Quality Strategy - part 2
• Based on these data, identify your most effective work practices.
• Incorporate these practices in your process artifacts– process scripts– checklists– forms
• Identify measures that predict process quality– use these as control variables
– set specifications for these variables
PSP Quality Strategy - part 3
• Track your performance against these specifications.
• Track your process to determine– when and if to change these specifications
– actions to take for further process improvement
Appraisal to Failure Cost Ratio
• Appraisal COQ =100*(design review time + code time)/
(total development time)
• Failure COQ (Cost of Quality) =100*(compile time + test time)/(total development time)
• A/FR (Appraisal to Failure Cost Ratio) ratio =
(Appraisal COQ)/(Failure COQ)
Yield vs. A/FR
•High A/FR ratios appear to lead to higher yields
•70+% yields not achieved without A/FRs near 1.0 or above
•high A/FR does not guarantee high yield - the appraisal time must be spent effectively
Test Defects vs. A/FR
•Defects are reduced by high A/FR ratios
•To get very low numbers of test defects, A/FR values of above 2.0 appear required.
•With A/FRs between 1.0 and 2.0, low test defects are occasionally obtained.
•With an A/FR of below 1.0, low test defects are rare.
Yield and A/FR vs. Productivity
•Productivity has considerable variation among individuals.
•In some cases, high yield produces higher productivity but in others it does not.
•High A/FR also sometimes results in high productivity and sometimes not.
•Yield and A/FR are somewhat independent of productivity.
Process Benchmarks
•It is desirable to have high values for–yield
–A/FR
–productivity
•Since yield and A/FR are closely related, a yield vs A/FR benchmarking chart would not be useful.
•Yield vs. productivity or A/FR vs. productivity would likely be useful benchmarking charts.
Defect Removal Strategies - part 1
• Focused inspections and reviews– high level design reviews - structural issues– detail level design reviews - logic correctness– code reviews - implementation issues
• Do not address– system issues in DLD– design issues in code reviews
Defect Removal Strategies - part 2
• Do thorough reviews the first time and then trust them.
• Do thorough unit tests– black box– white box
• Do system tests once unit testing is done– integration– system– regression
Defect Prevention
• Finding defects is expensive so it is better to avoid them in the first place
• Defect prevention costs are only incurred once, but their savings impact every future project
• For PSP, defect prevention actions include improved design methods and prototyping
Defect Prevention Strategies
• Set priorities for defect types that are– found most frequently– the most troublesome– easily prevented
• In setting priorities consider the defects most often found in integration and system testing
• Incorporate your prevention actions in your process scripts, checklists, and forms
Defect Prevention Process
• Follow an established schedule
• Select 1 or 2 defect types for initial action
• Track and evaluate the effectiveness of the defect prevention actions
• Make needed adjustments and continue