17
Slide 1 SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London Improved Software Defect Prediction
Slide 1
SPIN23 February 2006
Norman Fenton
Agena Ltd and
Queen Mary University of London
Improved Software Defect Prediction
Slide 2
delivery
Pre-release testing Post-release operation
Using fault data to predict reliability
Slide 3
Pre-release vs post-release faults?
0
0
Post-release faults
10
20
30
40 80 120 160
Pre-release faults
?
Slide 4
Pre-release vs post-release faults: actual
Post-release faults
0
10
20
30
0 40 80 120 160
Pre-release faults
Slide 5
Software metrics….?
Slide 6
Regression models….?
Slide 7
Solution: causal models (risk maps)
Slide 8
What’s special about this approach?
• Structured
• Visual
• Robust
Slide 9
The specific problem for Philips
time
Defectsfound
predicted
actual
Slide 10
Background to work with Philips
Fixed Risk Map (AID)
2000
MODIST
Agena Risk
2001 20032002 2004
Reliability models
Slide 11
Projects in the trial
MM&C (9)28%
TV (16)50%
DVD (7)22%
(actual number of projects shown in brackets)
Slide 12
Factors used at Philips
• Existing code base…
• Complexity and management of new requirements ...
• Development, testing, rework, process …
• Overall project management …
Slide 13
Actual versus predicted defects
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 500 1000 1500 2000 2500
Actual defects
pre
dic
ted
def
ects
Correlationcoefficient
95%
Slide 14
Validation Summary (Philips’ words)
“Bayesian Network approach is innovative and one of the most promising techniques for predicting defects in software projects”
“Our evaluation showed excellent results for project sizes between 10 and 90 KLOC”
“Initially, projects outside this range were not within the scope of the default model so predictions were less accurate, but accuracy was significantly improved after standard model tailoring”
“AgenaRisk is a valuable tool as is”
Slide 15
Specific benefits
• Accurate predictions of defects at different phases
• Know when to stop testing
• Identify where to allocate testing
• Minimise the cost of rework
• Highlight areas for improvement
• Use out of the box now if you have no data
• Approach fully customisable
• Models arbitrary project life-cycles
Slide 16
Summary
• Risk maps – the way forward
• Validation results excellent
Slide 17
…And
You can use the technology NOW
www.agenarisk.com
