Agile metrics and quality

transcript

Agile Metrics and Quality

Metrics -> Improvements

Use metrics to• Improve quality• Shorten product cycle time• Decrease costs• Decrease risks• Increase predictability and reliability

Control Limits and Variances

We can have limits for• WIP• Defects• Team size• Velocity• Time-boxes• Others?We can analyze variances to

understand the special causes of variation

Focus on Trends• Trends matter more than values• Trends and spikes are only the symptoms• Trends are leading indicators

Images from MountainGoat Software

Improve quality◦Quality is measured by passed tests

and customer acceptance◦Quality is defined by Definition of

Done and Acceptance Criteria◦Defect trends to use in Agile projects

Number of open defects Escaped defects Defect Cycle Time Defect spill-over

◦Test Automation is critical!

Defect Injection and Resolution

What is the trend of total Active Defects?Image from MountainGoat Software

Analyzing and Preventing Defects

Fishbone / 5 Why Technique

Pareto Chart

Escaped Defects

Calculation:

◦ For all releases, find all defects related to the release found after the release date

◦ Add up all the defects

◦ Can be captured per day / week / month / … or per sprint / release

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

• Used to measure the quality of delivered code

Escaped Defects over Time

Defect Cycle Time• Rapid resolution of defects is important• Average bug-fix time can also be tracked for

different priority bugs

1 2 3 4 5 6 7 8 9 100

Defect Cycle Time

Sprint No.

Desired Threshold

Defect Spill-over• Resolution of defects for a sprint story within the

sprint is important• Definition of Done can include a criteria like “No P1

defects”• We can also tracked whether spilled-over defects are

being closed in the next sprint

1 2 3 4 5 6 7 8 9 100

Defect Spill Over

Sprint No

f defe

Test AutomationCritical factor for regression testingCoverage should be as high as

possible

Shorten product cycle time

Cumulative Flow Diagrams◦Lead Time◦Cycle Time

CFD – An ElaborationFe

What does this CFD say?

Not startedStartedCompleted

Too much Work-In-Progress!

Image from MountainGoat Software

Identifying bottlenecks with a CFD

Where is the bottleneck?In DB Procs – After the widening Analysis phase

1 2 3 4 5 6 7 8 8 10 11 12 13Day

Image from http://www.soliantconsulting.com

Decrease costs• Earned Value Management• Wastage

Cost Variance (CV) = EV – ACSchedule Variance (SV) = EV – PVCost Performance Index (CPI) = EV / ACSchedule Performance Index (SPI) = EV / PVEstimate at Completion (EAC) = BAC / CPI or

= AC + BAC – EVEstimate To Complete (ETC) = EAC – ACVariance at Completion (VAC) = BAC – EAC

PV – Planned ValueEV - Earned ValueAC - Actual costBAC - Budget @ CompletionEAC - Estimate @ CompletionETC- Estimate to CompletionVAC- Variance @ Completion

Earned Value Analysis

EVM - InterpretationPV EV AC Schedule Cost

$1 $1 $1 On schedule On cost

$2 $2 $1 On schedule Under cost

$1 $1 $2 On schedule Over cost

$1 $2 $2 Ahead of schedule On cost

$1 $2 $3 Ahead of schedule Over cost

$1 $2 $1 Ahead of schedule Under cost

$3 $2 $1 Behind schedule Under cost

$2 $1 $3 Behind schedule Over cost

$2 $1 $1 Behind schedule On cost

EVM - ProgressPV ACEV

Ref: Earned Value and Agile ReportingAnthony Cabri, Mike Griffiths

WastageAny time not dedicated to working on

the highest priority story / task on the sprint backlog

Should be tracked over time and by cause

1 2 3 4 5 6 7 8 9 10 11 120

Wastage (in hrs)

Decrease risks• WIP Limits• Small Iterations• More predictability

Increase Predictability / Reliability

• Velocity• Lead Time• Cycle Time

Metrics to Avoid• Velocity comparisons• Productivity e.g. Story Points per day• Effort Estimation accuracy• Number of stories completed• LOC/Function Points• Code quality

Questions?

Contact Anu.khendry@gmail.com

Agile metrics and quality

Documents