How can metrics help in improving our effectiveness?

transcript

Metrics Workshop

Gaetano Mazzanti

@mgaewsj

agile42

Expectations

Which metrics are you using?

What is the goal?

Problems, anyone?

Exercise

Penny Game

How large (and valuable) are your batches?

Velocity, oh my

Correlation between Story Points and Time?

Ian Carroll collected data from 25 teams across 4 different organizations, 12500 work items in total – here are data from one of those teams

https://iancarroll.com/2016/04/18/no-correlation-between-estimated-size-and-actual-time-taken/

Story Points vs # of Stories

# of Stories

Exercise

Throughput Forecast

Trial1

Trial2 Trial3 Trial4 Trial5 Trial6 Trial7 Trial8 Trial9 Trial10

Trial11

7 11 7 5 17 5 10 16 19 7 10 5 13 13 5 7 6 5 5 3 5 16 6 5 6 19 5 3 5 3 6 3 5 7 10 5 6 8 8 6 5 7 19 10 16 8 10 16

48 56 56 31 62 53 45 53 40+ 50+ 50+ 30+ 60+ 50+ 40+ 50+

Sum of all samples for each trial by column (upper) / Nearest “tens” grouping rounded down (lower)

by Troy Magennis https://creativecommons.org/licenses/by-nc-sa/2.0/

Throughput Forecast

by Troy Magennis https://creativecommons.org/licenses/by-nc-sa/2.0/

1. MinimumSum(a): 31 MaximumSum(b): 70

2. MinimumSumwithlowestremoved(c): 39 MaximumSumwithhighest

removed(d): 68

3. Sumalltrials(e): 581 Averagealltrials(e/11):

4. Sumalltrialswithextremesremoved(f): 480 Averagewithextremes

removed(f/9): 53

Actualdataaverage6weekthroughput=57.75.Howclosewasyouraverage?

5. Probabilitiesofachievingatleastnstoriesforasix-weektimespan

SixWeek

Throughput

Counttrialsumgroupsat

least30,40,50,etc.stories

(Count/11)

Likelihood

Atleast30stories 11 1 Atleast40stories 9 0.82 Atleast50stories 7 0.63 Atleast60stories 3 0.28 Atleast70stories 1 0.09 Atleast80stories 0 0 Atleast90stories 0 0

Thisvalueis0to1

Multiplyitby100togetapercentage.0%=nochance,

100%meanseverytrialachievedatleastthislevel.

but… beware

Exercise

Simpson’s Paradox

Simpson’s paradox Global Natural Treat

Live 108 153

Die 123 120

Natural 47% live Treat 56% live

Women Natural Treat

Live 57 32

Die 100 57

Natural 36,3% live Treat 36,0% live

Men Natural Treat

Natural live Treat live

Simpson’s paradox

Global Natural Treat

Live 108 153

Die 123 120

Women Natural Treat

Live 57 32

Die 100 57

Natural 36,3% live Treat 36,0% live

Men Natural Treat

Live 51 121

Die 23 63

Q: why do we need metrics?

really, why do we need metrics?

key reason: to improve

really, why do we need metrics?

decisions

why do we need metrics?

predictability

key questions about your metrics

what do you want to improve?

what do you want to learn?

which metrics?

matter to customers no yes

end-to-end (global)

functional (local)

typical

pizza delivery

fast delivery

accuracy and quality

predictability

what matters to customers

just three metrics?

Lead Time

Throughput

Little’s Law

Items In Queue = Arrival Rate * Waiting Time

Lead Time = Work In Progress / Throughput

ideas elaborate do delivered validate

Lead Time

commitment point

1-Feb 3-Feb 5-Feb 7-Feb 9-Feb 11-Feb 13-Feb 15-Feb 17-Feb 19-Feb 21-Feb

scatterplot

days average

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Backlog To Do In Progress Done

throughput

more about CFDs

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

arrivals

departures

arrivals

departures

arrivals

departures

average lead time

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

average lead time

leading

lagging

some CFD issues

flat line

growing area

staircase

scatterplot

source ActionableMetrics

scatterplot (only bugs)

source ActionableMetrics

frequency distribution the shape of uncertainty*

source ActionableMetrics *Dominica DeGrandis

Weibull distribution

Weibull(1, 0.5)

Weibull(2, 0.5)

Weibull(3, 1)

efficiency

process efficiency = total time active time ________

20 5 = 25%

!1!!2!!3!!4!!5!!6!!7!!8!!9!10!11!12!13!14!15!16!17!18!19!20!

elaborate do validate deliver

waitingactive

correlations…

Productivity (throughput)

Quality (# of bugs)

Predictability (variance in throughput)

Responsiveness (Time in Process)

Stability

correlation with WIP

source Larry Maccherone

correlation with Team Focus

correlation with Team Stability

Exercise

FeatureBan

by Mike Burrows of Positive Incline http://creativecommons.org/licenses/by-sa/4.0/ https://blog.agendashift.com/2016/06/02/featureban-2-0/

tempus fugit

focus on trends

(not on absolute values)

"If you torture the data long enough, it will confess to anything you'd like"

Ronald Coase, economist

Gaetano Mazzanti

@mgaewsj

gaetano.mazzanti@agile42.com

agile42

How can metrics help in improving our effectiveness?

Education