Agile MetricsVelocity is NOT the Goal

Michael “Doc” NortonGrouponDirector of Engineeringdoc@groupon.com@DocOnDev

Husband Father

GRANDFATHERHE’S SOOO CUTE!!

What is “Agile Velocity”?

WORK UNITS DELIVERED OVER TIMEUMMM.... WOULDN’T THAT BE “AGILE SPEED”?

A LAGGING INDICATORTELLS US ABOUT SOMETHING THAT ALREADY HAPPENED

PLANNING BY VELOCITYUSING THE PAST TO PREDICT THE FUTURE

0

10

20

30

40

03-Dec 17-Dec 31-Dec 14-Jan

Ideal Burn Down

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Ideal

Velocity Graph

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

25

2827

2829

28

24

28

30

27

Velocity

Yesterday’s Weather

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

30 25 28 27 28 29 28 24 28 30

25

2827

2829

28

24

28

30

27

Weather Actual

Weather Burn Down

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Ideal Actual Estimate

THAT DOESN’T APPLY HERE, TONY.I’M SURE WE CAN MAKE THIS BETTER

Rolling Average

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

30 25 27 27 28 28 28 26 27 2730 25 28 27 28 29 28 24 28 30

25

2827

2829

28

24

28

30

27

Weather Rolling Actual

Rolling Burn Down

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Ideal Actual Estimate

All The Burn Downs

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Ideal Actual Rolling Weather

DO YOU FEEL CONFIDENT?ARE THESE NUMBERS GOOD?

STANDARD DEVIATIONSHOWING OUR CONFIDENCE

Standard Deviation:The Basics

IT’S NOT THAT HARDLET’S BREAK IT DOWN

Standard Deviation:Break-Down

Standard Deviation:Break-Down

CALCULATE THE MEAN OF THE POPULATION

Standard Deviation:Break-Down

SUBTRACT THE MEAN FROM EACH ITEM AND SQUARE THE RESULT

Standard Deviation:Break-Down

AVERAGE THOSE RESULTS AND GET THE SQUARE ROOT

Calculate the Mean

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

25

2827

2829

28

24

28

30

27

Velocity

(25+28+27+28+29+28+24+28+30+27)/10 = 27.4

SORRY, TONY.JUST LET ME DO THIS ONE THING.

“I feel like you’re not really hearing me.”

- Tony Robbins

Square the differences

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

25

2827

2829

28

24

28

30

27

Velocity

(25-27.4)**2 = (-2.4)**2 = 5.76(28-27.4)**2 = (0.6)**2 = 0.36(27-27.4)**2 = (-0.4)**2 = 0.16(28-27.4)**2 = (0.6)**2 = 0.36(29-27.4)**2 = (1.6)**2 = 2.56

(28-27.4)**2 = (0.6)**2 = 0.36(24-27.4)**2 = (-3.4)**2 = 11.56(28-27.4)**2 = (0.6)**2 = 0.36(30-27.4)**2 = (2.6)**2 = 6.76(27-27.4)**2 = (-0.4)**2 = 0.16

(25+28+27+28+29+28+24+28+30+27)/10 = 27.4

Square Root Result Mean

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

25

2827

2829

28

24

28

30

27

Velocity

(25-27.4)**2 = (-2.4)**2 = 5.76(28-27.4)**2 = (0.6)**2 = 0.36(27-27.4)**2 = (-0.4)**2 = 0.16(28-27.4)**2 = (0.6)**2 = 0.36(29-27.4)**2 = (1.6)**2 = 2.56

(28-27.4)**2 = (0.6)**2 = 0.36(24-27.4)**2 = (-3.4)**2 = 11.56(28-27.4)**2 = (0.6)**2 = 0.36(30-27.4)**2 = (2.6)**2 = 6.76(27-27.4)**2 = (-0.4)**2 = 0.16

(25+28+27+28+29+28+24+28+30+27)/10 = 27.4

(2.84)**1/2 = 1.685

Standard Deviation Is:

0

6

12

18

24

30

1 2 3 4 5 6 7 8 9 10

25

2827

2829

28

24

28

30

27

Velocity

1.685

Burn Downs w/1SD

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Ideal Actual Estimate High Low

Burn Downs w/2SD

0

125

250

375

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Ideal Actual Estimate High Low

Set A:10, 11, 9, 10

Set B:10, 14, 9, 7

Set A:10, 11, 9, 10

Set B:10, 14, 9, 7

Mean: 10 Mean: 10

Rolling: 10 Rolling: 10

Set A:10, 11, 9, 10

Set B:10, 14, 9, 7

Mean: 10 Mean: 10

Deviation: 0.7 Deviation: 2.5Rolling: 10 Rolling: 10

Set A:10, 11, 9, 10

Set B:10, 14, 9, 7

Mean: 10 Mean: 10

Deviation: 0.7 Deviation: 2.5Rolling: 10 Rolling: 10

Set A:10, 11, 9, 10

Set B:10, 14, 9, 7

Mean: 10 Mean: 10

Vel: 9.3-10.7 Vel: 7.5-12.5

WHOOPS.NOW TONY’S ANGRY.

“You know what? Just... Do what you want. I don’t care.”

- Tony Robbins

YOU CAN’T MANAGE WHAT YOU CAN’T MEASURESAYS WHO?

SAYS WHO?

SAYS WHO?

Dr. W. Edwards DemingFather of the Quality Evolution

You can’t manage what you

can’t measure.

Dr. W. Edwards DemingFather of the Quality Evolution

Don’t be ridiculous!

Dr. W. Edwards DemingFather of the Quality Evolution

Running a company on

visible figures alone [is deadly]

Seven Deadly Diseasesof

Western Management1) Lack of constancy of purpose2) Emphasis on short-term profit3) Evaluation of performance, merit rating or annual reviews4) Mobility of top management5) Running a company on visible figures alone6) Excessive medical costs7) Excessive legal damage awards

THE HAWTHORN EFFECTTHAT WHICH IS MEASURED, WILL IMPROVE

What matters is not

setting quantitative

goals but fixing the method by which those

goals are attained

REPAIR THE ROOT CAUSESRATHER THAN THE SYMPTOMS

INSTABLE VELOCITYWHAT DOES IT TELL US?

POOR STORY COMPOSITIONYOU WANT CONSISTENT (SMALL) STORIES

TOO MUCH WIPGET THE HIGHEST VALUE DONE FIRST

DEPENDENCY ON OTHER TEAMGET EVERYONE YOU NEED ON THE TEAM

STABLE VELOCITYIT WON’T EVER BE PERFECT

INCREASE VELOCITYDO WE ALWAYS WANT TO GO FASTER?

GOODHART’S LAWMAKING A TARGET FOR A METRIC DESTROYS THE METRIC (AND THE TARGET)

What matters is not

setting quantitative

goals but fixing the method by which those

goals are attained

REPAIR THE ROOT CAUSESRATHER THAN THE SYMPTOMS

Scatter Diagrams

Scatter Diagrams

0

5000

10000

15000

20000

0 2 4 6 8

Velocity & Complexity

0

12.5

25

37.5

50

0 25 50 75 100

Velocity & Value

0

12.5

25

37.5

50

0 10000 20000 30000 40000

Velocity & Coverage

0

12.5

25

37.5

50

0 22.5 45 67.5 90

Cumulative Flow Diagrams

SAMPLE BACKLOGWHAT MATTERS HERE IS THE STAGES OF WORK

CUMULATIVE FLOW DIAGRAMVELOCITY DOESN’T ALWAYS TELL US ENOUGH

What does this tell us?

0

5

10

15

20

1 2 3 4 5 6 7 8 9 10

Velocity

What does this tell us?

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10

Deployed Ready for Approval In Testing In Progress Ready To Start

Balanced Metrics

Balanced Metrics

0

4

8

12

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Velocity

Balanced Metrics

0

4

8

12

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

12.5

25

37.5

50

Velocity Quality

Balanced Metrics

0

4

8

12

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

12.5

25

37.5

50

Velocity Quality Avg. Hours

Balanced Metrics

0

4

8

12

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

12.5

25

37.5

50

Velocity Quality Avg. Hours Team Joy

Michael “Doc” NortonGrouponDirector of Engineeringdoc@groupon.com@DocOnDev

THANK YOU!

AttributionsPlanning By Velocity Photo

http://www.flickr.com/photos/carfull/4984803509/sizes/l/in/photostream/Are You Confident Photo

http://www.brookston.org/photoblog/images/20060113083815_thumbs_up.jpgHugh Laurie Photo

http://www.freakingnews.com/pictures/58000/Hugh-Laurie-with-a-Thermometer-58178.jpgBack to The Future

http://www.wallpaperdev.com/imageres/1920x1440-back-to-the-future-wallpaper-hd.jpgSuccess Baby

http://ncclassof2013.files.wordpress.com/2013/04/success.jpgDouble Dutchess (Standard Deviants)

http://www.mtviggy.com/wp-content/uploads/2013/04/double_dutchess.jpgMath is hard

http://www.washingtonpost.com/blogs/answer-sheet/files/2013/03/math.jpgPast Doesn’t Equal The Future

http://sphotos-b.xx.fbcdn.net/hphotos-ash4/402800_10151021129543907_1273189465_n.jpgYou’re not hearing me

http://farm3.staticflickr.com/2670/3948482669_93d49c1dc8_o.jpgAngry Tony

http://marcianitosverdes.haaan.com/wp-content/uploads/2012/07/TonyRobbins_thumb.jpgHolding you back

http://sphotos-a.xx.fbcdn.net/hphotos-ash4/p480x480/382197_426710810746267_1475892304_n.jpg