Steve Lawrence - Agile Metrics

transcript

Agile MetricsSteve LawrenceSlice Consulting Pty Ltd

Your pic

flickr.com/angelabethell/

STEVELAWRENCE

@goriansteve

A Metrics talk by Steve Lawrence

“ “When done correctly; an economic

framework will shine a bright light into all the dark corners of product

development

Don Reinertsen, The Principles of Product Development FlowVapor trail

Establish an Economic Framework

Our overall goal is to influence economic outcomes

Our most important decisions involve tradeoffs between multiple measures of performance

Why do we want to capture metrics

What is the purpose

How will they be used

Who are they for

Why Metrics?To support decision making processes

Measure Value (Product) or Process

To affirm and reinforce Lean and Agile principles

To measure outcomes

To follow trends, not numbers

Reveal, rather than conceal, context and significant variables

Provide fuel for meaningful conversation

At the current churn rate, 75% of the S&P will be replaced by 2027

If a measurement happens at all, it is because it must have some conceivable effect on decision and behavior. If we can't identify what decisions could be affected by a proposed measurement and how that measurement could change them, then the measurement simply has no value.

“How to Measure Anything” By Douglas W. Hubbard

““

What For?

Support organisational objectives

Optimise learning

Should be simple, big & visible

Well-understood and easily adopted

Guide actions and decisions

“Escape Velocity” By Geoffrey Moore

Bad Metrics

Collected because they always have been

Encourage gaming of the system

Result in bad behavior

Ignore the system

Don’t align with the Why!!

@goriansteve

1. Using metrics as levers2. Using a convenient metric (rather than one

that provides critical insight)

3. Bad analysis4. Motivating people to hide

information5. Too costly measures6. Too many measures (information overload)

7. Too few measures (unbalanced)

The Seven Deadly Sins of Agile Measurement

Larry Maccherone, Rally Software

Types of Metrics

Internal vs. External

Qualitative vs. Quantitative

System Efficiency vs. Local Efficiency

Hypotheses, Experiments, and Little Bets

Some Common Measures

estimatingwe are not so good at

Remember

comparisonwe are good at

cstrate

Productivity Metrics

Value Points Delivered per Time Period

Cost per Value Point

Concept To Cash

Revenue per Employee

Lead Time per Story

Mean Time to Ticket Resolution

SLA Achievement Metrics

Velocity

Stable Teams Creates an almost 2:1 difference in throughput in Teams that are 95% or more dedicated compared to teams that are less than 50% dedicated.

AND60% better productivity40% better predictability60% better responsiveness.

Predictability

Say / Do Ratio

Velocity Variance

Cycle Time per Story Point

Feature Comparison

Epic Comparison

flickr.com/uggboy

Customer Satisfaction

Net Promoter Score Trending

Customer Satisfaction Trending

Repeats / Renewals

flickr.com/amne//

Quality

Maintenance Complexity

Trending

Defect Density

Issue Re-introduction Rate

Defect Arrival / Kill Rate

Unit Test Coverage

Auto Functional Test Coverage

flickr.com/ivyfield/

Employee Satisfaction

Employee Satisfaction Trending (surveys)

Attrition Rates

Pain Scale

Retrospective Outcomes

Responsiveness

Cycle Time per Story

Lead Time per Story

Queue / Batch Size

Average Impediment Lifetime

Mean Time to Release

Mean Time to Fixflickr.com/y500

By themselves Metrics don’t tell the story. Trends Do!!

Velocity - a guide not a target

How Velocity Works

Whoa - What does your velocity look like over the last 3 iterations?

Well over the last 3 iterations we completed 45, 80 and 70 story points.

This iteration we completed 90 story points and next iteration we will do 160

Ok, so what has changed in your team or your work that makes you think you can achieve 160 story points?

Nothing, but to satisfy our customer we have to.

Velocity

poor cadence, unpredictable

Velocity

better cadence, more predictable

Program Velocity?

23 points per sprint

235 points per sprint

Team A

Team B

Examples of BadnessI want the ‘Blue’ team to work on my projects because their velocity is higher

I want to compare the output of people in the Team

We committed to 120 points and completed them all and carried forward 30?

As a manager I have to constantly drive my teams to ensure they meet the goals ‘we’ set

Burndown Charts

@goriansteve #agileaus

Manage Flow - Watch your WIP

Lean Start Up: Experiments

“Lean Start Up” By Eric Ries

Proposed Solution

The Lean Canvas 30-Jun-2014Iteration #1

Impact Mapping Communications Plan

Problem Conceptual Solution

NotesUnique Value PropositionSingle, clear, compelling message that states why the solution will be different and worth experimentation

Metrics Outcomes

This is a sample text, insert your

own text,

This is a sample text, insert your own

Lean Canvas is adapted from The Business Model Canvas (http://www.businessmodelgeneration.com) and is licensed under the Creative Commons Attribution-Share Alike 3.0 Un-ported License.

PRODUCT

Sample text

Sample Text

FeatureSample

Sample Text

FeatureDelta

Anticipated Improvement

Sample Text

Sample text

Sample Text

Deploy / Pivot / RIP

Top features

own text,

TasksTasks

Feature

TasksTasks

Expected Outcome

Relative Size S/M/L

Learnings

WSJF Prioritising

user value + time value + RR | OE

job size

“Images used with permission of Scaled Agile, Inc. See ScaledAgileFramework.com for more information

Feature User Business

Time Criticality

Risk Reduction/Opportuni

ty Enablement

Cost of Delay

Job Size

Presentation5 8 6 19 10 1.90

Sales Proposal 10 10 9 29 7 4.14

Remote Training

feedback 3 3 3 9 4 2.25

My Example

In our experience no single sensitivity is more eye-opening than cost of delay

Don Reinertsen, The Principles of Product Development Flow

Cost of Delay

AGILE EVM

Measures Schedule and Performance – not Value

Forecasts in financial units

Expects everything to be defined up front

No Assertion of Quality

PV, EV, CPI, SPI, ETC, EAC

Capacity Planning as a Metric

Waiting Times more than double as utilisation moves from 80% to 90% and double again as it moves from 90% to 95%

Control Queues Not Capacity Utilisation

Waiting on SME’s Longer Cycle Times

Waiting on Sign Offs Increased Risk

Management Reviews More Variability

Big Upfront Analysis More Overhead

Waiting on Releases Lower Quality

Waiting More Administration

Waiting Less Motivation

Waiting Flow On effects

Why do we want to capture metrics What decisions or behaviours do you wish to impact

How will they be usedLever or to facilitate feedback

Who will use the metricsAre the metrics fit for purpose

@goriansteve

Why do we want to capture metrics

What is the purpose

How will they be used

Who are they for

Thank You!

steve@sliceconsult.co.nz

@goriansteve

0404523668

Defect Density

My Hypothesis = 95-100%

Draw 2 parallel lines

At one end of the parallel lines, draw a line at right

angles to them

At the other end of the parallel lines, draw an

inverted V

At one end of the inverted V, draw 2 parallel lines

On the end of the parallel lines you just drew, draw a line at right angles to them

Customer collaboration over contract negotiation

Individuals and interactions over processes and tools

Better Outcomes

Steve LawrenceSlice Consulting Pty Ltd

Your pic

Thanks for listening…