23
Harold van Heeringen Benchmarking agile software development
Harold van Heeringen
Benchmarking agile software development
2Drs. Harold van Heeringen
Graduated in Business economics at the University of Groningen in 1997
>20 years experience in IT, >15 years in software measurement and metrics
Married, 3 kids, living in Veendam (North of the Netherlands)
Hobbies – Chess, soccer and software metrics:
METRI – Consultant Estimation & Performance Measurement
NESMA – Board member International cooperation and partnerships
ISBSG – Immediate Past President (2011-2019)
COSMIC – Dutch representative in the International Advisory Council (IAC)
ICEAA – Trainer of CEBoK chapter 12: Software Cost Estimation
sCEBoK – initiator and module developer
Introducing me
@haroldveendam
haroldveendam
haroldvanheeringen
METRI: www.metrigroup.comISBSG: www.isbsg.orgNesma: www.nesma.org
3
METRI proposition
4
Low industry Performance Measurement maturity
5
High Performance Teams – really?
-80%
-60%
-40%
-20%
0%
20%
40%
60%
80%
100%
Low Performing team
Industry Average
High Performing team
6
Agile Hypecycle
2019
7
See the big picture
8
The industry practice: story points
9• Productivity Effort hours spent
• Cost Efficiency Team cost
• Velocity Duration (months)
• Product Quality Defects Delivered
• Code Quality Metrics Maintainability ReliabilityPerformanceSecurityTechnical Debt
Key metrics for Agile teams
Size of the delivered Software Product
Size of the delivered Software Product
Size of the delivered Software Product
Size of the delivered Software Product
10
Function Points?
When Agile Teams Think about Function Points
They Imagine This…..
Its ugly, old, and evil
It may even eat children!
Something we did in the 80’s, and even then it always failed!
But… we also used the meter, the liter, the kilo in the 80’s
And still do…
Function Points measure functionality regardless
• The Technical implementation (e.g. programming language)
• The Implementation method (e.g. Agile)
• Other non-functional requirements (e.g. availability)
11•They see overdocumentation
Why not?
12•They see over-waterfall
Why not?
13•They see a management tool
Why not?
14
• Implement functional sizing without bothering the teams!
• similar to IFPUG and Nesma FP• ISO standard - ISO 19515:2019• OMG/CISQ Standard
• Implement in the CI/CD pipeline of Agile teams• No waste for the teams, while delivering value for management
Automated Function Points
15 Management Decisions
TEAM
Stakeholders
Senior management is responsible and accountable
TEAM
SP metrics
FP metrics
Type of Decision Measurement Responsibility
Determine backlog
priority Story Points Product owner
Sprint backlog items Story Points
Team / product
owner
Check progress SBI's Story Points Scrum master
Type of Decision Measurement Responsibility
Team size estimation Function Points Management
Performance
measurement Function Points Management
Long term estimation Function Points Management
Benchmarking Function Points Management
Budgetting Function Points Management
16
• Agile Team Performance Measurement, based on standards• Trends through time • High performance teams vs. Low performance teams – learn and improve!• Benchmark! METRI Data engine or ISBSG D&E data repository
• But what about the quality of the product?
Dashboard
0
2
4
6
8
10
12
14
16
18
Productivity (h/FP)
Productivity (h/EFP) Benchmark
€ -
€ 100
€ 200
€ 300
€ 400
€ 500
€ 600
€ 700
€ 800
Cost Efficiency (h/FP)
Cost Efficiency (€/EFP) Benchmark
0
1
2
3
4
5
Project Quality(Defects/FP)
Project Quality (Defects/EFP)
Benchmark
0
50
100
150
200
250
Velocity (FP/month)
Speed (EFP/month) Benchmark
17
Product Metrics –static code analysis
18
The total picture
Risk of critical
failures in
production
Risk of
performance /
scaling issues
Risk of security
breaches
Ease and speed
of modifying
Ease and speed
of learning
RISK MAINTAINABILITY
PRODUCTIVITY
Overall
application
functional size
Performance and
productivity
ISO Standards for
automated function points
BENCHMARK
19
Practical case
0,81
1,121,16
1,23
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
Productivity Index
Productivity Index
0,82
1,06
1,12
1,19
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
Cost Index
Cost Index
0,88
1,19
1,241,21
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
Quality Index
Quality Index
0,91
1,03
1,08
1,15
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
Velocity Index
Velocity Index
Market
average
20
Product Quality Metrics
2,672,82
2,97 3,01
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Total Quality Index (TQI)
Total Quality Index (TQI)
2,84 2,85 2,88 2,91
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
Robustness
Robustness
3,15 3,17 3,17 3,18
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
Efficiency
Efficiency
2,15
2,672,88
3,12
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
Security
Security
2,862,98 3,01
3,12
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Changeability
Changeability
3,15 3,15 3,10 3,09
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Transferability
Transferability
21
Conclusions
Understand the difference in metrics and the use of metrics
• Team metrics vs Management metrics
• Story Point metrics vs. Function Point metrics
Don’t use Story point metrics for management decision making.
Don’t use Function Point metrics in the agile team, unless the team sees the value and wants to use them.
Implement manual or automatic functional size measurement without bothering the teams
Quality is part of the productivity!
The goal is not to punish, but always to improve!
22
A bright future!
Standardized performance metrics based on FPStandardized product metrics2019: Story Points
Disclaimer2018 METRI. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system,
transmitted or made public, in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise, without the prior
written consent of METRI.
Disclaimer2019 METRI. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system,
transmitted or made public, in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise, without the prior
written consent of METRI.
