Using the ISBSG data to improve your organization success - van Heeringen (Metrikon 2013)

transcript

Harold van Heeringen,

ISBSG president

� We measured 350 FP, so... What is the estimate?

� � Effort, duration, quality, risk?

� � Price per FP?

� Why are we losing most of the bids we do?

� Competitive advantage? Or not?

Size does matter, but….

� Competitive advantage? Or not?

� What is our productivity compared to the industry?

� We are going to send out a request for bid based on price/FP

� How can we compare the proposals?

� How can we exclude unrealistic (overoptimistic) ones?

� What should the market average bid approximately look like?

� I did a project and the client thinks it was too expensive

� How did I perform against similar projects?

Historical data of completed projects!!

The answer…

Historical data of completed projects!!

� International Software Benchmarking Standards Group

� Independent and not-for-profit

� Full Members are non-profit organizations, like DASMA, IFPUG, FiSMA, QESP and NESMA

Historical data: ISBSG repositories

� Grows and exploits two repositories of software data:

� New development projects and enhancements (> 6000 projects)

� Maintenance and support (> 1000 applications)

� Everybody can submit project data

� DCQ on the site

� Anonymous

� Free benchmark report in return

� Mission: “To improve the management of IT resources by both business and government, through the provision and exploitation of public repositories of software engineering knowledge that are standardized, verified, recent and representative of current technologies”.

� All ISBSG data is � validated and rated in accordance with its quality guidelines

� current

� representative of the industry

� independent and trusted

� captured from a range of organization sizes and industries

Website and portal

� ISBSG wishes to help restore the confidence in IT, by giving transparency of what is realistic

� Improved project success rates

� Better customer satisfaction

ISBSG � IT Confidence

� Better customer satisfaction

� Shorter time to market

� Better alignment with the business/customer

� Easier contracting, resulting in better relationships between customer and supplier

3 Cases from my experience

� Case 1: Telecom project reality check on the expert estimate

� Case 2: Assessing the competitive position of an organization

� Case 3: Supplier Performance Measurement

Case 1� A telecom company wished to develop a new Java

system for the maintenance of subscription types;

� A team of experts studied the requirements documents and filled in the WBS-based estimation documents and filled in the WBS-based estimation calculation (bottom-up estimate);

� They decide that an estimate of 5.500 hours and a duration of 6 months should be feasible;

� The project manager decided not to believe the experts ‘on their blue eyes’ only and wished to carry out a reality check.

ISBSG Reality Check: Effort� An estimated FPA comes up with the expected size:

� Min: 550 FP, likely 850 FP, Max 1300 FP

� Selecting the most relevant projects in the ISBSG D&E repository show the next results:

Functional SizeFunctional Sizeshow the next results:

PDR (h/FP)

Min. 3,2

Percentile 10% 4,3

Percentile 25% 6,2

Median 8,9

Percentile 75% 12,9

Percentile 90% 19,8

Max. 34,2

PDR (h/FP)

Min. 3,2

Percentile 10% 4,3

Percentile 25% 6,2

Median 8,9

Percentile 75% 12,9

Percentile 90% 19,8

Max. 34,2

550 850 1300

3.410 5.270 8.060

4.895 7.565 11.570

7.095 10.965 16.770

Functional Size

550 850 1300

3.410 5.270 8.060

4.895 7.565 11.570

7.095 10.965 16.770

Functional Size

5.500 hours seems optimistic

ISBSG Reality Check: Duration� Same analysis is possible

� Also, formulas have been published in the Practical Project Estimation book

� For instance:� For instance:

table C-2.2 Project Duration, estimated from software size only

Functionele omvang 550 FP

C uit tabel 0,507

E1 uit tabel 0,429

Duration = C * Size^E1

Duration = 7,6 elapsed months

550 850 1300

Duration 7,6 9,2 11,0

Result� Expert estimate was assessed optimistic

� Adjusted Estimate:

� Effort: 8.000 hours

Duration: 10 months� Duration: 10 months

� This turned out to be quite accurate!

� The project manager now always carries out a reality check and is ‘spreading the word’.

Case 2� Senior management of a software company wondered how

competitive they were when it comes to productivity.

� Many bids for projects were lost and they wished to improve, especially their Microsoft .Net department.

� Analysis of the bids by department showed the next figures:

Nr. of bids 23

Average PDR in bid 16,3 h/FP

Average Size (FP) 230 FP

Average teamsize 6 fte

PDR (h/FP)

Min. 3,2

Percentile 10% 3,8

Percentile 25% 5,9

Median 7,6

Percentile 75% 12,9

Percentile 90% 18,9

Max. 34,2

ISBSG data analysis

Case 2� Analysis of the bid phase showed a number of issues:

� Estimates were extremely pessimistic due to severe penalties in case of overruns;

� In a number of stages, risk surcharges were added;� In a number of stages, risk surcharges were added;

� They wished to work in fixed team of 6 fte, but ISBSG data shows that the project size was usually to small for this teams size to be efficient;

� Because of the knowledge that the department bids were not market average (or better), the bid process was redesigned, making the company more successful!

Case 3� An organization outsourced all of their IT work to one

supplier;

� Therefore, the competition was gone and potentially the supplier could charge whatever they wish;the supplier could charge whatever they wish;

� The ISBSG data was used to construct the productivity targets for the supplier;

� Bonus/malus arrangements were agreed upon based on these targets.

Case 3: Supplier targets

Target PDR PDR (h/FP)

Target PCR PCR (EUR/FP)

jan-13 feb-13 mrt-13 apr-13 mei-13 jun-13 jul-13 aug-13 sep-13 okt-13

Target Defects/1000 FP Defects/1000 FP

The supplier is measured continuously and still has to make his target for the first time!

The organization is happy that the trends show improvement and they feel in control.

Other useful purposes� Analyze the difference in productivity or quality

between two (or more) types of projects:

� Traditional vs. Agile

� Outsourced vs. In-house� Outsourced vs. In-house

� Government vs. Non-government

� One site, multi site

� Reuse vs. no reuse

� � ISBSG Special Analysis reports

Traditional vs. Agile� Agile productivity: 10 – 20% gain after 1 year of

adoption

� Agile cost: 20-40% lower after 1 year of adoption

� Agile time-to-market: 10-60% less� Agile time-to-market: 10-60% less

� Agile quality (post production defects): 2-8% higher

� �10 Take Aways from Reifer Agile Report (www.isbsg.com)

Special reports� Impact of Software Size on Productivity

� Government and Non-Government Software Project Performance

� ISBSG Software Industry Performance report� ISBSG Software Industry Performance report

� ISBSG The Performance of Business Application, Real-Time and Component Software Projects

� Estimates – How accurate are they?

� Planning Projects – Role Percentages

� Team size impact on productivity

� Manage your M&S environment – what to expect?

� Many more

Country reportWhere is Germany??

Government vs. Non-government

Role percentages

� Everybody wants to use data

� But nobody wants to submit data… Why not?

� Is it hard?

Is there a risk?

We need data!

� Is there a risk?

� Is the reward not big enough?

� Why not try it? You’ll get a free Benchmark report and 100 portal credits in return!!

� Are there any factors preventing you?

� WWW.ISBSG.ORG

Conference offer

DASMA members always get a 10% discount, using the code provided earlier this year.

Dia 26

H.S.1 van Heeringen; 15-11-2013

Thank you!

NEtherlands Software Metrics users Association

To celebrate our anniversary NESMAwill organise the IWSM Mensura

October 6-8, 2014

2014.iwsm-mensura.org

October 6-8, 2014

HistoricalHistorical landmarkslandmarks

Modern Modern landmarkslandmarks

EuropeanEuropean CulturalCultural CapitalCapital 20012001

The city of modern The city of modern architecturearchitecture

LargestLargest soccerstadiumsoccerstadium in the in the NetherlandsNetherlands

WelcomeWelcome toto

RotterdamRotterdam

BusiestBusiest port in Europeport in Europe

2014.iwsm-mensura.org

I hope to see you next year

October 6-8, 2014