Experimental Study Using Functional Size Measurement in Building Estimation Models for Software...

SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010

Nelly Condori-Fernandez, Luigi Buglione, Maya Daneva, Olga Ormandjieva

1

http://users.encs.concordia.ca/~sera2010/


Goals of the presentationGoals of the presentation: G1. Discuss the estimation process in a software project, moving from initial requirements and their inner nature G2. Propose a possible hybrid approach for improving such process, mixing two different viewpoints on software, looking both at the project as well as the product entities G3. Measure in a controlled case study the effectiveness of such approach, noting possible issues for next improvements

2


◆ Software Requirements FR vs NFR NFR: Non-Functional Requirements

◆ Sizing & Estimating Why the need for sizing requirements? The cone of uncertainty Possible approaches

Project management approach Project Size Unit (PSU) Product functional approach FSMM COSMIC (ISO/IEC 19761)

◆ A possible approach Predicting CFP with PSU Scale types Refined relationships between CFP and PSU

◆ Results Context and sample projects Student’s Project Historical Data

◆ Conclusions & Prospects

AgendaAgenda

3


Software Requirements Software Requirements FR vs NFR

• Q: what is a requirement?

A: “A software capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documentation” [Leffingwell & Widrig, 2003]

• General types of requirements:

• Functional Requirements (FR)

...For each shot the system shall notify the players whether the shot was a hit or miss...

• Non- Functional Reqs (NFR)

..The response time shall be no more than 1 seconds for 95% of responses and no more than 2 seconds for the remaining responses...

4


Software Requirements Software Requirements NFR: Non-Functional Requirement

• In literature, NFRs are referred to as… -ilities Constraints Quality attributes Quality of service requirement More?

• IEEE-STD 830-1998 defines NFR as… …“software requirement that describes not what the software will do, but how the software will do it…” More definitions?

• Nature of NFRs… Subjective: viewed and interpreted differently by different people Relative: interpretation and importance vary depending on the considered system Interacting: attempts to achieve one NFR can hurt or help achievement of other Global and scattered: one NFR affects multiple functionalities, or the whole system More?

5








AgendaAgenda

6


Sizing & Estimating Sizing & Estimating Why the need for sizing FRs and NFRs?

• Effort is a function of Size • Most effort estimation models use size as input for cost estimation

Most widely used metric of the size of a finished system is source lines of code (SLOC), delivered source instructions (DSI) Two metrics of size applicable from the requirements specification phase are COSMIC Function Points (CFP) and Project Size Unit (PSU)

• The cone of uncertainty

Source: Boehm B., Software Engineering Economics, Prentice-Hall, 1981

7


Resources ProductProcess

Project

Organization/ SBU

Measurement

fsu (e.g. UFP, CFP, …)

Sizing & Estimating Sizing & Estimating Entities to be measured: STAR taxonomy

8


1980 1985 1990 1995 2000

Allan Albrecht FPA

IFPUG 4.0

IFPUG 4.1

MkII FPA

MkII FPA 1.3

NESMA

COSMIC-FFP – COSMIC-FFP – ISO/IEC ISO/IEC 1976119761

Sizing & Estimating Sizing & Estimating History of Functional Size Measurement Methods (FSMM)

Note: recently also FISMA FPA become an ISO International Standard (IS 29881:2008)

9


or

Engineered

Devices

SOFTWARE

USERS

Stor

age

Har

dwar

e

« Frontend »

« Backend »

ENTRIES

EXITS

EXITS

ENTRIES

BOUN

DARY

STORE PERSISTENT DATA(‘WRITE’)

RETRIEVE PERSISTENT DATA(‘READ’)

DATA MANIPULATIONOR TRANSFORMATION

Sizing & Estimating Sizing & Estimating COSMIC Measurement Method

10


Sizing & Estimating Sizing & Estimating Some thoughts on measurable entities…

…an 'application' (software) is not the project, therefore it cannot be represented in size terms only by a product metric such as FP (generically, as fsu). …the 'container' (the project) is larger than its 'content' (the product). …therefore, how could it be possible to size the bigger entity by the littler one and therefore to make all the subsequent technical and economical assumptions on such size unit?

Container (Project)

Content (Product)

…a possible answer could be to consider at the same time (or at least, in different moments during the SLC lifetime) sizing measures for different entities (project, product)

11


• Rationale:Rationale: is the productivity ratio (as now applied) meaningful or not? Overall productivity is underestimated (no “Quality | Technical points”) on

the upper part of the formula to counterbalance the overall project effort on the lower part

Sizing & Estimating Sizing & Estimating Some thoughts on measurable entities…is something missing?

Project Size: the size of a software project, derived by quantifying the (implicit/explicit) user requirements refereable to the scope of the project itself

12


Sizing & Estimating Sizing & Estimating Project Size Unit (PSU)

∑ ∑= =

=TQMi

jiLMHj

weighttaskPSU,, ,,

*

• Project Size Unit (PSU) Origin: created in 2003, it’s a PM-based technique for taking care of all

possible user requirements – no matter the type – within the project framework

Goal: to create a virtual sizing unit for the whole project Logical entity to count: WBS tasks, classified by several criteria Weights: complexity weights by effort ranges (periodically revised) Typical usage: internal (process improvement), but also external when the

weighting system is stable among stakeholders (benchmarking) Input: the IFPUG UFP formula URL: www.semq.eu/leng/sizestpsu.htm

• Application Scope Projects, not only those for software ones (NewDev, Enh) Project: “a temporary endeavor undertaken to create a

unique product, service, or result” (PMBOK2008, Glossary)

13

http://www.semq.eu/leng/sizestpsu.htm

http://www.geocities.com/lbu_measure/psu/psu.htm

http://www.semq.eu/leng/sizestpsu.htm


UR

1. Define HLR and refine them into RHLR

2. Translate RHLRs into WBS’s tasks, assigning an effort

3. Classifying tasks per type (M/Q/T), SLC phase and complexity

Sizing & Estimating Sizing & Estimating Project Size Unit (PSU): an example

14



4. Count tasks frequencies by type, SLC phase and complexity

5. Count PSU

6. Compute effort distribution by task type (M/Q/T) and SLC phase

Sizing & Estimating Sizing & Estimating Project Size Unit (PSU): an example

15



Sizing & Estimating Sizing & Estimating Sizing measures and possible gathering moments in the SLC

16








AgendaAgenda

17


• Early Size (in CFP) prediction of FR and NFR determines the likely future values of product size based on existing PSU measure of the same product

• Advantages1. allow for accurate size prediction of all FR and NFR, including those which

are not (yet) stated in measurable terms2. reduces the size measurement effort at this early stage

A possible approach A possible approach Predicting CFP with PSU

• Theoretical aspect1. Analyze the scale types of PSU and CFP2. Given the lowest scale type, identify the corresponding to it type of

admissible transformation (relation) between both units of measurement• Empirical aspect

1. Use available project data on PSU, CFP and Effort2. Derive the prediction formula

18


A possible approach A possible approach Scale type and admissible transformations

Source: Fenton N. & Pfleeger S.L., Software Metrics: A Rigorous and Practical Approach, 2° ed., Course Tech., 1998, ISBN 978-0534954253

Scale type Admissible transformations Examples

Nominal 1:1 mapping from M to M’ Labeling, classifying entities

Ordinal Monotonic increasing function

from M to M’, that is, M(x) ≥ M(y) implies M’(x) ≥ M’(y)

Preference, hardness, air quality, intelligence test (raw scores)

Interval M’=aM + b(a>0) Relative time, temperature (Fahreneit, Celsius), intelligence tests

(standardized scores)

Ratio M’=aM (a>0) Time interval, lenght, temperature (Kelvin)

Absolute M’=M Counting entities

19


• PSU: at least interval scale type respects the additive property

• CFP: at least ratio scale Ratio of two values is meaningful

• Both are valid size measurement units theoretically there should exist an admissible transformation of

type

A possible approach A possible approach Analysis of scale types of PSU and CFP

)0(* >+= kbCFPkPSU

20


1*1 bCFPkPSU ff += 2*2 bCFPkPSU nffn +=

A possible approach A possible approach Refined relationships between CFP and PSU

• PSU and CFP respect the additive property

• Relationships between PSU and CFP

nfPSUPSUPSU f += nfCFPCFPCFP f +=

21








AgendaAgenda

22

SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 23

First Results First Results Experimental design: Research Questions and Variables

RQ1: In what extent does the product non-functional size determine the project size?

RQ2: In what extent does the product functional size determine the project size?

Type of Variables

Response variable (Dependent)

Product size (calculated by the students)Project size (calculated by an expert)

Factor (Independent) Project size measurement method: PSUProduct size measurement method: COSMIC and COSMIC-NFSM

Parameters - Application domain (web-application domain), - Experience using size measurement methods,- Quality of requirements specification.

1. Null hypothesis, H10. Project Functional Size cannot be estimated from Product Functional Size.

2. Null hypothesis, H20. Project Non-Functional Size cannot be estimated from Product Non-Functional Size.

SERA 2010 - Condori-Fernandez, Buglione, Daneva, Ormanjieva © 2010 24

Group 1 Group2 Group 11 Group 3

Requirement , Design and Analysis

Work - Breakdown Structure Expert

Applying CFP and NFSM

Documentation Measurement

Applying PSU

First Results First Results Experimental Procedure

55 third-year students enrolled at Concordia University (Montreal,

Canada).

The experiment was organised as mandatory part of the “Software Measurement” course


First Results First Results Context and sample projects

• Measurement Process: Moving by the same problem statement, each team had to:

1. estimate the effort in man/hours on the info available at the feasibility stage2. calculate the two sizes (CFP, PSU) 3. realizing the web project4. determine the actual effort at the project closure stage5. calculate the two final sizes (CFP, PSU)

• Size Units adopted: COSMIC Measurement Manual, v3.0

Layers: Application layer Perspective used: end user

PSU Measurement Manual, v1.21 4 complexity effort ranges based on projects’ data H/MH/ML/L

25


First results First results Students’ Project Historical Data

26


First results First results Students’ Project Historical Data & Regression analysis

897.29*003.1 −= FURFUR CFPPSU

27

Hypothesis H10 is not rejected:

Null significance, p= 0.24

However, excluding two data points from the analysis:


First results First results Students’ Project Historical Data & Regression analysis

04.29*61.4 += NFRCFPPSUNFR

28

Hypothesis H20 is rejected:

Medium significance, p= 0.033








AgendaAgenda

29


• Main Issue/Goal observed Improve predicatibility of project effort estimation as earlier as possible

by obtaining the lowest ARE/MREas possible• State-of-the-art

Typical usage of single sizing & estimation methods (e.g. expert-based, analogy, parametric-based), covering a single perspective per time

• Possible solution Use at least two sizing methods according to their pros&cons during the

whole SLC phases• Challenge…

Predicting CFP of FUR and (more importantly) NFR moving from project scope knowledge captured in PSU estimates

• Possible advantages Early productivity analysis from the predicted CFP size Such solution can be automated within a PM tool (e.g. MS-Project)

• Next actions/Prospects A wider application of such approach on a larger number of projects, in

order to validate it and stress eventual weakenesses

Conclusions Conclusions && Prospects Prospects

30


Thanks for your attention Thanks for your attention !!

31


Nelly Condori-Fernandez <[email protected]>

Luigi Buglione: <[email protected]>

Maya Daneva: <[email protected]>

Olga Ormandjieva: <[email protected]>

32

Date post:	03-Nov-2014
Category:	Documents
Upload:	luigi-buglione
View:	711 times
Download:	0 times

Experimental Study Using Functional Size Measurement in Building Estimation Models for Software...

Documents