Contingency resources are reserves that are set aside to manageand handle the impact of risk events in order to protect projectsfrom producing undesirable results. Project managers estimate,allocate and manage appropriate contingency resources foreffective project management. In addition, contingency resourceshelp project managers efficiently manage organizational resources.Therefore, contingency estimation plays a fundamental role indefining project management plans that produce best projectresults.
Typical parameters involved in software development projectsinclude cost, risk, budget, schedule, quality, and specifications. Thecost is the most critical parameter of software development projects(Pfleeger and Atlee, 2006), which is represented in man-monthsunits of effort. Furthermore, the risk events of software projects canresult in adverse monetary consequences that cause shortfalls in theestimated cost of software projects (DeMarco and Lister, 2003;Kumar, 2002). Therefore, for software projects, the contingency
Please cite this article as: Uzzafer, M., A contingency estimation model for so10.1016/j.ijproman.2012.12.002
ftware
resources are reserves of man-months effort which provide a bufferagainst cost shortfalls due to the impacts of risk events andsafeguards software projects from producing undesirable results(Bannerman, 2008; Janne and Lyytinen, 2000; Kitchenham et al.,2003). The allocation of contingency resources allows softwareorganizations to take calculated risks; hence, the estimatedcontingency resources represent the maximum risk tolerance ofan organization (Cooper et al., 2005; Khamooshi and Cioffi, 2009;Touran, 2003).
The contingency estimation is about effective cost estimation(Alkoffash et al., 2008; Menzies et al., 2006) and risk assessment(Bennatan, 1994; Cooper et al., 1985; Dey et al., 1994; Wysocki,2006). Different levels of cost commitments help to manage theimpact of different sets of risk events; hence, contingencyestimation should be correlated with the estimated cost and riskof software development projects. Therefore, contingency estima-tion involves modeling the cost and risk of software developmentprojects (Tom and Susannah, 1988).
The estimated and allocated contingency resources are notalways fully utilized because some risk events may not occurduring the development of a software project. Therefore, thecontingency resources are best managed and utilized within a
projects, International Journal of Project Management (2013), http://dx.doi.org/
2 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
portfolio of software projects because only a few of the projects inthe portfolio would actually invoke the contingency reserves(Bannerman, 2008; Kitchenham et al., 2003).
Researchers in the field of software engineering have attemptedto estimate the contingency resources needed for software projectsand have proposed various contingency estimation models.However, the main drawback of the existing models for softwaredevelopment projects is their lack of generality (Kellner et al.,1999). Managers of software project seek flexible models that canbe easily implemented with their current project managementpractices. Furthermore, some traditional contingency estimationmodels do not consider the estimated cost whereas others do notconsider the risk; hence, these models generate contingencyestimates that do not correlate with the cost and risk of softwaredevelopment projects.
This research work presents a contingency estimation modelthat is independent of specific software development processes;hence, it offers a flexible solution to software organizations andmanagers of software projects. Furthermore, the model considersthe impact of risk events on the estimated cost; consequently, theproposed model generates contingency estimates that arecorrelated with the cost and risk of software development projects.This research work also focuses on the scalability of themodel to aportfolio of software projects.
Furthermore, the contingency estimates should be sub-additivewhich is essential to optimize a software project consisting ofmultiple development tasks. Similarly, sub-additive estimates areessential to optimize a portfolio of software projects (Uzzafer,2010). Sub-additivity occurs when the estimate of the whole is lessthan or at most equal to the combined estimate of the parts; forexample, for an estimate f of parts A and B, the sub-additivityproperty states that f(A+B)≤ f(A)+ f(B) (Lange, 2003; Uzzafer,2010). The contingency resources for a software project should besub-additive, which explain that the contingency estimates of asoftware project should be less than the sum of the contingencyresources needed for the development of each individual task of thesoftware project. Similarly, a portfolio of software project shouldrequire less contingency resources then the combined contingencyresources of all the projects in the portfolio. Hence, this researchwork aims to ensure that contingency estimates are sub-additive.
The proposed contingency estimation model for softwaredevelopment projects is inspired by a financial contingencyestimation model. Analogies from the financial domain havebeen used in the past for software development projects, see forexample the work of Kitchenham et al. (2003), Boukendour (2005)and Costa et al. (2007). The financial contingency estimationmodels use the financial risk measure (CAS, 2007). The financialrisk measure maps the impact of a set of financial risk events on theprobabilistic model of financial investments; therefore, theestimated contingency protects financial investments against a setof risk events (Acerbi and Drik, 2003; Acerbi et al., 2001; CAS,2007; Morgan/Reuters, 1996; Yamai and Toshinao, 2005).
Section 2 of this paper discusses the traditional contingencyestimation models. The section describes generic (parametric andnon-parametric) probability models of cost and discusses aparametric probabilistic model of cost of software developmentprojects. Furthermore, the section presents different categories of
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
risk and discusses a software risk measure model that measuresthe impact of a set of risk events on the probabilistic model of costof software projects. Section 3 presents the proposed contingencyestimation model and discusses different scenarios of contingen-cy estimations. Section 4 explains the validation procedure of themodel using different model validation techniques. Section 5presents a case-study of the proposed contingency estimationmodel on the data of real software development projects. Theresults of the case-study show that the contingency estimates ofthe model are comparable to the real software developmentexperiences. Section 6 draws some final conclusions.
2. Literature review
2.1. Traditional contingency estimation models
Fairley (1995) defines contingency as a percentile of thedistribution of estimated cost of software projects; however,percentile values of skewed probability distributions are notalways sub-additive (Uzzafer, 2010). Kitchenham and Linkman(1997) suggested the deployment of contingency resources onlyto deal with the uncertainties of software projects; hence, theirmodel does not consider other types of risk. Kansala's (1997)contingency estimation model is based on a single valuequantification of risk; however, single value representations ofrisks and costs are unreliable (Fairley, 1995; Kitchenham andLinkman, 1997; Kitchenham et al., 2003). Briand et al. (1998)mapped the impacts of risk events onto a triangular distributionwhere the mean of the triangular distribution determines thecontingency. The triangular distribution, however, suffers fromsymmetric bias and generates a large mean when the maxima ofthe distribution is large (David, 2008). Gillian and Robert (2001)represented the estimated cost of software projects with Gaussiandistributions, which assumes that the cost estimates on eitherside of the distribution are equally likely. Kitchenham et al.(2003) modeled the contingency resources with a probabilitydistribution based on an insurance premium model. Theysuggested using the contingency resources to handle risk eventsthat are unknown; however, the contingency resources estimatedto handle unknown risk events cannot be justified. Boukendour(2005) used stock option theory to estimate the contingencyresources for software development projects; the model relies onsingle value estimate of cost. Jantzen et al. (2006) modeled theimpact and probability of risk events using a predefined set ofpercentages. This model is therefore limited to a few pre-definedvalues of risk impacts and probabilities.
2.2. Software cost
The cost is the most important consideration in the softwareprojects (Kitchenham et al., 2003; Pfleeger and Atlee, 2006); theestimated cost encompasses all other parameters of softwareprojects. The cost estimationmodels for software projects producessingle point estimates of costs, represented here as E. Lately,researchers used probabilistic models (parameter, non-parameter)to represent the cost of software projects. Kitchenham et al. (2003,2005) used a gamma distribution (parametric model) to represent
projects, International Journal of Project Management (2013), http://dx.doi.org/
3M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
the cost of software projects. Whereas, Fairley (1995) used theConstructive Cost Model (COCOMO) (Boehm, 1981) for costestimation and constructed a non-parametric distribution throughMonte Carlo simulation to represent the cost.
The random values of costs of software development projectsare represented by X which has an underlying parametricprobability distribution [Appendix A.1]. The minimum value ofX is said to be bounded by zero, whereas the maximum value of Xis considered unbounded (Kitchenham and Linkman, 1997;Kitchenham et al., 2003). Furthermore, the expectation of X isrepresented with E X½ �; the expected cost is defined as thebudgeted cost. The maximum cost a software organization iswilling to invest in a software project is the risk tolerance of theorganization and is defined as the 100ath percentile of Xwhich isrepresented as xa. The development of software project requiressome cost (Kitchenham and Linkman, 1997), therefore, theminimum estimated cost for the development of software projectsis defined as the 100bth percentile of X and represented as xb.
Kitchenham et al. (2003, 2005) proposed using gammadistributions, Γ(k,θ), to model the random estimated cost ofsoftware projects such that X~Γ(k,θ) where k and θ are theshape and spread parameters of the gamma distribution, respec-tively. The expectation of the gamma distribution is defined asE X e Γ k; θð Þ½ � ¼ k� θ (Ross, 2007). Uzzafer (submitted forpublication) explained that single value estimates of cost (E) can
Table 1SEI's list of risk events for software projects.
5. Work environma. Quality Attib. Cooperationc. Communicad. Morale
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
be modeled with a gamma distribution by mapping E with theexpectation of the gamma distribution such thatE↦E XeΓ k; θð Þ� �and then approximating the distribution parameters k and θ. Agood approximation of the shape of a gamma distribution can beachieved by setting k=2 and estimating the θ usingE↦E X½ � ¼ kθ(Uzzafer, submitted for publication).
In general, the managers of software development projectsassign costs and probabilities to different risk events, whichgenerate a sequence of cost samples and produce a discretenon-parametric distribution of cost. Consider n samples of costwhich forms a sequence Xi where i is the sample index such that∑k P Xi≤Xkf g ¼ 1 and the mean of the samples is defined asE Xi½ � [Appendix A.2]. The random discrete samples of cost (Xi)can be generated using different techniques. For example, Boehm(1991) explained that the software manager can assign each riskevent a cost estimate with an associated probability based onthe expert opinion. Whereas, Fairley (1995) used Monte Carlosimulations to generate random samples of cost and theirprobabilities.
2.3. Software risk
Researchers have adopted different expressions of risk for theirdomains; in essence, risk is the possibility of loss. The central
4 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
notion of risk is that it is an event whichmay or may not take place;hence, its occurrence is uncertain and it results in adverse monetaryconsequences. Therefore, for software projects, the impact of riskevents adds additional cost to the expected cost (E X½ �) and movesthe expected cost towards higher values of X.
2.3.1. Risk categoriesContingency resources are limited resources; while differ-
ent risk events cause varying degrees of impacts. Therefore,software organizations manage the impact of risk events basedon their capacity of risk tolerance (Khamooshi and Cioffi,2009; Touran, 2003). Some risk events are highly likelyto occur; whereas, certain risk events are equally likely tooccur; while, others are low probability risk events. Uzzafer(submitted for publication) proposed three categories of riskevents for software projects namely: nominal-case, worst-case,and extreme-case risk events.
Software Engineering Institute (SEI) established a list ofpotential risk events of software development projects, shown inTable 1, which is used for the identification of risk during therisk assessment process (Carr et al., 1993; Higuera and Haimes,1996; Williams et al., 1999). The risk events are grouped intothree classes which are further sub-divided into elements andattributes. The risk assessment process identifies the risk eventsand establishes the impacts and probabilities of risk events usingdifferent techniques. Therefore, any risk event of a softwaredevelopment project could be in any of the risk category basedon its impact and probability.
2.3.1.1. Nominal-case risk events. Nominal-case risk events areusually known and are highly likely to occur during thedevelopment lifecycle of software projects (Bannerman, 2008).Hence, the cost impact of nominal-case risk events is representedwithin the expected cost (E X½ �) of software projects.
2.3.1.2. Worst-case risk events. Worst-case risk events areequally likely to occur; these risk events result in monetary
Fig. 1. Estimated cost X w
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
consequences that are within the risk tolerance. Therefore, thecost impacts of such risk events are mapped to random values ofcosts which are between the expected cost and the risk tolerance.
2.3.1.3. Extreme-case risk events. Extreme-case risk events arelow probability events with large amounts of adverse cost impacts.Extreme-case risk events cause cost impacts that push the estimatedcost beyond the risk tolerance and results in unbounded values ofcost.
2.3.2. Software risk measureUzzafer (submitted for publication) presented a model to
measure the risk of software projects that relies on theprobabilistic (parametric, non-parametric) representation ofcost. The software risk measure model estimates the expectationof maximum cost impacts due to ϑ percent worst-case riskevents nearer to risk tolerance. For a parametric model of cost(X), the cost arising from ϑ percent worst-case risk events isdefined to be between the xε (100εth percentile of X) and xa(100ath percentile of X), such that ε=(a−ϑ). Fig. 1 illustratesthe map of ϑ percent worst-case risk events on X along with themapping of risk categories on to X where X is represented using agamma distribution.
For X, the software risk measure is determined by the followingequation (Uzzafer, submitted for publication):
ρϑ XjX b;a½ �� � ¼ 1
a−εð Þ E XI xb≤X≤xaf gI xε≤Xf g� �� � ð1Þ
where I ∙f g is the indicator function, i.e., I conditionf g ¼0;condition¼fasle1;condition¼true
n. Eq. (1) presents the expectation of random costs
between xε and xa.For a non-parametric model ( Xi), the cost impact of ϑ
percent worst-case risk events is defined to be between the costsamples XYε and XYa . Therefore, the software risk measure isthe expectation of cost samples between the samples XYε and
ith map of risk events.
projects, International Journal of Project Management (2013), http://dx.doi.org/
5M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
XYa , which is defined as follows (Uzzafer, submitted forpublication):
ρϑ XijX Yb;Ya½ �� �¼ 1
a−εð ÞhE XiI XYb
≤Xi≤XYaf gI XYε≤Xif gh i
þ XYbb−P Xi≤XYb
� �� �−XYa P Xi≤XYaf g−að ÞÞ
þ XYε ε−P Xi≤XYεf gð Þi
ð2Þ
Fig. 2 illustrates the samples Xi; in-addition it shows thesamples XYb
, XYa and XYε along with their probabilities, i.e.,P Xi≤XYb
� �, P Xi≤XYaf g and P Xi≤XYεf g, respectively.
The software risk measure models, ρϑ(X|X[b,a]) andρϑ XijX Yb;Ya½ �
� �, are generic and does not assume any specific
probabilistic (parametric, non-parametric) model of the cost(Uzzafer, submitted for publication).
3. Software contingency estimation model
3.1. Towards improved contingency estimation
DeMarco and Lister (2003) hypothesize that estimationmodels should be based on project management and development
Fig. 3. Probability dist
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
practices. A survey about the use of estimation models indicatedthat only 14% of organizations use these models (Heemstra,1992; Lederer and Prasad, 1992). One reason for such a lowutilization of estimationmodels in the software industry is that thetraditional models for software development projects are notgeneric and are dependent on specific tools and practices (Kellneret al., 1999). Hence, the adoption of such models requiresadjusting the existing project development tools, methods andpractices which remains a challenge. This lack of generalitydeprives software managers of the flexibility to choose thedevelopment processes and methods that best fit their needs.
Furthermore, some traditional models produce estimates thatare not well correlated with the cost and risk of software projects.In addition, no guidelines have been proposed to scale thesemodels to a portfolio of software projects and the sub-additivityof the estimated contingencies remains questionable.
3.2. Contingency estimations using software risk measure
The proposed contingency estimation model is inspired from afinancial contingency estimation model (CAS, 2007). The lossesof a financial investment are on the left side of the distributionwhere the financial investment reduces to lower values. However,
ribution of cost X.
projects, International Journal of Project Management (2013), http://dx.doi.org/
Fig. 4. Cumulative probability distribution of cost X.
6 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
instead of the losses being the low percentiles of the distribution,the random costs impacts are the high percentiles of the costdistribution of software projects. The impact of risk events movesthe expected cost towards the higher values of costs. For aparametric model of X, the software risk measure estimates theexpected cost due to the maximum impact of ϑ percentworst-case risk events that are nearer the risk tolerance. Therefore,the contingency estimate is defined as the buffer between theexpected cost of the project and the expected cost due to themaximum impacts of risk events, which is defined as follows:
Contingency ¼ ρϑ X X b;a½ ��� �
−E X½ �� ð3Þ
The contingency estimation model, expressed in Eq. (3),produces estimates of man-months reserves to abate themaximum
Fig. 5. Probability distribution o
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
impact of ϑ percent risk events within the defined risk tolerance.For a non-parametric model of Xi, contingency estimation modelin Eq. (3) takes the following form:
Contingency ¼ ρϑ Xi X Yb ;Ya½ ��� �
−E Xi½ �� ð4Þ
The proposed contingency estimation model can be easilyadopted with any project development environment, since themodel is generic for the cases of X and Xi, and thus provides aflexible option to software managers. The model takes a holisticapproach that enables the correlation of contingency resourceswith the estimated cost and risk of software projects. Furthermore,the validation of the model confirms that the model generatessub-additive contingency estimates and the model is scalable to aportfolio of software projects; these properties are discussed inSections 4.2 and 4.3, respectively.
f discrete cost samples, Xi.
projects, International Journal of Project Management (2013), http://dx.doi.org/
RELY Required software reliability 1.15DATA Database size 1.09CPLX Product complexity 1.00TIME Execution time constraint 1.11STOR Main storage constraint 1.06
Fig. 6. Cumulative distribution of discrete cost samples, Xi.
7M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
3.2.1. Example of a parametric caseWhen the distribution of cost, X, is specified; for this case, the
values to xa, xb and ϑ, or alternatively, the values to a, b and ϑ canbe assigned. Then the estimation of a, b and xε from xa, xb and ϑ orthe estimation of xa, xb and xε from a, b and ϑ are straight forwardbecause any 100 q th percentile of X obeys the relationship q=P[X≤xq]=FX(xq) and any random value xq adheres to xq=FX
−1(q).These values are used in Eq. (1) to estimate ρϑ(X|X[b,a]), which isthen used in the contingency estimation model, expressed inEq. (3), for the contingency estimation.
Assume that, for a software project, X is modeled with agamma distribution, i.e., X~Γ(k,θ) having expectation of E X½ � ¼2.5 man-months. Hence, the gamma distribution parameters areestimated as k=2, θ=1.25 such that E X½ � ¼ k� θ ¼ 2:5. Theproject manager assign values of a=0.75, b=0.25 and ε=a−ϑ=0.65; then the estimated values of xa, xb and xε are 3.3658, 1.2016and 2.7736 man-months, respectively. The software risk measuremodel, Eq. (1), then produced ρ0.1(X|X[0.25,0.75])=3.0559 man-months. This explains that the software project having expectedcost of 2.5 man-months can experience a cost of 3.0559man-months due to the impact of worst-case risk events thatare within 10% probability of the risk tolerance of xa= 3.3658man-months. Therefore, the contingency for this software
project is, ρ0:1 XijXi 0:25;0:75½ �
−E X½ � ¼ 3:0559−2:5 ¼ 0:5559
man-months. This explains that a contingency reserve of 0.5559man-months is essential to protect the software project from themaximum impacts of worst-case risk events within the risk
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
tolerance. Figs. 3 and 4 shows the probability density andcumulative probability distribution of X~Γ(k=2,θ=1.25),respectivley.
3.2.2. Example of a non-parametric caseWhen the discrete cost samples,Xi, are specified along with the
associated probabilities; for this discrete case, values are assignedto a, b andϑ and the respective samples are estimated fromXi sinceany sample XYq is at Yq=max k : FXi Xkð Þ≤q� �
. Alternatively, thevalues to XYa , XYb
and XYε are assigned and their associatedprobabilities, P Xi≤XYaf g, P Xi≤XYb
� �and P Xi≤XYεf g, are
estimated. These values are then used in Eq. (2) to measure the riskof the software project; the Eq. (4) then produces the contingencyestimate for the software project.
Consider a software project where the project manager hasestimated the cost samples,Xi, along with their probabilities basedon the expert opinion as shown in Fig. 5. The expectation of theestimated cost samples is 2.5 man-months. Furthermore, theproject manager has estimated the minimum and maximum costs
PVOL Platform volatility 0.87PCON Personnel continuity 0.84ACAP Analyst capability 1.00DOCU Documentation to match lifecycle 1.13PCAP Programmer capability 1.16AEXP Applications experience 1.10SITE Multi-site operations 1.17LTEX Language and tool experience 0.91TOOL Use of software tools 1.00SCED Required development schedule 1.10Product=∏ i(cost Drivers)i 1.8201
projects, International Journal of Project Management (2013), http://dx.doi.org/
8 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
for the software project to be the 25th (b=0.25) and 75th (a=0.75)percentiles of Xi, respectively. The manager wants to abate theimpact of 10% worst-case risk events under 75th percentile, i.e.ε=a−ϑ=0.75−0.1=0.65. Therefore, from the sequence Xi, thevalues of XYa¼0:75 , XYb¼0:25
and XYε¼0:65 are estimated as 3.3635,1.2048 and 2.6104 man-months, respectively, along withtheir respective probabilities which are P Xi≤XYa¼0:75
� �¼ 0:756,
P Xi≤XYε¼0:65
� �¼ 0:6540 and P Xi≤XYb¼0:25
� �¼ 0:2540, respec-
tively. The cumulative probabilities of the discrete samples,Xi, areshown in Fig. 6. Applying the software risk measure model from
Eq. (2) produces ρ0:1 XijXi 0:25;0:75½ �
¼ 3:0120 man-months.
Therefore, expected cost could reach to 3.0120 man-months dueto the impact of worst-case risk events that are within the 10%probability of risk tolerance. Therefore, the software contingency
estimate using Eq. (4) is ρ0:1 XijXi 0:25;0:75½ �
−E X½ � ¼ 3:0120−2:5 ¼ 0:512 man-months. This explains that a contingency of0.512 man-months is needed to protect this software project from
Fig. 8. Cumulative probability distribut
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
the maximum impacts of the worst-case risk events within the risktolerance of 3.3635 man-months.
4. Model validation
Lindland et al. (1994) proposed that a model should befeasibly validated, meaning that the validation goals should beachieved with the specified modeling activities to the point inwhich further modeling activities would not be economical.The goals of the validation of the contingency estimation modelinclude the fidelity, sub-additivity and the scalability tests.The goal of the fidelity validation checks whether the modelproduces the expected output; the goal of the sub-additivityvalidation ensures the sub-additive behavior of the contingencyestimates of the model; while, the goal of the scalability ensuresthat the proposed model is scalable to a portfolio of softwareprojects. In-addition, the validation process ensures that the
ion of X and estimated percentiles.
projects, International Journal of Project Management (2013), http://dx.doi.org/
9M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
model is tested with different cost estimation techniques, i.e.,COCOMO-II and bottom-up cost estimations.
4.1. Fidelity test
To achieve the goal of the fidelity test, a software project isconsidered where the project manager uses the ConstructiveCost Model II (COCOMO-II) for cost estimation which isdefined as follows (Boehm et al., 2000, 2010; Dillibabu andKrishnaiah, 2005):
E ¼ ac � Sbc � EAF; ð5Þ
where E is the estimated cost in man-months, S represents thesoftware size in kilo-lines of the delivered source code (KDOC),EAF is the effort adjustment factor, which is the productof seventeen software cost drivers, bc is the sum of five parameters(Wi) and defined as bc ¼ 1:01þ 0:01�∑5
i¼1Wi, and ac is aconstant set to 2.94.
The software manager estimated the size of the project to be S=8.62 KDOC and selected the values of Wi and EAF, which areshown in Tables 2 and 3, respectively. The values of Wi resultedin bc=1.01+0.01×0.84=1.0184. With these values theCOCOMO-II, expressed in Eq. (5), produces E ¼ 48 man-months of cost.
This estimated cost, E, is mapped to the expectation ofthe gamma distribution such that E↦E XeΓ k; θð Þ� �
and thedistribution parameters are estimated as k=2,θ=24 and E X½ � ¼2� 24 ¼ 48. The project manager decided to model the costimpacts of 10% (ϑ=0.1) worst-case risk events within the risktolerance. Furthermore, project manager estimated the minimum(xb) and maximum (xa) costs to be about 24 man-months and 62.4man-months, respectively. Using these values (xa=62.4, xb=24,ϑ=0.1), the estimated values of a, b and xε are a=0.7326, b=0.2642 and xε=51.34 (ε=a−ϑ=0.7326−0.1=0.6326). Fig. 7and 8 shows the probability density and cumulative probabilitydistribution of X~Γ(k=2,θ=24), respectively. The software riskmeasure model, Eq. (3), then generated ρ0.1(X|X[0.2642, 0.7326])=57.5 man-months. Therefore, the contingency estimate for thissoftware project is ρ0:1 X X 0:2642; 0:7326½ �
�� �−E X½ ��
=57.5−48=9.5 man-months.
The contingency estimation model predicts that 9.5 man-month of resources are needed to abate the risk of maximumcost impacts due to 10% worst-case risk events within the risktolerance. This contingency estimate is approximately 19% ofthe expected cost of 48 man-months of the software project,which is in agreement with a real software developmentexperience, as reported by Fairley (1995), hence, the modelproduces the expected output and fulfils the goal of the fidelityvalidation.
4.2. Sub-additivity test
Consider a software project having estimated cost modeledas X. The software project has n software development taskshaving estimated costs of X1…Xn. Therefore, contingency
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
estimates of the software project are sub-additive if thefollowing expression holds:
ρϑ XjX b;a½ �� �
−E X½ � : bρϑ X1jX1 b;a½ �� �
−E X1½ � þ…
þ ρϑ Xn Xn b;a½ ��� �
−E Xn½ �� ð6Þ
The expression in Eq. (6) is the sub-additive form of theproposed contingency estimation. The estimated costs of Xcould be the cost of a portfolio of software projects and X1…Xncould be the costs of individual projects within the portfolio. Asimilar expression for Xi can be expressed as follows:
ρϑ XijX Yb;Ya½ �� �
−E X½ � : bρϑ X1ijX1 Yb;Ya½ �� �
−E X1i½ � þ…
þ ρϑ Xni Xn Yb;Ya½ ��� �
−E Xni½ �� ð7Þ
The goal of the sub-additivity test is achieved through bottom-up cost estimation of a software project. The project consists oftwo software development tasks with individual estimated costs ofE1 =10 and E2 ¼ 15 man-months. Therefore, using the bottomup cost estimation the overall cost of the software project is thesummation of the costs of each task of the project, i.e. E ¼ E1þE2=10+15=25man-months. These costs are thenmapped to theexpectations of the gamma distributions, such that E1↦E X1½ �,E2↦E X2½ � and E↦E X½ �, where X1 and X2 represents the costdistributions of tasks 1, 2, respectively, and X represents costdistribution of the software project. The distributions parameters(k, θ) are then estimated as E1 e Γ 2;5ð Þ, E2 e Γ 2;7:5ð Þ andE e Γ 2;12:5ð Þ. Furthermore, the values of b and ϑ are set at b=0.25, ϑ=0.1 and the value of a is varied from 0.65 to 0.99 in stepssuch that a=[0.65 0.75 0.85 0.95 0.99].
For this software project, the contingency estimates aresub-additive when the contingency estimate of the project is lessthan the sum of the contingency estimates of each individual taskof the software project; this inequality is expressed in Eq. (8):
ρϑ XjX b;a½ �� �
−E X½ � : bρϑ X1jX1 b;a½ �� �
−E X1½ �þ ρϑ X2jX2 b;a½ �
� �−E X2½ � ð8Þ
Table 4 shows the contingency resources estimated atdifferent levels of risk tolerance. For example, at a risktolerance of a= 65%, the contingency estimates for task 1 andtask 2 areρ0:1 X1 X1 0:25;0:65½ �
�� �−E X1½ ��
=1.22 man-months and
ρ0:1 X2 X2 0:25;0:65½ ��� �
−E X2½ ��=3.48 man-months, respectively,
their sum is therefore ρ0:1 X1 X1 0:25;0:65½ ��� �
−��
E X1½ �Þþρ0:1 X2 X2 0:25;0:65½ �
�� �−E X2½ �� ��
=4.70 man-months. The con-tingency of the software project, however, is estimated to be
projects, International Journal of Project Management (2013), http://dx.doi.org/
10 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
ρ0:1 X X 0:25;0:65½ ��� �
−E X½ � ¼�0.41 man-months for the same
tolerance level of 65%. This shows that the contingencyrequired for the project is less than the combined contingencyrequired for all the tasks of the project. The contingencyestimates for different risk tolerance levels shows thatcontingency estimates are sub-additive. These results provethat the model generates sub-additive estimates of contingencyresources.
4.3. Scalability test
The test for the scalability of the proposed contingencyestimation model is achieved by constructing a portfolio of tensoftware projects having costs ofE1…E10, respectively. Usingthe bottom-up cost estimation model, the estimated cost of theportfolio, E, is the summation of the expected costs of all thesoftware projects in the portfolio, i.e., E ¼ E1þ …þ E10 ¼3693 man-months. The estimated costs of the portfolio ismapped to the expectation of a gamma distribution such that
Table 6Actual, estimated costs and contingency estimates.
Project no. Actual Estimated (E X½ �) ρϑ(X|X[b,a])
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
Ep↦E XpeΓ kp; θp� �� �
. Similarly, the estimated cost of theprojects of portfolio are mapped such that Ej↦E XjeΓ kj; θj
� �� �,
j∈N, respectively, where j is the project number.The smallest and the largest of these, in terms of cost, have
estimated costs of 100 man-months and 595.23 man-months,respectively. Table 5 illustrates the estimated costs and riskmeasures of each software project assuming a=0.75, b=0.25,ϑ=10% and ε=a−ϑ=0.75–0.1=0.65. In-addition, Table 5shows the contingency estimates for each software project. Forexample, the project with an estimated cost of 100 man-monthscan experience random costs from xε=110.88 to xa=133.93man-months due to the impacts of 10% of worst-case riskevents near the risk tolerance of xa, resulting in an expected riskof 122.61 man-months. Therefore, the contingency is estimatedto be 22.61 man-months. The sum of contingency estimates ofall the software projects is 848.37 man-months.
Furthermore, Table 5 shows that the portfolio has anestimated cost of 3693 man-months and the risk tolerance of4962 man-months; therefore, the estimated contingency for theportfolio is 810.7 man-months. These results reveal that the
11M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
estimated contingency of the portfolio is less than the combinedcontingency of all of the software projects of the portfolio, i.e.,
ρϑ Xp Xp b;a½ ����
−E Xp� �
b∑10
j¼1ρϑ Xj Xj b;a½ �
��� −E Xj
� ��. The con-
tingency estimation model produces sub-additive estimates ofcontingency for the portfolio; hence, the scalability validationproves that the contingency estimation model is scalable to aportfolio of software projects.
5. Case-study of a software project
The aim of the case-study is to understand how the proposedcontingency estimation model would perform in a real softwaredevelopment environment. The proposed contingency estimationmodel should produce contingency estimates which are compa-rable with the actual risk experienced during the development ofreal software projects.
5.1. Case design
The data for real software projects are obtained from the studyby Lum et al. (2002). They collected data from 19 softwareprojects of NASA in an effort to understand the differencebetween the estimated costs and actual costs of software projects.These NASA projects used COCOMO (Boehm, 1981) forsoftware cost estimation and include the development of a widerange of software applications under different platforms. Theactual and the estimated costs of the project are shown in Table 6.
Fairley (1995) reported that software development organiza-tions allocate project resources based on the 70th percentile ofthe distribution of cost. Therefore, the software risk measureis estimated at a risk tolerance of the 75th percentile of theestimated cost for 10% of the worst-case risk events. Therefore,the expectation of the cost impacts attributable to risk eventswill have approximately 70% probability of the distribution.The probability of the minimum cost is assumed to be the 25thpercentile of the distribution of the cost.
For this case study, the single-point cost estimates are mappedto the expectations of gamma distributions for cost modeling; thencontingency resources are estimated using the model expressed inEq. (3). In-addition, Table 6 shows the risk measures andestimated contingencies for all the software projects. In-addition,Table 6 shows the percentage of difference between the actual costand the sum of the estimated cost and contingency.
Furthermore, any project that has an actual cost which is ±20% (Fairley, 1995) of the sum of the estimated cost and thecontingency is called a successful project and any project thathas an actual cost that is ±100% of the sum of the estimatedcost and contingency is called an extreme project. Additionally,the remaining projects are called challenged projects.
5.2. Case-study results
The case-study results in Table 6 shows that the actual costof the software project 1 is 446.34 man-months while the sumof the estimated cost and contingency is about 337.44
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
man-months. Therefore, the actual cost is about 32% morethan the estimated cost and contingency combined, whichmakes this project a challenged project. Similarly, the projects3, 5, 8, 9, 11, 12, 13, 17, and 18 are challenged projects.
On the other hand, project 4 has an estimated cost andcontingency of 499.21 and 110.99 man-months, respectively;hence, the actual cost of 592.28 man-months is about 3% lessthan the sum of the estimated cost and contingency of 610.02man-months. This is an example of a successful project;similarly, the projects 7, 10, 14, 15 and 16 are classified as thesuccessful projects.
While, Software project 6 has an estimated cost of 66.74 man-months and the estimated contingency of 14.83 man-months. Theactual cost of this project is 230.48 man-months, which is about182% more than the estimated cost and contingency; hence thisproject is classified as an extreme project. Similarly, the projects 2and 19 show the extreme deviations of more than 100% andhence they are extreme projects.
These NASA software projects were completed before 1994(Madachy et al., 2006). The 1994 chaos report of StandishGroup (Haughey, 2009) showed that the software projectscompleted by 1994, 16% of the software were successful, 53%were challenged and 31% were failed projects. The results ofthis case study reveals that of 19 projects, ten projects (53%)are challenged projects, six projects (31%) are successfulprojects and the remaining three projects (16%) are extremeprojects. Therefore, these results are in agreement with realsoftware development experiences.
6. Conclusions
Contingency estimates are critical for the successful manage-ment and development of software projects since contingencyresources provide a buffer against software development risk.Therefore, accurate contingency estimates would help softwaremanagers to avoid undesirable results when managing softwaredevelopment projects. This paper has presented a contingencyestimation model; the model is generic and independent of the costestimation and risk assessment models. Hence, it provides aflexible choice for software organizations and managers ofsoftware development projects. Furthermore, the model producescontingency estimates that are correlated with the cost and risk ofsoftware projects and considers the risk tolerance of softwareorganization for contingency estimations. In addition, the model isscalable to portfolios of software projects because contingencyreserves are best utilized within a portfolio. The model generatessub-additive contingency estimates, which is essential for manag-ing a portfolio of software projects or managing an individualsoftware project consisting of different development tasks.
The proposed contingency estimation model uses the measureof risk of software development projects. Although, any model tomeasure the risk of software projects can be adopted, the proposedcontingency estimation model is demonstrated with Uzzafer(submitted for publication) software risk measure model. Futureresearch work can focus on adopting different risk measuringtechniques for software development projects. For the validationof the proposed model the estimated cost is modeled with a
projects, International Journal of Project Management (2013), http://dx.doi.org/
12 M. Uzzafer / International Journal of Project Management xx (2013) xxx–xxx
gamma distribution, future research studies can adopt differentprobabilistic (parametric, non-parametric) representations of costof software projects.
The results of the case-study confirm the Standish group'sreport for the software projects completed before 1994. TheStandish group further reported that by 1998 percentage ofsuccessful projects increased to 26, then by 2004 it was 29%(Galorath, 2008) and by 2009 the success in the softwareindustry stood at 32% (Haughey, 2009). One factor whichcontributed to the increase in the number of successful softwareprojects is the improvements in the software cost estimationtechniques. For example, COCOMO-II was introduced in theyear 2000 as a successor of COCOMO. The contingencyestimations benefits from these improvements in the costestimation techniques for software projects.
Appendix A
A.1
The X has a cumulative distribution function, FX(∙), such thatFX(x)=P[X≤ x], where x is a single realization of the randomvariable X. The 100qth percentile of the random estimated cost,X, is defined as q=P[X≤xq]=FX(xq), where q∈ (0, 1] and xq isthe inverse function of FX(x) such that FX
−1(q)= xq (Papoulis,1991). The expectation of X is estimated from E X½ � ¼ ∫xf X xð Þ,where fX(x) is the probability distribution of X (Papoulis, 1991).
A.2
For discrete samples, Xi, the probability of the kth sample,FXi Xkð Þ ¼ P Xi≤Xkf g, suggests thatXi≤Xk if and only if at leastk of the n random samples of the sequence are less than or equal toXi (Montgomery and Runger, 2007; Ross, 2007). Any 100qthpercentile ofXi is defined as the sample whereP Xi≤Xkf g ¼ q. Arandom variable Yq=max k : FXi Xkð Þ≤q
� �counts the number of
samples of the sequence that are less than or equal to the 100q thpercentile of Xi (Montgomery and Runger, 2007; Ross, 2007).Therefore, the sample XYa of the sequence Xi is the sample forwhich FXi Xkð Þ≤a, where a is the probability of the 100 athpercentile of Xi. Similarly, the samples XYb
and XYε are thesamples for which FXi Xkð Þ≤b and FXi Xkð Þ≤ε, respectively,where b and ε are the 100 bth and 100 εth percentiles of Xi. Theexpectation of samples, Xi, is defined as E Xi½ � ¼ ∑kXkf Xi
Xkð Þ(Papoulis, 1991).
References
Acerbi, C., Drik, T., 2003. Expected shortfall: a natural coherent alternative tovalue at risk. Economic Notes 31 (2), 379–388.Acerbi, C., Nordio, C.,Sirtori, C., 2001. Expected shortfall as a tool for financial risk management.(online: http://ideas.repec.org/p/arx/papers/cond-mat-0102304.html (AccessedOctober 2010)).
Alkoffash, M.S., Bawaneh, M.J. Al, Rabea, A.I., 2008. Which software costmodel to choose in a particular project. Journal of Computer Science 4 (7),606–612.
Bannerman, P., 2008. Risk and risk management in software projects: areassessment. The Journal of System and Software 81, 2118–2133.
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
Boukendour, S., 2005. Estimating software cost contingency using optionstheory. International Conference on Information Technology: Coding andComputing (ITCC 2005). Las Vegas, Nevada, pp. 825–828.
Briand, L.C., El Emam, K., Bomarius, F., 1998. COBRA: a hybrid method forsoftware cost estimation, benchmarking and risk assessment. InternationalConference on Software Engineering. Kyoto, Japan, pp. 390–399.
Carr, M.J., Konda, S.L., Monarch, I., Walker, C.F., Ulrich, F.C., 1993.Taxonomy Based Risk Identification, CMU/SEI-93-TR-006. SoftwareEngineering Institute, Pittsburgh, U.S.A.
Casualty Actuarial Society (CAS), 2007. Dynamic Risk Modeling Handbook.(online: http://www.casact.org/research/drm/(Accessed October 2010)).
Cooper, D.F., MacDonald, D.H., Chapman, C.B., 1985. Risk analysis of aconstruction cost estimate. International Journal of Project Management 3 (3),141–149.
Costa, H.R., Barros, M.O., Travassos, G.H., 2007. Evaluating software projectportfolio risks. Journal of Systems and Software 80 (1), 16–31.
David, V., 2008. Risk Analysis: A Quantitative Guide, 3rd edition. Weily.DeMarco, T., Lister, T., 2003. Waltzing With Bears: Managing Risk on
Software Projects. Dorset House Publishing Company.Dillibabu, R., Krishnaiah, K., 2005. Cost estimation of a software product using
COCOMO II. 2000 model—a case study. International Journal of ProjectManagement 23 (4), 297–307.
Dey, P., Tabucanon, M., Ogunlana, S., 1994. Planning for project controlthrough risk analysis: a petroleum pipeline-laying project. InternationalJournal of Project Management 12 (1), 23–33.
Fairley, R., 1995. Risk Management for Software Projects. IEEE Software 11(3), 57–67.
Galorath, D., 2008. Software project failure costs billions, better estimation &planning can help. Galorath Incorporated. (Online: http://www.galorath.com/wp/software-project-failure-costs-billions-better-estimation-planning-can-help.php (10 October 2011)).
Gillian, A., Robert, A., 2001. Managing software project risk. TASSC. (Online:http://www.tassc-solutions.com/downloads/SoftwareRisks.pdf (AccessedOctober 2010)).
Haughey, D., 2009. Why software projects fail and how to make them succeed.ProjectSmart. ProjectSmart . (online: http://www.projectsmart.com/articles/why-software-projects-fail.php (Accessed December 2010)).
Heemstra, F.J., 1992. Software cost estimation. Information and SoftwareTechnology 34 (10), 627–639.
Janne, R., Lyytinen, K., 2000. Components of software development risk howto address them? A project manager survey. IEEE Transactions on SoftwareEngineering 26 (2), 98–112.
Jantzen, K., Gillian, A., Robert, A., 2006. Estimating the effect of project risksin software development projects. (online: http://www.tassc-solutions.com/downloads/EstimatingRisks.pdf (Accessed October 2010)).
Kansala, K., 1997. Integrating risk assessment with cost estimation. IEEESoftware 14 (3), 61–67.
Kellner, M.I., Madachy, R.J., Raffo, D.M., 1999. Software process simulationmodeling: Why? What? How? Journal of System and Software 46, 91–105.
Khamooshi, H., Cioffi, D.F., 2009. Program risk contingency budget planning.IEEE Transaction on Engineering Management 56 (1), 171–179.
Kumar, R.L., 2002. Managing risks in IT projects: an options perspective.Information Management 40 (1), 63–74.
Lange, K., 2003. Applied Probability. Springer.Lederer, A.L., Prasad, J., 1992. Nine management guidelines for better cost
estimating. Communications of the ACM 35 (2), 51–55.Lindland, O.I., Sindre, G., Solvberg, A., 1994. Understanding quality in
conceptual modeling. IEEE Software 2 (11), 42–49.Lum, K., Powell, J., Hihn, J., 2002. Validation of spacecraft software cost estimation
models for flight and ground systems. (online: http://trsnew.jpl.nasa.gov/dspace/bitstream/2014/11968/1/02-0706.pdf (Accessed December 2010)).
Madachy, R., Boehm, B., Wu, D., 2006. Comparison and assessment of costmodels for NASA flight projects. International Forum on COCOMO andSoftware Cost Modeling. 8 November 2006.
Menzies, T., Chen, Z., Hihn, J., Lum, K., 2006. Selecting best practices for effortestimation. IEEE Transactions on Software Engineering 32 (11), 883–895.
Montgomery, D.C., Runger, G.C., 2007. Applied Statistics and Probability forEngineers. John Wiley & Sons Inc., Hoboken, NJ.
Please cite this article as: Uzzafer, M., A contingency estimation model for software10.1016/j.ijproman.2012.12.002
Morgan/Reuters, 1996. J.P.RiskMetrics—Technical Document. Morgan GuarantyTrust Company, New York.
Papoulis, A., 1991. Probability, Random Variables, and Stochastic Processes,3rd edition. McGraw Hill Companies.
Pfleeger, S.L., Atlee, J.M., 2006. Software Engineering, Theory and Practice,3rd edition. Pearson International Edition.
Ross, S.M., 2007. Introduction to Probability Models, 8th edition. Elsevier.Tom, G., Susannah, F., 1988. Principles of Software Engineering Management.
Addison-Wesley.Touran, A., 2003. Calculation of contingency in construction projects. IEEE
Transactions on Engineering Management 50 (2), 135–140.Uzzafer, M., 2010. A pitfall of software estimated cost. Proceedings of IEEE
International Conference on Information Management and Engineering(ICIME 2010). Chengdu, China, pp. 578–582.
Uzzafer, M., submitted for publication. Measuring risk of software projects.Williams, R.C., Pandelios, G.J., Behrens, S.G., 1999. Software Risk Evaluation
