j ourna l homepage: www.e lsev ie r.com/ locate /autcon
A decision support system for contractor prequalification for surety bonding
Adel Awad a,1, Aminah Robinson Fayek b,⁎a Hole School of Construction Engineering, Department of Civil and Environmental Engineering, University of Alberta, 1-051 Markin/CNRL Natural Resources Engineering Facility,Edmonton, Alberta, Canada T6G 2W2b Hole School of Construction Engineering, Department of Civil and Environmental Engineering, University of Alberta, 3-013 Markin/CNRL Natural Resources Engineering Facility,Edmonton, Alberta, Canada T6G 2W2
In the construction bonding business, a complex and comprehensive prequalification or assessment process isdone to evaluate contractor, project, and contractual risks. Previous studies have focusedmainly on contractorprequalification from the owner's or consultant's perspective. There remains a need for a model to evaluatethe contractor and project-specific aspects (i.e., project team and contractual risks) from the surety bondingperspective. This paper identifies and classifies the most relevant evaluation criteria that surety underwritersand brokers consider when evaluating a specific construction project for bonding purposes. Several datacollection techniques (questionnaires, one-on-one interviews, and interacting group meetings), with highlyexperienced surety experts, were conducted to compile a comprehensive and detailed list of the evaluationcriteria. Both fuzzy logic and expert systems are combined to develop a decision support system (DSS) for usein contractor and project evaluation. An approach for fuzzy membership function estimation is presentedusing a traditional membership function estimation technique, integrated with data of contractorprequalification cases. Thirty-eight alternative system configurations are investigated to determine themost accurate one. The system is validated using 32 prequalification cases, and the accuracy of the system isfound to be 84.0%.
Construction is a risk-filled, uncertain, and dynamic environment,where two projects are rarely ever the same [1,2]. Contractor failure isalways possible, even for capable andwell-established contractors [3].For that reason, owners search for ways to mitigate the risk of con-tractor failure. One such technique is surety bonding, where the risk ofproject completion is transferred to the surety company [4,5].
Therefore it is a critical decision for a surety company to bond acontractor for a construction project [3]. The riskmust be estimated andreduced asmuch as possible via a complex evaluation (prequalification)process for the contractor. Many quantitative and qualitative evaluationcriteria must be taken into consideration in the contractor prequalifica-tion process [3].
This paper presents a decision support system (DSS) for suretybrokers and underwriters that helps them to assess the risk of con-tractor default in construction projects. The DSS was developed inclose collaboration with major surety broker and surety underwritingcompanies in Canada. With this tool, surety professionals can betterdecide whether or not to bond a contractor for specific construction
project, and contractors can identify areas that need improvement inorder to obtain bonding for construction projects. This paper alsoidentifies and classifies themajor evaluation criteria, including projectspecifics and contractual risks, necessary to advance the state of the artin surety underwriting.
The proposed DSS combines both fuzzy logic and expert systems tocreate a more structured, organized, and objective approach to use incontractor/project risk evaluation for surety underwriting purposes.This fuzzy expert DSS decreases subjectivity in the evaluation criteriaby creating predefined rating scales for the quantitative criteria, anddefining reference variables used to quantify values on the ratingscales of the qualitative criteria. This paper describes themethodologyused to create the fuzzy expert DSS, and focuses, in particular, on a newapproach for fuzzy membership function (MBF) estimation, whichcombines both knowledge-based (using the horizontal method) anddata-integration approaches. A validation of the system with hypo-thetical cases of contractor/project bonding evaluation is presented.Finally, areas for future research and development are outlined.
2. Background and previous research
2.1. Surety bonding in construction
In the construction industry, a surety company assumes the risksassociated with contractor prequalification by agreeing to bond a
90 A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
contractor for a construction project [4]. Fig. 1 shows a summary ofthe steps for obtaining the bonding facility. Most surety companieswork through surety brokers or a surety professional agent. Therefore,to obtain a bond, a contractor must first contact a construction suretybroker and provide the required business information. The brokerorganizes an information file on the contractor and submits it to theappropriate surety company according to the contractor's profile andneeds [4]. The surety underwriter then conducts a new contractorprequalification process that may require more in-depth informationabout the contractor's business. The underwriter's objective in thisprocess is to quantify the ability of the contractor to complete theconstruction project [6].
The contractor prequalification process occurs in two phases. Thefirst phase (contractor prequalification) begins when the contractorseeks a relationship with the surety company. The evaluation criteriaconsidered by the surety company during this phase can be placed inthree categories: character, capacity, and capital. The second phasebegins when the contractor requests bonding for a specific construc-tion project, and the surety underwriter conducts a second morecomprehensive prequalification (surety underwriting) process. Thesecond surety underwriting process includes evaluation of the projectspecifics and the contractual risks.
2.2. Contractor prequalification and surety underwriting studies
2.2.1. Contractor prequalificationMany studies exist on the topic of contractor prequalification. Al-
Sobiei et al. [7] developed a decision-making mechanism to assist
Broker Analyzes Data to Study Contractor`s Business and Needs
Broker Tailors Contractor’s Submission
Contractor Submits Relevant Data To Surety Broker
Surety UnderwriterRequests More Data
A Relationship Between Surety Underwriter and Contractor is Established
Contractor Contacts Surety Agent or Broker
First Phaseof Surety Evaluation
(Character, Capacity, and Capital)
Fig. 1. Summary of b
owners in predicting the likelihood of contractor default and inselecting the most suitable risk management method. Their researchalso compared two artificial intelligence techniques (artificial neuralnetworks [ANNs] and genetic algorithms [GAs]) that can be used forcontractor evaluation models. Eight cases were used for validation,yielding a prediction accuracy of 75% based on NN training and 88%based on GA training. Khosrowshahi [8] used ANNs to develop amodel to predict the suitability of contractors to tender for publicclients' projects, based on eleven prequalification attributes. Nguyen[9] presented a method for contractor prequalification that uses fuzzyset theory to incorporate subjective criteria inherent in the evaluationprocess. The evaluation criteria, and the assigned weights that reflecttheir importance, were predetermined based on the opinion of suretyprofessionals. Sonmez et al. [10] used an evidential reasoningapproach to develop a model for the multi-criteria contractorprequalification process. The model was not validated. Plebankiewicz[11] developed a fuzzy model for contractor prequalification from theowner's perspective. The model considered different contractorevaluation criteria and objectives that the owner wants to achievein the project. The model was not validated. Lam et al. [12,13]investigated the suitability of using the Support Vector Machine(SVM) method in contractor prequalification for construction projectprocurement, and presented a SVM-based decision support frame-work for contractor prequalification. The efficacy of the SVM modelwas validated in a case study, and the results were compared with theresults of using ANNs and principal component analysis (PCA) for thesame case study. The results showed that the SVM model was moreeffective than ANNs and PCA. Lam and Yu [14] used the principle of
Broker Analyzes Data and May Request More Data
Contractor Submits Relevant Data To Surety Broker
Underwriter Takes an In-depthLook at the Contractor’s
Entire Business
Broker Submits Data toSurety Underwriter
Surety Agrees to Issue Bondfor the Project
Contractor Requests Bondingfor a Specific Project
Second Phase of Surety Evaluation
(Project Specifics, Contractual Risks)
onding process.
91A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
multiple kernel learning (MKL) for decision support in contractorprequalification. Their study measured the accuracy and efficiency ofthe SVMmethod versus that of theMKL using a case study. The resultsshowed that MKL performed slightly better than SVM. Zavadskas et al.[15] developed a multi-attribute model for contractor evaluation andassessment that used the Hodges–Lehmann rule to determine theevaluation criteria values. Brauers et al. [16] presented a MOORAmethodology for multi-objective contractor ranking that uses ratioanalysis and dimensionless measurement to rank the contractors in anon-subjective way.
2.2.2. Surety underwritingFew studies have been done on the topic of surety underwriting.
Kangari et al. [17] presented a quantitative model to prequalify theperformance of construction companies from a financial perspective.The model was not validated. Kangari and Bakheet [18] developed alist of themajor factors that impact surety underwriting, in addition tofive evaluation forms to support the process. However, their study didnot provide a method to predict the outcome of the contractor'sperformance based on the data gathered from the forms. Severson etal. [19] investigated discrete choice modeling to predict the likelihoodof a claim occurrence on a construction surety bond. Their researchconsidered five financial evaluation criteria, and was validated using40 projects. The discrete choice model was found to have an accuracyof 87.5%. Bayraktar and Hastak [20] developed a conceptual, scoring-based contractor evaluation system that considers contractor-specificcriteria in the three categories of character, capacity, and capital.Further research by Marsh and Fayek [1,5,21,22] focused on the firstphase of the underwriting process (Section 2.1); they developedSuretyAssist, a decision support model that incorporates contractor-specific evaluation criteria (character, capacity, and capital) [1,5,22].SuretyAssist was validated in thirty-one historical cases, and asensitivity analysis was conducted to select the best system configu-ration. The model yielded an accuracy of 81.0%.
Having identified contractor evaluation from different perspec-tives, the previous research in the area of construction contractorprequalification and surety underwriting provides a point of depar-ture for the research presented in this paper. A model or amethodology for the second phase in the underwriting process hasnot been addressed in previous research; no study integratedcontractor-specific, project-specific, and contract-specific risk evalu-ation criteria in the underwriting process. There remains a need for amore structured and organized contractor/project prequalificationdecision-support system, not to replicate the surety practitioners'decisions but to enhance them. The research presented in this paperfocuses on the more advanced contractor evaluation process duringsurety underwriting (the second evaluation phase) when seekingbonding for a construction project.
The DSS presented in this paper fills an important gap in existingmodels by presenting a suitable way to integrate all the evaluationcriteria required for the second phase of the surety underwritingprocess and provide a comprehensive assessment tool to assist suretyexperts in their decision making. Moreover, the integration of expertknowledgewith prequalification cases (data) to build a surety DSS (aspresented in this paper) is a relatively new research area. Many of thecrucial evaluation criteria for the underwriting process depend uponexpert knowledge and subjective judgment. Therefore, incorporatingexpert judgment into the DSS is highly important. However, factoringexpert judgment into the evaluation criteria makes many criteria notonly subjective but also uncertain. Many criteria are qualitative andsome are quantitative. Combining all of these criteria into a singleassessment tool becomes a complex process, especially since therelationships between the criteria are non-linear and difficult todetermine. Fuzzy logic, an artificial intelligence technique, can handleuncertainty and subjectivity, and incorporate both quantitative andqualitative criteria into decision-making models. Combining fuzzy
logic with expert system to create DSS that have the ability to includeexpert knowledge and subjective judgment, or intuition, into thedecision-making process advances the state of the art in the suretyunderwriting process [22].
3. Development of the DSS
3.1. Surety underwriting criteria
Determining the evaluation criteria included in the proposed DSSoccurred into two steps. The first stepwas to develop an initial list thatincluded all the evaluation criteria to be considered in suretyunderwriting. The second step was to refine the initial list to includeonly the most important evaluation criteria.
3.1.1. Initial list of evaluation criteriaA comprehensive list was developed to include all the evaluation
criteria that should be considered for bonding a contractor for aspecific project. All the criteria presented in the previous studies onsurety underwriting [1,5,9,21,22], contractor prequalification [23–33],and contractor selection models [11,13,18,34–41], were included inthe list. Fifteen historical contractor prequalification cases from theparticipating surety underwriting and broker companies were used togenerate additional evaluation criteria. For the historical cases, all thedocumentation, information, and minutes of meetings betweensurety professionals and contractor representatives were reviewedand discussedwith the surety experts, to determine the importance ofthe collected information in the prequalification process and how itmight impact their bonding decision.
A group of five surety experts (with no less than 10 yearsexperience each) were selected to participate in the research: twoexperts came from underwriting companies, and three experts camefrom surety broker companies. One of the participating brokercompanies is one of the top Canadian surety broker companies thathandles the accounts for thirteen of Canada's top fifteen constructioncompanies. The company has over 500 offices in more than 120countries. The two underwriters had 23 and 10 years of experience inthe surety industry, and the three brokers had 26, 15, and 10 years ofexperience in the surety industry, specifically in construction. The laststage in developing the initial list of criteriawas to holdfifteen one-on-one interviews (three meetings each) with the five surety experts.
Each expert was asked to incorporate his or her thought processwhen evaluating general contractors and projects for surety bonding.After these one-on-one interviews and meetings, nine interactinggroup meetings were held to discuss the proposed list of evaluationcriteria, and add any helpful notes [42]. The result of this step was acomprehensive list that documented every factor, found in theliterature and determined during interviews (104 in total), thatpertained to contractor and project evaluation.
3.1.2. Relative importance of evaluation criteriaA questionnaire was developed to determine the relative impor-
tance of the evaluation criteria (i.e., inputs), meaning their influenceat a lower level in the model on the criteria (i.e., output) at a higherlevel. The following are samples of the questions included in thequestionnaire:
• What is the influence of the “Owner Type”, “Owner Funding”, and“Owner/Owner Agent Experience” on the “Owner” evaluation?
• What is the influence of the “Owner” evaluation, “Subcontractors”evaluation, and “Contractor” evaluation on the “Project Team”
evaluation?
The questionnaire was distributed among the same surety expertsparticipating in the study, whowere asked to evaluate the importanceof each criterion on a scale of 1–7: with 1 being the least importantand 7 being the most important. The seven-point scale was selected
Table 1Evaluation criteria and method of assessment.
Criterion name Quantification Red flags Favorable if
Project aspectsOwner (sub-model 1)
Owner type Public or private If private PublicOwner funding Rating scale of 1–7 b4 HigherOwner/owner agentexperience
Scope gaps Rating scale of 1–7 b4 HigherOverall (subcontractors)prequalification
Rating scale of 1–7 b4 Higher
ContractorYear end evaluation(sub-model 3)Working capital trend Real numbers (percent) b10% HigherTangible net worth trend Real numbers (percent) b10% HigherGross profit margin trend Real numbers (percent) b0% HigherNet profit margin trend Real Numbers (percent) b0% HigherDebt to equity ratio Real numbers (ratio) N2:1 LowerGross profit margin Real numbers (percent) b5% HigherNet profit margin Real numbers (percent) b2% Higher
Current evaluation(sub-model 4)Cash flow Rating scale of 1–7 b4 HigherOperating line Rating scale of 1–7 b4 HigherWork on hand Rating scale of 1–7 b4 Higher
Project specifics/scope(sub-model 5)Project type/complexity Rating scale of 1–7 b4 HigherProject size Rating scale of 1–7 b4 HigherProject location Rating scale of 1–7 b4 HigherCost breakdown Rating scale of 1–7 b4 HigherSchedule Rating scale of 1–7 b4 HigherProject risk Rating scale of 1–7 b4 Higher
Contractual riskContract form Standard, owner worded,
or combinedIf ownerwording
Standard
Contract clauses (sub-model 6)Payment Rating scale of 1–7 b4 HigherWarranty Rating scale of 1–7 b4 HigherIndemnity Rating scale of 1–7 b4 HigherSchedule extensions andprice adjustment
Rating scale of 1–7 b4 Higher
Damages/penalties/bonuses Rating scale of 1–7 b4 HigherToxic and hazardoussubstance and materials
Rating scale of 1–7 b4 Higher
Disputes/arbitration Rating scale of 1–7 b4 HigherDesign concerns Rating scale of 1–7 b4 HigherBonding/security Rating scale of 1–7 b4 Higher
92 A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
on the basis of its efficiency, and its ability to capture variations inexperts' opinion, without presenting too many, or too few, choices(leading to vacillation or lost data) [43,44].
The questionnaire results were used to identify the most importantcriteria, and to eliminate those with a minor impact on the bondingbroker's or underwriter's judgment. The score from each participant foreach criterion was given equal weight to calculate an average rating.Equal weight was given to all the participants' scores, because they allhave excellent experience (10 years or more) in the process ofcontractor prequalification. The average ratings were then used toreduce the number of criteria, and to generate the rules that logicallyrelate each input variable (i.e., the evaluation criteria) to the outputvariable (i.e., contractor/project overall prequalification). The genera-tion of the rules is explained later in this paper. Due to the large numberof evaluation criteria and to the practical limitations of fuzzy expertsystems, a hierarchical organizational structure was created for theinput criteria, and their number was reduced. The criteria that had anaverage importance value of less than 4.0 (a cutoff value recommendedby the surety experts) on the questionnairewere eliminated, leaving 32out of 104 criteria to have a high influence on surety underwritingdecisions. According to the surety expertswhoparticipated in the study,the evaluation criteria that address both project and contractual risksare themost important evaluation criteria to be used in the evaluation ofgeneral contractors when they request bonding for specific projects.Therefore, the evaluation criteria were grouped under two maincategories: project aspects and contractual risk.
3.1.3. Quantification and description of evaluation criteriaThe final list of the contractor and project evaluation criteria is
presented in Table 1. The first main category, “Project Aspects,”includes the “Project Team” (Owner evaluation, Subcontractor eval-uation, and Contractor evaluation) and the “Project Specifics/Scope,”(Project Type/Complexity, Project Size, Project Location, Cost Break-down, Schedule, and Project Risk). Each subcategory is divided intoone or twomore levels of detail. For example, “Contractor” evaluationis divided into two categories, “Year-End” evaluation and “Current”evaluation. Current evaluation, in turn, has three subdivisions: CashFlow, Operating Line, and Work on Hand. The second main category“Contractual Risk,” includes specific criteria to analyze the contractdocuments, such as the form of contract (Standard, Combined,Standard with Some Modifications, or Owner Worded), and thespecific contract clauses (payment, warranty, indemnity, scheduleextensions and price adjustment, damages/penalties/bonuses, toxicand hazardous substance and materials, disputes/arbitration, designconcerns, and bonding/security).
The hierarchical structure of the fuzzy-expert DSS presented inTable 1was used for three purposes: 1) to reduce the required numberof rules, because rules in fuzzy expert systems grow exponentiallyaccording to the number of input criteria for a single rule block; 2) todivide the fuzzy expert DSS into six smaller systems (i.e., fewer inputcriteria) so as to apply the proposed approach for membershipfunction (MBF) estimation, explained later in this paper; and 3) toprovide intermediate assessments of criteria categories to helpidentify specific areas for improvement. In the six smaller systems,groups of lower level criteria provide inputs into separate rule blockswhose output is higher-level, intermediate variables. These interme-diate variables then form the input for the next layer of intermediatecriteria, until a single output (i.e., overall prequalification) is obtained.
Table 1 also presents the scales used to quantify the evaluationcriteria, the threshold values (red flags), below which there is a causefor concern for the variable, and the favorable trends, as suggested bysurety experts. The red flags were created to enable the broker orunderwriter to conduct further research regarding the variable thatcreates a red flag.
The DSS considers thirty-two evaluation criteria: eight are quan-titative, twenty-two are qualitative, and two are categorical, such as
owner type (Public or Private). Table 1 presents the system's quan-titative criteria, which appear as a percentage (e.g., working capitaltrend), a ratio (e.g., debt to equity ratio), or a number of years(owner/owner agent experience). A rating scale of 1–7was created forall the qualitative criteria. To reduce subjective interpretation duringthe rating of the qualitative criteria, four interacting group meetingswere held with the participating surety experts to determine how thevalues for the qualitative criteria could be objectively evaluated. Theoutcome of these meetings was a set of reference variables, used toobjectively quantify the qualitative criteria and to define each scalevalue (1 to 7) for each of the qualitative criteria. For example, a set ofsix reference variables were used to define the predetermined ratingscale for the “Project Risk” criterion: 1) prepared project risk profile;2) quality of project risk assessment; 3) effect on the project cost andtime; 4) contingency assignment; 5) prepared risk mitigation plan;and 6) existence of a risk management team.
93A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
3.2. Membership function estimation
Membership functions (MBF) describe and represent the fuzzyexpert DSS input and output evaluation criteria, and the linguisticterms used for each criterion. The membership value indicates thedegree of belonging of an element on the relevant scale to thelinguistic terms. Membership values are between 0 and 1, where avalue of 0 indicates non membership, and a value of 1 indicates fullmembership. Estimating MBF is a vital step in creating any fuzzysystem, and the success of the system depends on it. However, MBFestimation is one of the most challenging aspects in designing fuzzysystems. It is difficult to evaluate the correctness of the MBF by usingany particular method. In addition, the techniques used for estimationneed to be flexible so that the MBF can be easily adjusted, or tuned, tooptimize the system's performance. Moreover, the choice of the MBFestimation method depends on the nature of the problem and thetype of data available [44–47]. Medasani et al. [44] pointed out that,for most applications, several methods must be incorporated toconstruct MBF, because many methods are difficult to use in practicalapplications, and because, generally, the applications are unique.Developing the context in which these methods will be applied iscrucial [47] andmust be considered before deciding on whichmethodis appropriate.
For building predictive models, such as fuzzy expert DSS, andespecially for MBF estimation, there are two approaches that can befollowed. The first approach is to use expert knowledge, if a group ofsubject matter experts is available. For example, in fuzzy membershipfunction estimation, the membership values are assigned subjectivelyby experts, based on their knowledge and past experience. The secondapproach is to use a set of research-related data to build themodel. Thedesign of the model components is governed by using historical anddocumented data that represent the research problem domain. Forexample, in fuzzy MBF, the fuzzy membership values are calculatedfrom collected data (i.e., cases).
Surety professionals do not currently document all evaluation(input) criteria used for the proposed DSS. Therefore, there are nodata that can be used for MBF estimation. In order to consider theissue of evaluation of MBF quality or correctness, a number of hypo-thetical contractors prequalification cases were developed to be usedfor the estimation of MBFs. Estimation of the MBF for the proposedDSS occurred in two steps. The first step used expert knowledge(knowledge-based initial estimation step) to initially estimate theMBF, and the second step used the developed contractor prequalifica-tion cases (data integration step) to evaluate the quality of the
a
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4 5 6 7
Mem
ber
ship
Val
ue
POOR
Rating Scale
Fig. 2. (a) Project type/complexity initial membership function estimated by horizon
estimated MBFs. Before the MBF estimation process began, the par-ticipating experts were consulted on the most appropriate linguisticterms to describe the input and output variables, and on thenumerical values used to quantify the variables.
3.2.1. The knowledge-based initial estimation stepThe first step in the process of MBF estimation used the horizontal
method, a traditional MBF estimation technique that depends onexpert knowledge. During this step, a second questionnaire wascreated to estimate the initial membership functions for the inputevaluation criteria. The questionnaire contained questions about theproposed values of the elements of each fuzzy set and the degree ofmembership of each value in the linguistic terms for the inputevaluation criteria. After being asked to identify a collection ofelements in the universe of discourse for each criterion, the expertswere then asked to answer some yes or no questions in the form,“Does xi belong to the concept of fuzzy set A?” [48].
For each value (xi) the number of positive (yes) answers wascounted, and the ratio of the positive answers to the total number ofreplies was computed. This ratio was treated as a membership degreeof the concept, at the given point of the universe of discourse. Fig. 2(a)shows an example of the estimatedMBF for a Project Type/Complexityevaluation criterion using the horizontal method. The x-axis repre-sents the rating scale used to quantify the evaluation criterion, and they-axis represents the corresponding membership value.
3.2.2. Data integration stepThe second step in the MBF estimation process used sixty-three
hypothetical contractor prequalification cases to select the bestsolution from the previously calculated MBFs, as will be explainedlater in the paper. The evaluation criteria representing the input to theDSS were divided into six groups representing the inputs of six sub-models, as presented in Table 1. Each sub-modelwas named accordingto its output (evaluation) and contains a number of input criteria. Theevaluation criteria for the six sub-models represent all the evaluationcriteria presented in Table 1, except for the contract form criterion,which has no level of evaluation criteria below it. The contract formcriterion is also represented by crisp (discrete) values, so it does notneed membership functions for its linguistic terms.
3.2.2.1. Membership function interpolation. After the initial MBFsestimation using the horizontal method, the estimated MBFs werethen transformed (interpolated) to two of the most practical andcommonly used shapes: triangular and trapezoidal [38,49]. Fig. 2(b)
1 2 3 4 5 6 7
b
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Mem
ber
ship
Val
ue
AVERAGE GOOD
Rating Scale
tal method, (b) Interpolation of project type/complexity membership function.
94 A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
illustrates an example of the MBF interpolation process (linearapproximation). According to the approximation process, severalsolutionswere found to represent the calculatedmembership functionsfor each linguistic term. All approximated shapes (whether triangular ortrapezoidal) for the actual datawere considered as alternative solutions.The target of this step was to select the most appropriate shapes andparameters for inputs' MBFs for developing the DSS. The selectionprocess depended on production of an objective value to measure theperformance status for each solution.
The triangular and trapezoidal MBF were described using fourparameters,a,b,c, and d; in triangular functions b=c. All the possiblelinear approximations for the MBFs were investigated. Table 2 showsan example of the ten different possible solutions for the “Owner” sub-model. For example, in the first solution, under the “Owner Funding”criterion, the “POOR” linguistic term is represented by trapezoidalMBFwith the parameters (1, 1, 2.5, and 5); the “AVERAGE” linguistic term isrepresented by triangular MBF with the parameters (2, 5, 5, and 7);and the “GOOD” linguistic term is represented by trapezoidal MBF(5, 6.75, 7, and 7). In the “Owner” sub-model, the solutions representedonly two sub-criteria:“Owner funding” and “Owner/owner agentexperience.” The third sub-criterion “Owner type” was representedusing discrete values.
3.2.2.2. Testing alternative solutions for sub-models. Linear approxima-tions for the calculated MBFs were performed for each sub-model;some had ten solutions, such as “Subcontractors” and “Year-EndEvaluation,”while other sub-models had only six solutions. Each sub-model was investigated as a separate model, in order to select the bestMBF representation for the sub-model input criteria. All of thedifferent solutions for each sub-model were implemented using afuzzy expert system shell, FuzzyTECH® [50].
Surety underwriters and brokers do not currently document allevaluation criteria (inputs) that were established in this study.Therefore, ninety-five hypothetical cases were created, to cover thefull range of possible contractor evaluation scenarios. Two-thirds (63)of the cases were used for the DSS development stage, while one-third(32) was used for the validation and sensitivity analysis stage. Each ofthe participating experts was asked to develop input values for caseshe or she believedwouldmost likely happen in reality. The cases weredistributed among the experts, who were asked to provide theappropriate output values according to the input values. As a result ofthis process, each of the hypothetical cases contained the values ofeach input evaluation criterion, and the corresponding sub-modeloutput value, in addition to the corresponding overall prequalificationvalue. For example, in the “Current Evaluation” sub-model, if the“Cash Flow,” “Operating Line,” and “Work On Hand” values are 4, 1,and 7, respectively, then the “Current Evaluation” output value is 3 (asdetermined by surety experts according to the input values).
Table 2Membership Function Solutions for Owner Sub-Model.
All the alternative solutions for each sub-model were developedusing the same configuration (rule base, rules' degrees of support,fuzzy operator, implication method, rule aggregation method,defuzzification method). The only difference between each solutionfor the same sub-model was in the MBFs that represent the inputcriteria. A comparison between the accuracy of the different solutionstherefore reflects the quality of the membership functions.
The values of the input evaluation criteria were presented to eachsub-model to predict the output value. For example, sixty-threehypothetical cases containing values for “Cash Flow,” “Operating Line,”and “Work On Hand”were presented to the “Current Evaluation” sub-model to predict the value of “Current Evaluation”. A comparison wasmade between the predicted output values and actual output values(developed by participating experts). The average percent error ofeach solution for each sub-model configuration was calculated usingEq. 1 [1,5], where
Average Percent Error =∑n
i=1Predicted Outputi−Actual Valuei
Actual Valuei
��������
� �
n× 100
ð1Þ
“Predicted Output” is the output value provided by the modelaccording to the input values for each case; “Actual Value” is theoutput value provided by the underwriter or broker for each case; i isthe individual case number; and, n is the total number of cases.
Fig. 3 shows the graphical representation of the average errorpercentage obtained from testing each solution for each of the six sub-models. The x-axis represents the alternative solution number, andthe y-axis represents the corresponding average percent error foreach solution. For example, for the “Subcontractor” sub-model, tenalternative solutions for MBFs were investigated, and the averagepercent error for the solutions ranged from 20.8% to 23.6%. In eachcase, the solution with the lowest average percent error was selectedto build the final fuzzy expert DSS. For the “Subcontractor” sub-model,the third solution was the best one (average error of 20.8%). For the“Owner” sub-model, the best solution had an average error of 16.0%.For “Year End Evaluation,” the best solution had an average error of19.4%. For “Current Evaluation,” the best solution had an average errorof 12.4%. For “Contract Clauses,” the best solution had an average errorof 18.4%; and, for “Project Specifics/Scope,” the best solution had anaverage error of 17.3%.
3.3. Rule base development
Fuzzy rules consist of a condition (If part) and conclusion (Thenpart) and represent the experts' reasoning process in the fuzzy expertsystem. In developing the fuzzy expert DSS, the data obtained from the
Fig. 3. Results of testing of sub-models' alternative solutions.
95A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
first questionnaire (Section 3.1) was used to create a rule base for thesystem. The average relative importance weights of the input criteriawere used to determine the rule base that represents the relationbetween the inputs and the output. Three influence levels (based onthe average relative importance weights on a scale of 1 to 7) weredefined for each input criterion: b5, 5 to 6, or N6. Correspondinginfluence levels were defined as: “Minor Influence,” “ModerateInfluence,” or “High Influence,” respectively. The rule base was thencreated according to the influence levels of the inputs. As an example,the “Current Evaluation” sub-model contains three inputs (“CashFlow,” “Operating Line,” and “Work on Hand”) that have averageimportance weights equal to 6.8, 4.0, and 4.8, respectively. Theseaverage importance weights translate to influence levels of “HighInfluence,” “Minor Influence,” and “High Influence,” respectively.According to the influence level for each input, the rules were gen-erated. If “Cash Flow” is “GOOD,” and “Operating Line” is “POOR,” and“Work On Hand” is “POOR,” then the output (“Current Evaluation”) is“AVERAGE”. Because “Cash Flow” has a higher influence level, it isgiven greater weight than “Operating Line” or “Work on Hand” in thedetermination of the output level.
All possible combinations between the inputs' linguistic termswere considered to generate a complete rule base, i.e., if there are Ninputs, eachwith Zmembership functions, then the complete rule basecontains ZN rules.
4. DSS validation and sensitivity analysis
The final fuzzy expert DSS (including all the contractor/projectevaluation criteria and sub-models described previously) was imple-mented using a fuzzy expert system shell, FuzzyTECH® [50].
The fuzzy expert system shell consists of three parts: fuzzification,inference engine, and defuzzification. In fuzzification, the systemcalculates the degree of membership for each linguistic term thatdefines each criterion value. Then, it applies a fuzzy operator to themembership values from each evaluation criterion to link the com-binations of evaluation criteria to overall contractor/project prequa-lification as a single value for each rule. The MIN (minimum) fuzzyoperatorwas initially used as the fuzzy operator in this step. In the nextstep, the inference engine applies an implication method for each ruleto the output variable's membership function. The PROD (product)was used as an implication method. The last step in the inferenceengine is rule aggregation. Rule aggregation is the process of com-bining the output sets from each rule into a single output fuzzy set. TheMAX (maximum) rule aggregation method was initially used.Defuzzification, the last step, determines a crisp value from the outputfuzzy set. The CoM (center ofmaximum)methodwas initially selectedas a defuzzification method.
A base casemodelwas built using all the initial operators describedpreviously, along with piecewise linear membership functions (e.g.,
96 A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
triangular or trapezoidal), estimated during the MBF estimation step.The base case model and thirty-eight alternative system configura-tions were developed to determine which system configurationproduced the most accurate results. The configurations consideredthe different input aggregation methods [MIN (minimum); MAX(maximum); AVG (average); PROD (product); MIN/AVG (minimum/average); and MIN/MAX (minimum/maximum)], different rule ag-gregation methods [MAX (maximum), BSUM (bounded sum)], anddifferent defuzzification methods [COM (center of maximum); MOM(middle of maximum); Fast COA (fast center of area); Hyper COM(hyper center of maximum)]. The product method [PROD (product)]was used for rule implication, as it is the only available method inFuzzyTECH for implication. The characteristics of the system config-urations are shown in Table 3.
The fuzzy expert DSS was validated using the thirty-two hypo-thetical contractor/project prequalification cases. Each case containeda value for all the evaluation criteria (Table 1) and the correspondingoutput (overall contractor/project prequalification), based on partic-ipating surety experts' opinion. The average percent error of eachsystem configuration was calculated using Eq. 1. In Eq. 1, the fuzzyexpert DSS output is the crisp rating provided by the system'sdefuzzification process. The actual rating is the rating given to thecontractor by the underwriter or broker. Table 3 presents the thirty-eight different system configurations that were tested, along with theaverage percent error and 95% confidence intervals from evaluatingthe hypothetical contractor/project prequalification cases. The mostaccurate system configuration, number 24, consists of piecewise linearmembership functions, MIN (minimum) for input aggregation, PROD
Table 3System configuration for validation and sensitivity analysis.
Base Piece linear MIN PROD MAX1 Piece linear MAX PROD MAX2 Piece linear AVG PROD MAX3 Piece linear PROD PROD MAX4 Piece linear MIN/AVG PROD MAX5 Piece linear MIN/MAX PROD MAX6 Piece linear MIN PROD BSUM7 Piece linear MAX PROD BSUM8 Piece linear AVG PROD BSUM9 Piece linear PROD PROD BSUM10 Piece linear MIN/AVG PROD BSUM11 Piece linear MIN/MAX PROD BSUM12 Piece linear MIN PROD MAX13 Piece linear MAX PROD MAX14 Piece linear AVG PROD MAX15 Piece linear PROD PROD MAX16 Piece linear MIN/AVG PROD MAX17 Piece linear MIN/MAX PROD MAX18 Piece linear MIN PROD BSUM19 Piece Linear MAX PROD BSUM20 Piece Linear AVG PROD BSUM21 Piece Linear PROD PROD BSUM22 Piece Linear MIN/AVG PROD BSUM23 Piece Linear MIN/MAX PROD BSUM24 Piece Linear MIN PROD MAX25 Piece Linear MAX PROD MAX26 Piece Linear AVG PROD MAX27 Piece Linear PROD PROD MAX28 Piece Linear MIN/AVG PROD MAX29 Piece Linear MIN/MAX PROD MAX30 Piece Linear MIN PROD BSUM31 Piece Linear MAX PROD BSUM32 Piece Linear AVG PROD BSUM33 Piece Linear PROD PROD BSUM34 Piece Linear MIN/AVG PROD BSUM35 Piece Linear MIN/MAX PROD BSUM36 Piece Linear MIN PROD MAX37 Piece Linear PROD PROD BSUM38 Piece Linear MIN/MAX PROD BSUM
(product) for implication, MAX (maximum) for rule aggregation, andFast CoA (fast center of area) for defuzzification. This system con-figuration has an average percent error of 16.0% (i.e., 84.0% accuracy),with a 95% confidence interval between 12.0% and 20.1% (i.e., 88.0%and 79.9% accuracy).
According to the input values for each case, the DSS provides theuser with the contractor prequalification for both intermediatevariables and the final output (i.e., overall contractor and projectprequalification) on a defuzzified scale of 1 to 7. For the final output,this rating scale is represented by five MBF. Each defuzzified value onthe 1 to 7 rating scale for the final output is described by the followinglinguistic terms, respectively: “Not Qualified”, “Somewhat Qualified”,“Below Average Qualified”, “Average Qualified”, “Above AverageQualified”, “Very Qualified”, and “Extremely Qualified”. If the overallcontractor prequalification is “AverageQualified” (i.e., 4) or higher, thecontractor will likely be accepted for bonding. A report consisting ofthe input and output values can then be printed to document thecontractor's prequalification case.
5. Conclusions and future work
A fuzzy expert decision support system (DSS) was developed tohelp surety underwriters and brokers in the second phase of the suretyunderwriting process and to provide a systematic and structuredapproach to this complex process. The fuzzy-expert DSS was validatedwith a number of hypothetical cases of contractor prequalification.Five senior surety professionals provided input to the determination ofthe contractor evaluation criteria and the model development.
od Defuzzification method Average percent error 95% confidence interval
COM 19.3% 15.1%–23.5%COM 27.9% 17.9%–37.9%COM 19.6% 11.9%–27.3%COM 31.3% 21.7%–40.9%COM 24.5% 16.3%–32.6%COM 25.6% 17.0%–34.2%COM 23.7% 17.5%–29.9%COM 27.9% 17.9%–37.9%COM 27.9% 17.9%–37.9%COM 24.2% 17.7%–30.6%COM 27.9% 17.9%–37.9%COM 27.9% 17.9%–37.9%MOM 28.6% 20.6%–36.5%MOM 37.5% 31.8%–43.2%MOM 24.5% 16.6%–32.4%MOM 30.9% 20.4%–41.4%MOM 24.5% 16.6%–32.4%MOM 25.7% 17.9%–33.4%MOM 24.7% 17.1%–32.3%MOM 37.5% 31.8%–43.2%MOM 37.5% 31.8%–43.2%MOM 22.6% 15.1%–30.1%MOM 37.5% 31.8%–43.2%MOM 37.5% 31.8%–43.2%Fast COA 16.0% 12.0%–20.1%Fast COA 27.9% 17.9%–37.9%Fast COA 27.7% 17.9%–37.6%Fast COA 30.8% 21.0%–40.6%Fast COA 27.0% 17.9%–36.1%Fast COA 27.6% 17.9%–37.4%Fast COA 23.7% 17.5%–29.9%Fast COA 27.9% 17.9%–37.9%Fast COA 27.9% 17.9%–37.9%Fast COA 24.1% 17.6%–30.5%Fast COA 27.9% 17.6%–37.9%Fast COA 26.1% 19.3%–32.8%Hyper COM 17.1% 13.0%–21.1%Hyper COM 23.6% 16.1%–30.1%Hyper COM 28.2% 17.5%–38.9%
97A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
A comprehensive, detailed list of the evaluation criteria forcontractor and project prequalification was compiled following athorough literature review, and review of contractor prequalificationcases, fifteen one-on-one interviews and nine interacting groupmeetings with the participating surety experts. Numerical scaleswere defined for the quantitative evaluation criteria, and, rating scales,using reference variables, were developed to quantify the qualitativecriteria. For all criteria, critical threshold values and favorable trendswere determined.
A new approach for fuzzy membership function estimation wasdeveloped. The new approach incorporates the horizontal MBF esti-mation technique, which depends on expert knowledge, with someprequalification cases (data integration). Finally, the fuzzy expert DSSwas validated with a number of hypothetical cases of project bondingevaluation.
The proposed fuzzy expert DSS offers several advantages to suretyprofessionals who conduct surety underwriting. The system improvessurety underwriters' and brokers' reliance on judgment and experi-ence to validate their underwriting decisions. It also provides a struc-tured, organized, and objective approach to evaluate subjective, anddifficult to quantify, criteria in contractor qualification for a specificproject, to formalize and quantify complex decision making and makeits logic easy to trace. Finally, the proposed system can assist con-struction contractors to self-assess and to discover areas for improve-ment to better obtain bonding for construction projects.
Further improvements to the fuzzy expert DSS are currently underinvestigation. One important evaluation component, contractors'organizational practices, will be incorporated. Contractor organiza-tional practices have not been evaluated in the surety underwritingprocess, but their inclusion may enhance decision making, and allowthe evaluation of a contractor's plans regarding practices such as safetymanagement, quality management, time management, cost manage-ment, and many other practices that can contribute to project successto be considered. However, this evaluation component requires moreresearch before incorporation into the existing DSS, and surety expertsfrom across Canada have been invited to contribute to the process.
For increasing the DSS's accuracy and ability to adapt, more actualcontractor prequalification cases will be collected from surety un-derwriters and brokers, to enable the use of other soft computingoptimization techniques, such as artificial neural networks (ANNs)and/or genetic algorithms (GAs) in developing the fuzzy expert systemcomponents (membership functions, the rule base, and rules' degreesof support). Using these techniques will allow for future calibration ofthe model components.
Acknowledgments
The authors would like to acknowledge Mr. Andre Giasson andMr.Brian Davidson of AON Reed Stenhouse, Mr. Jason Smith of Foster ParkBaskett, Ms. Betty Shellnutt of AXA Pacific, and Mr. Greg Forsythe ofThe Guarantee Company of North America for their time and expertisein providing input while conducting this research.
References
[1] K. Marsh, A fuzzy expert system decision-making model to assist suretyunderwriters in the construction industry, M.Sc. Thesis, Dept. of Civil andEnvironmental Engineering, University of Alberta, Edmonton, Alberta, 2008.
[2] L. Chao, M.J. Skibniewski, Fuzzy logic for evaluating alternative constructiontechnology, Journal of Construction Engineering and Management 124 (4) (1998)297–304.
[3] Surety Information Office (SIO), The importance of surety bonds in construction,2011, available online http://www.sio.org/pdf/importanceof.pdf.
[4] J.S. Russell, Surety bonding and owner-contractor prequalification: comparison,Journal of Professional Issues in Engineering 116 (4) (1990) 360–374.
[5] K. Marsh, A.R. Fayek, Development of a fuzzy expert system for suretyunderwriting, Proceedings, Construction Research Congress, Seattle, Washington1 (2009) 219–228.
[6] A. Awad, A.R. Fayek, Developing a framework for construction contractorqualification for surety bonding, Proceedings, Construction Research Congress,Banff, Alberta 2 (2010) 899–908.
[7] O.S. Al-Sobiei, D. Arditi, G. Polat, Managing owner's risk of contractor default,Journal of Construction Engineering and Management 131 (9) (2005) 973–978.
[8] F. Khosrowshahi, Neural network model for contractors' prequalification for localauthority projects, Engineering Construction and Architectural Management 6 (3)(1999) 315–328.
[9] V.U. Nguyen, Tender evaluation by fuzzy sets, Journal of Construction Engineeringand Management 111 (3) (1985) 231–243.
[10] M. Sonmez, G.D. Holt, J.B. Yang, Applying evidential reasoning to prequalifyingconstruction contractors, Journal of Management in Engineering 18 (3) (2002)111–119.
[11] E. Plebankiewicz, Contractor prequalification model using fuzzy sets, Journal ofCivil Engineering and Management 15 (4) (2009) 377–385.
[12] K.C. Lam, E. Palaneeswaran, C.Y. Yu, A support vector machine model forcontractor prequalification, Automation in Construction 18 (3) (2009) 321–329.
[13] K.C. Lam, M.C. Lam, D. Wang, Efficacy of using support vector machine in acontractor prequalification decision model, Journal of Computing in CivilEngineering 24 (3) (2010) 273–280.
[14] K.C. Lam, C.Y. Yu, A multiple kernel learning-based decision support model forcontractor pre-qualification, Automation in Construction, 2010 available onlinehttp://dx.doi.org/10.1016/j.autcon.2010.11.019.
[15] E.K. Zavadskas, Z. Turskis, J. Tamošaitiene, Contractor selection of construction in acompetitive environment, Journal of Business Economics and Management 9 (3)(2008) 181–187.
[16] W.K.M. Brauers, E.K. Zavadskas, Z. Turskis, T. Vilutien, Multi-objective contractor'sranking by applying the MOORA method, Journal of Business Economics andManagement 9 (4) (2008) 245–255.
[17] R. Kangari, F. Farid, H.M. Elgharib, Financial performance analysis for constructionindustry, Journal of Construction Engineering and Management 118 (2) (1992)349–361.
[18] R. Kangari, M. Bakheet, Construction surety bonding, Journal of ConstructionEngineering and Management 127 (3) (2001) 232–238.
[19] G.D. Severson, J.S. Russell, E.J. Jaselskis, Predicting contract surety bond claimsusing contractor financial data, Journal of Construction Engineering andManagement 120 (2) (1994) 405–420.
[20] M.E. Bayraktar, M. Hastak, Scoring approach to construction bond underwriting,Journal of Construction Engineering and Management 136 (9) (2010) 957–967.
[21] K. Marsh, A.R. Fayek, A decision-making model for surety underwriters in theconstruction industry based on fuzzy expert systems, Joint InternationalConference on Computing and Decision Making in Civil and Building Engineering,Montreal, Canada, 2006.
[22] K. Marsh, A.R. Fayek, SuretyAssist: a fuzzy expert system to assist suretyunderwriters in evaluating construction contractors for bonding, Journal ofConstruction Engineering and Management 136 (11) (2010) 1219–1226.
[23] J.S. Russell, M.J. Skibniewski, Decision criteria in contractor prequalification,Journal of Management in Engineering 4 (2) (1988) 148–164.
[24] G.D. Holt, P.O. Olomolaiye, F.C. Harris, Factors influencing UK construction clients'choice of contractor, Building and Environment 29 (2) (1994) 241–248.
[25] J.S. Russell, A knowledge-based System Approach to the Contractor Prequalifica-tion Process, PhD Thesis, Purdue University, Purdue, West Lafayette, USA,1988.
[26] J.S. Russell, Decision models for analysis and evaluation of constructioncontractors, Construction Management and Economics 10 (3) (1992) 185–202.
[27] G.D. Holt, P.O. Olomolaiye, F.C. Harris, Evaluating prequalification criteria incontractor selection, Building and Environment 29 (4) (1994) 437–448.
[28] S.T. Ng, R.M. Skitmore, Client and consultant perspectives of prequalificationcriteria, Building and Environment 34 (5) (1999) 607–621.
[29] S.T. Ng, R.M. Skitmore, Contractor selection criteria: A cost–benefit analysis, IEEETransactions on Engineering Management 48 (1) (2001) 96–106.
[30] North American Development Bank, Prequalification of contractors, available onlinehttp://www.nadb.org/pdfs/procurementdocs/Prequal_Contractors.pdf 2009.
[31] M. Egemen, A.N. Mohamed, Different approaches of clients and consultants tocontractors' qualification and selection, Journal of Civil Engineering andManagement 11 (4) (2005) 267–276.
[32] N. Banaitienė, A. Banaitis, Analysis of criteria for contractors' qualification evaluation,Technological and Economic Development of Economy 12 (4) (2006) 276–282.
[33] D. Singh, R.L.K. Tiong, Contractor selection criteria: investigation of opinions ofSingapore construction practitioners, Journal of Construction Engineering andManagement 132 (9) (2006) 998–1008.
[34] W.K.M. Brauers, E.K. Zavadskas, Z. Turskis, T. Vilutien, Multi-objective contractor'sranking by applying the MOORA method, Journal of Business Economics andManagement 9 (4) (2008) 245–255.
[35] E.K. Zavadskas, Z. Turskis, J. Tamošaitiene, Contractor selection of construction in acompetitive environment, Journal of Business Economics and Management 9 (3)(2008) 181–187.
[36] O. Abudayyeh, Z.S. Zidan, S. Yehia, D. Randolph, Hybrid prequalification-based,innovative contracting model using AHP, Journal of Construction Engineering andManagement 23 (2) (2007) 88–96.
[37] Y. Li, X. Nie, S. Chen, Fuzzy approach to prequalifying construction contractors,Journal of Construction Engineering and Management 133 (1) (2007) 40–49.
[38] D. Singh, R.L.K. Tiong, A fuzzy decision framework for contractor selection, Journalof Construction Engineering and Management 131 (1) (2005) 62–70.
[39] C.H. Wong, Contractor performance prediction model for the United Kingdomconstruction contractor: study of logistic regression approach, Journal ofConstruction Engineering and Management 130 (5) (2004) 691–698.
98 A. Awad, A.R. Fayek / Automation in Construction 21 (2012) 89–98
[40] I.M. Mahdi, S.M. Riley, S.M. Fereig, A.P. Alex, A multi-criteria approach tocontractor selection, Engineering, Construction and Architectural Management 9(1) (2002) 29–37.
[41] K.C. Lam, T. Hu, S.T. Ng, M. Skitmore, S.O. Cheung, A fuzzy neural networkapproach for contractor prequalification, Construction Management and Eco-nomics 19 (2) (2001) 175–188.
[42] M.R. Hallowell, J.A. Gambatese, Qualitative research: application of the Delphimethod to CEM research, Journal of Construction Engineering and Management136 (1) (2010) 99–107.
[43] H.E. Osgood, G. Suci, P. Tannenbaum, The Measurement of Meaning, University ofIllinois Press0-252-74539-6, 1957.
[44] S. Medasani, J. Kim, R. Krishnapuram, An overview of membership functiongeneration techniques for pattern recognition, International Journal of Approx-imate Reasoning 19 (1998) 391–417.
[45] W. Pedrycz, F. Gomide, An introduction to fuzzy sets: analysis and design, ABradford Book, MIT Press, Cambridge, Massachusetts, London, England, 1998.
[46] M.R. Civanlar, H.J. Trussell, Constructing membership functions using statisticaldata, Fuzzy Sets and Systems 18 (1986) 1–13.
[47] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications, PrenticeHall PTR, Upper Saddle River, NJ, 1995.
[48] W. Pedrycz, F. Gomide, Fuzzy Systems Engineering: Toward Human-CentricComputing, John Wiley & Sons, Inc., Hoboken, New Jersey, 2007.
[49] P. Lorterapong, O. Moselhi, Project-Network analysis using fuzzy sets theory,Journal of Construction Engineering and Management 122 (4) (1996) 306–318.
