EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 229 Source: Financial Models Using Simulation and Optimization by Wayne Winston
USING ANALYTICAL HIERARCHY PROCESS TO SELECT A JOB
When multiple objectives are important to a Decision Maker,
it is often difficult to choose between alternatives.
Example: You are choosing a job. One might offer the highest starting salary, but rate
poorly on your other objectives: quality of life, closeness to your family. Another
job offer might rate highly on these latter objectives but has a relatively low
starting salary. Which one do you choose?
THOMAS SAATY’s ANALYTICAL HIERARCHY PROCESS Provides a powerful tool that can be used to make decisions
when multiple objectives are present.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 230 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Four Objectives: Objective 1: High Starting Salary
Objective 2: Quality of life in city where job is located
Objective 3: Interest of Work
Objective 4: Nearness of job to family.
• The AHP process generates a weight for each objective. By convention the
weights sum up to 1.
Weights For Objectives:
OBJECTIVE DESCRIPTION Weight Objective 1 Salary 0.5115 Objective 2 Life quality 0.0986 Objective 3 Work interest 0.2433 Objective 4 Near family 0.1466
TOTAL WEIGHT 1.0000
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 231 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CONCLUSION:
• High Starting Salary (HSS) is your most important objective.
• HSS is twice as important as the interest of the work.
• HSS is approximately three times as important as nearness to your family.
• HSS is approximately five times as important Quality of Lie in the city where
you work.
YOU HAVE OBTAINED THREE JOB OFFERS The following Table described how each Job scores on the four objectives.
Determining best job Matrix of scores
Salary Life quality Work interest Near familyJob 1 0.5714 0.1593 0.0882 0.0824 Job 2 0.2857 0.2519 0.6687 0.3151 Job 3 0.1429 0.5889 0.2431 0.6025
TOTAL WEIGHT1.0000 1.0000 1.0000 1.0000
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 232 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CONCLUSION:
• Job 1 is twice as important in terms of salary as job 2 and three times as
important as job 3.
• Job 3 is twice as important in terms of Quality of Life as job 2 and three
times as important as job 1.
• Job 2 is more than twice as important in terms of Interesting Work as job 3
and eight times as important as job 1.
WHICH ONE DO YOU CHOOSE?
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 233 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CALCULATE COMBINED SCORES:
Job 1 Score = 0.5115*0.5714+0.0986*0.1593+
0.2433*0.0882+0.1466*0.0824 = 0.3415
Job 2 Score = 0.5115*0.2857+0.0986*0.2519+
0.2433*0.6687+0.1466*0.3151 = 0.3799
Job 3 Score = 0.5115*0.1429+0.0986*0.5889+
0.2431*0.6687+0.1466*0.6025 = 0.2786
Conclusion: Job 2 gets the highest score. Hence, the analysis above recommends taking job
number 2.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 234 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Note that:
• Combined Score for Jobs is a Matrix-Vector product that can be easily
calculated using the MMULT function in Excel.
Inputs to the above analysis are:
• Weights for each objective
• Scores for each Alternative on each of the Objectives
OUTPUT OF ANALYTICAL HIERARCHY PROCESS PROVIDES THESE WEIGHTS AND SCORES!!
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 235 Source: Financial Models Using Simulation and Optimization by Wayne Winston
PAIRWISE COMPARISON MATRIX OF OBJECTIVES Suppose we have N objectives. Then consider the N*N Matrix A, such that:
=
NNN
N
aa
aaA
,1,
,11,1
where
ai,j= “Importance of Objective i compared to Objective j.
BUT HOW DO WE GET SPECIFIC VALUES FOR ai,j?
<
>=
j Objective asImportant less is i Objective1j Objective asImportant as is i Objective1
j Objective thanimportant more is i Objective1
, jia
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 236 Source: Financial Models Using Simulation and Optimization by Wayne Winston
ANSWER: THROUGH EXPERT JUDGEMENT QUESTIONNAIRES
INTERMEZZO: WHAT IS EXPERT JUDGMENT?
IT IS NOT! A Group of Experts in
a Room deciding on Numbers.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 237 Source: Financial Models Using Simulation and Optimization by Wayne Winston
STRUCTURED APPROACH TO CAPTURING AN EXPERTS KNOWLEDGE BASE
AND CONVERT HIS KNOWLEDGE BASEINTO QUANTITATIVE ASSESSMENTS.
EXPERTS
SUBSTANTIVE
NORMATIVE
1. DECOMPOSITION OF EVENT OF INTERESTTO A MEANINGFULL
LEVEL FOR SUBSTANTIVE EXPERT
KNOWLEDGABLE ABOUT THE SUBJECT MATTER
AND EXTENSIVE EXPERIENCE
MODELERS SKILLED IN DECOMPOSITION
AND AGGREGATION OF ASSESSMENTS
EXPERT JUDGEMENT ELICITATION PROCEDURE
2. ELICITATION OF JUDGMENT OF
SUBSTANTIVE EXPERT FACILITATED
BY NORMATIVE EXPERT
3. AGGREGATION OF JUDGEMENTS
BY NORMATIVE EXPERT
ELICITATION PROCESS =MULTIPLE CYCLES
(AT LEAST 2)
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 238 Source: Financial Models Using Simulation and Optimization by Wayne Winston
EXAMPLE: THE DELPHI METHOD
• Early 1950: Developed by RAND Corporation as spin-off of an Air Force
Research Rroject, "Project Delphi".
• 1963: Wider audience due to 1963 RAND Study "Report on a long-range
Forecasting study".
Probably, best known method to date of
Eliciting and synthesizing expert judgment.
STEP 1: Monitoring Team defines set of issues and selects sets of Respondents who are
experts on the issues in question. Respondents do not know who other
respondents are, and the responses are anonymous. Preliminary questionnaire
is sent for comments, which are then used to establish a definitive questionnaire.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 239 Source: Financial Models Using Simulation and Optimization by Wayne Winston
STEP 2: Questionnaire is sent to respondents. Monitoring Team analyses the answers.
STEP 3: The set of responses is sent back together with 25% lower and 25% upper
responses. The respondents are asked if they wish to revise the initial
predictions. Those who answered outside of the above range are asked to give
arguments.
STEP 4: The revised predictions are analyzed by the monitoring team and the outliers for
arguments are summarized. GOTO STEP 2.
TYPICALLY THREE ROUNDS!
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 240 Source: Financial Models Using Simulation and Optimization by Wayne Winston
EXAMPLE OF DELPHI QUESTIONNAIRE # 1
Questionnaire # 1 This is the first in a series of four questionnaires intended to demonstrate the use of the
Delphi Technique in obtaining reasoned opinions from a group of respondents.
Each of the following six questions is concerned with developments in the United States
with the next few decades.
In addition to giving your answer to each question, you are also being asked to rank the
questions from 1 to 7. Here “1” means that in comparing your ability to answer this question
with what you expect the ability of the other participants to be, you feel that you have the
relatively best chance of coming closer to the truth than most of the others, while a “7”
means that you regard that chance as relative least.
Source: Helmer (1968)
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 241 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Rank Answer*
1. In your opinion, in what y ear will the median familiy income (in1967 dollars) reach twice its present amount?2. In what y ear will the percentage of electric automobiles along allautomobile in use reach 50 percent?3. In what y ear will the percentage of households that are equippedwith computer consoles tied to a central computer and databankreach 50 percent?4. By what y ear will the per-capita amount of personal cash transactions(in 1967 dollars) be reduced to one-tenth of what it is now?5. In what y ear will power generation be thermonuclear fusion becomecomercially competitive with hydroelectric power?6. By what y ear will it be possible by commercial carriers to get fromNew York's Time Square to San Fransisco's Union Suare in half thetime that is now required to make that trip?7. In what y ear will a man for the first time travel to the Moon, stayfor at least 1 month, and return to earth?* "Never" is alson an acceptable answer
Please also answer the following question, and give your name (this is for identification purposesduring the exercise only ; no opinions will be attributed to a particular person).
Check One: I would like
I am willing but no anxious
I would prefer not
participate in the three remaining qustionnaires
Name (block letter please):………………………………………..
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 242 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CRITIQUE ON DELPHI METHOD: (Sackman's Delphi Critique (1975))
Methodological
• Questions are vague are often so vague that it would be impossible to
determine when, if ever, they occurred.
• Furthermore, the respondents are not treated equally.
• Many dropouts. No Explanation for # of dropouts given or researched, nor are
effects assessed on eventual assessment. Does Delphi convergence because
of boredom in stead of consensus?
• Sackman argues that experts and non experts produce comparable results.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 243 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Comparison to other Methods (Delbecq, Van de Ven, and Gusstafson, 1975);
• Method 1: "nominal group technique"; participants confront each other directly
in a controlled environment.
• Method 2: "no interaction model”; initial assessments are simply aggregated
mathematically.
Results:
• Nominal group technique superior to the others,
• Delphi worst of the three.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 244 Source: Financial Models Using Simulation and Optimization by Wayne Winston
EXPERT JUDGMENT ELICITATION PRINCIPLES (Source: Experts in Uncertainty, Roger M. Cooke)
1. Reproducibility: It must be possible for scientific peers to review and if necessary reproduce all
calculations. This entails that the calculation model must be fully specified and
the ingredient data must be made available.
2. Accountability: The source of Expert Judgment must be identified.
3. Empirical Control: Expert probability assessment must in principle be susceptible to empirical
control.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 245 Source: Financial Models Using Simulation and Optimization by Wayne Winston
4. Neutrality: The method for combining/evaluating expert judgements should encourage
experts to state true opinions.
5. Fairness: All Experts are treated equally, prior to processing the results of observations
PRACTICAL EXPERT JUDGMENT
ELICITATION GUIDELINES
1. The questions must be clear 2. Prepare an attractive format for the questions and graphic format for the
answers
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 246 Source: Financial Models Using Simulation and Optimization by Wayne Winston
3. Perform a dry run 4. An Analyst must be present during the elicitation 5. Prepare a brief explanation of the elicitation format, and of the model for
processing the responses. 6. Avoid Coaching 7. The elicitation session should not exceed 1 hour.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 247 Source: Financial Models Using Simulation and Optimization by Wayne Winston
ELICITATION PROCEDURES
• Direct Procedures: Ask for Probabilities\Measures of Central
Tendency\Measures of Variability
• Indirect Procedures: Use Betting Strategies; Paired Comparisons
Example Betting Strategies: Indifference when Expected payoffs are the same
1. Betting Stategies
Event: Lakers winning the NBA title this season
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 248 Source: Financial Models Using Simulation and Optimization by Wayne Winston
STEP 1: Offer a person to choose between following the following bets, where
X=100, Y=0.
Bet for Lakers
Lakers Win
Bet against Lakers
Lakers Win
Lakers Loose
Lakers Loose
Max Profit
X
-X
-Y
Y
STEP 2: Offer a person to choose between following the following bets, where
X=0, Y=100. (Consistency Check)
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 249 Source: Financial Models Using Simulation and Optimization by Wayne Winston
STEP 3: Offer a person to choose between following the following bets, where
X=100, Y=50.
STEP 4: Offer a person to choose between following the following bets, where
X=50, Y=100. (Consistency Check)
Continue until point of indifference has been reached.
Assumption:
When a person is indifferent between bets the
expected payoffs from the bets must be the same.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 250 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Thus:
X*Pr(LW) – Y*Pr(LL)= -X*Pr(LW) + Y*Pr(LL) ⇔
2*X*Pr(LW) – 2*Y*(1- Pr(LW) )=0 ⇔
Pr(LW) = YXY+ .
Example: X=100, Y=50 ⇒ Pr(LW)= 13 ≈ 33.33%
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 251 Source: Financial Models Using Simulation and Optimization by Wayne Winston
2. Paired Comparisons of Situations
Issaquah class ferry on the Bremerton to Seattle route in acrossing situation within 15 minutes, no other vessels around,
good visibility, negligible wind.
Other vessel is a navy vessel Other vessel is a product tanker
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 252 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Question: 1 89
Situation 1 Attribute Situation 2 Issaquah Ferry Class -
SEA-BRE(A) Ferry Route - Navy 1st Interacting Vessel Product Tanker
Crossing Traffic Scenario 1st Vessel - 0.5 – 5 miles Traffic Proximity 1st Vessel -
No Vessel 2nd Interacting Vessel - No Vessel Traffic Scenario 2nd Vessel - No Vessel Traffic Proximity 2nd Vessel - > 0.5 Miles Visibility - Along Ferry Wind Direction -
0 Wind Speed - Likelihood of Collision Avoidance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9
Situation 1 is worse <====================X====================> Situation 2 is worse
9: VERY MUCH MORE LIKELY to result in a collision. 7: MUCH MORE LIKELY to result in a collision. 5: MODERATELY LIKELY to result in a collision. 3: SOMEWHAT MORE LIKELY to result in a collision. 1: EQUALY LIKELY to result in a collision.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 253 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Underlying Model for Paired Comparison Questionnaire
)(0
1 1
)X Failure,Propulsion|AccidentPr( XYBTeP=
1X 1 Scenario Traffic = 2X 2 Scenario Traffic =
nsinteractioway -2 includingVector )X(Y =
Paired Comparison
( ) )()(
)2(0
)(0
2
121
1
)X Failure,Prop.|AccidentPr()X Failure,. Prop|AccidentPr( XYXY
XY
XYT
T
T
eePeP −== β
β
β
( ) )()( )X Failure,Prop.|AccidentPr()X Failure,. Prop|AccidentPr( 21
2
1
XYXYLN T −=
β
1
2
3
4
Calibration to Accident Data
Calibrate to convert relative probabilities to absolute probabilities by solving for
P0 e.g. fix the total number of expected collisions over a given time period.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 254 Source: Financial Models Using Simulation and Optimization by Wayne Winston
BACK TO THE ANALYTICAL HIERARCHY PROCESS
PAIRWISE COMPARISON MATRIX OF OBJECTIVES TO OBTAIN WEIGHTS FOR OBJECTIVES
Suppose we have N objectives. Then consider the N*N Matrix A, such that:
=
NNN
N
aa
aaA
,1,
,11,1
where
ai,j= “How much more important is objective
i compared to Objective j.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 255 Source: Financial Models Using Simulation and Optimization by Wayne Winston
<
>=
j Objective asImportant less is i Objective1j Objective asImportant as is i Objective1
j Objective thanimportant more is i Objective1
, jia
ASSUMPTION:
• There exists a set of numbers iw , i=1,…, n such that:
iw = Relative importance of objective i compared to the other objectives.
j
ijii
n
ii w
waww =>=∑=
,1
,0,1
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 256 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Note that:
ji
j
ii
jij a
www
wa
,,
11===
BUT HOW DO WE GET SPECIFIC VALUES FOR ai,j? STEP 1: Introduce a quantitative scale for measuring importance
1: Equally Important
3: Slightly More Important
5: Strongly More Important
7: Very Strongly More Important
9: Absolutely More Important
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 257 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CRITIQUE: Where does this scale come from? Why five categories and why a
scale from 1 to 9? Do the analysis result depend one these choices?
ANSWER: There is instability in the final scores that are being calculated and in
the RANKINGS of the alternatives as well (AHP is not perfect, but very practical
and widely applied method).
STEP 2: Develop a questionnaire. Use an attractive graphical format for the questions.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 258 Source: Financial Models Using Simulation and Optimization by Wayne Winston
For example:
Objective ObjectiveQuality of Life Starting Salary
LHS is Preferred RHS is Preferred
9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9
Abs
olut
ely
Mor
e Im
porta
nt
Ver
y S
trong
ly M
ore
Impo
rtan
Stro
ngly
Mor
e Im
porta
nt
Slig
htly
Mor
e Im
porta
nt
Equ
ally
Impo
rtant
Slig
htly
Mor
e Im
porta
nt
Stro
ngly
Mor
e Im
porta
nt
Ver
y S
trong
ly M
ore
Impo
rtan
Abs
olut
ely
Mor
e Im
porta
nt
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 259 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Total number of objectives is N.
What is the number of pairwise comparison you need to ask?
ANSWER:
2N
Answers are typically summarized in a Matrix Form
=
1212
41
21221
21
211
51
4251
A
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 260 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Back to our Job Selection Example: HOW DO WE CALCULATE THE WEIGHT VECTOR
Tnwwww ),,,( 21=
FROM THIS MATRIX?
Consider the case of 4 Objectives:
=
4
4
3
4
2
4
1
4
4
3
3
3
2
3
1
3
4
2
3
2
2
2
1
2
4
1
3
1
2
1
1
1
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
A
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 261 Source: Financial Models Using Simulation and Optimization by Wayne Winston
STEP 3. Calculate:
jj
N
iiN
i j
iN
iji ww
w
wwa 11
11, ===
∑∑∑ =
==
STEP 4. Calculate Normalized matrix ANorm with elements
i
j
j
i
j
jiN
iji
jiji w
w
ww
w
a
a
aa ====
∑=
11~ ,
1,
,,
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 262 Source: Financial Models Using Simulation and Optimization by Wayne Winston
=
4444
3333
2222
1111
wwwwwwwwwwwwwwww
ANorm
If Subjective Judgment is perfect the top
the elements in each row should be the same
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 263 Source: Financial Models Using Simulation and Optimization by Wayne Winston
ANSWER MATRIX
Salary Life quality Work interest Near family Salary 1 5 2 4 Life quality 1/5 1 1/2 1/2 Work interest 1/2 2 1 2 Near family 1/4 2 1/2 1 Sum 1.95 10 4 7.5
“NORMALIZED MATRIX”
Salary Life quality Work interest Near family
Salary 0.513 0.500 0.500 0.533
Life quality 0.103 0.100 0.125 0.067
Work interest 0.256 0.200 0.250 0.267
Near family 0.128 0.200 0.125 0.133
Sum 1.0000 1.0000 1.0000 1.0000
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 264 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CONCLUSION: Subjective Judgment is NOT PERFECT
STEP 4. Estimate Weight iw for objective i such that:
∑=
=N
jjii a
Nw
1,
~1
Note that:
∑ ∑∑= ==
=
N
i
N
jji
N
ii a
Nw
1 1,
1
~1
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 265 Source: Financial Models Using Simulation and Optimization by Wayne Winston
∑∑∑
∑∑∑ = =
=
= =
=
=
=N
j
N
iN
iji
jiN
i
N
jN
iji
ji
a
aNa
aN 1 1
1,
,
1 1
1,
, 11
∑∑ ∑∑ == =
=
==
=
N
j
N
j
N
ijiN
iji
Na
aN 11 1,
1,
11111
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 266 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Back to our Job Selection Example:
Normalized matrix
Salary Life quality Work interest Near family Weights
Salary 0.513 0.500 0.500 0.533 0.5115
Life quality 0.103 0.100 0.125 0.067 0.0986
Work interest 0.256 0.200 0.250 0.267 0.2433
Near family 0.128 0.200 0.125 0.133 0.1466
Sum 1.0000 1.0000 1.0000 1.0000 1
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 267 Source: Financial Models Using Simulation and Optimization by Wayne Winston
CHECKING CONSISTENCY IN JUDGMENTS
HENCE!
=
4
4
3
4
2
4
1
4
4
3
3
3
2
3
1
3
4
2
3
2
2
2
1
2
4
1
3
1
2
1
1
1
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
ww
A 1 2 3 4( , , , )Tw w w w w=
1
2
3
4
4444
ww
Awww
=
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 268 Source: Financial Models Using Simulation and Optimization by Wayne Winston
THEREFORE:
4
4
4
4
4
4
4
3
3
2
2
1
1
=
=
wwwwwwww
wwA
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 269 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Back to our Job Selection Example:
Product Ratios
2.0774 4.0611
0.3958 4.0161
0.9894 4.0672
0.5933 4.0459
CI 0.0159
Consistency Index:
1-nn - s Ratio'Average
=CI
Note That: When expert judgment is perfect CI should be zero
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 270 Source: Financial Models Using Simulation and Optimization by Wayne Winston
• Suppose an Expert would be filling out the questionnaire at random and we
would calculate the associated CI, what would that value be?
The experiment above can be conducted using computerized random answer. By
conducting this experiment a great number of times one can calculate:
RI – INDEX = average CI – index.
If pairwise comparison matrix is an N*N Matrix the following RI’ indices have
been calculated:
n 2 3 4 5 6 7 8 9 10 RI 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.51
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 271 Source: Financial Models Using Simulation and Optimization by Wayne Winston
THOMAS SAATY’s suggests:
>
<
nciesInconsiste Serious10.0
rysatisfactoy consistenc of Degree10.0
RICIRICI
Back to our Job Selection Example:
• CI = 0.0159
• RI=0.9
• CI/RI=0.0176
CONCLUSION : JUDGEMENT IS CONSISTENT
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 272 Source: Financial Models Using Simulation and Optimization by Wayne Winston
PAIRWISE COMPARISON MATRIX OF ALTERNATIVES TO OBTAIN SCORES FOR ALTERNATIVE IN EACH OBJECTIVE
Suppose we have N objectives and M alternatives. Then consider the N matrices
Ai of size M*M, such that:
=
MMM
M
i
aa
aaA
,1,
,11,1
where
ai,j= “How much more important is alternative
i compared to to alternative j in objective i.
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 273 Source: Financial Models Using Simulation and Optimization by Wayne Winston
<
>=
j eAlternativ asImportant less is i eAlternativ1j eAlternativ asImportant as is i eAtlernativ1
j eAlterantiv thanimportant more is i eAlternativ1
, jia
ASSUMPTION:
• There exists a set of numbers iw , i=1,…, n such that:
iw = Relative importance of alternative i compared to the other objectives.
j
ijii
n
ii w
waww =>=∑=
,1
,0,1
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 274 Source: Financial Models Using Simulation and Optimization by Wayne Winston
Note that:
ji
j
ii
jij a
www
wa
,,
11===
BUT HOW DO WE GET SPECIFIC VALUES FOR ai,j?
ANSWER:
Same as before through
Paired Comparison Questionnaire
EMSE 388 – Quantitative Methods in Cost Engineering
Lecture Notes by Instructor: Dr. J. Rene van Dorp Chapter 20 - Page 275 Source: Financial Models Using Simulation and Optimization by Wayne Winston
OBJECTIVE: STARTING SALARYObjective ObjectiveJob 1 Job 2
LHS is Preferred RHS is Preferred
9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9
Abs
olut
ely
Mor
e Im
porta
nt
Ver
y S
trong
ly M
ore
Impo
rtan
Stro
ngly
Mor
e Im
porta
nt
Slig
htly
Mor
e Im
porta
nt
Equ
ally
Impo
rtant
Slig
htly
Mor
e Im
porta
nt
Stro
ngly
Mor
e Im
porta
nt
Ver
y S
trong
ly M
ore
Impo
rtan
Abs
olut
ely
Mor
e Im
porta
nt