Asymmetrical adjustments in
preference elicitation.
The effect of multicriteria conflict.
Stéphane Deparis, Ph.D.
IBM Research
Ireland Lab
OUTLINE
1. Multicriteria decision aid : normative, descriptive, prescriptive.
2. Multicriteria conflict
3. An experiment on matching judgments
4. Results
5. Recommendation for the practice of elicitation
2
MULTICRITERIA DECISION
Car 14 3.25 0 784
Subway 22 1.70 3 13
Bus 32 1.70 8 352
Taxi 11 18.60 2 637
Walking 50 0 0 30
3
Duration(min) Cost
(Euros)
Total waiting
time
(min)
CO2 emission
(g)
Montparnasse → Etienne Marcel
• Evaluations of the options
• Preferences of the decision-maker
a g1(a) g2(a) … gn(a)
b g1(b) g2(b) … gn(b)
… … … … …
e g1(e) g2(e) … gn(e)
4
g1 g2 … gn
a ↔ (g1(a), g2(a), … , gn(a))
Preference structure:
• total order b > e > d > a > c
• total preorder b > e ~ d > a > c
• partial preorder b > a, d > c ~ e and a ? c
EVALUATION MODEL
TWO TYPES OF PREFERENCE MODELS
SYNTHESIS CRITERION OUTRANKING
Aggregating performances
in a global performance.
ex : additive aggregation
g(a) = ∑ kivi( gi(a) )
Comparisons
Total preorder
g(a) > g(b) = g(c)…
Pairwise comparisons
« Majority vote » by criteria
Aggregating
Incomplete structure
5
m
i=1
aSb ¬(aSb)
bSa a~b a<b
¬(bSa) a>b a?b
AMPLITUDE OF MULTICRITERIA CONFLICT
6
Taxi 11 18.60
Car 14 5.25
Subway 21 1.70
Bus 32 1.70
Walk 50 0
Time
(min)
Cost
(Euros)
Dominance
AMPLITUDE OF MULTICRITERIA CONFLICT
7
Low conflit
Taxi 11 18.60
Car 14 5.25
Subway 21 1.70
Bus 32 1.70
Walk 50 0
Time
(min)
Cost
(Euros)
AMPLITUDE OF MULTICRITERIA CONFLICT
8
High conflit Time
(min)
Cost
(Euros)
Taxi 11 18.60
Car 14 5.25
Subway 21 1.70
Bus 32 1.70
Walk 50 0
THREE APPROACHES IN DECISION-AID
9
Prescriptive Recommendation
Descriptive Real behaviour
ex: Kahneman et Tversky
Normative Rationality
ex: Von Neumann and Morgenstern1944
THREE APPROACHES IN DECISION-AID
10
Prescriptive Recommandation
Descriptive Real behaviour
Normative Rationality
PRESCRIPTIVE APPROACH
11
Considering the preferences of a real decision-maker
to parameterize a formal model.
elicitation
Preferences
Preferential
parameters Set of alternatives
Multicriteria
aggregation
Preference Structure
THREE APPROACHES IN DECISION-AID
12
Prescriptive Recommandation
Descriptive Real behaviour
Normative Rationality
DESCRIPTIVE APPROACH
• Procedural invariance
« Normatively equivalent procedures for assessing
preferences should give rise to the same preference order. »
Kahneman et al. 1988
13
Most important criterion
favored (Kahneman, Sattath, Slovic 1988)
Choice
(a1, a2) ? (b1, b2)
Matching
(a1, a2) I (b1, ? )
EXPERIMENT
16
Objective :
investigate the effect of
multicriteria conflict
on the expression of
incomplete preferences
When conflict induces the expression of incomplete
preferences, European Journal of Operations Research, 221
(2012), pp. 593-602
ISOPREFERENCE CHAINS : A MATTER OF TASTE
17
0
200
400
600
800
1000
1200
14005 4 3 2 1
Ren
t
Distance to city center
attractive
neutral
unattractive
COMPARISONS WITH VARYING CONFLIT
18
ISOPREFERENCE CHAINS : A MATTER OF TASTE
19
0
200
400
600
800
1000
1200
14005 4 3 2 1
Ren
t
Distance to city center
attractive
neutral
unattractive
ISOPREFERENCE CHAINS : A MATTER OF TASTE
20
0
200
400
600
800
1000
1200
14005 4 3 2 1
Ren
t
Distance to city center
attractive
neutral
unattractive
ISOPREFERENCE CHAINS : A MATTER OF TASTE
21
0
200
400
600
800
1000
1200
14005 4 3 2 1
Ren
t
Distance to city center
attractive
neutral
unattractive
RESULTS
24
P 73%
I 9%
R 18% I 21%
P 79%
IMPLICATIONS FOR THE ELICITATION OF PREFERENCES
Pairwise comparisons → trade-off information
(Ratio, Smart, Swing weights)
When there is a large multicriteria conflict between two
options, an « indifference » answer is ambiguous:
equivalent or incomparables ?
Authorize the expression of incomparability
or restrain to low conflict comparisons.
25
OUTLINE
1. Multicriteria decision aid : normative, descriptive, prescriptive.
2. Multicriteria conflict
3. An experiment on matching judgments
4. Results
5. Recommendation for the practice of elicitation
26
EXPERIMENT
27
Objective :
investigate the effect of
multicriteria conflict
on the response asymmetry
during a double matching
DOUBLE MATCHING
28
A
B B
A’ A
Indifference Indifference
forward backward
Asymmetry δ = A’1 – A1
PARTICIPANTS : 29 students (ECP and Paris-Dauphine)
928 double matchings
OPTIONS : Apartments
CRITERIA : Rent (€)
Time to go to the city-center (min)
CONTEXT
29
VARIABLE : DIRECTION OF MATCHING
30
improving
degrading
improving
degrading
VARIABLE : MULTICRITERIA CONFLICT
31
ideal 5
attractive 15
rather attractive 25
neutral 35
rather unattractive 60
unattractive 90
worse 120
criterion: time (min)
Conflict level 1
lowest
VARIABLE : MULTICRITERIA CONFLICT
32
criterion: time (min)
Conflict level 2
ideal 5
attractive 15
rather attractive 25
neutral 35
rather unattractive 60
unattractive 90
worse 120
VARIABLE : MULTICRITERIA CONFLICT
33
criterion: time (min)
Conflict level 4
maximal
ideal 5
attractive 15
rather attractive 25
neutral 35
rather unattractive 60
unattractive 90
worse 120
OBSERVED ASYMMETRY (DEGRADING MATCHINGS)
34 δmoy = -16 min t(28)= -11.1, p<.001
Also observed by
Willemsen et Keren
2003
backward
forward
Commuting time
rent
OBSERVED ASYMETRY (IMPROVING MATCHINGS)
35 δmoy = +11 min t(28)= 13.7, p<.001
Non significant in
Willemsen et Keren
2003
forward
backward
rent
Commuting time
EFFECT OF MULTICRITERIA CONFLICT
36
Effect of conflict : F(3, 81)=142.7, p<.001
Effect of direction : F(1,27)=13.2, p=.001
Interaction : F(3, 81)=7.5, p<.001
unfavourable
favourable
Conflict
EFFECT OF MULTICRITERIA CONFLICT
37
Effect of conflict : F(3, 81)=142.7, p<.001
Effect of direction : F(1,27)=13.2, p=.001
Interaction : F(3, 81)=7.5, p<.001
unfavourable
favourable
Conflict
Asymmetry
towards stimulus,
In minutes
EFFECT SIZE
38
63 min
29 min
EXPERIMENT ON BIMATCHING : SUMMARY
• Greater knowledge of the multicriteria decision behaviour in
response to matching.
• The amplitude of multicriteria conflict magnifies the
asymmetry between forward and backward matchings.
• Effect of strong magnitude. Larger on unfavourable double
matchings.
39
IMPLICATIONS FOR THE ELICITATION OF PREFERENCES
Matching tasks are widely used to measure trade-off
information (ex : Keeney et Raiffa 1976).
• Use low conflict tasks.
• Use favouring matching tasks.
40
STIMULUS AS A REFERENCE POINT
41
|v(-∆l)| = v(+∆t) v(-∆’t) = v(+∆l)
or v(+∆l) < |v(-∆l)|
v(+ ∆’t ) < v(-∆’t) < v(+∆t)
INTRANSITIVITY OF INDIFFERENCE
42
a I b
b I a’ but a’ ∆ a
a I b
b I c
c I d
d I e
but a R e
• G. Fischer et al., Goal-based construction of preferences: Task goals
and the prominence effect, Management Science, 45(8), pp. 1057-1075,
1999
• R.L. Keeney and H. Raiffa. Decisions with multiple objectives :
Preferences and value trade-offs. J. Wiley, New York, 1976.
• M.C. Willemsen and G. Keren. The meaning of indifference in choice
behavior: asymmetries in adjustments embodied in matching.
Organizational Behavior and Human Decision Processes, 90(2), pp.342-
359, 2003.
• A. Tversky, S. Sattath, and P. Slovic. Contingent weighting in judgment
and choice. Psychological review, 95(3):371, 1988.
• J. Von Neumann, O. Morgenstern, A. Rubinstein, and H.W. Kuhn.
Theory of games and economic behavior. Princeton Univ Pr, 2007.
REFERENCES
46