Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | matthew-robinson |
View: | 239 times |
Download: | 0 times |
Why composite should be non compensatory
Giuseppe MundaUniversitat Autonoma de Barcelona
Dept.of Economics and Economic History
2
3
Step 1. Developing a theoretical
framework
Step 2. Selecting indicators
Step 3. Multivariate analysis
Step 4. Imputation of missing data
Step 5. Normalisation of data
Step 6. Weighting and aggregation
Step 7. Robustness and sensitivity
Step 8. Association with other variables
Step 9. Back to the details (indicators)
Step 10. Presentation and dissemination
Index construction- Steps[OECD-JRC Handbook]
4
Based on:
5
Structure of the presentation
Linear Aggregation Rule•Social Choice and Aggregation Rules•Non-Compensatory Aggregation Rules•Conclusions
6
Linear Aggregation Rule
7
Linear aggregation rules and preference independence
The variables X1, X2,..., Xn are mutually preferentially independent if every subset Y of these variables is preferentially independent of its complementary set of evaluators. The following theorem holds:
given the variables X1, X2, ..., Xn, an additive aggregation function exists if and only if these variables are mutually preferentially independent.
8
Preferential independence implies that the trade-off ratio between two variables is independent of the values of the n-2 other variables, i.e.
, 0x ySz
x, y Y ,z Z
Preferential independence is a very strong condition from both the operational andepistemologicalpoints of view.
9
The meaning of weights in linear aggregation rules
• “Greater weight should be given to components which are considered to be more significant in the context of the particular composite indicator”. (OECD, 2003, p. 10).
• Weights as symmetrical importance, that is "… if we have two non-equal numbers to construct a vector in R2, then it is preferable to place the greatest number in the position corresponding to the most important criterion." (Podinovskii, 1994, p. 241).
10
To use the compensatory approach in practice, we have to determine for each normalised variable, a mapping : RX ii which provides an interval scale of
measurement and to assess scaling constants in order to specify how the compensability should be accomplished, given the scales i between the different variables. Note that
the scaling constants appearing in the compensatory approach depend on the scales i ;
thus they do not characterise the intrinsic relative importance of the indicators (Roberts, 1979).
11
The aggregation of several criteria implies taking a position on the fundamental issue of compensability.
Compensability refers to the existence of trade-offs, i.e. the possibility of offsetting a disadvantage on some criteria by an advantage on another criterion.
E.g. in the construction of a composite indicator of environmental sustainability a compensatory logic (using equal weighting) would imply that one is willing to accept, let’s say, 10% more CO2 emissions in exchange of a 3% increase in GDP.
12
The implication is the existence of a theoretical inconsistency in the way weights are actually used and their real theoretical meaning.
For the weights to be interpreted as “importance coefficients ” (the greatest weight is placed on the most important “dimension”) non-compensatory aggregation procedures must be used to construct composite indicators.
Linear Aggregation is a correect rule iff
•Mutual preferential independence applies•Compensability is desirable•Weights are derived as trade-offs
14
Social Choice and Aggregation Rules
15
The Plurality Rule
Number of criteria 3 5 7 6 a a b c
16
Example (rearranged from Moulin, 1988, p. 228; 21 criteria and 4
alternatives )Objective: find best country
A first possibility: apply the plurality rule the country which is more often ranked in the first position is the winning one.
Country a is the best.
BUT Country a is also the one with the strongest opposition since 13 indicators put it into the last position!
# of indicators 3 5 7 6
1st position a a b c
2nd position b c d b
3rd position c b c d
4th position d d a a
This paradox was the starting step of Borda’s and Condorcet’s research at the end of the 18th century, but the plurality rule corresponds to the most common electoral system in the 21st century!!!
17
Borda approach
Rank a b c d Points
1st 8 7 6 0 3
2nd 0 9 5 7 2
3rd 0 5 10 6 1
4th 13 0 0 8 0
Frequency matrix (21 criteria 4 alternatives)
Jean-Charles, chevalier de Borda
18
Borda approach
Borda solution: bcad
Now: country b is the best, not a (as in the case of the plurality rule)
Rank a b c d Points
1st 8 7 6 0 3
2nd 0 9 5 7 2
3rd 0 5 10 6 1
4th 13 0 0 8 0
Frequency matrix (21 criteria 4 alternatives)
8 3 24
5 9 2 7 3 44
10 5 2 6 3 38
6 7 2 20
a
b
c
d
Borda score:
The plurality rule paradox has been solved.
19
Condorcet approach
0 8 8 8
13 0 10 21
13 11 0 14
13 0 7 0
a b c d
a
b
c
d
Outranking matrix (21 criteria 4 alternatives)
Jean-Antoine-Nicolas de Caritat, Marquis de Condorcet
20
Condorcet approach
• For each pair of countries a concordance index is computed by counting how many individual indicators are in favour of each country (e.g. b better than a 13 times).
• “constant sum property” in the outranking matrix (13+8=21)
0 8 8 8
13 0 10 21
13 11 0 14
13 0 7 0
a b c d
a
b
c
d
Outranking matrix (21 criteria 4 alternatives)
21
Condorcet approach
• Pairs with concordance index > 50% of the indicators are selected.
• Majority threshold = 11 (i.e. a number of individual indicators bigger than the 50% of the indicators considered)
• Pairs with a concordance index ≥ 11: bPa= 13, bPd=21(=always), cPa=13, cPb=11, cPd=14, dPa=13.
0 8 8 8
13 0 10 21
13 11 0 14
13 0 7 0
a b c d
a
b
c
d
Condorcet solution: c b d aYet, the derivation of a Condorcet ranking may sometimes be a long and complex computation process.
Outranking matrix (21 criteria 4 alternatives)
22
Which approach should one prefer?
Both Borda and Condorcet approaches solve the plurality rule paradox. However, the solutions offered are different.
Borda solution: b c a dCondorcet solution: c b d a
In the framework of composite indicators, can we choose between Borda and Condorcet on some theoretical and/or practical grounds?
23
An Original Condorcet’s Numerical Example
Number of criteria 23 17 2 10 8 a b b c c b c a a b c a c b a
24
Number of indicators
23 17 2 10 8
1st a b b c c
2nd b c a a b
3rd c a c b a
Rank a b c Points
1st 23 19 18 2
2nd 12 31 17 1
3rd 25 10 25 0
0 33 25
27 0 42
35 18 0
a b c
a
b
c
Example with 60 indicators
[Condorcet, 1785]
Frequency matrix
Outranking matrix
58, 69, 53a b c
concordance threshold =31 aPb, bPc and cPa (cycle)???
Borda approach Condorcet approach
25
From this example we might conclude that the Borda rule (or any scoring rule) is more effective since a country is always selected while the Condorcet one sometimes leads to an irreducible indecisiveness.
However Borda rules have other drawbacks, too
26
• Both social choice literature and multi-criteria decision theory agree that whenever the majority rule can be operationalized, it should be applied. However, the majority rule often produces undesirable intransitivities, thus “more limited ambitions are compulsory. The next highest ambition for an aggregation algorithm is to be Condorcet” (Arrow and Raynaud, 1986, p. 77).
27
Non-Compensatory Aggregation Rules
28
Condorcet approach
Basic problem: presence of cycles, i.e. aPb, bPc and cPa
The probability of obtaining a cycle increases with both N (indicators) and M (countries) [Fishburn (1973, p. 95)]
29
Condorcet approach
Condorcet himself was aware of this problem (he built examples to explain it) and he was even close to find a consistent rule able to rank any number of alternatives when cycles are present…
… but Maximilien de Robespierre
30
Condorcet approachBasic problem: presence of cycles, i.e. aPb, bPc and
cPa
Furter attempts made by Kemeny (1959) and by Young and Levenglick (1978) … led to:
Condorcet-Kemeny-Young-Levenglick (C-K-Y-L) ranking procedure
31
C-K-Y-L ranking procedure
Main methodological foundation: maximum likelihood concept.
The maximum likelihood principle selects as a final ranking the one with the maximum pair-wise support.
What does this mean and how does it work?
Indic. GDP Unemp. Rate
Solid wastes
Income dispar.
Crime rate
Country
A 25,000 0.15 0.4 9.2 40
B 45,000 0.10 0.7 13.2 52
C 20,000 0.08 0.35 5.3 80
weights .166 .166 0.333 .166 .166AB = 0.333+0.166+0.166=0.666
BA = 0.166+0.166=0.333
AC = 0.166+0.166=0.333
CA = 0.166+0.333+0.166=0.666
BC = 0.166+0.166=0.333
CB = 0.166+0.333+0.166=0.666
C-K-Y-L ranking procedure
Indic. GDP Unemp. Rate
Solid wastes
Income dispar.
Crime rate
Country
A 25,000 0.15 0.4 9.2 40
B 45,000 0.10 0.7 13.2 52
C 20,000 0.08 0.35 5.3 80
weights .166 .166 0.333 .166 .166
A
B
C
A B C
0 0.666 0.333
0.333 0 0.333
0.666 0.666 0
C-K-Y-L ranking procedure
AB = 0.333+0.166+0.166=0.666
BA = 0.166+0.166=0.333
AC = 0.166+0.166=0.333
CA = 0.166+0.333+0.166=0.666
BC = 0.166+0.166=0.333
CB = 0.166+0.333+0.166=0.666
Indic. GDP Unemp. Rate
Solid wastes
Income dispar.
Crime rate
Country
A 25,000 0.15 0.4 9.2 40
B 45,000 0.10 0.7 13.2 52
C 20,000 0.08 0.35 5.3 80
weights .166 .166 0.333 .166 .166
A
B
C
A B C
0 0.666 0.333
0.333 0 0.333
0.666 0.666 0
ABC = 0.666 + 0.333 + 0.333 = 1.333
BCA = 0.333 + 0.666 + 0.333 = 1.333
CAB = 0.666 + 0.666 + 0.666 = 2
ACB = 0.333 + 0.666 + 0.666 = 1.666
BAC = 0.333 + 0.333 + 0.333 = 1
CBA = 0.666 + 0.333 + 0.666 = 1.666
C-K-Y-L ranking procedure
35
The Computational problem
The only drawback of this aggregation method is the difficulty in computing it when the number of candidates grows.
With only 10 countries 10! = 3,628,800 permutations (instead of 3!=6 of the example)
To solve this problem of course there is a need to use numerical algorithms
C-K-Y-L ranking procedure
36
Conclusions
Compensabiliity is a fundamental concept in choosing an
aggregation rule
38
Aggregation ESI rank
with LIN rank with NCMC
Change in Rank
Azerbaijan 99 61 38
Spain 76 45 31
Nigeria 98 69 29 South Africa 93 68 25 Im
prov
emen
t
Burundi 130 107 23
Indonesia 75 114 39
Armenia 44 79 35
Ecuador 51 78 27
Turkey 91 115 24
Det
erio
ratio
n
Sri Lanka 79 101 22
Average change over 146 countries 8
Table 3. ESI rankings obtained by linear aggregation (LIN) and non-compensatory rule (NCMC): countries that largely improve or worsen their rank position
39
0
5
10
15
20
25
Lom
bard
y (I
T)
Mad
rid
To
sca
ny (
IT)
Nav
arra
Cat
alo
nia
Ven
eto
(IT
)
Rio
ja
Bal
ear
ic Is
land
s
Pai
s V
asco
Ara
gon
Val
enc
ia
Mur
cia
Sic
ily (
IT)
Can
tab
ria
Cas
tilla
y L
eon
And
alu
cia
Cas
tilla
la M
anch
a
Can
ary
Isla
nds
Krit
i (G
R)
Atti
ki (
GR
)
Ast
uria
s
Ext
rem
adu
ra
Ga
licia
St.
Ella
da (
GR
)
Th
essa
lia (
GR
)
Mu
lti-
Cri
teri
a R
ank
(30
sce
nar
ia)
Figure 1: Multi-criteria based ranking1
40
0
5
10
15
20
25
Na
varr
a
Ma
drid
Lo
mb
ard
y (I
T)
Pa
is V
asc
o
To
sca
ny
(IT
)
Ve
ne
to (
IT)
Rio
ja
Att
iki (
GR
)
Krit
i (G
R)
Ara
go
n
Ext
rem
ad
ura
Ca
talo
nia
Sic
ily (
IT)
Mu
rcia
Ba
lea
ric I
sla
nd
s
Va
len
cia
Th
ess
alia
(G
R)
Ca
na
ry I
sla
nd
s
Ca
still
a y
Le
on
Ca
nta
bria
An
da
luci
a
Ast
uria
s
St.
Ella
da
(G
R)
Ca
still
a la
Ma
nch
a
Ga
licia
Lin
ea
r R
an
k(3
0 s
ce
na
ria
)
Figure 2: Additive (linear) based ranking1
Using Borda too
The KEI example …
42
Methodological scenarios for the development of the KEI composite
Aggregation function Additive
(linear) Multiplicative (geometric averaging)
Non-compensatory multi-criteria analysis
All (total 29) Sub-dimensions included One-at-a-time excluded
All (total 7) Dimensions included One-at-a-time excluded
Preserved Pillar Structure
Not preserved z-scores Min-max Normalisation Raw data Factor analysis Equal weighting Weighting Data envelopment analysis
The frequency matrix (Borda type approach) of a country’s rank in each of the seven dimensions and the overall KEI was calculated across the ~2,000 scenarios~2,000 scenarios.
43
Ran
k 1
Ran
k 2
Ran
k 3
Ran
k 4
Ran
k 5
Ran
k 6
Ran
k 7
Ran
k 8
Ran
k 9
Ran
k 10
Ran
k 11
Ran
k 12
Ran
k 13
Ran
k 14
Ran
k 15
Ran
k 16
Ran
k 17
Ran
k 18
Ran
k 19
Ran
k 20
Ran
k 21
Ran
k 22
Ran
k 23
Ran
k 24
Ran
k 25
Ran
k 26
Ran
k 27
Ran
k 28
Ran
k 29
Sweden 54 46Denmark 55 30 14Luxembourg 36 4 14 25 4 7 7 4Finland 18 23 29 9 11 11USA 11 32 2 4 39 9 4Japan 4 7 18 32 36 4UK 2 5 16 38 39Netherlands 86 4 4 7Ireland 4 61 14 4 9 9Austria 18 50 18 7 7Belgium 11 4 11 57 16 2France 4 14 18 11 54EU15 4 57 39EU25 4 4 14 32 39 7Germany 7 79 4 7 4Slovenia 7 41 38 14Estonia 4 36 25 21 11 4Malta 7 13 9 21 23 27Cyprus 36 7 4 23 23 7Spain 4 4 32 25 29 7Czech. Rep. 4 7 30 39 5 7 7Latvia 20 36 11 21 7 5Italy 29 18 9 29 9 7Greece 4 4 4 29 18 21 7 14Lithuania 4 41 13 32 11Hungary 2 13 13 57 2 14Portugal 4 4 7 11 61 14Slovakia 4 7 18 71Poland 100Legend:Frequency lower 15%Frequency between 15 and 30%Frequency between 30 and 50%Frequency greater than 50%
Knowledge Economy Index
44
REFERENCES• Borda J.C. de (1784) – Mémoire sur les élections au scrutin, in
Histoire de l’ Académie Royale des Sciences, Paris.• Brand, D.A., Saisana, M., Rynn, L.A., Pennoni, F., Lowenfels, A.B.
(2007) Comparative Analysis of Alcohol Control Policies in 30 Countries, PLoS Medicine, 0759 April 2007, Volume 4, Issue 4, e151, p. 0752-0759, www.plosmedicine.org
• Condorcet, Marquis de (1785) – Essai sur l’application de l’analyse à la probabilité des décisions rendues à la probabilité des voix, De l’ Imprimerie Royale, Paris.
• Fishburn P.C. (1973) – The theory of social choice, Princeton University Press, Princeton.
• Fishburn P.C. (1984) – Discrete mathematics in voting and group choice, SIAM Journal of Algebraic and Discrete Methods, 5, pp. 263-275.
45
• Munda G. (1995), Multicriteria evaluation in a fuzzy environment, Physica-Verlag, Contributions to Economics Series, Heidelberg.
• Munda G. (2004) – “Social multi-criteria evaluation (SMCE)”: methodological foundations and operational consequences, European Journal of Operational Research, vol. 158/3, pp. 662-677.
• Munda G. (2005a) – Multi-Criteria Decision Analysis and Sustainable Development, in J. Figueira, S. Greco and M. Ehrgott (eds.) –Multiple-criteria decision analysis. State of the art surveys, Springer International Series in Operations Research and Management Science, New York, pp. 953-986.
• Munda G. (2005b) – “Measuring sustainability”: a multi-criterion framework, Environment, Development and Sustainability Vol 7, No. 1, pp. 117-134.
• Munda G. and Nardo M. (2005) - Constructing Consistent Composite Indicators: the Issue of Weights, EUR 21834 EN, Joint Research Centre, Ispra.
• Munda G. and Nardo M. (2007) - Non-compensatory/Non-Linear composite indicators for ranking countries: a defensible setting, Forthcoming in Applied Economics.
• Munda G. (2008) - Social multi-criteria evaluation for a sustainable economy, Springer-Verlag, Heidelberg, New York.