1

THE WORLDWIDE

GOVERNANCE INDICATORS

Methodological and Analytical Issues

Gaia Dallera6 June, 2012

2

INTRODUCTION1. Introduction2. Defining Governance3. Governance Data Sources for the WGI4. Constructing the Aggregate WGI

Measures5. Using and Interpreting the WGI Data6. Analytical Issues7. Conclusions

3

INTRODUCTION The WGI are a research project to develop cross-

country indicators of governance.

We will focus on the methodology and key analytical issues relevant to the overall WGI project.

4

2. DEFINING GOVERNANCE “the traditions and institutions by which authority in

a country is exercised….

..This includes three areas:

1. The process by which governments are selected, monitored and replaced.

2. The capacity of the government to effectively formulate and implement sound policies.

3. The respect of citizens and the state for the institutions that govern economic and social interaction among them”.

5

OPERATIONALIZATION: INDICATORS

No. Dimension Indicators Code

1. Selected, Monitored and Replaced

Voice and Accountability VA

2. Selected, Monitored and Replaced

Political Stability and Absence of Violence/Terrorism

PV

3. Formulate and implement sound policies

Government Effectiveness GE

4. Formulate and implement sound policies

Regulatory Quality RQ

5. Respect of citizens and state for institutions

Rule of Law RL

6. Respect of citizens and state for institutions

Control of Corruption CC

6

3. GOVERNANCE DATA SOURCESThese indicators are based on several hundred variables optained from 31 different data sources capturing governance perception

Availability of the underlying data from (with few exceptions) the individual data sources (transparency and replicability)

The WGI data sources reflect the perceptions of a very diverse group of respondent:

There are four categories: 1. “Surveys” of domestic firms and individuals with first hand

knowledge of the governance situation 2. “Public sector data providers”3. Commercial business information providers4. Non-governamental organizations

7

3. GOVERNANCE DATA SOURCES (CONT’D)

An important qualification is that the sources of data provide different coverage across countries (some cover a majority of countries and some cover small groups of countries).

Each individual variable are not comparable across countries

8

9

4. CONSTRUCTING THE AGGREGATE WGI MEASURES Combines the six aggregate governance indicators using a statistical

tool known as the “Unobserved Component Model” (UCM)

The underlying premise of this statistical approach is straightforward – each of the individual data sources provides an imperfect signal of some deeper underlying notion of governance that is difficult to observe directly

This means that we face a signal extraction problem: How can we isolate an informative signal about the unobserved governance

component common to each individual data source? How can we combine the many data sources to get the best possible signal of

governance in a country? Therefore we construct a linear regression model in order to:

• Standardize the data from these diverse sources into comparable units;• Construct an aggregate indicator of governance as a weighted average;• Construct a margin of error that reflects the unavoidable imprecision in

measuring governance.

10

4. CONSTRUCTING THE AGGREGATE WGI MEASURES (CONT’D) Regression model

ykj=ak +bk(gj+ejk)

Different thing to note about this regression model is that the error term is considered as an independent variable.

“a” and “b” are parameters that reflect the fact that different sources use different units to measure governance.

The error term and precisely its variance captures two sources of uncertainty.

11

4. CONSTRUCTING THE AGGREGATE WGI MEASURES (CONT’D) We consider the estimate of governance as a weighted

average of the re-scaled scores for each country.

E[gj I yj1,…., yjk ]= ∑ wk

The re-scaling puts the observed data from each source into the common units we have chosen for unobserved governance.

The larger the weights, the smaller the variance of the error term.

A crucial observation is that there is an unavoidable uncertainty around this estimate of governance.

yjk - ak

bkk=1

K

12

5. USING AND INTERPRETING THE WGI DATA We report the aggregate WGI measures

in two ways:

• Standard normal units of the governance indicator ranging from -2.5 to +2.5.

• Percentile rank terms ranging from 0 (lowest) to 100 (highest).

13

14

15

5. USING AND INTERPRETING THE WGI DATA (CONT’D)

Size of the confidence intervals varies across countries.

Resulting confidence intervals are substantial relative to the units in which governance is measured.

16

5. USING AND INTERPRETING THE WGI DATA (CONT’D) Many of the small differences in estimates of

governance across countries are not likely to be statistically significant at reasonable confidence intervals, since the confidence intervals are likely to overlap.

Example of overalapping confidence intervals:

Indicator Country

Coefficient

Scale

Control of Corruption

Jamaica 5.0 4.0 to 8.0

Control of Corruption

Peru 6.0 4.0 to 6.0

17

5. USING AND INTERPRETING THE WGI DATA (CONT’D)

This is not to say that aggregate indicators cannot be used to make cross country comparisons.

There are in fact many pair-wise country comparisons that are statistically significant.

63% of the pairwaise comparisons the confidence intervals do not overlap

18

5. USING AND INTERPRETING THE WGI DATA (CONT’D) For example:• 2009 Control of Corruption indicator covers 211

countries;• Total of 21,155 pair-wise comparisons.

% of Compariso

ns

Confidence Intervals

Significance

63% 90% confidence intervals do not overlap

Signals statistically significant differences in the indicator

73% 75% confidence intervals do not overlap

Same as above

19

6. ANALYTICAL ISSUES AGGREGATION METHODOLOGY (CONT’D) UCM has three main advantages:

• It maintains some of the cardinal information in the underlying data.

• It provides a natural framework for weighting the re-scaled indicators by their relative precision.

• It naturally emphasizes the uncertainty associated with aggregate indicators. In fact it formalizes the signal extraction problem and by doing that it provides a rationale for a more inclusive approach to combining data from different types of sources.

20

6. ANALYTICAL ISSUES AGGREGATION METHODOLOGY (CONT’D)

Unbalanced vs balanced samples:

• Refers to the fact that WGI use all available data sources for all countries as opposed to using only those data sources that cover all countries and in all time periods (permitting balanced comparison).

21

6. ANALYTICAL ISSUES:USE OF PERCEPTION DATA This choice is based on the view that

perception data has a particular value in the measurement of governance because:

• Agents (citizens and enterprise) base their action on their perceptions;

• In many areas of governance there are few alternatives to relying on perception data e.g. corruption ;

• The “jure” notion of laws often differs from the de-facto reality.

22

6. ANALYTICAL ISSUES:USE OF PERCEPTION DATA (CONT’D) Potential problems:1. Interpretation of subjective data

• Perception data is imprecise; • But all measures of governance are imprecise proxies for the broader

concept;• Therefore it underscores the importance of using empirical methods

and taking seriously the extent of imprecision. 2. Systematic biases in perception data on governance introduced

by:• Different types of respondents may differ systematically in their

perceptions of the same underlying reality;• The ideological orientation of the organization providing the subjective

assessments of governance. • Possibility that different providers of governance perceptions data rely

on each other’s assessments (correlated perception errors).

• BUT THERE IS LITTLE EVIDENCE OF SUCH BIASES!!!!

23

7. CONCLUSIONS

WGI reports six dimensions of governance covering 200 countries since 1996.

Updated annually.

Based on hundreds of variables from many different data sources.

24

7. CONCLUSIONS (CONT’D)

Due to the inherently unobservable nature of the true level of governance in a country, any observed measure is only a proxy.

Consequence is that our estimates are subject to non-trivial margins of error.

25

7. CONCLUSIONS (CONT’D)

Don’t over interpret small differences in performance (across countries or over time).

Presence of errors does not mean that the WGI cannot be used to make meaningful comparisons.

Estimation of and emphasis on such margins of error is intended to enable users to make more sophisticated use of imperfect information.

26

7. CONCLUSIONS (CONT’D) In fact it is possible to make meaningful

comparisons cross-country and over time.

Almost 67% of all cross-country comparisons in 2009 result in highly significant differences at 90% confidence intervals.

More than one quarter of countries show a significant change in at least one of the six WGI measures during the 2000-2009 decade.

27

THANK YOU