Helpdesk Research Report www.gsdrc.org [email protected]Indicators for conflict, stability, security, justice and peacebuilding Róisín Hinds and Becky Carter 18.06.2015 Question What indicators are being used to measure progress in conflict, stability, security, justice and peacebuilding? Contents 1. Overview 2. Conflict and violence indicators 3. Peacebuilding indicators 4. Stability indicators 5. Justice and safety indicators 6. Performance indicators for crisis response 7. Proxy indicators 8. References 9. Additional information 1. Overview This rapid review identifies some of the current and emerging indicators that are being used to measure progress in conflict, stability, security, justice and peacebuilding. It is not a comprehensive mapping of indicators, but rather a rapid overview of some of the readily available literature on indicators on these themes. There are a range of indicators that have been used to measure progress in these themes, and increasing recognition of the merits of measuring achievements and progress toward goals (Holzapel 2014). Signatories to the New Deal for Fragile States, for instance, committed to developing a set of indicators for
Indicators for conflict, stability, security, justice and peacebuilding
3
peace- and conflict-related indicators are top-down, originate from the global north, and fail to capture
local voices (Mac Ginty 2013b). Existing peace indicators have also been criticised for failing to portray the
often subtle differences within and between communities (Mack 2014).
Indicators for conflict, stability, security, justice and peacebuilding
4
2. Conflict and violence indicators
There are a number of sophisticated instruments and indicators that seek to pre-emptively identify potential incidences of conflict and violence; however, there is a
more limited range of indicators which seek to measure progress made in conflict or violent situations. In a rapid literature review, Walton (2011) provides an
overview of the evidence on conflict early warning systems (CEWS) and identifies some of the most commonly-used indicators of identify the risk of violent conflict.
These systems typically distinguish between ‘long-term structural factors, medium-term proximate or accelerator factors, and short-term trigger factors (Ibid, p. 1).
While longer-term indicators are treated as fairly unproblematic in the literature, shorter-term indicators are treated more cautiously as they typically rely on
qualitative data or locally-generated information.
Indicators that seek to measure progress in conflict and violence typically assess a country’s political and institutional framework for supporting transition and
recovery (New Post-Conflict Performance Indicators Framework), identify sources of conflict and the local institutional capacity to deal with them (Measuring
Progress in Conflict Environments), and score the quality of democracy, and economic and political management in transition countries (Heidelberg Conflict
Barometer).
Various critiques are offered in the literature concerning the utility and precision of conflict and violence indicators. Some authors question the accuracy of conflict
indicators, noting that they tend to be ‘top-down’, and measured by government and political elites who may have a limited connection with communities
experiencing the conflict or transition to peace (Mac Ginty 2013b). Mac Ginty (2013) suggests that to combat this lack of local input, supplementary reports to existing
top-down indicators could be produced that are based on bottom-up information, and have their research design influenced by the local communities under study.
Indicator and link Status Purpose Areas covered How are scores determined? Information used to determine performance
Measuring Progress in Conflict Environments (MPICE) - United States Institute for Peace http://www.usip.org/sites/default/files/MPICE_final_complete%20book%20%282%29.pdf
Published 2010 The MPICE indicators aim to identify potential sources of continuing violent conflict and instability, and to gauge the capacity of indigenous
There are five core sectors in the MPICE indicators: safe and secure environment; political moderation and stable democracy; rule of law; sustainable economy; and social well-being. Each of these sectors are divided into two subsectors, conflict
Data is aggregated and analysed to establish trends over time.
Data collection methodologies include content analysis (which involves surveying media publications), expert knowledge, quantitative data, and survey/polling data.
Indicators for conflict, stability, security, justice and peacebuilding
5
organisations to overcome them.
drivers and institutional performance.
Minorities at risk project (MAR) http://www.cidcm.umd.edu/mar/
Ongoing MAR is a university based research project that monitors and analyses the status and conflicts of politically-active communal groups in countries which have a population of at least 50,000.
The Minorities at Risk Organizational Behavior (MAROB) dataset is a subsidiary of the MAR Project. Initiated in 2005, the purpose of the project is to answer fundamental questions focusing on the identification of factors that motivate some members of ethnic minorities to become radicalized, to form activist organizations, and to move from conventional means of politics and protest into violence and terrorism.
MAR maintains data on 284 politically active ethnic groups. Information on ranking is not readily available.
It combines quantitative and qualitative data, including minority group assessments (a risk assessment and an analytical summary and minority group chronologies.
Uppsala Data Conflict Program (UDCP) http://www.pcr.uu.se/research/UCDP/
Ongoing. Coverage is global with information from 1946 onwards.
Records ongoing violent conflict and provides a number of databases on organised violence and peace-making. The UCDP database is one of the most well-used data sources on armed conflict and ‘its definition is becoming a standard in how conflicts are systematically
Data on a broad range of aspects of violence, including armed conflict, peace agreements, and organised violence.
Information on the process of ranking is not readily available.
Key word search of the Factiva database, which includes over 10,000 different newswires, newspapers and other sources. Articles are downloaded by human coders and manually sorted. Information is gathered and then coded according to UDCP’s criteria for different types of organised violence.
Indicators for conflict, stability, security, justice and peacebuilding
6
defined and studied’2.
At the second stage, coders use other types of material, including newly published books, journals, NGOs publications, and online databases within the sphere of organised violence3.
Heidelberg Conflict Barometer - European Commission’s Instrument for Stability. http://www.hiik.de/en/konfliktbarometer/
Ongoing. Published annually since 1992; latest report 2014.
Annual analysis of global conflict events, which includes a detailed examination of conflict dynamics and processes.
Different conflict factors, including non-violent and violent crises, wars, coup d’etats, peace negotiations. Also looks at changes in the intensity of conflict, including types and use of weapons, personnel, causalities, and destruction.
The barometer distinguishes between five levels according to the dynamic conflict model. Details of the methodological approach are provided in the recent 2014 report4.
The data sources that are used in the barometer are unclear.
Armed Conflict Location and Event Data Project (ACLED) http://www.acleddata.com/
Ongoing Comprehensive public collection of political violence data for developing states.
Contains information on the specific dates and locations of political violence, the types of events, the groups involved, fatalities, and changes in territorial control. This can be used for medium and long term analysis, and mapping of political violence across countries.
n/a Event data is derived from a variety of sources, including reports from developing countries and local media, humanitarian agencies, and research publications.
Indicators for conflict, stability, security, justice and peacebuilding
7
New Post-Conflict Performance Indicators Framework (PCPI) - World Bank http://www.worldbank.org/ida/ISIA/PCPI-Q&A.pdf
Unclear The PCPI assesses the quality of a country’s policy and institutional framework to support a successful transition and recovery from conflict.
The PCPI is organised around four clusters: economic management and structural policies; social inclusion and human development; governance; and post-conflict risk.
For each criterion, countries are scored on a scale of 1 (low) to 6 (high). The ratings are averaged to obtain the overall score.
Scores are based on a variety of indicators, observations and judgements that is originated at the World Bank and elsewhere.
3. Peacebuilding indicators
There is a broad body of indicators that seek to capture progress in peacebuilding. However, some authors cautions that ‘many of the approaches to measuring
peace favoured by international organisations, INGOs and donor governments are deficient’ (Mac Ginty 2013, p. 1). Limitations are attributed to analysis being too
broad or narrow, and aggregated statistical formats ‘represent[ing] the conflict-affected areas in ways that are meaningless to local communities’ (Ibid). Lack of
accurate data has also been highlighted as a key factor constraining the accuracy and utility of peacebuilding indicators. In a chapter for Accord, Mack (2014, p. 109),
for instance, notes that though there has been some progress in agreeing on a list of preliminary indicators for the New Deal’s Peacebuilding and Statebuilding Goals
(PSG)5, there are ‘few reliable sources of data available to populate them and…some of the datasets that have been proposed are not reliable enough to provide a
useful guide to progress’. He furthers that some fragile state governments ‘have resisted common indicators claiming they primarily reflect the interests of donors’
(Mack 2014, p. 109). In countries without robust statistical systems, household surveys are the only means of generating the reliable data and these can be politically
embarrassing or even damaging to governments (Ibid).
The complexities of understanding ‘peace’ and ‘peacebuilding’ have additionally hampered the creation of effective indicators .Mac Ginty (2013a, p. 58) cautions,
‘it is not always clear that peace or peacebuilding indicators are measuring peace. In part this stems from the illusive nature of peace and the absence of an agreed
definition…Many measures of peace focus on proxies that are believed to have a bearing on the extent of peace or peacefulness…Many of these proxies are defensible
(for example, there is significant evidence linking militarism and military spending to war) but they tend to be proxy explanations for conflict, or the absence of
conflict, rather than for peace’.
5 The five goals are: legitimate politics; security; justice; economic foundations; and revenues and services. For further details see: http://www.newdeal4peace.org/peacebuilding-and-statebuilding-goals/
Indicators for conflict, stability, security, justice and peacebuilding
8
Many peace-related indicators are tied to evaluation processes and are auditing and compliance tools that say little about the peacefulness of society (Mac Ginty
2013a).
Universal indicators on peacebuilding have also been criticised for failing to capture the complexities of local political, cultural and historical context. Country-level
indicators, by comparison, can be designed to take on board unique circumstances (Mack 2014).
Indicator and link Status Purpose Areas covered How are scores determined?
Information used to determine performance
Everyday Peace Indicator project - George Mason University (US), Institute for Justice and Reconciliation (South Africa), and the University of Manchester (UK) http://everydaypeaceindicators.org/
As of May 20156, piloting in three communities in each of four countries: South Africa, Zimbabwe, Uganda and South Sudan.
Aims to investigate alternative, bottom-up indicators of peace.
The communities themselves choose the areas they use to assess changes in peace and conflict in their locality, through focus group participatory research methods.
Community perception. These indicators are tested by the wider community in the locality using mobile phone technology.
International Dialogue on Peacebuilding and Statebuilding (IDPS) / “New Deal“ Technical Working Group - Common Indicators
A 2014 independent evaluation of the New Deal7 reports there has been no use of these
Interim product to be piloted by volunteer G7 countries to measure progress on five Peacebuilding and Statebuilding Goals (PSGs) in the New Deal for engagement in Fragile
34 indicators organised by the five PSGs: legitimate politics; security; justice; economic foundations; revenue and services. (In conjunction with this work, pilot countries also developed national
Draft guidance is given for some of the indicators; for others it has not been developed.
The guidance proposes data sources for each indicator. Suggestions include perceptions surveys, government and UN statistics.
States. Intended for national governments and international partners to track progress on PSGs within a country, over time.
indicators to measure PSGs8).
Globally-Accepted Indicators (GAIN) for peacebuilding - Catholic Relief Services http://static1.1.sqspcdn.com/static/f/752898/9984888/1296501826553/GAIN.pdf?token=MI4p7zFMvbSTEMHMVu9NvGG6cWA%3D
GAIN initiative started in 2008, and the GAIN peacebuilding indicators were designed in 2010.
To improve the quality of indicator selection and use and to contribute to a more efficient M&E design process. There is little emphasis on the use of the indicator outside of specific project or program context.
18 indicators encompassing increased equity, social cohesion, church action, civic engagement, extractives, sexual and gender-based violence, interfaith dialogue and cooperation, and youth.
Each template includes calculations for analysis, and tips for interpreting the results against the project’s broader objectives. Each indicator is also linked to a theory of change.
Each template includes suggested qualitative and quantitative data collection questions.
Indicators for the post-2015 Sustainable Development Goal 16 - The United Nations Statistical Commission (UNSC)
Expected to be endorsed in 2016.
To measure progress on the sustainable development goal 16: “Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective,
The draft indicators are organised under 12 targets, covering violence, terrorism and crime, abuse of children, rule of law and access to justice, illicit financial and arms flows, corruption and bribery, institutions,
The draft framework does not yet explain how scoring will work9.
The draft framework does not yet give guidance on information sources.
accountable and inclusive institutions at all levels”
decision-making, global governance, legal identity, freedom to information, non-discrimination.
Global Peace Index - Institute for Economics and Peace http://www.visionofhumanity.org/#/page/indexes/global-peace-index
Published its eighth annual report in 201410
1) Ranks the peacefulness of 162 independent states covering 99.6 percent of the world’s population, and 2) gauges global peace.
The 22 GPI indicators are classified under three broad themes: ongoing domestic and international conflict; societal safety; and security and militarisation.
Each indicator is
weighted based on their relative importance on a
scale of 1-5. Two sub-component weighted
indices are calculated,
with a weight of 60 per cent applied to the
measure of internal
peace, and 40 per cent for external peace.
Scores are based on a range of sources and judgments.
4. Stability indicators
A variety of analytical frameworks and instruments have been developed to measure certain dimensions and indicators of state fragility. Often, indicators and indexes
present a hierarchically organised list of states according to their performance against certain state functions (Mcloughlin 2012, p. 29). Similar to other indicators,
however, indicators measuring stability have been criticised for being ‘subjective, arbitrary in terms of where they draw the line between performing and non-
performing institutions, and for inconsistencies within and between them’ (Ibid). Some authors additionally caution that aggregate scores can fail to capture how
Indicators for conflict, stability, security, justice and peacebuilding
11
state capacity varies across functions11. A number of the identified indicators that look at stability and fragile states are not currently operational, such as the Political
Instability Index and the Index of State Weakness in the Developing World.
Indicator and link Status Purpose Areas covered How are scores determined?
Information used to determine performance
Fragile States Index http://library.fundforpeace.org/library/cfsir1423-fragilestatesindex2014-06d.pdf
Ongoing. The latest Index was published in 2014.
Annual ranking of 178 nations based on their levels of stability and the pressures they face.
The Fragile States Index is based on the twelve primary social, economic and political indicators of the Conflict Assessment Software Tool (CAST), developed by the Fund for Peace12.
The Index is based on the Fund for Peace’s proprietary CAST analytical platform. Data from three sources are triangulated and subjected to critical review to obtain final scores for the Fragile States Index.
It is unclear what sources are used to determine performance in the Fragile States Index.
Political Instability Index http://viewswire.eiu.com/site_info.asp?info_name=instability_map&page=noads&rf=0
The Index covers the period 2009-2010.
The Political Instability Index shows the level of threat posed to governments by social protest.
The Index covers 165 countries and looks at various factors that have contributed to political instability, including inequality, state history, corruption, status of minorities, level of social provision, and regime type13.
The overall index on a scale of 0 (no vulnerability) to 10 (highest vulnerability) has two competent indexes: an index of underlying vulnerability; and an economic distress index.
Index draws on political science literature. In particular, the Political Instability Task Force (PITF) based at George Mason University.
Index of State Weakness in the Developing World – Brookings Institution
Last produced in 2008.
The Index ranks and assesses 141 developing nations according to their relative performance in four critical
State weakness is measured according to each state’s effectiveness in delivering economic, political, security and social welfare.
The indicator scores are standardised and aggregated, creating individual and basket scores ranging from 0.0 (worst) to 10.0(best). The
The Index relies on four ‘baskets’, each of which contains five indicators:
Economic: GNI per captia; GDP growth; inflation; regulatory quality
11 The GSDRC Topic Guide on Fragility has a comprehensive list of different approaches to measuring fragility. See: Mcloughlin (2012). 12 For further details see: http://fsi.fundforpeace.org/indicators 13 See: http://viewswire.eiu.com/index.asp?layout=VWArticleVW3&article_id=874361472
spheres: economic, political, security and social welfare.
four basket scores are then averaged to obtain an overall score for state weakness14.
Political: Government
effectiveness; rule of law; voice and accountability; control of corruption; freedom ratings
Security: conflict intensity; political stability and absence of violence; incidence of coups; gross human rights abuses; territory affected by conflict
Social welfare: child mortality; primary school completion; undernourishment; per cent population with access to improve water sources, and improved sanitation facilities; life expectancy.
5. Justice and safety indicators
There are a number of indicators that seek to measure progress in justice and safety. The UN’s Rule of Law indicators, for instance, has a particular focus on criminal
justice institutions. While many of the approaches involve universal indicators, there is some emerging projects that involving supporting countries to develop their
own justice indicators. For instance, the Harvard Kennedy School, with funding from DFID, has been supporting state officials and civil society organisations in Jamaica,
Sierra Leone, and Nigeria to develop and use their own indicators15. There is also some literature available offering guidance on how to develop effective justice
Indicators for conflict, stability, security, justice and peacebuilding
13
indicators. The UN’s Office on Drugs and Crimes, alongside UNICEF, have produced a manual for the measurement of juvenile justice indicators, which provides
practical methodological tools (UNODC 2006). The Harvard Kennedy School has also provided guidance on how tackle some of the practical challenges in developing
indicators for programme that aim to reduce violence against women and girls (Foglesong 2012).
Indicator and link Status Purpose Areas covered How are scores or indicators determined?
Information used to determine performance
Rule of Law Index – World Justice Project (WJP) http://worldjusticeproject.org/rule-of-law-index
Ongoing, 2015 report has been produced16.
Measurement of how the rule of law is experienced in everyday life around the world.
Indicators are organised around 8 themes: Constraints on government powers; Absence of corruption; Open government; Fundamental rights; Order and security; Regulatory enforcement; Civil justice; and criminal justice.
Performance is assessed using 44 indicators across 8 categories, each of which are scored and ranked globally and against regional and income peers17.
100,000 household and 2,400 expert surveys.
UN Rule of Law Indicators http://www.un.org/en/events/peacekeepersday/2011/publications/un_rule_of_law_indicators.pdf
Aggregate and individual governance indicators for 213 economies over the period 1996-2009.
Initial focus on criminal justice institutions, including the police and other law enforcement agencies, the courts, prosecution and defence, and corrections.
The indicators focus on the fundamental aspects of criminal justice institutions as they relate to the rule of law, including a focus on capacity, performance, integrity, transparency and accountability. They also monitor how vulnerable groups are treated by these organisations.
Indicators are grouped in 25 baskets, which include both rated (expressed as a score between 1.0 and 4.0) and unrated indicators.
There are four main sources of data for the indicators: public survey; expert survey; document review; administrative and field data.
Indicators for conflict, stability, security, justice and peacebuilding
14
Indicators in Development: Safety and Justice - Harvard Kennedy School (HKS) http://www.hks.harvard.edu/programs/criminaljustice/research-publications/measuring-the-performance-of-criminal-justice-systems/indicators-in-development-safety-and-justice/project-overview
Operated over a five year period between 2009-2014
HKS has been supporting state officials and civil society organisations in Jamaica, Sierra Leone and Nigeria to develop and use their own indicators to initiative, reinforce, and communicate progress toward strategic goals in justice and safety.
Areas of focus include sense of public safety and response to crime; police effectiveness; pre-trial detention; police-prosecution coordination; coordination of customary and formal courts.
Country-led with support from the HKS.
Unclear.
6. Performance indicators for crisis response
In a rapid literature review, Rohwerder (2013) provides an overview of some of the readily available indicators and guidance used by organisations providing crisis
response. Despite a wealth of indicators relating to humanitarian response18, there is little engagement in the literature with regard to specific performance
indicators for crisis response and the strengths and weaknesses of these indicators. The importance of indicators in general is flagged up but often no actual examples
of indicators are provided. Very few organisations have a specific set or performance indicators, which measure the effectiveness of their interventions in crisis
response.
18 See for example: http://www.alnap.org/eha; http://www.alnap.org/resource/5674.aspx; http://www.alnap.org/resource/5666; http://www.sphereproject.org/
Indicators for conflict, stability, security, justice and peacebuilding
15
7. Proxy indicators
There are a number of indicators that do not directly seek to measure progress in conflict, stability, justice and peacebuilding, but which could be seen as proxy
indicators for a number of areas of this report. A selection of these are detailed below:
Indicator and link Status Purpose Areas covered How are scores or indicators determined?
Information used to determine performance
Bertelsmann’s Transformation Index http://www.bertelsmann-transformation-index.de/16.0.html?&L=1
The Status Index ranks countries according to their quality of democracy and market economy as of 31st January 2013. The Management Index ranks countries according to their leadership’s political management performance between January 2011 and January 2013.
Scores the quality of democracy, market economy and political management in 129 developing and transition countries.
Political transformation, economic transformation, transformation management.
Country experts access the extent to which a total of 17 criteria have been met for each of the 129 countries. A second country expert reviews these assessments and scores. In the final step, consistency is ensured by to subjecting each of the 49 individual scores to regional and interregional calibration processes19.
Broad body of data including official economic data, constitutional provisions, regulatory or competition policy, and social safety nets20.
World Governance Indicators
1996-2013. Reports aggregate and individual governance
Six dimensions of governance: voice and accountability; political stability and absence
Indicators combine the views of a large number of enterprise, citizens and expert survey
1981-2011. Contains quantitative information of government respect for 15 internationally recognised human-rights practices.
Variables can be grouped into three main categories: violations of physical integrity, such as torture; civil rights, including freedom of speech and freedom of assembly; and the political, economic and social rights of women.
Expert coding. Every unit of analysis is independently coded by two coders and areas of dispute are mediated with senior staff.
Indicators are based on US State Department and Amnesty International reports.
