Fragile states: Perspectives from evaluationsAFDB Evaluation Week, 5th December 2012.Presentation by Beate BullEvaluation Department, Norwegian Agency for Development Cooperation, Norad.

Norwegian development aid budget

Side/Page 218.04.23

Perspectives from peacebuilding /fragile states evaluations based on:

•Evaluations done for and by the Norwegian development cooperation Agency, and reviewing others

•A meta-review of evaluations of support to state building by Gravingholt, J. og Leininger, J. 2012

•OECD/DAC Guidance on evaluating peacebuilding activities in settings of conflict and fragility

Side/Page 318.04.23

Side/Page 418.04.23

Norad Evaluation Department’s Annual Reports (2011,2010,2009)

“Evaluation of Norwegian Development Cooperation with Afghanistan 2001-2011”, (2012), http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace;-«Pawns of Peace. Evaluation of Norwegian peace efforts on Sri Lanka 1997-2009 (2011)”, http://www.norad.no/no/resultater/publikasjoner/evalueringer/publikasjon?key=386346(2011)-“Aiding the Peace”: A Multi-donor Evaluation of Support to Conflict Prevention and Peacebuilding Activities in Southern Sudan 2005-2010. ITAD Ltd., United Kingdom. http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace-“Evaluation of the Danish education support to Afghanistan 2003-2010», 2012, http://um.dk/en/danida-en/results/eval/afgh/-“Assessing the Impact of Development Cooperation in North East Afghanistan 2005 – 2009.” 2010. Final Report. BMZ. Böhnke, J. R., J. Koehler, and Ch. Zuercher. -“Evaluation of the Danish engagement in and around Somalia 2006-10”, 2011 http://www.netpublikationer.dk/um/11094/

- “Evaluation of UNDP support to conflict-affected countries in the context of UN Peace Operations”, Draft final (not to be quoted), Evaluation Office, UNDP, Forthcoming January 2013.-Evaluation of Norwegian Peace efforts in Haiti 1998-2009 (2009).

Findings across the evaluations

1) That there are weaknesses in the analysis of the situation and of the conflict in both the planning and implementation phase –

– Implications: - limit relevance of the intervention/support/

- limit the evaluation’s possibility to say something bout about relevance (Afghanistan/ South Sudan)

- reduces the likelihood of appropiate conflict analysis being conducted

- More…

Side/Page 518.04.23

Findings from the evaluations cont.

• 2) Not enough resources are set aside to follow up and assess progress, during implementation (South Sudan, Afhganistan, Haiti and Sri Lanka). – At times, the staffing in embassies were far from adequate

(Afganistan- in particular, South Sudan, Haiti)Implications:– Not enough resources to quality assure programmes – are they on

the right track (that we do what we say we shall do): (Afghanistan-50 % of scools not adapted to girls needs (latrines/protective walls)).

– The danger of aid not being relevant/not adapted to changing context (are we doing the right thing?) or maybe contribute to aggraving the conflict(s)

– Not enough resources to monitor how or whether aid becomes a stake in the conflict

– Corruption

Side/cccPage

618.04.23

Findings from evaluations cont.

• 3) Too much emphasis from donors are put on harmonisation and coordination at the capital level in the partner country at the cost of sharing knowledge about local context, adapting activities to local conditions an presence in the field.

• Example: South Sudan: Donor Coordination meetings did not revolve around sharing conflict analyses, and discussing how to coordinate aid to address local conflicts, but bigger diplomatic issues: referendum 2010.

• Example: Afghanistan: the Norwegian funding to the ARTF remained remarkably consistent over the years despite important changes in the context

Side/Page 718.04.23

Some key challenges to peacebuilding evaluations

•The threat of violent conflict•The reality is often complex, stakes are high, everything becomes political,– ‘all voices to be heard’? How to be perceived as impartial and balanced-key to the credibility of the evaluation?•Evaluations can do harm (evaluators leave, others stay behind)

Side/Page 818.04.23

Side/Page 918.04.23

How to deal with a challenging context…

•How to conduct conflict analysis•How to conduct conflict sensitive evaluations (do no harm)•How to analyse theories of change and their underlyingassumptions•Surprises that are expected and those unexpected

Examples of findings using conflict analysis in evaluations

• The evaluation of peacekeeping efforts in the Democratic Republic of Congo, 2011. The evaluation uses a conflict analysis to assess the peace efforts, and finds that two important drivers of conflict were neglected by the donors: the land conflicts and the mineral exploitations.

• South Sudan: the multi-donor evaluation of conflict prevention and peace building activities 2005-2010, found that disarmement initiatives, not using a conflict analysis and thereby a do no harm approach as a basis for programming, led to - at least in one instance - a disarmed local community being subject to armed cattle raids from neighbouring community

Side/Page 1018.04.23

Findings from a (2012) meta review of evaluations of support to state building in fragile states:

Side/Page 1118.04.23

Few evaluations are concerned with:

- explicating a theory of change;

- constructing a credible counterfactual;

- and making use of quantitative methods where possible.

Which leads to:-An inbuilt tendency to reproduce the conventional wisdom -instead of testing implicitly assumed causality chains-or exploring what the alternatives would have been.

What do the evaluations of peacebuilding-field look at?

• Peace building evaluations conducted by many donors and mulitlateral organisations seem to have focused on understanding and mapping the terrain and their own internal organisation,

• They are concerned with:– Coordination between different actors, planning, – whether a conflict analysis is used or not for programming (most often

it is not), – types of interventions, (socio-economic, humanitarian, peace

building, governance) – conflict sensitivity, – Inputs, activities, and outputs

Rather than• results, what works and what does not work (what do we base our

knowledge on/ what assumptions do we base the interventions on=are we doing the right thing)

Side/Page 1218.04.23

Side/Page 1318.04.23