7/28/2019 Building RBM

1/62

IPDETModule4:

BuildingaResults-B

MonitoringandEvaluSystem

Results-based-based Monitorin

1 2 3 4 5 61

7/28/2019 Building RBM

2/62

IPDET

Introduction

ImportanceofResults-Based

WhatIsResults-BasedM&E?

ReasonstoDoResults-Base

Traditionalvs.Results-Based

TenStepstoBuildingResults

M&E

7/28/2019 Building RBM

3/62

IPDET

Importance

Growingpressuresindevelo

countriestoimproveperform

publicsectors Tracksresultsofgovernment

overtime

Isamanagementtool

7/28/2019 Building RBM

4/62

IPDET

Planning forImprovement SelectingRealistic Targets51

Conducting aReadinessAssessment3

Selecting KeyIndicators toMonitorOutcomes7

The Role ofEvaluations

2

Agreeingon Outcomesto Monitor andEvaluate

4

Baseline Data onIndicators Where Are WeToday?

6

Monitoringfor Results RepFind

7/28/2019 Building RBM

5/62

IPDET

ThePowerofMeasurin

Results

Ifyoudonotmeasureresults,youca

successfromfailure

Ifyoucannotseesuccess,youcann

rewardit Ifyoucannotrewardsuccess,youar

rewardingfailure

Ifyoucannotseesuccess,youcann

Ifyoucannotrecognizefailure,youc Ifyoucandemonstrateresults,youc

publicsupport

7/28/2019 Building RBM

6/62

IPDET

Results-BasedMo

Results-basedmonitoring(w

monitoring)isacontinuous

collectingandanalyzinginforcomparehowwellaproject,

policyisbeingimplementeda

expectedresults

7/28/2019 Building RBM

7/62

IPDET

Results-BasedEva

Results-basedevaluationisa

assessmentofaplanned,on

completedinterventiontodetrelevance,efficiency,effectiv

impact,andsustainability

7/28/2019 Building RBM

8/62

IPDET

DifferencebetweenRe

BasedMonitoringand

Evaluation Monitoringisfocusedontracking

movementtowardstheachievem

specific,predeterminedtargets

Evaluationtakesabroaderview

intervention,consideringnotonly

towardstatedgoals,butthelogic

initiative,aswellasitsconseque Bothareneededtobeabletobe

policies,programs,andprojects

7/28/2019 Building RBM

9/62

IPDET

KeyTypesofMoni

Res

Implem(Mean

Outcomes

Impacts

Results

Inputs

Activities

Outputs

Implementation

7/28/2019 Building RBM

10/62

IPDET

ComplementaryRo

MonitoringandEvaMonitoring9 Clarifiesprogramobjectives

9 Linksactivitiesandtheir

resourcestoobjectives

9 Translatesobjectivesintoperformanceindicatorsandsets

targets

9 Routinelycollectsdataonthese

indicators,comparesactual

resultswithtargets9 Reportsprogresstomanagers

andalertsthemtoproblems

Eva9 Analyzeswhy

wereorwere

9 Assessesspe

contributions9 Examinesimp

9 Exploresunint

9 Provideslesso

significantac

programpoterecommenda

improvement

7/28/2019 Building RBM

11/62

IPDET

Reasons:

Providescrucialinformationabousectorperformance

Providesaviewovertimeontheproject,program,orpolicy

Promotescredibilityandpubliccoreportingontheresultsofprogra

Helpsformulateandjustifybudge

Identifiespotentiallypromisingprorpracticesbystudyingpilots

(continuedonnextslide)

7/28/2019 Building RBM

12/62

IPDET

Reasons:(cont.)

Focusesattentiononachievingoimportanttotheorganizationanditsstakeholders

Providestimely,frequentinforma

Helpsestablishkeygoalsandou

Permitsmanagerstoidentifyandtocorrectweaknesses

Supportsadevelopmentagendashiftingtowardsgreateraccountaforaidlending

7/28/2019 Building RBM

13/62

IPDET

Comparison

Traditional

inputs

activities outputs

Results-Based

combinestra

assessmento

andimpacts

allowsorgani

modifyandm

adjustmentstchangeimple

processes

7/28/2019 Building RBM

14/62

IPDETStep1:Conductin

ReadinessAssessmPlanning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

Ten Steps to BuildinResults-Based M&E S

7/28/2019 Building RBM

15/62

7/28/2019 Building RBM

16/62

IPDET

Incentives

Sortouttheanswerstothese

WhatisdrivingtheneedforbuM&Esystem?

WhoarethechampionsforbuusinganM&Esystem?

WhatismotivatingthosewhocbuildingandM&Esystem?

Whowillbenefitfromthesyste

Whowillnotbenefit?

7/28/2019 Building RBM

17/62

IPDET

Rolesand

Responsibilities Ask:

Whataretherolesofcentralandlin

assessingperformance?

Whatistheroleofparliament?

Whatistheroleofthesupremeaud

Doministriesandagenciessharein

oneanother?

Whointhecountryproducestrustw

Whereinthegovernmentaredatau

7/28/2019 Building RBM

18/62

IPDET

OrganizationalCap

Assesscurrentgovernmentcmonitorandevaluate:

Technicalskills Managerialskills

Existingdatasystemsandthe

Technologyavailable

Fiscalresourcesavailable Institutionalexperience

7/28/2019 Building RBM

19/62

IPDET

BarrierstoM&E

Doanyofthesebarrierstobuildisystemnowexist? Lackoffiscalresources

Lackofpoliticalwill

Lackofchampion

Lackofexpertise&knowledge

Lackofstrategy

Lackofpriorexperience Howdoweconfrontthesebarrie

7/28/2019 Building RBM

20/62

IPDET

KeyQuestionsforPredict

SuccessinBuildinganM&

DoesaclearmandateexistforM&E? PRSP?,Law?CivilSociety?Other?

Istherethepresenceofstrongleadersseniorlevelsofthegovernment?

Isthereadesiretoseeresourceandplinkedtothebudget?

Howreliableisinformationthatmaybepolicyandmanagementdecisionmaki

Howinvolvediscivilsocietyasapartngovernment?

Aretherepocketsofinnovationthatcabeginningpracticesorpilotprograms?

7/28/2019 Building RBM

21/62

IPDET Step2:AgreeingPerformanceOutco

MonitorandEvalPlanning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomesto Monitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

22/62

IPDET

WhyanEmphasis

Outcomes? Makesexplicittheintendedobjec

ofgovernmentaction (Knowwhereyouaregoingbeforeyouge

Outcomesarewhatproduceben

Clearlysettingoutcomesiskeyto

andbuildingresults-basedM&E

Important!Budgettooutputs,maoutcomes!

7/28/2019 Building RBM

23/62

IPDET

DevelopingOutcom

OnePolicyArea:E

2.Primary

schoollearningoutcomesfor

childrenare

improved

1.Nationschildrenhave

improvedaccess

topre-school

programs

Baselines:Indicators:Outcomes:

7/28/2019 Building RBM

24/62

IPDET

Outcomes:

Outcomesareusuallynotdir

measured onlyreportedon

Outcomesmustbetranslatedkeyindicators

Whenchoosingoutcomes,D

alone! agreementiscrucia

7/28/2019 Building RBM

25/62

IPDET Step3:SelectingIndicatorstoMon

OutcomesPlanning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitorOutcomes7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

26/62

IPDET

Indicators

Aspecificmeasure,thatwhe

systematicallyovertimeindic

progress(ornot)towardasptarget

Indicatorsasksthequestion:

Howwillweknowsuccesswh

7/28/2019 Building RBM

27/62

IPDET

IndicatorDevelopm

CREAM

Clear

Relevant Economic

Adequate

Monitorable

7/28/2019 Building RBM

28/62

IPDET

MatrixforSelecting

Indicators

Indicator Datacollectionmethod

Frequency& cost tocollect

Who willcollectdata?

Difficultyto collect

Datasource

1.

2.

3.

7/28/2019 Building RBM

29/62

IPDET

DevelopingSetofO

IndicatorsforOnePolicyAr

1.%ofGrade6students

scoring70%orbetteron

standardizedmathandscience

tests2.%ofGrade6students

scoringhigheronstandardized

mathandsciencetestsin

comparisontobaselinedata

1. %ofeligibleurbanchildren

enrolledinpre-school

education2.%ofeligibleruralchildren

enrolledinpre-school

education

BaselinesIndicators:Outcomes:

2.Primary

schoollearning

outcomesforchildrenare

improved

1.Nations

childrenhave

improvedaccesstopre-school

programs

7/28/2019 Building RBM

30/62

IPDET

DevelopingIndicat

Developyourownindicatorsyourneeds

Developinggoodindicatorsutakesmorethanonetry

Arrivingatfinalindicatorswil

Stateallindicatorsneutrallyincreasein ordecreasein

Pilot,Pilot,andPilot!

7/28/2019 Building RBM

31/62

IPDETStep4:GatheringBa

DataonIndicatorPlanning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators WhereAre We Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

32/62

IPDET

BaselineDataand

Sources Baselinedata: Measurementstofindoutwhereare

Primarysource: gatheredspecificallyfortheproject

Secondarysource: collectedforanotherpurpose,butc

from:withinorganization,governme

internationaldatasources,etc.

cansavemoneybutbecarefultoetrulytheinformationyouneed

7/28/2019 Building RBM

33/62

IPDET

PossibleSources

Writtenrecords

Individualsinvolvedwiththe

Thegeneralpublic

Trainedobservers

Mechanicalmeasurements

Geographicalinformationsys

7/28/2019 Building RBM

34/62

IPDET

DesignDataCollec

Methods1. Decide how to obtain the da

need from each source

2. Prepare data collection inst

3. Develop procedures for usedata collection instruments

7/28/2019 Building RBM

35/62

IPDET

Conversation

with

concerned

individuals

CommunityInterviews

Field

visits

Reviews of

official

records (MIS

and admin

data)

Participant

Observation

Key informant

interviews

Focus

Group

Interviews

Questionnaires

Informal/Less Structured Methods More Structu

One-T

Surv

Directobservation

7/28/2019 Building RBM

36/62

IPDET

ContinuingExample,Develop

DataforOnePolicyArea: Edu

1.47%in2002sc

70%orbetterinm

50%in2002scor

orbetter

2.Mean%score

forGrad6studen

mathwas68%,a

forscience

1.75%urbanin

2.40%ruralin20

1.%ofGrade6students

scoring70%orbetteron

standardizedmathand

sciencetests

2.%ofGrade6students

scoringhigheronstandardized

mathandsciencetestsin

comparisontobaselinedata

2. Primary

schoollearningoutcomesfor

childrenare

improved

1. %ofeligibleurbanchildren

enrolledinpre-school

education

2.%ofeligibleruralchildren

enrolledinpre-school

education

1. Nations

childrenhave

improvedaccess

topre-school

programs

BaselineIndicators:Outcomes:

7/28/2019 Building RBM

37/62

IPDET Step5:PlanningImprovements Se

RealisticTargetPlanning forImprovement SelectingRealistic Targets51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

38/62

IPDET

Targets:

The quantifiable levels of thethat a country or organization

achieve at a given point in timExample:

Agricultural exports will increasthree years by 20% over the b

7/28/2019 Building RBM

39/62

IPDET

IdentifyingExpectedor

LevelofImprovementR

Targets

Baseline

Indicator

Level

Desired

Level of

Improvement

Assumes afinite and

expected levelof inputs,

activities, andoutputs

+ =

7/28/2019 Building RBM

40/62

IPDET

Caution:

Ittakestimetoobservethee

improvements,therefore:

Berealisticwhensettingtarge Avoidpromisingtoomuchand

programmingyourselftofail

7/28/2019 Building RBM

41/62

IPDET

ContinuingExample,Setting

TargetsforOnePolicyArea:

1.47%in2002sc

70%orbetterinm

50%in2002scor

orbetter

2.Mean%score

forGrad6studen

mathwas68%,a

forscience

1.75%urbanin

2.40%ruralin20

1.%ofGrade6students

scoring70%orbetteron

standardizedmathand

sciencetests

2.%ofGrade6students

scoringhigheronstandardized

mathandsciencetestsin

comparisontobaselinedata

2. Primary

schoollearningoutcomesfor

childrenare

improved

1. %ofeligibleurbanchildren

enrolledinpre-school

education

2.%ofeligibleruralchildren

enrolledinpre-school

education

1. Nations

childrenhave

improvedaccess

topre-school

programs

BaselineIndicators:Outcomes:

7/28/2019 Building RBM

42/62

IPDETStep6:Monitoringfor

Planning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoringfor ResultsRF

7/28/2019 Building RBM

43/62

IPDET

KeyTypesofMoni

Res

Implem(Mean

Outcomes

Impacts

Results

Inputs

Activities

Outputs

Implementation

7/28/2019 Building RBM

44/62

IPDET

Implementation

MonitoringLinksto

ResultsMonitoring

Means andStrategies(Multi-year andAnnual WorkPlans)

Means andStrategies(Multi-year andAnnual WorkPlans)

Means andStrategies(Multi-year andAnnual WorkPlans)

Means andStrategies(Multi-year andAnnual WorkPlans)

Means andStrategies(Multi-year andAnnual WorkPlans)

Means andStrategies(Multi-year andAnnual WorkPlans)

Target 1 Target 2 Target 3

Outcome

7/28/2019 Building RBM

45/62

IPDET

Target 1

Partner 1

Partner 2

Partner 3

Means & Strategy

Partner 1

Partner 2

Partner 3

Means & Strategy

Partn

Partn

Partn

Means &

Target 2

Outcome Outcome

Impact

7/28/2019 Building RBM

46/62

IPDET

SuccessfulMonito

Systems Tobesuccessful,everymon

systemneedsthefollowing:

ownership management

maintenance

credibility

7/28/2019 Building RBM

47/62

7/28/2019 Building RBM

48/62

IPDET

EvaluationWhent

Anytimethereisanunexpectedresult

furtherinvestigation

Whenresourceorbudgetallocationsa

acrossprojects,programs,orpolicies

Whenadecisionisbeingmadewheth

expandapilot

Whenthereisalongperiodwithnoim

andthereasonsforthisarenotclear Whensimilarprogramsorpoliciesare

divergentoutcomes

7/28/2019 Building RBM

49/62

IPDET

EvaluationMeansStrategy

Whetherwearedoingtherigh Rationale/justification Cleartheoryofchange

Operation

Whetherwearedoingthingsr Effectivenessinachievingexpe Efficiencyinoptimizingresourc Clientsatisfaction

Learning

Whethertherearebetterway

Alternatives Bestpractices Lessonslearned

7/28/2019 Building RBM

50/62

IPDETStep8:ReportingFin

Planning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults ReFi

7/28/2019 Building RBM

51/62

IPDET

AnalyzingandRep

Data Givesinformationonthestatusoprograms,andpolicies

Providescluestoproblems

Createsopportunitiestoconsiderimprovementsinthe(projects,prpolicy)implementationstrategies

Providesimportantinformationovtrendsanddirections

Helpsconfirmorchallengetheory

7/28/2019 Building RBM

52/62

IPDET

AnalyzingData

Examinechangesovertime

Comparepresenttopastdata

trendsandotherchanges

Themoredatapointsyouhavcertainyouareofyourtrends

?Time

Improving accessto rural markets

Acc

ess

Tim

Improvingto rural m

Acce

ss

7/28/2019 Building RBM

53/62

IPDET

PresentingData

Presentmostimportantdata

Useanappendixorasepara

fordetailedtechnicaldata Usevisualpresentations(cha

graphs,maps)toillustratean

keypoints Avoiddatadumps

7/28/2019 Building RBM

54/62

IPDETStep9:UsingFindi

Planning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

55/62

IPDET

TenUsesofResul

Findings Respondstoelectedofficials anddemandsforaccountability

Helpsformulateandjustifybudge

Helpsinmakingoperationalresoallocationdecisions

Triggersin-depthexaminationsoperformanceproblemsexistandcorrectionsareneeded

Helpsmotivatepersonneltocontprogramimprovements

(continuedonnextslide)

7/28/2019 Building RBM

56/62

IPDET

TenUsesofResul

Findings(cont.) Monitorstheperformanceofcontgranteesagainstperformanceta

Providesdataforspecial,in-deptevaluations

Helpsprovideservicesmoreeffic

Supportsstrategicandotherlongplanningefforts(byprovidingbasinformationandlatertrackingpro

Communicatesbetterwiththepubuildpublictrust

7/28/2019 Building RBM

57/62

IPDET

NineStrategiesfor

SharingInformatio Empowerthemedia

EnactFreedomofInformation legisla

InstituteE-government

Addinformationoninternalandextern Publishannualbudgetreports

Engagecivilsocietyandcitizengroups

Strengthenparliamentaryoversight

StrengthentheOfficeoftheAuditorGe Shareandcompareresultsfindingswidevelopmentpartners

7/28/2019 Building RBM

58/62

IPDET Step10:SustaininM&ESystemwithi

OrganizationPlanning forImprovement Selecting RealisticTargets

51

Conducting aReadinessAssessment

3

Selecting KeyIndicators toMonitor Outcomes

7

The Role ofEvaluations

2

Agreeingon Outcomes toMonitor andEvaluate

4

Baseline Data onIndicators Where AreWe Today?

6

Monitoring forResults RF

7/28/2019 Building RBM

59/62

IPDET

CriticalComponen

CrucialtoSustainin Demand

Clearrolesandresponsibilitie

Trustworthyandcredibleinfo

Accountability

Capacity

Incentives

7/28/2019 Building RBM

60/62

IPDET

LastReminders

ThedemandforcapacitybuildingTheonlywayanorganizationcandownhill

Keepyourchampionsonyoursidthem!

EstablishtheunderstandingwithofFinanceandtheParliamentthsystemneedssustainedresource

Lookforeveryopportunitytolinkinformationtobudgetandresourdecisions

(continuedonnextslide)

7/28/2019 Building RBM

61/62

IPDET

LastReminders(co

Beginwithpiloteffortstodemonstrresults-basedmonitoringandevalu

Beginwithanenclavestrategy(e.g

innovation)asopposedtoawhole-governmentapproach.

Monitorbothimplementationprogrresultsachievements.

Complementperformancemonitorevaluationstoensurebetterunderspublicsectorresults.

7/28/2019 Building RBM

62/62

To continue on to the

Next Module click here

To return to the

Table of Contents click here

http://../Narrative/M_05-na.pdfhttp://../Narrative/M_05-na.pdfhttp://../Narrative/M_05-na.pdfhttp://../Other/index.pdfhttp://../Other/index.pdfhttp://../Other/index.pdfhttp://../Narrative/M_05-na.pdfhttp://../Other/index.pdf