1

Designing a Monitoring and Evaluation System for a Rural Travel and Transport Project

Michael BambergerGender and Development Group

The World BankRTTP Conference

Pretoria, South AfricaDecember 2 1999

2

A good M/E system helps ensure that a project

• Is being efficiently implemented

• Is reaching the intended target groups

• Is achieving its intended objectives

3

A good M/E system will

Monitor the use of project inputs Monitor the effectiveness of the project

implementation process Monitor the production of project outputs Assess project impacts on the target communities Assess the effectiveness of project outputs in

producing the intended short-term and long-term impacts.

Assess the extent to which these impacts can be attributed to the effects of the project.

4

An RTT M/E system should

• Extract lessons and best practices for the design of future projects

• Provide guidelines for the modification of project design

• Provide adequate data for the evaluation of program impact

5

The Basic Monitoring and

Evaluation Model

6

Figure 1 Monitoring and Evaluation Model of the Project Implementation Process

Economic and Politicalcontext within which theproject is implemented

Institutional contextwithin which the projectis implemented

Projectplanningand Inputs

ProjectImplementationprocess

Project outputs Project impacts Projectsustainability

Socio-economic characteristics of the communitiesaffected by the project

7

Stages of the project cycle covered by the M/E system

• Project planning and inputs

• Implementation process

• Output

• Short and long term impacts

• Project sustainability

8

Project implementation and impacts are also affected by

• The organizational and institutional context

• The economic and political context

• The social and economic characteristics of the communities

9

Figure 3 Apply the Monitoring and Evaluation Model to the Tanzania Village Travel and Transport Project

Economic and Politicalcontext within which theproject is implemented

Institutional contextwithin which the projectis implemented

ProjectInputsRoadconstruction, donkey,bicycles,constructionof waterpumps,technicalassistance

ProjectImplementation processFocus groups,communitymanagementgroups includingwomen, training,female extensionworkers

Project outputsImproved andnew roads andfootpaths, womenuse bicycles anddonkeys,handpumps,grinding mills.

Short and longterm impactstravel timereduced, easieraccess to markets, ,more childrenattend school,income increases,women enter labormarket, women’sstatus in familyenhanced

Sustainability

InfrastructuremaintainedStrongercommunitygroupsNetworks createdCost-recoveryAgencies moregender sensitive

Socio-economic characteristics of the communities affected bythe project

10

Key questions for designing a M/E system

• What indicators should be used to measure and monitor each stage of the project?

• How should the information be collected?

• How to measure quality and effectiveness?

• How and when to communicate findings to managers and stakeholders?

11

Key questions continued….

• How do we assess the benefits and impacts produced by the project?

• How to assess whether pilot projects have been successful.

12

Developing and using indicatorsGood indicators should

• Cover all stages of the project cycle• Provide essential information needed to

monitor progress, identify problems and assess results

• Provide information which can easily be used by managers and policy makers

• Be objective and easily measurable• Include the minimum number possible

13

Examples of indicators for a feeder road project

• Planning and inputs

• Implementation process

• Outputs

• Stakeholder groups consulted• Funds approved and available• Equipment available• All community included in

planning and implementation• Targets set for women

representatives• Local organizations

established at district level• Km. Roads constructed/

upgraded

14

Feeder road indicators page 2

• Impacts • Increased household income

• Higher proportion of girls attending school

• Increased agricultural production

• Wider range of goods available in villages

15

Feeder road indicators page 3

• Sustainability • Cash payments for road maintenance

• Community labor for road maintenance

• Condition of roads 1, 2 and 5 years after construction

16

M/E systems must identify all stakeholders and ensure their

information needs are covered.

• All community groups affected by the project

• Local government agencies

• National government agencies

• NGOs

• International development agencies

17

Evaluationstudies

andevaluation systems

18

Individual M/E studies can be conducted

each using the methods best suited to the particular issue

being studied

But to assess the overall progress of a program

involving many different activities ….

An evaluation system is required

Logical framework analysis [LOGFRAME]

Is one of the best methods for setting up a monitoring and

evaluation system

21

LOGFRAME

helps define the basic assumptions on which the project

design and implementation systems are based.

22

LOGFRAME

monitors the following logical sequence of activities……

23

Ifcertain inputs are used effectively

thencertain outputs will be produced

if the outputs are produced

thencertain impacts will be achieved

24

Example

Using LOGFRAME to monitor and evaluate the impacts of a

feeder road on women

25

If feeder roads are upgraded

thenwomen will take more

agricultural produce to marketthen

household income will increaseand women’s economic status

will improve

26

Main kinds of M/E studiesSee Handout Annex I for details

Question

1. Are RTTP activities being implemented on schedule?

2. Are women fully involved in all stages?

3. Are the activities producing their intended benefits?

Study

• Monitoring study

• Gender responsiveness assessment study

• Impact evaluation

27

Main kinds of studies page 2

Question

4. Are the activities sustainable?

5. Were the pilot interventions successful and could they be replicated?

Study

• Sustainability assessment

• Replicability assessment