Changes in and Goals of USAID M&E Reporting Requirements

February 6, 2007Jackie Doremus, CARPE M&E Consultant

Outline of Presentation

Success of M&E to date – Broadly and the Operational Plan case study

Overall goals of CARPE M&E – why do we want this information?

Brief overview of current M&E system Proposed CARPE M&E Calendar Feedback from Partners’ Phase IIA

final reports

Outline of Presentation

Proposed updates to CARPE M&E system SO level indicator Updated matrix MOVs Narrative structure for LS consortia GIS data

Feedback and distribution of tools

Success of CARPE M&E System: Broadly

M&E system developed to satisfy all CARPE USAID reporting requirements, finalized Feb 2005

Effectively communicated results to Congress, helping ensure a fourth year of funding

Success of CARPE M&E System: A case study

December 2006 saw major changes in USAID’s reporting structure New, comprehensive reporting tool, the

Operational Plan, required for each Operating Unit

Entered into a web-based system, FACTS: Foreign Assistance Coordination and Tracking System

Data requirements much greater than previous USAID reports

Very little ‘warning’ or collaboration with the field on the content of the report

Success of CARPE M&E System: A case study

CARPE team was only SO in USAID/DRC to not return to partners for repeated information requests to satisfy FACTS*

Operational Plan design mimicked CARPE database design

Information entered into FACTS was auditable and robust due to MOVs and matrices

FACTS stats from Matrix

FACTS required activity budgeting Average % of the total LS budget spent

on “Training/Capacity Building” is 14% CARPE funds led to at least 2,163

people receiving training with FY06 funds

FY06 LS funds were used to improve management on 32.6 M ha

Goals of CARPE M&E

FUNDING Determine project warrants funding Use all available resources to communicate results and

ensure funding Regular conduit of communication between

partners and CTO Better to have structured and well-anticipated report

submission than ad hoc requests and changing requirements

Must efficiently use time and resources of partners during reporting submission and of CARPE team during processing of reports Matrix requires an initial investment from partners to learn

format, then an annual large investment when consortia create the workplan

Landscape leaders will need to help new subpartners

Goals of CARPE M&E

USG Grant Management

Performance-Based

Agreements

Communication with

Stakeholders

Conservation Knowledge

CARPE M&E

Overview of CARPE M&E

Semi-Annual Reporting Cycle: 1. A workplan matrix and narrative 2. A Semi-Annual Report matrix and

narrative 3. An Annual Report matrix and narrative

MOVs

A one-time GIS data submission

Overview of CARPE M&E 1. Partners submit

a workplan matrix and proposed budget Analyzed by

CARPE using rubric (ex )

FY06 Workplans were submitted in CARPE IIB RFA proposals

Review Criteria Approval

Substantive Content of Work Plan

Distribution of tasks is appropriate given the current stage of the LUP process Yes

Distribution of tasks between macro zones is balanced (50+% outside PA’s) Yes

Tasks are included that constitute a reason- able stakeholder engagement strategy Yes

Tasks include people-centered alternative livelihoods options to conservation threats

Yes

Data collection is appropriate to support current stage of the LUP process Yes

Training contributes to sustainable local institutional conservation capacity Yes

Coherence of Work Plan

Tasks are coherently linked to PMP. Yes

Work plan tasks are adequate for achieving specific BM’s. Yes

Activity categories are appropriate. Yes

Task responsibilities are clearly defined. Yes

Tasks are reasonable to achieve in given timeframe. Yes

Nature of tasks is clearly defined Yes

Task quarterly progress schedule (in %) is identified and reasonable Yes

Performance Monitoring

IR’s and macro zones are appropriately identified Yes

BM’s are consistent with the PMP Yes

MOV’s are acceptable, verifiable and document achievement of the BM’s Yes

Budget

Budget is segmented by macro zone, activity cat., USAID-cost share, partner Yes

Overview of CARPE M&E Workplan data is entered into a MS Access

database The database links Landscape financial and

performance data Financial: Activity budgeting by zone type, partner,

and country, semi-annual expenditure rate Performance: Each zone’s name, benchmark, size,

country location, and type Allows CARPE rough comparisons between

landscapes and a way to quickly scale up the regional program

Eventually data will be represented spatially using zone and landscape polygons

Overview of CARPE M&E

0

5

10

15

hectares

Millions

FY06 FY07 FY08fiscal year of funding

Planned ha under improved management by zone type

PACBNRMERZ

Overview of CARPE M&E 2. Partners submit Semi-Annual Report

(SAR) matrix and narrative SAR is a tool for partners to communicate to the

CARPE Operating Unit on progress toward benchmarks and any proposed changes Adaptive management—flexible Required in Cooperative Agreements

Level of Effort: No major changes made to matrix submitted as

workplan unless changes in benchmarks Percentage accomplished on right-side is updated

CARPE uses SAR performance (financial and programmatic) to influence budgeting of LS funds

Overview of CARPE M&E 3. Partners submit an Annual Report matrix,

narrative, and MOVs Matrices: Final results are updated into database for

reporting to Washington FACTS, Global Climate Change, 118/119, Performance

Report, etc

Narratives: Success stories and reporting is created from accomplishments listed here

MOVs: catalogued, cross-checked, and evaluated Important as a check to show that results were achieved Required by USAID data quality assessment Esp important due to limitation of field visits Some published on CARPE website

Proposed CARPE Reporting Calendar

CARPE analysis, workplan approval, and final budgetSeptember 1, 2007

Site visits Feb-Aug 2008

FY07 Workplan and FY06 SAR submitted

August 1, 2007

FY06 Final Report and MOVs submitted

Dec 1, 2007

CARPE analyzes data

for submission to Washington

Dec-Jan

Try to continue approximately this cycle for length of agreements to help planning, though flexibility is necessary

Updating of M&E System All proposed updates are small and have been

balanced with need for stability in reporting Changes will:

Better measure CARPE results as the program matures

Ensure CARPE receives complete information from bundled consortia Segments reporting was automatically disaggregated by

country Respond to Partners’ feedback in Final Reports Improve data quality Help CARPE reporting burden with reduction in team

size

Summary of Partners’ Feedback Thank you! Feedback was very helpful Across the board notes:

Request limitation in future changes to format – has stabilized since Feb 2005 workshop

Matrix is resource and time intensive Request CARPE distribute a clear reporting calendar to help

partners’ plan meeting logistics and time management Most partners find the matrix an important and useful

tool Some have incorporated it into their own LS monitoring plans Useful for team building and will be important for guiding new

consortia and managing sub-grants Smaller segments and segments with many donors found it

burdensome Request for French translation of reporting tools

Matrix available in French, guidance currently is not Recommendation for more site visits by CTO

Proposed Updates to M&E System:SO level indicator

SO Level indicator from PMP: “Population status for selected biodiversity

“indicator” species such as: wide-ranging “landscape” species and/or ecological keystone species (e.g. elephants, large predators) and/or globally threatened species (such as, mountain gorillas, bonobos, etc.)”

Program maturity Unit of measurement described in PMP: No. of

individuals of indicator species Timing: Biennial/triennial Compatible with SOFR Indicators Will be separated and not linked to tasks

Updated Matrix People trained summarized on separate page

Reported by “training event” Disaggregated by gender No. of days trained

On-going tasks link to MOVs and LUP progress should be described as best as possible

Percentages now given semi-annually on workplanning (right) side

Changes from Workplan or SAR marked in RED font and explained in narrative no “comparison percentages” (since changes highlighted in

red) Zone names

Add zone ID (assigned by CARPE, see future handout) Add country where zone is found in parentheses, below name

Updated Matrix Break down budgets by country whenever

possible, even within the same NGO Example: WWF in TNS Cameroon and CAR Linked to Transformational Diplomacy

reporting requirements Annual Reports and Workplans will be

posted on the CARPE website (without financial data) Feedback form

Updates: Means of Verification (MOVs)

Hyperlinking has helped control the MOVs, however some were missing or mis-linked for FY05

If MOVs are missing without explanation, CARPE will report the benchmark as not achieved in Washington reporting

MOVs are sifted through by CARPE team and appropriate material will be published on the CARPE website

Narrative Updates

Narrative format will continue in the same format as before, with increased page limits to accommodate LS consortia Suggestions welcome in feedback form

Annual Report will include “success stories” with photos, like the Final Reports of Phase IIA

GIS Data CARPE partners are active in LUP in

more than 150 zones Shapefile data received for Phase IIA

zones Quality was uneven, few partners sent in

metadata, required great time and resources from UMD/NASA team to clean data

Destination for shapefiles CARPE website CARPE management tool

GIS next steps

Landscapes will give CARPE the names and contact information for their GIS experts so that they can be contacted directly

The polygons for the remaining zones for Phase IIB will be sent in to CARPE by a date agreed at this workshop Suggestion: one month from now?

Feedback and distribution of tools A form has been created and feedback on

all of these issues is welcome and appreciated Feedback will be compiled and a consensus

distributed by end of workshop. Most importantly: Reporting Calendar GIS point of contact and polygon due date

Feedback and distribution of tools

The updated tools (Workplan, SAR, and AR matrices and narratives), zone ids, summary of changes, and this presentation will be available on the CARPE website and can be transferred electronically by flashdisk at this workshop after feedback is compiled and incorporated

The summary of the decisions made here on CARPE M&E will also be distributed in paper after the feedback has been compiled