Changes in and Goals of USAID M&E Reporting Requirements
February 6, 2007Jackie Doremus, CARPE M&E Consultant
Outline of Presentation
Success of M&E to date – Broadly and the Operational Plan case study
Overall goals of CARPE M&E – why do we want this information?
Brief overview of current M&E system Proposed CARPE M&E Calendar Feedback from Partners’ Phase IIA
final reports
Outline of Presentation
Proposed updates to CARPE M&E system SO level indicator Updated matrix MOVs Narrative structure for LS consortia GIS data
Feedback and distribution of tools
Success of CARPE M&E System: Broadly
M&E system developed to satisfy all CARPE USAID reporting requirements, finalized Feb 2005
Effectively communicated results to Congress, helping ensure a fourth year of funding
Success of CARPE M&E System: A case study
December 2006 saw major changes in USAID’s reporting structure New, comprehensive reporting tool, the
Operational Plan, required for each Operating Unit
Entered into a web-based system, FACTS: Foreign Assistance Coordination and Tracking System
Data requirements much greater than previous USAID reports
Very little ‘warning’ or collaboration with the field on the content of the report
Success of CARPE M&E System: A case study
CARPE team was only SO in USAID/DRC to not return to partners for repeated information requests to satisfy FACTS*
Operational Plan design mimicked CARPE database design
Information entered into FACTS was auditable and robust due to MOVs and matrices
FACTS stats from Matrix
FACTS required activity budgeting Average % of the total LS budget spent
on “Training/Capacity Building” is 14% CARPE funds led to at least 2,163
people receiving training with FY06 funds
FY06 LS funds were used to improve management on 32.6 M ha
Goals of CARPE M&E
FUNDING Determine project warrants funding Use all available resources to communicate results and
ensure funding Regular conduit of communication between
partners and CTO Better to have structured and well-anticipated report
submission than ad hoc requests and changing requirements
Must efficiently use time and resources of partners during reporting submission and of CARPE team during processing of reports Matrix requires an initial investment from partners to learn
format, then an annual large investment when consortia create the workplan
Landscape leaders will need to help new subpartners
Goals of CARPE M&E
USG Grant Management
Performance-Based
Agreements
Communication with
Stakeholders
Conservation Knowledge
CARPE M&E
Overview of CARPE M&E
Semi-Annual Reporting Cycle: 1. A workplan matrix and narrative 2. A Semi-Annual Report matrix and
narrative 3. An Annual Report matrix and narrative
MOVs
A one-time GIS data submission
Overview of CARPE M&E 1. Partners submit
a workplan matrix and proposed budget Analyzed by
CARPE using rubric (ex )
FY06 Workplans were submitted in CARPE IIB RFA proposals
Review Criteria Approval
Substantive Content of Work Plan
Distribution of tasks is appropriate given the current stage of the LUP process Yes
Distribution of tasks between macro zones is balanced (50+% outside PA’s) Yes
Tasks are included that constitute a reason- able stakeholder engagement strategy Yes
Tasks include people-centered alternative livelihoods options to conservation threats
Yes
Data collection is appropriate to support current stage of the LUP process Yes
Training contributes to sustainable local institutional conservation capacity Yes
Coherence of Work Plan
Tasks are coherently linked to PMP. Yes
Work plan tasks are adequate for achieving specific BM’s. Yes
Activity categories are appropriate. Yes
Task responsibilities are clearly defined. Yes
Tasks are reasonable to achieve in given timeframe. Yes
Nature of tasks is clearly defined Yes
Task quarterly progress schedule (in %) is identified and reasonable Yes
Performance Monitoring
IR’s and macro zones are appropriately identified Yes
BM’s are consistent with the PMP Yes
MOV’s are acceptable, verifiable and document achievement of the BM’s Yes
Budget
Budget is segmented by macro zone, activity cat., USAID-cost share, partner Yes
Overview of CARPE M&E Workplan data is entered into a MS Access
database The database links Landscape financial and
performance data Financial: Activity budgeting by zone type, partner,
and country, semi-annual expenditure rate Performance: Each zone’s name, benchmark, size,
country location, and type Allows CARPE rough comparisons between
landscapes and a way to quickly scale up the regional program
Eventually data will be represented spatially using zone and landscape polygons
Overview of CARPE M&E
0
5
10
15
hectares
Millions
FY06 FY07 FY08fiscal year of funding
Planned ha under improved management by zone type
PACBNRMERZ
Overview of CARPE M&E 2. Partners submit Semi-Annual Report
(SAR) matrix and narrative SAR is a tool for partners to communicate to the
CARPE Operating Unit on progress toward benchmarks and any proposed changes Adaptive management—flexible Required in Cooperative Agreements
Level of Effort: No major changes made to matrix submitted as
workplan unless changes in benchmarks Percentage accomplished on right-side is updated
CARPE uses SAR performance (financial and programmatic) to influence budgeting of LS funds
Overview of CARPE M&E 3. Partners submit an Annual Report matrix,
narrative, and MOVs Matrices: Final results are updated into database for
reporting to Washington FACTS, Global Climate Change, 118/119, Performance
Report, etc
Narratives: Success stories and reporting is created from accomplishments listed here
MOVs: catalogued, cross-checked, and evaluated Important as a check to show that results were achieved Required by USAID data quality assessment Esp important due to limitation of field visits Some published on CARPE website
Proposed CARPE Reporting Calendar
CARPE analysis, workplan approval, and final budgetSeptember 1, 2007
Site visits Feb-Aug 2008
FY07 Workplan and FY06 SAR submitted
August 1, 2007
FY06 Final Report and MOVs submitted
Dec 1, 2007
CARPE analyzes data
for submission to Washington
Dec-Jan
Try to continue approximately this cycle for length of agreements to help planning, though flexibility is necessary
Updating of M&E System All proposed updates are small and have been
balanced with need for stability in reporting Changes will:
Better measure CARPE results as the program matures
Ensure CARPE receives complete information from bundled consortia Segments reporting was automatically disaggregated by
country Respond to Partners’ feedback in Final Reports Improve data quality Help CARPE reporting burden with reduction in team
size
Summary of Partners’ Feedback Thank you! Feedback was very helpful Across the board notes:
Request limitation in future changes to format – has stabilized since Feb 2005 workshop
Matrix is resource and time intensive Request CARPE distribute a clear reporting calendar to help
partners’ plan meeting logistics and time management Most partners find the matrix an important and useful
tool Some have incorporated it into their own LS monitoring plans Useful for team building and will be important for guiding new
consortia and managing sub-grants Smaller segments and segments with many donors found it
burdensome Request for French translation of reporting tools
Matrix available in French, guidance currently is not Recommendation for more site visits by CTO
Proposed Updates to M&E System:SO level indicator
SO Level indicator from PMP: “Population status for selected biodiversity
“indicator” species such as: wide-ranging “landscape” species and/or ecological keystone species (e.g. elephants, large predators) and/or globally threatened species (such as, mountain gorillas, bonobos, etc.)”
Program maturity Unit of measurement described in PMP: No. of
individuals of indicator species Timing: Biennial/triennial Compatible with SOFR Indicators Will be separated and not linked to tasks
Updated Matrix People trained summarized on separate page
Reported by “training event” Disaggregated by gender No. of days trained
On-going tasks link to MOVs and LUP progress should be described as best as possible
Percentages now given semi-annually on workplanning (right) side
Changes from Workplan or SAR marked in RED font and explained in narrative no “comparison percentages” (since changes highlighted in
red) Zone names
Add zone ID (assigned by CARPE, see future handout) Add country where zone is found in parentheses, below name
Updated Matrix Break down budgets by country whenever
possible, even within the same NGO Example: WWF in TNS Cameroon and CAR Linked to Transformational Diplomacy
reporting requirements Annual Reports and Workplans will be
posted on the CARPE website (without financial data) Feedback form
Updates: Means of Verification (MOVs)
Hyperlinking has helped control the MOVs, however some were missing or mis-linked for FY05
If MOVs are missing without explanation, CARPE will report the benchmark as not achieved in Washington reporting
MOVs are sifted through by CARPE team and appropriate material will be published on the CARPE website
Narrative Updates
Narrative format will continue in the same format as before, with increased page limits to accommodate LS consortia Suggestions welcome in feedback form
Annual Report will include “success stories” with photos, like the Final Reports of Phase IIA
GIS Data CARPE partners are active in LUP in
more than 150 zones Shapefile data received for Phase IIA
zones Quality was uneven, few partners sent in
metadata, required great time and resources from UMD/NASA team to clean data
Destination for shapefiles CARPE website CARPE management tool
GIS next steps
Landscapes will give CARPE the names and contact information for their GIS experts so that they can be contacted directly
The polygons for the remaining zones for Phase IIB will be sent in to CARPE by a date agreed at this workshop Suggestion: one month from now?
Feedback and distribution of tools A form has been created and feedback on
all of these issues is welcome and appreciated Feedback will be compiled and a consensus
distributed by end of workshop. Most importantly: Reporting Calendar GIS point of contact and polygon due date
Feedback and distribution of tools
The updated tools (Workplan, SAR, and AR matrices and narratives), zone ids, summary of changes, and this presentation will be available on the CARPE website and can be transferred electronically by flashdisk at this workshop after feedback is compiled and incorporated
The summary of the decisions made here on CARPE M&E will also be distributed in paper after the feedback has been compiled