© Commonwealth of Australia 2003

The Quality Assessment Framework

The 16th APEC Ministerial meeting

“Ministers approved a new Quality Assessment Framework, which will replace the ECOTECH Weightings Matrix, as a tool to foster good quality project proposals and an efficient allocation of APEC resources.”

2005 Senior Officials Meeting (SOMI), Seoul, Korea

Senior Officials endorsed the Small Group on Evaluation’s 2005 Work Program

The Work Program further develops the approach to strengthen the quality of APEC projects and implements the key directive from the 16th AMM Joint Statement.

Three key recommendations for implementation in 2005

1. The evaluation frameworks be adopted and implemented for Operational Account Projects from 2005 onwards.

2. The Guidebook on APEC Projects be updated to incorporate the evaluation frameworks, including replacing the ECOTECH Weightings Matrix and updating the project application form

3. APEC Working groups continue to receive technical assistance and support in implementing frameworks

Steps to achieving recommendations

Projects being proposed for funding through the Operational Account from 2005 onwards will be assessed against the Assessment Framework prior to BMC

Continuation of the trial process of the Assessment, Monitoring and Evaluation Framework (AM&E) focussing on M&E

The Secretariat will complete the revisions of the Guidebook and project application form

AM&E coordinator (Michelle Lowe) will coordinate support to the working groups.

Australia will continue to fund an evaluation consultant.

The Quality Assessment, Monitoring and Evaluation Frameworks (AM&E)

The Assessment Framework

Will the project contribute to APEC’s mission?

The Monitoring Framework

Is the project on track to achieving its objective?

The Evaluation Framework

Have the objectives been achieved?

How to use the Quality Assessment Framework (QAF)

Each working group establishes a small group to assess project proposals

16 design criteria to be assessed

Each design criterion is scored from 3 to 0.

-Good practice (3)-Satisfactory (2)-Marginally satisfactory (1)-Weak (0) -Not applicable (0)

All criteria are of equal value, there is no weighting system

Comments made against each criterion can be a simple yes or no.

Add scores together to get a final score. Projects can be ranked in order of merit.

Lead shepherds ensure the framework is objectively filled in and are either kept informed or participate in the group. They should send the filled out QAF to the proposal proponent and project overseers to allow them to make revisions if required.

Submit final project proposal, with QAF attached, to BMC. This will clearly indicate the strengths and weaknesses of the proposal to the Committee.

Examples

1. Does the proposal clearly state which one of the APEC priority themes of the 1996 Manila Declaration will be addressed?

One, and only one, priority themes in the 1996 Manila Declaration should be chosen as the project goal. For example: Safeguarding the quality of life through environmentally sound growth.

Comments (if any) AssessmentYes 2

9. Is the implementation methodology described in the proposal both clear and effective? Have the main risks to implementation been identified?

Explanation: Implementation should be well-planned and logical, and based on a breakdown of functions over time. Main risks or assumptions should be identified and practical strategies prepared to manage them.

Comments AssessmentNo, needs more detail 1

13. Does the proposal have a strategy to make project benefits sustainable?

Explanation: Project benefits should last well beyond project completion. For example, regular up-dates of documents and training, ensuring that manuals are in local language, etc.

Comments Assessment

Yes 3

16.Final statement of suitability for APEC funding

If implemented, is the project likely to make a sufficient contribution to APEC’s mission to justify its funding? Considers all relevant factors plus any

others specific to a working group.

Comments Assessment

Yes, high priority 3

Phase 2- Assessment, Monitoring and Evaluation

The Monitoring and Evaluation Frameworks will be trialled and refined in 2005. With a view to being adopted at the 17th AMM

and implemented in 2006.

Final AM+E Frameworks will be reviewed annually.

Benefits: Ensures projects on track, helps achieve the desired outcomes, enhances APEC credibility and supports regional development

Next steps for Working group projects

Lead Shepherds should ensure projects being put to the BMC for Operational Account funding have been assessed using the Quality Assessment Framework (QAF)

The Working Group should consider trialling the M&E Frameworks in 2005 (particularly those projects involved in the initial trial of the QAF)

Contacts

For any further advice on the AM&E Frameworks please contact:

Michelle Lowe

ml@apec.org

Ph (65) 6772 7658