Post on 23-May-2020
transcript
1
Terms of Reference- Senior Monitoring and Evaluation Expert
PROJECT FINAL EVALUATION
Project Title: Strengthening of the Democratic Process in Egypt Post Title: Short-term Senior Monitoring and Evaluation Expert Application Deadline: 10 April 2016
I. Background and Context: The UNDP electoral assistance project “Strengthening of the Democratic Process in Egypt” was signed in November 2011 between the High Elections Committee, the Ministry of Foreign Affairs and UNDP and was designed to support national efforts to enhance the credibility and sustainability of electoral institutions and processes, with a particular emphasis on capacity development and women’s empowerment. The project focused on four components: (i) strengthening public outreach and voter information, (ii) upgrading technical and operational capacity of national electoral authorities; (iii) incorporating “lessons-learned” and best practices in the subsequent elections processes, and (iv) increasing access of women and rural dwellers to their citizenship rights. The project, which was completed in December 2015, had a total budget of approximately US$ US$20,668,187 supported by 12 Development Partners namely Austria, Australia, Belgium, Germany, Ireland, Japan, Norway, Netherlands, Romania, Sweden and Switzerland and UNDP. Within the first component “strengthening the public outreach and voter information” the project supported SIS/HEC/PEC in a) the implementation of the public outreach campaigns; b) the dissemination of nationally devised gender sensitive voter information products; c) the implementation of a media relations strategy through organization of media trainings and workshops; and d) the production of a media-elections handbook to enhance the media’s understanding of the electoral cycle. Furthermore, during 2013-2015, the project supported SIS/HEC in the production of a gender sensitive civic and voter education programme for broadcast, TV and print, through the Voter Education Initiative focusing on rural women and youth. Within the second component “upgrading technical and operational capacity of national electoral authorities”, the project provided technical assistance to SIS/HEC/PEC/MOI by a) developing and producing training materials such as illustrated procedural manuals and simulation videos for judges and arranging an orientation on electoral procedures, b) upgrading IT infrastructure and procuring various electoral materials and equipment including the procurement of a data centre to host the electoral register and all electoral related information; c) availing opportunities for electoral authorities to participate in international for a to share comparative knowledge and experiences and d) organizing 6 workshops and international conferences in Cairo on different electoral topics. Within component 3, “incorporating “lesson-learned” and best practices of the 2011 elections for subsequent electoral processes”, the project helped organize lessons-learned workshops after various electoral events such that future electoral processes benefit from knowledge sharing and experiences. Furthermore, an Arabic lexicon of electoral terminology was developed, published and disseminated in order to increase the understanding within the Arab region of the different electoral terms in eight countries. Within component 4, “increasing access of women and rural dwellers to their citizenship rights in the 2011 and future elections”, the project contributed to the Women Citizenship Initiative led by
2
UNWomen and which aims to facilitate the on-going national registration process for the issuance of national ID cards for women over a period of three years. As of November 2015, the National Council for Women (NCW) became the lead national partner for the initiative. Nonetheless, MPMAR was responsible for issuing 181,868 ID cards to date. The Ministry has developed different implementation models to speed up the process of ID cards issuance and to maximize efficiency. The initiative is supported by a strong gender information campaign on TV, radio, social media and printed materials. Throughout its implementation , the project worked with various counterparts including the High Electoral Commission (HEC), the Presidential Electoral Commission (PEC), the Ministry of Communication and Information Technologies (MCIT), the Ministry of Foreign Affairs (MoFA), the Ministry of Interior (MoI), the Ministry of State for Administrative Development (MSAD) (currently the Ministry of Planning, Monitoring and Administrative Reform (MPMAR)), the State Information Service (SIS), the National Council for Women (NCW), the Social Fund for Development (SFD), the Information Decision Support Centre (IDSC); United Nations UNWomen and the International Foundation for Electoral Systems (IFES).
II. Evaluation purpose:
As stipulated in the signed project document, the project is expected to conduct a project evaluation to analyse the achievements of the project against its original objectives while providing project partners with an independent review of project outputs. The evaluation will review technical and managerial aspects and consider issues of effectiveness, efficiency, relevance, impact and sustainability. The evaluation will identify factors that have facilitated and/or impeded the achievement of objectives and should result in recommendations and lessons learned that could benefit the various stakeholders. The evaluation will specifically consider aspects of monitoring and evaluation; capacity building in electoral administration; and gender mainstreaming.
III. Evaluation scope and objectives:
Assess the status of project results and how they have been achieved with an assessment of UNDP’s contribution/approach
Identify factors that have facilitated and/or impeded the achievement of objectives
Assess the relevance, efficiency, effectiveness, and sustainability of the project
Assess the impact of external and internal factors on the effectiveness of the project
Assess the efficiency and the adequacy of the management arrangements of the project
Evaluation of the project strategy in enhancing national capacity in electoral administration
Identify lessons learned and good practices with regards to project implementation and partnerships
Assess the Monitoring and Evaluation framework used by the project
The evaluation will cover the duration of the project starting from November 2011 till the project completion in December 2015, and will also take into account those activities that were carried out before the signing of the project document.
IV. Evaluation Criteria:
The project will be evaluated against the following criteria:
Relevance: the evaluation will assess the degree to which the project remained relevant to the context during which it was implemented and in accordance with the priorities of the
3
Government of Egypt, the mandate of UN electoral assistance and UNDP goals in Democratic Governance.
Efficiency/management: the evaluation will assess the outputs realized in relation to the inputs, timeframe, funds and expertise provided, looking for example at whether the management structure was appropriate.
Effectiveness: the evaluation will assess the extent to which the project outputs, outcomes and results have been achieved.
Impact and sustainability: the evaluation team should assess the lasting change brought about by the project.
Gender sensitivity: the evaluation will assess to what degree was the project sensitive to the inclusion of Gender.
V. Evaluation questions
Evaluation questions set the parameters of the evaluation and when answered, will give users of the evaluation the information they seek in order to make decisions, take action or add to knowledge. Evaluation Questions should include at least the following questions:
Were stated outputs achieved?
What progress toward the outputs has been made?
What factors have contributed to achieving or not achieving intended results?
Were the actions to achieve the outputs and outcomes effective and efficient? Has the project been effective in providing support to the electoral authorities in Egypt in effectively conducting elections in the areas stated in the project?
Did the project make the best use of its resources to achieve its results? Has the project been efficient in implementing its activities?
What factors contributed to effectiveness or ineffectiveness?
Has the project partnership strategy been appropriate and effective? To what extent the project has been able to build and promote its partnership with other relevant stakeholders for greater results?
What efforts have been made to encourage sustainability of the project results?
What are the lessons-learned of this project for stakeholders in and outside Egypt?
VI. Methodology
1. Preparation of Inception Report: The methodology that will be used by the evaluators should be presented in the inception brief and the final report in detail. The methodology must be agreed upon between UNDP, the evaluators and project partners prior to the start of the evaluation.
2. Data Collection: a. Documentation review: This would include a) the Project Document with the
description of the project, its intended outcome and outputs, the baseline for the outcome/outputs and the indicators used; b) Project Progress Reports; c) Project Technical Reports including training materials, publications etc.
b. Key participants interviews with MOFA, UNDP, key focal points in HEC, PEC, and SIS, MOI, MPMAR, Development Partners, the Chief Electoral Advisor (via skype call) and the Project Coordinator (via Skype Calls), former project staff, UNWomen, and other partners such as Al Ahram Center for Strategic Studies and international partners such as IFES.
3. Preparation of Draft Final Report: with comments and inputs from various stakeholders and partners.
4
4. Preparation of Final Report
VII. Composition of the Evaluation Team The final evaluation will be carried out by a team of three national consultants: Monitoring and Evaluation Expert; Electoral Expert and Gender Expert to evaluate the gender dimensions of the project. All three consultants should not have participated in the project preparation and/or implementation and should not have any conflict of interest with project related activities. The estimated duration of the Evaluation is 2 months with a level of effort of 25 working days.
The Evaluation Team will be responsible for producing the following deliverables: 1. Inception Reports: each of the evaluation experts will be responsible for preparing an inception
report for their respective focus area 2. Draft Evaluation Reports: each of the evaluation experts will be responsible for preparing a
draft report for their respective focus area for discussion with MOFA and UNDP 3. Final Evaluation Reports (See Evaluation Report Template in Annex 3): each of the evaluation
experts will be responsible for submitting a final report for their respective focus area 4. Brief Executive Summary for each of the reports
VIII. Time-frame for the evaluation process
The evaluation will take place over a period of 25 days. The tentative schedule follows:
Planned Activities Tentative
Days
Desk review, Briefing Meetings with MOFA and UNDP and preparation of the evaluation i n c e p t i o n r e r e p r e p o r t r e p o r t design,
3 days
Finalization and Presentation of Inception Report to MOFA and UNDP
2 days
Meetings with National Partners, Development partners, Project Team etc Reference Group for feedback
15 days
Preparation of Draft Report and Presentation of draft findings to MOFA and UNDP
3 days
Finalize and submit Report
2 days
Total 25 days
To facilitate the evaluation process, MOFA and UNDP will assist with the organization of meetings with the relevant electoral authorities, development partners, institutions and key stakeholders. MOFA and UNDP will be responsible for preparing and coordinating the full agenda of the final evaluation in consultation with the relevant national and international stakeholders. It is anticipated that the mission will meet the following key partners including:
HEC Chair and spokesperson
Former Chair of PEC
Representatives of relevant ministries including Ministry of State for Administrative development, Ministry of Interior, State Information Services, Information Decision Support Centre
5
MOFA
Representatives of Think Tanks and Research Centers namely Al Ahram Center for Political and Strategic Studies and Baseera
Representatives of Development Partners
Representative of UN Women
UNDP
IX. Roles and Responsibilities of the Senior Monitoring and Evaluation Expert
Facilitate meetings with national counterparts and development partners
Provide guidance in preparing Evaluation deliverables
Coordinate with the Evaluation team to consolidate inputs from Evaluation team and ensure timely delivery of Evaluation products
Prepare an inception brief identifying Monitoring and Evaluation aspects of project formulation and implementation to be addressed and the proposed methodology of data collection and analysis
Prepare a separate M&E report to be integrated into the final evaluation report and presentation
X. Deliverables of Senior Monitoring and Evaluation Expert:
1. Inception Brief to outline main evaluation issues that will be addressed, including the relevant evaluation questions and the proposed and final methodology that has been agreed upon before the evaluation is set to begin. The Senior M&E Expert will prepare an inception brief on the M&E aspects of the project. The brief will be presented to MOFA and UNDP in a joint meeting with the Electoral and Gender Experts.
2. Draft Evaluation Report and Presentation of Findings— The draft report covering M&E issues of the project will be prepared and presented by the Senior M&E Expert. The report will be shared with both MOFA and UNDP, following which a meeting will be held to discuss it, along with reports prepared by the Electoral and Gender Experts, to ensure that the evaluation meets MOFA and UNDP expectations as stipulated in the Evaluation Terms of Reference.
3. Final Evaluation Report including Executive Summary
XI. Qualifications: The Senior Monitoring and Evaluation Expert will possess the following qualifications and
competencies:
Advanced university degree in Public Policy and Administration, Law, International Law, Development Management or related discipline with 5 years’ experience
Recognized experience in Monitoring and Evaluation and results-based management evaluation methodologies
Recognized experience in evaluating development projects
Familiarity with recent UNDP Monitoring and Evaluation Policy
Experience with multilateral or bilateral supported projects
Fluency in English and possession of strong technical writing and analytical skills
6
XII. Evaluation of Applicants
Individual consultants will be evaluated based on a cumulative analysis taking into consideration the combination of the applicants’ qualifications and financial proposal. The award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:
Responsive/compliant/acceptable, and Having received the highest score out of a pre-determined set of weighted technical and
financial criteria specific to the solicitation. Only the highest ranked candidates who would be found qualified for the job will be considered for the Financial Evaluation. Technical Criteria – 70% of total evaluation
Weight
Relevant professional experience 30
Proposed methodology, its appropriateness to the assignment, and timeliness of the implementation plan
50
Previous working experience on similar assignments
20
Total 100
Financial Criteria – 30% of total evaluation To be computed as a ratio of the Proposal’s offer to the lowest price among the proposals received by UNDP. Applicants receiving a score less than 70% will be technically disqualified.
XIII. Terms of Payment
30% upon submission of Inception Report on M&E
30% upon submission of Draft Evaluation Report on M&E
40% upon submission of Final Report, Executive Summary on M&E
XIV. Evaluation ethics
All UNDP Programme and project evaluations are to be conducted in accordance with the principles
outlined in the UNEG ‘Ethical Guidelines for Evaluation’ and the UNEG Code of Conduct for
Evaluation in the UN System. Both documents can be found at the following link:
http://www.uneval.org/search/index.jsp?q=ethical+guidelines
Evaluations of UNDP-supported activities need to be independent, impartial and rigorous. Each evaluation should clearly contribute to learning and accountability. Hence, evaluators must have personal and professional integrity and be guided by propriety in the conduct of their business.
7
Evaluators:
Must present information that is complete and fair in its assessment of strengths and weaknesses so that decisions or actions taken are well founded.
Must disclose the full set of evaluation findings along with information on their limitations and have this accessible to all affected by the evaluation with expressed legal rights to receive results.
Should protect the anonymity and confidentiality of individual informants. They should provide maximum notice, minimize demands on time, and: respect people’s right not to engage. Evaluators must respect people’s right to provide information in confidence, and must ensure that sensitive information cannot be traced to its source. Evaluators are not expected to evaluate individuals, and must balance an evaluation of management functions with this general principle.
Evaluators should consult with other relevant oversight entities when there is any doubt about if and how issues should be reported.
Should be sensitive to beliefs, manners and customs and act with integrity and honesty in their relations with all stakeholders. In line with the UN Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender equality. They should avoid offending the dignity and self-respect of those persons with whom they come in contact in the course of the evaluation. Knowing that evaluation might negatively affect the interests of some stakeholders, evaluators should conduct the evaluation and communicate its purpose and results in a way that clearly respects the stakeholders’ dignity and self-worth.
Are responsible for their performance and their product(s). They are responsible for the clear, accurate and fair written and/or oral presentation of study limitations, findings and recommendations.
Should reflect sound accounting procedures and be prudent in using the resources of the evaluation.
The Evaluators must sign an Evaluation Staff Agreement form at the start of their contract (see Annex 2).
XV. Implementation Arrangements
The principal responsibility for managing this evaluation lies with MOFA and UNDP. The report will be cleared by MOFA and UNDP. The time frame above does not include two weeks of unpaid time, during which MOFA and UNDP Egypt will analyse, provide comments to the evaluators and share the draft report with different stakeholders.
Interested applicants are invited to send a CV, P11 form and cover letter (specifying the title of the consultancy); proposed methodology outlining how the Consultant will execute the assignment; written sample of prior evaluation and/or assessment work; and detailed financial proposal to the following e-mail: alya.hegazy@undp.org
Vacancy Notice issued on: 27 March 2016 Deadline for application is: 10 April 2016
8
ANNEX 1
Documents to be consulted— this is a list of important documents and Webpages that the evaluator(s)
should read at the outset of the evaluation and before finalizing the evaluation design and the inception
report.
Project Document and any revisions
Websites: o www.undp.org.eg o https://www.elections.eg/
UNDAF and UNDP CPD/CPAP
UNDP M&E Yellow Handbook
Project operational guidelines, manuals and systems
Quarterly Progress Report and detailed activity progress reports
Project Annual reports
Minutes of Board meetings and other project management meetings.
Presentations and other inputs to Board Meetings and project management meetings
Combined Delivery Report
Atlas Reports (such as the AWP and Project Budget Balance report)
Project Implementation Reviews
UNDP User Guide (relevant sections)
Other Reference Documents produced by the project
Citizenship Initiative project document
Mapping study of UN WOMEN
Concept notes on the various round tables and sub-regional conferences
Reports on study tours and lessons learnt exercise of HEC, PEC
Reports of the internal UNDP meetings with the UNDP electoral assistance projects in Libya and Tunisia.
Action plan
9
ANNEX 2
United Nations Evaluation Group – Code of Conduct for Evaluation in the UN System
Evaluation Consultants Agreement Form
To be signed by all consultants as individuals (not by or on behalf of a consultancy company) before a
contract can be issued.
Agreement to abide by the Code of Conduct for Evaluation in the UN System
Name of Consultant:
__________________________________________________________________
Name of Consultancy Organization (where relevant):
________________________________________
I confirm that I have received and understood and will abide by the United Nations Code of Conduct
for Evaluation.
Signed at ( ) on ( )
Signature: __________________________________________________________________
10
ANNEX 3
EVALUATION REPORT TEMPLATE AND QUALITY STANDARDS
This evaluation report template is intended to serve as a guide for preparing meaningful, useful and
credible evaluation reports that meet quality standards. It does not prescribe a definitive section-by-
section format that all evaluation reports should follow. Rather, it suggests the content that should be
included in a quality evaluation report. The descriptions that follow are derived from the UNEG
‘Standards for Evaluation in the UN System’ and ‘Ethical Standards for Evaluations’.
The evaluation report should be complete and logically organized. It should be written clearly and
understandable to the intended audience. In a country context, the report should be translated into
local languages whenever possible (see Chapter 8 for more information). The report should also include
the following:
Title and opening pages—must provide the following basic information:
Name of the evaluation intervention
Time-frame of the evaluation and date of the report
Countries of the evaluation intervention
Names and organizations of evaluators
Name of the organization commissioning the evaluation
Acknowledgements
Table of contents—must always include boxes, figures, tables and annexes with page references.
List of acronyms and abbreviations
Executive summary—A stand-alone section of two to three pages that should:
Briefly describe the intervention of the evaluation (the project(s), programme(s), policies or
other intervention) that was evaluated.
Explain the purpose and objectives of the evaluation, including the audience for the evaluation
and the intended uses.
Describe key aspect of the evaluation approach and methods.
Summarize principle findings, conclusions, and recommendations.
Introduction—Should:
Explain why the evaluation was conducted (the purpose), why the intervention is being
evaluated at this point in time, and why it addressed the questions it did.
Identify the primary audience or users of the evaluation, what they wanted to learn from the
evaluation and why and how they are expected to use the evaluation results.
Identify the intervention of the evaluation (the project(s) programme(s) policies, or other
intervention—see upcoming section on intervention.)
11
Acquaint the reader with the structure and contents of the report and how the information
contained in the report will meet the purposes of the evaluation and satisfy the information
needs of the report’s intended users.
Description of the intervention—provides the basis for report users to understand the logic and asses
the merits of the evaluation methodology and understand the applicability of the evaluation results. The
description needs to provide sufficient detail for the report user to derive meaning from the evaluation.
The description should:
Describe what is being evaluated, who seeks to benefit, and the problem or issue it seeks to
address.
Explain the expected results map or results framework, implementation strategies, and the key
assumptions underlying the strategy.
Link the intervention to national priorities, UNDAF priorities, corporate multi-year funding
frameworks or strategic plan goals, or other programme or country specific plans and goals.
Identify the phase in the implementation of the intervention and any significant changes (e.g.,
plans, strategies, logical frameworks) that have occurred over time, and explain the implications
of those changes for the evaluation.
Identify and describe the key partners involved in the implementation and their roles.
Describe the scale of the intervention, such as the number of components (e.g., phases of a
project) and the size of the target population for each component.
Indicate the total resources, including human resources and budgets.
Describe the context of the social, political, economic and institutional factors, and the
geographical landscape within which the intervention operates and explain the effects
(challenges and opportunities) those factors present for its implementation and outcomes.
Point out design weaknesses (e.g., intervention logic) or other implementation constraints (e.g.,
resource limitations).
Evaluation scope and objectives—The report should provide a clear explanation of the evaluation’s
scope, primary objectives and main questions.
Evaluation scope—the report should define the parameters of the evaluation, for example, the
time period, the segments of the target population included, the geographic area included, and
which components, outputs or outcomes were and were not assessed.
Evaluation objectives—the report should spell out the types of decisions evaluation users will
make, the issues they will need to consider in making those decisions, and what the evaluation
will need to achieve to contribute to those decisions.
Evaluation criteria—the report should define the evaluation criteria or performance standards
used. The report should explain the rationale for selecting the particular criteria used in the
evaluation.
12
Evaluation questions—Evaluation questions define the information that the evaluation will
generate. The report should detail the main evaluation questions addressed by the evaluation
and explain how the answers to these questions address the information needs of users.
Evaluation approach and methods—The evaluation report should describe in detail the selected
methodological approaches, methods and analysis; the rationale for their selection; and how, within the
constraints of time and money, the approaches and methods employed yielded data that helped answer
the evaluation questions and achieved the evaluation purposes. The description should help the report
users judge the merits of the methods used in the evaluation and the credibility of the findings,
conclusions and recommendations. The description on methodology should include discussion of each
of the following:
Data sources—the sources of information (documents reviewed and stakeholders), the rationale
for their selection and how the information obtained addressed the evaluation questions.
Sample and sampling frame—If a sample was used: the sample size and characteristics; the
sample selection criteria (e.g., single women, under 45); the process for selecting the sample
(e.g., random, purposive); if applicable, how comparison and treatment groups were assigned;
and the extent to which the sample is representative of the entire target population, including
discussion of the limitations of sample for generalizing results.
Data collection procedures and instruments—Methods or procedures used to collect data,
including discussion of data collection instruments (e.g., interview protocols), their
appropriateness for the data source, and evidence of their reliability and validity.
Performance standards—the standard or measure that will be used to evaluate performance
relative to the evaluation questions (e.g., national or regional indicators, rating scales).
Stakeholder participation—Stakeholders’ participation in the evaluation and how the level of
involvement contributed to the credibility of the evaluation and the results.
Ethical considerations—The measures taken to protect the rights and confidentiality of
informants (see UNEG ‘Ethical Guidelines for Evaluators’ for more information).
Background information on evaluators—the composition of the evaluation team, the
background and skills of team members, and the appropriateness of the technical skill mix,
gender balance and geographical representation for the evaluation.
Major limitations of the methodology—Major limitations of the methodology should be
identified and openly discussed as to their implications for evaluation, as well as steps taken to
mitigate those limitations.
Data analysis—the report should describe the procedures used to analyse the data collected to answer
the evaluation questions. It should detail the various steps and stages of analysis that were carried out,
including the steps to confirm the accuracy of data and the results. The report also should discuss the
appropriateness of the analyses to the evaluation questions. Potential weaknesses in the data analysis
and gaps or limitations of the data should be discussed, including their possible influence on the way
findings may be interpreted and conclusions drawn.
13
Findings and conclusions—the report should present the evaluation findings based on the analysis and
conclusions drawn from the findings.
Findings—should be presented as statements of fact that are based on analysis of the data. They
should be structured around the evaluation questions so that report users can readily make the
connection between what was asked and what was found. Variances between planned and
actual results should be explained, as well as factors affecting the achievement of intended
results. Assumptions or risks in the project or programme design that subsequently affected
implementation should be discussed.
Conclusions—should be comprehensive and balanced, and highlight the strengths, weaknesses
and outcomes of the intervention. They should be well substantiated by the evidence and
logically connected to evaluation findings. They should respond to key evaluation questions and
provide insights into the identification of and/or solutions to important problems or issues
pertinent to the decision-making of intended users.
Recommendations—the report should provide practical, feasible recommendations directed to the
intended users of the report about what actions to take or decisions to make. The recommendations
should be specifically supported by the evidence and linked to the findings and conclusions around key
questions addressed by the evaluation. They should address sustainability of the initiative and comment
on the adequacy of the project exit strategy, if applicable. Recommendations should also provide
specific advice for future or similar projects or programming.
Lessons learnt—As appropriate, the report should include discussion of lessons learnt from the
evaluation, that is, new knowledge gained from the particular circumstance (intervention, context
outcomes, even about evaluation methods) that are applicable to a similar context. Lessons should be
concise and based on specific evidence presented in the report.
Report annexes—suggested annexes should include the following to provide the report user with
supplemental background and methodological details that enhance the credibility of the report:
ToR for the evaluation
Additional methodology-related documentation, such as the evaluation matrix and data
collection instruments (questionnaires, interview guides, observation protocols, etc.) as
appropriate
List of individuals or groups interviewed or consulted and sites visited
List of supporting documents reviewed
Project or programme results map or results framework
Summary tables of findings, such as tables displaying progress towards outputs, targets, and
goals relative to established indicators
Short biographies of the evaluators and justification of team composition
Code of conduct signed by evaluators