Document of The World Bank FOR OFFICIAL USE ONLY Report No: 83161-AM INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT AND INTERNATIONAL DEVELOPMENT ASSOCIATION PROJECT APPRAISAL DOCUMENT ON A PROPOSED CREDIT IN THE AMOUNT OF SDR 9.8 MILLION (US$15 MILLION EQUIVALENT) AND ON A PROPOSED LOAN IN THE AMOUNT OF US$15.0 MILLION TO THE REPUBLIC OF ARMENIA FOR AN EDUCATION IMPROVEMENT PROJECT February 18, 2014 Human Development Sector Unit Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized
Transcript
Document of
The World Bank
FOR OFFICIAL USE ONLY
Report No: 83161-AM
INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT AND INTERNATIONAL DEVELOPMENT ASSOCIATION
PROJECT APPRAISAL DOCUMENT
ON A
PROPOSED CREDIT
IN THE AMOUNT OF SDR 9.8 MILLION (US$15 MILLION EQUIVALENT)
AND ON A
PROPOSED LOAN
IN THE AMOUNT OF US$15.0 MILLION
TO THE
REPUBLIC OF ARMENIA
FOR AN
EDUCATION IMPROVEMENT PROJECT
February 18, 2014
Human Development Sector Unit Europe and Central Asia Region
This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
ii
CURRENCY EQUIVALENTS
(Exchange Rate Effective December 31, 2013)
Currency Unit = Armenian Dram AMD409 = US$1 US$1.54 = SDR 1
FISCAL YEAR
January 1 – December 31
ABBREVIATIONS AND ACRONYMS
ANQA National Quality Assurance Agency GoA Government of Armenia ASRA Accounting Standards of the Republic
of Armenia HEI Higher Education Institution
ATRC Assessment and Testing Center IBRD International Bank for Reconstruction and Development
CEP Center for Education Project ICT Information and Communication Technologies
CFAA Country Financial Accountability Assessment
IDA International Development Association
CIF Competitive Innovation Fund IFRs Interim Financial Reports
CPS Country Partnership Strategy IFAC International Federation of Accountants
ECA Europe and Central Asia ISA International Standard on Auditing ECACDG Europe and Central Asia Capacity
Development Grant M&E Monitoring and Evaluation
EDI Early Development Index MoES Ministry of Education and Science EMF Environmental Management
Framework NACET National Center for Education Technology
EMIS Education Management Information System
PAS Procurement Accredited Staff
EMP Environmental Management Plan PDO Project Development Objectives EIP Education Improvement Project PFM Public Financial Management EQRP Education Quality and Relevance
Project PIP Project Implementation Plan
ETF European Training Foundation PRAMS Procurement Risk Assessment and
Management System FA Financing Agreement READ Russia Education Aid for Development FMM Financial Management Manual RoA Republic of Armenia GCF General Conversion Factor TIMSS Trends in International Mathematics and
Science Study GDP Gross Domestic Product UNDP United Nations Development Programme GPN General Procurement Notice VET Vocational Education and Training
Regional Vice President: Laura Tuck Country Director: Henry G. R. Kerali
Acting Sector Director: Alberto Rodriguez Acting Sector Manager: Andrea Guedes
Total Project Cost: 37.50 Total Bank Financing: 30.00
Financing Gap: 0.00
.
vi
Financing Source Amount
BORROWER/RECIPIENT 7.50
International Bank for Reconstruction and Development 15.00
International Development Association (IDA) 15.00
Total 37.50
.
Expected Disbursements (in USD Million)
Fiscal Year 2015 2016 2017 2018 2019 2020
Annual 8.00 7.00 5.00 5.00 3.00 2.00
Cumulative 8.00 15.00 20.00 25.00 28.00 30.00
.
Proposed Development Objective(s)
The Project Development Objectives (PDO) are to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper secondary schools, and support improved quality and relevance in higher education institutions in Armenia.
.
Components
Component Name Cost (USD Millions)
Component 1. Enhancing the Quality of General Education 27.50
Component 2. Mainstreaming of the Competitive Innovation Fund (CIF) for Higher Education Institutions (HEI) into full implementation
6.25
Component 3. Project Management, Monitoring and Evaluation 3.75
.
Institutional Data
Sector Board
Education
.
Sectors / Climate Change
Sector (Maximum 5 and total % must equal 100)
Major Sector Sector % Adaptation Co-benefits % Mitigation Co-benefits %
Education General education sector 50
Education Tertiary education 30
Education Pre-primary education 20
Total 100
I certify that there is no Adaptation and Mitigation Climate Change Co-benefits information applicable to this project.
.
vii
Themes
Theme (Maximum 5 and total % must equal 100)
Major theme Theme %
Human development Education for the knowledge economy 70
Human development Education for all 30
Total 100
.
Compliance
Policy
Does the project depart from the CAS in content or in other significant respects? Yes [ ] No [ X ]
.
Does the project require any waivers of Bank policies? Yes [ ] No [ X ]
Have these been approved by Bank management? Yes [ ] No [ X ]
Is approval for any policy waiver sought from the Board? Yes [ ] No [ X ]
Does the project meet the Regional criteria for readiness for implementation? Yes [ X ] No [ ]
.
Safeguard Policies Triggered by the Project Yes No
Environmental Assessment OP/BP 4.01 X
Natural Habitats OP/BP 4.04 X
Forests OP/BP 4.36 X
Pest Management OP 4.09 X
Physical Cultural Resources OP/BP 4.11 X
Indigenous Peoples OP/BP 4.10 X
Involuntary Resettlement OP/BP 4.12 X
Safety of Dams OP/BP 4.37 X
Projects on International Waterways OP/BP 7.50 X
Projects in Disputed Areas OP/BP 7.60 X
.
Legal Covenants
Name Recurrent Due Date Frequency
Schedule 2, Section I, A1 X CONTINUOUS
Description of Covenant
The Republic of Armenia shall maintain, at all times during Project implementation, the CEP as the Project Implementation Unit (PIU) within MOES, with a mandate, staffing and other resources satisfactory to the Bank (“PIU” or “CEP”)
Name Recurrent Due Date Frequency
Schedule 2, Section I, C2 X CONTINUOUS
viii
Description of Covenant
The Republic of Armenia shall ensure that the Micro-project Sub-grants and CIF Sub-project Sub-grants are made to Beneficiaries in accordance with eligibility criteria and procedures set forth in the Operational Manual and in Section I.C. of the Loan Agreement and the Financing Agreement.
Name Recurrent Due Date Frequency
Schedule 2, Section I, C3 X CONTINUOUS
Description of Covenant
Except as the Association may otherwise agree in writing, no proposed Micro-project and/or Sub-project shall be eligible for financing under unless the Recipient has determined the proposed Micro-project and/or Sub-project satisfies the eligibility criteria set out in the Loan Agreement, Financing Agreement and the Operational Manual.
Name Recurrent Due Date Frequency
Schedule 2, Section I, D1 X CONTINUOUS
Description of Covenant
The Republic of Armenia shall ensure that the Project is carried out in accordance with the Environmental Management Framework (EMF) and site-specific Environmental Management Plans (EMPs).
Name Recurrent Due Date Frequency
Schedule 2, Section I, D2 X CONTINUOUS
Description of Covenant
Except as the Bank may otherwise agree in writing, no proposed Micro-project or Sub-project shall be eligible for financing unless the Recipient has determined, on the basis of an appraisal, carried out in accordance with guidelines acceptable to the Bank, that the respective Micro-project or Sub-project satisfies the eligibility criteria for safeguards specified in the Project Operational Manual.
Name Recurrent Due Date Frequency
Schedule 2, Section II, A1 X CONTINUOUS
Description of Covenant
The Republic of Armenia shall monitor and evaluate the progress of the Project and prepare Project Reports in accordance with the provisions of Section 4.08 of the General Conditions and on the basis of the indicators acceptable to the Bank.
Name Recurrent Due Date Frequency
Schedule 2, Section II, B1 X CONTINUOUS
Description of Covenant
The Republic of Armenia shall maintain or cause to be maintained a financial management system in accordance with the provisions of Section 4.09 of the General Conditions.
.
Conditions
Name Type
Operational Manual adopted in a manner satisfactory to the Association/Bank Effectiveness
Description of Condition
The approval by the Ministry of Education and Science of an Operational Manual, including specific
ix
Operational Manuals for the grants for Pre-school Micro-projects and CIF Sub-projects, which is satisfactory to the Association/Bank.
Team Composition
Bank Staff
Name Title Specialization Unit
Mario Cristian Aedo Inostroza Sr Education Economist Team Lead ECSH2
Benedicta T. Oliveros-Miranda Procurement Analyst Procurement Analyst ECSO2
Satik S. Nairian Program Assistant Program Assistant ECCAR
Darejan Kapanadze Senior Environmental Specialist
Environmental Specialist ECSEN
Gabriel C. Francis Program Assistant Program Assistant ECSHD
Sarah G. Michael Senior Social Development Specialist
Social Development Specialist ECSSO
Garik Sergeyan E T Consultant E T Consultant ECSO3
Anush Shahverdyan Consultant Consultant ECSH2
Daniel Kutner Junior Professional Associate Junior Professional Associate ECSH2
Non Bank Staff
Name Title Office Phone City
Petros Keshishian Consultant 5103035770 Los Angeles
1
1. STRATEGIC CONTEXT
A. Country Context
1. Armenia’s remarkable growth in the 2000s was followed by a severe downturn during the 2009 global financial crisis and a moderate recovery since then. Between 2001 and 2008, growth rates increased to double-digit levels, with economic growth averaging 12 percent per year, resulting in poverty gains and enhanced job creation. In 2009, Armenia experienced a sharp downturn, due to the economic crisis, with output contracting more than 14 percent. The economy recovered after the contraction. GDP grew by 2.2 percent in 2010, picking up to 4.7 percent in 2011 and 7.2 percent in 2012. Preliminary data show that growth is now expected to level at around 4.2 percent in 2013, with agriculture, trade, and services showing lower growth than in the previous year and construction declining. Growth in industrial production and exports remains relatively strong.
2. Poverty and inequality increased in the aftermath of the 2009 crisis. The incidence of poverty rose from 27.6 percent of the population in 2008 to 34.1 percent in 2009, 35.8 percent in 2010, and leveled off at 35 percent in 2011.1 The situation worsened in the rural areas where poverty increased from 27.5 percent in 2008 to 36 percent two years later, before decreasing to 34.5 percent in 2011. The share of those living in extreme poverty grew from 1.6 percent in 2008 to 3.7 percent in 2011. Poverty also became deeper and more severe, with a poverty gap of 7.9 percent in 2011 (versus 5.1 percent in 2008) and poverty severity of 2.4 percent (versus 1.4 percent in 2008). Inequality as measured by the Gini coefficient increased on the basis of both consumption (from 0.24 to 0.27) and income aggregates (from 0.34 to 0.37).
3. In terms of shared prosperity, the bottom 40 percent in the income distribution of the population enjoyed strong consumption growth, but was affected by the global crisis. The Bank’s indicator of shared prosperity, i.e., the growth rate of consumption per capita of the bottom 40 percent, demonstrates that economic growth in Armenia benefitted the poor and vulnerable more than the overall population. During 2006-2008, mean real consumption per capita of the bottom 40 percent outpaced that of the population as a whole, 6.5 percent to 5.2 percent. The 2008 crisis eroded the growth in average consumption that had been enjoyed both by the bottom 40 percent and by the overall population. As a result, between 2006 and 2011, the consumption per capita of the bottom 40 percent grew at 0.4 percent per year while that of the population as a whole stagnated.
B. Sectoral and Institutional Context
4. Over time, considerable progress has been made in improving access to preschool, general education and higher education in Armenia.2 The number of 5-6 year-old children
1 Based on analysis in the 2012 Social Snapshot and Poverty in Armenia produced by the NSS: In 2011, the general and the extreme poverty lines per adult equivalent per month were estimated at AMD 36,158 (US$ 97.1) and AMD 21,306 (US$ 57.2), respectively. The general poverty line represents the amount of expenditure required to purchase the typical food basket that provides 2,232 calories per day per capita and a minimal number of non-food items. 2 The education system in Armenia includes preschool education for children of ages 3-6; general education comprising primary schools (grades 1-4), middle schools (grades 5-9), and high schools (grades 10-12); specialized and technical secondary schools; and higher education. The system consists primarily of public institutions, except for universities. In the 2009/2010 academic year, 96 percent of the 1,421 general secondary education schools were state schools.
2
enrolled in the preschool program has increased to 65 percent for the school year 2012-2013. It is estimated that the enrollment rate would increase to about 75 percent by 2015. However, it is still far from reaching the government's objective of 90 percent pre-school enrollment by 2017. In general education, Armenia has achieved nearly universal enrollment rates at the primary (grades 1-4) and lower secondary (5-9 grades) levels. In 2009, the net enrollment rates for primary education were 99 percent and for lower secondary education, 96.2 percent. There has been a significant growth in gross enrollment for higher education, from 19.6 percent in 2001 to 28.6 percent in 2008. The increase compares well with the countries at a similar economic level in the ECA region. Enrollment is slightly skewed toward males in primary and lower secondary education, while in upper secondary and tertiary education the trend is reversed with more females than males enrolled in those levels.
5. A wide range of reforms aimed at improving quality and efficiency has also taken place in the education sector. The government developed the National Curriculum Framework, standards and syllabi; it extended the general education system from 10 to 12 years, and it introduced a network of high schools that offer specialized streams. Furthermore, there has been a concerted effort to provide all 1,431 schools in Armenia with computers and Internet access for students and teachers. The government is developing more e-content to maximize the benefits of Internet access. The government has introduced a teacher certification system under which teachers need to take mandatory training and be tested every five years. It is developing comprehensive professional development policies, including a ranking system and rank-based salary differentiations. In addition to in-service teacher training, the government has begun investing in pre-service teacher training. In the mid-2000s, the government introduced a per-student financing mechanism. It resulted in substantial efficiency gains, including increased student-teacher ratios and stronger school autonomy. At the same time, it established the Assessment and Testing Center (ATC), which enhanced the capacity to measure student performance and introduced a transparent, unified entrance examination for universities and colleges. In higher education, the comprehensive set of reforms began in 2005, when Armenia joined the Bologna Process. A notable development was the establishment of an independent quality monitoring agency, the National Center for Professional Education Quality Assurance (ANQA) in 2009. In addition, the government adopted a higher-education financing strategy in 2011 that addressed the weaknesses in the current system. Based on the strategy, the government plans to make changes to the existing financing system, including introduction of a Competitive Innovation Fund to support demand-driven projects that enhance market relevance of the academic programs, needs-based scholarships, and scholarships for priority fields.
6. Notwithstanding these achievements and reforms, important challenges remain for the Armenia education sector. First, quality of education remains a key challenge, given that the results of international student assessments do not show that the reforms have yet led to improved student performance. The latest results of the Trends in International Mathematics and Science Study (TIMSS) demonstrated that Armenian students’ performance at 4th and 8th grades did not improve compared to 2003 and 2007. When comparing 2003 and 2011 data, TIMSS results show no significant change in Grade 4 mean scores in mathematics, but significant declines in scores for Grade 8 mathematics (-12 points), in Grade 4 science (-20 points) and in Grade 8 science (-24 points). The differences between 2007 and 2011 are more striking, with an average point decrease across the four subject areas assessed at 50 points.
3
Second, even though the enrollment rate for the one-year preschool program has improved considerably in recent years, it still remains low by international standards. Moreover, at the current pace of expansion, it is not likely to meet the government’s own target at 90 percent by 2017. Third, high school infrastructure requires immediate attention. Most of Armenia’s 107 high school buildings are over-designed (spacious hallways, excessively high ceilings and windows, underground air-raid shelters, etc.), use energy inefficiently and are expensive to operate and maintain, have asbestos in their insulation of roofs and piping, many have structurally damaged infrastructures, lack proper seismic stability and present poor sanitary conditions. The need for addressing the infrastructure conditions is obvious and even crucial for the quality of the education and safety of the students. Fourth, there are disconnects between education and job opportunities in Armenia as education does not necessarily help citizens get jobs.3 There are two likely primary causes for the high unemployment rates and the inefficient use of their qualifications at all education levels. One is the overall weak economic growth and the sluggish labor market demand, which is a matter beyond the education reforms, at least in the short run. The other is the gap between the formal qualifications of graduates and the skills employers actually require. Employers in Armenia are increasingly pointing out that those graduating from education lack the skills needed to succeed in today’s market. Communications, critical and creative thinking skills, teamwork abilities, command of foreign languages and ICT skills are needed more than ever to improve productivity and competitiveness, both in the lagging manufacturing sector, and in the growing service sector. The inability of educational institutions to supply graduates with a balanced set of skills may have major consequences on economic growth.
7. The government is fully aware of these challenges and, despite working in a financially constrained context, remains committed to education. The State Program of Education Development in the Republic of Armenia 2011-2015 shows this commitment. In particular, it emphasizes the need of (i) ensuring that all children are ready to learn; (ii) building an education system that can provide good quality education for all; (iii) establishing a diverse tertiary education and research system responsive to the labor market needs; and (iv) improving productivity by endowing tomorrow’s labor force with the appropriate set of skills. Similarly, the country is expected to increase its spending in education from the current 2.4 percent of GDP to 2.6 percent in 2014 and to three percent in the medium term.
C. Higher-Level Objectives to which the Project Contributes
8. The proposed project responds to the government's reform priority areas in the education sector: extending preschool coverage, improving the quality of general education and promoting greater links between higher education institutions and the labor market. The proposed project is aligned with the Bank's 2013 Country Partnership Strategy (CPS) for the Republic of Armenia FY2014-FY2017 (P144255) and supports two of the four key result areas identified in the CPS. These are:
3 According to National Statistical Service of the Republic of Armenia, 2011, Labor Market in the Republic of Armenia 2006-2010, the unemployment rates in 2010 were 19.5 percent among those with tertiary education, 20.3 percent among those with secondary specialized educations and incomplete tertiary, 20.3 percent among those with a vocational education, and 19.0 percent among those with a general secondary education. The rate was especially high among youth (15-24 year-old) at 38.9 percent and even higher among the youth in urban areas at 48.6 percent.
4
Enhancing human capital through better access to quality services, including healthcare, education, culture, and basic infrastructure: investing in the human capital of Armenian’s future workers is one of the most important ways in which the World Bank can help spur the country’s economic competitiveness. By improving access to quality pre-primary education, enhancing conditions for better learning outcomes in high schools and increasing market relevance of tertiary education programs through the Competitive Innovation Fund, the proposed project aims to directly help Armenian authorities put the country on a path to sustainable economic growth.
Employment expansion through high-productivity, decently paid jobs: the proposed project focuses on improving the skills set that graduates bring to the labor market. By improving school readiness of entrants to primary education, enhancing conditions for better learning outcomes in high schools, and increasing market relevance of tertiary education programs through the Competitive Innovation Fund, the proposed project would contribute to stepping up both productivity increases and employment growth.
9. Additionally, the proposed project would complement the ongoing second Education Quality and Relevance Project (P107772), as well as the ongoing Bank-Executed Russia Education Aid for Development (READ) Armenia program (TF012402), in developing a holistic approach to improving the quality of education in Armenia.
2. PROJECT DEVELOPMENT OBJECTIVE(S)/GLOBAL ENVIRONMENT OBJECTIVE(S)
A. Project Development Objectives
10. The Project Development Objectives (PDO) are to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper-secondary schools, and support improved quality and relevance in higher education institutions in Armenia.
B. Project Beneficiaries
11. The project contributes to the Bank’s twin goals of Reducing Poverty and Promoting Shared Prosperity through targeted investments in preschool, general school, upper secondary and higher education. The project’s direct beneficiaries include a total of about 35,000 children in preschools and students in upper-secondary and tertiary levels. The poverty reduction goal would be addressed through targeting the poorest and hard to reach communities located in mountainous and borderline areas as part of the preschool education sub-component. More specifically, the project’s direct beneficiaries would be 12,0004 children aged 5 and 6 living in rural impoverished areas that lack preschool services. Approximately 10,200 high school students would also see their schools refurbished every year. These institutions would provide more appropriate teaching and learning environments, including safer infrastructure, adequate
4 Direct beneficiaries during the Project’s implementation would be approximately 2,400 children per year for each of the project’s five years. However, if the sustainability of coverage expansion is assured, many more children would benefit from the preschool facilities set up under the Project after its completion.
5
ICT equipment and digital teaching and learning materials. The targeted high schools would cover all provinces, which are known as Marzes in Armenia, ensuring that the project would have a country-wide impact. Moreover, improvements in the quality of education of high schools have important pro-poor implications. Approximately 34 percent of students in grades 10-12 (this number is likely to be significantly higher in the high schools targeted), regardless of the type of school, receive private tutoring to prepare for University Entrance Examinations. Better education would help level the field and make students from vulnerable families more capable and likely to progress to higher education. The quality of general education would be increased through the revisions and improvements in curriculum and school-based student assessment activities. This would indirectly benefit approximately 370,000 students per year. The support for Tertiary Education through the envisioned mainstreaming of the Competitive Innovation Fund (CIF) would impact approximately 3,000 students per year, whose academic programs would be modernized and made more responsive to the needs of employers.
C. PDO Level Results Indicators
12. The expected outcomes of the proposed project are the following:
Improved school readiness among those entering first grade in participating communities.
Increased number of high schools targeted by the project which meets Armenia’s construction & safety standards.
Increased number of high schools equipped with education resources provided by the project.
Share of grants established under the Competitive Innovation Fund achieving at least 80 percent of their proposed objectives.
3. PROJECT DESCRIPTION
A. Project Components
13. The proposed project aims at supporting the Government of Armenia in two main endeavors: enhancing the quality of pre-primary and general education, and fostering the relevance of tertiary education through mainstreaming the Competitive Innovation Fund (CIF), or expanding it from a pilot program to one with full implementation.
14. The proposed project builds on the ongoing reform efforts and achievements made to date in the education sector of Armenia. The World Bank has supported the design and implementation of many of these reforms through a series of education projects. The bulk of the proposed interventions under this project, such as support to preschool and upper secondary education, curriculum revisions, integration of ICT, mainstreaming the Competitive Innovation Fund, build upon the successful implementation experience of the APL 2 (ongoing) and APL 1 education projects supported by the Bank. The new interventions are targeted at expanding the previous efforts and ensuring sustainability of the core reform areas, which require a long-term support to produce the expected outcomes. The detailed links of the proposed project components with the previous and ongoing Bank’s support are outlined in Annex 2.
6
Component 1: Enhancing the Quality of General Education (US$27.5 million equivalent).
15. Subcomponent 1.1. Promoting School Readiness and equal opportunities at the start of General Education. The objective of this subcomponent is twofold: first, it would support the government’s efforts to increase preschool access and enrollment, focusing on vulnerable populations, as a means of improving the school readiness of five and six-year-old children entering primary education. By the end of 2017, it is expected that 90 percent of five and six-year-olds would be enrolled in preschool institutions. Second, this component would finance activities aimed at improving the quality of preschool education offered by community-based initiatives.
16. The design of this sub-component has benefited from the implementation experience and achievements made as part of the ongoing engagement with the government under the on-going Education project (P107772), which supports school readiness via preschool education micro-projects. This subcomponent would maintain focus on improving the quality of service delivery through providing adequate pedagogical support to teachers in partnership with the Ministry of Education. In particular, the subcomponent would support: (i) dissemination activities for community leaders, parents and communities at large about preschool opportunities and benefits to startup these services in the community and associated grant procedures for micro-projects; (ii) evaluation and selection of micro-projects; (iii) provisions of grants for the implementation of the micro-projects; (iv) provision of technical assistance for the implementation of the micro-projects; and (v) monitoring and evaluation of the impact of the micro-projects. The grants extended to communities would support refurbishment of classrooms, purchase of equipment and furniture, provision of pedagogical materials and teacher training.
17. Subcomponent 1.2. Enrichment of Upper Secondary Schools. The objective of this subcomponent is to strengthen the recently established network of high schools by supporting the rehabilitation of selected facilities in poor physical conditions and improvements in all high schools in teaching and learning conditions, including the provision of contextualized digital learning materials, modern equipment, and school furniture.
18. This subcomponent would support the rehabilitation and refurbishment of a limited number of high schools. It is expected that approximately 17 out of a total of 107 high schools in Armenia would benefit from this subcomponent. These schools would be selected in accordance to the selection criteria agreed with the government (see Annex 2). Given that Armenia is prone to earthquakes, the feasibility study for each of the schools would include experts’ assessments of their seismic conditions. The rehabilitation works supported by the project would ensure compliance to seismic resistance standards and, where needed, strengthening works would be carried out. The project would also ensure project selected schools provide accessibility for disabled students, faculty and family members and ensure compliance with all relevant legislation on accessibility. To ensure that the newly upgraded school infrastructure is maintained, the component would also support the development and adoption of a school maintenance manual and training in the use of that manual to upgrade the capacity of school managers and administrators in this area.
7
19. This subcomponent would also finance activities aimed at improving the teaching and learning conditions of all high schools. It would finance the purchase of modern information and communication technologies (ICT) equipment, educational furniture and the development of contextualized digital learning materials to be used in classrooms in all high schools in Armenia. This subcomponent would also support training teachers on using modern teaching methodologies and digital learning materials in the classrooms. To monitor the changes and improvements, the subcomponent would finance classroom observations and other studies. It would also fund meetings with key stakeholders to bolster the social accountability of the high school reforms and of the investments made under the project. This endeavor would ensure the education Ministry would receive feedback from parents and members of the community.
20. Subcomponent 1.3. Improving data-collection and monitoring of the education system performance. This component would improve the National Center of Education Technology’s (NACET) capacity to monitor Armenian schools and provide them with adequate ICT coverage. This sub-component would also provide, if needed, the financing for the participation of the country in large-scale international student assessment studies.
21. Starting in 2013, NACET’s ICT services have been covering all educational institutions in Armenia, including Vocational Education and Training (VET) and Higher Education Institutions (HEIs). NACET is tasked with managing data from all educational institutions in Armenia and making educational statistics available both to the public and the authorities in a systematic and organized fashion. By the end of the project, it is expected that NACET would be playing the role of national center for educational statistics and providing decision-makers and key stakeholders with timely and relevant education information on the whole spectrum of Armenia’s education system.
22. The subcomponent would support the following activities (i) the development of an ICT education strategy for the National Center for Educational Technology (NACET); (ii) the rehabilitation and refurbishment of the NACET’s facilities; (iii) the provision of hardware and software platforms needed to run EMIS information systems; (iv) the establishment of technical service mechanisms for the maintenance of hardware and software in Armenian schools; (v) training of NACET staff on management, planning, and monitoring and evaluation; (vi) training teachers and non-teaching staff of schools, universities and vocational education and training institutions on the integration of ICT into teaching and learning processes as well as training of users of EMIS in schools and institutions; and (vii) financing, if needed, the participation fees for subsequent editions of the International Trends in Mathematics and Science Studies (TIMSS).
23. Subcomponent 1.4. Supporting further improvements in the quality of education through curriculum revisions. The objective of this subcomponent is to bring the National Curriculum Framework for grades 1-12 into conformity with the requirements of the General Education and State General Education Curriculum law approved by the government Decree No. 1088, dated July 28, 2011. The proposed curriculum revisions would be outcome-oriented and prioritize identifying the key competencies that students should have acquired by the end of each level of education. The proposed project would finance technical assistance and consultative workshops to align with the law the National Curriculum Framework, the State Standard for General Education, the subject standards and syllabuses. The curriculum revision
8
would be done by Working Groups for the purposes of which the credit proceeds would be used to hire subject experts, school teachers, university professors, and experts from the National Institute of Education. The related activities would include in-service training for teachers, new curriculum materials for teachers and dissemination workshops. The Ministry of Education would be responsible for the implementation of the modified curriculum including training of teachers through the National Institute of Education, and alignment of textbooks through the existing textbook revolving fund.
Component 2: Mainstreaming of the Competitive Innovation Fund (CIF) for Higher Education Institutions (HEI) into full implementation (US$6.25 million equivalent).
24. The objective of this component is to support the roll-out of the competitive grant program for HEIs that was developed and piloted under the Education Quality and Relevance Project 2 (EQRP2) (P107772). The fund was designed to support the best proposals prepared by HEIs with the aim of improving the quality, promoting modernization and particularly the labor-market relevance of the academic programs as well as increasing efficiency, equity and the development of new academic programs of HEIs. The CIF would support on a competitive basis projects in the following three main areas: (i) quality improvement, (ii) relevance improvements; and (iii) efficiency promotion. Innovative project proposals with a focus on underserved groups or regions would be particularly encouraged.
25. Specific activities to be financed by this component include: (i) update of the fund’s operational processes and procedures; (ii) provision of grants to HEIs; (iii) monitoring of the fund’s implementation process, including surveys of grant holders and potential applicants and private sector partners; (iv) tapping private-sector and international potential donors to bolster the fund’s sustainability; (v) special trainings for HEIs managerial staff and other relevant staff members on project management skills, procurement issues, and project monitoring and evaluation issues; and (vi) an assessment of the impact of the component.
Component 3: Project Management, Monitoring and Evaluation (US$3.75 million equivalent).
26. The key objective of this component is to provide continued support for the management and monitoring of project implementation activities and outcomes. This component would provide the financing and support for the effective functioning of the Center for Education Project (CEP) as the project implementing unit within the Ministry of Education and Science. Specific activities to be supported by this component are the following: (i) Project Implementation Unit, the CEP, with core staff hired as consultants; (ii) implementation and management of the procurement processes, disbursements, and financial management, project monitoring, and reporting; (iii) acquisition of minor additions/replacements of office furniture and equipment; (iv) local travel, utilities and publications, translations, small office repair, office supplies, fuel, Internet service, bank commission charges and vehicle maintenance and repair; (v) implementation and management of special studies planned; (vi) facilitate coordination, communication flows and dissemination of information with participating institutions; (vii) developing and overseeing a system to collect and address beneficiary feedback and grievances related to the project; (viii) facilitate working group arrangements and the organization of seminars and workshops; and (ix) training to CEP staff in procurement,
9
disbursements, information technology, safeguards, project management and other key areas identified and proposed in the course of project implementation.
B. Project Financing
27. The proposed project is to be implemented over a period of five years, between 2014 and 2019, and would be financed by an IBRD loan in the amount of US$15.0 million equivalent, on standard IBRD terms, and by an IDA Credit in the amount of US$15.0 million equivalent, on standard IDA terms. The government counterpart contribution is estimated at US$7.5 million, integrated within the annual government budget allocation for the education sector.
Project Cost and Financing
Project Components Total Project cost IBRD or IDA
Financing % Financing
1. Enhancing the Quality of General Education 2. Mainstreaming of the Competitive Innovation Fund for Higher Education Institutions 3. Project Management, Monitoring and Evaluation Total Costs
27,500,000
6,250,000
3,750,000
37,500,000
22,000,000
5,000,000
3,000,000
30,000,000
80
80
80
80
Total Project Costs Front-End Fees Total Financing Required
37,500,000 0
37,500,000
30,000,000 0
30,000,000
C. Lessons Learned and Reflected in the Project Design
28. The project reflects key lessons learned under the previous Bank-supported education operations in Armenia. First, a realistic evaluation of the institutional capacity for managing reforms and projects and an appropriate project scope enhance the chances of success. Some of the proposed areas are familiar to Armenian counterparts from previous projects, such as preschool micro-grants and involvement in the Competitive Innovation Fund. In newer areas, such as infrastructure for high schools, a considerable amount of technical assistance has been provided during preparation, and detailed implementation plans have been prepared so that there is greater awareness of the tasks. Second, project implementation is considerably enhanced when project concept and objectives are based on stakeholder consensus and borrower priorities. The design of the project has largely been shaped by the Armenian government and specifically the education Ministry, based on its priorities, as discussed in the Country Partnership Strategy (CPS). The government has been proactive throughout project preparation and has produced an extensive Project Implementation Plan (PIP).
29. In terms of its design, the project reflects key lessons learned from international experience and from Armenia.
a. Preschool education. International evidence documents the significant impact of
quality preschool education on children’s school readiness (including their cognitive,
10
social, emotional and physical development) and on their capacity to become productive citizens. Lack of access to preschool has been internationally documented as detrimental to children’s capacity to learn in Grade 1 and beyond, to transition successfully from one grade to another, to complete their education and to eventually become fully contributing and productive members of society. Evaluation studies undertaken under the ongoing EQRP2’s support to preschool in Armenia shows the important positive impact of these services on the Early Development Index of children in the project’s supported preschools as compared with students with no preschool experience.
b. Impact of improvements in the teaching and learning environment on student learning. Glewwe et al5 conducted a literature review followed by a meta-analysis study of the relationship between school resources and test scores. Better resources such as textbooks, basic furniture, blackboards, school libraries and better infrastructure were found to have a positive impact on test scores. The authors concluded that “a fully functional school – one with better-quality roofs, walls or floors, with desks, tables and chairs and with a school library – appears to be conducive to student learning.” Branham6 studied over 200 schools in the Houston Independent School District and found that the quality of school infrastructure has a significant effect on student attendance and drop-out rates. Empirical evidence shows that poor infrastructure in schools leads to lower attendance and higher drop-out rates. Finally, Cuyvers et al7 analyzed the impact of school infrastructure on the well-being of students in Flemish secondary schools. The authors found empirical evidence on the importance of school infrastructure for the well-being of students and concluded that the quality of school infrastructure has a strong impact on a student’s perception of his or her well-being. It is important to underscore the fact that the evidence available is drawn mainly from regions whose needs may differ from those of Armenia in substantive ways. However, the overall assessment is that regardless of the region, improving infrastructure would lead to better learning outcomes by strengthening the motivation of teachers and students and reducing absenteeism.
c. Competitive Innovation Funds. International experience suggests that innovation funds are highly effective mechanisms for improving transparency and efficiency in the budget allocation for tertiary education, and for boosting educational quality and relevance within tertiary institutions. They offer four main types of benefits: First, the competitive nature of these funds generally makes them more efficient instruments for the allocation of public funding than more traditional approaches based on budgetary planning. Second, they are effective mechanisms for improving educational quality and relevance. Third, their incentives system fosters changes in professional attitudes and institutional culture. Fourth, innovation funds are very flexible and can quickly respond to changing policy priorities. In Armenia, where public financing of higher education has declined over the last decade, the government’s goals to raise enrollment and improve the quality of education requires
5 Glewwe, P., Hanushek, E., Humpage, S. & Ravina, R. (2011) School Resources and Educational Outcomes in Developing Countries: A Review of the Literature from 1990 to 2010, NBER Working Paper 17554. 6 Branham, D. (2004) “The Wise Man Builds His House Upon the Rock: Effects of Inadequate School Building Infrastructure and Attendance”, Social Science Quarterly, 85 (5), p. 1112-1128. 7 Cuyvers, K., G. De Weerd, S. Dupont, S. Mols, and C. Nuytten (2011) “Well-being at school: does infrastructure matter”, CELE Exchange 2011/10, OECD.
11
that total spending must increase beyond current projections. Competitive funding appears to be the most effective and politically appealing mechanism to do so. Under the EQRP2, a pilot application of a CIF is underway. Although it is too early to assess its impact on outcomes, important lessons learned from an ongoing process assessment of the CIF would be incorporated by the government before proceeding to mainstream the CIF under this proposed project.
4. IMPLEMENTATION
A. Institutional and Implementation Arrangements
30. The Ministry of Education and Science would be responsible for the implementation of the project supported by the services of the Center for Education Project (CEP) acting as the Project Implementation Unit. CEP is a project management unit operating within the Ministry. It was created in 1996 and it has gained significant strength in project management through the implementation of Bank-funded education projects, including the two past education projects. CEP would continue its activities of administering, coordinating, implementing, overseeing project information-sharing and communications, and monitoring and evaluating the proposed project activities, as well facilitating the decision-making process for the proposed project. CEP would be also responsible for the procurement and financial management functions (including flow of funds, planning and budgeting, accounting, internal controls, audit and financial reporting). CEP comprises a core group of technical, administrative and support staff.
31. The Ministry of Education and CEP would work within the existing organizational structure of the Ministry supporting state management institutions (the Ministry’s technical departments, provincial education departments, the State Inspectorate of Education), as well as key beneficiaries such as the National Center for Education Technology (NACET), selected communities and universities. To implement the first component, the General Education Department (in charge of preschool, primary and secondary education) and the office of the Chief of Staff (in charge of school infrastructure) at the Ministry of Education would play leading roles and through CEP would coordinate with the provincial departments of education (Marzes), with community heads and councils, and with pre-schools and high schools. The Ministry and CEP would work closely with NACET to strengthen monitoring of Education outcomes envisioned under the project. To monitor student learning, the Ministry of Education would coordinate its activities with the Assessment and Testing Center as the participating entity involved in implementing TIMSS. To implement the project’s second component, the Higher Education Department and CEP would work closely with the selected grant-recipient institutions. The Ministry would also work with institutional entities related to tertiary education, such as the newly established Armenian National Quality Assurance Agency and higher education institutions. The specific division of project responsibilities among the different actors is provided in Annex 2.
32. Given the scope of the project, there are several relevant stakeholders for the project. They include education Ministry officials, nongovernmental organizations, higher education institutions, teachers, parents, and other institutions. To properly address the needs and concerns that these stakeholders may have, the Ministry has conducted and it would continue to hold stakeholder consultations during the design and implementation phases of the project.
12
B. Results Monitoring and Evaluation
33. The Project Development Objective level and intermediate result indicators would be monitored using the following sources and methodologies: (i) regular data collection process; (ii) baseline and follow-up surveys; and (iii) reports prepared by the CEP.
34. CEP would be responsible for coordinating monitoring activities and for bringing together the representatives of the education Ministry to monitor the project’s objectives and results and for communicating with the World Bank according to the frequency of reports described in Annex 1. Monitoring and Evaluation activities under the project would be integrated into regular monitoring functions of the Ministry. Project indicators have been selected on the basis of the data requirements already planned as part of the Ministry’s regular data collection. 35. The project envisions strong beneficiary/stakeholder participation in project monitoring and supervision activities on an ongoing basis. Project websites and social media applications would provide a forum for students, parents, educators and other beneficiaries to voice their feedback on the project on an ongoing basis and to respond to online surveys and discussion threads. Online forums would be supplemented by regular public meetings in project areas. In addition to tracking beneficiary feedback on project implementation, these forums would also be used to track perceptions of project governance, including transparency, responsiveness and relevance.
36. Under the proposed project, efforts would be directed at gradually building up the Ministry’s capacity in policy planning, monitoring and evaluation. These areas of responsibilities have largely been assumed by the CEP, but the center would work closely with the department of education policy and planning at the Ministry of education, building their capacity and gradually transferring these responsibilities to them. The key responsibilities to be transferred include: a) quality reporting; b) internal and external evaluations, studies and assessments; and c) monitoring and evaluating the project’s outcomes and results.
C. Sustainability
37. The sustainability of the proposed project would be determined by four aspects: first, the government’s ownership of the project; second, an experienced CEP with staff in project management mainstreamed into the education Ministry; third, institutional capacity building of the technical level staff and fiduciary management in the education sector under a strengthened and evolving organizational structure; and fourth, fiscal sustainability and cost-effectiveness of the project activities.
38. With regard to the first three aspects and as previously indicated (i) the government of Armenia has shown strong ownership throughout project preparation, producing an extensive project concept and a detailed Project Implementation Plan; (ii) the project implementing agency, CEP, has over 10 years of successful experience of World Bank projects; and (iii) through Bank support, the education Ministry has been strengthened in terms of its capacity for coordination functions associated with policy, planning, monitoring and evaluation.
13
39. For the fourth element, the sustainability of the proposed project activities after the project completion is expected to be high. A flexible expansion of preschool education is a cost-effective way for the government to reach its enrollment targets. Local communities would commit to financing the preschools’ first year’s operating expenses, while the government is committed to paying teachers’ wages from the start of the second year. The recurrent cost associated with the periodic maintenance of the rehabilitated high schools would be funded by the education Ministry’s budget. The Competitive Innovation Fund requires the replenishment of resources to ensure its sustainability. It is expected that the central government funds for the CIF would be complemented by the private sector and other donors. The project would support technical work for the design of an incentive scheme for private contributions to the CIF. It would fund studies to assess the impact of the CIF on key outcomes at the tertiary education level and disseminate its results to raise awareness on the benefits of these investments.
5. KEY RISKS AND MITIGATION MEASURES
A. Risk Ratings Summary Table
Risk Category Rating
Stakeholder Risk Moderate
Implementing Agency Risk
- Capacity Moderate
- Governance Low
Project Risk
- Design Moderate
- Social and Environmental Moderate
- Program and Donor Low
- Delivery Monitoring and Sustainability Low
- Other (Optional)
- Other (Optional)
Overall Implementation Risk Moderate
B. Overall Risk Rating Explanation
40. The overall implementation risk is moderate. The key risks are the following (a more detailed description can be found in Annex 4):
a. Stakeholder Risk. Given the need to engage and coordinate with many stakeholders,
there is a risk that the needs of some of them would not be properly addressed by the proposed project, which may affect its implementation.
14
b. Capacity Risk. There are three risks which may affect project implementation capacity: first, the Center for Education Project may not be able to retain qualified staff for project implementation due to low salaries. Second, the education Ministry and the CEP have limited experience in the application of Bank's safeguards policies and the procurement of works under the Bank's guidelines. This may introduce capacity gaps that need to be closed by project's effectiveness. Third, local authorities’ capacity to implement the planned expansion under the preschool subcomponent may be low, which may require capacity building training for local governments and other stakeholders during the project implementation.
c. Design Risk. At the community level there are some risks for the “Promoting school readiness and equal opportunities at the start of general education” subcomponent. First, there may be the risk of limited demand for community-based preschool education and low number of service providers due to the newness of the initiative. Second, there may be a lack of understanding and knowledge of the benefits of preschool for potential service providers and parents. Third, some communities may not have adequate expertise to prepare the required grant proposals. For the “Strengthening key institutions for monitoring educational outcomes and monitoring student learning in Armenia” subcomponent, there is the center's staff may not be adequately trained to fulfill NACET’s newer and broader range of expected responsibilities.
d. Social and Environmental Risk. The proposed project would finance the upgrading of approximately 17 school buildings. The environmental risks associated with the proposed works are typical for rehabilitation of medium-size buildings located within settlements, and are modest in scale. Managing the expected negative impacts and keeping them to the acceptable minimum would require adhering to good construction practices and applying specific mitigation measures which would be provided in the Environmental Management Plans (EMPs) that the government is producing for each of the project beneficiary schools.
6. APPRAISAL SUMMARY
A. Economic and Financial Analysis
41. The proposed project addresses different levels and aspects of the education system and the economic and financial analysis developed in Annex 6 examines the costs and benefits of each component (and subcomponent) separately. The analysis assumes that the benefits and costs can be added and subtracted. The benefit of the project is defined as the difference between the expected gains under the project and what would be expected to happen in its absence (counterfactual). Moreover, other benefits follow from the changes in the composition of the labor force (more qualified and productive workers) in the long term. The costs considered not only the project's direct cost, but also the long-term costs and the fiscal effort the government would have to make in order to sustain these investments. The fiscal burden includes higher maintenance cost of the renovated high schools, the expansion of pre-primary education and the additional costs of the expected increase in enrollment in all education levels. Part of the burden can be shared with the families that choose private over public education. The analysis conducted indicated that the project presents a benefit-to-cost ratio of 3.2. A sensitivity analysis of the results was performed on the key parameters of the economic
15
evaluation model (dropout rates, higher progression to tertiary education, and better integration of graduates in the labor market) confirming the view that the project is a good investment for Armenia.
B. Technical
42. The development objective would be achieved through its technically sound design, focusing on policy advice, institutional capacity building and support for broad-based reforms of education. The proposed project would help the government consolidate a system-wide reform focusing on promoting school readiness and equal opportunities, improving the teaching and learning environments in high schools and enhancing the responsiveness of tertiary education institutions to labor market requirements. Technical assistance would be provided to strengthen the ability of the education Ministry to implement the reforms. Managerial and analytical capacity building would take place at all levels to ensure that the various parts of the organization are effectively coordinated. Quality assurance and oversight would also be strengthened.
C. Financial Management
43. The financial management assessment of the project was performed by the Bank’s Financial Management (FM) specialist in October 2013. The information available from the assessment of the Armenia project preparation grant (ECAPDEV) for this project conducted in November 2012 as well as from the FM implementation support and supervision mission of EQRP2 project conducted in December 2012 was also used.
44. As a result of the assessment, it was established that the Center for Education Project has adequate FM arrangements in place, and these are acceptable to the Bank for the implementation of this EIP Project. Currently, CEP successfully implements the EQRP2 Project, and this EIP Project is largely built on the strong implementation experience and structure of EQRP2. In Particular: (i) significant experience of the CEP’s FM staff in implementing Bank-financed projects over the past several years; (ii) adequate accounting system and software utilized by the CEP, (iii) FM arrangements similar to those of the active project implemented by the CEP and found to be adequate, and (iv) unmodified audit reports issued on EQRP2 project’s financial statements.
45. The Treasury system is being used to maintain the designated accounts of Bank-financed projects including this project, which plans to open the designated account in the Treasury. In addition, the country budget system would be used for this project. For all the other FM elements, the CEP’s respective systems are going to be used for this particular project.
D. Procurement
46. Procurement would be carried out in accordance with the World Bank's “Guidelines: Procurement of Goods, Works, and Non-Consulting Services under IBRD Loans and IDA Credits & Grants by World Bank Borrowers,” dated January 2011; and “Guidelines: Selection and Employment of Consultants under IBRD Loans and IDA Credits & Grants by World Bank Borrowers,” dated January 2011, and the provisions stipulated in the Legal Agreement. The
16
World Bank Guidelines on Preventing and Combating Fraud and Corruption in Projects Financed by IBRD Loans and IDA Credit and Grants dated October 15, 2006 and revised on January 2011, would also apply.
47. The Ministry of Education and Science (MoES) of Armenia would be responsible for the implementation of the project supported by the services of a Center for Education projects (CEP) acting as the Project Implementation Unit. An assessment of the capacity of CEP to implement procurement actions for the new project was carried out by the procurement staff assigned to the project in June 2013. It was done on the basis of the CEP’s performance in implementing the ongoing Second Education Quality and Relevance Project (EQRP2). The CEP’s director has significant experience in implementing World Bank-financed projects. It has also two procurement staffers, among others, who have good knowledge and experience of the Bank's Procurement and Consultant Guidelines. This capacity can therefore be considered satisfactory. However, it should be pointed out that CEP has limited experience in procuring and implementing works contracts, which is a major component of the project.8 The procurement risk for the project is rated as “moderate” given: (a) that CEP has satisfactory experience in implementing Bank-financed projects and has good knowledge of the Bank's Procurement and Consultant Guidelines; and (b) the mitigation measures identified in Annex 3. This is already reflected in the Procurement Risk Assessment and Management System (PRAMS) in the portal, which was completed on October 31, 2013 by the Procurement Accredited Staff (PAS) assigned to the project.
E. Social
48. The social impacts of this project are expected to be positive, with improved access to, quality of, and relevance of, education being primary development objectives. The project activities would include particular attention to underserved communities and population groups. Ensuring gender sensitivity and inclusion of people with disabilities and other vulnerable groups would also be prioritized across all project activities, including mechanisms for stakeholder engagement. As such, the project would be expected to improve equity within the education sector and coverage of the poor and vulnerable by education services. By supporting strengthened capacity for teachers and education officials, including the NACET, the project would be expected to have ongoing positive impacts for the education system of Armenia beyond its implementation period.
49. All civil works (rehabilitations) would take place within the existing footprint of educational facilities. As such, OP 4.12 is not triggered. Nonetheless, the project PIU would be oriented on World Bank Social Safeguards Policies and would monitor compliance with these policies by all project partners throughout the life of the project.
F. Environment
50. The project would support upgrading of the selected school buildings. Rehabilitation works are expected to have some temporary negative impacts typical for reconstruction and
8 During EQRP1 the CEP carried out the procurement of six work contracts for rehabilitation of (i) four NIE regional centers, (ii) the Assessment and Testing Center and (iii) the building currently hosting CEP and NACET.
17
rehabilitation of small to medium-size buildings. OP/BP 4.01 Environmental Assessment is triggered. According to the principles of this safeguard policy, and based on the outcome of environmental screening, the project is classified as Category B. Environmental risks of its implementation would be of a modest scale and limited to the construction sites and to the period of construction. Managing these impacts and keeping them to the acceptable minimum would be achieved effectively by adhering to good-construction practices and applying specific mitigation measures to be provided in the Environmental Management Plan (EMP) for each civil work.
51. An Environmental Management Framework (EMF) was developed as part of the project preparation process, while the site-specific EMPs would be worked out later on, as the beneficiary institutions get identified and detailed designs for their rehabilitation become available.
52. School buildings suggested for rehabilitation with support of the project are predominantly old and there is some likelihood of some of these buildings carrying a historical value. OP/BP 4.11 is triggered to ensure that the process of school selection would involve checking if any of the buildings are formally entered into the list of the nation's historical and/or cultural heritage. In such cases the building renovation designs would be cleared with the national authorities responsible for the preservation of cultural heritage to ensure that historic/cultural value of such buildings are not compromised as a result of improvement works. These provisions are included into the EMF.
53. The education Ministry is the implementing agency for the project. Its unit called the Center for Education Project would act as this project’s day-to-day implementation unit, responsible for procurement, disbursement, safeguards application and M&E. Over the project’s preparation stage, one of the CEP’s staff members has been appointed to oversee the environmental work. It was agreed with the CEP that an environmental specialist would be hired once the project enters into operation and resources become available. Hiring and maintaining an environmental specialist in CEP throughout the period of civil works performance under the project would be a legally binding commitment of the borrower.
The Project Development Objectives (PDO) are to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper secondary schools, and support improved quality and relevance in higher education institutions in Armenia.
These results are at Project Level
.
Project Development Objective Indicators
Cumulative Target Values Data Source/ Responsibility for
Indicator Name Core Unit of Measure
Baseline YR1 YR2 YR3 YR4 End Target Frequency Methodology Data Collection
1. Improved school readiness among those entering first grade in participating communities (Indicator to be measured at the aggregate level as well as separate by gender).
Text
Baseline collected in September 2014 for treatment & control group for the communities selected in the first round. For the communities to be selected in the second round, the baseline will be collected in September 2015.
EDI scores are higher in treatment group as compared to control group for the communities selected in the first round and this difference is statistically significant.
EDI scores are higher in treatment group as compared to control group for the communities selected in the first round and this difference is statistically significant.
EDI scores are higher in treatment group as compared to control group for the communities selected in the first and second round and this difference is statistically significant.
EDI scores are higher in treatment group as compared to control group and this difference is statistically significant
Annual EDI measurements (starting from Year 2)
Survey on the Early Development Index dynamics of the children enrolled in the preschool micro projects institutions and in a control group comprising children without preschool education.
CEP
19
2. Increased number of high schools targeted by the project which meet Armenia’s construction & safety standards.
4. Share of grants established under the CIF achieving at least 80 per cent of their proposed objectives.
Percentage
0.00 0.00 0.00 100.00 100.00 100.00 Years 3 and 5 CEP Progress Report
CEP
.
Intermediate Results Indicators
Cumulative Target Values Data Source/ Responsibility for
Indicator Name Core Unit of Measure
Baseline YR1 YR2 YR3 YR4 End Target Frequency Methodology Data Collection
Five to six year old children enrolled in preschools supported by the project. (Indicator to be measured at the aggregate level as well as separate by gender)
Number 0.00 0.00 600.00 1200.00 1800.00 2400.00 Annual
Questionnaire (MoES and provincial/Marz Education Department reports)
MoES, MEDs, CEP
Number of community-based pre-schools supported by the project.
Number 0.00 0.00 30.00 60.00 90.00 120.00 Annual
CEP Progress Report
CEP
Improved student attendance in high
Text Baseline collected for
Student attendance
Student attendance is
Student attendance
Student attendance is
Annual Attendance
Independent monitoring
CEP
20
schools targeted by the project (Indicator to be measured at the aggregate level as well as separate by gender).
treatment & control high schools (collected in school year 2013-2014)
is higher in treatment high schools with refurbished activities completed as compared to control group high schools.
higher in treatment high schools with refurbished activities completed as compared to control group high schools.
is higher in treatment high schools with refurbished activities completed as compared to control group high schools.
higher in treatment high schools as compared to control group high schools and this difference is statistically significant
Survey contracted by CEP to track student attendance in project supported high schools and in a control group.
Number of classrooms rehabilitated in high schools resulting from project interventions
Education Management Information System for general and higher education integrated and in operation.
Text No No Integrated system designed
Integrated system piloted
Integrated system fully operational
Integrated system fully operational and expanded nationwide
Annual NACET Reports NACET, CEP
System for the maintenance of hardware and software of Armenian schools designed and in operation
Text No No System designed
System piloted
System fully operational and expanded nationwide
System fully operational and expanded nationwide
Annual NACET Reports NACET, CEP
Number of administrative staff trained in the use of the integrated Education Management Information Systems (Indicator to be measured at the aggregate level
Number of higher education institutions benefiting from the CIF’s component
Number 0.00 4.00 4.00 10.00 10.00 10.00 Annual
CEP Progress Reports (calls for proposals happen in the first and third years of the project)
CEP
Number of partnerships between public and private institutions established under the CIF
Number 0.00 2.00 2.00 4.00 4.00 4.00 Annual
CEP Progress Reports (calls for proposals happen in the first and third years of the project)
CEP
.
22
Annex 1: Results Framework and Monitoring
Armenia: Education Improvement Project (P130182)
Results Framework
Project Development Objective Indicators
Indicator Name Description
1. Improved school readiness among those entering first grade in participating communities (Indicator to be measured at the aggregate level as well as separate by gender).
This indicator would be measured by the Early Development Index, which provides a community-level measure of young children’s development in five domains: language and cognitive skills, emotional maturity, physical health and well-being, communication skills and general knowledge, and social competence.
2. Number of high schools targeted by the project which meets Armenia’s construction & safety standards.
Armenia is country with high seismic risk. However, many of its schools were built before 1994 - when the new safety standards were introduced - and risk collapse in case of seismic activity.
3. Number of high schools equipped with education resources provided by the project.
Education resources include computers and ICT material.
4. Share of grants established under the CIF achieving at least 80 per cent of their proposed objectives.
According to the Competitive Innovation Funds' Operational Manual, each proposal must include a detailed Results Framework with measurable and quantifiable outcome and output indicators. Projects might tackle quality of education, relevance or efficiency. Due to the diversity in the range of possible projects, this indicator would measure what share of these outcome and output indicators of each grant have been met.
.
Intermediate Results Indicators
Indicator Name Description
Five to six year old children enrolled in preschools supported by the project. (Indicator to be measured at the aggregate level as well as separate by gender)
Enrollment in preschools in the participants’ communities.
Number of community-based pre-schools supported by the project. Number of micro-projects awarded within the framework of the project.
Improved student attendance in high schools targeted by the project (Indicator to be measured at the aggregate level as well as separate by gender).
High schools' infrastructure tends to be very poor. Among other issues, many of these schools do not have proper heating systems, which lead to high absenteeism during the winter. However, attendance is not adequately monitored at the moment. Hence, the project would include external monitors in order to capture real attendance in the baseline.
Number of classrooms rehabilitated in high schools resulting from project interventions
This indicator is inspired in a core indicator that measures classrooms built in primary schools. Since this project does not focus on primary education, the indicator was adjusted to upper secondary education.
Education Management Information System for general and higher education integrated and in operation.
Currently, the management information system for general education and vocational and tertiary education work separately. Users need to consult two different systems to get valuable information. An integrated system means that users can get information from a single source. Operational means that users can get relevant reports from the system according to their access profiles.
System for the maintenance of hardware and software of Armenian All schools in Armenia are equipped with hardware and software that allow them to feed a central system
23
schools designed and in operation with timely information on enrollment and attendance of their students. However, there are challenges to be faced by schools in remote areas, where it is difficult to provide maintenance services. The project is helping NACET to increase its capacity of catering to all schools in Armenia. A fully operational system needs to be capable of providing online support plus timely visits on-site to solve hardware or software problems.
Number of administrative staff trained in the use of the integrated Education Management Information Systems (Indicator to be measured at the aggregate level as well as separate by gender).
Staff from the Ministry of Education and Science, schools and Marzes would be trained to extract relevant information on enrollment and attendance.
System for learning assessment at the primary level Core indicator which measures whether or not the basic elements of a system for learning assessment exist at the primary level, and the degree to which that assessment is able to provide useful information for education policy and practice. This is assessed by checking whether the system has or not the following two characteristics: - The official purpose of the assessment is to measure overall student progress toward agreed system learning goals; - The assessment is given to a representative sample or census of the target grades or age levels.
Utility of the learning assessment system Utility of the learning assessment system is assessed by checking whether it possesses the following three characteristics: - Data are analyzed and results are reported to education policymakers and/or the public; - Results are reported for at least one of the following student subgroups: gender, urban/rural, geographic region; - The assessment exercise is repeated at least once every 5 years for the same subject area(s) and grade(s).
Number of subjects whose standards and syllabuses have been revised and endorsed by the government
Curricula revision has taken place in 2009. However, due to the extension of the length of general education from 10 to 12 years, refinements are needed to assure the syllabi are coherent and adequate.
Number of higher education institutions benefiting from the CIF’s component
This measures the number of higher education institutions selected under the project for CIF funding as individual universities or as part of consortiums.
Number of partnerships between public and private institutions established under the CIF
Partnerships between public and private institutions are expected to make public institutions more responsive to the labor demand (and market incentives). This indicator also takes into account the participation of research centers that are not higher education institutes.
24
Annex 2: Detailed Project Description
ARMENIA: Education Improvement Project 1. The proposed project builds on the ongoing reform efforts and achievements made to date in the education sector of Armenia. Since 2004 Armenia has developed the National Curriculum Framework, standards, and syllabuses, and extended the general education system from 10 to 12 years, with the introduction of high schools that offer specialized streams. It established the Assessment and Testing Center (ATC), which enhanced the capacity to assess student performance and introduced a transparent unified entrance examination for universities and colleges.9 All schools in Armenia got access to the internet by the beginning of 2013, allowing students and teachers even in rural areas to be readily connected to the most up-to-date information. The government is developing more e-content to maximize the benefit of internet access. Teacher management is another core area for reforms. The government has introduced the mandatory teacher attestation system under which teachers need to take mandatory training and be certified every five years. 2. The bulk of the proposed interventions under this project, such as support to preschool and upper secondary education, curriculum revisions, integration of ICT and mainstreaming the Competitive Innovation Fund build on the successful implementation experience of the APL 2 (ongoing) and APL 1 education projects supported by the Bank. The new interventions are targeted at expanding the previous efforts and ensuring sustainability of the core reform areas, which require a long-term support to produce the expected outcomes. The detailed links of the proposed project components with the previous and ongoing Bank’s support are outlined in table 1 below. 3. The proposed project aims to support the government of Armenia in two main endeavors: enhancing the quality of general education and fostering the relevance of tertiary education through mainstreaming the Competitive Innovation Fund. Component 1: Enhancing the Quality of General Education (US$27.5 million equivalent).
4. Subcomponent 1.1. Promoting School Readiness and equal opportunities at the start of General Education. The objective of this subcomponent is twofold: first, it would support the government of Armenia’s efforts to increase preschool enrollment, focusing on vulnerable populations (including minority ethnic groups and children of disabilities), as a means of improving school readiness of five and six year old children entering primary education. By the end of 2017, the government’s goal is to have 90 percent of five and six year old children enrolled in preschool institutions. Second, this component would finance activities aimed at improving the quality of preschool education offered by community-based groups.
9 Based on the System Assessment and Benchmarking for Education Results (SABER) for Student Assessment—a diagnostic tool that benchmarks education policies according to evidence-based global standards and best practice—the four dimensions of the Armenia’s student assessment system is assessed on the four-level scoring system (Latent – Emerging – Established - Cutting Edge) as follows: “Cutting Edge” for Classroom Assessment; “Established” for Exams and International Large-Scale Assessment; and “Emerging” for National Large-Scale Assessment.
25
Table 1: Links between the Proposed Components and Previous and Ongoing Work
Education Level or Function
Proposed Education Improvement Project
EQRP2 EQRP1 Read ARMENIA TF
Preschool Education Subcomponent 1.1 (: Grants to communities for
improved access and quality (Yes)
Subcomponent 1.1: Grants to communities for
improved access and quality
N/A N/A
Primary and Lower Secondary Education
Subcomponent 1: Curriculum revisions (No)
Subcomponent 1.2 and subcomponent 2.4: Teacher training and
professional development Reforming pre-service
teacher training
Component 1 and Component 2: Development of National
curriculum and subject syllabi Teacher training/professional
development.
Support for Classroom Assessment: Pre-service & in-service
teacher training on student assessment and measurement
Upper Secondary Education
Subcomponent 1.2: Educational equipment and
materials (Yes) Renovation and retrofitting
of schools (No)
Subcomponent 1.4: Resource Centers Training of teachers Training of principals Training of librarians Introduction of reforms in
secondary education
Component 1: Development of National
curriculum and subject syllabi
N/A
Vocational Education and Training (VET)
EU assistance programmes for reduction of poverty through reforms in Vocational Education and Training
EU assistance programmes for reduction of poverty through reforms in Vocational Education and Training
EU assistance programmes for reduction of poverty through reforms in Vocational Education and Training
EU assistance programmes for reduction of poverty through reforms in Vocational Education and Training
Higher Education Component 2: Implementation of the
competitive Innovation Fund (Yes)
Subcomponent 2.1 and Subcomponent 2.3: Implementing Quality
Assurance at national and University level
Pilot Competitive Innovation Fund
Support for CBT: Computer Based Entrance
Exam
26
Education Level or Function
Proposed Education Improvement Project
EQRP2 EQRP1 Read ARMENIA TF
Monitoring and Evaluation Subcomponent 1.3: Implementation of tertiary
education management information system and integration of EMISs for general and tertiary education (Yes)
Participation in TIMSS (Yes)
Subcomponent 1.3 and Subcomponent 2.2 (: Development and
implementation of EMIS for secondary education
Development of EMIS for tertiary education
Participation in TIMSS
Component 1: Establishment of the
Assessment and Testing Center
Support for large scale assessment: National large scale
assessment Support for TIMSS analysis
and dissemination Tailored training for
Assessment and Testing Center staff
ICT in Education Subcomponent 1.3: Strengthening of NACET
(Yes)
Subcomponent 1.3: Educational technologies
in schools Establishment of Internet
and intranet connection for all schools
Development of digital content
Teacher training in ICT Staff management in ICT
Component 2: Educational technologies in
schools Internet connection Development of digital
content Strengthening of NACET Teacher training in ICT
school network optimization Design of the per capita
financing mechanism
N/A
*: (Yes) means that the component/subcomponent is a continuation of previous project activities while (No) means otherwise.
27
5. This subcomponent would support the establishment of sustainable community-based preschool education units under a competitive grant financing scheme. First, a study would be undertaken to identify communities with disadvantaged population and potential organizations able to provide preschool educational services. Based on the results of this study, the competitive grant scheme designed under the ongoing project (P107772) would be adjusted to serve the needs of the poorest communities.10 These communities would provide less monetary contributions than the communities covered under the ongoing EQRP2 project. Community leaders, parents of students and representatives of potential service providers would be encouraged to participate in the grant financing mechanism, and would be trained to prepare competitive proposals with the aim of establishing low-cost and sustainable preschool education institutions. The best proposals prepared by the communities would be selected based on criteria to be defined under the competitive grant scheme. Priority would be given to communities that either do not have preschools or whose preschool facilities do not meet the demand for them. It is expected that about 120 community-based preschool education initiatives would be established using project funds, during the implementation of this subcomponent. The grants extended to the selected communities would target both infrastructure and quality improvements via financing refurbishment of classrooms, purchase of equipment and furniture, provision of pedagogical materials and teacher training. The teacher trainings would be provided by the Step-by-Step program. 6. The design of this sub-component has benefited from the implementation experience and achievements made as part of the ongoing engagement with the government under the ongoing Education APL2 project, which supports school readiness via preschool education micro-projects. The focus on the quality of service delivery through teacher training and provision of pedagogical materials has proved instrumental for improving the school readiness of five and six year old children in participating communities. The Baker Tilly monitoring survey (2012) assessing the impact of pre-school micro-projects on the participating children’s Early Development Index (EDI) indicates positive development dynamics across all measured indicators, namely 50 percent improvement in children’s development as a result of pre-school attendance. The difference of the results of children in treatment and control groups are also tangible, indicating higher development in children attending preschools. This sub-component would maintain focus on improving the quality of service delivery through providing adequate pedagogical support to teachers in partnership with the Ministry of Education. 7. Specific activities to be financed by this subcomponent include (i) dissemination activities for community leaders, parents and communities at large about preschool opportunities and benefits to startup these services in the community and associated grant procedures for micro-projects; (ii) evaluation and selection of micro-projects; (iii) provisions of grants for the implementation of the micro-projects; (v) provision of technical assistance for the implementation of the micro-projects; and (iv) monitoring and evaluation of the impact of the micro-projects.
10 The operational manual for the preschool grants, approved by the Government and the Bank, includes a detailed description of the composition and functions of the governing board for the grants and different grant participants (CEP, provincial education departments, communities, and schools); the criteria for selecting the communities; the implementation of promotion activities in the selected communities; the procedures for submitting proposals; the criteria for selection and approval of the proposals; the details for grant implementation; the arrangements for the monitoring and evaluation of the grants; the procurement methods, the financial management procedures, and the flow of funds for the grants.
28
8. The expected outputs of the subcomponent are (i) the establishment of accessible and sustainable preschool education services in the selected communities; (ii) increases in parents’ and communities’ participation in pre-schools; and (iii) increase in enrollment in preschool education. 9. The main risks for the implementation of the subcomponent include (i) negative demographic trends resulting in reduced numbers of children and associated increased costs per child of preschool provision; (ii) insufficient participation/interest of communities in implementing micro-projects; (iii) delays in contributions from communities and parents to implement micro-projects; and (iv) lack of properly trained preschool teachers in the selected communities.
10. In terms of institutional and implementation arrangements, the division of responsibilities is as follows:
At the central level, the Ministry of Education would be responsible for the overall
management and oversight of this project, as well as for providing relevant pedagogical guidance and reference documents through the Division of Preschool Education and Inspectorate through the MoES and the National Institute of Education.
At the Marz (provincial) level, education departments would be progressively empowered to play a role in the sensitization phase, in the selection of community proposals, and in monitoring and evaluation of implementation.
At the community level, community heads and schools would be progressively empowered to bear more responsibility in the financing and in the overall management of quality preschool services.
11. Subcomponent 1.2. Enrichment of Upper Secondary Schools. The objective of this subcomponent is to enrich the recently established network of high schools by supporting rehabilitation of selected facilities and improvements in teaching and learning conditions, including the provision of contextualized digital learning materials in classrooms and modern equipment and furniture in all high schools. 12. This subcomponent would support the rehabilitation and refurbishment of high schools recently established by the government. Many of these high schools operate in buildings constructed during the Soviet time or even earlier, whose main infrastructure deficiencies include lack of appropriate heating systems, potential seismic instability and humidity problems. It is expected that approximately 17 out of a total of 107 high schools in Armenia would benefit from this subcomponent and offer upper secondary educational services in improved teaching and learning environments.
13. The selection of the schools is based on objective criteria presented in the table 2 below with schools assessed by an infrastructure expert team. Technical conditions of the school buildings – including structure and foundation, roof, floor, walls, and heating and electrical systems – accounts for most of the weight in an index of infrastructure needs. The year the school was constructed is considered, with a higher score given to older schools. Finally, the number of students and the number of villages serviced by the school are considered with, a higher score given to larger schools which service more villages.
29
Table 2: Selection criteria for high schools
Criteria Selected High Schools’ Conditions against the Criteria
A. Technical conditions of the school buildings ( Weight 64 percent total, 8 percent each category)
5. Coverings for the floor, plastering of the walls and ceilings, doors and windows
0 point 1 point 2 points 3 points 4 points
6. Sanitary facilities, including toilets, internal & external water supply and drainage piping
0 point 1 point 2 points 3 points 4 points
7. Heating System 0 point 1 point 2 points 3 points 4 points 8. Electric System 0 point 1 point 2 points 3 points 4 points B. Construction Year (Weight 12 percent)
Before 1960 1960-1991 1992-Present
Construction year of the schools 4 points 2 points 0 point C. Area of service (Weight 12 percent)
Only one region More than one region
The school is serving 0 points 4 points
D. School size (Weight 12 percent)
0-400 students 401-800 students More than 800 students
The total number of students is in the following category
0 points 2 points 4 points
Observations: One high school from each Marz would be rehabilitated. Schools can be considered beyond repair, in which case they would not be targeted by the project. The index ranges from 0 to 100 (100 being the highest priority for rehabilitation) and is calculated in the following way:
14∗ 8 ∗ 12 ∗ 12 ∗ 12 ∗
Where A, B, C and D are the points accumulated in each part. 14. Given that Armenia is a high-risk country in terms of natural hazards and potential earthquakes, the feasibility study for each of the proposed school would include an expert assessment of the seismic conditions. The rehabilitation works supported by the project would ensure compliance with seismic resistance standards and, where needed, strengthening works would be carried out. The rehabilitation of the selected schools would accommodate changes introduced in the curriculum with respect to spatial and functional requirements. The modifications triggered by the curriculum supported by the project may have implications on the number of regular classrooms, specialized classrooms for music, arts and sciences, recreation zones for meetings, group work and discussions and handicap access to assure inclusive education.
30
For this reason, the sub-component would finance the hiring of an experienced engineer, who in close collaboration with the Ministry and Curriculum Specialists would develop recommendations for space management and organization of interior functional layout during rehabilitation in accordance to the requirements of the modified curriculum. 15. To ensure that the newly upgraded school infrastructure is maintained, the component would also support the development and adoption of a school maintenance manual and training in the use of that manual to upgrade the capacity of school managers and administrators in this area.
16. The targeted schools would thus be able to provide improved and safer teaching and learning environments. Empirical evidence shows that poor infrastructure in schools leads to lower attendance and higher drop-out rates. Moreover, infrastructure and well-integrated ICT have a significant influence over measures of students’ well-being. Lastly, and more difficult to measure, poor infrastructure and neglect convey to the students the message that no one cares about the school, which decreases motivation among students.11 17. Since improving upper secondary education in Armenia requires much more than infrastructure, this subcomponent would also finance activities aimed at enriching the teaching and learning conditions of all 107 high schools. It would finance the purchase of modern information and communication technologies equipment and the development of contextualized digital learning materials to be used in classrooms in all upper secondary schools in Armenia. This subcomponent would also support teachers’ training activities on the use of such materials under a student-centered approach. The focus of these training activities would be on using modern teaching methodologies and digital learning materials in the classrooms. In order to monitor the changes in teaching and learning in high schools and assess the degree/effectiveness of ICT integration in teaching and learning, it would finance classroom observations and other studies. By the end of the project, it is expected that an increased percent of high schools teachers would be using modern teaching methodologies in the classrooms. 18. Specific activities to be financed by this subcomponent include (i) renovation and refurbishment of selected high schools; (ii) provision of up-to-date computers and other educational equipment and e-materials and furniture to all high schools; (iii) training of teachers in the use of new ICT resources in all 107 high schools; and (iv) monitoring and evaluation of the impact of the subcomponent. 19. The expected outputs of the component are (i) approximately 17 high schools renovated; (ii) modern ICT resources, such as computer equipment, educational facilities and other e-materials, introduced in all high schools; (iii) increased student enrollment in all high schools; and (iv) teachers trained in the use of new ICT-educational resources. 20. The main risks for the implementation of the subcomponent include (i) delays in coordinated actions by the different bodies involved in rehabilitation works and permits for the envisioned rehabilitation and renovation work in the schools and (ii) possible reluctance by teachers to use up to date facilities, educational equipment and e-materials in the high schools.
11 Branham, D. (2004) and Cuyvers et al. (2011)
31
21. In terms of institutional and implementation arrangements, the division of responsibilities is as follows:
At the central level, the Ministry of Education and Science (MoES) would be responsible for the implementation and oversight of civil works supported by this subcomponent;
Marz (provincial) Education departments would provide support to data collection and give feedback to the Ministry on specific implementation issues related to the high schools;
The high schools would be responsible for maintaining the newly refurbished infrastructure, and for ensuring that their teaching staff is properly trained and familiar with the new education and learning materials;
CEP would support the Ministry in the procurement of goods and services, as well as providing monitoring and technical assistance; and
The National Institute of Education would be in charge of providing classroom-based support to teachers. The institute would also participate in data collection, analysis and evaluation activities.
NACET would also participate in data collection, analysis, and evaluation activities. 22. Subcomponent 1.3. Improving data-collection and monitoring of the education system performance. This sub-component would improve the National Center of Education Technology’s (NACET) capacity to monitor the Armenian educational network, extend adequate ICT coverage to all educational institutions of the country and provide the financing for the participation of the country in large scale international student assessment studies. 23. Since 2013, the center’s ICT service has covered all educational institutions in Armenia, including VET and higher education institutions (HEIs). The center is tasked with managing data from all educational institutions in Armenia and making information available to the public and the authorities in a systematic and organized fashion. By the end of the project, it is expected to be playing the role of a national center for educational statistics and providing decision-makers and key stakeholders with quality information on the whole spectrum of Armenia’s education system. As the integration of ICT into teaching and learning processes is even more critical in rural areas due to the lack of physical and human resources to deliver educational services, this subcomponent would finance the development of an approach to foster the use of digital learning resources in rural classrooms. By the end of the proposed project, it is expected that the center’s Education Management Information Systems (EMISs) would produce quality information for policy making and planning at all levels of the education system of Armenia. 24. To this end, this subcomponent would support the development of an ICT education strategy for the NACET center, as the current absence of this strategy makes it more difficult to implement ICT-related activities, which includes the integration of information systems and the production of quality educational information. Second, it would support the rehabilitation and refurbishment of the center’s facilities, as it is outgrowing its current location and its responsibilities are broadening. Third, as it would be responsible for the operation of EMIS covering all levels of Armenia's education system, the subcomponent would finance the purchase of state-of-the-art hardware and software platforms to run such information systems in an integrated fashion. Fourth, it would finance the establishment of technical service mechanisms for the maintenance of hardware and software in Armenian schools. Fifth, it would finance the
32
training of center staff on management, planning, and monitoring and evaluation. Sixth, in terms of monitoring learning outcomes, the subcomponent would cover, if needed, the participation fees for the 2015 edition of the International Trends in Mathematics and Science Studies (TIMSS). On the user end fend, this subcomponent would finance: (i) training for teachers and non-teaching staff of schools, universities and vocational education and training institutions on the integration of ICT into teaching and learning processes; and (ii) training for users in schools and institutions on the use of quality information produced by these systems. 25. The expected outputs of the component are (i) a properly operating School Internet Network (Armenian Educational Network); (ii) appropriate operating systems for education management information; (iii) proper operating maintenance mechanisms (technical service) for school hardware and software; (iv) modern ICT in education strategy to guide the further activities in this direction; (v) about 3,900 representatives of preschools, schools, universities and vocational education and training institutions would benefit from these training activities; and (vi) Armenia participates in subsequent rounds of TIMSS. 26. The main risks for the implementation of the subcomponent include (i) target groups may not be willing to use ICT both in teaching and learning process and for administrative purposes; (ii) key representatives of schools and universities may not attend envisioned training and may not be eager to use the EMIS/TEMIS; (iii) operating difficulties could arise with some network equipment at schools and in the fiber optic channels of Armenia; (iv) weather conditions may not allow center vehicles to visit schools to repair hardware/software; (v) necessary parts for hardware repair are not distributed on time; and (vi) government has not included appropriate changes in the budget of the center regarding inclusion of new staff. 27. In terms of institutional and implementation arrangements, the National Center of Education Technology would play the central role as the implementation agency for all these processes. Further development of the capacity of the center to lead and support initiatives related to ICT use in education would be required, and formal and informal links to similar organizations in other countries, especially in Europe would be important mechanisms to help achieve this, as would targeted technical assistance and the development and utilization of a set of quantitative and qualitative monitoring and evaluation indicators. To monitor student learning, the Ministry of Education would coordinate its activities with the Assessment and Testing Center as the participating entity involved in implementing subsequent rounds of TIMSS. 28. Subcomponent 1.4. Supporting further improvements in the quality of education through curriculum revisions. The objective of this subcomponent is to bring the national curriculum framework including state curriculum standards and subject syllabuses and standards for grades 1-12 into conformity with the requirements of the General Education and State General Education Curriculum law approved by the government Decree No. 1088 of July 28, 2011. The proposed project would finance: technical assistance and consultative workshops to align the subject syllabuses and subject standards with the requirements of the law. The curriculum revision would be done by Working Groups for the purposes of which the credit proceeds would be used to hire subject experts, school teachers, university professors, and experts from the National Institute of Education. The Ministry of Education would be responsible for the implementation of the modified curriculum including training of teachers through the National Institute of Education.
33
29. Specific activities to be financed by this subcomponent include (i) provision of technical assistance to develop conceptual approaches for the revision of general education subject standards and syllabuses; (ii) creation of working groups for the revision of subject standards and syllabuses for primary, basic, secondary general and specialized education; (iii) revision and improvement of subject standards and syllabuses for primary, basic, secondary general and specialized as needed; (iv) creation of working groups on development of a module on short-term training courses to raise awareness of teachers on changes in the subject syllabuses and standards.; and (vi) training courses for textbook authors. 30. The expected outputs of the subcomponent are (i) revised subject syllabuses and standards aligned with the new uniform State Standard on general and special education at primary basic and secondary general and specialized school levels; and (ii) trained teachers who are adequately aware of the changes in the subject syllabuses and standards and in the revised methodology of students’ evaluation.
31. In terms of institutional and implementation arrangements the division of responsibilities would be as follows:
At the central level, the education Ministry would be responsible for the oversight of this subcomponent;
CEP would support the procurement of services, as well as providing monitoring and technical assistance; and
The National Institute of Education would be in charge or the revision of the curriculum and the alignment of the textbook and education materials to the revised curriculums.
Component 2: Mainstreaming of the Competitive Innovation Fund (CIF) for Higher Education Institutions (HEI) into full implementation (US$ 6.25 million equivalent).
32. The objective of this component is to support the institutionalization of the competitive grant program for HEIs that was developed and piloted under the Education Quality and Relevance Project 2 (EQRP2) (P107772). The CIF fund was designed to support the best proposals prepared by HEIs with the aim of improving the quality, promoting the modernization and labor market relevance of the academic programs, increasing efficiency and equity, and the development of new academic programs of HEIs in Armenia. This component would support the assessment of the pilot fund experience implemented under the ongoing EQRP2 and the review of the government’s approved operational manual for the fund based on pilot’s findings. Under this subcomponent representatives of HEIs would be encouraged to participate in the fund’s programs and would be trained in preparing competitive proposals, project management, procurement, financial management and monitoring and evaluation. 33. The project would finance studies to determine what skills employers seek in recent graduates, how satisfied they are with the skills of the current crop of graduates and what jobs graduates get when they enter the labor market. A communication campaign would be supported by this component to help upper secondary students make more informed choices of higher education institutions and careers. After the assessment, the component would finance the roll-out of CIF fund implementation nationwide. The selection of proposals prepared by HEIs would
34
follow the procedures established in the fund’s Operations Manual designed under the EQRP2 Project and approved by the government and the Bank in 2013. The CIF is expected to cover three main areas:
Quality improvement:
Curriculum and teaching methodology Alignment with Bologna requirements for full integration into the European Higher
Education Area International accreditation/ quality certification Establishment of teaching and learning support facilities
• Infrastructure: multipurpose labs/language facilities • Internal quality assurance and evaluation centers • Centers for collaborative teaching and learning/ remediation courses for
entering students
Relevance improvement: Establishment of new study programs: projects targeting improvements in
education and employability outcomes • New undergraduate/ graduate programs • Dual/joint degree programs • Student-centered approach to teaching and learning (emphasis on
competencies and skills) Involvement of employers in the educational process
• Governance/ Advisory Councils • Job fairs • Curriculum development and internships
Labor market observatories
Efficiency promotion: Rethinking and reorganization of institutional governance, management &
information systems Reform of institutional governance and quality management Consolidation of career tracks Distance education/blended learning
34. Through its different chapters the Operational Manual defines the requirements for the CIF's operationalization, transparency and accountability including a detailed description and functions of the CIF’s main organizational bodies (steering committee, secretariat, and proposals review panel); the operational guidelines and procedures for project selection (call for proposals, submission of proposals, eligibility and completeness, and proposal review and selection) and project implementation (notification and signature of the grant agreement, implementation, monitoring and evaluation of the projects, performance indicators, and monitoring and evaluation
35
of the CIF); the procurement methods including eligible and ineligible expenditures, and the financial management procedures for the grants. 35. Specific activities to be financed by this component include (i) update of the fund’s operational processes and procedures; (ii) provision of grants to HEIs; (iii) monitoring of the fund’s implementation process, including surveys of grant holders and potential applicants and private sector partners; (iv) tapping private-sector and international potential donors to bolster the fund’s sustainability: (v) special trainings for HEIs managerial staff and other relevant staff members on project management skills, procurement issues, and project monitoring and evaluation issues; and (vi) an assessment of the impact of the component. 36. The expected outputs of the component are (i) enhanced public-private partnership in higher education; (ii) modernized HEIs and more relevant higher education programs; and (iii) increased efficiency of HEIs. 37. The main risks for the implementation of the component include (i) insufficient capabilities within HEIs to prepare and implement grants; (ii) lack of partnership opportunities with NGOs, private sector, etc.; (iii) lack of sustainable co-financing from HEIs for all the phases of the project; and (iv) insufficient readiness of private and public sector to finance innovative projects of HEIs. 38. In terms institutional and implementation arrangements the education Ministry, through its Higher Education Department, would be responsible for technical matters while the CEP would assist on procurement and financial management, as well as providing monitoring and technical assistance for the implementation of the grants. Component 3: Project Management, Monitoring and Evaluation (US$3.75 million equivalent).
39. The key objectives of this component would be to provide continued support of mainstreamed arrangements for the management and monitoring of project implementation activities and outcomes. This component would provide funds for monitoring and evaluation studies and audits of both project financial statements and grants implementation. 40. Specific activities to be supported by the component are the following (i) financing a project implementation unit, the Center for Education project, with a core staff hired as consultants; (ii) implementing and managing the processes of procurement, disbursements, and financial management, project monitoring and reporting; (iii) acquiring small amounts of office furniture and equipment; (iv) financing local travel, utilities and publications, translations, small office repair, office supplies, fuel, Internet service, bank commission charges, vehicle maintenance and repair; (v) financing the implementation and management of planned special studies; (vi) facilitating the coordination, communication flows and dissemination of information with participating institutions; (vii) helping organize working groups, seminars and workshops; and (viii) providing training to staff in the CEP center in the areas covering procurement, disbursements, information technology, project management and other areas identified and proposed in the course of project implementation.
1. The Ministry of Education and Science of Armenia would be responsible for the implementation of the project supported by the services of the Center for Education Project (CEP) acting as the project implementation unit. The center is a project management unit established within the Ministry. It was created in 1996 and has gained significant strength in project management through the implementation of Bank-funded education projects, including two recent ones. The center would administer, coordinate, implement, monitor and evaluate the proposed project. It would also be responsible for the procurement and financial management functions, the flow of funds and for budgeting, accounting, and reporting. The center comprises a core group of technical, administrative and support staff. 2. The CEP center would work within the existing organizational structure of the education Ministry to support state management institutions (Technical Departments in the Ministry, provincial authorities, education departments, the State Inspectorate of Education), as well as key beneficiaries, such as the Ministry’s National Center for Education Technology. Although the Assessment and Testing Center (ATC) created under Education Quality and Relevance Project 1 (EQRP1) would not be a beneficiary institution under this proposed project, it would be a participating entity involved in monitoring and evaluation. CEP would also work with tertiary education institutions and entities related to tertiary education, such as the newly established Armenian National Quality Assurance Agency (ANQA). It would also coordinate with preschools and high schools, the other institutions targeted under the proposed project.
3. Given the scope of the project, there are several relevant stakeholders (Ministry officials, nongovernmental organizations, higher education institutions, teachers, parents, and other institutions) that can be influential or have a stake in the education sector. The proposed project strengthens the accountability framework between the Ministry, key educational agencies and the public by putting in place an integrated Education Management Information System and supporting the monitoring of student learning outcomes, which would provide solid and regular information on the status of the education system’s performance (e.g. student progression and completion rates). To ensure close collaboration between the schools and parents, the government has already established an interactive website that enables parents to monitor their children’s progress, ask the school administrators questions and share concerns about the teaching and learning process. To properly address the needs and concerns of these stakeholders, the Ministry would conduct stakeholder consultations and focus group discussions during the implementation of the proposed project. Investments under its first component are expected to contribute to quality improvements and reduced demand for private tutoring in the targeted high schools. The project would support surveys to monitor the scope of private tutoring in those schools and would identify shortcomings of the existing education system so that students without access to private tutoring could receive better education within the formal education system.
37
4. Under the proposed project, the Center for Education Project would continue its activities related to general administration, coordination, implementation, monitoring and evaluation, as well as facilitating the decision-making process of the project. It would comprise a core group of technical, administrative, and support staff. The detailed description of staff would be provided in the Operation Manual but it is envisioned to comprise two component/sub-component heads supported by a group of 12 technical specialists. The component/sub-component heads include: the preschool sub-component head, the high school enhancement sub-component head, the strengthening key institutions for monitoring educational outcomes head, the student and monitoring learning sub-component head, the curriculum sub-component head, and the higher education component head. The staff would include specialists in preschools, infrastructure and environment, information technology and networks and curriculum as well as specialists well- versed in the workings of the innovation fund. The center would also be supported by a lawyer, a chief accountant, two accountants, two procurement specialists, an office manager and assistant and a translator. 5. The Minister of Education would assume the overall leadership for implementation. The Ministry of Finance would be responsible for ensuring the proper flow of funds, the timely provision of counterpart funding, financial monitoring of project activities, and the replenishment of the special account and allocation of funds for project activities. The Deputy Minister of Education would be responsible for the policy dialogue related to general education (including preschool and high school) and to tertiary education. She would also interface with the technical departments in the Ministry, the provincial education departments, the State Inspectorate of Education and educational state institutions. The education Ministry’s technical departments include the General Education Department, the Education Policy and Planning Department, the Information, Analysis and Development Programs Department and the Higher Education department. These departments would coordinate their relevant areas of work with the state institutions. 6. The implementation arrangements under the proposed project would be governed by the guidelines and procedures set out in the Operational Manual (OM). The CEP has developed the manual, drawing from the lessons learned in the implementation of the EQRP1 and EQRP2 projects, and reflecting the financial management, procurement and reporting arrangements planned for the proposed project. The approval by the Ministry of Education and Science of an Operational Manual, including specific Operational Manuals for the grants for Pre-school Micro-projects and CIF Sub-projects, which is satisfactory to the Association, is a condition of effectiveness. Financial Management/Disbursement 7. The Financial Management (FM) arrangements of the CEP were reviewed and assessed as satisfactory and acceptable to the Bank for the implementation of this Education Improvement Project. The FM assessment undertaken in October 2013 confirmed that (i) accounting and reporting is performed using reliable and functional 1C accounting software; (ii) the filing system is adequate; (iii) the FM staff has extensive experience in Bank procedures and financial reporting;
38
(iv) the internal control system is adequate; and (v) results from previous FM implementation support and supervision missions and the latest annual audit of the active project were satisfactory. 8. The overall financial management risk for the project before and after mitigation measures is assessed as moderate. 9. The Center for Education Project is capable of preparing relevant budgets. Its financial manager prepares the project’s annual budget in co-operation with procurement specialists and component heads. The budget is based on the procurement plan, endorsed by the Ministry of Education and Science and approved by the Ministry of Finance. It is classified by categories, components, and sources of funds. All changes in procurement plan must be reviewed in advance by the CEP director and agreed with the Bank and with the Ministry of Finance before they can be incorporated in the annual budget. 10. The financial management and accounting staff of the CEP’s financial department are experienced in implementing WB-financed projects. They include an experienced financial manager (she was previously the CEP’s chief accountant) and two accountants one of whom worked at CEP as an accountant for several years and filled the vacant chief accountant’s position. Due to this change, currently there is a vacant position of an accountant, which CEP plans to fill by the time the project’s implementation begins. The CEP’s financial management staff is well aware of the Bank’s policies and procedures and has participated in a number of regional fiduciary workshops organized by the Bank. The latest one took place in April 2013 in Odessa, Ukraine. 11. The CEP utilizes 1C accounting software for the project. It has been used by a number of Armenian project implementation units for many years and is found to be adequate. The software automatically generates all forms of Interim Financial Reports (IFRs), except the SOE Withdrawal Schedule form. The software was recently updated to reflect recent changes in the local tax legislation. The accounting data backup is conducted on a monthly basis on separate CDs, which are kept in a fire-proof safe. 12. The accounting at the CEP is maintained on an accrual basis according to the former Accounting Standards of the Republic Armenia (ASRA). For the project reporting, the accrual basis would also be adopted. The accounting procedures are covered by the FMM. The current chart of account used for the active project would be adapted to the project’s requirements. 13. CEP has overall an adequate internal control system in place. The existing internal controls and procedures are adequately documented in the FMM. The CEP maintains a good and well-systemized filing system that allows quick retrieval of documentation. 14. Prior to the project effectiveness the project’s Operational Manual (including specific OMs for grants for Pre-school Micro-projects and CIF sub-projects), acceptable to the Bank, would be adopted by the MOED, where the funds flow and the controls over those grants would be described. In addition, prior to the project effectiveness, CEP would update its FMM to reflect specific activities and controls relevant to the project (a capacity-building action).
39
15. The CEP does not pay its director’s representation expenses and maintains no petty cash. All payments made directly by the CEP (including payments to grant recipient institutions) are executed via treasury transfers. In the meantime, these institutions (as separate entities) have their own petty cash boxes from which payments are made in cash also. The grants’ OMs (under the ongoing project) specify the petty cash limits. 16. The office manager is responsible for purchases based on the requests submitted for office supplies, which are approved by the financial manager and authorized by the director. 17. There are overall adequate internal controls over contract monitoring system in place at CEP. The contract monitoring of the payments made and invoices/acceptance acts received under each contract/grant is made by the chief accountant, who developed a specific Excel spreadsheet to trace the payments made and the amounts due. To tighten monitoring, the Bank team shared a more advanced sample spreadsheet (also presented during the fiduciary training in Odessa) to be elaborated and used by the CEP. The monitoring of the payments under the grants is also made via separately tailored Excel spreadsheet, which indicate the amounts paid and due, what reports have been received per each phase of the grants, as well as cumulative amounts that are reconciled to the total amount of the grants. Overall there are adequate controls over payment monitoring. 18. Once received from a contractor or a supplier, the invoices are checked and approved by the financial manager or chief accountant and authorized for the payment by the director. The financial staff of the CEP also verifies the invoices and acts of acceptance, reconciles them with accounting records, and executes the payment. Payments Orders are authorized by the financial manager (or acting chief accountant) and the CEP director. 19. The reconciliation of the project DA and GCF accounts for the project (held in Treasury) with project accounting records are made after each transaction or at least monthly by the acting chief accountant and is verified by the financial manager. The Treasury account statements are available online (via Treasury-Client system), with hard copies available upon request. Monthly reconciliation statements are approved by the director. 20. The Statements of Expenditures (SOEs) are generated from 1C accounting software and then finalized in Excel spreadsheets, which incorporate the expenditure to be included in the relevant SOE. 21. CEP conducts stocktaking of the fixed assets (FAs) annually. The FAs register is maintained in 1C accounting package. All fixed assets of the CEP are allocated to the personnel who are formally responsible for their condition and existence. All fixed assets have inventory tags attached. Considering the small size of the CEP, no internal audit function exists or is required. 22. Project management-oriented IFRs would be used for the project monitoring and supervision. The active project’s IFRs have always been received on-time and were acceptable to the Bank.
40
23. The format of IFRs has been confirmed during assessment and includes: (i) Project Sources and Uses of Funds, (ii) Uses of Funds by Project Activity, (iii) Designated Account Statements, (iv) A Statement of the Financial Position, and (v) SOE Withdrawal Schedule. CEP would be producing a full set of IFRs every calendar semester throughout the life of the project. These financial reports would be submitted to the Bank within 45 days of the end of each calendar semester. The first semester IFR would be submitted after the end of the first full semester following the initial disbursement. 24. CEP also submits monthly and quarterly reports to the government of Armenia by sources of funds showing project financing and expenditures by categories. 25. The audit of the project would be conducted (i) by independent private auditors acceptable to the Bank, on Terms of Reference (TOR) acceptable to the Bank and procured by the CEP, and (ii) according to the International Standards on Auditing (ISA) issued by the International Auditing and Assurance Standards Board of the International Federation of Accountants (IFAC). The CEP’s current auditing arrangements are satisfactory to the Bank, and it has thus been agreed that similar audit arrangements would be used for the project to cover the project financial statements. The sample audit TOR agreed with the Bank and the annual audited project financial statements would be submitted to the Bank no later than six months after the end of each fiscal year, also at the closing of the project. If the period from the date of effectiveness of the loan to the end of the Borrower’s fiscal year is no more than six months, the first audit report may cover financial statements for the period from its effectiveness to the end of the second fiscal year. The Borrower has agreed to disclose the audit reports for the project within one month of their receipt from the auditors and acceptance by the Bank, by posting the reports on its web site (http://www.cfep.am). Following the Bank's formal receipt of these reports from the Borrower, the Bank would make them publicly available according to World Bank Policy on Access to Information. The contract for the audit awarded during the first year of project implementation may be extended from year-to-year with the same auditor, subject to satisfactory performance. The cost of the audit would be financed from the proceeds of the project. 26. The accounting staff of the CEP is well aware of the Bank’s disbursement procedures. Under the ongoing project, CEP uses the Bank e-disbursement and e-signature systems. The CEP would establish a Designated Account (DA) specifically for this Project in the State Treasury maintained by Central Bank of Armenia, which is holding almost all DAs for current Bank-financed projects in Armenia. A project account would be opened in the Treasury for transfer of the government’s Counterpart Funding.
27. Project funds would flow from: (a) the Bank, either: (i) via the DA to be maintained in the Treasury, which would be replenished on the basis of SOEs or full documentation; or (ii) on the basis of direct payment withdrawal applications and/or special commitments, received from the CEP; and (b) the government, via the Treasury through normal budget allocation procedures initiated by the implementing agency in accordance with standard Treasury and Budget execution regulations. Withdrawal applications documenting funds utilized from the DA would be sent to the Bank at least every three months. The following disbursement methods may be used under the project: reimbursement, advance, direct payment and special commitment. The DA ceiling is
41
established at US$ 1,500,000 which is reflected in the Disbursement Letter (DL). Detailed instructions on withdrawal of loan proceeds are provided in the Disbursement Letter. Procurement
28. Procurement under the project would be carried out in accordance with the World Bank’s “Guidelines: Procurement of Goods, Works, and Non-Consulting Services under IBRD Loans and IDA Credits & Grants by World Bank Borrowers” (January 2011) and “Guidelines: Selection and Employment of Consultants Under IBRD Loans and IDA Credits & Grants by World Bank Borrowers” (January 2011), and the provisions stipulated in the Legal Agreement. The World Bank Guidelines on Preventing and Combating Fraud and Corruption in Projects Financed by IBRD Loans and IDA Credit and Grants dated October 15, 2006 and revised on January 2011, would also apply. 29. Procurement would be carried out by CEP, which implemented the first Education and Quality Relevance Project (EQRP) and is implementing the Second Education Quality and Relevance Project (EQRP2). CEP has satisfactory experience and knowledge of the “Bank’s Procurement and Consultant Guidelines”. The overall project risk for procurement after mitigation measures is moderate. General Procurement Notice 30. The General Procurement Notice (GPN) and its updates would be published in the UN Development Business and the Bank’s external website. The initial GPN would be published in February 2014. Procurement Plan 31. For each contract to be financed by the loan, the different procurement methods or consultant selection methods, the need for pre-qualification, estimated costs, prior review requirements, and time frame are agreed between the borrower and the Bank in the Procurement Plan. The procurement plan (see Appendix 1 to this Annex), which covers the entire project and its updates or modifications, shall be subject to the Bank’s prior review and its having no objection before implementation. The Procurement Plan and its updates would be published in the Bank’s external website. 32. Goods to be procured under the project are: equipment for high schools, furniture, vehicles, and office equipment. Works would include rehabilitation of approximately 17 high schools and renovation of NACET building. Consulting services would include, among others, development of design and technical specifications for the rehabilitation of high schools and the NACET building, development of social and environmental management plans, and supervision services of the relevant works. Thresholds for Procurement Methods 33. Goods: Goods and equipment estimated to cost US$300,000 or more would be procured through International Competitive Bidding (ICB). Goods estimated to cost less than $300,000 and
42
more than US$100,000 would be procured through National Competitive Bidding (NCB). Readily available off-the-shelf goods estimated to cost less than US$100,000 each may be procured through Shopping (SH) on the basis of at least three written quotations obtained from qualified suppliers. 34. Works: Works estimated to cost US$4,000,000 and more would be procured through ICB. Works estimated to cost less than US$4,000,000 and more than US$100,000 would be procured through NCB. Contracts estimated to cost less than US$100,000 each may be procured through Shopping (SH) procedures on the basis of at least three written quotations obtained from qualified contractors. 35. Consultant Services: Consultancy services to be provided by consulting firms estimated to cost US$300,000 or more would be procured through Quality and Cost Based Selection (QCBS) method. Consultancy services to be provided by consulting firms estimated to cost less than US$300,000 may be procured through Consultants’ Qualifications (CQ) method. For assignments estimated to cost less than US$200,000, the shortlist may comprise only national firms according to the paragraph 2.7 of the Consultant Guidelines. However, if foreign firms express interest, they shall be considered. Individual consultants would be selected in accordance with Section V of the Consultant Guidelines. Prior Review Thresholds 36. Prior Review thresholds are proposed as follows:
ICB - All; NCB - First two NCB contract for Goods and first 2 contracts for Works; All DC contracts; All contracts with consulting firms estimated to cost US$100,000 or more and all
Single Source Selection contracts; All contracts with individual consultants estimated to cost US$50,000 or more and all
Single Source Selection contracts. Mitigation Measures 37. Based on the working experience with CEP on the ongoing EQRP2, adequate mitigation measures have been put in place and would be closely monitored to ensure that the residual project risk is acceptable, including: (a) training on procurement of works for CEP staff and beneficiaries; (b) procurement prior and post reviews, the latter to be conducted once or twice a year; (c) monitoring of procurement progress against the procurement plan and recommending an action plan whenever there are perceived delays; (d) goods, works and consulting services would be packaged to generate the maximum competition; and (e) hiring a civil engineer who would review the technical parts of the civil works contracts (i.e. TORs for design companies, acceptance of deliverables such as architectural design, BoQ, qualification criteria in the Bid Docs, evaluation of bids, monitoring of the civil works etc.)
43
Appendix 1: Procurement Plan
Goods
Works
Goods
No According to the Cost Tables
Activities
No of Packages
Procurement Method
Review by WB PRIOR/ Post
Bidding Document submission
to W
B
No‐objection to the Bidding
Document
Invitation Date
WB No to Bid Evaluation
Report
Contract Signing Date
Completion Date
Contract Price by Contract
Contract Currency
Contract Price in USD
/date of
contract signing/
Supplier
Contract No
Status
Component 1. Enhancing the Quality of General EducationSub-component 1.2: Enriching Upper Secondary Schools
1
Equipment for High Schools /not included in the list of the
Environmental and Social Aspects 38. Component 2 of the project would support upgrading of the selected school buildings. Rehabilitation works are expected to have some temporary negative impacts typical for reconstruction/rehabilitation of small to medium-size buildings. OP/BP 4.01 Environmental Assessment was triggered. According to the principles of this safeguard policy, and based on the outcome of environmental screening, the project is classified as Category B. Environmental risks of the project implementation would be on a modest scale, limited to the construction sites and to the period of construction works. Managing these impacts and keeping them to the acceptable minimum would be effectively achieved by adhering to good construction practices and applying specific mitigation measures to be provided in the Environmental Management Plans (EMPs). An important principle for selecting school buildings and for preparing adequate design documents for their upgrading would be to screen them against any structural damage and instability, and to either address these faults or drop buildings that are beyond repair from the project. If the designed works imply replacement of damaged roofs, there is a high likelihood of encountering asbestos-containing old roofing materials. Handling of such hazardous construction waste at the work site and its disposal would require close supervision and control, in order to minimize exposure of construction workers and avoid future risks to public health that may arise in case of improper dumping. As all rehabilitation works under the project are expected to take place within the existing footprint of educational facilities and no land acquisition/use is expected, OP 4.12 is not triggered. 39. According to the project design, civil works would be implemented in multiple locations to be identified at a late stage of the project preparation and early into implementation. The nature of works to be undertaken at all of these locations would be pretty similar, with generally well-known environmental and social impacts. Therefore, an Environmental Management Framework (EMF) was developed as part of the project preparation process, while the site-specific EMPs would be worked out later on, as the beneficiary institutions are identified and detailed designs for their rehabilitation become available. The simplified EMP Checklist for Small Construction and Rehabilitation Activities would be used for most construction sites, while a fuller version of EMP may be required in the unlikely case of higher risk Category B works. A template for monthly field monitoring report is also provided in the EMF, as a user-friendly tool for tracking environmental performance of contractors and for reporting on the compliance with EMPs.
40. School buildings suggested for rehabilitation with support of the project are predominantly old and there is some likelihood of some of these buildings carrying a historical value. OP/BP 4.11 is triggered to ensure that the process of school selection would involve checking if any of the buildings are formally entered into the list of the nation's historical and/or cultural heritage. In such cases the building renovation designs would be cleared with the national authorities responsible for the preservation of cultural heritage to ensure that historic/cultural value of such buildings are not compromised as a result of improvement works. These provisions are included into the EMF. 41. The advanced draft of the EMF document was disclosed in-country in Armenian and English by posting it on the web page of the education Ministry. It was published through the Bank’s InfoShop as well. A public consultation meeting on the draft EMF was held in Yerevan on December 10, 2013. The EMF was then finalized and re-disclosed in-country and through the InfoShop.
47
42. The Ministry of Education and Science is the implementing agency for the project. CEP, under the Ministry, would act as the unit responsible for day-to-day management of project implementation, including procurement, disbursement, safeguards application, and M&E. Although the Ministry and CEP have been involved in the implementation of other Bank-supported projects in the past and performed well in that capacity, their experience in the application of safeguards policies is modest. In contrast with the proposed project, the previous ones did not include civil works and were classified as environmental Category C. The Bank team discussed with the CEP the institutional arrangements to close the gap in human capacity for the application of safeguards. It was agreed that for the period of the project preparation, CEP management would appoint one of its staff members as the point person for overseeing the safeguards compliance and would hire environmental and social specialist once the project goes into operation and resources become available. This specialist should have relevant experience and skills to monitor and ensure compliance of all project activities with Bank safeguards policies and would receive on-the-job coaching from the Safeguards Specialists of the World Bank Task Team promptly upon recruitment. Hiring and maintaining this specialist in CEP throughout the period of civil works performance under the project would be a legally binding commitment of the borrower. Updates on safeguards supervision and compliance would be included in all project progress reports provided to the Bank by CEP. Monitoring & Evaluation
43. The PDO level and intermediate results indicators would be monitored using the following sources and methodologies:
regular data collection process; baseline and follow-up surveys; and reports prepared by the CEP.
44. CEP would be responsible for bringing together the reports and representatives of the education Ministry for monitoring of the PDO and results, and communicating with the World Bank according to the frequency of reports described in Annex 1. M&E under the project would be integrated into regular monitoring functions of the Ministry. Project indicators have been selected on the basis of the data requirements already planned as part of the Ministry’s regular data collection process. 45. Under the proposed project, efforts would be directed at gradually building up the Ministry’s capacity to plan, monitor and evaluate policy. These functions so far have largely been shouldered by the CEP, which has significantly contributed to education reforms over the past 10 years. These key areas of management need to be mainstreamed and institutionalized in the Ministry. Some of its departments, such as the Information Analysis and Development Programs Department, appear to be already involved in some aspects of policy planning and monitoring. This department could be strengthened through capacity building, training and technical assistance to assume a stronger role in management and coordinating functions associated with policy, planning, monitoring and evaluation.
Given the need to engage and coordinate with multiple stakeholders with a wide range of objectives which cannot be fully accommodated by this project, there is a risk that political and broader stakeholder support for the ongoing reforms may diminish raising questions of sustainability for quality improvements which are necessary to complement the proposed project activities. This may affect project design and implementation.
To address this risk during all stages of project preparation and implementation extensive stakeholder consultations would take place.
Risk Description: Implementing Agency: The Center for Education Project (CEP) may not be able to retain qualified staff for project management and coordination due to low salaries. Financial Management: The overall financial management risk for the project before and after mitigation is assessed as moderate. Preschool Component: The capacity of local authorities and provincial education departments is low in relation to the planned
Risk Management:
Specific professional development opportunities through training and knowledge exchange will be promoted by the project to retain the qualified staff. Specific professional development opportunities through training and knowledge exchange will be promoted by the project to retain the qualified staff. Prior to the project effectiveness the project’s Operational Manual (including specific OMs for grants for Pre-school Micro-projects and CIF sub-projects), acceptable to the Bank, would be adopted by the MOED, where the funds flow and the controls over those grants would be described.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
The proposed project would provide capacity building training for local governments and other stakeholders for implementing project components.
49
preschool education expansion. Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Governance Rating Low
Risk Description: There is the risk of political interests interfering with targeting and selection mechanisms foreseen under the following activities: preschool expansion, selection of high schools to be rehabilitated, and Competitive Innovation Fund in higher education.
Risk Management:
Objective selection criteria and validation mechanisms would be agreed upon during project preparation and included in the POM; oversight would be in place during implementation.
Resp: Both Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
The Competitive Innovation Fund pilot experience undertaken under EQRP2 would be assessed and the grant financing scheme and selection criteria would be reviewed based on the findings of this assessment. Oversight would be in place during implementation.
Resp: Both Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Project Risks
Design Rating Moderate
Risk Description: Preschool Component: At the community level there is the risk of limited demand for community-based preschool education and low number of service providers due to the newness of the initiative and associated lack of understanding and knowledge of the benefits of the initiative for potential service providers and parents of children. There is also the risk that some communities may not have adequate expertise to prepare the required grant proposals, and/or that some communities may not be able to contribute to the preschool to the extent required by the project Strengthening key institutions for monitoring outcomes of educational institutions and for assessments of student learning in Armenia
Risk Management:
Preschool Component: To address this risk the Ministry of Education and Science would organize public awareness campaigns in poorer communities to inform stakeholders of expected benefits and train them for grant proposal preparation.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
Capacity building efforts would take place at the community level to ensure applications from poor communities. The grant scheme would introduce sensible options to ensure that poorer communities can be targeted for the preschool expansion component.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
Strengthening key institutions for monitoring outcomes of educational institutions and for assessments of student learning in Armenia component: To address these risks an ICT education strategy for NACET would be supported as a key activity for the project preparation through the ongoing ECAPDEV grant for project preparation.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
50
component: There is the risk that currently there is no ICT education strategy for NACET. The absence of an ICT education strategy may make it more difficult to implement ICT-related activities, including the integration of information systems and the production of quality educational information. There is the risk that NACET staff may not be adequately trained to fulfill the newer and broader range of expected responsibilities of NACET.
Risk Management:
NACET staff would be trained on management, planning, and monitoring and evaluation.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Social and Environmental Rating Moderate
Risk Description: There is the risk that the proposed upgrading of high school buildings may have some limited environmental and social impacts within the work sites. There is the risk that the project triggers other safeguard policies. There is the risk that the selected school sites present some possible structural damage and present inadequate seismic stability. Implementing Agency: The MoES and CEP have no experience in procurement of works and have limited experience in the application of safeguards policies as previous activities were classified as environmental Category C.
Risk Management:
To address these risks the implementing agency would prepare an Environmental Management Framework (EMF) for good environmental management for this proposed project. Site-specific Environmental Management Plans (EMPs) would be developed once individual buildings are identified and detailed designs are prepared for their rehabilitation. Structural soundness of the schools: The schools sites would be inspected for identification of a possible structural damage and for assessment of their seismic stability. Buildings damaged beyond repair would not be included into the proposed project, while those carrying structural damage would be reconstructed.
Resp: Both Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
The Bank team would work with the MoES and CEP in the course of the project preparation to carefully assess any institutional gaps that these institutions may have in safeguards policies and take action to ensure that these gaps are closed by the time of the project effectiveness.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Program and Donor Rating Low
Risk Description: Risk Management:
There is the risk of duplication of services if project finances grants for preschool expansion in areas already being covered by other donors or NGOs.
The implementing agency CEP-PIU would organize consultations with relevant donors to design coordinating mechanisms to avoid duplication of efforts or stepping in other donors' actions. Oversight would be in place during project implementation.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
51
Delivery Monitoring and Sustainability Rating Low
Risk Description: There is a risk of weak monitoring and evaluation due to low capacity and the multitude of stakeholders and service providers involved. There is a risk of project supported activities not sustained after project closing.
Risk Management:
To address this risk a comprehensive Management Information System (MIS) would be established for M&E purposes. In addition, the Project would provide technical assistance to service providers to ensure the quality of M&E systems.
Resp: Client Status: Not Yet Due
Stage: Both Recurrent:
Due Date:
Frequency: Yearly
Risk Management:
The Government has a strong record of sustaining the activities and/or institutions supported or created under previous World Bank projects. Teacher training, textbook alignment to new curriculum, NACET, ATC, preschools, ANQA, among others, are all examples of activities and/or institutions supported under previous projects which are now funded by the Government. For CIF the ministry is planning on using government and private funds for sustainability after project closing.
Resp: Status: Stage: Recurrent:
Due Date:
Frequency:
Overall Risk
Overall Implementation Risk: Rating Moderate
Risk Description:
A moderate risk was selected given as the Ministry of Education and Science and CEP-PIU have the required capacity and mechanisms to deal with implementation issues as demonstrated for successful implementation of Bank projects (EQRP1 and EQRP2).
52
Annex 5: Implementation Support Plan
Armenia: Education Improvement Project Strategy and Approach for Implementation Support
1. The implementation support strategy for the proposed project would include regular dialogue with the government, joint review of the project implementation and regular exercise of fiduciary oversight throughout implementation.
a. Regular dialogue with the government would facilitate early identification of problems and obstacles that could delay implementation. It would enable the timely provision of technical advice and support to remove such obstacles. This would help to identify issues as they emerge and address them through advice and support in an expeditious manner, without waiting for joint reviews.
b. Joint reviews would take place twice a year, aimed at examining the progress in
achieving agreed targets and results. The Bank Task Team would participate in the reviews with representatives of the GoA. During each review, the type of implementation support that is needed would be identified, followed by joint decisions on necessary assistance.
c. Fiduciary oversight would enable the Bank to fulfill its fiduciary obligations and ensure
compliance with its fiduciary standards through the ongoing supervision of the project’s financial management and procurement arrangements and results.
i. Financial management. The Bank would conduct risk-based financial
management implementation support and supervisions within a year from the project effectiveness, and then at appropriate intervals. During the project implementation, the Bank would supervise the project’s financial management arrangements in the following ways: (a) review the project’s semi-annual IFRs as well as the project’s annual audited financial statements and auditor’s management letters and remedial actions recommended in the auditor’s management letters; and (b) during the Bank’s on-site missions, review the following key areas (i) project accounting and internal control systems; (ii) budgeting and financial planning arrangements; (iii) disbursement arrangements and financial flows, including counterpart funds, as applicable; and (iv) any incidents of corrupt practices involving project resources. As required, a Bank-accredited Financial Management Specialist would participate in the implementation support and supervision process.
ii. Oversight on procurement would be provided through prior reviews in
accordance with the procurement thresholds. Supervision would be carried out twice a year, through both desk and on-site reviews of procurement arrangements and results, including post review of contracts selected in a random manner. As needed, on-site training on procurement may be provided upon request to the Ministry or CEP staff.
53
Implementation Support Plan 2. The table below reflects the implementation support plan for the proposed project.
Time Focus Skills Needed Resource
Estimate Partner Role
First 12 months
Technical and operational support: (i) enhancing the quality of General Education; (ii) supporting tertiary education through mainstreaming the competitive innovation fund; (iii) M&E; and (iv) overall implementation
Senior Education Economist (TTL)
10 weeks Participation in joint reviews
Technical support: (i) increasing access and quality of preschool education; (ii) enrichment of upper secondary schools; (iii) strengthening key institutions for monitoring; (iv) supporting further improvements in the quality of education through curriculum revisions; and (v) mainstreaming competitive innovation fund
Technical support: (i) curriculum revisions; (ii) enrichment of upper secondary schools; and (iii) competitive innovation fund
Senior Operations Officer
8 weeks
Technical and operational support Education Specialist (Consultant)
8 weeks
Financial management support Financial Management Specialist
3 weeks
Procurement support Procurement Specialist 4 weeks Environmental support Environmental
Specialist 4 weeks
13th – 60th months
Technical and operational support: (i) enhancing the quality of General Education; (ii) supporting tertiary education through mainstreaming the competitive innovation fund; (iii) M&E; and (iv) overall implementation
Senior Education Economist (TTL)
12 weeks Participation in joint reviews
Technical support: (i) increasing access and quality of preschool education; (ii) enrichment of upper secondary schools; (iii) strengthening key institutions for monitoring; (iv) supporting further improvements in the quality of education through curriculum revisions; and (v) mainstreaming competitive innovation fund
Education Specialist / Education Economist
8 weeks
Technical support: (i) curriculum revisions; (ii) enrichment of upper secondary schools; and (iii) competitive innovation fund
Senior Operations Officer
8 weeks
Technical and operational support Education Specialist / Education Economist Civil Engineers
4 weeks 8 weeks
Financial management support Financial Management Specialist
4 weeks
Procurement support Procurement Specialist 4 weeks Environmental support Environmental
Specialist 4 weeks
54
Annex 6: Economic and Financial Analysis
Armenia: Education Improvement Project
1. INTRODUCTION 1. The Education Improvement Project (EIP) addresses various constraints and challenges of the Republic of Armenia’s educational system in the areas of preschool, general and higher education. It is the natural continuation of a long-term engagement between the Bank and the Republic of Armenia, which includes the Education Quality and Relevance Project (EQRP) 1 and the ongoing EQRP 2. The proposed project focuses on various aspects of the education system such as increasing access and quality of preschool (subcomponent 1.1); enriching upper secondary schools through infrastructure rehabilitation and the introduction of new technologies into teaching (subcomponent 1.2); Improving data-collection and monitoring of the education system performance (subcomponent 1.3); supporting further improvements in the quality of education through curricula revision (subcomponent 1.4); and mainstreaming the Competitive Innovation Fund (CIF), which provides grants for Higher Education Institutions (HEIs) on a competitive basis (component 2). 2. This annex presents the economic analysis that provides the rationale for both public investment and Bank’s involvement in the abovementioned areas and a cost-benefit analysis that justifies the investment. Expected Development Impact 3. The EIP seeks to (i) improve school readiness of children entering primary education and enhance the learning environment in upper secondary education, and (ii) support increased labor market linkages of higher education institutions in Armenia. Upon the project’s completion, it is expected that school readiness among primary education entrants would be improved, the teaching and learning environments in high schools enhanced, and the link between higher education institutions and the labor market strengthened. 4. The impact being sought goes in line with the World Bank’s 2014-2017 Country Partnership Strategy (CPS), which guides the engagement between the Bank and the client. The strategy underscores skills development and increasing the relevance of education with the focus on job creation. However, it also highlights that the provision of quality education remains a priority that goes beyond economic growth. The EIP supports this strategy in several ways. The project supports pre-primary education which is strongly associated with the development of key cognitive and non-cognitive skills that provide the foundation on which further skills can grow. Moreover, through the enrichment of upper secondary schools and curricula revision, the project would improve the quality of general education and make it more relevant to the labor market. Finally, the Competitive Innovation Fund envisioned under the project has a direct impact on the relevance of higher education programs.
55
Rationale for public investment
5. General education in Armenia is primarily provided by the State. In 2012, over 98 percent of the students in grades 1-12 attended public schools.12 The rationale for public investment in education stems from the existence of market failures and equity concerns. The market failures are related to (i) informational asymmetries or bounded rationality,13 (ii) credit market imperfections, and (iii) externalities. For instance, parents may be unaware of the full extent of private returns to education and may fail to enroll their children in school (informational asymmetries). If households had to pay for their children’s education, it could be the case that the poorest families would not be able to afford tuition even if they wanted to, due to difficulties in financing (credit market imperfections). Finally, it is commonly accepted that even though individuals benefit from education, society at large also benefits as well (externalities). Education in all its levels has positive effects on social behavior, political participation, as well as contributing to overall economic growth, innovation and development. The fact that individuals do not capture all the returns to their investment leads to a level of education that is below the optimal level for society. The equity concerns arise as the poorest have usually low access to quality of education at all levels.
Rationale for World Bank involvement
6. The World Bank has broad experience in the areas included in the proposed project (preschool, rehabilitation and infrastructure upgrade, information and communication technologies, curriculum development, education management and information systems, and higher education), including many projects in Europe and Central Asia. Project has benefited from the Bank’s experience as would project implementation. In addition, the Bank has been actively engaged in projects in Armenia since the late 1990s, and it has acquired extensive knowledge of its challenges and opportunities to better assist the country.
2. COST-BENEFIT ANALYSIS
Main principles guiding the analysis
7. The benefit-to-cost analysis follows from an economic model that accounts for the cash flow generated by a person in his or her productive lifetime. Distinction is made between the different education levels (primary or less, completed secondary and completed tertiary education). The education premia (parameters of a Mincer equation) were estimated using the Integrated Living Conditions Survey (ILCS 2010). Using labor market statistics (Statistical Yearbook of Armenia 2011), the model was calibrated and the discounted expected cash flow generated by level of education and gender was estimated.14 This represents the present value of
12 Data from the Ministry of Education and Science for the academic year of 2011-2012 13 Bounded rationality is used when individuals’ decisions may be constrained by the availability of information, their own cognitive limitations and the time available to make the decision. An example is parents’ inability to see the expected benefit of pre-primary education, which may lead to low enrollment rates in that level. 14 The calibration of the model was done by using information from actual wages observed in Armenia to determine – using the parameters estimated in the Mincer equation – the wages for each age and education attainment. In particular, the actual wage of young (24 years old and under) university graduates was imputed as the “actual wage” of graduates in the model. The whole cash flow – for primary and secondary graduates as well – follows from the relationship between wages of each education and age groups found in the estimation of the Mincer equation.
56
all the income that an average person, with a given level of education and gender, would generate in her lifetime. The model accounts for the gender differences in education premium, participation and employment rates (between gender and levels of education) and unemployment benefits. Finally, a three percent yearly rate was used to discount the cash flows. In all components and subcomponents the benefits are defined as the difference between the gains under the project and what would be expected to happen in its absence (counterfactual). Table 1 shows the point estimates and the labor market indicators (labor force participation and employment) used in the analysis.
Table 1: Parameters for the estimation of discounted cash flow
Male Female Mincer equation (point estimates)
Return to Secondary education (relative to primary) 22.9% 29.1% Return to Tertiary education (relative to primary) 47.0% 75.9%
Labor Force Participation Primary or less 48.3% 35.2% Secondary 77.3% 58.5% Tertiary 84.9% 68.3%
Employment Primary or less 79.5% 88.1% Secondary 83.1% 77.3%
Tertiary 82.1% 79.0% Source: Labor Force Participation and Employment data come from Statistical Yearbook of Armenia 2011.
8. The analysis did not quantify the economic benefits of the subcomponent 1.3 “Improving data-collection and monitoring of the education system performance” and subcomponent 1.4 “Supporting further improvements in the quality of education through curriculum revision” due to difficulties in attributing measurable economic benefits to these subcomponents. While it is difficult to quantify the contribution of these interventions, it is clear that monitoring results is critical for evidence-based policy making. Also, curriculum revision is fundamental to ensure the content taught is relevant, up-to-date and follows international best practice. 9. It is important to note that the model employed in the analysis captures only part of the project’s benefits and therefore underestimates its potential benefits. For instance, preschool education is increasingly associated with better cognitive and non-cognitive skills that create better outcomes in a host of labor market and adult life indicators (better health, lower criminality, better informed voters, etc.). In the case of higher education, more relevant courses can shorten unemployment periods people’s lives. Also not included are potential increases in economic growth brought about by the project. Research by OECD (2010) has estimated the impact of improving academic quality (measured by results in international assessments) on countries’ future economic growth. According to the study, a 50 points increase in PISA scores (half standard deviation or one year and a quarter of education) is associated with 0.9 percentage points higher growth rates in the long-term (that is the impact would be felt over a period of 50 years).
57
Project’s expected net benefit
10. Given that the EIP addresses different levels and aspects of the education system, the economic and financial analysis developed below examines the costs and benefits of each component (or subcomponent) separately. 11. Table Error! Reference source not found.2 summarizes the figures for each component and subcomponent and presents the expected net benefit from the project.15 The table accounts not only for the project’s costs, but also for the long-term costs and fiscal effort Armenia would have to incur in order to sustain these investments.
Table 2: Education Improvement Project: Summary of Costs and Benefits (thousands of US$)
Component IDA/IBRD Financing
Total Economic
Cost
Total Economic
Benefit
Net Benefit
Benefit-to-Cost Ratio
Subcomponent 1.1 Promoting School Readiness and equal opportunities at the start of General Education
Subcomponent 1.3 Improving data-collection and monitoring of the education system performance
2,000 Not quantifiable
Subcomponent 1.4 Supporting Further Improvements in the Quality of Education through Curriculum Revision
500 Not quantifiable
Component 2 Mainstreaming of the Competitive Innovation Fund for Higher Education Institutions (HEI) into full implementation
5,000 5,032 17,780 12,748 3.5
Component 3 Project management, Monitoring and Evaluation 3,000 Not quantifiable
Total Education Improvement Project 30,000 62,149 201,055 138,906 3.2
12. The next sections present the details for each subcomponent as well as the sensitivity analysis performed. Expanding preschool education
13. EIP’s subcomponent 1.1 aims at increasing access and quality of preschool education. Activities under this component would focus on children who are five and six years old, with especial attention to vulnerable populations. The project’s goal is to raise the enrollment rate in preschool education in order to improve school readiness of children entering primary education. This fits well the government’s objective to raise the enrollment rate at this age from its current level of 75 percent to 90 percent by 2017. The number of children benefiting from this subcomponent is estimated to be 2,400 per year. A competitive grant scheme would be set up
15 The analysis is conducted using economic prices and it does not include about US$ 7.5 million for taxes.
58
and it is planned that 120 communities would be granted resources (total of US$ 1.5 million) to establish or upgrade preschool units. However, the resources would have to be coupled with government’s (or families’) resources to pay for service provision and sustain the investment. It is estimated that the sustainability effort would entail a fiscal effort of approximately 1 percent of the education budget.16 14. There is an accumulating body of evidence that shows that investment at an early stage is an excellent education policy.17 Leveling the human capital endowment at an early stage is more cost-effective than employing remedial strategies later on. In particular, it reduces the moral hazard issues that may arise when society chooses to redistribute resources to compensate those with poor labor market outcomes (Currie 2001). Research by Nobel laureate in Economics James Heckman praised early investment in disadvantaged children as “a rare public policy with no equity-efficiency tradeoff” as it promotes social justice and fairness by reducing inequality and at the same time raises society’s productivity. Both medical and economics literatures agree that gaps in cognitive and non-cognitive skills emerge early in life and persist over the lifecycle. Policies to mitigate these gaps later in life have proven to be less cost-effective. Table 3 presents a summary of early childhood development interventions and its measured outcomes.
Table 3: ECD interventions and their impact
Program Age range Outcomes Sources
Chicago Child-Parent Center and Expansion Program
Entry: 3-4 years old; exit: 9 years old
Increased high school completion; more years of schooling; reduced repetition rates; and higher college attendance.
Fuerst and Fuerst (1993); Reynold et al. (2002).
Perry Program Mean entry age: 4 years and 4 months
Benefits in education, criminality criminal activity; returns to investment are higher than other investment alternatives.
Heckman et al. (2010)
Head Start 3-5 years old Higher likelihood of completing high school and attending college; lower repetition rates; better academic performance; lower criminality rates
Currie and Thomas (1995); Garces et al. (2002)
Abecedarian Project 0-5 years old Increased future enrollment in higher education; improved exam results; and reduced pregnancy, use of drugs, incidence of smokers.
Campbell et al. (2002)
15. Drawing on this evidence, the economic model employed in this economic and financial analysis is based on the assumption that the expansion and quality enhancement of pre-primary education would lead to more years of schooling (less people with primary education or less and more tertiary education graduates) and lower repetition rates (as a consequence of increased school readiness). Children would be more likely to finish high school and progress to higher education, which in turn leads to higher productivity and income along the life-cycle and less unemployment.
16 Calculations based on the need to enroll approximately 6,000 children and on a budget of US$ 260 million. The calculations also assume that the government will have to finance the whole project (no families’ contributions). 17 Heckman and Masterov (2007), Cunha and Heckman (2006) and Carneiro and Heckman (2003) among others
59
16. The benefit of this subcomponent comes from the higher expected income generated by a more qualified and educated pool of people. The costs of the project include the capital cost of building and upgrading facilities to absorb the increasing demand for pre-primary education and the fiscal cost of service provision. The analysis considers a time span of 30 years and discounts the cash flow of the project in order to reach a benefit-to-cost ratio based on the expected benefits and costs accrued in the time frame considered. In line with previous research, the estimated benefit-to-cost ratio is 3.3 (with a cost of approximately US$38 million and a benefit of US$58 million, coming to a net benefit of around US$40 million, see Table 2). 17. In addition to the quantifiable effects described above, it is expected that there would be other positive effects (externalities) associated with increased preschool enrollment, such as reduced criminal activity, drugs usage, and teenage pregnancy, among others. Hence, it is likely that the figures presented underestimate the real impact of such intervention. Enrichment of upper secondary schools
18. In 2008, the government of Armenia approved the high school reform strategy. It is aimed at creating high quality learning environment for students in upper secondary (grades 10-12). The original plan was to reorganize 700 general schools (grades 1-12) into 550 basic schools (grades 1-9) and 150 high schools. The rationale behind the reform is that the new model brings in economies of scale by putting together more upper secondary pupils, and consequently, teachers. This leads to a more efficient use of facilities such as laboratories and libraries, and the possibility of offering different streams that specialize in mathematics/science or humanities. In principle, this would improve the learning environment and improved quality. 19. Eventually, 107 high schools were created. The new high schools have specialized streams in mathematics/science or humanities, new laboratories, libraries, advanced teaching and learning materials, and better-trained teachers. However, many of these high schools were in dire conditions (most were constructed between the 1950s and the early 1990s and likely have asbestos in the insulation of their roofs and piping; many of them have structurally damaged infrastructures, lack proper seismic stability and rely on sub-standard equipment). The EIP seeks to enhance the learning conditions in the approximately 17neediest high schools by rehabilitating old buildings that lack proper infrastructure and educational conditions. Through an investment of US$18 million, the proposed project aims at significantly improving teaching and learning conditions in these schools, in order to have an impact on students’ outcomes. The investment would follow a two-pronged approach: US$ 16 million would be invested in rehabilitating the approximately 17 neediest schools and US$2 million would be used to purchase modern ICT equipment and learning material for all 107 high schools. Besides improving the quality of teaching and learning, better infrastructure is also expected to reduce absenteeism and dropout rates, thus having a significant effect on the education system.18
18These schools face severe problems with heating systems (critical during the winter) and humidity. The poor conditions affect both teachers’ and students’ health and consequently academic performance through absenteeism and inability to concentrate during classes (which are not beneficial to the fullest extent even if students and teachers are present but can’t make the best of time on task due to the low temperatures).
60
20. The analysis of this subcomponent is based on the assumption that the renewed high schools, given their better infrastructure and increased access to modern and relevant learning material, would improve graduation rates and their students would go on to attain higher education. In addition, repetition and dropout rates would fall. The estimates assume that in these high schools, there are as many boys as girls, that the unit cost (per student) in the renewed high schools would be 20 percent higher than what is currently spent per student in grades 10-12, and that the average high school would serve 600 students per year. Moreover, calculations also consider maintenance expenses and depreciation. The benefits stem from the fact that these schools would turn out more tertiary education graduates, which would be more productive in the labor market and therefore earn higher income. Additionally, they would consume less unemployment benefits. The important underlying assumption is that due to the improved environment in the high schools, the graduation and transition to tertiary education would rise. Better educated people are more productive, earn more and spend less time unemployed during their lives. The analysis compares the expected discounted flow income of the beneficiaries under the project with what would be expected otherwise. The costs involve not only the capital costs of rehabilitation, but also the fiscal effort. The project would have a direct impact on over 10,000 students per year and, for the purpose of the analysis, a time span of 30 years is considered. The expected net benefit in 2013 values is approximately US$86 million with a benefit-to-cost ratio of 3.2 (total benefit of US$125 million and cost of US$39 million). 21. This subcomponent is likely to have an important impact on equity as 34 percent of students in high schools attend private tutoring in order to prepare for the University Entrance Examinations. Providing better learning conditions would level the field and enable students that can’t afford private tutors to compete for places in the universities. Competitive Innovation Fund
22. The CIF would provide grants for higher education institutions that submit proposals aiming at improving quality, promoting modernization and labor market relevance, and increasing efficiency, equity and access to academic programs (component 2). The project envisions providing US$5 million for grants between 2014 and 2017. The quantitative analysis focuses on the following expected benefits of this component: improvements in labor market relevance (measured by less time looking for the first job), and increases in efficiency (measured by the reduction in time taken to graduate and by reduced dropout rates). 23. The economic model, as described above, calculates beneficiaries’ expected cash flow and compares it with what would have been in the absence of the project. More labor market relevance reduces the time spent looking for the first job (which means more income per beneficiary and less expenditures in unemployment benefits). More efficient and relevant programs would reduce the amount of time the average student takes to graduate (less expenditure on higher education). Finally, reduced dropout rates – resulting from more relevance and increased student interest – would cause expenditures to go up (as students stay longer in the tertiary education system), but this would be outweighed as graduates of tertiary education generate more income than secondary education graduates. The costs include the grants that would be competitively awarded and the extra amount of time spent in school.
61
24. The analysis assumes there would be 2,500 beneficiaries per year between 2014 and 2024 (starting with 625 in 2014 beneficiaries and stabilizing at 2,500 by 2017), that the average decrease in time searching for the first job is of only one month and that student would take 1 semester less to graduate. In addition, it is assumed that there is perfect gender parity among the beneficiaries. Under these circumstances, the expected net benefit in 2013 values is approximately US$13 million with a benefit-to-cost ratio of 3.5 (cost of US$5 million and benefit of US$18 million). 25. It is important to note that this exercise does not account for some positive externalities. In the model, the beneficial impact of labor market relevance comes only as a consequence of less time searching for the first job. However, the project also envisions improving the quality of graduates. The productivity gain is not quantified and could be very significant. Moreover, if the project succeeds in targeting students whose families are in the lower income quintiles, the benefits are likely to be even higher as these students usually have fewer opportunities in the labor market than high income youth. Sensitivity analysis
26. Table 4 below provides the results of the sensitivity analysis. Besides the baseline (see parameters in (Table 1), the sensitivity analysis calculates the net benefit and benefit-to-cost ratio of each component under the following scenarios: (i) the impact of the project is half of the expected; (ii) returns to education are one standard deviation lower than the point estimate; (iii) returns to education are one standard deviation higher than the point estimate; (iv) discount rate is 5 percent; and (v) discount rate is 1 percent. Under scenarios (i) and (ii) the model was recalibrated in order to fit the earnings data.
Note: Half-sized improvements considers that the impact of the project on upper secondary graduation rates, transition to higher education, reduction in time searching for first job, and reduction in time taken to graduate in higher education institutions is half of the expected (baseline).
27. The sensitivity analysis performed confirms that the project is a good investment for Armenia.
62
References
Branham, D. (2004) “The Wise Man Builds His House Upon the Rock: Effects of Inadequate School Building Infrastructure and Attendance”, Social Science Quarterly, 85 (5), p. 1112-1128. Campbell, F. A., C. T. Ramey, E. Pungello, J. Sparling, and S. Miller-Johnson (2002) "Early Childhood Education: Young Adult Outcomes from the Abecedarian Project," Applied Developmental Science, 6(1), p. 42-57. Carneiro, P. and J. Heckman (2003) “Human Capital Policy.” National Bureau of Economic Research Working Paper 9495, Cambridge, MA. Cuyvers, K., G. De Weerd, S. Dupont, S. Mols, and C. Nuytten (2011) “Well-being at school: does infrastructure matter”, CELE Exchange 2011/10, OECD. Cunha, F. and J. J. Heckman (2006) “Investing in our young people”, Unpublished manuscript, University of Chicago, Department of Economics. Currie, J. (2001) “Early Childhood Education Programs” The Journal of Economic Perspectives, 15(2), p. 213-238. Currie, J. and D. Thomas (1995) “Does Head Start Make a Difference?”, The American Economic Review, 85(3), p. 341-364. Fuerst, J. S., and D. Fuerst. 1993. “Chicago Experience with an Early Childhood Program: The Special Case of the Child Parent Center Program”, Urban Education, 28, p. 69–96. Garces, E., D. Thomas and J. Currie (2002) “Longer-Term Effects of Head Start”, The American Economic Review, 92(4), p. 999-1012. Heckman, J. J., S.M. Hyeok, R. Pinto, P. Savelyev and A. Yavitz, Adam (2010) "A New Cost-Benefit and Rate of Return Analysis for the Perry Preschool Program: A Summary," IZA Policy Papers, 17, Institute for the Study of Labor (IZA). Heckman, J. J. and D. V. Masterov (2007). “The productivity argument for investing in young children”, Review of Agricultural Economics 29(3), p. 446–493. OECD (2010). “The High Cost of Low Educational Performance: The Long-Run Economic Impact of Improving PISA Outcomes”, Organization for Economic Co-operation and Development, Paris. Reynolds, A. J., J. A. Temple, D. L. Robertson, and E. A. Mann (2002) “Age 21 Cost-Benefit Analysis of the Title I Chicago Child-Parent Centers”, Educational Evaluation and Policy Analysis, 24(4), p. 267-303.
LL ee ss ss ee rr CC aa uu cc aa ss uu ss MM oo uu nn tt aa ii nn ss
ZZaa
nngg
ee zzuu
rrRR
aann
ggee
VV aa rr dd ee nn ii ss RR aa nn gg ee
L O R R IL O R R I
A R A G AA R A G AT S O T NT S O T N
A R A R AA R A R ATT
KO
TK
OTAA
YK
YK
’’
YEREVYEREVANANA R M AA R M AV I RV I R
S H I R A KS H I R A K
TTAAV U S HV U S H
G E G H A R KG E G H A R K ’’ U N I KU N I K ’’
S Y U N I KS Y U N I K ’’
VVAAY O T SY O T S ’’
D Z O RD Z O R
AlaverdiAlaverdi
DilijanDilijan
SevanSevan
AraratArarat
MegriMegri
GorisGoris
AngekhakotAngekhakot
VVaikaik
JerJermukmuk
VVardenisardenis
MarMartunituni
ArArtsvashentsvashenArArtiktik
TTashirashir
KarKarmirmir
StepanavanStepanavan
GyumriGyumri
IjevanIjevan
VVanadzoranadzor
ArArmavirmavir
HrazdanHrazdan
AshtarakAshtarakGavarGavar
ArArtashattashat
YYegegnadzoregegnadzor
KapanKapan
YEREVYEREVANAN
HHrraa
zzddaann
L O R R I
A R A G AT S O T N
A R A R AT
KO
TAY
K’
YEREVANA R M AV I R
S H I R A K
TAV U S H
G E G H A R K ’ U N I K ’
S Y U N I K ’
VAY O T S ’
D Z O R
Alaverdi
Dilijan
Sevan
Ararat
Megri
Goris
Angekhakot
Vaik
Jermuk
Vardenis
Martuni
ArtsvashenArtik
Tashir
Karmir
Stepanavan
Gyumri
Ijevan
Vanadzor
Armavir
Hrazdan
AshtarakGavar
Artashat
Yegegnadzor
Kapan
YEREVAN
AZERBAIJAN
GEORGIA GEORGIA
AZERBAIJANISLAMIC REPUBLIC
OF IRAN
TURKEY
Kura
Arpa
Arpa
Hra
zdan
Aras
Vor otan
Aras
Aras
Aras
LakeSevan
MingechevirReservoir
L e s s e r C a u c a s u s M o u n t a i n s
Za
ng
e zu
rR
an
ge
V a r d e n i s R a n g e
Aragats(4090 m)
ToNaxçivan
To Naxçivan
To Ordubad
To Füzili
ToQubadli
ToQubadli
ToGäncä
To Gäncä
To T’bilisiTo T’bilisi
To Borjomi
To Gäncä
ToKars
43°E
44°E
44°E
45°E
46°E 47°E
39°N
40°N 40°N
41°N 41°N
43°E
47°E
ARMENIA
This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.
