Final Evaluation Report - UNICEF · PDF filefinal evaluation report i evaluation of...

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual

Education in Viet Nam, 2006 – 2014. FINAL EVALUATION REPORT

i

EVALUATION OF UNICEF-SUPPORTED MOET’S INITIATIVE OF MOTHER TONGUE BASED BILINGUAL EDUCATION

IN VIET NAM 2006 - 2014 -

Final Evaluation Report

December 2015



ii

TABLE OF CONTENTS

ABBREVIATIONS AND ACRONYMS v

ACKNOWLEDGEMENTS 1

1.0 INTRODUCTION 8

2.0 EVALUATION, PURPOSE, OBJECTIVE AND SCOPE 9

2.1 Purpose 9

2.2 Evaluation Objectives 10

2.3 Scope 10

3.0 EVALUATION METHODOLOGY 11

3.1 Data Collection Methods and Sources 12

3.2 Data Analysis 13

3.3 Data Gaps and Limitations 13 3.3.1 Data quality assurance and sound analysis 14 3.3.2 Data gaps and limitations 14

3.4 Ethical Standards 16

3.5 The Theory of Change 17

4.0 FINDINGS 20

4.1 Relevance 20 4.1.1 Defining relevance 20 4.1.2 Summary regarding Relevance 20 4.1.3 Relevance to the Vietnamese - context 21 4.1.4 Relevance to international treaties and commitments 22 4.1.5 Relevance to the Vietnamese legal and policy frameworks 25 4.1.6 Role in improving educational policy regarding the ethnic minority people 28 4.1.7 Relevance to UNICEF´s renewed focus on equity 29

4.2 Effectiveness 30 4.2.1 Defining effectiveness 30 4.2.2 Summary on effectiveness 30 4.2.3 Effectiveness in creating conditions to improving the learning outcomes of participating children 31 4.2.4 Effectiveness in creating conditions to improving the capacity of teachers and education managers 35 4.2.5 Effectiveness in considering contextual factors in MTBBE design and implementation 38

4.3 Impact 41



iii

4.3.1 Defining impact 41 4.3.2 Summary on impacts 41 4.3.3 What impacts did the MTBBE approach have on the learning outcomes of participant students? 42 4.3.4 Impact on teachers, school management, board members and all level education managers 50 4.3.5 Impact on parents and the community 52

4.4 Efficiency 54 4.4.1 Defining efficiency 54 4.4.2 Summary on efficiency 54 4.4.3 Were activities cost-efficiency? 55 4.4.4 Were objectives achieved on time? 61 4.4.5 Was the programme or project implemented in the most efficient way compared to alternatives? 63

4.5 Sustainability 64 4.5.1 Defining sustainability 64 4.5.2 Summary on sustainability 64 4.5.3 Continuation of the programmes´ results in the AR MTBBE sites 67 4.5.4 Extent of capacity for scaling up in sites similar to the pilots 71 4.5.5 Elements in place/not in place for sustainability 73

5.0 LESSONS LEARNED 77

Identification of lessons learned 77

5.2 What could or should be done differently in future replications and/or scaling up? 78

6.0 CONCLUSIONS 79

7.0 RECOMMENDATIONS 83

REFERENCES 88

ANNEXES 92

Annex 1 – Terms of Reference 93

Annex 2 - Project Overview 109

Annex 3 – Evaluation Design Matrix 119

Annex 4 – List of People Consulted 139

Annex 5 - Additional tables and graphs regarding the effects of the Action Research MTBBE on students´ learning outcomes 145

Annex 6 - Selected testimonies on different issues 148

Annex 7 – AR MTBBE Detailed Financial Information 158

Annex 8 – Systematised list of benefits of the Action Research MTBBE 160

Annex 9 – Additional information for the SWOT 162

Annex 10 – Summary of findings, conclusions and recommendations 169

Annex 11 – Questionnaires 183



iv

LIST OF TABLES Table 1. Number of key informant interviews and focus groups, planned and actual .................................... 13 Table 2. Brief general information of the target provinces ................................................................................... 22 Table 4. SWOT analysis of implied factors on the sustainability of MTBBE scaling up .................................... 73

LIST OF FIGURES Figure 1. Percentage of MTBBE first cohort students according to grade and general scores ....................... 45 Figure 2. Percent of students by language dimensions and scores in MOTHER TONGUE. School year 2012-

2013, 1st and 2nd cohorts .................................................................................................................................... 45 Figure 3. Percent of students by language dimensions and scores in Vietnamese School year 2012-2013, 1st

and 2nd cohorts ..................................................................................................................................................... 46 Figure 4. Percent of fourth graders (First cohort) according to provinces and MTBBE / Non-MTBBE

students, by scores in Vietnamese ................................................................................................................... 48 Figure 5. Per cent of fourth graders (First cohort) by provinces and MTBBE / Non-MTBBE students, by

scores in Mathematics ........................................................................................................................................ 48 Figure 6. MTBBE Costs by component ....................................................................................................................... 58 Figure 7. MTBBE Costs 2008-2014 ............................................................................................................................... 60 Figure 8. Schematic interpretation of the main intervening factors in the development of the AR MTBBE in

the three participating provinces ..................................................................................................................... 70 Figure 9. Path of factors involved in the replica and scaling up of the MTBBE in areas with characteristics

similar to existing project sites ......................................................................................................................... 76



v

ABBREVIATIONS AND ACRONYMS AR MTBBE Action Research on Mother Tongue-Based Bilingual Education BE Bilingual Education BOET Bureau of Education and Training Department (MOET´s district

level) CEMA Committee for Ethnic Minorities DOET Provincial Education and Training Department (MOET´s provincial

level) EAPRO UNICEF´s East Asia and Pacific Regional Office ECED Early Childhood Education Department of MOET EMED Ethnic Minority Education Department of MOET EM Ethnic Minority EML Ethnic Minority Language (s) NIESAC National Institute of Educational Strategy and Curriculum (under MOET) MLE Multilingual Education MoU Memorandum of Understanding between the Ministry of

Education and Training and United Nations Children Fund Viet Nam of 15 August 2007, which defines the common understanding and operating modalities between the parties.

MOET Ministry of Education and Training MT Mother Tongue MTBBE Mother Tongue-Based Bilingual Education NIL National Institute of Linguistics



vi

PED Primary Education Department of MOET PID Project Initiation Document RCEME Research Centre for Ethic Minority Education, VIES SIL Summer Institute of Linguistics SWOT Strengths, Weaknesses, Opportunities and Threats Analysis ToR Terms of Reference UN United Nations UNESCO United Nations for Educational, Scientific and Cultural Organization UNEG United Nations Evaluation Group UNICEF United Nations Children's Fund VIES Viet Nam Institute for Educational Sciences1

1 This is the new name of the National Institute for Education Strategies and Curriculum

since 2009.



1

ACKNOWLEDGEMENTS

We thank to all relevant Government agencies, local and international organizations,

experts, specialist, managers, who actively engaged in the preparation, field work and

analysis of the evaluation. We express great appreciation to Viet Nam Ministry of

Education and Training, Departments of Education and Training in Lao Cai, Gia Lai and Tra

Vinh, Ethnic Council of National Assembly/, UNICEF Viet Nam and UNICEF Regional Office

for East Asia and the Pacific region for valuable guidance, inputs, technical supports,

expertise and financial support for this evaluation. In particular, our sincere gratitude goes

to MOET Vice Minister Nguyễn Vinh Hiển, Trần Thị Thắm- Deputy Director of Primary

Education Department, Prof. Nguyễn Văn Kha- Former General Director of VIES and his

team, Hà Đức Đà, RCEME director and his team, other entities within the MOET, managers

of DOETs, BOETs; Madam Trieu Thi Nai, Vice-Chairman of the Ethnic Council of the

National Assembly, and Nguyen Manh Quynh, Vice Director of the Ethnic Minority

Department of the National Assembly Office; Mr.Youssouf Abdel-Jelil, UNICEF Viet Nam

Representative, Jesper Moller-Deputy Representative. Thanks to the close technical and

management support by Joyce Patricia Bheeka- Education Chief, Vu Manh Hong-Chief,

Monitoring and Evaluation and Dinh Phuong Thao and Nguyen Thi Thanh Nga, Inclusive

Education Specialists, Nguyen Minh Nhat, Education Officer.

The Evaluation Reference Group was a valuable source of help with their comments on the Inception Report and the drafts of the Evaluation Report. In the same vein we recognize the UNICEF Regional Office for East Asia and the Pacific whose valuable feedback improved this Evaluation Report. Teachers, parents, community representatives, interpreters of the local languages deserve a special acknowledgement for their warm reception and readiness to answer our questions. Without them we could not have done this work. School children became the best part of our mission because their spontaneity, contagious joy and dreams tell us how much the MTBBE is transforming their lives as well as what to expect for the future of Viet Nam.



2

EXECUTIVE SUMMARY Viet Nam has 54 officially recognised ethnic groups, many of which have their own distinct language and live in remote and economically disadvantaged areas2. By 2009 the total ethnic minority (EM) population was about 12.5 million, or 14 percent of the total population3. The official language of instruction in schools is Vietnamese, which is used to teach all children, regardless of their level of knowledge of this language. This has created a “language barrier” for many EM children who have limited, or in some cases no, understanding of and/or proficiency in Vietnamese. In this context, two key issues facing education in Viet Nam are: 1) how to cope with the needs of many different language groups, and 2) how to fulfil the right to education for all children in order that they learn and develop to their full potential. The Action Research4 on Mother Tongue-Based Bilingual Education (AR MTBBE) has been an important initiative of UNICEF´s Education Programme during two Country Programmes of Cooperation 2006-2011 and 2012-2016. The AR has been implemented in the provinces of Lào Cai, Gia Lai and Trà Vinh, with the participation of students from three EM groups, i.e. Mong, J´rai and Khmer. The programme covered six years from pre-primary (one year) to primary education (five years) levels. This is a cooperation project between the Ministry of Education and Training (MOET) and UNICEF since school year 2008 with two cohorts of EM children. By the end of school year 2013-2014 (May 2014), the first cohort completed a six-year learning programme, followed by the second cohort by the end of the school year 2014-2015 in May 2015. The main objectives of the AR MTBBE are as follows: (i) To implement a valid and feasible design of bilingual education (BE) in ethnic minority languages (EML) and Vietnamese in pre-primary and primary schools; and (ii) To contribute to the development of policies and practices, including legal frameworks, that will promote the use and development of EML as a means to improve access, quality and equity of education and other social services. The aim of this final evaluation, which covers the period 2007-2014, is to make an objective assessment of the MTBBE approach implemented through the AR MTBBE to date and document evidence and lessons learned for replication and scaling-up in the coming years. This evaluation will serve MOET and UNICEF in fulfilling their mandate to “Successfully implement the AR MTBBE to inform the subsequent development of an appropriate and sustainable policy for BE in Viet Nam.” Other important users of the evaluation report are relevant policy-makers and decision-makers in MOET, CEMA,

2 The World Bank, 2009, page 51. 3 2009 Housing and Population Census of Viet Nam. 4 Action research is defined as observation, reflection, and planning activities which allow for improvement,

correction, documentation and evaluation of project activities, in this case, what is happening in the selected

schools.



3

National Assembly, subnational authorities, education managers, teachers, UN agencies and various development partners. The methodology for this evaluation was developed in accordance with the quality criteria set out in the context of the Organization for Economic Co-operation and Development (OECD) Development Assistance Committee’s (DAC) Quality Standards for Development Evaluation5, in line with the United Nations Evaluation Group’s (UNEG) Standards6 and Norms7, and with special reference to the provisions of the Global Evaluation Reports Oversight System8 (GEROS).

The methodology was grounded in the Terms of Reference (ToR) and the approved Inception Report, where the latter included a detailed Evaluation Design Matrix. In accordance with UNEG Standard 3.10, paragraph 21, the approved Inception Report constitutes the extent of the mutually agreed upon work programme. The Evaluation follows a mixed methods approach, which created opportunities for the stakeholders’ participation in providing insights about pertinent issues. This methodology used multiple lines of evidence that yielded robust information for the analysis. The evaluation started in October 2014 and ended in August 2015. Fieldwork for data collection took place during the second half of November 2014, including a validation workshop. Main Findings Relevance The MTBBE approach is relevant and responsive to the socioeconomic, legal and policy contexts of Viet Nam and UNICEF´s focus on equity. MTBBE translates the Vietnamese legal and policy frameworks into a more inclusive and equitable education system, as expressed in the 2013 Constitution, revised 2009 Education Law, Education Strategic Development Plan (2011-2020), among other relevant laws and policy frameworks. From the cultural point of view, the relevance of the programme is closely linked to its potential to contribute to the preservation and development of ethnic minority languages (EML) and cultural practices and beliefs. Despite the attested enabling implementing environment, a review of the Vietnamese key legal and policy instruments suggests the need for revisiting and clearly strengthening the existing policies and laws towards MTB-MLE.

5 http://www.oecd.org/development/evaluation/qualitystandards.pdf 6 http://www.uneval.org/document/detail/22 7 http://www.uneval.org/document/detail/21 8 http://www.unicef.org/evaluation/index_60830.html



4

Effectiveness The programme has been effective in developing and implementing a context-appropriate MTBBE model that has created conditions to improve the learning outcomes of participating EM children and the professional capacity of teachers and education managers. By offering the programme from pre-school to the end of primary education and adopting a flexible approach, the curriculum design allowed the development of both the students’ mother tongues and Vietnamese right from the first day of schooling as well as offered optimal conditions for the transfer of knowledge from one language to the other. Nevertheless, there were reports of misapplication of flexible teaching methodologies, as well as some hesitation about the teaching of Vietnamese as a second language, in particular in earlier stages of the programme. Impact The MTBBE has produced factual and perceived impacts on its intended beneficiaries. According to different internal/official assessments and evaluations, children’s learning outcomes have been at a high level in all schools of the three provinces, as compared with national standards. In addition, children displayed confidence, self-esteem and good communication skills. Teachers, school management, board members and all levels of education managers generally supported the programme and recognized its effect on improving their competences to successfully apply the MTBBE design and contributed to develop policies and practices promoting the use of EMLs. Parents showed awareness and support to the MTBBE and recognized its benefits in the life of their children.

Efficiency Evidence provided by MOET and UNICEF indicated that monetary and time resources were used in an economically acceptable manner. Stakeholders expressed awareness of the cost vs. results relationship, and most think that the unit costs of the programme seemed high but compensated by its returns (opportunity costs), particularly its immediate impact on learning outcomes and teacher training, as well as its contribution to capacity building for policy making. Sustainability The AR MTBBE has created favourable basic conditions for future replication and scaling up of the bilingual programme. In addition the socio-political climate is also supportive of the scaling up of the MTBBE programme to new schools and new ethnic minority groups in Viet Nam.



5

However, for sustainable scaling up of the MTBBE programme, following adjustment of the relevant laws, including Education Law and sub-laws is needed: EMLs can be used as medium of teaching and learning, together with supportive policies and permanent dialogue between central and provincial governments. Gender and Equity The MTBBE approach has also proved its relevance to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. By helping ethnic minority children overcome the language barrier in school, MTBBE was regarded as one of the platforms that can be used to improve the academic performance of these children and set the ground for official use of EML in other formal settings, including in the health and legal sectors. As a part of the equity efforts, the MTBBE has contributed to good participation of girl students. The evidence indicates girls had as good academic performance as boys. Overall Conclusion The main conclusion of this evaluation is that the AR MTBBE has proved to be a valid and feasible education approach for the context of Viet Nam, as evidence showed relevance, effectiveness, impact, efficiency, sustainability and as a contribution to achieve education equity. Evidence gathered from the three pilot sites indicated that the programme had a substantial positive effect on students, teachers, education managers, parents and beneficiary communities, thus contributing to improve the learning outcomes of the participating ethnic minority children. The success of the AR MTBBE suggested that EM groups in Viet Nam may be better served and included in social life if EMLs are officially used in other formal arenas, including in the health and legal sectors. Main Recommendations The recommendations are based on the findings and conclusions of this evaluation. These recommendations are intended to inform the design and implementation of the on-going and future MTBBE programmes in Viet Nam. They are chiefly directed to MOET and its relevant departments,. A few recommendations are also directed to UNICEF, as the promotor and the main supporter of the implementation and management of the AR MTBBE, and to other development partners. The recommendations were developed in close consultation with key stakeholders, including UNICEF, MOET and local level education authorities. They are not listed in any priority order, but thematically organized based on the evaluation criteria. ,

1. Develop a Project Initiation Document (PID), including a theory of change, for the next

programme delivery, clearly defining performance indicators to be used in the

management and assessment of its overall success. This should be part of any

programme/project initiative.

2. Review the existing Vietnamese legal and policy frameworks with the view of

strengthening them clearly towards MTBBE.



6

3. Show strong political commitments to introduce mother tongue learning by establishing

a specific policy on how the MTBBE approach should be implemented in Viet Nam.

4. In the current context of curriculum and general education reform in Viet Nam,

incorporate MTBBE as one of the solutions to improve quality and equity for ethnic

minority children in this country.

5. Conduct careful sociolinguistic studies before implementing the MTBBE approach in new

EM groups in Viet Nam as this may give indications about the relevance and the likelihood

of success of the programme in those settings.

6. Develop a strategic advocacy/communication plan to ensure that relevant institutions

issue policies/guidance on the use of EML in other official arenas, including in basic service

delivery sectors such as health, child protection and legal.

7. In the scaling up phase, replicate the design and the flexible approach adopted during

the pilot stage, as they yielded a strong form of bilingual education, which is

recommended for the socio-cultural context of Viet Nam. This design should form part of

the future policy on MTBBE in Viet Nam.

8. As part of the preparation of the scaling up phase, organize a seminar/workshop aimed

at analyzing and reviewing the teaching-learning methodologies used in the MTBBE to

recommend adjustments to make them more responsive and appropriately applied.

9. Adopt multilingual-oriented second language methodologies in the teaching and

learning of Vietnamese as part of the MTBBE approach. This may include the involvement

of NGOs promoting the teaching and learning of Vietnamese as a second language in Viet

Nam (e.g. Plan International) in training workshops and/or continuing professional

development programmes for MTBBE teachers.

10. Conduct (longitudinal) studies of former MTBBE students’ performance at secondary

school in order to assess the educational impact of MTBBE beyond primary education

level. The results of those studies will provide further context-specific evidence about the

feasibility and validity of the MTBBE for ethnic minority students in Viet Nam.

11. Develop a Behaviour Change Communication Strategy (campaign) to raise awareness

of parents and communities about the purpose and benefits of the MTBBE and encourage

their support and participation, as well as to mobilize collective action in support of the

programme.

12. Scale up the MTBBE programme, involving more schools and EM students, thus

maximizing the academic and sociocultural benefits of the approach and reducing its costs

by capitalizing on economies of scale.



7

13. Consolidate all bilingual reference materials (i.e. teaching guides, grammars,

dictionaries, additional reading and reference books, and visual aids) in the list of basic

materials to be produced and supplied to MTBBE schools.

14. Mobilize independent writers and publishers to produce low-cost and/or no-cost

reference materials in EML for MTBBE schools; this may include award offers and

promotion of contests and competitions culminating in the publication of the best

manuscripts, including on-line publication.

15. Develop a strategy for a gradual expansion and scaling up of the MTBBE programme.

This should have two goals: (a) the continuity and dissemination of MTBBE in the three

pilot provinces and (b) the design and implementation of new MTBBE interventions in

provinces with characteristics similar to those of the provinces involved in the pilot stage

or in other provinces fitting MOET’s criteria.

16. Continue to provide financial support for teacher training and development of teaching

and learning materials to the experimental provinces for three to five years while they will

continue to find other donors to sustain the programme.

17. Provide support to teachers for them to effectively deliver a multi-language policy by

ensuring pre- and in-service training which lead to the improvement of their own

proficiency in the language of instruction and in teaching skills in ethnic minority

languages, including early literacy development in these languages.

18. Establish and maintain a dialogue with relevant provincial authorities in order to clarify

responsibilities and find solutions for the financial needs of local governments to sustain

current MTBBE efforts and results and set the bases for the expansion and scaling up of

the programme in those provinces.

19. Bring language of instruction to the forefront of dialogue on education sector funding

with the government partners, particularly in relation to both quality and equity concerns,

ensuring the coverage of MTBBE is highlighted as international indicator of education

quality.

20. Mobilize resources to support the introduction of MTBBE and offer technical assistant

to develop flexible competency targets, literacy materials and an assessment framework in

multiple languages.



8

1.0 INTRODUCTION9 The Action Research on Mother Tongue-Based Bilingual Education (AR MTBBE) was implemented by the Ministry of Education and Training (MOET) in collaboration with UNICEF Viet Nam in Lào Cai, Gia Lai and Trà Vinh provinces with three ethnic minority languages, namely, Mong, J´rai, and Khmer. MTBBE was one of the solutions to fulfil the Government of Viet Nam’s commitment to improve the quality of education for ethnic minorities (EM). The AR MTBBE had two objectives: (i) To implement a valid and feasible design of bilingual education in ethnic minority languages (EMLs) and Vietnamese in pre-school and primary schools; and (ii) to contribute to the development of policies and practices, including legal frameworks that will promote the use and development of EMLs as a means for improving access, quality and equity of education and other social services. MOET and UNICEF were key institutional stakeholders of the AR MTBBE. Within MOET, the pilot involves different specialized departments, i.e. Viet Nam Institute for Educational Sciences (VIES), Research Centre for Ethnic Minority Education (RCEME which was part of VIES and assigned as focal point of the pilot implementation), Early Childhood Education Department (ECED), Primary Education Department (PED) and Ethnic Minority Education Department (EMED). MOET´s Vice-Minister, UNICEF´s Deputy Representative and the Directors of MOET´s departments form the Steering Committee. In addition, the Advisory Group included representatives of the ECE, PED, EMED, RCEME, NIESAC, CEMA and NIL, as well as senior staff of UNICEF, UNESCO and The World Bank. At the local level MOET is represented by Provincial Research Management Boards and District Action Research Management Boards, both adding the representation of provincial and district education managers, school principals and teacher associations. During the implementation, the AR MTBBE continuously received technical support from well known international experts such as Carol Benson, Margaret Franken, Andy Smart, Susan and Denis Malone and other experts from the Summer Institute of Linguistics (SIL), as well as the best national experts of the Viet Nam Linguistic Institute for the design, curriculum and materials development, training of master teachers´, Action Research planning and operation, learning outcome assessment, and Primary Classroom Language Mapping. The Ethnic Council of the National Assembly, was another important partner in promoting the MTBBE approach and policy. Also CEMA was recognised as having an important role in policy development. 9 For a more detailed project overview, See Annex 2



9

The specific responsibilities of UNICEF in the AR MTBBE have been: To provide support to MOET in the overall implementation and management of the

Action Research, through technical assistance and/or contracting of international specialists, as per the Research Overall Design.

To ensure that the initial and on-going design, technical aspects, and overall quality of the AR are based on sound practices according to inputs from international specialists working with national counterparts.

To liaise with and report to interested donors to support the raising of financial resources for the implementation of the AR activities throughout the entire Research cycle.

To provide support to RCEME in ensuring participation of relevant institutes and organizations such as NIL, CEMA, Teachers´ Colleges and Pedagogical departments and universities, and in relevant AR activities.

2.0 EVALUATION, PURPOSE, OBJECTIVE AND SCOPE

2.1 Purpose10 The main purpose of the evaluation is to generate substantive evidence and knowledge to inform national policies on mother tongue bilingual education and the subsequent reduction of education inequities in Viet Nam as well as strengthen the sub-national commitment in implementing the MTBBE approach. The main users of the evaluation are relevant policy-makers and decision-makers in the MOET, CEMA, National Assembly, subnational authorities, education managers, teachers, UN agencies and various development partners. The first MTBBE student cohort entered pre-primary school in 2008. By May 2015 the second cohort finished the five grades of the primary level, which marked the end of the AR MTBBE. UNICEF- MOET ´s Education Programme Annual Work plan 2014 anticipated an independent Evaluation. This evaluation was designed to make an objective assessment of the MTBBE approach implemented through the AR MTBBE to date and document evidence and lessons learned for replication and scaling-up in the coming years. This evaluation would also serve MOET and UNICEF to inform the subsequent development of an appropriate and sustainable policy for bilingual education in Viet Nam. By supporting organizational learning and accountability, this evaluation was expected to help related agencies continually to improve their performance and results. In addition to that this evaluation would support planning and decision-making, and provide a basis for informing relevant education policies that promote the well-being of all children. In sum, this evaluation served to improve results and stakeholder satisfaction.

10 ToR, 2013, p.2-3



10

2.2 Evaluation Objectives The Terms of Reference (ToR) stated the following objectives:

To assess the extent to which the MTBBE approach has been relevant, effective, and efficient and has contributed to changes at the outcome and impact levels, including the contribution to the better learning outcomes of the participating children (first cohort and second cohort). The evaluation will also seek to determine whether the MTBBE approach is sustainable.

To analyse whether the results obtained through the overall strategy of implementing the MTBBE approach at both the national and sub-national levels have contributed to the reduction of education inequities and to the fulfilment of children´s rights to education in Viet Nam.

To identify lessons learnt on design and implementation and to provide recommendations for on-going and future MTBBE in Viet Nam.

To recommend actions to advocate for supportive policies on the use of mother tongues in bilingual education for the improvement of quality of education in ethnic minority areas.

To identify good practices, innovative interventions and shortcomings (in terms of implementation of MTBBE including inclusiveness and effectiveness, child friendly learning environment, participation of students, parents and the community people), positive effect on individual children, teachers, parents, community leaders and local authorities, etc.

2.3 Scope The evaluation covered the period 2007-2014, as per the signed Memorandum of Understanding (MoU) on 15 August 2007 between MOET and UNICEF Viet Nam. The pilot phase involved two cohorts of students – the first cohort completed their studies in 2014 and the second in May 2015. The evaluation was guided by evaluation criteria and key questions as set out in the ToR, and later refined and detailed in the approved Inception Report. The evaluation covered all criteria and questions in accordance with the available information. The third OECD DAC criterion, “impact” is understood as the “final results of a programme or policy on the intended beneficiaries and, where possible, on comparison groups. This criterion reflects the cumulative effect of donor-supported programmes of cooperation and national policy initiatives.11 In this regard the evaluators triangulated a mix of evidence based on facts and perceptions. The criterion on “sustainability”, particularly on the question of scaling up in areas with characteristics similar to existing project sites, became particularly challenging because the team had to cope with incomplete or contradictory information on key issues, such as the

11 UNICEF, 2013, page 10.



11

number and the geographical distribution of different ethnic populations, the extent to which they are alone or mixed with others, and what linguistic situations emerge from this distribution.

3.0 EVALUATION METHODOLOGY

The methodology was developed in accordance with the quality criteria set out in the context of the Organization for Economic Co-operation and Development (OECD) Development Assistance Committee’s (DAC) Quality Standards for Development Evaluation12, in line with the United Nations Evaluation Group’s (UNEG) Standards13 and Norms14, and with special reference to the provisions of the Global Evaluation Reports Oversight System15 (GEROS).

In particular, the methodology was grounded in the Terms of Reference (TOR) and the approved Inception Report, where the latter included a detailed Evaluation Design Matrix (see Annex 2). In accordance with UNEG Standard 3.10, paragraph 2116, the approved Inception Report constitutes the extent of the mutually agreed upon work program.

The evaluation was structured in hierarchical sequence by the evaluation criteria (i.e. relevance, effectiveness, impact, efficiency, and sustainability), and within these by the corresponding evaluation questions and sub-questions. These “were then analysed by the types of evaluation question (descriptive, normative, cause-effect), quantitative and qualitative measures or indicators, the type of evaluation design (experimental; quasi-experimental and non-experimental), the multiple line of evidence, also referred to as data sources (documents, interviews, focus groups, site visits, etc.), approaches to data collection (e.g. sample or census), data collection instruments (e.g. questionnaires), and notes on approaches to data analysis. The original questions and the sub-questions were included in the Annexes. The key questions are as follows:

To what extent is the MTBBE approach relevant to the socioeconomic, legal and policy contexts of Viet Nam and UNICEF´s renewed focus on equity?

12 http://www.oecd.org/development/evaluation/qualitystandards.pdf 13 http://www.uneval.org/document/detail/22 14 http://www.uneval.org/document/detail/21 15 http://www.unicef.org/evaluation/index_60830.html 16 UNEG Standard 3.10 (paragraph 21) states: “The responsibilities of the parties who agree to conduct an

evaluation (specifying what, how, by whom and when what is to be done) should be set forth in a written

agreement in order to obligate the contracting parties to fulfill all the agreed upon conditions, or if not, to

renegotiate the agreement. Agreements, such as the Terms of Reference, should be established at least in the

follow areas: financing, time frame, persons involved, reports to be produced or published, content,

methodology, and procedures to be followed. Such an agreement reduces the likelihood that

misunderstandings will arise between the contracting parties and makes it easier to resolve them if they arise.

Providing an inception report at the start of the evaluation is a useful way of formalizing such an agreement

and ensuring proper interpretation of the Terms of Reference.”



12

How effective is the MTBBE approach to improving both the learning outcomes of participating children, and the capacity of teachers and education managers?

What impacts did the MTBBE approach have on four stakeholders groups: (A) MTBBE students; (B) Teachers, school managements, board members; (C) Parents and members of the community; and (D) Managers from grassroots to central levels?

Does the actual implementation of the approach use resources in the most efficient (economical) manner to achieve expected results?

To what extent should the benefits of the MTTBE approach be continued after the donor funding ceases?

3.1 Data Collection Methods and Sources To meet the challenge of the evaluation questions it was necessary to gather reliable and valid quantitative and qualitative data. To this end the evaluation team followed a mixed methods approach that created opportunities for the stakeholder participation in providing insights about pertinent issues. This methodology used multiple lines of evidence that yielded robust evidence. The evaluation started in October 2014 and ended in August 2015. Fieldwork for data collection took place during the second half of November 2014, including a validation workshop with the Steering Committee members. UNICEF Viet Nam staff assisted the evaluation team with travel logistic and arranging interview appointments in the three provinces where the AR MTBBE schools are located, as well as interviews and other research activities in Ha Noi. The evaluators reviewed 277 documents of background information previously sent by UNICEF Viet Nam. This information was complemented by other documents collected in the field. The other methods of data collection were face-to face in-depth interviews with key informants and focus groups with children, some provincial and district educational authorities and school managers, as well as teachers and parents. The evaluation team interviewed 62 key informants at the central, provincial and district levels, and conducted 22 focus groups in which 163 teachers, parents and children participated.17 The sample of key informant covered the entire range of informants at the central level (education authorities, other high level governmental authorities and donors). At the local level these interviews covered educational authorities of the three provinces, seven districts, and seven schools participating in the AR MTBBE. Participants in the focus groups included the majority of currently active MTBBE teachers, and a moderate number of parents and students. The sample for the focus groups had a gender-balanced representation. See Table 1 and Annex 4.

17 The original planned number of focus groups participants was 112.



13

Table 1. Number of key informant interviews and focus groups, planned and actual

Key-informant interviewee category Estimated # in

Target Population Actual # of Interviews

Key informant interviews with MOET, other central level managers and representatives of the donor community

18 22

Key informant interviews with DOET and other provincial and local authorities

30 26

Key informant interviews with school principals

7 6

Key informant interviews with UNICEF management

3 8

Total interviewees 58 62

Focus groups category Actual # of Focus Groups

Teachers 8

Parents 8

MTBBE students 4

Students of the mainstream system 2

Total number of sessions 22

Total number of participants 163

For evidence regarding the Impact section (Section 4.3), the evaluation team used

secondary data provided by MOET, which had been used in the preparation of the

“Programme Brief” and the Annual Student Assessments; both publications had been

produced by MOET with technical support from UNICEF.

3.2 Data Analysis Evidence from documents, key informant interviews and focus groups was entered into NVivo for reviewing and coding in accordance with the questions contained in the Evaluation Design Matrix. To ensure the quality of the analysis, triangulation and validation of findings were undertaken from all sources including the documents, interviews and site visits. No conclusions were based on only one opinion. An initial internal peer review process involved UNICEF Viet Nam and UNICEF´s East Asia and Pacific Regional Office, and later Central Level stakeholders.

3.3 Data Gaps and Limitations



14

3.3.1 Data quality assurance and sound analysis The Inception Report warned about different challenges that the evaluators might find in assuring data quality (reliability and validity) and sound analysis. To meet such challenges particular measures were taken as described below:

Data were collected from representatives of all key stakeholders.

Multiple lines of evidence were used including secondary sources (files, documents, databases, specialized literature) and primary sources (individual and group interviews, focus groups).

The questions contained in the interview questionnaires and guides addressed all evaluation criteria, questions and indicators as presented in the Evaluation Design Matrix. Each line of evidence included program staff, recipients, and stakeholders.

Information was carefully handled by intensive use of NVivo software for systematization/coding and analysis. Facts were carefully distinguished from opinions or perceptions.

All evaluators participated in entering the data and, in turn, validating the analysis by means of collegial oversight/review of the information.

UNICEF Viet Nam, UNICEF´s EAPRO and the Evaluation Reference Group also participated in reviewing different drafts of the evaluation report.

3.3.2 Data gaps and limitations

A weak base for generating performance indicators. The AR MTBBE was undertaken on the basis of the MoU and the AR Implementation Framework. There was no proper Project Initiation Document (PID), which usually defines the project to form the basis for its management and the assessment of overall success. The PID normally contains the program goal, purposes, justification, management and roles, inputs, and an execution schedule. The PID is key to develop an adequate and precise monitoring system based on performance indicators. Without a PID, performance indicators were hard to identify, thus evaluators had to review the MoU and its Annexes for this purpose. These two documents offered clues for generating performance indicators but they lacked precision in quantifying most of the specified activities. A project monitoring system, which could be a simple Excel control table, easily shows the key performance indicators throughout the execution period. Usually evaluators receive this information to assess how close each indicator is from the target. Evaluators also verified this information in the field by observing, for instance, whether the textbooks were supplied to a school in the quantity specified in the monitoring system. In the case of this evaluation, the Research Team did not find evidence that such a monitoring system was operating; therefore, they concluded that the monitoring of the AR MTBBE was focused on learning indicators rather than other performance indicators (e.g. number of training workshops; developing, printing and



15

delivery of teaching and learning material), where the latter were probably registered as a part of MOET´s general inventory control system rather than as a separate one. Secondary data used for key assessments on impact. The key data of this evaluation were those related to learning outcomes. In this regard, the evaluation team and its UNICEF counterpart agreed that primary data collection regarding students´ learning outcomes would be unnecessary because MOET already had data resulting from previous annual assessments. The main results of these evaluations had been already published by MOET, with the support of UNICEF, in the "Programme Brief" and “Annual Assessment Reports”. It was expected that the evaluation team should consider this information to be available for use as the basis of Section 4.3.3 ("What impact did the MTBBE approach have on participant learning outcomes of students?"). This information was considered to be primary information in the case of MOET´s assessment reports, but it would be managed as secondary information for the case of the present evaluation. Because MOET had twice hired two different agencies to collect and analyse data related to student learning outcomes, some inconsistencies in the assessment methodology could be expected. MOET provided a copy of Excel and some SPSS (2013-2014) databases. This set of information did not allow the evaluation team to rebuild a complete matrix or matrices of data regarding all students, all indicators and all years by means of which to process whatever cross-tabulation was needed. In some cases the information was already grouped by schools or provinces; in other cases some indicators were available for some years but not for all. In short, most of the information available for the evaluation team had been previously processed and analysed. On this basis the evaluation team prepared Section 4.3.3. As is common with secondary information, the evaluation team first assumed the data were reliable and valid since there was evidence that (the data) had been collected and analysed with appropriate methodological rigor18. The evaluation team did not review the methodology used in the collection of data and organization of the database, although it remained attentive to detect any inconsistency. In fact, the evaluation team found virtually no evidence of inconsistencies in the data up to becoming an obstacle to (i) assess general academic performance of each of the two student cohorts during the entire primary school period; or (ii) comparing these results by province. The evaluation team only encountered a problem trying to compare the performance of students participating in the program with non-participating students throughout the period (primary school years). The data analysis and preparation of the evaluation report was scheduled to take place

18 MOET had standards for the assessment of student learning outcomes (testing criteria,

methods and protocols for the different subjects). UNITEC also hired international

consultants to prepare manuals about the oral and written versions of Vietnamese Placement

Test for primary school. The results of these tests were monitored by the consultant Steve

Walter to guarantee that regional variation in the use of Vietnamese was properly

considered.



16

before Christmas 2014. Therefore, the evaluation team had little time to wait for additional information to be sent after the team's visit to Viet Nam. In addition, data on impact as well as that related to the other evaluation criteria were triangulated with other informational sources. MOET´s data on student learning outcomes proved to be consistent with testimonial information obtained from the interviewees (education managers, teachers, students, parents). Time was a constraint for more in-depth observation of the learning environment. The evaluation team visited three participating provinces according to an agenda of interviews, which was more intense than anticipated. This generated an opportunity to interview authorities, teachers, children and parents. However, time was very short to go deeply into the different issues. For example, it might have been interesting to observe the development of some classes to get a first-hand impression of the teaching-learning process. As is known, this is a time-consuming activity that was not planned because of its cost implications. The selection of participating children without discrimination and bias was important; however, this was out of reach in the case of this evaluation. Two baseline surveys had been conducted at the beginning of the school year 2006-2007 and at the beginning of 2007 to ensure selected criteria were well communicated to local authorities, schools, teachers and parents. The evaluation team conducted discussions with parents and members of local communities in their EM language to ensure active participation. Annex 2 of the MoU presents the criteria for selecting the target pre-primary and primary schools. Those criteria expressed preference for small schools; pre-primary schools should be included in or located close to a primary school; schools should be linguistically homogenous, i.e., close to 100% students should speak the target EM language; the schools should have competent and qualified teachers in the EM language as well as in Vietnamese (preferable), among other similar criteria. It was also important to procure classes where the number of boys and girls was almost the same. Besides these criteria no others had been expressed, for instance criteria to prevent subtle or open discrimination against EM children in a particular physical or psychological situation. The evaluators were not informed by local interviewees about episodes of discrimination. According to the evaluators´ experience with similar programmes in other developing countries, discrimination against special students, when detected, represented only isolated cases. The detection of this problem becomes difficult with the type of data collection tools usually used in evaluations where the detection of discrimination and bias against participant children is not a priority, given the time and cost that most appropriate research tools would imply.

3.4 Ethical Standards



17

UNICEF Ethical Principles and Premises of the Evaluation19, UNICEF Ethical and Principle Guidelines for the Reporting on children and young people under 18 years old20, as well as UNEG Norms and standards for the protection of confidentiality, dignity, rights, respect of the values of the visited communities, and child welfare (friendly environment and talks) were applied throughout the entire processes of primary data gathering, analysing and report writing. In particular these involved those on Gender and Human Rights, including child rights. Interviews were held on an informed and voluntary basis. In most cases, interviewees permitted the recording of the sessions to facilitate the note-taking process. Testimonial material had been used in a confidential manner to protect the anonymity of respondents. Ethnic minority children in selected schools, including MTBBE and non-MTBBE students, participated in six focus group sessions, which were previously authorized by the respective school authorities, thanks to UNICEF Viet Nam and MOET recommendations. In preparation for these meetings the evaluation team participated in a special session in Hanoi where UNICEF´s staff provided information on protocols to be followed to respect child rights.

3.5 The Theory of Change This section addresses the following evaluation question: What was the theory of change of the intervention, as summarized or indicated in a reconstructed results chain through which inputs are translated into successive higher levels of results? The theory of change is summarized in the results chain shown below. The results chain displays the relation of the inputs to expected outputs, outcomes and the long term results. The displayed results chain is an ex-post interpretation undertaken by the evaluation team; it was designed after a careful review of the MoU and the AR implementation framework 2007-2014. It represents the reconstructed rationale of the intervention, which is implicit rather than explicit. Draft versions of the results chain were modified according to the comments received from reviewers of this Report. The AR implementation framework shows results-oriented planning, but without the guide of an explicit Theory of Change. Evaluators also noticed the lack of a PID for the MTBBE, which would have mainly described the goal and purposes, justification, management and roles, inputs, and execution schedule. This explained why the MoU between MOET and UNICEF dated 15 August 2007 became very important for grasping the project´s rationale. The lack of a PID became an obstacle to the evaluability of the project given the value

19 See ToR, Annex 5 20 See ToR, Annex 6



18

which such a document has to contrast the plan versus success, as well as to generate performance indicators. The Inception Report identified as a challenge the fact that “different stakeholders may have different perspectives on key programme performance measures”. A suggested solution was: The “Evaluators will (re) construct the Theory of Change, Logic Model and Performance Indicators that support the programme to compare with the stakeholders perspectives”.21 The theory of change facilitates a quick general assessment of the performance, although the lack of the PID became a real obstacle to generate specific performance indicators, particularly those of a quantitative nature. Considering this limitation and whenever pertinent, findings reported in the next section were linked to the results chain, particularly regarding both the effectiveness and impact evaluation criteria. As illustrated by the reconstructed results chain shown below, the final expected result of the AR MTBBE would be the contribution to the development of an applicable and sustainable policy on bilingual education. This policy had to be based on both the existing legal framework and evidence on the advantages of the MTBBE approach. For this general purpose it was necessary to implement the AR MTBBE in some carefully chosen provinces and schools, where two cohorts of EM students had to attain learning outcomes according-to-national-standards learning. This would require (i) trained educational managers and teachers, (ii) all-level (local, provincial, national) stakeholders´ support, and (iii) attention from MOET and policy-makers to adopt the MTBBE approach as one solution to improve the quality of EME. That final result and outcomes would be attained after overcoming several challenges, such as the lack of a specific MTBBE curriculum according to MOET´s standards, appropriate learning and teaching materials, training opportunities for teachers and education managers, and orientation for policy-makers. Also, monitoring, evaluation and reporting activities, based on action-research had to be organised to achieve quality and effectiveness control. Inputs in the form of both funding (UNICEF, donors, MOET) and inter-institutional close coordination and management had to be guaranteed as triggering condition of the proposed programme.

21 Revision 2 20141031, page 38.

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014. FINAL EVALUATION REPORT

19



20

4.0 FINDINGS

4.1 Relevance

4.1.1 Defining relevance The OECD Network on Development Evaluation defines relevance as “the extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.”22 Based on this definition, a project is relevant when

• It captures congruently planners´ vision and beneficiaries´ perception of their needs;

• It responds to changing and emerging development priorities and needs in a responsive manner;

• It gets acceptance and capacity to operate within the local context. In evaluating relevance, the ToR included questions to assess the extent the AR MTBBE had considered national priorities as emerging from the socioeconomic, legal and policy contexts of Viet Nam. It also included the extent to which the initiative aligns with UNICEF´s renewed focus on equity.

4.1.2 Summary regarding Relevance The general question on relevance was: To what extent is the MTBBE approach relevant to the socioeconomic, legal and policy contexts of Viet Nam and UNICEF´s renewed focus on equity? Finding No.1. The MTBBE approach is relevant and responsive to the socioeconomic, legal and policy context of Viet Nam and UNICEF´s focus on equity

MTBBE has proved to be relevant and responsive to the Vietnamese context at both national and subnational levels. MTBBE translates the Vietnamese legal and policy framework for a more inclusive and equitable educational system, as expressed in the Constitution, Education Law, Education Strategic Development Plan (2011-2020), among other relevant laws and policy frameworks. From the cultural point of view, the relevance of the programme is closely linked to its potential to contribute to the preservation and development of ethnic minority languages and cultural practices and beliefs. Despite the attested enabling implementing environment, a review of Vietnamese key legal and policy instruments suggests the need of revisiting/ strengthening the existing policies and laws clearly towards MTB-MLE. The MTBBE approach has proven to be relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. By helping ethnic minority children to overcome the language barrier in school, MTBBE can be regarded as one of the platforms that can be used to improve the academic performance of these children and lay the ground for official use of EML in other formal settings, including in the health and legal sectors.

22 OECD Network on Development Evaluation, n.d., page 13.



21

A relevant approach?

To the socioeconomic context?

To international treaties and commitments?

To legal and policy frameworks?

To UNICEF´s renewed focus on equity?

There is a need to strengthen legal and policy frameworks clearly towards MTBBE

Sections 4.1.3 to 4.1.5 responded to each of the sub-questions on relevance by providing the summary evidence that substantiates Finding No.1.

4.1.3 Relevance to the Vietnamese - context

The first sub-question regarding relevance was: To what extent is the MTBBE approach relevant and responsive to the Vietnamese context and needs at both national and sub-national level?

Language and poverty are key factors in explaining the educational gap between the Kinh majority and children from ethnic minorities. Language is also a barrier for the social inclusion of the ethnic minority children. As soon as MTBBE is a solution to overcome this language barrier and its negative effects on equity, social capital and social cohesion, it becomes relevant in the context of Viet Nam.

Several structural barriers hamper upward social mobility of ethnic minority groups. In the realm of education, poverty and language23 have been identified as paramount problems. Most school aged children from ethnic minorities are put at a disadvantage when Vietnamese is the only medium of instruction in schools. As abundant scientific evidence has shown, students who do not learn their mother tongue proficiently have greater difficulty learning another language well.24 Within this context, MTBBE becomes relevant because it offered one promising solution for overcoming the language barrier that affects most ethnic minority children. The goal of recent educational reforms is to increase the learning potential of all children, which is very difficult to attain when poverty, languages and other major obstacles exist. Viet Nam needs a better-educated population to sustain and revitalize its rapid economic growth.

23 In the case of poverty Johnson and Theberge, 2007; and Votruba-Drzal, 2006 are good references. In

the case of language, see Snowling and Hulme, 2011; Dutcher, 2004; Bransford, Brown, & and Cocking,

1999. Snowling and Hume (2011) state that “Individual differences in language development affect the

course of learning to read… Children with broader oral language weaknesses (particularly weaknesses in

vocabulary knowledge and grammatical skills) are at risk of reading comprehension difficulties (this

includes children with a poor command of English because it is not their native tongue)”. 24 Dutcher, 2004, p. 10.



22

At the subnational (provincial, local) level, the greater relative importance of some ethnic groups must be considered. For instance, in Lào Cai province ethnic minority groups represent about two thirds (64%) of the population, in Gia Lai 61% and in Trà Vinh 32%. The AR MTBBE´s participant ethnic groups form the bulk of the ethnic minority groups in those provinces, i.e. J´rai 34%, Khmer 30% and Mong 22% of the total population. [See Table 2]. Hence, by creating conditions for improving access to basic quality education, MTBBE might greatly empower provinces with an important share of ethnic minority people to be better positioned in contributing to and taking advantage of economic growth as well as the relatively increasing offer of social services. This comment considers that MTBBE becomes relevant in communities with relatively homogenous EM populations. s This means that one EM language is really predominant as the home language of almost all school-age children. Despite the fact that there is no disaggregated data on the % of each ethnic groups in the province

Table 2. Brief general information of the target provinces

General information Province

Gia Lai Lào Cai Trà Vinh

Region Central Highlands Northwest Mekong Delta

(South)

Land area (km2) 15,536.9 6,383.9 2,215

Population (million) 1.3 0.66 1.0

Density (Persons/Km2)

86 103 461

Rural population (%) 70 78 84

Ethnic groups 39 including Kinh, J´rai 34%, Ba Na 14%, others: Gie Trieng , Xo Dang, Co Ho, Nhang, Thai, Muong

26 including Kinh 36%, Mong 22%, Tay 16%, Dao 14%, Giay 5%, others: Phu La, San Chay, Na Nhi and the La Chi

3 including Kinh 68%, Khmer 30%, Hoa 2%

Source: General Statistics Office of Viet Nam. Statistical Yearbook of Viet Nam 2013. Density data is a rough estimation.

Language is not only a barrier in education, but it is also a cause of social exclusion, like mothers not attending health awareness campaigns or having limited aspirations of upward mobility in the labour market. Language also plays a role in strengthening social cohesion in the community and society, including mobility, inclusion and social capital.

4.1.4 Relevance to international treaties and commitments The second sub-question on relevance was: To what extent is the MTBBE approach relevant to international treaties and commitments to which Viet Nam already signed up?



23

International agreements encourage governments to observe cultural and linguistic dimensions as crucial components of human rights. The approach is relevant because its design considers and builds on these agreements by fostering MTBBE as one of the best practices which many countries have adopted to express deference for cultural diversity and recognition of its pedagogical value to develop the full potential of children from ethnic minorities.

Viet Nam appears in the list of countries that have signed international ratification, accession or succession treaties and conventions, such as the UNESCO Convention against Discrimination in Education (1960), the International Convention on the Elimination of all Forms of Racial Discrimination (1965), the International Convention on Economic, Social and Cultural Rights of 1966, the United Nations Convention on the Rights of the Child of 198925. This body of international agreements encourages governments to observe cultural and linguistic dimensions as crucial components of human rights. This way, governments become encouraged to avoid discrimination against other cultures, e.g. ethnic minority groups. Education is a field in which open or discreet (indirect) discrimination may act against ethnic minorities. In this case international legislation has been clear regarding the right of all children to access compulsory and free primary education and has opened the door to allow that “children living in exceptionally difficult situations” have special, positive treatment, such as provided in affirmative action policies. A difficult situation is not only to live in a conflict area or have physical impairments; there are countries in which children of ethnic minorities live also under structurally difficult conditions because of poverty and social exclusion. Recognizing this fact, an increasing number of countries have been introducing MTBBE. By doing so these countries show respect to cultural diversity while making use of tested pedagogical methods that develop the full potential of children from ethnic minorities and minimize school absenteeism and dropout. The MTBBE approach is relevant regarding this matter because its design considers and builds on these international laws. The UNESCO Convention against Discrimination in Education (CADE) and The Convention on the Rights of the Child (CRC) are two examples of pertinent international laws to which Viet Nam has subscribed. The CADE, which entered into force in 1962, recalls the Universal Declaration of Human Rights, particularly the principle of non-discrimination and the proclamation that every person has the right to education.

25 Minority Rights Group and UNICEF, 2009, pages 222-233; Human Rights Library of the University of

Minnesota.



24

The CRC entered into force in 1990. The Convention recognized that, “in all countries in the world, there are children living in exceptionally difficult conditions, and that such children need special consideration”. In those States in which ethnic, religious or linguistic minorities or persons of indigenous origin exist, a child belonging to such a minority or who is indigenous shall not be denied the right, in community with other members of his or her group, to enjoy his or her own culture, to profess and practice his or her own religion, or to use his or her own language (Article 30).



25

4.1.5 Relevance to the Vietnamese legal and policy frameworks The third sub-question on relevance was: To what extent is the MTBBE approach relevant to Constitution, Education Law, and Law on Child Care, Protection and Education, national policies, and others?

“MTBBE operationalizes the body of legislation and educational policies addressed to benefit ethnic minority children; it shows a path to go beyond words”.26 The Government of Viet Nam has tested several approaches in the past to guarantee ethnic minorities´ access to education while respecting their rights to use their own language and preserve their cultures. However, the results have not met the expectations. The failure greatly responds to the fact that mother tongue has not been in the role of language of learning, a practice supported by some articles of the educational legislation. MTBBE challenges the assumptions behind this legislation, which leaves little room for the scaling up of this programme.

Relevance to the Constitution The AR MTBBE was highly relevant to the Constitution because it shows a way to honour cultural diversity in schools where this diversity is a problem (instead of an opportunity) for learning. The MTBBE approach promised that a second language can be learned well when a child first becomes literate and fluent in his/her own language. Article 5 of the new Constitution of Viet Nam (2013) clearly expresses that although Vietnamese is the national language:

every nationality has the right to use its own language and system of writing, to preserve its national identity, and to promote its fine customs, habits, traditions and culture;

the State implements a policy of comprehensive development, and provides conditions for the national minorities to promote their internal abilities and to develop together with the nation.

Relevance to legislation on education The Vietnamese constitution, education laws and many education policy documents have strongly supported the use of ethnic minority languages (EML) in education. These documents gave equal support to the learning of Vietnamese by ethnic minority people, thus emphasizing bilingual education as a good strategy for ethnic minority learners. However, they are not explicit about the use of EML as languages/media of teaching and learning in formal education.

26 High level official from MOET.



26

Article 7 of the Education Law (Law No.38/2005/QH11) points out, that Vietnamese is the official language to be used in schools and other educational institutions. It also prescribes that “The State shall enable ethnic minority people to learn their spoken and written languages in order to preserve and develop their ethnic cultural identity, helping pupils from ethnic minorities easily to absorb knowledge when they study in schools and other educational institutions. The teaching and learning of these languages shall be conducted in accordance with the Government regulations”. The revised Education Law 200927 maintained the same idea, thus repeating that the teaching and learning of EML should be in line with Government´s regulations. Governmental Decree 82 of July 2010 (82/2010/ND-CP) regulates “the teaching and learning of EMLs including the conditions, content, methods and forms of teaching and learning: teacher training programmes, policies for teachers and learners for the teaching and learning of EMLs”. Article 6 clearly indicates that EMLs are to be taught as a subject at all education levels and categories (paragraph 1). In sum, EMLs were taught in school only as subjects. So far MTBBE was implemented only on an experimental basis. For expansion, MTBBE would require either a revision of legal framework in the education or authorities tolerating de facto situations, where the second is not a recommendable alternative. Legislators have a legitimate concern with the importance that Vietnamese has for people´s full social inclusion and socioeconomic mobility. The Education Law was based on the assumption that learning Vietnamese at early age is beneficial for ethnic minority children. However, the clear learning gaps between the Kinh majority and most ethnic minority (EM) children suggests that this assumption is partially inaccurate. EM children are not learning Vietnamese sufficiently well as to guarantee the expected benefits at school and in society as a whole. As shown by international experience, and by the evaluation of the AR MTBBE, Vietnamese can be learned better if EM children first develop appropriate levels of oral competency and literacy in their own languages.

Relevance to the Vietnamese policy framework The policy reference for MTBBE comes mainly from central and sectorial plans ratified by the Government. The most important policies, valid for the period of the AR MTBBE, are the National Education for All (EFA) Action Plan 2003-2015, the Socio-Economic Development Strategy (2006-2010 and 2011-2020), the Education Strategic Development Plan (2006-2010 and 2011-2020), the Resolution No.29 (November 2013) of the Central Steering Committee of the Communist Party of Viet Nam on comprehensive reform with socialist orientation and global integration, and the Prime Minister´s Decision on Approval of the School Education Textbook and Curriculum Innovation Scheme (27 March 2015).

27 Law Amending and supplementing a number of articles of the Education Law. The National Assembly

No.44/2009/QH12, 25 November 2009.



27

The EFA Action Plan 2003-2015, in its updated version of 2012, recommends the design of an inclusive development strategy that merges all relevant short-term measures, such as the extension of the role of special schools, expansion of training in inclusive teaching methods and the provision of additional materials and resources for disadvantaged children (EFA, 2012, page 8). Under the framework of this strategy, the EFA Action Plan highlights the relevance of MTBBE and other approaches28. For EFA, MTBBE is a “promising” option to, for example,

improve learning outcomes of ethnic minority pupils, and as a condition for the success of this effort, the promotion of teachers´ training and support and offer a

preventive strategy against school repetition and dropout.29 Other documents include those promoting inclusive education measures addressed to disadvantaged populations. For instance, (i) The Socio-Economic Development Strategy (2011-2020) and the Education Strategic Development Plan (2011-2020); (ii) The Socio-economic Development Strategy, dated 10/29/2012 and approved during the 6th Conference of the Communist Party; and, (iii) The Resolution No.29-NQ/TW, dated 11/04/2013, on “fundamental and comprehensive innovation in education serving industrialization and modernization in a socialist-oriented market economy during international integration”, ratified in the 8th session of the Central Steering Committee of the Communist Party of Viet Nam. The most recent Prime Minister´s Decision on Approval of the School Education Textbook and Curriculum Innovation Scheme (No.404/QD-TTg, 27 March 2015) operationalizes the aforementioned Resolution No.29 (November 2013). This way the new scheme proposes, for the entire education system (Grades 1 to 12), curriculum and textbook development aligned with the education reform. The new curriculum combines knowledge and practice as well as competences and student-centred teaching methodologies, including special treatment to meet the needs of the most vulnerable children. Budget provisions allocate public resources for developing and piloting the curriculum and compiling and piloting a set of textbooks, including bilingual textbooks (Vietnamese and EMLs). For all these documents, education plays a leading role for which it has to experience a comprehensive renovation. In this sense, it is necessary to consolidate the universalization of primary education and pursue the universalization of secondary education and the pre-school. A part of these efforts is improving the quality of education and the development of policies focused on promoting education in the most disadvantaged regions to reduce inequality in education access.

28 The source mentions five approaches, i.e. PEDC or Vietnamese reading model, JICA´s “Lesson Study”,

Teacher Action Research which is a Belgian Project, Escuela Nueva model in Viet Nam supported by the

World Bank, and finally, a group integrated by different projects supported by Oxfam UK, Enfant et

Développement, Save the Children and UNICEF “which provide a more professional approach” (EFA

Programme, 2012, page 32). Escuela Nueva, Save the Children and UNICEF projects use mother tongue

in the learning process. 29 Education for All Action Plan, 2012, pages 35-36.



28

In sum, the Vietnamese policy framework sees education in a prominent role for development and creates a favourable environment for reform and focus on the most disadvantaged regions. This creates good will and guarantees resources for new curricula and textbook approaches. However, it remains a challenge to legislate on the use of EMLs as language of learning beyond an experimental basis of the MTBBE approach.

4.1.6 Role in improving educational policy regarding the ethnic minority people A fourth sub-question on relevance was: What role has MTBBE played in terms of improving educational policy with respect to the ethnic minority people of the country?

MTBBE has played an outstanding role in improving educational policy on bilingual education for ethnic minority people. It has implemented an MTBBE design to show the advantages of this approach, and it has become a benchmark practice inspiring innovation in educational policy. MTBBE has gained the attention and receptivity of educational authorities and policy makers toward bilingual education, which is key to enlarge the legal space this approach needs for its expansion.

MTBBE was organized to play a key role for improving educational policy related to ethnic minority people of Viet Nam. First, the AR MTBBE has implemented a valid and feasible design of bilingual education in EMLs and Vietnamese in pre-school and primary school. The design was a reference experience to show the advantages of the approach in terms of learning outcomes. The experience included training and advocacy addressed to policymakers, education managers, principals, teachers, students and communities to attain their understanding and support to MTBEE. Simultaneously great efforts were made on developing teaching, learning, reference and advocacy materials in EMLs. Second, the AR MTBBE contributed to the development of policies and practices for promoting the use and development of EMLs. The next sections on effectiveness and impact will examine evidence in this regard. Some outstanding results of this commitment were the following:

• In 2010 the provincial authorities of Lào Cai decided to scale up the MTBBE approach. It was a provincial initiative with the potential to include other ethnic minority communities (see Annex 4, Box 2). It showed an open-minded attitude of education authorities to the new approach.

• The AR MTBBE gained momentum and the attention of the National Assembly Office (see details in Section 4.2.6).

• In 2012 a Primary Classroom Language Mapping exercise was implemented in

Lào Cai. This was an important information tool for policy makers and sustainability of MTBBE that MOET and UNICEF agreed to produce30 upon the request of the Lào

30 2011 Annual Work Plan.



29

Cai Department of Education and Training and Primary Education Department (MOET). The initiative focused on generating more accurate information on language use and identifying locations where the MTBBE can be replicated. Beyond this focus, the study was considered useful for supporting the development of a standard Language Mapping tool for Viet Nam and for contributing to the development of the education system, making it more relevant and matching the multilingual nature of Viet Nam.31 . Lào Cai has been utilizing the findings from this tool for education sector planning. The province of Dien Bien also utilised this tool to conduct their language mapping. In addition The World Bank is now utilizing this tool for its supported project in East Timor.

• The 2012 updated version of the EFA Action Plan 2003-2015 recognized the potential of the MTBBE approach to respond to educational needs of ethnic minority groups, and recommends that strategies, such as SEDS 2011-2020 and ESDP 2011-2020 for the period 2011 to 2020, consider the implementation of an inclusive development strategy for the benefit of disadvantaged children.32 A component of this strategy would be the delivering of mother tongue-based bilingual or multilingual education and its scaling up to other provinces.33 For some testimonies on this issue see Annex 6, Box 3 and Box 8.

4.1.7 Relevance to UNICEF´s renewed focus on equity The fifth sub-question on relevance is: To what extent is the MTBBE approach relevant to UNICEF´s renewed focus on equity?

The MTBBE approach is specifically focused on ethnic minority children, which are the most deprived and vulnerable children in Viet Nam. MTBBE contributes to remove the language barrier that is considered one of the main causes of low learning outcomes, school absenteeism and dropout. During adult life this educational disadvantage turns into other social disadvantages in the labour market, in the access to and use of services, and others. Given this commitment, the MTBBE approach is strongly relevant to UNICEF´s renewed focus on equity, by which it tries both to reach the most deprived and vulnerable children and remove barriers to service provisions, particularly in the case of the poor and marginalised34.

Since ethnic minority children face a language barrier to learning at school in Viet Nam and in other spheres (labour market, business, social services), actions to overcome this barrier become relevant to achieve equity. For UNICEF´s focus on equity, AR MTBBE is a key project, particularly as a vehicle to promote inclusive policies on education and other social spheres. The following is a testimony on this perspective:

31 MOET, SIL International and UNICEF, 2012, page 5. 32 EFA, 2012, page 8. 33 EFA, 2012, page 36. 34 UNICEF, 2010, page 1



30

For UNICEF equity is a global level issue, not country specific. Viet Nam has a different context. In this country there is a good investment in education. The Government has developed a universal primary education. However, about 4% of the target population is outside the education system. These are children from ethnic minority groups, children with disabilities, children who are out of school due to socio-economic reasons and children of immigrants, very poor people. This is why there is a focus on inclusive education with broad meaning of a functional education system that would include all children regardless of their background, ethnicity or development status. The MTBBE approach was part of this effort together with inclusive education for children with disabilities. A direct work with poor children or migrant children will be the next step of the UNICEF education programme in the future.35

In sum, The MTBBE approach offered a pertinent solution to overcome the language barrier in an alignment to international, national and sectorial legal frameworks as well as policy and development strategies. However, the existing policies and laws in Viet Nam are lacking clear and explicit statements about the use of EML as media of teaching and learning, which would facilitate the expansion of MTBBE.

4.2 Effectiveness

4.2.1 Defining effectiveness

The OECD defines effectiveness as “a measure of the extent to which an aid activity attains its objectives”. Two questions are typical in this regard; these are:

To what extent were the objectives achieved or likely to be achieved?

What were the major factors influencing the achievement or non-achievement of the objectives?36

4.2.2 Summary on effectiveness

The general question on effectiveness was: How effective was the MTBBE approach to improving both the learning outcomes of participating children and the capacity of teachers and education managers? Were contextual factors taken into account in the approach design/ implementation of interventions? Finding No.2. The Programme has been effective in developing and implementing a context–appropriate MTBBE model that has created conditions to improve the learning outcomes of participating children and the capacity of teachers and education managers.

The effectiveness criterion makes reference to the attainment of the following Programme objectives: (i) implementing BE models in EML at pre-school and primary levels, and (ii) contributing to policy making capacity in education at local and central levels. Turning those objectives into reality has required inputs and actions regarding curriculum design, teaching and learning materials development, and capacity/ improvement of teachers and education management, to mention the most important activities. National standards were met in the case of curriculum and educational materials. By offering the programme from

35 UNICEF Official from the Education Section 36 OECD Network on Development Evaluation, n.d. page 13



31

pre-school to the end of primary education and adopting a flexible approach, the curriculum design allowed the development of both the students’ mother tongues and Vietnamese right from the first day of schooling as well as offered optimal conditions for transfer of knowledge from one language to the other. The in-service training of teachers has developed/ improved mastery of innovative BE teaching methods and self-learning skills, as well as action research competences. Educational managers were involved in activities oriented to improve their EML knowledge, comprehension and support, and in this way, increase their potential for policy making at different levels. The AR MTBBE considered contextual factors at both stages, i.e. design and implementation. In meeting challenges from structural factors the MTBBE had an effective advocacy component. Right from the preparation phase and throughout the pilot phase, there were awareness campaigns and lobbying initiatives chiefly targeting parents, communities, politicians, policy makers, donor agencies and NGOs. As a result of these advocacy measures, key stakeholders got acquainted with the basic principles of MTBBE and/or understood the relevance of this approach to the Vietnamese context. The attention and support devoted to the programme by participating communities, key policy makers and other members of the public can be regarded, at least in part, as a result of the effectiveness of the advocacy component of the AR MTBBE.

Effective?

To improving learning outcomes?

To improving capacities of teachers and education managers

To consider contextual factors in the design/implementation of interventions

Sections 4.2.3 to 4.2.5 respond to each one of the sub-questions on effectiveness by providing the summary evidence that substantiates Finding No.2.

4.2.3 Effectiveness in creating conditions to improving the learning outcomes of participating children The first sub-question on effectiveness was: How effective was the MTBBE approach to improving the learning outcomes of participating children?

The MTBBE approach created the necessary conditions to improving the learning outcomes of participating children. The MTBBE model, with variants for each of the three participating MT, was implemented and assessed as a result of developing both curriculum and teaching and learning materials according to national standards.

The MTBBE model was implemented mainly by the following actions:

Designing of the MTBBE pilot Action Research

Curriculum development according to MOET standards

Developing supporting material according to MOET standards. There are other actions oriented to the implementation of the MTBBE models, e.g. training of teachers and education managers, which are being considered under the second sub-question on effectiveness (Section 4.2.4).

MTBBE models: Have they been successfully implemented and assessed?



32

There was no question that the AR MTBBE model, with its three variants, was implemented and there was also good evidence about actions on monitoring and assessment. The MTTBE went through two stages, a first one on designing and a second one on implementation. The designing stage started in 2006 while the implementation stage commenced in 2008 in selected pre-primary and in 2009 in primary schools in Gia Lai, Trà Vinh and Lào Cai provinces with three EM languages: J´rai, Khmer and Mong.37 In preparation of the implementation stage, several actions on curriculum design and teaching and learning material development were carried out. As planned, the first cohort of students entered pre-school in September 2008 in eight selected schools of the above mentioned provinces. During the next school year the second cohort of students entered pre-school while the first was ready to start First Grade. By June 2014, the first student cohort was finishing primary school (the Sixth Grade), this way completing the period designed for the AR MTBBE programme. As expected, the second student cohort also finished Sixth Grade in 2015. The AR MTBBE anticipated actions to guarantee the models would be under permanent monitoring and assessment. These actions were the responsibility of technical staff and education managers at central, provincial and district levels. Students´ progress was assessed annually, which provided valuable data about the impact of the programme on learning outcomes.38 The first student assessment was carried out in September 2008 collecting baseline data for the first cohort of pre-primary students. The assessment focused on five development components in both the mother tongue and Vietnamese39. A second assessment, carried out in May 2009 replicated the preceding study to find that “significant progress in all five components” had occurred. Teachers and learning material were also objects of this last assessment. The assessment also extended to a sample of non-MTBBE students since 2011. As a result, valuable information was collected and used to improve textbooks, provide insights for teacher training, and strengthening the teaching-learning process.40This participative monitoring process turned into a routine activity of the Primary Education Department in collaboration with the RCEME. These two entities were aware of the importance the annual assessments had as a

37 MoU 15 August 2007, page 1. 38 See for instance, UNICEF-MOET. The Programme Brief 3 of September 2012, page 12. 39 The following were the five assessment components in mother tongues: (a) listening comprehension of

spoken language; (b) understanding simple concepts; (c) understanding antonyms; (d) arranging pictures

and story-telling according to pictures; and (e) recognizing mother tongue alphabets. The five components

in Vietnamese were: (a) listening comprehension of spoken language; (b) knowing the names and

functions of parts of the body, objects, trees and animals; (c) implementing commands; (d) naming

people/animals and their activities; and (e) arranging pictures in sequence (MOET-UNICEF, 2009, pages

4-5). 40 MOET-UNICEF, 2009, pages 6-7.



33

tool to demonstrate the importance of transitioning from mother tongue to Vietnamese as well as to control the quality and effectiveness of the programme. Starting with the Assessment Report 2012-2013, the objective of setting up a database for the project became manifest41. As observed in the different assessment reports, each indicator was constructed according to methodological procedures commonly used by education specialists around the world. Programme actions attained the purpose of having the AR MTBBE models operating as planned. How successful these models were in improving MTBBE students´ learning outcomes is a question to be discussed in Section 4.3, which corresponds to the impact criterion. Curriculum development: Did it follow MOET standards?

The MTBBE programme was based on a curriculum design, developed by national and international experts and in line with national standards. MOET´s adoption of this design and the fact that it yielded students with appropriate competences for their grade level indicate its success in terms of meeting national standards. By offering the programme from pre-school to the end of primary education and adopting a flexible approach, the curriculum design allowed the development of both the students´ mother tongues and Vietnamese right from the first day of schooling as well as offering optimal conditions for a smooth transfer of knowledge from one language to the other. Curriculum development was one key initial activity of the AR MTBBE. To lead this process a Bilingual Curriculum Design Team was formed during the last quarter of 2007, which comprised key national technical experts42. In supporting this team, UNICEF contracted an international specialist to work directly with the Viet Nam counterpart and Carol Benson, the main international advisor in the design of the Action Research MTBBE. The core task of the Bilingual Curriculum Design Team was to develop a bilingual curriculum framework based on the national standards for the EML. That is, at the end of grade 5 students must meet the national standards. In fact, the current Vietnamese standards would be the EML standards, and the Vietnamese standards should represent appropriate levels for second language learners.43 The Bilingual Curriculum was approved in April 2008. The Curriculum Framework identified eight (8) “Strands” and their respective “sub-strands” for the Mother Tongue and Vietnamese. Each strand was then detailed for pre-primary and each grade of the primary

41 Report of the Survey Results of Grade 3 and 4 on Learning Outcomes of MTBBE Students. Academic Year

2012-2013. Hanoi, 2014, page 3. 42 The team was led by the Research Centre for Ethnic Minority Education and benefitted from the

participation of about 30 persons at central and provincial levels. 43 MoU 15 August 2007, Annex 3: Research Design, page 4.



34

level. The strands were the following: Listening and speaking, reading, writing, phonology and orthographic conventions, vocabulary, grammar, text types and text forms.44 In 2011 UNICEF hired another international expert to refine the MTBBE preschool and primary curriculum for three subjects, i.e. Mathematics, Vietnamese and Mother Tongues. Additionally, the expert provided clearer and more specific indicators for the monitoring of students´ progress. It was expected that this exercise would allow the consultant to support capacity building of MOET experts to ensure the transfer of skills and knowledge necessary for the curriculum revision. In this way the MTBBE curriculum was updated and legitimated in its alignment with the competencies required by MOET.45

Teaching and learning materials: Do they meet MOET standards? Developing teaching and learning materials followed the MOET standards since its Research Centre for Ethnic Minority Education (RCEME) led this process directly. UNICEF contracted Andy Smart, international expert in bilingual material development, to support the national team at the beginning, including the training and development of a material development strategy. The strategy for the development of teaching and learning materials was flexible, experimental and grounded in the curriculum framework. In the development of the supporting material, language aspects were considered, such as: (i) Narrative and imaginative, (ii) conceptual, (iii) phonological, and (iv) social and/or communicative. Based on those aspects RCEME proposed a minimum requirement of materials to be supplied by the project.46 By May 2013, 543 copies had been printed; almost 60 copies for preschool students and almost 20 copies to train teachers. About 2,900 copies of textbooks for all primary grades in subjects like Mother Tongue (textbook and a workbook for writing), Vietnamese, Mathematics, Social Sciences, Natural Sciences, History, and Geography were printed and delivered.47 RCEME instructed the developers of these materials to base their work on general principles, such as, culturally adapted, close to children´s experience, consistently integrating different learning levels, and without complicated concepts.

44 M. Franken (July) 2008. Curriculum Framework (July 5).

45 ToR. UNICEF. Consultancy Notice Edu 2015-01NOTICE EDU 2011-01 46 These materials consisted of “big books” (10 textbooks series per language), picture books (10 textbook

series per language), wall charts (10-12 textbook series per language), teacher´s methodology guides (one

series per language), oral literature resource books (one series only in mother tongues), Jigsaw puzzles,

and letter cards and picture cards (flash cards). 47 MOET. Annual Progress Report 2013.



35

In sum, the MTBBE approach implemented AR MTBBE models for the participating children of the three ethnic minority populations. Regarding this purpose it was necessary to develop curriculum and teaching and learning materials meeting national standards.

4.2.4 Effectiveness in creating conditions to improving the capacity of teachers and education managers The second sub-question on effectiveness is: How effective is the MTBBE approach to improving the capacity of teachers and education managers?

The MTBBE approach created the necessary conditions to improving the capacity of teachers and education managers. Teacher training has provided competences based on updated BE teaching and research methods. There has been a retraining offer at different levels with a problem-solving orientation. Education managers have received training oriented to facilitate policy-making on education at different levels, particularly at provincial and local levels.

Did MTBBE create conditions to improving the capacity of teachers? By means of different training activities the MTBBE opened opportunities for teachers in attaining or improving key professional competences like the mastering of (i) the mother tongue and the familiarity with its respective culture, (ii) innovative language-teaching and student-centred approaches, and (iii) action research methodologies.

First actions in training

At the inception stage of the MTBBE a two-track approach had to be employed to support teacher development:

1. A rapid training programme on bilingual education for pre-primary and First Grade teachers, as well as education managers to successfully participate in the pilot rollout of the first year.

2. Long term pre-service and in-service training programmes on bilingual education (BE) for current and future teachers and education managers, in cooperation with teacher training colleges and pedagogical departments at universities.

RCEME took the charge of establishing and supervising a Teacher Training Committee composed of key national technical experts, while UNICEF contracted international specialists to support the MOET´s work on teacher development. Budget figures registered about 564 teachers trained during the project period.

Mastering MT and familiarity with local culture

Since the start of the training process, experts advised on the need to prepare highly competent teachers in the MT and high familiarity with the respective culture. This meant that most teachers had to be ethnic minority teachers because they had the highest



36

probability to share the same linguistic and cultural background with the student and have the national teacher training system which implies an acceptable mastery of Vietnamese48. This advice was surely based on internationally well-known information and best practice gathered from successful MTBBE and multilingual education (MLE).

Mastering language-teaching and student-centred approaches

Teacher training has also emphasized another internationally well-known best practice of MTBBE and MLE, which is the mastery of innovative language teaching and general pedagogic approaches. Most interviewed teachers and educational managers reported participation in Total Physical Response training (language teaching) and student-centred methodologies (general pedagogic)49.

Competence on action research implementation

Training teachers on the theory and practice of action research was a prominent added value of the MTBBE approach. Regarding this fact an official document stated that:

The project has taught teachers to observe, monitor and record academic progress of each student in the class, and to make reasonable adjustments in the teaching process to help them (students) understand the lesson. For this purpose the teachers use a notebook in which they track the process of learning. They take note of new words, errors in spelling or pronunciation. In sum, they take note of important events in the classroom. In this way teachers develop the habit of following the learning of students and the habit of collecting evidence to assess how much progress the students are making.50

This training on action research has been synergetic with the training on innovative teaching methodologies, thus becoming training oriented to problem-solving. For instance, the first assessment of the pilot revealed that some teachers/administrators were moving “weak” students to a lower grade or school. This practice was immediately discouraged; teachers were invited to pay more attention to students with some learning problems. Training content also took note of this practice to provide appropriate teaching tools. As indicated in one document source:

Teachers alone do not guarantee quality. Training and retraining to meet the standards is an important task of the AR MTBBE.51

One good example of the project commitment for teacher quality is the following testimony from one Representative of the Provincial Education and Training Department (DOET) in Gia Lai province:

48 Benson, C., 2008, Appendix 3, page 2. 49 Total Physical Response “is a language of teaching based on the co-ordination of speech and action…

It attempts to teach languages through physical activity” (Baker & Jones, 1998, page 681). Student-

Centred Learning Approach is a cooperative learning environment where teachers question students

and students question teachers; thus students lost their fear to participate in classroom activities (Freeman,

1998, page 168; Al Murshidi, 2014, page 31). 50 MOET-VIES-RCEME, 2014, page 27. 51 MOET-VIES-RCEME, 2014, page 14.



37

For this project we recruited the best J’rai teachers from the district. In the beginning they had difficulties in reading and writing in J’rai. Because of that, the project provided training for the teachers, twice a year at the beginning of the academic year and in the middle of the year. They had 20-day training workshops. In these workshops they learned J’rai and improved their teaching methodologies. As a result of those training workshops most of the teachers have good skills in both J’rai and Vietnamese.

Training as a permanent and multi-level activity

The immediately above testimony (DOET Representative in Gia Lai) is also supporting the fact that training is more than a single event activity; it is a permanent activity originated by different sources. Teacher training was offered twice a year, in July and December. The content of training and retraining of teachers included several topics: fostering basic knowledge, such as the teaching of MT, Vietnamese and Mathematics, teaching and research methodologies, and the use of textbooks and other learning materials. There was also permanent technical support at three levels,

Direct central technical support. Three times during the school year experts and textbook authors from the central level provided technical support to help teachers solve problems of knowledge and teaching methods associated with MTBBE. In this vein a guide on teaching and research implementation was developed.

Technical and professional support at local level. MOET, DOET and the Management Boards of Schools organized regular professional activities on a monthly or weekly basis to help teachers in the management of learning materials and designing plans for teaching and implementation.

Remote technical support. MOET - regularly helped teachers tackle difficulties in teaching and research practices.52

Did MTBBE create conditions to improving the capacity of education managers? MTBBE has included actions to increase the policy-making potential of education managers in supporting EML. Programme actions have included provincial, local and central level management boards. One international designer of the pilot indicated that for a contribution to policies and practices promoting the use and development of EMLs it was necessary “to work with provincial and district education officials to develop strategies so that more ethnic minority people can move into teaching and educational decision making positions.” The other concern of the same consultant was to have access to a proper infrastructure to solve matters on remuneration, methods of teacher selection, monitoring of performance,

52 (MOET-VIES-RCEME, 2014, page 21).



38

the production, storage and transport of school materials, the purchase and distribution of school supplies, etc.”53 In other words, for the designer decentralization was a key factor in building policy-making capacity because it turns this capacity into a bottom-up generating process; decentralization facilitates the fact that local needs may be solved locally. The right to use and develop an ethnic minority language is a question that may be better understood locally. Decentralization facilitates that this understanding translates into policy and actions that sooner or later may have an impact on nation-wide policies. It is important to mention that decentralization is a component of Viet Nam policy framework54; consequently, training provincial and local level education managers on the MTBBE approach aligns well with this national policy. These ideas permeating the AR MTBBE design are the root-motivation of the Programme´s approach to provincial and local education management. Budget figures indicate that about 162 managers of all levels received training from the programme. Evidence also shows that this initiative resulted in the expected outcomes, as illustrated by an official testimony that argues that the current success of the MTTBBE approach was also the result of support and compromise of local education authorities (provincial, district and school level)55. In addition to the above-mentioned local efforts, the Programme paid attention to central-level management. The same source indicated that “strengthening MOET´s leadership and management capacity, and the interdepartmental coordination committee within MOET to carry out the Action Research successfully” were among the list of the AR MTBBE’s top priorities.

4.2.5 Effectiveness in considering contextual factors in MTBBE design and implementation The third sub-question on effectiveness is: Were contextual factors taken into account in the approach design/ implementation of interventions?

Contextual factors were taken into account in the approach design/ implementation of interventions. There is evidence of awareness of the importance of considering contextual factors in the MTBBE design. Actions were undertaken to meet the challenges of the identified contextual factors.

Contextual factors in MTBBE design

53 Benson, 2008, Appendice 3, page 2. 54 Although decentralization is not free of debate, it is a fact that legislation on this issue exists -from

Constitution down to financial policy. An example is the now outdated Council of Ministers’ Resolution

186/HDBT on Fiscal Decentralization to Local Governments, which initiated the fiscal decentralization of

Viet Nam in 1990. An interesting updated reflections on decentralization in Viet Nam can be found on:

https://www.btcctb.org/fr/blog/challenge-making-decentralization-work-vietnam 55 Programme Brief of June 2010, pages 6-7.

https://www.btcctb.org/fr/blog/challenge-making-decentralization-work-vietnam



39

Reports by the main designing consultants (Maloy, Benson) show clear awareness of the

contextual challenges the program was going to face. The design expressed a will to

consider these challenges and, above all, it translated this will into specific preventive

measures addressed to future, permanent identification of new contextual factors. The

program was designed to be sensible enough to detection and solving problems, as best as

possible. The most outstanding illustration of this idea is the management, learning and

participatory tool called action research approach: The Action Research approach, monitoring and evaluation: It is proposed that the action research, monitoring and evaluation be integrated and systematic. Stakeholders involved at all levels of the bilingual pilot project will conduct their own ongoing action research, i.e. make small-scale interventions in the functioning of bilingual schooling under actual conditions and close examination of the effects. This includes everyone from teachers to MOET administrators in their own information-gathering and action-taking activities, overseen by committees at each level: local, provincial and national. The committees will meet periodically during each school year, most often at the local level and at least twice per year on the national level, to discuss progress and any adjustments that need to be made to the program.

Benson, 2006, page 17, italics as in the original56. One important concrete fact that illustrates how the design of the program considered local contextual factors was the set of selecting criteria of the target schools:

Children should come from a single minority ethnic group; Teachers must speak the mother tongue of the students; Curriculum and textbooks must be available. The last condition includes the participation of experts on these areas and academics that speak the respective EML. An EML must have a written system.

Manager of MOET´s Primary Education Department. The development of learning material is also an exposing process of the attention to

contextual factors – where the specific ethnic minority culture is the general most

frequently recommended factor to be considered by developers.

Contextual factors in MTBBE implementation International and national experiences taught that stakeholders` support is a key

contextual factor for the implementation of MTBBE/MLE57. The MTBBE designers showed

awareness of the importance of support from parents/ local communities and all level

policy makers. Consequently, the MTBBE designed and planned activities regarding this

56 See also Annex 3, page 3-4 of the MOU between MOET and UNICEF, 15 August 2007. 57 ”Unless and until social attitudes change…resistance to bilingual education will continue regardless of

official national policy or research demonstrating its effectiveness in educating language minority and

majority children (Ricento & Hornberger, 1996, page 416. See also, Benson C. , 2004; Burton, 2013;

Martinez & Hinojosa, 2012.



40

endeavour that later turned into implementation. The next paragraphs illustrate some

actions that the programme undertook to meet the challenge of this factor.

Advocacy with parents and communities Parents and communities` understanding and support had to be extended to key regional and community organizations. The designers also suggested that, if possible, adult literacy programmes in the MT should be run in parallel to generate positive synergy created by inter-generational reading. They also agreed that pilot schools should only run in supportive communities, thus awareness campaigns had to start early in the process.58 By following these ideas, orientation workshops on MTBBE were organised by community leaders and parents in each target community (2008) where bilingual user-friendly brochures in the local language were distributed (2008). In addition, media announcements in each corresponding language were developed and distributed on a yearly basis.

Advocacy with policy makers Several events with policy makers have been undertaken, like the one of November 2010 that allowed the participation of high-level officials from the Ethnic Council of the National Assembly at an international conference in Bangkok on Language, Education and the Millennium Development Goals. The Viet Nam delegation, which comprised officials from The Committee for Ethnic Minorities (CEMA) and MOET, shared the MTBBE experience in Viet Nam with representatives from other countries. Another sound event was the round table discussion that the proponents of the Programme organized in November 2010. About 100 persons participated in this event, including five standing vice-presidents and 22 members of the Ethnic Council of the National Assembly and 15 senior officials from the National Assembly Office59. One year later the Ethnic Council of the National Assembly organized a second round table, which had the same level of attendance but was enhanced with the presence of representatives from UN agencies, donors, international NGOs, and leaders and educational managers from provinces.60 A critical outcome of these events was the recognition that ethnic minority children learn best through their own mother tongues, and MTBBE approaches are an effective means of learning Vietnamese. An important agreement during the first round table was for the National Assembly to undertake regular visits to target provinces and schools to improve understanding and awareness of the AR MTBBE.61 Two high level officials of the Ethnic Council of the National Assembly reported to the evaluators that the visits to provinces and schools took place as programmed. Both interviewed persons were participants of such visits that helped them to confirm the positive effect of the pilot on children, teachers, educational authorities and parents.

58 Benson 2008, 2006. 59 MOET-UNICEF, 2011, page 6-7. 60 MOET-UNICEF, 2012, page 6. 61 MOET-UNICEF, 2011, page 7.



41

Policy advocacy has also been extended to the Communist Party, besides the Committee for Ethnic Minorities and, of course, MOET in order to promote and scale up the implementation of the MTBBE approach.

4.3 Impact

4.3.1 Defining impact OECD defines impact in the following terms: “The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended.62

UNICEF´s Global Evaluation Quality Assurance System (GEROS 2013) defines impact as the final results of a programme or policy on the intended beneficiaries and, where possible, on comparison groups. Impact reflects the cumulative effect of donor supported programmes of cooperation and national policy initiatives.63 The most frequent questions on impact are about the changes brought by the project´s outcomes and its value added on the beneficiaries.

4.3.2 Summary on impacts The general question regarding impacts is the following: What impacts did the MTBBE approach have on four stakeholders groups: (A) MTBBE students; (B) Teachers, school management, board members; (C) Parents and community people; (D) Managers from grassroots to central levels? Finding No. 3: The MTBBE has produced factual and perceived impacts on its intended beneficiaries/participants. According to different internal/official evaluations children´s learning outcomes have been acceptable in all of the participating schools of the three provinces compared to national standards. Simultaneously, children displayed confidence, self-esteem and communication skills. - Teachers, school management, board members, education managers at all levels generally support the programme and recognize its effects in improving their competencies to successfully apply the MTBBE design and contribute to develop policies and practices promoting the use and development of EML. Parents show awareness of and support to the MTBBE and recognize its benefits in the life of their children.

The appropriateness of the curriculum design, the effectiveness of teacher training and the quality of teaching and learning materials had positive impacts on students’ academic performance. This leads to the conclusion that MTBBE has proven to be an effective approach to a successful inclusion of ethnic minority children into the Vietnamese basic educational system. As a matter of fact, the evaluations carried out in all three participating provinces showed that the programme attained an acceptable student retention rate over the six years of study and the students’ performance was high in both MT and Vietnamese, especially in listening and speaking components, and satisfactory scores were obtained in Mathematics. Comparative studies undertaken throughout the course of the pilot programme indicate that students in the MTBBE programme scored better in both Vietnamese and Mathematics than students who were not in the programme. These outcomes came together with other positive effects on students’ self-esteem, confidence and communication skills. The pilot phase of the programme benefited from regular in-service training and support, mainly targeting

62 OECD Network on Development Evaluation, n.d. page 14 (bold letter are not in the original). 63 UNICEF, 2013, page 10.



42

teachers and education managers. As a consequence of the training initiatives undertaken, key participants reported improvements in teachers’ literacy skills in EML, teaching and self-learning skills, research methodologies and attitudes toward teaching. These improvements explain teachers heightened self-esteem and confidence, in particular on the part of ethnic minority teachers. The impact of training initiatives on education managers includes better perception of the challenges faced by ethnic minority children when educated solely in Vietnamese, increased awareness of cultural rights, and development of skills needed to manage bilingual schools. The evidence produced shows that the MTBBE programme had the support of parents and communities, who mainly saw it as a relevant contribution to the promotion and preservation of their languages and ethnic traditions and values. Despite reports of scepticism and apprehension at the inception of the programme, parents’ and communities’ attitudes and perceptions changed over the course of the programme especially as a consequence of perceived positive academic results exhibited by participating students and MTBBE advocacy initiatives. Parents’ and communities’ discomfort was based on the view that students in the MTBBE programme would not learn Vietnamese properly, the language of social mobility in Viet Nam. This mind-set changed when they perceived that, in addition to exhibiting good kills in their mother tongues, students in the MTBBE programme were also performing well in Vietnamese. Therefore, parents’ and community members’ active participation in school activities and overall support to the programme is closely linked to perceived and attested academic and sociocultural impacts of the MTBBE programme on participating children and communities.

Impact of the approach?

On MTBBE students

On teachers, school management, board members

On all-level managers

On Parents and community people

Sections 4.3.3 to 4.3.6 respond to each of the sub-questions on impact by providing the summary supporting evidence of Finding No.3.

4.3.3 What impacts did the MTBBE approach have on the learning outcomes of participant students?

The most important test for the MTBBE approach is its capacity to impact positively on the learning outcomes of the participating students. In this regard, MTBBE exhibits a remarkable advantage as a valid model for educating ethnic minority children. School dropout rates of both student cohorts were low along the period of the programme; when they occurred, these withdrawals were a result of parent migration to other communities. An important achievement under this type of cultural context was the fact that girl students were a majority since the beginning to the end of the respective cohort period. Both cohorts finished primary school with remarkable school attainments. Almost no one scored “weak”, and a trend is observed for a gradual increase of the percentages reaching “good” and “excellent” score category. Periodical assessment between comparable samples of MTBBE and non-MTBBE students shows the first group scoring higher in Mathematics and in Vietnamese. Good learning of these two subjects according to national standards becomes the best indicators of the effectiveness and impact of the MTBBE. These accomplishments also translate into good effects on attitudes and habits of mind. Participating students expressed confidence, self-esteem, and enthusiasm for learning.



43

School retention and girls participation

The evaluation team had data on learning outcomes that allowed a breakdown by sex only regarding enrolment. Thus, the analysis does not distinguish grade scores by girls and boys. However, two facts suggest that girls´ and boys´ scoring could not be very different between them. These facts are: (i) girls were a slight majority for both cohorts in all provinces; and (ii) any student of both cohorts scored “weak” during the period, except few exceptional cases of about 1%. AR MTBBE has worked with two student cohorts. The first cohort started Preschool [kindergarten] in 2008. Children of this cohort reached Grade 5, the last year of the primary education level in Viet Nam, in June 2014. The second cohort started in 2009 to the end of June 2015. [See Table 3].

Table 3. MTBBE Cohorts and school grades

September

2008 2009 2010 2011 2012 2013 2014

1st cohort students (2008 - 2014)

Preschool 5 years

old Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

2nd cohort students (2009-2015)

Pre-

school 5 years old

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

The first MTBBE student cohort began with 262 pre-school students; 52% of them were girls. This level of student enrolment remained the same for the next year, when the cohort entered into the elementary school. Students were distributed into eight (8) schools, the first three located in the Province of Lào Cai with Mong as the students’ MT (L1). The next three schools were situated in the Province of Gia Lai; there, J’rai was the students’ L1. The last two schools were located in the Province of Trà Vinh where the students’ L1 was Khmer. In half of the schools two classes were organized. Three schools organized only one class. Hàm Giang School is the only one, which offered three classes. In total, the first cohort was distributed into fourteen classes each with no more than 25 students. [See Annex 5, Table A1.] Of 262 students of the first cohort, 247 completed the pilot and 15 students dropped out of school because their parents migrated to other communities. According to these figures, the dropout rate for this cohort during the entire period was 5.7%, and the average annual dropout rate was 1.5%. Is the first rate acceptable? First, we must consider the external origin of the motive for the dropout, which was household migration. For a comparison with national standards two sources are useful: (1) The 2003 SAVY Survey, which estimated a dropout rate of 17.2% for the entire primary school level in Viet Nam,



44

but in the case of ethnic minorities this rate was 39.8%; (2) the EFA´s 2015 target of a dropout rate was 0.5% for primary school grades 1-4 (EFA Programme 2012, page 34). The case of girls, who started and finished the cohort period as a majority, i.e. 52% (at preschool) and 53% (at Grade 5) is interesting. From the original 132 girls at first grade, only four left school during the period. This achievement may say little or a lot depending on the cultural environment in which learning takes place. For example,

Mong people did not allow girls to attend school. This was the traditional thinking. When I first started to teach in this school, there were very few girls. Since the implementation of the MTBBE, more girls attended.64

The cases of Gia Lai and Trà Vinh were different. The first is a matrilineal culture; so, boys and girls are educated equally. In the second case the interviewees assured the evaluators that there were no cultural restrictions for girls attending school unless household survival strategies obliged them to opt for having girls at home; as in other situations boys are the ones selected to participate in family income-generating activities. In sum, in terms of student retention, this pilot initiative showed an impressive result for the first cohort. The second cohort of MTBBE students had a smaller number of students, 248 compared with 262 of the first cohort. [See Annex 5, Table A2.] Currently the students of this cohort are close to finishing Grade 5, with 10 less students compared to the 2010 number. There is no drop out children in MTBBE classes. 1.4% did not contine learning in MTBBE as their families moved to other location.. So far, this cohort also promises an acceptable achievement in terms of student retention. In this second cohort girls have increased their representation to 57% at Grade 1, and 58% at Grade 4.

General learning outcomes In general terms, the evaluation of the students of the two cohorts indicates remarkably similar results in terms of learning outcomes. All enrolled children of the first cohort successfully finished primary school, no one scoring “weak”. See Figure 1 below. The second cohort seems to be following the same route. [See Annex 5, Figure A1]. The line over the bars for both figures represents the average behaviour or tendency of the “excellent” score. This indicates that, in general, student scores managed to improve during the respective cohort period, i.e. the per cent of students scoring “Excellent” tended to increase gradually; the same happened in the case of the per cent of students scoring “good”. This is an interesting result, particularly from grade 3 and up when Vietnamese broadens its role in the learning process, indicating that MTBBE is effective as a smooth transition to the mainstream system.

64 Teacher, Ban Pho School.



45

Figure 1. Percentage of MTBBE first cohort students according to grade and general scores

Source: Prepared by the evaluation team based on the Action Research MTBBE Data Bank

Details regarding student achievements on Mother Tongue and Vietnamese

The evaluation of students´ learning outcomes regarding MT and Vietnamese distinguished two general dimensions in mastering the language. First was the listening and speaking skill, and the second was the capacity to read and comprehend. In general, students show better ability to listen and speak than to read and understand. Taking an example of the school year 2012-2013, the students of both cohorts, in this case Grade 4 for the first cohort and Grade 3 for the second cohort, tended to score better in listening and speaking than in reading and understanding. This situation happened both in the Mother Tongue and in Vietnamese. [See Figures 2 and 3.] Figure 2. Percent of students by language dimensions and scores in MOTHER TONGUE. School year 2012-2013, 1st and 2nd cohorts

Source: Prepared by the evaluation team based on the Action Research MTBBE Data Bank As is known, reading and comprehension imply a greater capacity for abstraction than listening and speaking. Abstraction capacity is expected to grow according to age and exposure to diverse intellectual challenges. AR MTBBE students have experienced different problems in developing reading and understanding competences. Some of the problems were attributable to the complexity of grammar of their own language, as in the case of



46

Khmer. For example, during interviews with Khmer students, some of them declared that the alphabet and some words of the MT were more difficult for them than the Vietnamese alphabet. Teachers of Hàm Giang School confirmed these children´s perceptions:

We try our best to explain in Khmer to help children understand well. However, for Grades 4 and 5, the level of difficulty of language is high, especially loan words from Pali and Sanskrit make students feel overwhelmed. Thus the teachers have to explain carefully and in much detail to help students understand the meaning of words.

Figure 3. Percent of students by language dimensions and scores in Vietnamese School year 2012-2013, 1st and 2nd cohorts


Due to weakness of the written systems of most EML, children could be more exposed at the end, and more familiar with, Vietnamese letters and words. Sometimes they capture letters and words from commercials, governmental signs, banners, etc. Thus, the written manifestations of their MT could be known less than those of the official language. In other cases, problems are related to teachers´ mastery of MT like this testimony reveals: “Many teachers can speak but cannot read and write in EML”. The same respondent indicated: “The Khmer teachers are better because they can read and write in their language. The Mong teachers are the least qualified; their Vietnamese is also limited because of isolation”.65 Problems of this nature also occur in the case of the J’rai language. There the most important problem is with regional variations or dialects:

Of course, we know about some concerns raised by communities over the language varieties. In the material for kindergarten we initially used the Pleiku J’rai variety. However, we found that this variety has features of Vietnamese given the close contact between these languages in this area; it borrows many words from Vietnamese. So we changed to J’rai Chor, from the Ayun Pa, Krongpa region, as the dialect of reference. J’rai varieties from other regions have less Vietnamese influence. However, we still use some technical

65 MOET´s technical staff in Ha Noi.



47

words from J’rai Pleiku when J’rai Chor does not have those words. So, we now tend to use the original words from those varieties in the textbooks. When we started to do that, teachers and students had some difficulties in understand the materials. However, since we then produced glossaries, with lists of Vietnamese words translated into different varieties of J’rai, they finally appropriated those words and even taught them to the communities around Pleiku.66

Performance of MTBBE students compared to Non-participant students

The Primary Education Department benefitted from the technical assistance of an international consultant to assess annually the learning outcomes of MTBBE students. As a part of this effort, comparisons of MTBBE students to a group of non-participant children in MTBBE activities were made. The sample of non-participant children was chosen according to specific criteria67. For comparison, Vietnamese and Mathematics were considered the most appropriate subjects. Mathematics was selected because the MTBBE curriculum is identical to the national one; the mastery of Vietnamese is considered a passport for inclusion of ethnic minority children into the national society. Good learning of these two subjects according to national standards would be the best indicators of the effectiveness and impact of the MTBBE approach.

Since the start of these comparisons in 2009, MTBBE students have shown better performance in the aforementioned subjects than non-MTBBE students. Some of the results were presented in the Programme Brief of June 2010 (page 5) where the following note appears:

A comparison group of students who did not participate in MTBBE classes was also assessed in May 2009, in the same categories [or components of language mastery] with the exception of recognizing mother tongue alphabets… Key findings include: students who participated in MTBBE scored higher than the comparison group in four areas; and a significant difference between the two groups was identified in arranging pictures…[For Vietnamese], again, students who participated in the MTBBE scored higher than the comparison group in all five areas, thus highlighting the overall educational strength of the MTBBE approach.

66 DOET textbook local developer, Gia Lai. 67 The criteria for selecting the sample of non-participant children in the MTBBE pilot were the

following: (1) Ethnic minority student with non-MTBBE learning in the schools having MTBBE classes;

(2) Ethnic minority student with non-MTBBE learning in a school of the district having MTBBE; (3)

Kinh students in a school of the district having MTBBE students were randomly selected.



48

Figure 4. Percent of fourth graders (First cohort) according to provinces and MTBBE / Non-MTBBE students, by scores in Vietnamese


Figures 4 and 5 (main text) and A4 and A5 in Annex 2 present comparisons between MTBBE students and non-MTBBE students in the subjects of Vietnamese and Mathematics in the three participating provinces and for both cohorts during the school year 2012-2013. The results are similar, i.e. in general MTBBE students tended to score better than non-MTBBE students. Figure 5. Per cent of fourth graders (First cohort) by provinces and MTBBE / Non-MTBBE students, by scores in Mathematics

Source: Prepared by the evaluation team based on the Action Research MTBBE Data Bank In sum, the comparative analysis above shows that MTBBE intervention is making a difference. Participant students have attained literacy in their MT and by this means they have experienced a smooth transition to Vietnamese, whose scores look more competitive than the ones of their counterparts in the mainstream programme. A superior performance is also clear in the case of Math.



49

Other learning outcomes The use of MT helps students to be more confident in learning. Courage and confidence help them to be positive, proactive, and creative. These gains in attitudes and habits of mind are the result, and also a condition, for continuing improving, this way preparing students for the next educational stage. After the second year of implementation, the Annual Progress Report 2010 (page 7) indicated the following: “Grade 1 students like to go to school. They are more proactive, natural, and mentally stimulated. Class atmosphere is the same as that of Kinh students who study Vietnamese in favourable areas”. The same report stated that all students showed close to perfect attendance and behaviour in the classroom. In the MOET-UNICEF Programme Brief 3 of September 2012 (page 6), the assessment of this matter continues to show great results:

The approach continues to receive positive feedback at all levels and positive views have been expressed on the benefits of the approach, particularly the education outcomes of children themselves, from the progress in learning results to their self-confidence and self-esteem. According to local education managers and project staff in all implementing schools, students of MTBBE classes maintain the highest attendance record, enjoy a dynamic learning atmosphere and are proud of their classes and schools.

The MOET Progress Report of 2014 indicates again that in the year 2013-2014 both student cohorts achieved perfect behaviour at school and were able to pass to the next grade. This document also informs that:

The use of the mother tongue helps students a lot to be more confident in learning. Courage and confidence help them to be positive, proactive, and creative. Ethnic minority children easily express their thoughts in their mother tongue, something that could be very difficult if [it] were the case of talking in Vietnamese. At Grade 3 the capacity to speak in Vietnamese has increased remarkably. At this moment they can talk using both languages. It is a smooth transition that creates the basis for improving both languages (page 32).

This documentary information is in harmony with the different testimonies collected from interviews and focus group sessions, for example,

I like both languages. I gained a lot of knowledge.68 Children participate more and are more creative in the classroom than their peers who are not in this programme. J’rai children who learn in Vietnamese do not express themselves fluently in this language, especially in front of a visitor.69

68 MTBBE student of Ban Pho School, Lào Cai 69 Deputy Principal in a school of Gia Lai



50

Obviously, the main differences between the two groups of students is that MTBBE students learn in their MT, and when they transit to Vietnamese, they are more active and more confident in taking part in learning activities, even in their daily life as well.70

More testimonies are presented in Annex 4, Box 4.

4.3.4 Impact on teachers, school management, board members and all level education managers The second sub-question on impact is: What impact (positive or negative changes) did the MTBBE approach have on teachers, school management, board members and all level managers?

The design of the AR MTBBE paid important attention to in-service training and to support of teachers and school authorities. These activities prepared the teachers for a better knowledge of the EML, for implementing updated teaching and learning methodologies and for having a proactive attitude by means of both action research techniques and displayed confidence and self-esteem. School authorities have gained better perception of the linguistic and other learning problems of ethnic minority students and have also increased their own awareness of cultural rights. This way, school managers are now better prepared to manage bilingual schools. Provincial and higher-level educational managers have also been a target of training and advocacy events, which have strengthened their policy-making capabilities regarding EML as well as MTBBE/MLE.

Impacts on teachers Evidence shows positive impact on teachers. The AR MTBBE has not only improved the capacity of teachers to meet basic quality requirements of the pilot, it has also enhanced their confidence, self-esteem and commitment toward students and school. Testimonies of programme effects on teachers´ attitudes came from teachers and local educational authorities. These testimonies confirm that training was the trigger for a new teacher attitude. Immediate results of these training activities were confidence, enthusiasm, peer communication, articulated efforts, access to information on teaching methodologies and sense of being supported by a body of technical staff and education authorities.

70 School Principal of a Trà Vinh school.



51

Teachers are more enthusiastic. They have to learn a lot and do a lot of self-learning. They have to learn Mong to teach in Mong. They have to research and study learning materials. They have to learn Mong culture, so that they can integrate it into the education programme. This improves teachers’ ability to self-learning. Teachers have to translate materials from or to the target mother tongue. This provides teachers with opportunities to improve their teaching capacity. They also exchange views with textbook authors; this way they understand the teaching materials better and become very confident. They have strong teaching techniques compared to other teachers.71 MTBBE has helped us in our professional development. We have received training in new teaching methods, such as Total Physical Response. Now we can use different methods, be more flexible, and able to get that learning be a fun activity. Training workshops are good not only to learn new methods; we also have the opportunity to share experiences, to suggest teaching method improvements.72 Most of MTBBE teachers are very enthusiastic about teaching in this programme. The principals also follow suit. The DOET officials are also happy with the programme because they can see the learning outcomes of the programme.73

[See more testimonies in Annex 4, Box 5.]

Impact on teachers, school management board members and all level educational managers The AR MTBBE has improved the capacity of teachers, education managers, initially by means of training. Now they show a high level of awareness about the importance of MTBBE. They have improved their knowledge of the MT and the particular administrative and methodological conditions which this approach requires for its functionality. The evaluators were greatly surprised by observing how well informed and committed the managers appeared to be. Their different testimonies acknowledged that training has been useful for them in different matters. Some went deeper into mastering a MT. Some feel their leadership capacity improved as a result of their involvement with the programme. One important contribution was the application of MTBBE teaching/learning methodologies to the mainstream programme.

Via some training workshops for MTBBE project and other types of workshop, my management skills have been greatly improved. I also attended a workshop on second language learning in Lào Cai two years ago that was presented by experts from a foreign country. Though this workshop did not discuss in-depth the teaching methods for any specific class or programme, it did so for the general one. However, we apply this to the MTBBE Action Research and the mainstream programme, especially at the primary level. This means that my professional capacity has been improved… [Exchanging experience

71 School Principal in a Lào Cai school. 72 Teachers, Nhi Truong School, Trà Vinh 73 DOET Authority, Gia Lai.



52

with MTBBE schools] helped me changing my perception of educational leadership; that is the focus on changing teaching methods, which was previously less focused, since previously more attention was paid to administrative activities. Innovation in teaching methods is a key point to enhance educational quality. Also staff from the Research Institute has provided a lot of experience in running and managing school activities, not only in workshops but also outside the workshops, to enhance the learning quality.74 The AR MTBBE has also been useful to education managers. I recognize that it had some impact on me. I had a chance to attend workshops on education management. The project helped me to update my knowledge of teaching methods. This knowledge was an important asset in my classroom observation activities. So, the project contributed to improve the quality of education in this province.75

[See more testimonies in Annex 4, Box 6.]

4.3.5 Impact on parents and the community The third sub-questions on impact is: What impacts (positive or negative changes) did the MTBBE approach have on parents and the community?

The AR MTBEE has attained parents´ and communities´ confidence in the project as well as their

participation in school activities. Simultaneously, the MTBBE has enhanced their appreciation for their ethnic traditions and values. For MTBBE this is an important achievement since parents from ethnic minorities usually distrust treatment they consider discriminatory for their children. Parents´ attitude changed because of advocacy initiatives, as some of them recognize, but probably the academic accomplishments of their children were the best remedy against initial distrust. Although the majority of parents speak their MT at home, they feel Vietnamese is their children´s passport to future achievements in education, labour market, social status, etc. Before the inception of the pilot, some parents could be satisfied because their mother tongue was being considered as a language of instruction by the education authorities76. But at the same time, they were worried that their children would not be proficient in Vietnamese. For example:

I was initially worried about the objectives of this programme. I thought by learning in J’rai my child would never learn or understand Vietnamese. However, when he was at grade 3, I realized that he could perform well in Vietnamese too. So, I am now very happy with the outcomes of this programme, I am no longer worried that children in the programme will not speak Vietnamese.77

74 DOET Leader, Trà Vinh. 75 DOET Representative, Gia Lai. 76 Benson, 2008, 2006 “A bilingual design has great symbolic value because the languages are side-by-

side in the materials and in instruction. The student and family see that their language as well as

Vietnamese can be used in reading and writing and for teaching and learning, as well as describing

scientific, mathematical and other concepts. This represents a big step toward gaining community support

and involvement in mother tongue-based bilingual education.” (Benson, 2006, page 21) 77 Parent, Lu Tu Trong School, Gia Lai.



53

Different assessment reports took note of parents´ and the community´s support for the AR MTBBE. This is consistent with the evaluators’ observation and testimonies collected in the three participating provinces. Furthermore, during the pilot period, parents observed that their children had significantly advanced in education and had positive attitudes towards learning. Testimonies suggest that, in many cases, children´s enthusiasm has brought their parents closer to their schools. Consequently, communication between parents and teachers and also between parents and children has greatly improved. In some households children and parents help each other in understanding MT and Vietnamese alphabets and words. Pride of ethnic cultural values and traditions is emerging at home and also extends to some instances of community life. Important too is the effect that the pilot seems to have on girl attendance and retention at school, particularly in the case of Lào Cai schools, where Mong tradition is stricter regarding the subordinate role of education in a girl’s life, as demonstrated in the following examples of testimonies from parents.

When this programme started, we were confused. We thought that children would only learn in J’rai. We asked ourselves how they would learn Vietnamese. However, after three years we changed our minds. We realized that they know both J’rai and Vietnamese. So we started to like the programme.78 I’m a Khmer but I don’t know Khmer letters. This makes me unhappy. In my opinion, the students who are in the MTBBE Action Research are always more advanced to those of the mainstream programme. My son can read everything that appears on TV, but not I. The more they know Khmer the more they want to learn.79

We provide learning objects and some school supply. When the school calls us we do volunteer work, like gardening, contributing bamboo to make a fence, and whatever the

school needs.80

[See additional testimonies in Annex 4, Box 7.]

In summary, the MTBBE programme has produced positive impact in the life of ethnic minority children and their parents. They have found a source of pride in the revival of their traditional values and better consideration to their MT. The opportunity to learn Vietnamese by means of mastering first the MT has facilitated the path to other cognitive skills, such as Mathematics, and positive attitudes like confidence and self-esteem. This direct impact, in addition to other contributions for improving conditions for policy making on EML, lead to conclude that the MTBBE approach has proven itself in the pilot test on feasibility and validity in the context of Viet Nam. It is also important to add that in general terms MTBBE has also contributed to the objectives of UNICEF Viet Nam on gender and equity. Designers of the AR MTBBE

78 Parent, Ngo May School, Gia Lai. 79 Parents, Ham Giang School, Trà Vinh. 80 Parents, Ban Pho School, Lào Cai.



54

recommended special measures to attain and keep a gender balance in the participating schools, i.e. implementers had to exert special effort for guaranteeing that girls would be around half of the total student enrolment of each school. Evidence presented above indicates that this designers’ purpose was met. At the beginning and at the end of each cohort period girls were a slight majority. Beyond this accomplishment, as in the case of boys, not a single girl was classified in a “weak”81 learning situation at the end of the process. During the visits to schools the evaluators observed no great differences between girls and boys regarding colloquial performance to outsiders. Girls and boys showed confidence and enthusiasm, the former looking even much more expressive. MTBBE is per se an equity-oriented policy instrument as soon as it attempts to fill ethnic and gender gaps in education by providing an effective opportunity to ethnic minority children to overcome the language barrier. The efforts in terms of policy made under the MTBBE approach are also addressed to attain equity educational goals in favour of ethnic minority populations.

4.4 Efficiency

4.4.1 Defining efficiency According to OECD DAC (2002), “efficiency is a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results”, which in turn, are defined as the combination of outputs, outcomes and impact. An “economical” management is a situation when the costs of used scarce resources approximate the minimum needed to achieve planned objectives. In other words, “efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is an economic term which is used to assess the extent to which aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted” (OECD DAC, 2010). Frequent questions on efficiency are the following: Were activities cost-efficient? Were objectives achieved on time? Was the programme or project implemented in the most efficient way compared to alternatives?

4.4.2 Summary on efficiency The general question on efficiency is the following: Does the actual implementation of the approach use resources in the most efficient (economical) manner to achieve expected results? Finding No. 4: Evidence provided by MOET and UNICEF Viet Nam indicates that monetary and time resources were used in an economically acceptable manner. Stakeholders expressed awareness of the cost vs. results relationship. Most think that the results of the programme are high. Its cost is offset by its returns, particularly its immediate impacts on learning outcomes and teacher training, as well as its contribution to capacity building for policy making.

81 “Weak” is used in this context to indicate that a student does not master the minimal standard

competences required for that grade at the moment of the assessment.



55

Teacher training and deployment were efficiently managed. The participating schools had enough trained teachers. Overall, the development and provision of school materials were also undertaken in an efficient manner. Despite the general satisfaction in relation to the quality and provision of school materials, some key actors, including teachers and school managers, still expected greater availability of complementary materials, such as teaching guides, grammars, dictionaries and visual aids. The project objectives were achieved on time. This is the result of synchronisation with the school cycle. It was important to assure that participating schools would receive the necessary inputs on time to assure good quality services. Complementing this effort, the policy environment also had to work in alignment with MTBBE progress, thus producing the synergies to turn this pilot into a benchmark of good educational practices. The higher cost of the MTBBE programme resulted from the cost of training activities and the development of learning/teaching materials (87% of the expenditures). As international experiences have shown, and as was also recognized by key participants in the evaluation, the reliance on regular in-service training, the innovative effort placed on material development and the small size of the student population served were the main factors explaining the relatively high costs incurred in the pilot phase. This suggests that, in a post-pilot phase, the cost of MTBBE programming will be lower when MTBBE becomes mainstream programme in Viet Nam. Despite its relatively high cost, the immediate and long-term benefits attached to the programme make it a worthwhile investment. The results of the programme included: (i) the improvement of retention and completion rates in the participating schools, with a special focus on girls; (ii) the improvement of the professional capacity of ethnic minority teachers; (iii) the emergence of a new cadre of developers of education material in ethnic minority languages; (iv) the revival of parents’ and communities’ participation in formal education of ethnic minority children; and (v) the renewed societal awareness over the language challenges faced by ethnic minority children in an education system exclusively conducted in Vietnamese. The MTBBE programme has opened new perspectives for the education of ethnic minority children and a path towards a better inclusion of ethnic minority groups in the wider society.

An efficient programme?

Cost-efficiency?

Time-efficiency?

Comparative efficiency

4.4.3 Were activities cost-efficiency?

Compared to its outputs, outcomes and impact the AR MTBBE has been a cost-efficient programme, although data limitation in the analysis should be considered. Resources for the AR MTBBE were provided on time and accordingly to the scale of activities. In fact UNICEF was able to assure funds that anticipated a larger number of schools and participating students than the ones finally approved. Evidence indicated that the budget was prudently managed; expenditures were only for capacity building, teaching and learning materials, and monitoring and evaluation activities. Capacity building, which included teacher training and advocacy actions, represented almost half of the expenditures while teaching and learning materials reached almost two fifths. In some ways these expenditures represent the additional cost that MTBBE implies, at the inception stage, since regular MOET expenditures for the mainstreaming programme are not included. Most expenditures occurred during the first two years because the need to design the curricula (pre-school and primary school), provide in-service training for teachers, develop educational materials and assure the support of parents, communities, and all levels of education authorities.



56

The data MOET provided the basic financial information used in this section,82 which was complemented with data from UNICEF Viet Nam. 83 However, it lacks a frame of reference to compare these actual expenditures with the planned ones in terms of quantities and time. Precisely, this is one role of a Project Initiation Document (PID), particularly its budget section. As reported before, the MoU between UNICEF and MOET of 15 August 2007 and one of its Annexes, the Action Research Implementation Framework, guided the implementation of the AR MTBBE, instead of a PID. Specific financial information is not found in the MoU and the Implementation Framework. Regarding this issue the MoU only states that UNICEF would be responsible “To liaise with and report to interested donors to support the raising of financial resources for the implementation of Research activities throughout the entire Research cycle” (3.7.3, page 4). At inception, the AR Implementation Framework listed the different activities to be implemented on a yearly gross chronogram, but no budget figures are provided. However, the Annual Work plans of the UNICEF Education Programme included AR MTBBE budgetary data according to activities. Consequently, the lack of a budgetary benchmark (the PID budget), made UNICEF´s budget source the most convenient benchmark. One additional problem has to be considered: not all of the budget categories supplied by MOET are comparable to the UNICEF ones due to different criteria or levels or disaggregation. As for an external comparison, the AR MTBBE could not be contrasted to another similar alternative, e.g. the approach of strengthening Vietnamese for ethnic minority children, because the evaluation team could not obtain the respective budget data. A gross comparison was made with Viet Nam public expenditure data on the primary education sector; thus this became a gross reference in lieu of a fairer, appropriate benchmark.

Available resources UNICEF met its responsibility of mobilizing funds for the AR MTBBE. Financial resources were enough and ready when needed. UNICEF estimated costs of about US$5.2 million for the period 2008-2014 to fulfil its commitment in relation to the AR MTBBE.84 Before and during its implementation, UNICEF submitted various project proposals to mobilize financial resources from different donors. The result was spectacular. The Norwegian Government contributed US$1.6 million for the period 2006-2010, and others, including UNICEF’s general resources and fund, contributed US$2.4 million. Each of these donations has been managed according to the conditions set out by each donor.

82 MTBBE Information for Cost/Efficiency Analysis (Thong Tin dé phan tích tính hieu quá chi phí). 83 AR MTBBE Planned Budget extracted from UUNICEF Education Programme Annual Work plans

2008-2014. 84 It was agreed that MOET would provide only in-kind counterpart resources (administration, offices and

some equipment, human resources), which at the end were not registered in a systematic manner, thus the

cost reports do not include such contribution.



57

Previously UNICEF had incurred some expenses to contract international consultants for designing and promoting the project. During the pre-implementation phase 2006-2007 US$505,000 were spent on advocacy, orientation of parents and communities about the AR, literacy training of EML, a baseline survey to select the target schools, and the development of bilingual education materials for pre-primary education. UNICEF also anticipated that about half of the financial resources would be disbursed during the first three years of the period 2008-2014. This prediction was, considering the important initial efforts, to be done in terms of teacher training and the preparation of learning/teaching materials. Thus, the project would need to be efficient in supplying the basic inputs to the pilot on a yearly basis.

Budget execution During the period 2008-2014 the total real expenditure amounted to close to US$3.5 million. Compared with the estimated costs (US$5.2 million), the actual expenses were estimated to be about 33 per cent below UNICEF’s original estimates. There are several explanations for this budgetary difference. First and most important, UNICEF´s initial budget estimation was based on a larger focus population of students and teachers, as expressed in the project proposal submitted to IKEA Social Initiative Secretariat in 200885. Second, there is evidence of a prudent disbursing policy. There are documents with specific instructions of international consultants on how to avoid costs on both teacher training and printing of learning materials. Several testimonies indicate that implementers followed such instructions. Additionally, MOET’s in-kind contributions allowed some cost savings, such as overhead, local and office equipment, national experts, etc. In some districts, educational authorities reportedly contributed by preparing classrooms where the AR MTBBE was to be implemented86. Third, according to interviewees´ testimonies, schools received the basic inputs they needed to guarantee good project implementation, such as training and learning/teaching materials, only with a brief delay at the inception stage87. Of course, some interviewees still expected better availability of complementary materials, such as, teaching guides, grammars, dictionaries, visual aids, etc.

85 Focus population: “5,000 pre-primary and primary school children between the age of 5-14 half of

whom are girls in 12 pre-primary schools and 13 primary schools in 3 provinces.” 86 One example: “DOET have equipped learning facilities such as desks, chairs, boards. Enhance minor

repairs and furnish small facilities. Create good conditions for teachers to improve their knowledge of

Khmer and Vietnamese.” (DOET Officer, Trà Vinh). 87 “Initially we used to start the new school year using photocopies. The textbooks came two or three

weeks later. However, this problem does not exist anymore as we receive the proper textbooks before the

start of the school year.” (Teacher, Ia Phi School, Gia Lai).



58

Finally, the disbursement amount during the period (about US$3.5 million) is close to the US$4.0 million that UNICEF had initially collected from different donors.

Resource allocation AR MTBBE expenditures present an interesting distribution that leaves an added value to the basic education sector of Viet Nam. In fact, the total resources were invested in developing both human capital and teaching and learning materials. This was possible because wages, salaries and other administration costs were financed from MOET´s regular budget. This also means that the AR MTBBE budget reflects well the cost of this type of education in its initial stage over the regular cost of mainstreaming education. Almost half of the total expenses (49%) were for capacity-building activities that benefited teachers, school authorities and other education managers. Training includes different workshops on BE methodologies and MTBBE implementation and monitoring, as well as advocacy activities, such as study visits, international workshops, conferences. Another important component is learning and teaching materials (38%), which include the costs in developing the curriculum for the MTBBE, and translating, printing and distributing educational materials. A third budget component is monitoring and evaluation (13%), which includes technical assistance visits to schools and annual assessment of students’ learning outcomes. [See Figure 6 and Table 1 in Annex 7]. Figure 6. MTBBE Costs by component

Source: Prepared by the evaluation team based on the Action Research

MTBBE Data Bank. Original data are in Table 1, Annex 7 Costs were at a peak in 2010-2011 and 2011-2012, particularly for training and learning/teaching materials. Expenditures increased to US$725,000 in 2011. During those years the eight target schools were delivering educational services to two student cohorts simultaneously. By September 2011 and September 2012, the first and the second cohorts, respectively, were reaching Grade 3. As is known, after this grade there is more balance between MT and Vietnamese; this seems to reduce the demand for learning/teaching materials and training. Figure 7 helps to observe better that the first three years, as predicted, required the major efforts in terms of training and learning/teaching materials;



59

therefore costs reached their peak at Grade 3. This seems to suggest that costs are similar to those under a regular curriculum during the last two years of primary school. The project started from non-existing pre-service training on teaching EML under the MTBBE approach. In-service training is more expensive and necessary even when pre-service training is available. The cost of printed materials per unit declines when the number of printed units increases. These economies of scale can be realized with the scaling up of the AR MTBEE.



60

Figure 7. MTBBE Costs 2008-2014

Source: Prepared by the evaluation team based on the Action Research MTBBE Data Bank. Original data are in Table 2, Annex 7.

Cost vs. results An economic perspective is incomplete if costs are not compared to results88; from this analysis the MTBBE pilot can be better assessed as an alternative to improve the learning outcomes of children from ethnic minorities, the mastery of Vietnamese included89. Profuse international evidence backs up MTBBE. The mastery of MT guarantees both better success at school and faster learning of a second and more languages. So, official concern with the learning of Vietnamese can be met. Enhanced results are to be expected if children are exposed to a friendly learning environment characterized by a student-centred methodology and adequate infrastructure support. The MTBBE pilot is an initiative that has been working to demonstrate the advantages of this approach. Relatively high cost has been an argument brought in by some critics. However, international experiences show that immediate and long-term outcomes surpass costs.

88 Cost is meaningless in itself. It makes sense only in relation with what one gets in return for the cost

incurred. Even a high cost policy can be perfectly reasonable on economic grounds, if the outcome is

`worth it´; and paying for something which is worth paying for, is a quintessentially sound economic

decision (Grin, 2003, page 11). 89 As informed before, other alternatives implemented in Viet Nam to UNICEF supported MTBBE

implemented in Viet Nam PEDC or Vietnamese reading model, JICA´s “Lesson Study”, Teacher Action

Research which is a Belgian Project, Escuela Nueva model in Viet Nam supported by the World Bank, and

finally, a group integrated by different projects supported by Oxfam UK, Enfant et Développement, Save the

Children and UNICEF, based on the MT as language of learning.



61

Annex 8 presents a summary list of results, which are mentioned in the programme´s documents and by testimonies of different informants participating in this on-going evaluation. Add to the list the contribution of the programme to change the mind-set of education managers when allowing that a language other than Vietnamese was used as language of learning. This contribution by itself might well compensate the incurred costs.

4.4.4 Were objectives achieved on time?

The implementation of the AR MTBBE occurred in close coordination with the school cycle, year by year. Coordination was necessary to assure that the two participating cohort students received education simultaneously to the mainstream programme. Both student cohorts started and finished activities on the expected dates. There is also documentary and testimonial evidence supporting the statement that schools received the expected inputs (trained teachers, teaching and learning materials, etc.) within the expected time and quantities. On the other hand, the programme has contributed to the development of policies and practices promoting EML - the respective actions being carried out within the planned time span. All these lead to conclude that objectives were achieved on time.

Time in supplying the necessary inputs for the participating schools was crucial in order to work in alignment with the official school schedule. Activities regarding the curriculum designs (for pre-school and primary school), textbook development, and teacher in-service training and support have to be undertaken in harmony with the school calendar. In this regard, interviews with different informants (education managers, school authorities, teachers, and parents) included questions addressed to assess whether training events, educational materials and other inputs and activities were received within the expected time and in the required amount. There were only few complaints regarding some delays in the first delivery of textbooks at the start of the programme. However, besides this initial problem, none of the different types of respondent manifested dissatisfaction regarding the timing and amount of supplies. Training and advocacy activities addressed to the objective of contributing to the development of policies and practices for EML also required synchronisation with the academic activities. For instance in the case of the Province of Lào Cai, its decision to expand the MTBBE approach to several schools required lobbying to assure permission for the respective implementation. A recent event enhancing the policy environment for EML referenced the MTBBE pilot as a good practice regarding teaching-learning methodologies and has also benefitted from UNICEF contribution. This event is the Prime Minister´s Decision on Approval of the School Education Textbook and Curriculum Innovation Scheme (No.404/QD-TTG, 27 March 2015) that adopts a new basic curriculum to align with the education reform and includes consideration of the needs of the most vulnerable children, and a compiling and piloting of a set of textbooks including bilingual textbooks (Vietnamese and EML). The Prime Minister´s decision is timely before decisions that should be taken on the future (scaling up) of the MTBBE.



62



63

4.4.5 Was the programme or project implemented in the most efficient way compared to alternatives?

Compared to the annual per capita expenditure of the Government of Viet Nam in primary education, the MTBBE estimated cost per pupil appears to be higher. However, this is not an exceptional finding compared to international experience, which also document high initial costs of this type of programme. This fact is explained by the initial necessary investment in textbooks development and the in-service training of teachers. Later on, the marginal cost of scaling up initiatives will tend to diminished, particularly if pre-service teacher training becomes the usual alternative. The expenditure on primary education is not the best benchmark to compare the AR MTBBE, since a more precise comparison should focus on the cost-efficiency of at least two similar programmes. Despite the result attained by a more pertinent comparison, the AR MTBBE shows a list of remarkable results that demonstrate its best compensatory side.

In Viet Nam several alternative approaches on EM education have been implemented by different specialised agencies, such as MOET (strengthening of Vietnamese language), JICA (Lesson Study), World Bank (“Escuela Nueva”)90. However, cost data for these different programmes were not available, a fact that hampered a most appropriate contrast. In compensation this section provides a more general comparison. Considering a number of 485 students [270 of them girls] as the final count of two student cohorts leaving primary school in 2014 and 2015, the per capita cost of this particular initiative, non-monetary complementary contributions aside, is estimated to be about US$7,177.00. This amount is for bilingual basic education from kindergarten to Grade 5 in the target schools, which had rural or semi-urban characteristics. This means an average per capita cost of about US$1,196.22 annually or about US$100.00 each month. This cost is higher than the average public per capita spending in primary education in Viet Nam,91 based on 2010 and 2013 data in this regard. According to World Bank statistics, Viet Nam expended US$338.00 per pupil in primary education (25.3% of the per capita GDP) in 2010. Assuming the same percentage, the per capita expenditure in primary education would be about US$483 by 2013. This comparison roughly indicates that AR MTBBE´s cost was relatively high (roughly 2.5 times higher). As one official of RCEME in Ha Noi argued, “The cost is high because this is the first project. Small numbers of participating schools and students lead to high costs. However, if the

90 See footnote 27, page 18. 91

Public expenditure per pupil as a percentage of GDP per capita is the total public expenditure per

student in primary education as a percentage of GDP per capita. Public expenditure (current and capital)

includes government spending on educational institutions (both public and private), education

administration as well as subsidies for private entities (students/households and other private entities

(World Bank, 2015).



64

project continues, there will only be a need to reprint the materials already produced and the costs would be lower. If teacher training is integrated in teacher training colleges, the cost of this will no longer exist”. From a social perspective, costs would be transferred. However, costs would be lowered due to economies of scale. The above argument coincides with international experience. Investigation regarding the cost of bi/multilingual education is categorical in affirming that initial costs are higher than conventional monolingual education. The differences come from EML standardization and the production of educational materials.92 In the case of AR MTBBE, training represented about half of the costs, because the programme paid remarkable attention to quality. According to international research, costs are going to be the same for any language education model in terms of teacher training, if quality of education becomes a real goal.93 In general, the AR MTBBE is a particular pilot programme offering high quality bilingual education to a small group of children located in relatively isolated communities that simultaneously rendered other social benefits.

4.5 Sustainability

4.5.1 Defining sustainability According to OECD DAC, “Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable”. Key questions regarding sustainability are the following: To what extent did the benefits of a programme or project continue after the donor funding ceased? What were the major factors, which influenced the achievement or non-achievement of sustainability of the programme or project?94

4.5.2 Summary on sustainability The general question on sustainability is: To what extent did the benefits of the MTTBE approach continue after the donor funding ceased?

92

Grin, 2003, page 20. 93

Regarding this point, Heugh (2011) argues, “A close examination of the different literacy/language

education models proposed for or in use in African countries and their implication for teacher education

suggests that training costs will be nearly the same whether or not the curriculum uses mother tongue or

official/national languages for instruction. The situation in most countries is such that teachers are not

sufficiently well trained or prepared for the classroom. Curriculum changes are being implemented in

many countries. These two realities mean that there need to be changes in the pre-service teacher

training/education programmes and in the in-service teacher education programmes (if they exist); or that

in-service teacher education programme must be developed (if they do not exist). These provisions

(expenses) are mandatory to ensure the successful implementation of the curriculum” (page 271). 94 OECD Network on Development Evaluation, n.d., page 14.



65

Finding No. 5: The AR MTBBE has created basic favourable conditions for future replication and scaling up of the bilingual programme. In addition, the socio-political climate is also supportive of the scaling up of the MTBBE programme to new schools and other ethnic minority groups in Viet Nam.

The AR MTBBE has created favourable basic conditions for future replication and scaling up of the bilingual programme. In addition the socio-political climate is also supportive of the scaling up of the MTBBE programme to new schools and new ethnic minority groups in Viet Nam. However, for a sustainable scaling up of the MTBBE programme there will be a need to revisit and revise the sub-law papers to allow EML to be a language of learning, together with supportive policies and permanent dialogue between central and provincial governments. The policy guidelines regarding MTBBE replication and scaling up issued by MOET represent a solid progress in terms of regulating and ensuring quality of the programme. Nevertheless, this step appears not to be enough to mobilize all relevant stakeholders to embark on the process. This can be substantiated based on the analysis of the different actions and reactions from the participating provinces. After reviewing the cases of Gia Lai, Lào Cai and Trà Vinh, the message being sent has been that although provincial authorities support the scaling up of MTBBE, the provinces cannot afford the process by themselves. However, beyond cost, other constraining factors could be at play, such as diversity of EM groups in one community or school, and existence of other alternative approaches. Most respondents did not bring these other possible issues into discussion. This may be so because cost was the real problem, or because it conceals the lack of local high commitments, or simply, because the respondents were incapable to perceive other underlying constraining factors. In the case of Lào Cai, the MTBBE pilot generated interest and motivation up to a point that provincial authorities opted for the scaling up of the intervention. This province faces almost the same or worse financial limitation than the others two; nevertheless, the scaling up of MTBBE became mainly supported by the provincial budget. It seems clear that the high political commitment at the local level was the key factor there. Current conditions favour the continuation, and even the scaling up, of the MTBBE in the provinces of Gia Lai and Trà Vinh; in Lào Cai continuation is no longer an issue since the scaling up of the programme already took place. Whether cost or insufficient political commitment or any other issue were a constraining factor in the other participating provinces, dialogue between central and local authorities, and complementary advocacy actions form UNICEF and MOET could make the difference. At the time of this evaluation, the province of An Giang was close to starting an MTBBE pilot. For this purpose this province was learning from the experience of Trà Vinh. However, regarding the expansion of the MTBBE to other areas, similar to the ones of the project sites, more information is needed for a satisfactory answer. Thus, and instead of an answer to the second sub-question, the evaluators propose a set of questions to be employed in future studies about the situation of the EM groups in Viet Nam. The evaluators also propose an interpretative model of factors affecting the process of replication and scaling up. This model, which also constitutes a lesson learned of the intervention, emerged from the study of the pilot´s three participant provinces and could be used as a guide for future research.

A sustainable approach?

Can it continue in the pilot sites? Can it be replicated/ scaled up in areas similar to the project sites?

It would depend of the convergence of favourable conditions

Subsections 4.5.3 to 4.5.5 respond to each of the sub-questions on sustainability by providing evidence that substantiates Finding No.5. Questions regarding lessons learned



66

and about what to do differently in future replications and/or scaling up are answered in Section 5 and 6.



67

4.5.3 Continuation of the programmes´ results in the AR MTBBE sites The first sub-question on sustainability is: To what extent are the results (impacts and outcomes) likely to continue in three AR MTBBE sites when the external support from UNICEF is withdrawn?

All key informants of the three target provinces did mention the high cost of both training teachers (in service training) and educational materials as the constraining factors for the future of the MTTBE. These two factors would be affecting more a scaling up process, but not so much the continuity of the existing project sites. Since continuity conditions are in place, only minor adjustments and support would be necessary. Each province already has a sufficient number of trained teachers for the pilot schools and a body of trained/proven developers of educational materials. It is just a question that local education authorities consolidate the provincial governments´ commitments and economic support to finance monitoring and evaluation activities and the updating, printing and distribution of the teaching and learning materials, where some financial/technical support comes from external sources, like donors or the Central Government. Complementarily, it would be advisable that: (i) UNICEF and other development partners prepare an advocacy strategy addressed to managers of the education field and the provincial and district governments to strengthen the commitment toward MTBBE; (ii) MTBBE teachers could enrol their most competent colleagues in tutorial activities (needed to assure teacher quality improvement) to the extent that the impact on teachers’ skills achieved during the pilot phase could bring the cost of training down; (iii) Provincial/district education authorities take full advantage of the Prime Minister´s recent Decision on Approval of the School Education Textbook and Curriculum Innovation Scheme (404/QĐ-TTg of 27th March 2015) for support on curricular and textbook matters.

“Of course, the biggest challenge is the number of qualified teachers. If we have enough qualified teachers, we will sustain and expand the program. We must also provide sufficient resources and learning materials/textbooks.” Using different words most key informants repeated this opinion of a DOET official in Sa Pa District of Lào Cai. This message emphasizes the importance of qualified teachers and the teaching and learning materials to continue or expand the MTBBE. These inputs, as seen before, have a relatively high cost, particularly at the inception of the programme. How to cope with this cost without UNICEF´s support? For most respondents this was the main concern. In Lào Cai the answer to the last question came as early as 2010, when government provincial authorities took the decision of expanding the number of MTBBE schools.

From 2011 to 2015, each school year, a new group of 210 children aged five will enter the MTBBE classes. At the end of the 2015-2016 school-year in Lào Cai a total of 1,300 students will have been taught through the MTBBE approach in pre-school, followed by MTBBE primary education, providing a model to other provinces committed to improving



68

ethnic minority children´s learning through a bilingual education approach. For this plan, the Lào Cai DOET has estimated that a budget of 8,000,000,000 VND (approximately 205,000 USD) is required.95

The resources for funding the scaling up in Lào Cai came from a combination of the public provincial budget with donors´ support - UNICEF included. Lào Cai took the risk for going beyond the continuity of the pilot´s five classes in three districts (2008 to 2015). This decision has not been free of some internal criticism, as is usual in the case of affirmative action policies addressed to excluded and/or vulnerable populations. What counts is the fact that Lào Cai is showing a path to follow in case other provinces decide to continue or scale up the MTBBE. Despite the fact that Lào Cai has opted for scaling up of the MTBBE96, some local authorities also expressed concern for the future, as they anticipate the end of the donors´ first wave of funding in 2016. Maybe this concern does not consider enough the legacy of the pilot, which left about 200 trained teachers plus 100 Mong teachers whom the Teacher Training College had prepared under a pre-service mode. This represents an accumulated human capital together with the experience achieved in terms of developing educational materials in Mong. Future expansion can take full advantage of these accumulated resources, which will contribute to lowering costs. It is also to be expected that public support (policy-makers, CSO, ONG, donors) for MTBBE may change positively to increase the opportunities of future technical and financing support. In the case of the other two provinces there were no reports of scaling up initiatives under consideration, although the continuation of the actual pilot schools seems to be a more probable decision. As seen, there are encouraging and discouraging factors in effect, such as the ones listed below.

Encouraging factors Discouraging factors 1. Close to 150 teachers of each target province were trained by the programme. They represent an important highly motivated group favouring the continuation and expansion of the MTBBE approach. 2. All stakeholders in the three provinces expressed support for the MTBBE and would like to see it continue and be scaled up. In Trà Vinh Buddhist monks, a respectable leading institution, and the local public university promote conservation and the development of the Khmer language. This university has a strong linguistic department, which has collaborated with the pilot. The aforementioned institutions are

1. The perception that a large number of teachers still need to continue training and coaching to improve their competences. This perception is low in the case of Trà Vinh and higher in the case of Gia Lai. In Lào Cai teachers´ low quality is of high concern. It is a discouraging factor because commitment to a more favourable situation implies cost. 2. The perception that updating and printing learning material have high costs. 3. Important language variations (“dialects”) have produced problems in the quality of the educational materials, although developers think this situation is under control and some future alternatives are discussed.

95 MOET-DOET, 2011, page 6. 96 Currently Lào Cai has 32 classes (5 from the pilot) in operation.



69

potential partners for pre-service and in-service training.

4. In Gia Lai some complaints were manifested against MOET (“More leadership is needed”; “MOET is involved in a political game on MTBBE issues”).

Despite the importance of these discouraging factors, some key informants perceive that continuity, and even expansion of MTBBE, is probable to be achieved as soon as viable solutions to finance the cost can be explored in a dialogue between MOET and provincial governmental authorities. Advocacy support from UNICEF would be advisable as well as its mediation to mobilize funding from donors, which will help provincial governments to cope with the cost of the MTBBE. From the analysis of the three pilot provinces, the evaluators identified the main intervening factors in the development of the pilot MTBBE. [See Figure 8]

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014. FINAL

EVALUATION REPORT

70

Figure 8. Schematic interpretation of the main intervening factors in the development of the AR MTBBE in the three participating provinces

Geographic isolation Local authorities closer to citizens plus other unknown factors

Support from UNICEF – MOET (MTBBE initiative)

LÀO CAI General enthusiasm for the MTBBE approach

Slower process of cultural assimilation

High political will (provincial government authorities)

Search for funds ended in success

Scaling up

Low geographic

isolation

TRIGGER FACTOR: PILOT AR MTBBE

AR MTBBE produces,

TRÀ VINH

Advanced cultural assimilation

Enthusiasm for the MTBBE approach by school and community stakeholders

Unknown factors

High probability for MTBBE continuity

- Awareness about the situation of Ethnic minority children

Buddhist Monks as influencing promoters of cultural traditions and values

Support from Buddhist Monks

Modest political will (provincial government authorities)

- Operationalization of the MTBBE approach

Some hesitation on scaling up

- Greater appreciation of Ethnic minority languages and culture

Moderate geographic isolation

Enthusiasm for the MTBBE approach on the side of school and community stakeholders

Unknown factors

Moderate to high? Probability for MTBBE continuity

GIA LAI Modest political will (provincial government authorities)

Moderate process of cultural assimilation

Some hesitation on scaling up

Discouraging

factors at play

(cost of inputs,

scarcity of good

EM teachers)

Discouraging

factors at play higher than in the

other two cases

Discouraging

factors at play

(cost)

Discouraging

factors at play

(cost, scarcity of

good quality EML

teachers)



71

4.5.4 Extent of capacity for scaling up in sites similar to the pilots The second sub-question on sustainability is: To what extent can the MTBBE approach be replicated and scaled up in areas with characteristics similar to current project sites? The third question on sustainability is: What elements are/are not in place for sustainability? An overlap is unavoidable in answering these two questions; this is why the summary below includes the evaluators´ answer to both sub-questions.

MTBBE has left an important experience on how to implement the MTBBE approach. The early scaling up of the MTBBE in Lào Cai has also come to add value on this matter. Both initiatives are now a reference point for future actions for replication, and have also contributed to the institutional strengthening of MOET at different levels. There is now a relatively high number of trained MTBBE teachers as well as developers of EML educational materials. Add to these a good number of decision-makers, academics, parents and communities that understand and support the MTBBE. MOET´s technical staff now seems to understand the MTBBE approach very well, a favourable fact to help policy-makers in broadening the legal space needed to expand the programme. The education legislation should permit the use of an EML as a language of instruction in all of MOET authorised schools. With the benefit of favourable affirmative-action policies addressed to ethnic minorities some financial obstacles could be overcome. In addition, the lack of sufficient information for decision-making in EML affairs requires attention. In this regard, UNICEF and donors could consider supporting the Primary Classroom Map of Languages in interested/targeted provinces. Finally, potential threats that may arise in the course of the coming years are to be considered, such as those coming from an eventual economic downturn or political setback, which could cut public resources addressed to social programs. In sum, the MTBBE approach can be replicated in areas with similar characteristics to existing project sites as soon as policy, will and viability come together. In this context, viability means a situation that meets the requirements set by MOET to authorise new MTBBE initiatives, which include warranty for financing, that in turn, implies the support of educational and government authorities at the provincial level. Primary Classroom Maps of Languages could be expected to produce an enhanced assessment on viability.

To what extent can the MTBBE approach be replicated and scaled up in areas with characteristics similar to current project sites? For a satisfactory, well-documented answer, this sub-question requires a good deal of general and detailed information about the situation of the EM populations in Viet Nam. Most of this information was not readily available to the evaluation team; in fact, there is a lack of information about ethnic minorities, “from how they are classified by ethnic group to where they are located in the



72

country to their levels of economic [status]”97. The World Bank estimates 54 ethnic minority groups, 65% of these persons live in two regions, the Northern Mountains and the Central Highlands.98 Due to this limitation of data, a satisfactory answer to the sub-question is not possible. Instead, among others, the initiation of studies regarding the following questions is recommendable:

• What characteristics of the pilot areas are key determinants/pre-conditions for the replication/scaling up in other areas?

• What is the linguistic situation of those areas? Which would be the target mother tongue of each area? What is the predominant EML? With which other languages does the predominant EML coexist?

• Which of these areas meet the criteria MOET requires to approve new MTBBE initiatives?

• What would be the estimated cost for the training of teachers and how should they be trained (pre-service, in-service)?

• Which provincial and district governments have similar characteristics to those of the pilot sites?

• What is the financial capacity of these provincial governments to meet all or part of the costs required to sustain an MTBBE programme?

• What advocacy actions would be needed to win support from the target communities and the key provincial/local stakeholders?

• How could those participant communities/schools and other key provincial/local stakeholders keep the program strong, and how could they mitigate their weaknesses?

The answers to the above questions would determine the magnitude of the challenge of expansion in terms of inputs, required actions and costs. It is possible that a single study would not be enough to respond satisfactorily to the above questions. In this case, a successive approximation to the problem would be recommendable; that is, starting with previously identified specific areas, identify priorities, as, for example, starting with known high-ethnic-minority-population density areas, as in the case of the pilot program. Figure 9 offers a model, based on the experience of the MTBBE pilot, which could be useful for guiding future research on this matter. The model identifies factors that could be at play in the process of replication/scaling up. These factors are the following: local authorities commitments and or local context, cost of inputs/actions, availability of funding, and capacity of the targeting/requesting provinces to meet MOET´s criteria for authorizing the functioning of an MTBBE programme. Athorities commitments is considered the main explanatory factor to make scaling up possible. Authorities commitments will also facilitates the access to funding and contributes to reduce some 97 The World Bank, 2009, page 51. 98 The World Bank, 2009, page 1-2.



73

cost by different means, particularly those regarding policies and direct public interventions (e.g. public works). In turn, political will is the result of awareness of cultural and social rights, visibility of the target EM group, among other factors. Higher cost is considered to constrain scaling up. In contrast, the availability of funds facilitates scaling up. The convergence or synergy of facilitating factors provides better guarantees for scaling up to become a reality.

4.5.5 Elements in place/not in place for sustainability The third sub-question on sustainability leads to the identification of in place/not in place elements for sustainability that the evaluation team summarises by means of a SWOT analysis (strengths, weaknesses, opportunities, and threats). This analysis was also useful to design the model presented in Figure 9. The evaluation team developed this analysis on the basis of the available testimonial (the most) and documentary information. [See Table 4].

Table 4. SWOT analysis of implied factors on the sustainability of MTBBE scaling up

STRENGTHS WEAKNESSES

+ The Pilot MTBBE has proven its learning advantages. Now it is a matter of replication and scaling up in areas with characteristics similar to existing project sites. + Strong stakeholder agreement on MTBBE learning potential. + Parents/communities, local key education and political actors support the MTBBE programme (in the current target provinces). + Favourable policy environment supporting proactive education and general social policy for EM children/ communities. + MOET has adopted MTBBE as one solution to meet the educational problems of the EM population. + MOET has policy guidelines to regulate the scaling up of the MTBBE99.

- MOET´s central-provincial/district dialogue seems to lack enough fluency for overcoming negative/ill-informed perceptions on policy and financial support for MTBBE scaling up. - MOET´s support is limited to technical assistance. - Few but important local education authorities still need reinforcement to increase their awareness of the social importance and functionally (e.g. smooth transition in language) of MTBBE. - There is limited guidance and implemenetion of aggressive affirmative action policy for ethnic minority populations, particularly on the regular (not only pilot) use of MT as a language of learning and the (extra) financing of the respective initiatives. - There is insufficient good linguistic mapping to estimate better the potential magnitude of MTBBE demand (e.g.

99 Local scaling up will be authorized under the following conditions or criteria: (i) the language of the

ethnic minority group should have a written system; (ii) the ethnic minority group has to live in a specific

area, not a dispersed one; (iii) there should be a sufficient number of teachers who can speak the EML to

be used in schools; and (iv) there should be available teaching and learning resources to implement the

programme. The local governments will decide on the implementation. The DOETs are supposed to

develop the plans and submit them to the provincial plan. It is up to the provincial authority to approve



74

+ Recent curricular reform allows pre-service training of teachers on MTBBE. + An important number of policy makers (National Assembly) and decision makers (Central and local governments) are aware of the importance of MTBBE as an instrument for successful education/social inclusion of ethnic minority population. This might lead to an even clearer policy for MTBBE promotion and financing (i.e. education policy within a better defined affirmative action policy framework). + MOET´s leadership, ownership, and technical capacity to lead MTBBE scaling up have been greatly reinforced.

languages/communities meeting MOET´s requirements for authorizing MTBBE initiatives) and already available resources (e.g. MT speaking teachers; local institutions able to support MT development). - Provincial governments are heterogeneous on financial matters. Generally the ones with more financial problems are also the ones with the highest proportions of EM population.

OPPORTUNITIES THREATS

+ Viet Nam continues growing at a relatively high pace. Fiscal space for social expenditures and transfers for provincial governments may grow accordingly. + Clear awareness that growing in the global economy demands great investment in human capital. Pro-reform education environment. + Clear awareness of the need for increasing efforts on development equity and inclusiveness. Different social gaps need to be filled; there are ethnic disparities among them. + Policies on decentralisation have been favouring province/district´s financial and decision-making autonomy, although there remain challenges to be met. + Provincial governments can make their own decisions on education policies, although aligned with the Education Law, which is becoming amicable to some cultural curricular adaptations. + Donors are ready to support efforts

- Slowdown of the actual growth rate could turn into a reduction of the fiscal space for social programmes and transfers to provincial governments. - The development of reactive political pressures to maintain Vietnamese as the exclusive language of learning. - The political atmosphere becoming unfavourable in some provinces to scaling up the MTBBE because of cost and/or misunderstanding of affirmative action policies favourable to ethnic populations. - Donors´ resources become insufficient to meet most provincial demand for scaling up MTBBE.

that plan, or not. It is expected that the Department of Childhood will advocate for the implementation of

the MTBBE in the provinces. Source: Evaluators identified these conditions from interviews with several

key respondents of MOET. The latter warned that these were working, still-under-discussion criteria.



75

aligned to the Nation´s education priorities (e.g. EFA). + Donors are allowed to direct financing of priority local initiatives.



76

Figure 9. Path of factors involved in the replica and scaling up of the MTBBE in areas with characteristics similar to existing project sites

Socio-economic conditions

Cost of inputs/actions

Linguistic conditions of the target area

Meeting MOET´s standards for authorizing MTBBE replication/scaling up initiatives

Replication/Scaling up

Awareness on cultural and social rights

Political will/context

Visibility of the EM group

Funds availability (public, donors)

Other factors

(+) Direct relationship. E.g. more availability of funds facilitates a scaling up process; authorities

commitments (or local context) facilitates a scaling up process.

( - ) Indirect relationship. High costs constraint a scaling up process

Relationships exist in a qualitative sense, not in a statistical sense.

(+)

(+)

(+)

(+)

(+)

(-)



77

5.0 LESSONS LEARNED The lessons summarised below are grounded in the general findings and conclusions of this evaluation but also respond to the following specific evaluation questions: What worked well and deserves continuation? What needs improvement or be done differently? The approach adopted in the identification of the lessons learned takes into account the prospects of replication and scaling-up of the programme in Viet Nam but also considers the relevance of those lessons beyond the Vietnamese context.

Identification of lessons learned

Lesson 1: Design of the MTBBE model A thoughtful design is a road for success. The design and the flexible approach to MTBBE adopted in the pilot is a model of good practice to be replicated in Viet Nam and elsewhere. By linking pre-school and primary school levels and allowing a flexible use of Vietnamese and EMLs, the design adopted facilitates the development of both the students’ mother tongues and second language right from pre-school, thus preparing them to learn content subjects in either language of schooling. The design offers important conditions for the development of bilingual/multilingual, bi-literate/multi-literate and bicultural/multicultural citizens. All these attributes make the design adopted superior to transitional models of MTBBE commonly adopted in developing countries.

Lesson 2: Teachers’ capacity Teachers’ capacity is key to a successful implementation of an education programme. Intensive and regular in-service training adopted in the pilot phase allowed the development of language skills and the understanding of the bilingual teaching methodologies, including the flexible bilingual education. Teachers’ enhanced capacity has an impact on their confidence and self-esteem, which are essential for a conducive teaching and learning environment. The fact that MTBBE students reach the required national standards by the end of grade 5 can, in great deal, be attributed to quality teaching capacity offered in the programme. This indicates that investment in teachers´ capacity building is a necessary condition for a successful implementation of MTBBE programmes, in particular in contexts where there is no (substantial) tradition of using vernacular languages in education.

Lesson 3: Control of the pilot programme A pilot programme needs to keep rigorous control of both participating and non-

participating settings. MOET, UNICEF and other partners did not face the pressures coming from an unplanned increase of the participating



78

classes/schools/communities, as happens in some other international cases (e.g. Mozambique 2003-2010) and its negative consequences on the quality of the process (e.g. the same funds for more participants). Lào Cai’s decision on the MTBBE scale-up did not take MOET by surprise. The process was considered an expansion of the pilot but with its own structure and funds. MOET’s management of the MTBBE has become a benchmark for conducting a social experiment, like the AR MTBBE.

Lesson 4: Decentralization of programme implementation The involvement of local actors in the planning and implementation of MTBBE programmes contributes to capacity building of teachers and local managers and fosters ownership, which are important preconditions for a sustainable implementation of any programme after the pilot phase. Lào Cai’s decision to scale-up the programme well before the end of the pilot stage can be regarded as an evidence of the grassroots´ appropriation of MTBBE, which started as a top-down initiative. This is facilitated in contexts where a decentralized approach to governance is adopted, as is the case of Viet Nam.

Lesson 5: Monitoring and evaluation The Action Research model as a monitoring tool fosters collaborative identification of problems and local solutions in a timely fashion. The involvement of local and central level actors in action research activities allows a good flow of information and ownership of the programme. Therefore, the Action Research model followed in the programme is more than a fashionable name, it is a competence for self and team learning and a trigger of ownership. Documented regular assessment of MTBBE students’ learning outcomes allows the production of accurate and reliable data necessary for comparisons with national standards and strengthens the evidence needed for advocacy actions promoting the use of vernacular languages.

Lesson 6: Commitments and engagement at all levels Commitments at all level is crucial for success, but it has to be fostered through advocacy and clear benchmarks. Commitments at all levels and decision takers need help in understanding unknown approaches as well as evidence that the approach produces outstanding results. This fosters policy makers’ commitment and support to innovative programmes.

5.2 What could or should be done differently in future replications and/or scaling up?



79

Lesson 7: Design of the MTBBE AR The lack of a Project Initiation Document with an explicit theory of change constrains monitoring and evaluation of any project since it becomes hard to generate performance indicators and contrast planned activities versus success. Programme goals, chain results and performance indicators should be clearly defined at the preparation phase.

Lesson 8: Teacher training Reliance on intensive in-service training makes capacity building very costly, as recurrently attested internationally. The resources channeled to in-service training could be invested in other components of the programme, including the development of reference and reading materials. Although a focus on in-service training may be unavoidable in a pilot stage of a programme, the findings of this evaluation and international experience suggest that for a large and sustainable implementation of innovative education programmes pre-service training should have a more prominent role. This should not mean overlooking in-service training, which is also crucial for continuous professional development of teachers and education managers at all levels.

Lesson 9: Definition of reference language varieties The language varieties used as reference in MTBBE programmes should be carefully selected based on scientific criteria, including consultation with beneficiary communities, before their actual implementation. Although this was generally done in the preparation phase of the pilot programme, during the implementation in some cases there was a need to change the reference varieties used in teaching and learning material because communities did not feel comfortable with the selected varieties, as happened in Gia Lai. Communities may opt out of MTBBE programmes just because they do not identify themselves with the language varieties used in schools, so due attention should be paid to this issue.

6.0 CONCLUSIONS The main overall conclusion of this evaluation is that the AR MTBBE has proved to be a valid and feasible education approach for the context of Viet Nam, as evidence shows relevance, effectiveness, impact, efficiency, sustainability, and contribution to the objectives of UNICEF Viet Nam regarding gender and equity. The specific conclusions listed below are based on the main findings of the evaluation and, in turn, they are the base for the recommendations presented in the next section. [See Annex 10 Relationship among findings, conclusions and recommendations] Project Design



80

Conclusion No.1: The lack of a PID constrains the evaluation of any project since it becomes hard to contrast planned activities versus success as well as to generate performance indicators. Although the MoU and the Implementation Framework of the AR MTBBE offer clues for generating performance indicators, the fact that they lack precision makes it difficult to make accurate judgements about the overall success of the programme. An important exception on this matter was the learning outcomes, for which the programme presented periodical assessments based on indicators employed by the education authorities and specialists world-wide. The lack of a Theory of Change also limited its use as an instrument for planning and monitoring the process from inputs to outcomes/impacts and with this the value added of the programme to the development goals of the country and/or UNICEF. Relevance Conclusion No.2: The MTBBE approach translates international and Vietnamese legal and policy frameworks for a more inclusive and equitable educational system and is relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. However, the fact that the existing policies and legal frameworks in Viet Nam are not explicit about the use of EMLs as languages of teaching and learning and the absence of a specific policy on how MTBBE should be implemented may constrain the expansion and scaling up of the programme, as expressed by key stakeholders, in particular those from the local level. Conclusion No.3: The MTBBE approach has proven its potential to respond to educational needs of ethnic minority groups as well as its potential to contribute to the preservation and revival of ethnic minority languages and cultural practices and beliefs. However, the evaluation suggests that differences in terms of historical trajectories and language attitudes and practices in Viet Nam may lead to different perceptions about the relevance of the approach. Conclusion No.4: As international experience has shown, this evaluation also indicates that bilingual education should not just be viewed as a response to the learning challenges faced by ethnic minority children. It has also to be seen from a broader perspective, linked, for example, with efforts for inclusiveness, cultural preservation and improvement of the living conditions of ethnic minorities in Viet Nam. The rationale here is that the same way children perform better in school because a language familiar to them is used as medium of teaching and learning, ethnic minority people in Viet Nam can be better served and participate more effectively in social, economic and political life if their languages are legitimately used as vehicles of communication in official arenas. Effectiveness



81

Conclusion No. 5: The MTBBE programme was based on an effective curriculum design, one that allowed the development of both the students’ mother tongues and Vietnamese and offered optimal conditions for transfer of knowledge from one language to the other. Taking into account international experience and the results of this programme, it can be said that the design adopted has the potential to produce bilingual and bi-literate students, that is, those who can speak, read and write in both Vietnamese and their mother tongues as well as facilitate learning of content subjects in both languages of schooling and set the conditions for the maintenance of ethnic minority languages. Conclusion No. 6: The pilot phase of the programme benefitted from regular in-service training and support that gave MTBBE teachers and education managers the opportunity to improve their professional capacity. Despite the overall effectiveness of those initiatives, reports of misapplication of flexible teaching methodologies and insecurity in the teaching of Vietnamese as a second language suggest that capacity building should continue to be one of the top priorities of programme implementation, in particular at the scaling up phase. Indeed, international experience indicates that capacity building is key for a successful implementation of MTBBE programmes, in particular in contexts where there is no long tradition of using local languages in education, as it is the case with some EMLs in Viet Nam. Conclusion No. 7: Teaching and learning of Vietnamese as a second language is continued to be taken as one of the solutions addressing language challenges for young ethnic children in Viet Nam. However, as international experience has shown second language teaching-learning methodologies are not necessarily in conflict with the MTBBE approach. Second language teaching-learning methodologies should be applied in MTBBE-MLE as long as they draw on approaches that promote the use of bi/multilingual resources in classroom interaction as a legitimate communicative and pedagogical strategy. Impact Conclusion No.8: The overwhelming positive academic results obtained by ethnic minority children, with special reference to girls, and its consistency with national standards confirm the pedagogical value of an education programme based on learners’ mother tongues. The familiarity with the language of instruction makes EM children feel at ease in the classroom and creates the conditions for their active participation, which are important preconditions for learning. The positive results achieved in Vietnamese also show that, instead of hindering, MTBBE facilitates the learning of the second language, the language regarded as the gateway to social integration and mobility in Viet Nam. The general conclusion is that MTBBE has the potential to contribute to mend learning outcome gaps between Kinh and EM children thus paving the way towards inclusion and equity in education and in the Vietnamese society at large.



82

Conclusion No. 9: Regular in-service training and support had a significant impact on teachers and education managers. These initiatives led to improvements in teachers’ professional capacity and prompted positive attitudes towards teaching, in particular on the side of ethnic minority teachers; hence their heightened self-esteem and confidence. By involving education managers in training initiatives, the programme created the conditions for an effective management of bilingual schools as they not only got acquainted with MTBBE philosophies and methodologies but also became more sensitive to the challenges faced by ethnic minority children in contexts where they are educated solely in Vietnamese. The technical preparedness of education managers, in particular those from the local level, is an important condition for the sustainability of the programme and for bottom-up initiatives promoting the use of EMLs in education, as happened in Lào Cai. Conclusion No. 10: While recognizing the sociocultural value of using EMLs in formal education, parents and other community members were initially unsure about the pedagogical value of MTBBE, in particular in creating opportunities for effective learning of Vietnamese. The fact that their attitudes and perceptions changed over the course of the programme, especially when they perceived substantial academic results exhibited by participating students, suggests that the achievement of positive educational and sociocultural outcomes may prove to be the best way to advocate for MTBBE. In other words, while drawing on international experiences, advocacy initiatives should involve key actors (students, teachers, parents, education officials, etc.) and use the results from the pilot phase as the main arguments in favour of the MTBBE. Efficiency Conclusion No.11: As international experiences have shown, and also recognized by key participants in this evaluation, the expenditure of the pilot programme was higher than the average per pupil expenditure in primary education in Viet Nam because of its pilot nature. The reliance on regular in-service training, the innovative effort placed on material development and the small size of the student population served are the main factors which explain the relatively high costs incurred in the pilot phase. However, the positive results obtained, including the improvement of learning outcomes of ethnic minority students and change of the mind-set of education managers regarding the role and value of EMLs in education indicate that the MTBBE is a worthwhile investment. In fact, as attested internationally, in a post-pilot phase, the cost of the programme is likely to be lower and similar to that of the mainstream programme in Vietnamese. Conclusion No.12: The availability of basic and good quality teaching and learning materials is a necessary precondition for a successful implementation of any educational programme. The evidence gathered suggests that this precondition was, in general, fulfilled in the AR MTBBE. However, in addition to very basic materials, it is also necessary to furnish bilingual schools with complementary material to aid teaching and learning, in



83

particular in EMLs. As attested internationally, when students, teachers and parents understand that education programmes in low status languages are poorly equipped in terms of teaching and learning aids (and also in terms of quality of the teaching staff) when compared to programmes in the mainstream language, as it is the case of Vietnamese, they may opt out of those programmes. This is to be avoided, in particular considering the equity agenda underlying the provision of education in EMLs. Sustainability Conclusion No. 12: As international experience has shown, in addition to technical preparedness, political commitment at all levels is one of the key factors for a successful and sustainable implementation of MTBBE initiatives. Although, in general, these conditions are in place in Viet Nam, there are some other factors which may threaten the sustainability of the programme if not adequately addressed. These include mainly claims of lack of financial resources to implement the programme in some provinces and the lack of a clear policy on MTBBE in Viet Nam. In fact, stakeholders generally perceive the need of financial support for teacher training and material development, especially at the district and provincial levels, as the main challenge for a sustainable replication and scaling up of the programme. Other less voiced constraining factors include the difficulty in managing language diversity in some provinces and the existence of alternative approaches to education of ethnic minority children, especially the teaching of Vietnamese as a second language, usually regarded as less expensive and less demanding. Gender and equity Conclusion No. 13: The MTBBE approach has also proven to be relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. By helping ethnic minority children to overcome the language barrier in school, MTBBE can be regarded as one of the platforms that can be used to improve the academic performance of these children and set the ground for official use of EML in other formal settings, including in the health and legal sectors. As a part of the equity efforts, the MTBBE made special efforts to have good participation of girl students. The programme achieved this objective well since girls came to represent a small majority of the total enrolment. The evidence indicates that girls’ academic performance was as good as that of boys.

7.0 RECOMMENDATIONS The recommendations are based on the findings and conclusions of this evaluation. These

recommendations are intended to inform the design and implementation of the on-going

and future MTBBE programmes in Viet Nam. They are chiefly directed to MOET and its



84

relevant departments, as the policy proponents and implementers of education

programmes. A few recommendations are also directed to UNICEF, as the advocate and

the main supporter of the implementation and management of the AR MTBBE, and to

other development partners. The recommendations were developed in close consultation

with key stakeholders, including UNICEF, MOET and local level education authorities, who

provided their feedback to debriefings and earlier versions of the evaluation report. They

are not listed in any priority order, given the close interconnection between them, but

thematically organized based on the evaluation criteria.

Project Design Recommendation No.1: Develop a PID, including a theory of change, for the next programme delivery, clearly defining performance indicators to be used in the management and assessment of its overall success. This should be part of any programme/project initiative. Responsibility: UNICEF and MOET. Relevance Recommendation No. 2: Review the existing relevant Vietnamese legal and policy frameworks with the view to strengthening them clearly towards MTBBE. Responsibility: MOET, in coordination with the CEM- National Assembly and with the support of UNICEF. Recommendation No. 3: Show strong political will to introduce mother tongue learning by establishing a specific policy laying down how the MTBBE approach should be implemented in Viet Nam. Responsibility: MOET. Recommendation No. 4: In the current context of curriculum and general education reform in Viet Nam, incorporate MTBBE as one of the solutions to improve quality and equity for ethnic minority children in this country.

Responsibility: MOET leading the advocacy component, with the support of UNICEF.

Recommendation No. 5: Conduct careful sociolinguistic studies before implementing the MTBBE approach in new ethnic minority groups in Viet Nam. This may give indications about the relevance and the likelihood of success of the programme in those settings.



85

Responsibility: MOET, in coordination with UNICEF in the search of resources and advocacy. Recommendation No. 6: Develop a strategic advocacy/communication plan to ensure that relevant institutions issue policies on the use of EML in other official arenas, including in basic service delivery sectors such as health, child protection and legal. Responsibility: UNICEF, MOET and other institutions promoting social inclusiveness in Viet Nam. Effectiveness Recommendation No.7: In the scaling up phase, replicate the design and the flexible approach adopted during the pilot stage as they yielded a strong form of bilingual education, which is recommended for the socio-cultural context of Viet Nam. This design should form part of the future policy on MTBBE in Viet Nam. Responsibility: DOETs and MOET. Recommendation No.8: As part of the preparation of the scaling up phase, organize a seminar/workshop aimed at analysing and reviewing the teaching-learning methodologies used in the MTBBE to recommend adjustments to make them more responsive and appropriately applied. Responsibility: MOET and UNICEF. Recommendation No.9: Adopt multilingual-oriented second language methodologies in the teaching and learning of Vietnamese as part and parcel of the MTBBE approach. This may include the involvement of NGOs promoting the teaching and learning of Vietnamese as a second language in Viet Nam (e.g. Plan International) in training workshops/continuing professional development programmes for MTBBE teachers. Responsibility: MOET leading the process, with the support of UNICEF. Impact Recommendation No.10: Conduct (longitudinal) studies of former MTBBE students’ performance at secondary school in order to assess the educational impact of MTBBE beyond primary education level. The results of those studies will provide further context-



86

specific evidence about the feasibility and validity of the MTBBE for ethnic minority students in Viet Nam. Responsibility: MOET (RCEME/VIES), in coordination with UNICEF in the search of resources and advocacy. Recommendation No.11: Develop a Behaviour Change Communication Strategy (campaign) to raise awareness of parents and communities on the purpose and benefits of the MTBBE and ensure their support and participation as well as to mobilize collective action in support of the programme.

Responsibility: MOET, in collaboration with UNICEF.

Efficiency Recommendation No.12: Scale up the MTBBE programme, involving more schools and EM students, thus maximizing the academic and sociocultural benefits of the approach and reducing its costs by capitalizing on economies of scale. Responsibility: MOET (DOETs, RCEME, VIES). Recommendation No.13: Consolidate all reference materials (i.e. teaching guides, grammars, dictionaries, additional reading and reference books, and visual aids) in the list of basic materials to be produced and supplied to MTBBE schools.

Responsibility: MOET (DOETs, RCEME, VIES). Recommendation No.14: Mobilize independent writers and publishers to produce low-cost and/or no-cost reference materials in EML for MTBBE schools; this may include award offers and promotion of contests and competitions culminating in the publication of the best manuscripts, including on-line publications.

Responsibility: MOET (DOETs, RCEME, VIES).

Sustainability

Recommendation No.15: Develop a strategy for a gradual expansion and scaling up of the

MTBBE programme. This should have two goals: (a) the continuity and dissemination of

MTBBE in the three pilot provinces; and (b) the design and implementation of new MTBBE

interventions in provinces with characteristics similar to those of the provinces involved in

the pilot stage or in other provinces fitting MOET’s criteria.

Responsibility: MOET, in coordination with UNICEF and relevant DOETs.



87

Recommendation No.16: Continue to provide financial support for teacher training and development of teaching and learning materials to the experimental provinces for three to five years while they will continue to find other donors to sustain the programme.

Responsibility: MOET, UNICEF and other donors of the education sector in Viet Nam.

Recommendation No.17: Provide support to teachers for them to effectively deliver a

multi-language policy by ensuring pre- and in-service training which lead to the

improvement of their own proficiency in the language of instruction and in teaching skills

in ethnic minority languages, including early literacy development in these languages.

Responsibility: MOET.

Recommendation No.18: Establish and maintain a dialogue with relevant provincial

authorities in order to clarify responsibilities and find solutions for the financial needs of

local governments to sustain current MTBBE efforts and results and set the bases for the

expansion and scaling up of the programme in those provinces.

Responsibility: MOET and UNICEF.

Recommendation No.19: Bring language of instruction to the forefront of dialogue on

education sector funding with the government partners, particularly in relation to both

quality and equity concerns, ensuring the coverage of MTBBE is highlighted as

international indicator of education quality.

Responsibility: UNICEF and other development partners.

Recommendation No.20: Mobilize resources to support the introduction of MTBBE and

offer technical assistant to develop flexible competency targets, literacy materials and an

assessment framework in multiple languages.

Responsibility: UNICEF and other development partners.



88

REFERENCES Al Murshidi, G. (2014). The Impact of Student-Centered Learning Approach Through Workshops Conduction on the UAE University Female Students´ Confidence. European Journal of Business and Management Review. Vol 2 , 31-43 (September). Baker, C., & Jones, S. (1998). Encyclopedia of Bilingualism and Bilingual Education. Clevedon, Toronto, Johannesburg: Multilingual Matters. Freeman, R. (1998). Bilingual Education and Social Change. Clevedon, Toronto, Johannesburg: Multilingual Matters. Benson, C. (2004). Bilingual Schooling in Mozambique and Bolivia: From Experimental to Implementation. Language Policy (3), 47-66. Benson, C. (2006). Final Report Consultancy in Bilingual Education. Presented to UNICEF VN. Hanoi: (December 15). Benson, C. (2007). Final Report. Second Consultancy in Bilingual Education. Hanoi: (February). Benson, C. (2008). Final Report. Fourth Consultancy in Bilingual Education for UNICEF VN. Hanoi: Final Report. 25 March. Benson, C. a. (2013). Language Issues in Comparative Education. Inclusive Teaching and Learning in Non-Dominant Languages and Cultures. (S. Publishers, Ed.) Rotterdam. World Bank. (2009). Country Social Analysis: Ethnicity and Development in Viet Nam. Washington D.C.: The International Bank for Reconstruction and Development/The World Bank. Burton, L. (2013). Mother Tongue-Based Multilingual Education in the Philippines: Studying Top-Down Policy Implementation from the Bottom Up. Ph. Dissertation, University of Minnesota . Dutcher, N. (2004). Expanding educational opportunities in lenguistically diverse societies. Washington: Second Edition, Center for Applied Linguistics. Edmondson, J., & Gregerson, K. (2007). The languages of Viet Nam: Mosaics and Expansions. Language and Linguistic Compass , 1 (6), 727-749. Education for All Program. (2012). Viet Nam EFA Action Plan 2003-2015. Review and update 2012. Hanoi.



89

Evaluation, O. N. (n.d.). Evaluating Development Co-Operation Summary of Key Norms and Standards. Second Edition. Gardner, R. (1994). On Service Teacher Education. En Second International Encyclopaedia of Education (pág. Vol. X). Pergamon Press. Government of Viet Nam. (2013). Millenium Development Goals Full Report 2013. Achievements and challenges in the progress of reaching Millenium Development Goals of Viet Nam. Hanoi: New Technology Printing Joint Stock Company. Gouleta, E. (2006). Language Learning and Linguistic Policy in Education: Considerations for Successful Bilingual Programs in Madagascar. Washington: Desk study commissioned by the World Bank. Grin, F. (2003). The Economics of Language Policy Implementation: Identifying and Measuring Cost. . En A. (. Neville, Mother Tongue-Based Bilingual Education in Southern Africa: The Dynamics of Implementation. Cape Town: Proceedings of a Symposium held at the University of Cape Town Wolkswagen Foundation and PRAESA. Hartmann, A., & Linn, J. (2007). Scaling Up: A Path to Effective Development. Washington: International Food Policy Institute, Special Edition, www.ifpri.org/sites/default/files/publications/oc63ch44.pdf. Heugh, K. (2011). Cost Implications of the Provision of Mother Tongue and Strong Bilingual Models of Education in Africa. En A. a. Ouane, Optimizing Learning, Education and Publishing in Africa - The Language Factor. A Review and Analysis of Theory and Practice in Mother Tongue and Bilingual Education in Sub-Saharan Africa (págs. 255 - 288). Hamburg and Tunisia: UNESCO - ADEA. Martinez, L., & Hinojosa, M. (2012). Parents, Perceptions and Attitudes for Denying Bilingual Education. Focus and Colleges, Universities and Schools , 6 (1), 1-8. Minority Rights Group International and UNICEF. (2009). State of the World´s Minorities and Indigenous People 2009. London. MOET-UNICEF. (2009). Programme Brief. Action Research on Mother Tongue-Based Bilingual Education. Creating Learning Opportunities for Ethnic Minority Children. Hannoi. MOET-UNICEF. (2011). Education Programme Brief. Action Research on Mother Tongue-based bilingual education: Achieving quality, equitable education for ethnic minority children. Hanoi (February).



90

MOET, SIL International and UNICEF. (2012). Lào Cai Primary Classroom Language Mapping. Hannoi. MOET-UNICEF. (2012). Action Research on MTBBE: Improving the equity and quality of education for EM Children in Viet Nam. Hanoi: (September). MOET-UNICEF. (2013). Action Research 2013 Annual Assessment Report. Hannoi: (June). MOET-VIES-RCEME. (2014). Report. Summary of MTBBE Research Action. First Student Cohort (2008-2014) and Academic Year 2013-2014. Hanoi. Morra, L. a. (2009). The Road to Results. Designing and Conducting Effective Development Evaluations. Washington DC: The World Bank. OECD DAC. (2002). OECD DAC Glossary of Key Terms in Evaluation and Results Based Management. Paris: OECD DAC. OECD Network on Development Evaluation. (n.d.). Evaluating Development Co-Operation Summary of Key Norms and Standards. 2nd. Edition. OECD DAC. (2010). OECD DAC Glossary of Key Terms in Evaluation and Results Based Management. Paris: OECD DAC. Ricento, T., & Hornberger. (1996). Unpeeling the Onion: Language Planning and Policy and the ELT Professional. TESOC Quarterly , 30 (3), 401-427. The Comunist Party of Viet Nam. Socio Economic Development Strategy 2011-2020. En w. V. translation (Ed.), Eleventh Congress of the Communist Party of Viet Nam. Hanoi. The Danish Education Network. (2007). Mother Tongue and Bilingual Education. A collection of Conference Papers. Conference on Mother Tongue and Bilingual Education. Copenhagen: November 28-29. The United Nations in Viet Nam. (2003). Millennium Development Goals. Closing the Millennium Gaps. Hanoi. The World Bank. (2009). Country Social Analysis. Ethnicity and Development in Viet Nam. Summary Report. Washington. The World Bank. (2015). World Data. Recuperado el 25 de January de 2015, de http://data.worldbank.org/indicator/SE.XPD.PRIM.PC.ZS/countries



91

Thomas, C. (2009). A positively plurilingual world: promoting mother education. En M. R. International, State of the World´s Minorities and Indigenous People (págs. 82-91). London. University of Minnesota. (12 de January de 2015). The Human Rights Library of Minnesota. Obtenido de Ratification of International Human Rights Treaties: http://www1.umn.edu/humanrts/research/ratification-vietnam.html Truong Huyen, C. (2011). Eliminating Inter-Ethnic Inequalities? Assessing Impacts of Education Policies on Ethnic Minority Children in Viet Nam. Oxford: Young Lives, Department of International Development, University of Oxford. Truong, B. (2014). Fiscal Decentralization in Vietnam: Outcomes and the Way Forward. National Institute for Finance, Ministry of Finance. Phnom Penh: Asian Regional Seminar on Public Financial Management. UNDP. (2011). Social Services for Human Development. Viet Nam Human Development Report 2011. Hanoi: Inis Communication. UNDP. (2014). Human Development Report 2014. Sustaining Human Progress: Reducing Vulnerabilities and Building Resilience. New York: PBM Graphics. UNICEF Viet Nam. (2008). Action Research on Mother Tongue Based Bilingual Education for Ethnic Minority Students in Viet Nam. Social Initiative (IKEA) Project Proposal. Hanoi. UNESCO . (2008). Improving the Quality of Mother Tongue-Based Literacy and Learning. Case Studies from Asia, Africa and South America. Bangkok: UNESCO Asia and Pacific Regional Bureau for Education. UNICEF. (2010). Narrowing the Gaps to Meet the Goals. UNICEF. (2013). Global Evaluation Reports Oversight System (GEROS). n.d.: n.d. Law Amending and Supplementing a number of articles of the Education Law. The National Assembly No, 4. 2. Vandermoortele, M., & Bird, K. (2010). Viet Nam´s Progress on Economic Growth and Poverty Reduction: Impressive Improvements. London: Overseas Development Institute. World Bank. (2014). http://www,worldbank.org/en/country/Viet Nam/overview.



92

ANNEXES Annex 1 – Terms of Reference Annex 2 – Project overview Annex 3 – Evaluation Design Matrix Annex 4 – List of people consulted Annex 5 – Additional tables and graphs regarding the effects of the Action Research MTBBE on students´ learning outcomes Annex 6 – Selected testimonies on different issues Annex 7 – AR MTBBE Detailed Financial Information Annex 8 – Systematised list of benefits of the Action Research MTBBE Systematised list of benefits of the Action Research MTBBE Annex 9 – Additional information for the SWOT analysis Annex 10 – Relationship among findings, conclusions and recommendations Annex 11 - Questionnaires



93

Annex 1 – Terms of Reference



94



95



96



97



98



99



100



101



102



103



104



105



106



107



108

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in

Viet Nam, 2006 – 2014. FINAL REPORT

109

Annex 2 - Project Overview 1. Background, Purpose and goal100

1.1 Background Viet Nam is an ethnically diverse society with 54 officially recognized ethnic groups, many of which have their own distinct language and live in remote and economically disadvantaged areas in the country. The total ethnic minority population is approximately 12.5 million, representing 14 per cent of the total population of about 90.5 million. Vietnamese is the official language of instruction in schools, which is used to teach all children, regardless of their level of knowledge of this language. This has created a `language barrier´ for many ethnic minority children who have limited, or in some cases no understanding and/or proficiency in Vietnamese. In this context, two key issues facing education in Viet Nam are: 1) how to cope with the needs of many different language groups; and 2) how to fulfil the right to education for all children in order that they learn and develop to their full potential. The Constitution, Education Law and many other educational policy documents of Viet Nam have provided strong support for the use of EMLs in education. Bilingual education is the education in which young children can start learning in their home language first (mother tongue) as the language of instruction and then gradually learn in the second language (national language – Vietnamese). Based on a strong foundation of competences built in their home language, young children can learn the second language. The Action Research on Mother Tongue Based Bilingual Education (MTBBE) has been an important initiative of the Education Programme within two Country Programmes of Cooperation 2006-2011 and 2012-2016. The action research has been implemented in the provinces of Lào Cai, Gia Lai and Trà Vinh (Northwest, Central Highlands and Mekong Delta in the South, respectively) with participation of students from pre-primary to primary education level of three ethnic minority groups: Mong, J´rai and Khmer. This is a cooperation project between MOET and UNICEF since school year 2008 with two cohorts of ethnic minority children. By end of school year 2013-2104 (May 2014), the first cohort completed six year learning in this programme, while the second cohort will finish their respective cycle by May 2015. 1.2 Purpose and goal The objectives of the Action Research are:

To implement a valid and feasible design of bilingual education in EMLs and Vietnamese in pre-school and primary schools.

To contribute to the development of policies and practices, including legal frameworks that will promote the use and development of EMLs as a means to improve access, quality and equity of education and other social services.

100 This section basically repeats the TOR.



110

2. Location Table A1 and Table A2 summarize details of the locations where MTBBE pilot is implemented. Figure 1 shows the location on the map of Viet Nam. Table A1. MTBBE Action Research location

Province District Commune School 5-year old ethnic children

Gia Lai Chu Pah Ia Phí MN Ia Phí J’rai

Ia Grai Ia Der MN 30-4 J’rai

Lào Cai Sa Pa Lao Chai MN Lao Chai Mong

Bác Hà Ban Pho MN Ban Pho Mong

Si Ma Cai Sín Chéng MN Sín Chéng Mong

Trà Vinh Cầu Ngang Nhi Truòng Nhi Truòng Khmer

Trà Cú Hàm Giang Hàm Giang Khmer

Source: MOET, Annual Progress Report 2010

Table A2. Brief general information of the target provinces

General information Province

Gia Lai Lào Cai Trà Vinh

Region Central Highlands Northwest Mekong Delta (South)

Land area (km2) 15,536.9 6,383.9 2,215

Population (million) 1.3 0.66 1.0

Density (Persons/Km2) 88 103 461

Rural population (%) 70 78 84

Ethnic groups 39 including Kinh, J´rai, Ba Na, Xo Dang, Gie Trieng

26 including Kinh 36%, Mong 22%, Tay 16%, Dao 14%, Giay 5%, others: Phu La, San Chay, Na Nhi and the La Chi

3 including Kinh 68%, Khmer 30%, Hoa 2%

Source: Source: General Statistics Office of Viet Nam. Statistical Yearbook of Viet Nam 2013. Density data is a rough estimation.



111

Figure A1. Map location of the three target provinces

Note: Data on class enrolment belong to the school year 2010-2011 Source: MOET-UNICEF. 2012. Programme Brief 3 on Action Research on MTBBE: Improving the equity and quality of education for ethnic minority children in Viet Nam (September), page 4.

3. Components Interventions and inputs of the Action Research MTBBE were the following (a) Design and development of methodologies, materials, teacher training plans, baseline surveys and assessment mechanism for measuring learning outcomes and an implementation framework; (b) Awareness-raising and support from policymakers, education, managers, principals, teachers, students and community members; (c) Development of teaching and learning materials for pre-school and primary grades one and two; reference and advocacy materials in three EMLs; (d) Teacher-training in mother tongue literacy, bilingual education teaching methodologies and techniques; (e) Policy dialogue and advocacy with decision policy makers at all levels on the model for continuation and replication in other provinces in ethnic minority areas. During the implementation, a number of high level international experts in the field of bilingual education were brought in to support the design, materials development, teacher training areas (ToR, page 2). 4. Financing



112

For the implementation of the MTBBE Action Research UNICEF Viet Nam mobilized US$2.4 million from various sources including its general resources. This was instrumental to the development of the design phase and the kick-off of the implementation phase. At the end UNICEF has been able to request for the programme the amount of US$5,244,000.00 during 2008-2014. This amount was not budgeted in exact terms since the beginning because, at the time of signing the MoU (2007), MOET lacked information for a better estimation of the amount of financial resources that would be needed. MOET had the commitment to also provide in-kind counterpart resources in addition to those coming from its regular budget. In other words, resources were agreed under flexible conditions. 5. Main Stakeholders and organization management 5.1 Main Stakeholders Key institutional stakeholders of the MTBBE Action Research are MOET and UNICEF. Within MOET, the pilot programme involves different specialized departments, i.e. VIES, RCEME (it is part of VIES and the focal point of the Programme), PED and ECED. All the aforementioned entities integrate the Steering Committee by means of the MOET´s Vice-Minister, UNICEF´s Deputy Representative and the Directors of the MOET´s departments. The Advisory Group also adds the participation of representatives of EMED, NIESAC, and CEMA as well as senior staff of UNICEF, UNESCO and World Bank. At the local level MOET is represented by Provincial Research Management Boards and District Action Research Management Boards, both adding the representation of provincial and district education managers, school principals and teacher associations.

During the implementation time, the Programme got technical support from the VietNam Linguistic Institute and the Summer Institute of Linguistic (SIL) for master teachers training, MTBBE planning, and the Map of language. The Ethnic Council/National Assembly, by means of the Vice Chairperson of this Council and the Deputy Director of the Ethnic Department, has been another important partner in promoting MTBBE policy. In the matter of policy the Committee for Ethnic Minority also deserves to be mentioned.

The specific responsibilities of UNICEF in the MTBBE Programme have been: Provides support to MOET in the overall implementation and management of the Research,

through technical assistance and/or contracting of international specialists, as per Research Overall Design.

To ensure that the initial and on-going design, technical aspects, and overall quality of the Research are based on sound practices according to inputs from international specialists working with national counterparts.

To liaise with and report to interested donors to support the raising of financial resources for the implementation of Research activities throughout the entire Research cycle.



113

Provides support to RCEME in ensuring participation of relevant institutes and organizations such as NIL, CEMA, Teachers´ Colleges and Pedagogical departments a universities, in relevant Research activities (, page 4).

This role is in agreement with the general assistance UNICEF Viet Nam provides to MOET to address learning gaps in the education system and improve the quality of education at preschool, primary and lower secondary education. As it is known one priority of UNICEF is the field of Basic Education and Gender Equality, which includes improving children´s developmental readiness for school, especially for excluded children among disadvantaged groups, via community-sponsored childhood education and health initiatives. 5.2 Organization management A high-level Steering Committee, co-chaired by MOET Vice-Minister and the UNICEF Deputy Representative has been responsible for overall management and decision-making. Other members of the Steering Committee are UNICEF Education Section Chief, Director of PED, Director of ECED, Director of EMED, and Director of RCEME. This Committee had met at a minimum once a year and its specific responsibilities are:

To make decisions on the strategic issues related to the overall management and implementation of the Research, as well as on issues submitted by the Advisory Group;

To decide on the scope and timeframe of the Research activities;

To approve the application of the pilot curriculum and teaching/learning materials; and

To review the progress reports submitted by RCEME and the Advisory Group (MoU, page 5).

The provision of technical inputs and guidance and the review of progress reports prepared by RCEME have been the main functions of an Advisory Group, co-chaired by the RCEME Director and the UNICEF Chief of Education. Other members of this group would be representatives of PED, ECED, EMED, RCEME, NIESAC101, CEMA and NIL, the International Cooperation Department of MOET, and senior staff of UNICEF, UNESCO and The World Bank. The Advisory Group had to meet on a quarterly basis. Figure A2. Management diagram

101 This Institute’s name changed again to Viet Nam Institute for Educational Sciences since 2008.



114

5.3 Implementation status of the pilot Timeline of the Action Research Implementation Framework indicates that the pilot is already implementing closing activities, which includes the finalization of Grade 5 of the second cohort, i.e. the second semester to finish in May 2015. At the end of the present evaluation, UNICEF will undertake intense advocacy activities in favour of the MTBBE´s scale up where necessary and feasible. 5.4 The project´s context With a size of 330,972 square kilometres102, Viet Nam is located on the Indochina Peninsula in Southeast Asia. The number of inhabitants reached 90.5 million by 2014103, becoming one of the most populated countries in the world104. The 2009 Housing and Population Census classified most of the inhabitants (86%) as belonging to the Kinh (Viet) ethnic group, while about 11 million were part of 53 ethnic minority groups. The Kinh ethnic majority speaks Vietnamese, an Austroasiatic, tonal Mon-Khmer language that uses a Romanized Vietnamese alphabet (quốc ngữ) since the 17th century. Minority groups speak a variety of languages, including Tai-Kadai, Khmer, Chinese, Nùng, Mong, and J’rai105. The social dynamic of this cultural mosaic has been synthetized in the following terms,

102 Report No.1809/BC-BTNMT, May 15th 2014 of Minister of Natural Resources and Environment. 103 2014 Inter-censal Population and Housing Survey (IPS). 104It ranks 13th in the World, and is the eight most populous in Asia.

http://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population

http://en.wikipedia.org/wiki/List_of_Asian_countries_by_population 105 The number of spoken languages varies among sources. For instance, Edmonson and Gregerson count about 100

distinct languages, Wikipedia reports 65 plus the official Vietnamese, three sign languages and French, and Ethnologies’

lists 109, from which 108 are living and 1 is extinct. Of the living 1 is institutional, 15 are developing, 50 are vigorous,

http://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population

http://en.wikipedia.org/wiki/List_of_Asian_countries_by_population



115

The multi-ethnic nation-state of Viet Nam with 53 official minority nationalities (plus Vietnamese) has been transfigured by a turbulent history of invasions, migrations, demi expansions, and the rise and fall of cultures. The result of these forces is a populace divided into: (i) a majority ethnicity, the Kinh or ethnic Vietnamese, who expanded from a traditional geographic base in the Red River Basin, and (ii) a large number of minority ethnicities, with smaller populations, homelands in highland enclaves, less developed economies, and slowly changing cultures.106

Article 5 of the 2013 Constitution defines Viet Nam as a socialist republic, a “unified nation of all nationalities living on the territory of Viet Nam”. All these nationalities are declared “equal, solidary, mutually respect and assist in their developments; all acts of national discrimination and division are strictly forbidden.”107

Viet Nam is also a country ruled by a single-party. In 1986, the Communist Party of Viet Nam introduced reforms for a socialist-oriented market economy. The reforms encouraged private ownership in industries, commerce and agriculture. As the World Bank (2014) recognizes, these reforms became a development success story that transformed Viet Nam: ...from one of the poorest countries in the world, with per capita income below $100, to a lower middle income country within a quarter of a century with per capita income of $1,960 by the end of 2013. Viet Nam has already attained four of its ten original Millennium Development Goal (MDG) targets and is likely to attain three more by 2015 108 (World Bank, 2014).

Over the last decade, the growth rate averaged 6.4% annually; however, it declined to 5.4% in 2013 and probably will decline in 2014. The last happened under a framework of macroeconomic stabilization that reduced inflation from 23% in August 2011 to about 4.2% one year later. Another important attainment is the outstanding progress in reducing poverty, from about 60% in the 1990s to below 10% by 2012 (World Bank, 2014). Some scholars attribute poverty reduction to equitable economic policies that distributed land during the first stage of the 1986 reforms, invested in remote areas, and subsided education and healthcare (Vandermoortele & Bird, 2010, page 5). All these advances are reflected in Viet Nam’s relative high ranking in the Human Development Index (HDI). In 2013, the HDI value was 0.638— which is in the medium human development category— positioning the country at 121 out of 187 countries and territories. Between 1980 and 2013, Viet Nam’s HDI value increased from 0.463 to 0.638, an increase of 37.8 per cent or an average annual increase of about 0.98 per cent (UNDP, 2014). However, this progress in the HDI could be even better if Viet Nam had multiplied its efforts in social development. As the Human Development Report 2011 indicated, income growth has made a more significant contribution to progress in the HDI than either life expectancy and/or education indicators. Progress in social development, including health and 37 in trouble, and 5 are dying”. See http://en.wikipedia.org/wiki/Category:Languages_of_Viet Nam;

http://www.ethnologue.com/country/VN 106 Edmondson & Gregerson, 2007, page 745. 107Unofficial English translation by International IDEA´s Constitution Building Programme. www.ConstitutionNet.org 108 Extreme poverty and hungry eradication, universal primary education (by the national standard), and gender equality

and empower women are among the MDG targets already accomplished, according to the MDG Full Report 2013

(Government of Viet Nam, 2013, p.7).

http://en.wikipedia.org/wiki/Category:Languages_of_Vietnam

http://www.ethnologue.com/country/VN

http://www.constitutionnet.org/



116

education, has been less rapid, thus contributing to slower growth in the HDI at the national and subnational level109. Of high concern is the fact that education indices have slowed over the past decade, in particular from 2004 to 2008. This may indicate cause for concern considering the importance of education for human development and the prominence attached to education in the development priorities identified by the Socio-Economic Development Plan 2011-2015 and the Socio-Economic Development Strategy 2011-2020 (UNDP, 2011, pages 1-2). A consequence of a relatively lower priority to social investment and its quality, compared to investment in economic growth, is the increase of social inequality110, which particularly affects, among other disadvantaged and vulnerable groups, the ethnic minority population. Ethnic minorities face structural and policy barriers to capture a greater portion of the benefits of economic growth. This situation manifests itself in the disparities in poverty levels between ethnic minorities and the rest of the population:

The average income of ethnic minority households in poor and remote areas is only one-sixth of the national average. The fact that the concentration of minorities among the poor and extremely poor has increased substantially in recent years, while the national poverty rate has been declining indicates that the speed of poverty reduction for ethnic minorities is far below the national standard and that of the Kinh´s.111 Ethnicity still determines the level of poverty in many cases, and some ethnic minorities fall into the trap of intergenerational poverty that has yet to be broken in a systematic way.112

Table 3 shows several indicators that make evident the huge disparities between the Kinh and the ethnic minorities. By 2010 the poverty rate for the ethnic minorities was five times higher (66%) than that of the Kinh (13%). Disparity in living standards (poverty gap) inside ethnic minorities’ households is also higher than among the Kinh. In terms of education, there is another huge gap as indicated by the differences in literacy rate and school attendance for both ethnic populations. The literacy rate is lower in the case of ethnic minorities than in the Kinh´s, 80% compared to 96%. School attendance for school age children is also in favour of the Kinh majority, 90% vs. 95%. Access to public utilities, such as electricity, improved water and sanitation is also lower in the case of ethnic minorities. Table A3. Socio-economic Indicators by ethnicity (%)

Kinh/Hoa Ethnic minority

Poverty rate in 2010 (by expenditure) 13.0 66.3

Poverty gap in 2010 (by expenditure) 2.7 24.3

Literacy rate for population aged 10 and older in 2010 96.2 79.8

School attendance rate for children aged 6-14 in 2010 95.0 89.8

Households with access to electricity grid in 2010 90.9 69.6 109 During the period from 2004 to 2008, the income index rose by 13 percent compared to 15 percent between 1999 and

2004, the life expectancy index increased by 2 percent compared to 8 percent between 1999 and 2004, and the education

index by 1 percent between 2004 and 2008, compared to 3 percent between 1999 and 2004 (UNDP, 2011, page 14). 110 The Gini coefficient for income rose from 0.418 in 2002 to 0.434 in 2008. The gap between the incomes of the richest

and poorest quintiles was 8.94 times in 2008, up from 8.1 times in 2002 (UNDP, 2011, page 25) 111 Government of Viet Nam, 2013, page 3. 112 EFA Programme 2012, page 11.



117

Households with access to improved water in 2010 69.2 18.4

Households with access to improved sanitation in 2010 98,9 83.2

Source: General Statistics Office, Viet Nam Household Survey 2010. Cited by Government of Viet Nam, 2013:4

For the Viet Nam Human Development Report 2011, some regions and socio-economic groups are at risk of being left even further behind, undermining Viet Nam’s progress towards higher levels of human development. This is a critical issue, which deserves greater priority from policymakers and decision makers. Human development, and in particular improved health and education outcomes, must be at the heart of Viet Nam’s future progress, in its own right and not only as an instrument to achieving greater wealth and economic growth (UNDP, 2011, page. 4). The Viet Nam Human Development Report 2011 emphasized poverty and insufficient public social investment as the main causes of the education gaps between ethnic minority groups and the Kinh/Hoa majority. The Millennium Development Goals Full Report 2013 indicates that reducing school dropout rates, especially in remote areas113, has been critically challenging. The main reasons are families´ financial problems –that usually discourage children to attend school, and so use them in home chores- and the learning capacity of students:

Language barriers are among critical factors that affect learning outcomes of ethnic minority groups. Ethnic minorities who cannot speak fluent Vietnamese encounter great difficulties in comprehending learning materials; this language barrier makes them lag behind other students in terms of learning and they drop eventually when they find they cannot catch up with their classmates’.114

Since about three decades ago the government has paid attention to the role of mother tongues in education. For instance, the 2003 Millennium Development Goals cited a CEM study on education policies that found that while the quality of teachers varied from district to district, language barriers contributed to low quality of education for these areas. Vietnamese speaking teachers had important difficulties communicating with students in ethnic minority areas since textbooks were only available in Vietnamese.115 Besides supportive laws and policies, the government delegated to various institutions particular roles in the provision of education to ethnic minority populations, such as the Research Centre for Ethnic Minority Education (RCEME), the Primary Education Department (PED), the Early Childhood Education Department (ECED), the Ethnic Minority Education Department (EMED), the National Institute of Linguistics (NIL), the State Committee for Ethnic Minority (CEMA). Institutional efforts had included the use of different models of bilingual education during the period 1961-2004.116.

113 The most remote schools are likely to have less resources (such as supplementary reading books), have poorly trained

and less motivated teachers, and have many students who do not use the Vietnamese language at home and do not

receive support for their learning at home (EFA Programme, 2012, p. 31). 114 Government of Viet Nam, 2013, page 7. Language of instruction is still a strong but negative factor in determining

the quality and outcome of education for some ethnic minority students who come from communities that do not use the

national language, Vietnamese (EFA Programme, 2012, page 11). See also (Truong Huyen, 2011). 115 The United Nations in Viet Nam, 2003, page 13. 116 Truong Huyen, 2011, pages 7-8.



118

However, only the 2000-2005 World Bank supported model tested in the Central Highlands to teach Vietnamese to J’rai children through the students´ mother tongue has been considered successful.117 As some research concludes, Policy effort in this area shows government accepts that there is a problem and is concerned to reduce disparities between groups. However, a (World Bank) report concluded that the government’s policies on education for ethnic minority pupils have not been able to close the gap between the majority and minorities. It also highlighted differences between different ethnic groups.118 The above cited World Bank study is very precise in recommending that more ethnic minority teachers need to be trained and used effectively, and that bilingual education pilot projects need to be scaled up, particularly in the areas in which certain minorities are falling behind on education indicators.119 Different studies confirm that low progress in making laws and policies that encourage the use of mother tongue in the education of ethnic minority children could be related to a combination of two factors, i.e. insufficient awareness on the importance of the mother tongue and low mastering of the appropriate methodologies. At the level of implementation misconception, stereotypes, prejudices, and similar factors also come into play.120 Higher tendency to retire from school and lower quality education on the side of ethnic minority children will continue feeding the ethnic gap in education and other areas, such as the access to business and income opportunities, labour market, and social services. Ethnic minorities of Viet Nam need the resolve and commitment from the national and provincial governments and local communities to overcome their social disadvantages. They may need more audacious implementation of affirmative action. Something beyond what is already at play includes approaches such as the construction of schools in remote areas and the application of education fee discounts and exemptions for disadvantaged populations. The MTBBE has been recognized as a promising approach for those targeted policies, particularly in areas of ethnic minority concentration (e.g. EFA Programme, 2012). AR MTBBE pointed out the need of bilingual education models that really help ethnic minority children to defeat the language barrier at the most effective, efficient, and sustainable possible way.

117 UNICEF Viet Nam, 2008, page 5. 118 Truong Huyen, 2011, page 4; World Bank, 2009. 119 World Bank, page 53. 120 Truong, H., 2011, page 18; World Bank, 2009, page 45-46.

Annex 3 – Evaluation Design Matrix

A B C D E F G H J K L

Evaluation Criteria* and Questions*

Sub-question

Type of Sub-question121,122

Measure or Indicator123

Target or standard (normative)

Data Source124

Design125 Sample or Census

Data Collection Instrument126

Data Analysis127

Comments128

1. Relevance

1.1 To what extent is the MTBBE approach relevant and responsive to the Vietnamese context and needs at both national and sub-national level?

1.1.1 To what extent is the MTBBE relevant and responsive at the national level?

Descriptive For KII respondents’ qualitative perception; for D evidence.

Not applicable

D FG KII

Non-experimental

Sample C D KIICM S

NVivo T

1.1.2 To what extent is the MTBBE relevant and responsive at the sub-national level?

Descriptive For KII and FG respondents’ qualitative perception; for D evidence.

Not applicable

C D FG KII

Non-experimental

Sample C D FGC FGL KIIL S

NVivo T

1.2 To what Normative Degree of accordance

Provisions D Non- Sample D NVivo

121 There are three types (descriptive, normative, cause-effect). 122 Cause and effect questions often require an experimental or quasi-experimental design to ensure rigour. That is not possible in this evaluation due to

budgetary limitations. Hence attempts will be made to use responses to make counterfactual estimates: e.g. What would the situation have been if the intervention

had not taken place? 123 If the quantitative measures are expressed as percentages for measurement, they will at the reporting stage also indicate the number of cases (“n” value) on

which the percentage is based. 124 D=Document; KII=Key Informant Interviews; FG=Focus Group; SV=Site Visit. 125 There are three main kinds of design: Experimental; quasi-experimental and non-experimental. 126 For documents, a Document Coding Structure (DCS); for KII, KII Guides (KIIG); for focus groups, Focus Group Guides (FGG); for survey, survey

questionnaires (SQ). The specific data collection instruments corresponding to the KII and FG referenced will be designed after this matrix has been approved. 127 Examples: NVivo software (http://www.qsrinternational.com/products_nvivo.aspx). Line-of-evidence report (LER). Triangulation (T). 128 Typical comments are: Check the quality of a data set; indicate limitations of design; develop graphic from the data.

http://www.qsrinternational.com/products_nvivo.aspx

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014.

FINAL REPORT

120



Sub-question




Data Source124



Data Analysis127

Comments128

extent is the MTBBE approach relevant to international treaties and commitments that Viet Nam already signed up, e.g. CRC129

with international treaties and commitments For KII respondents’ qualitative perception; for D evidence;

of international treaties and commitments

KII

experimental

KIICM T

1.3 To what extent is the MTBBE approach relevant to Constitution, Education Law, and law on children care, protection and education, national policies,

1.3.1 To what extent is the MTBBE relevant to Constitution

Normative Degree of accordance with the Constitution For KII respondents’ qualitative perception; for D evidence.

Provisions of Constitution

D KII

Non-experimental

Sample D KIICM

NVivo T

1.3.2 To what extent is the MTBBE Education Law?

Normative Degree of accordance with Education Law. For KII respondents’ qualitative

Provisions of Education Law

D KII

Non-experimental

Sample D KIICM

NVivo T

129 CRC = [The meaning of “CRC” to be identified].


FINAL REPORT

121



Sub-question




Data Source124



Data Analysis127

Comments128

Decree 82 on the teaching and learning of EMLs in general education establishments and centres for continuing education? Institutions in the natio-nal education system? Education Law, Ministry of Education and Training’s Education Development Strategy 2011-2020, the Party Decision 29-NQ/TW issued at

perception; for D evidence.

1.3.3 To what extent is the MTBBE relevant to the Law on child care, protection and education?

Normative Degree of accordance with law on child care, protection and education. For KII respondents’ qualitative perception; for D evidence.

Provisions of law on child care, protection and education.

D KII

Non-experimental

Sample D KIICM

NVivo T

1.3.4 To what extent is the MTBBE relevant to national policies?

Normative Degree of accordance with national policies. For KII respondents’ qualitative perception; for D evidence.

Provisions of national policies.

D KII

Non-experimental

Sample D KIICM

NVivo T

1.3.5 To what extent is the MTBBE relevant to Institutions in the national education system?

Normative Degree of accordance with institutions in the national education system. For

Provisions of institutions in the national education system

D KII

Non-experimental

Sample D KIICM

NVivo T


FINAL REPORT

122



Sub-question




Data Source124



Data Analysis127

Comments128

the Party Congress XIII on comprehen-sive reform with socialist orientation and global integration?

KII respondents’ qualitative perception; for D evidence.

1.3.6 To what extent is the MTBBE relevant to MET’s Educational Development Strategy 2011-2020?

Normative Degree of accordance with MET’s Educational Development Strategy 2011-2020. For KII respondents’ qualitative perception; for D evidence.

Provisions of MET’s Educational Development Strategy 2011-2020.

D KII

Non-experimental

Sample D KIICM

NVivo T

1.3.7 To what extent is the MTBBE relevant to Party decision 29-NQ/TW?

Normative Degree of accordance with Party decision 29-NQ TW. For KII respondents’ qualitative perception; for D evidence.

Provisions of Party decision 29-NQ TW

D KII

Non-experimental

Sample D KIICM

NVivo T

1.4 What role has MTBBE played in

Descriptive For KII respondents’ qualitative perception; for D

Not applicable

D FG KII

Non-experimental

Sample D KIICM KIIL S

NVivo T


FINAL REPORT

123



Sub-question




Data Source124



Data Analysis127

Comments128

terms of improving educational policy with respect to the ethnic minorities of the country?

evidence.

1.5 To what extent is the MTBBE approach relevant to UNICEF’s renewed focus on equity?

Normative Degree of accordance with UNICEF’s renewed focus on equity. For KII respondents’ qualitative perception; for D evidence.

Provisions of UNICEF’s renewed focus on equity

D FG KII

Non-experimental

Sample D KIIU

NVivo T

2. Effectiveness

2.1 How effective is the MTBBE approach to improving the learning outcomes of

2.1.1 How effective is the MTBBE approach to improving the learning outcomes of Vietnamese?

Descriptive For FG respondents’ qualitative perception; for D evidence.

Not applicable

D FG KII

Non-experimental

Sample D PSSP T

For questions 2.1.1 and 2.1.2, D is most the review of the results


FINAL REPORT

124



Sub-question




Data Source124



Data Analysis127

Comments128

participating children (academic especially Vietnamese mastery and mathematics and non-academic results and children’s confidence, skills such as communica-tion, etc.?

2.1.2 How effective is the MTBBE approach to improving the learning outcomes of Mathematics?


Not applicable

D FG KII

Non-experimental

Sample D PSSP T

of past official tests that have measured children` learning outcomes, particularly those comparing MTBBE to non-MTBBE students.

2.1.3 How effective is the MTBBE approach to improving Non-academic learning outcomes?


Not applicable

D FG KII

Non-experimental

Sample D FGS FGSN

NVivo T

2.1.4 How effective is the MTBBE approach to improving children’s confidence?


Not applicable

D FG KII

Non-experimental

Sample D FGS FGSN

NVivo T

2.1.5 How effective is the MTBBE approach to improving communication?


Not applicable

D FG KII

Non-experimental

Sample D FGS FGSN

NVivo T


FINAL REPORT

125



Sub-question




Data Source124



Data Analysis127

Comments128

2.1.6 How effective is the MTBBE approach to improving other areas? Please specify.


Not applicable

D FG KII

Non-experimental

Sample D FGS FGSN

NVivo T

2.2 How effective is the MTBBE approach to improving the capacity of teachers and education managers?

2.2.1 How effective is the MTBBE approach to improving the capacity of teachers?


Not applicable

D FG KII

Non-experimental

Sample D FGT

NVivo T

2.2.2 How effective is the MTBBE approach to improving the capacity of education managers?


Not applicable

D FG KII

Non-experimental

Sample D FGL FGT KIICM KIISP

NVivo T

2.3 Were contextual factors taken into account in the approach design/ implementation of interventions?

2.3.1 Were contextual factors taken into account in the approach of interventions?


Not applicable

D FG KII

Non-experimental

Sample D FGL FGT KIICM KIISP KIIU

NVivo T

2.3.2 Were contextual factors taken into account in the design of

Descriptive For KII and FG respondents’ qualitative perception; for D

Not applicable

D FG KII

Non-experimental

Sample D FGL FGT KIICM KIISP

NVivo T


FINAL REPORT

126



Sub-question




Data Source124



Data Analysis127

Comments128

interventions? evidence. KIIU

2.3.3 Were contextual factors taken into account in the implementation of interventions?


Not applicable

D FG KII

Non-experimental

Sample D FGL FGT KIICM KIISP KIIU

NVivo T

2.4 What effects (positive or negative changes) did the MTBBE approach have on four stakeholder groups? A: MTBBE students; B: Teachers, school management, board members; C: Parents and community people; D:

2.4A.1 What positive or negative effects did the MTBBE approach have on student learning outcomes?

Cause and effect

For FG respondents’ qualitative perception; for D evidence

Not applicable

D Non-experimental

Sample DT FGC, FGL, FGP, FGS, FGSN, FGT

NVivo T

2.4A.2 What positive or negative effects did the MTBBE approach have on student attitudes?

Cause and effect

. For KII and FG respondents’ qualitative perception; for D evidence.

Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGS FGT FGSN KIISP

NVivo T

2.4A.3 What positive or negative effects did the MTBBE approach have on student self-confidence?

Cause and effect

For KII and FG respondents’ qualitative perception; for D evidence.

Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGS FGT FGSN

NVivo T


FINAL REPORT

127



Sub-question




Data Source124



Data Analysis127

Comments128

Managers from grassroots to central levels.

KIISP

2.4A.4 What positive or negative effects did the MTBBE approach have on student approaches to learning?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4A.5 What positive or negative effects did the MTBBE approach have on student approaches to schools?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4A.6 What positive or negative effects did the MTBBE approach have on student approaches to teachers?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4A.7 What positive or negative effects did the MTBBE approach have on

Cause and effect

For KII and FG respondents’ qualitative perception; for D

Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGS

NVivo T


FINAL REPORT

128



Sub-question




Data Source124



Data Analysis127

Comments128

girl student and gender issues at school levels?

evidence; for S quantitative.

FGT FGSN KIISP S

2.4A.8 What positive or negative effects did the MTBBE approach have on girl student and gender issues at community levels?

Cause and effect

For KII and FG respondents’ qualitative perception; for D evidence; for S quantitative.

Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGS FGT FGSN KIISP S

NVivo T

2.4B.1 What positive or negative effects did the MTBBE approach have on teachers’ in-class teaching and organizing skills in facilitating MTBBE learning?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.2 What positive or negative effects did the MTBBE approach have on teachers’ professional capacity of

Cause and effect


Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGT KIISP

NVivo T


FINAL REPORT

129



Sub-question




Data Source124



Data Analysis127

Comments128

MTBBE?

2.4B.3 What positive or negative effects did the MTBBE approach have on teachers’ attitude?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.4 What positive or negative effects did the MTBBE approach have on teachers’ self-confidence?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.5 What positive or negative effects did the MTBBE approach have on teachers’ approaches towards learning?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.6 What positive or negative effects did the MTBBE approach have on teachers’ approaches towards schools?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T


FINAL REPORT

130



Sub-question




Data Source124



Data Analysis127

Comments128

2.4B.7 What positive or negative effects did the MTBBE approach have on teachers’ approaches towards other teachers?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.8 What positive or negative effects did the MTBBE approach have on school management?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4B.9 What positive or negative effects did the MTBBE approach have on board members?

Cause and effect


Not applicable

Non-experimental


NVivo T

2.4C.1 What positive or negative effects did the MTBBE approach have on parents’ participation in school activities?

Cause and effect


Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGT KIISP

NVivo T

2.4C.2 What Cause and For KII and Not D Non- Sample D NVivo


FINAL REPORT

131



Sub-question




Data Source124



Data Analysis127

Comments128

positive or negative effects did the MTBBE approach have on parents’ participation in learning and extra-curricular activities?

effect FG respondents’ qualitative perception; for D evidence.

applicable FG KII

experimental

FGC FGL FGP FGT KIISP

T

2.4C.3 What positive or negative effects did the MTBBE approach have on parents’ relationships with schools and community?

Cause and effect

For KII and FG respondents’ qualitative perception; for D evidence;.

Not applicable

D FG KII

Non-experimental


NVivo T

2.4C.4 What positive or negative effects did the MTBBE approach have on parents’ acceptance and views about MTBBE?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4C.5 What positive or negative effects did the MTBBE

Cause and effect

For KII and FG respondents’ qualitative perception;

Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP

NVivo T


FINAL REPORT

132



Sub-question




Data Source124



Data Analysis127

Comments128

approach have on community’ participation in school activities?

for D evidence.

FGT KIISP

2.4C.6 What positive or negative effects did the MTBBE approach have on community’s participation in learning and extra-curricular activities?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4C.7 What positive or negative effects did the MTBBE approach have on community’s relation-ships with schools and parents?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4C.8 What positive or negative effects did the MTBBE approach have on community’s acceptance and views about

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T


FINAL REPORT

133



Sub-question




Data Source124



Data Analysis127

Comments128

MTBBE?

2.4D.1 What positive or negative effects did the MTBBE approach have on managers’ (all levels) support for and participation in implementation of Action Research Activities?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

2.4D.2 What positive or negative effects did the MTBBE approach have on managers’ acceptance of and views about the MTBBE approach and its continuity and scale-up?

Cause and effect


Not applicable

D FG KII

Non-experimental


NVivo T

3. Efficiency

3.1.3.1 Does the actual implementation of the approach

3.1.1 What is the total cost of the program, by component and product? ¿What

Descriptive. Mainly quantitative but also qualitative

D evidence including records provided by KII

Not applicable

D KII

Non-experimental

Census D KIICM KIIU

Cost analysis using Excel

An official MOE´s source will provide high detailed cost


FINAL REPORT

134



Sub-question




Data Source124



Data Analysis127

Comments128

use resources in the most economical manner to achieve expected results?

is the cost by beneficiary children?

information break down by component and product.

3.1.2 Are these costs a justified amount as compared to the programme´s results?

Normative D evidence Evidence from the programme´s evaluation

D The rest of the Evaluation Report

Non-experimental

D Cost-efficiency analysis

3.1.3 Are these costs an acceptable amount as compared to other similar programs or projects in Viet Nam or abroad?

Normative

D evidence.

Other similar programs in Viet Nam or abroad

D

Non-experimental

Sample D

NVivo T

Due to the confidential nature of this information it is very difficult to find information to compare with.

3.1.4 What are the perceptions that educational authorities have in terms of efficiency in allocating resources such as,

Descriptive For KII respon-dents’ qualitative perception.

Not applicable

KII Non-experimental

Sample KIICM KIIL KIISP KIIU FGT

NVivo T


FINAL REPORT

135



Sub-question




Data Source124



Data Analysis127

Comments128

teaching material and trained teachers?

4. Sustainability

41 To what extent are the results (impacts, if any and outcomes) likely to continue in 3 MTTBE Action Research sites when the external support from UNICEF is withdrawn?


Not applicable

D FG KII

Non-experimental

Sample D KIICM KIIL KIISP KIIU

NVivo T

4.2 To what extent can the MTBBE approach be replicated and scaled up in areas with


Not applicable

D FG KII

Non-experimental


NVivo T


FINAL REPORT

136



Sub-question




Data Source124



Data Analysis127

Comments128

characteristics similar to existing project sites (e.g. presence of students from one single ethnic group, Mong or J´rai or Khmer in the class?)

4.3 What elements are (or are not) in place for sustain-ability? (Leadership, ownership, financial resources, teaching and learning materials, qualified teachers, local/ DOET resources)

4.3.1 Is leadership in place for sustainability?


Not applicable

D FG KII

Non-experimental


NVivo T

4.3.2 Is ownership in place for sustainability?


Not applicable

D FG KII S

Non-experimental


NVivo T

4.3.3 Are financial resources in place for sustainability?


Not applicable

D FG KII

Non-experimental


NVivo T


FINAL REPORT

137



Sub-question




Data Source124



Data Analysis127

Comments128

4.3.4 Are teaching and learning materials in place for sustainability?


Not applicable

D FG KII

Non-experimental


NVivo T

4.3.5 Are qualified teachers in place for sustainability?


Not applicable

D FG KII

Non-experimental


NVivo T

4.3.6 Are local/ DOET resources in place for sustainability?


Not applicable

D FG KII

Non-experimental


NVivo T

4.4 What could or should be done differently in future replications and/or scaling up?


Not applicable

D FG KII

Non-experimental

Sample D FGC FGL FGP FGT KIICM KIIL KIISP KIIU

NVivo T

4.5 What are the lessons

Lessons learnt will be constructed


FINAL REPORT

138



Sub-question




Data Source124



Data Analysis127

Comments128

learnt about MTBBE in the Vietnamese context?

in the analysis phase


Education in Viet Nam, 2006 – 2014. FINAL REPORT

139

Annex 4 – List of People Consulted UNICEF VN

No. Interviewee Position

1 Youssouf Abdel-Jelil Representative

2 Jesper Moller Deputy Representative

3 Joyce Patricia Bheeka Chief Education Section

4 Vu Mang Hong Planning, Monitoring and Evaluation Specialist

5 Dihn Phuong Thao Inclusive Education Specialist

6 Nguyen Thi Thanh Nga Bilingual Education Specialist

7 Nguyen Minh Nhat Education Officer

8 Le Thi Minh Chau Education Specialist System Strengthening

Partners

No. Interviewee Position/ Agency

1 Ragnhild Dybdahl Deputy Head of Mission Counsellor, Norwegian Embassy

2 Hans Lambrecht Belgian Development Agency, BTC

3 Nico Vromant Flemish Association for Development Cooperation and Technical Assistance, VVOB

4 Sun Lei UNESCO Education Coordinator

5 Vo Hoang Nga UNDP

6 Nguyen Tam Giang World Bank

7 Nguyen Thi Tuyet Nhurg Child Fund

8 Le Thi Bich Hanh Education Manager, Plan International in Viet Nam

9 Nguyen Lam Giang New Zealand Embassy

Central Level Educational Authorities


1 Tran Cong Phong General Director, VIES

2 Phan Van Kha Former Director of Viet Nam Institute for Educational Sciences

3 Nguyen Thi Phuong Thao Vice Director, Research Centre for Ethnic Minority Education, VIES

4 Vi Van Dieu Head of Personnel and Organization, VIES

5 Do Tien Dat Head of Methodologies Department, VIES cum Math Material Developer

6 Tran Thi Tham Vice Director of the Primary Education Department

7 Nguyen Manh Quynh Vice-Director of Ethnic Minority Department

8 Ha Duc Da Director, Research Centre for Ethnic Minority



140

Education, VIES

9 Ngo Thi Thanh Thuy Researcher, Research Centre for Ethnic Minority Education

10 Nguyen Van Xuan Director of the Personnel Department, Committee for Ethnic Minorities

11 Truong Kim Minh Ex Director MTBBE AR in Lào Cai , Consultant for Escuela Nueva of Viet Nam Project

National Assembly


1 Trieu Thi Nai Vice Chairperson of the Ethnic Council, National Assembly

2 Nguyen Manh Quynh Vice Director, Ethnic Minority Department, National Assembly Office

Lào Cai

No. Interviewees Position Location

1 Nguyễn Anh Ninh Director DOET Lào Cai Province

2 Nguyễn Thị Kim Ngân Deputy Director, DOET Lào Cai Province

3 Mã Én Hằng Deputy Chief Lào Cai Province

4 Vũ Thị Hậu Head, training Lào Cai Province

5 Giàng Thị Gấm Principal Elementary Pre-school, Lào Cai

6 Nguyễn Thị Lý Vice Principal Elementary Pre-school, Lào Cai

7 Phùng Thị Thanh Deputy Head, Inspection and Quality Education

Lào Cai Province

8 Tran Phuong Anh DOET Lào Cai Province

9 Trần Phượng Anh Deputy Head, Primary School, Lào Cai

10 Tran Thi Thoa School Principal, Lao Chai Primary School, Sa Pa District

11 Bui Huy Cuong Head of Ethnic Minority Groups,

Sa Pa District

12 Mr. Hinh Director/Chairman of Board of Education and Training (BOET),

Sa Pa District

13 Nguyen Ha Deputy Director of BOET,

Bắc Hà District

14 Nguyen Thi Uyen

Vice Principal Ban Pho Primary School, Bác Hà District



141


15 Dang Phuong Dai

School Principal Si Ma Cai, Sin Cheng Primary School, Bac Ha District

16 Nguyễn Thị Quyên Focus Group (FG) Teachers *

Lao Chai Primary School

17 Hà Thị Nhung

18 Nguyễn Thị Hương

19 Vũ Kim Thanh

20 Mã Thị Mỷ

21 Bùi Thị Lan Anh

22 Châu A Tàu

23 Nguyễn Thị Thanh Mai

24 Giàng Thị Mải FG Parents Parents were selected at random. However, the number obeys to FG limits of no more than 10 participants on voluntary basis. Most parents were middle-age women.

Lao Chai Primary School, Sa Pa District 25 Lồ Thị Giống

26 Giàng A Sùng

27 Lý A Vảng

28 Lý A Lềnh

29 Lý A Khay

30 Giàng A Minh

31 Hảng A Seng

32 Lý A Sềnh

33 Lý A Măng

34 Giàng Thị Cở

35 Hoàng Thị Phương FG Teachers*

Ban Pho Primary School, Bác Hà District 36 Sùng Thị Mai

37 Hoàng Thị Hằng

38 Vùi Thị Lánh

39 Nguyễn Thị Kim Tiến

40 Tạ Thị Thúy Vân

41 Hoàng Thị Hằng

42 Doãn Thị Tuyết

43 Nguyễn Thị Vui

44 Lê Thị Kim Ngân

45 Vàng Thị Mình

46 Cháu Seo Pùa FG Parents** Ban Pho Primary School, Bác Hà District 47 Vàng Seo Chảo

48 Ly Thị Dúa

49 Lý Thị Say

FG Teachers*

Si Ma Cai, Sin Cheng Primary School, Bác Hà



142


50 Thào A Dìn If not all, most MTBBE teachers of this school were represented by this group.

District

51 Vàng Seo Chứ

52 Thào A Giăng

53 Thào Thị Tứ

54 Ma A Sùng

FG Parents** Si Ma Cai, Sin Cheng Primary School, Bac Ha District 55 Hảng A Chính

56 Thào Cù Vảng

57 Tráng A Chứ

58 Ly Sì Páo

59 Tráng Thị Sua

60 Ma A Din

* If not all, most MTBBE teachers of the different schools were represented by the group ** Parents were selected at random but its number at each group obeys to the no-more than 10 participants usually set for this type of research technique. Besides parents attend on voluntary basis. In most cases, women (usually mothers) over numbered men.

Gia Lai

# Interviewee Position Location

1 Do Thi Bé Deputy Principal

Ly Tu Trong Primary School in Ia Grai District

2 Nguyen Trong Ngoan (male) School Principal

Ia Phi Primary School in Chu Pah District

3 Nguyen Quang Thuan District DOET Representative

Ia Grai District

4 Do Thi Minh Hue Deputy Principal

Ngo May Primary School in Ia Grai District

5 Nhan Thi Hang Nga DOET Vice Director

Gia Lai Province

6 Kpa Pual and Rmah Vau (2) Material developers, Head and staff member of the Ethnic Minority Unit, Gia Lai DOET

Gia Lai Province

7 FG of MTBBE students’ Parents Parents** Ia Phi Primary School in



143

(8) Chu Pah District, Gia Lai Province

8 FG of MTBBE students (10) Students Ia Phi Primary School in Chu Pah District

9 FG of MTBBE teachers (6) Teachers* Ia Phi Primary School in Chu Pah District

10 FG of MTBBE students (11) Students Ly Tu Trong Primary School in Ia Grai District

11 FG of Parents of MTBBE students (3) Parents** Ly Tu Trong Primary School in Ia Grai District

12 FG of MTBBE teachers (5) Teachers* Ly Tu Trong Primary School in Ia Grai District

13 FG of Parents of MTBBE students (3) Parents** Ngo May Primary School in Ia Grai District

14 FG of MTBBE students (5) Students Ngo May Primary School in Ia Grai District

15 FG of Non-MTBBE students (5) Students Ngo May Primary School in Ia Grai District

16 FG of MTBBE teachers (4) Teachers* Ngo May Primary School in Ia Grai District

* If not all, most MTBBE teachers of the different schools were represented by the group ** Parents were selected at random but its number at each group obeys to the not-more than 10 participants usually set for this type of research technique. Besides parents attend on voluntary basis. In most cases, women (usually mothers) over numbered men.

Trà-Vinh


1 Kien Quan Provincial DOET Leader, Deputy Director

2 Thach Saron DOET Staff

3 Thach Doi Staff Ethnic Minority Department, DOET

4 Thron Thi Thanh Loan Staff Pre-school Education Department, DOET

5 Tran Chau Ngoc Ha Staff Pre-school Education Department, DOET

6 Nguyen Van Lai Head of Primary Education Department, DOET

7 Huynh Tan Nien Staff of Primary Education Department, DOET

8 Tran Hun Minh BOET Leader, District of Tra Cu

9 Kim Tan Loc School Principal, Ham Giang B Primary School, District of Tra Cu

10 Nguyen Ngoc Du DOET Deputy Leader, District of Cau Ngang



144

11 Huynh Khoane School Principal, Nhi Truong Primary School, District of Gau Nang

12 FG Parents (11) ** Nhi Truong Primary School, District of Gau Nang

13 FG Non-MTBBE Children (13)

Nhi Truong Primary School, District of Gau Nang

14 FG Teachers (6) * Nhi Truong Primary School, District of Gau Nang

15 FG MTBBE Children (11 children)

Ham Giang B Primary School, District of Tra Cu

16 FG Parents (12) ** Ham Giang B Primary School, District of Tra Cu

17 FG Teachers (5) * Ham Giang B Primary School, District of Tra Cu

* If not all, most MTBBE teachers of the different schools were represented by the group ** Parents were selected at random but its number at each group obeys to the not-more than 10 participants usually set for this type of research technique. Besides parents attend on voluntary basis. In most cases, women (usually mothers) over numbered men.



145

Annex 5 - Additional tables and graphs regarding the effects of the Action Research MTBBE on students´ learning outcomes

Table A1. First cohort. Number of MTBBE students by grade and school

No.

Elementary school

Mother tongue

Number of classes

Grade 1 2009-2010

Grade 2 2010-2011

Grade 3 2011-2012

Grade 4 2012-2013

Grade 5 2013-2014

Total Girl

Total Girl

Total Girl

Total Girl

Total Girl

1 Lao Chải

Mong

1 23 8 23 8 23 8 22 8 22 8

2 Bản Phố 2 38 21 38 21 38 21 38 21 38 21

3 Sín Chéng 1 2 33 16 33 16 32 16 32 16 32 16

4 Ia Phí

J’rai

2 30 18 29 17 28 16 28 16 28 16

5 Ngô Mây 1 25 19 24 19 23 18 23 18 23 18

6 Lý Tự Trọng 1 25 13 25 13 24 13 24 13 24 13

7 Hàm Giang B

Khmer

3 53 22 51 22 50 22 49 22 49 22

8

Nhị Trường B 2 35 19 35 19 35 19 32 18 31 18

Total 14 262 136 258 136 253 133 248 132 247 132

Drop out No.

4 0 5 3 5 1 1 0

Dropout rate

.015 0 .019 .019

.020 .008

.004 0

Source: MOET. MTBBE Report 2014

Ll Table A2. Second cohort. Number of MTBBE students by grade and school

No.

Elementary school

Mother tongue

No. of classes

Grade 1 2010-2011

Grade 2 2011-2012

Grade 3 2012-2013

Grade 4 2013-2014

Grade 5 2014-2015

Total Girl Total Girl Total Girl Total Girl Total Girl

1 Lao Chải

Mong

1 23 8 23 8 22 8 22 8

2 Bản Phố 2 28 18 28 18 28 18 28 18

3 Sín Chéng 1 2 38 18 38 18 38 18 38 18

4 Ia Phí J’rai 2 28 14 27 13 27 13 27 13



146

5 Ngô Mây 1 20 13 20 13 20 13 20 13

6 Lý Tự Trọng 1 24 17 24 13 24 17 24 17

7 Hàm Giang B

Khmer 3 50 32 49 32 47 30 48 30

8 Nhị Trường B 2 37 22 32 21 33 21 31 21

Total

12 248 142 241 136 239 138 238 138

Drop out No.

7 6 2 0 1 0

Dropout rate

.027 0.042 .009 0 .005 0

Source: MOET. MTBBE Report 2014

Figure A1. Percentage of MTBBE first cohort students according to grade and general scores


Figure A4. Percent of third graders (2nd Cohort) according to provinces and MTBBE / Non-MTBBE students, by scores in Vietnamese




147

Figure A5. Percent of third graders (2nd Cohort) by provinces and MTBBE / Non-MTBBE students, by scores in Mathematics




148

Annex 6 - Selected testimonies on different issues

Box 1. The flexible Bilingual Approach

This approach uses the mother tongue in grades 1 and 2 for teaching the essential content areas like mathematics and ethics. The learners´ materials (books and workbooks) will be used only in the L1, as well as the teacher´s manuals (unless it is determined that the manuals should be bilingual, considering teachers’ needs). This part of the approach is not flexible, because an effective bilingual model cannot introduce instruction in the L2 until students have learned enough of the L2 to understand. The flexibility in the medium of instruction can begin in grade 3 through the development of bilingual teaching and learning materials. Content area materials for grades 3 through 5 must be developed in a side-by-side format. This supports teachers in using bilingual methods such as Preview-Review, where lesson concepts are introduced at the beginning of the lesson and reviewed at the end of the lesson in the L1, but the main content in between is taught in the L2, or alternate days, where teachers use the L1 some days to present new content and the L2 other days to review or practice content. Bilingual materials also support students at all levels of transfer from L1 to L2.

The flexible bilingual approach

Pre-primary

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

L1- EML/ L2 -Viet

L1-EML L1-EML L1-EML L1-EML L1-EML

L1 L1 L1/L2 L1/L2 L1/L2

L2-Viet L2-Viet L2-Viet L2-Viet L2-Viet Preschool level builds oral L1 and L2

L1 for literacy and instruction in the early grades

Bilingual materials (suggest Grades 3-5) to ease transition

L2 oral in the early grades, then literacy transfer from L1

Source: , 1 August 2007, Annex 1 Research Design



149

Box 2. Expansion of the MTBBE

In light of the positive results achieved after 2 years of implementation of the MTBBE in Lào Cai, the Provincial Department of Education and Training (DOET) developed in 2010 its provincial plan for the continuation and expansion of the MTBBE approach. While the first and second cohorts of MTBEE students comprised 5 classes in three districts, the DOET has scaled up MTBBE to 12 classes of 214 students in 4 districts in the 2010-2011 school year. From 2011 to 2015, each school year, a new group of 210 children aged five will enter the MTBBE classes. At the end of the 2015-2016 school year in Lào Cai, a total of 1,300 students will have been taught through the MTBBE Action Research in pre-school, followed by MTBBE primary education, providing a model to other provinces committed to improving ethnic minority children learning through a bilingual education approach. Source: MOET-UNICEF. Education Programme Brief. 2011 (February), page 6

Box 3. Testimonies on MTBBE´s role on improving educational policies addressed to ethnic minorities of Viet Nam

Government policies are changing in part inspired by this initiative (MTBBE Action Research). These changes have been implemented yearly, including changes in textbooks. The development of new textbooks is taking place. The government has already a plan to develop the capacity of teachers in order to enable them teach in the mother tongues; this new policy started in 2010. The Ethnic Minority Council of the National Assembly … is assessing this bilingual programme in a very careful manner. We have also been assessing this programme for two years now.

Official from the Ethnic Minority Department

Five years ago no one accepted that Vietnamese is a second language for ethnic minority children but this has now changed. Now there are efforts to teach Vietnamese as a second language to ethnic minority children. Other initiatives to address the education challenges of ethnic minority children is the Viet Nam “Escuela Nueva” Project, a World Bank supported project based on the use of local communities to help teachers use EMLs to teach the children. One of the achievements of the Action Research Project was to change the minds of people that Vietnamese should be taught as a second language to ethnic



150

minorities. UNICEF´s official

Provincial and district level authorities are aware of the benefits of the MTBBE programme. They want to continue and expand the programme. At the central level, the programme is having an impact on the authorities. The programme has the support of different levels of the central government, including the Government Ethnic Council and the Committee for Ethnic Minorities at the National Assembly. After reading reports and after visiting the schools they recognized the impact of the programme and recommended MOET to continue and expand the Action Research Project. The National Assembly Committee for Ethnic Minorities hopes that this programme can improve the quality of education of ethnic minority children in Viet Nam. MOET was recommended to explore the possibilities to continue this programme and expand it to other minority groups. Since 2008 MOET declared this programme as one of the five solutions to promote the education of ethnic minority children. This position was expressed in documents sent to the different provinces. I think this is the basis for policy making and policy change in Viet Nam. However, I would like to share with you frankly that in order to have new policies we need time and support from UNICEF and other agencies and organizations. After completing the pilot phase of a programme, the continuation and drafting of policies take time. The following phases need planning and timeline… All pilot projects require resources and energy. This is a meaningful project. Official from a MOET´s research institute

I would like to remind you that this is an Action Research Project implemented in linguistically homogeneous schools. But there are many ethnic minority groups; and some time there is more than one group in the same school. In such cases we will have to figure out the solutions. We will have to look at different solutions and strategies to prepare these children for school. That is why we consider the MTBBE programme as just one of the solutions to address the education challenges of ethnic minority children. That is why we don’t want to have a special and separate policy for this programme. Whatever solution we find relevant for the education of ethnic minority students it needs to be considered and we will allow its implementation. There are conditions that need to be fulfilled before the implementation of a MTBBE programme: (i) children need to be from the same ethnic minority group; (ii) availability of teachers from the target ethnic minority group that can teach in that language; (iii) teachers need to be trained because although most of them can speak EMLs they cannot read and write in those languages; (iv) availability of textbooks. The development of textbooks requires the availability of material developers and intellectuals who speak the target languages so that they can participate in the process; (v) the target EML has to have a writing system. That is why we consider the MTBBE programme as just one solution to address the education challenges of ethnic minority children. There are other approaches to address this issue. We don’t develop different policies for each of such approaches. EMLs can also be taught as subjects not necessarily used as languages of teaching. Official from a MOET´s department.



151

The Communist Party plays a role at each school; it has first-hand information on how the programme is being implemented. There is high interest in this pilot, which is observed with positive eyes given children´s learning achievements and teaching performance. The programme has provided models and ways to meet those objectives. Ethnic minority children need to acquire both Vietnamese and their mother tongue in order to learn better. Why do I mention the Communist Party? Because it is a key actor in issuing policies to respond to a perceived demand. Education researcher.

Box 4. Testimonies about student learning outcomes of the MTBBE Approach

- I made progress in learning. - I like both languages. I gained a lot of knowledge. - I study two languages. I make a lot of progress. - I study two languages. I study better. In Grades 4 -5, I was an excellent student. - I made progress since grade 1, I was an average student but now I am an excellent student ** MTBBE students of Ban Pho School, Lào Cai The students from the MTBBE programme are more confident than those who are not in this programme. Learning in J’rai, the students easily understand the lessons. They communicate better than non-MTBBE students. They are very active in the classroom and motivated to learn.

Parent of a MTBBE student, Ngo May School, Gia Lai.

Children are more participative and creative in the classroom than their peers who are not in this programme. J’rai children who learn in Vietnamese do not express themselves fluently in this language, especially in front of a visitor.

Deputy Principal in a primary school of Gia Lai.

This programme is a good opportunity for Khmer children to know their mother tongue. Students can communicate much better than those in non-MTBBE programme. They also look more active and friendly in communication.

School Principal, Trà Vinh.

- When we come back from school, we have group work after dinner. The main activity is to do our homework. In these groups, better students can teach the weaker ones. We also have group works at school. - We have study groups to help each other in Math but also to learn Music. We can learn modern music, learn the notes and also traditional music played using bamboo instruments. MTBBE students, Ngo May School, Gia Lai.



152

As I have mentioned, MTBBE students are able to know and understand more about their language and culture than those of non-MTBBE programme, though they are all Khmer students. I can express the difference between the two groups of students as follow: Non-MTBBE students have good achievement as well, but MTBBE students have more understanding. MTBBE students are more confident in daily life than the other group of students. Non-MTBBE students sometimes are quite timid, for instance, a Grade 1 student may not understand the command “sit down”. Obviously, the main differences between the two group of students is that MTBBE students learn in their mother tongue, and when they transit to Vietnamese, they are more active and more confident in taking part in learning activities, even in their daily life as well. Students like to take part in the MTBBE programme very much. But also some non-MTBBE students have asked me “Why could we not attend the MTBBE programme?” This question shows that these students perceived that by engaging to this programme, student can learn in Khmer, and have more opportunity to sing, play and dance in Khmer. School Principal of a Trà Vinh school.

** The use of a hyphen (-) at the start of each sentence indicates more than one person given the testimony in the respective group.

Box 5. Testimonies of programme effect on teachers´ attitude

Teachers are more enthusiastic. They have to learn a lot and do a lot of self-learning. They have to learn Mong to teach in Mong. They have to research and study learning materials. They have to learn Mong culture, so that they can integrate it into the education programme. This improves teachers’ ability to self-learning. Teachers have to translate materials from or to the target mother tongue. This provides teachers with opportunities to improve their teaching capacity. They also exchange views with textbook authors; this way they understand better the teaching materials and become very confident. They have strong teaching techniques compared to other teachers. School Principal in a Lào Cai school. -I am J’rai and teach J’rai. This programme is very meaningful to me because it helped me to learn some more words in J’rai. I can now understand J’rai technical terms. As a teacher of J’rai I have benefitted from workshops organized for the teachers in this programme. These workshops helped me to improve my teaching skills. The people from the RCEME are always ready to help us when we face some difficulties. I learned terms such as “plus” and “minus”, which did not know in J’rai but in Vietnamese. -Although there are many J’rai teachers, not all can teach in this programme because it is very challenging. This is because we did not learn any J’rai in the college, but we now have to teach in this language. This is why it is challenging to teach in this programme. -We learned teaching methods at the college but you have to keep learning new methods. The authors of the books help us to understand the contents and also support the training



153

of teachers. We learned how to use both J’rai and Vietnamese to explain content to the students. -This programme has changed our teaching mentality. For example, we learned methods for team building. This applies both to teachers and students. -This project was an opportunity for us to develop our teaching skills. We are now more confident in teaching. In the mainstream programme most of us could only teach grades 1 and 2. Because of that, all knowledge acquired in the college had been lost. However, thanks to the training received, we have updated and expanded that knowledge and can now teach up to grades 4 and 5. Teachers, Lu Tu Trong School, Gia Lai.** The students and teachers from the MTBBE project have to work harder than their peers who are not in the project because they have to stay at school from morning to afternoon. School Principal, Gia Lai. -MTBBE has helped us in our professional development. We have received training in new teaching methods, such as TPR [total physical response]. Now we can use different methods, be more flexible, and able to get that learning be a fun activity. -Training workshops are good not only to learn new methods; we also have the opportunity to share experiences, to suggest teaching method improvements. There is also a self-learning of the mother tongue, we have to read a lot, to learn about Khmer culture. Teachers, Nhi Truong School, Trà Vinh. MOET held some training courses to improve teachers’ language competence. After receiving these training courses teachers feel more confident in teaching. For instance, in the case of some abstract or complicated words that are hard for student to understand, particularly at Grade 5, teachers try to use real objects or pictures helping students to capture the meaning of the word. Teacher, Ham Giang School, Trà Vinh. Despite the programme´s training efforts, and changes in the attitude of teachers, teacher training still remains a challenge that will need attention in case of replication: Teachers are from the same ethnic groups as the students. They meet the standards of MOET. They meet the criteria set for the MTBBE Action Research. Many teachers can speak but cannot read and write in EMLs. To respond to that, we conducted many training workshops before we started the programme. We also trained them in research methodology. Due to limited time, the quality of these teachers is not as good as expected. They have not met the requirements of the programme. They are still not good enough in their mother tongues. RCEME Leader, Ha Noi



154

MTBBE teachers have good teaching skills. However, they need to improve a bit more. Their pronunciation in Vietnamese is not accurate. They also need to improve at Math, especially if they are to teach grades 4 and 5. Because of that, they sometimes confuse the students in class. That is why most of J’rai teachers only teach from grade 1 to 3. They are not good enough to teach grades 4 and 5. So, if the programme continues or is expanded, then for grades 1 to 3 the main teachers must be J’rai; Vietnamese teachers will act as assistants. However, for grades 4 and 5, Vietnamese teachers must be the main teachers and J’rai teachers the assistants. Both J’rai and Vietnamese teachers should be trained. Vietnamese teachers should also be trained to speak, read and write the EMLs used in their schools. For them to be good teachers they need to be good at speaking, reading and writing in these languages. Most of MTBBE teachers are very enthusiastic about teaching in this programme. The principals also follow suit. The DOET officials are also happy with the programme because they can see the learning outcomes of the programme. DOET Leader, Gia Lai. ** The use of a hyphen (-) at the start of each sentence indicates more than one person given the testimony in the respective group.

Box 6. Testimonies of Education Managers regarding the effect of training on their leadership competences

Through this programme and as a vice principal, I must have strong knowledge and teaching techniques. The programme has helped me to learn Mong. Now I know more about training techniques while being supported by specialists. I participate in developing textbooks, learning materials for Mong teachers. School Principal in a Lào Cai School. The MTBBE programme has also been useful to education managers. I recognize that it had some impact on me. I had a chance to attend workshops on education management. The project helped me to update my knowledge on teaching methods. This knowledge was an important asset in my classroom observation activities. So, the project contributed to improve the quality of education in this province. Of course there were some management challenges in this project. One had to do with the selection of teachers. We chose J’rai teachers trained in teacher colleges. They could speak J’rai but they could not read and write in this language. They needed to be trained for that. In fact, there are many J’rai teachers, but only a few can teach in this language. DOET staff, Gia Lai. Via some training workshops for MTBBE project and other types of workshop, my management skills have been greatly improved. I also attended a workshop on second language learning in Lào Cai two years ago that was presented by experts from a foreign country. Though this workshop did not discuss in-depth the teaching methods for any



155

specific class or programme, it did so for the general one. However, we apply this to the MTBBE programme and non-MTBBE programmes, especially at the primary level. This means that my professional capacity has been improved. From that, I can recognize what I should do as a manager. In addition, from exchanging experiences with districts and MTBBE schools, I can learn more how to manage at a district level for specific MTBBE activities and to exercise educational leadership in general. Furthermore, it helped me changing my perception of educational leadership; that is the focus on changing teaching methods, which was previously less focused, since previously more attention was paid to administrative activities. Innovation in teaching methods is a key point to enhance educational quality. Also staff from the Research Institute has provided a lot of experience in running and managing school activities, not only in workshops but also outside the workshops, to enhance the learning quality. DOET Leader, Trà Vinh.

Box 7. Testimonies of parents and community about understanding and support to the MTBBE

Before, as cultural practice, Mong girls had less opportunity to attend school. Girls got married early. Parents understood the programme and saw advantages and hence allowed more girls to attend school. Now parents grasp better what children learn in school. Teacher, Ban Pho School, Lào Cai. When this programme started, we were confused. We thought that children would only learn in J’rai. We asked ourselves how they would learn Vietnamese. However, after three years we changed our minds. We realized that they know both J’rai and Vietnamese. So we started to like the programme. Parent, Ngo May School, Gia Lai. Quality of education of school is improved every day. For example, at the beginning, children used to go to school for half a day, now it is full day. Children also teach Mong to their parents. This changes parents’ view about education. This makes parents more involved in their children’s education. Before, only few families paid attention to their children’s education. A majority of parents did not care. Also, before only fathers attended teacher-parent meetings, now mothers attend too. Parents bought and built a school fence. Parents register with the commune to build



156

washroom facility. Parents allow children to attend school for the full day. Parents allow children to volunteer to help others and community. They do not force children to stay home to work. Children volunteer enthusiastically. School Principal, Lào Cai. -Parents of programme students participate more than those of non-programme students. -They also help in providing school supply. -Very few students are absent from school. Before, when children were sick, parents did not inform. Now, when children are sick, parents come to inform the teacher that their children are taken to see a doctor. Teachers, Ban Pho School, Lào Cai.** -We provide learning objects and some school supply. When the school calls us we do volunteer work, like gardening, contributing bamboo to make a fence, and whatever the school needs. -As parents we have no role inside classrooms. But we keep alert of teacher petitions for contribution. Parents, Ban Pho School, Lào Cai. -I have a girl in this programme. She can read both J’rai and Vietnamese. I can read in Vietnamese, but not in J’rai. So I help her when she has problems in Vietnamese but not in J’rai because my schooling was not in J’rai but in Vietnamese. -It is difficult for me to help my children at home because students work with various J’rai dialects, which are different from the one I speak. So we are learning those J’rai varieties from our kids. Parents, Ia Phi School, Gia Lai. I learn what my children learn in school, such as, planting trees, raising animals, preserve Mong tradition, Mong music during weddings and funerals. Parent, Lao Chai School, Lào Cai. -Non-MTBBE teachers teach only in Vietnamese. They do not need much collaboration from the parents or from the community. -In the past we thought that we were just farmers and could not help much the kids. However, now that J’rai is used in schools, we can check the books and understand that teaching in J’rai helps to preserve the J’rai language and culture. That is why we are happy to come to the classroom when a teacher needs us to share something with the children. We decorate the school and donate utensils. Before the programme we just wanted our children do came to schools and learn. Parents, Ngo May School, Gia Lai. -My parents help me to do my homework. They help me to do calculations and other things that I don’t understand.



157

-I teach my parents to read and write in J’rai and Vietnamese. I help them to pronounce and how to spell and write some words in J’rai. -I teach my mom to read and write in J’rai because my father already knows it. I open my books and she finds difficult the alphabet in J’rai, so I teach her. She can understand the easier letters, since she can read and write in Vietnamese only. Students, Ngo May School, Gia Lai. Parents love their children learning in Khmer from the beginning of the programme because Khmer people like their mother tongue to be preserved and developed. Thus, from the beginning they wanted their children to take part in this programme. Besides, during the deployment of the programme, by seeing that their children look more confident, children have opportunity to learn in their mother tongue, they can freely communicate in their mother tongue, parents liked the programme more. DOET Leader, Trà Vinh -I’m a Khmer but I don’t know Khmer letters. This makes me unhappy. In my opinion, the students who are in the MTBBE programme are always more advanced to those of the non-MTBBE programme. My son can read everything that appears on TV, but not me. The more they know Khmer the more they want to learn. -Before my child knew some Khmer and Vietnamese, but his Vietnamese was better. When he participated in MTBBE programme the proportion of Khmer and Vietnamese that he knew are equal. -Without this programme, children would not have learned Khmer. In the future they can work as translators. The more languages you know, the better you are. Parents, Ham Giang School, Trà Vinh. The community supports the programme. For example, they teach folk dance to students and tell stories in mother tongues and participate in all curricular activities aiming at teaching the mother tongues. They help the DOETs to advocate for the programme and train teachers. DOET leader, Lào Cai. The local authority tries to propagate the MTBBE programme to the community. They also help to translate stories, folklore and folksongs for children to Vietnamese. DOET staff, Trà Vinh. We have used local resources. In terms of teaching the Khmer language, we receive great help from monks in different pagodas who help to teach Khmer to children in the community. Even when conducting this programme, monks and Khmer people had provided a lot of ideas. Some other units also support us a lot as well. Trà Vinh University has so far helped us in teaching Khmer to teachers because it has specialists in Khmer language teaching. We will make use of this support. However, the role of MOET and the Research Institute for Ethnic Education is irreplaceable, especially in the adjustment of



158

textbooks. DOET Leader, Trà Vinh. ** The use of a hyphen (-) at the start of each sentence indicates more than one person given the testimony in the respective group.

Box 8. MTBBE Action Research´s long-lasting benefits on education

One of the objectives of the pilot programme is to influence policy change or change in the education system that would promote ethnic minority education. It seems that there are other potential benefits that may have lasting positive effects, beyond the change of language of instruction, and I would just like to mention them here so that they can be kept in mind and considered in future evaluations of the project:

1. Articulation of the curriculum between pre-primary and grade 1: Curriculum developers worked hard to articulate the two levels, which on the surface are quite different since pre-primary is organized developmentally by theme, while grade 1 takes a traditional approach to content area division. It appears that the use of materials like Big Books will promote further integration of developmentally appropriate activities from the pre-primary where they are well accepted into the early primary curriculum.

2. Streamlining of the national curriculum (by cutting out overlapping content between subjects as well as superfluous content overall), which could have a positive return effect on the national curriculum if the pilot curriculum is taken as a model that works.

3. New approaches to materials development, piloting, editing and publishing could have a positive effect on all educational publishing.

4. Bilingual materials and methods may be examined as models if MOET pursues its current policy of including English as a foreign language starting around grade 3.

Source: Carol Benson, 2008, page 8.

Annex 7 – AR MTBBE Detailed Financial Information

Table 1. MTBBE Action Research costs by component 2008-2014

Component Amount Percentage

All training costs 1,690,000 48.55

Learning materials (consultants, printing, etc.) 1,321,000 37.95

Monitoring and evaluation 470,000 13.50

Total 3,481,000 100.00




159

Table 2. MTBBE Action Research Costs by year and component Component 2008 2009 2010 2011 2012 2013 2014 TOTAL

All training costs 165,000 120,000 330,000 310,000 255,000 240,000 270,000 1,690,000

Learning materials (consultants, printing, etc.) 133,000 170,000 205,000 365,000 225,000 140,000 83,000

1,321,000

Monitoring and evaluation 20,000 50,000 80,000 50,000 50,000 50,000 170,000 470,000

Total 318,000 340,000 615,000 725,000 530,000 430,000 523,000 3,481,000




160

Annex 8 – Systematised list of benefits of the Action Research MTBBE

1. One student cohort of 247 ethnic minority children [132 girls] was able to graduate from the primary level. Presumably the 238 students [138 girls] of the second cohort will graduate too from Grade 5 by mid-2015. These figures will make a total of 485 students [270 female students].

2. Considering the context where target schools are located, the retention rate of these two cohorts is high, particularly in the case of girls. On the base of standardized tests, none of the MTBBE students indicated a “weak” learning condition. Periodical standardized comparisons between MTBBE students and samples of students of the mainstreaming system revealed better learning achievements on the part of the first group. As one local education authority argued, “it is clear that MTBBE students have a better understanding of the subjects, Vietnamese included, than their ethnic minority counterparts educated in the conventional system.”

3. MTBBE students became bilingual even before Grade 5; confidence, self-esteem, originated from a friendly learning environment, played a role for this achievement. Given this fact, they have better probability of success in the next educational level. Some of them have the potential to become future MTBBE teachers. Their future employability might be higher than that of other children with lower quality education.

4. The MTBBE programme trained about 564 main and auxiliary teachers during the pilot period. In the future these teachers are going to need less training than at the pilot stage; some may become trainers of new teachers.

5. Education managers were also trained. In the future these managers are going to need less training than at this pilot stage; some may become trainers of new managers and teachers.

6. About 2,900 copies of textbooks for all primary grades in subjects like mother tongue (textbook and a workbook for writing), Vietnamese, Mathematics, Social Sciences, Natural Sciences, History, and Geography were printed and delivery. In a future scaling up in areas of the same target languages and similar socio-economic conditions these textbooks will be ready for use requiring maybe only minor review.

7. The production of learning material left a group of ethnic minority technicians with experience in learning material development; they are a potential base for developing material teams in a scaling up stage.

8. The pilot is leaving a good base of supporters of the MTBBE, e.g. parents, teachers, managers, community leaders, policy makers, others.

9. The pilot would have contributed to governance [grateful parents and communities], ethnic cultural appreciation and pride. It can be the first solid step toward a society consolidating and developing the asset of language/cultural diversity.

10. The pilot has had a demonstration effect. The first achievements became a reference point for success and replication. In fact, the province of Lào Cai opted for a scaling-up of the approach and there are other provinces, like Giang, that have manifested their will to follow the same route.

11. Policy makers have been particularly impressed with the MTBBE approach: “Government policies are changing, in part inspired by the AR MTBBE initiative. These



161

changes have been implemented yearly, including changes in textbooks and teacher training in order to enable them teach in the mother tongues, this new policy started in 2010”130. Perception has changed from central to local educational authorities, from community to policy makers, that Vietnamese (and MT) is not to teach in a mechanical way131.

12. Impact on rethinking teaching methodology. MTBBE pilot is more than a reference for bilingual education. It has also valuable components of excellent student-centred methodology. The action research tool is a good example for basic and secondary educational levels. It is a proactive way to detect and solve problems in a participatory fashion. It brings together teachers, education managers, community and other stakeholders.

13. If AR MTBBE has proved to be a functional facilitating bridge for the transition toward high quality education of ethnic minority children, its replication where needed will constitute a key tool for fighting poverty, social exclusion and inequality. More observations are presented in Annex 6, Box 8.

In sum:

We witness change of attitudes, improvement of teachers’ qualifications and raised community awareness. The programme opened new perspectives for the education of ethnic minority children. 132 Since independence, Viet Nam manifested a will for delivering high quality education to ethnic minority children; we wanted to do it, we tried different approaches. This programme has taught us a promising way to go. This is its major asset.133 For any pilot programme initial expenses are huge, but those gradually reduce. The most important thing to consider is the outcomes of the MTBBE Action Research, which are high in relevance and effectiveness as different stakeholders recognized it.134 Now MTBBE is only an experiment and, because of this, it is expensive. However, the results indicate that this is a value for money relationship.135

130 Official from the Ethnic Minority Department of the National Assembly. 131 Official from VIES. 132 High level officer, VIES, Ha Noi. 133 National expert, Ha Noi. 134 High level officer, Department of Primary Education, Ha Noi. 135 DOET technical staff, Trà Vinh.



162

Annex 9 – Additional information for the SWOT analysis

MOET´s policy guidelines High-level educational authorities in Ha Noi have defined a policy guideline regarding MTBBE replication and scaling up. First, MTBBE has reached high-level approval as it fits well into the policy of the Communist Party regarding ethnic minority people in Viet Nam136. MOET, the Committee for Ethnic Minorities and the Ethnic Council of the National Assembly all support the programme. “This support is reflected in speeches and documents regarding the UNICEF initiative in particular. All this tells us that MTBBE is sustainable”.137 Second, facilitating changes for MTBBE are on the way. The Government is conducting a comprehensive education reform. “We are reforming the curriculum and general education. The curriculum reform will be discussed in the next meeting of the National Assembly. MTBBE is one of the solutions adopted by MOET to address the education challenges of ethnic minorities. Another project is the Viet Nam Escuela Nueva. The implementation of these initiatives will lead the government to propose solutions as to how to implement the outcomes in the new curriculum”.138 In the upcoming reform the proponents expect to develop a new management mode based on decentralization. The central level will be in charge of curriculum design and the provinces will provide the materials in a very flexible way. Even in terms of curriculum, the provinces will be allowed to make local adaptations to some relevant aspects. Based on the national curriculum, schools could develop their own curriculum. Once education sectors start making changes, MOET will also make further changes regarding the MTBBE. Third, local scaling up will be authorized under the following conditions: (i) the language of the ethnic minority group should have a written system; (ii) the ethnic minority group has to live in a specific area, not a dispersed one; (iii) there should be a sufficient number of teachers who can speak the EML to be used in schools; and (iv) there should be available teaching and learning resources to implement the programme. The local governments will decide on the implementation. The DOETs are supposed to develop the plans and submit them to the provincial plan. It is up to the provincial authority to approve that plan, or not. It is expected that the Department of Childhood will advocate for the implementation of the MTBBE in the provinces.

136 Emphasis on mention the Communist Party came from more than one interviewee; a fact that reveals

the importance given to the role of this party in the context of Viet Nam. 137 VIES´s high-level officer. 138 VIES´ high-level officer.



163

Fourth, training of teachers should be at the pre-service level. It is necessary to prepare the teachers to teach in two different areas: the MTBBE and the mainstream programme in Vietnamese. In order to reach this goal the teachers must be prepared three years before the start of the programme since the training programme takes that time. This condition is considered key to reduce dependency on international support. Fifth, the support from the central level is limited to technical assistance as soon as financial support is already integrated in the budget of the provinces. “Lào Cai is an example, they set the MTBBE as a priority for the education sector; consequently they allocated seven billion dons to that initiative. The province of An Giang is the fourth province that decided to implement the programme using the resources from the provincial budget and from the central budget.”139 Regarding this last paragraph another interviewee makes clear that financial support integrated in the budget of the provinces maybe insufficient, being so,

The provinces approve the education plans but they may need the financial support of the central government to implement the programme; they may also need to find support from NGOs, businessmen and international government agencies. For example, if UNICEF supports the development and supply of materials, many provinces could implement the programme.140

Finally, provinces are authorized to applying and receiving external support as indicated in the last quote above.

Capacity of replication The main Strength of MTBBE scaling up are the present general favourable policy environment and the results coming from the recent pilot experience. MOET´s policy guidelines could be considered the immediate policy framework for the scaling up process of the MTBBE in Viet Nam. This means that in general, current policy guarantees support to the MTBBE from national and provincial governments. How many provinces are in need of the MTBBE? How many meet the basic criteria implied in the current policy guidelines for a scaling up process? Insufficient information makes hard to answer these questions. A departure point could be an effort from MOET to prepare a Map of Needs and Conditions for the MTBBE. The Map would be based on demographic and education indicators at district level, such as, the number of ethnic minority populations, the number of ethnic minority school-age children, school enrolment rates, dropout rates, rates of school approval, and other key education indicators, all this information breaking down by ethnic group, age and gender. It is possible that most of this information is already at the reach of MOET, so the task will be

139 High officer of the Primary Education Department, MOET. 140 RCEME´s high officer



164

easier to be undertaking. Districts/provinces that combine the highest amounts of ethnic population with ethnic minority children showing the worse education indicators will be first in the ranking of needs that the Map will show. This will be a first layer of the Map. A second layer of the Map will be a ranking of provinces regarding the degree of fulfilment of MOET´s criteria to authorize a scaling up process in provinces. Both layers will indicate needs and conditions for starting MTBBE. Again, this would be just a starting point that will need updating, enrichment/adjustment with more and pertinent indicators (for instance, data from linguistic maps), to be done as soon as possible under the guidance of experts. The Map will support the decision-taking process to select the districts/provinces having priority for the scaling up process. The ranking of districts/provinces may change as soon as the database is enhanced, but meanwhile the scaling up will start with basic available information. A strategy is needed for the scaling up process. The Map will be one key tool of this strategy. As the Map is improved, the strategy will gain in refinement. What is implied in these advances on recommendations is the idea that the scaling up will need the support of a more proactive policy that materializes in a MTBBE scaling up strategy. The current MOET´s policy guidelines are a good start for this more proactive policy, i.e., the strategy could be prepared and socialized as soon as possible, as should the building of the Map. The point is to take advantage of the momentum produced by the general success of the MTBBE pilot.

Leadership and ownership Since the start of the programme MOET showed strong commitment as the leading national institution, a fact recognized in different documents written by external consultants and UNICEF´s officers. The evaluation team was greatly impressed by the quality of answers of most interviewees. Quality was manifested in how well informed and enthusiastic they looked within the realm of their responsibilities. Teachers, school principals, DOET and provincial authorities, central level educational authorities, technicians and political leaders of the National Assembly were able to provide information on details and shared views on their concerns and hopes for the future of the programme. These are indications of ownership and leadership. Nevertheless, some basic assumptions that nurture the action of this leadership may be in need of conceptual and policy refinement; for instance, the meaning of cultural



165

diversification141 in the case of Viet Nam, and the differences of language education models, such as, early immersion method of teaching Vietnamese, the early-exit and the late-exit bilingual education methods. This weakness may turn into strength by an adequate reinforcement of basic awareness efforts. The pilot programme built leadership and ownership. In general leadership seems to be in place for sustainability, but an important number of stakeholders perceive the scaling up process will still need some external, additional, financial support because it is not only a question of size but of quality of the scaling up in terms of friendly learning environment, active teaching methods, teaching Vietnamese as a second language142, recording daily practice, and so on.

Financial resources If provincial governments receive budget transfers from the central level and they have autonomy on how to expend those resources, it is to be expected that provinces can assume the cost of implementing promising initiatives such as the MTBBE. From a formal standpoint this is a correct expectation. However, it is well known that provinces with the highest concentration of an ethnic minority population are, in general, the ones with more financial problems. For local leadership to invest in the MTBBE may be politically attractive. However, leaders may be equally apprehensive of the financial cost of the programme during the inception stage. On these terms financing may become an obstacle for sustainability. To anticipate this risk more dialogue between central and provincial authorities is advisable which may lead to design appropriate incentives for the scaling up process.

Teaching and learning materials MOET has a long experience with developing learning materials for the mainstreaming the educational system. Additionally, international consultants brought in additional expertise regarding the pilot programme that has specially benefitted local teams for learning and teaching material development. As a result, Viet Nam has good know-how on developing teaching and learning materials. In the case of the provinces participating in the pilot programme, learning materials will only need review and updating if the scaling up process proceeds, although diversification of this material will be observed there as a challenge, e.g. dictionaries, grammars,

141

Cultural diversity is used here in the same vein of the UNESCO Universal Declaration on Cultural

Diversity of September 11, 2001. Article 1 of this declaration views cultural diversity “as a source of

exchange, innovation and creativity… cultural diversity is as necessary for humankind as biodiversity is

for nature. In this sense, it is the common heritage of humanity and should be recognized and affirmed for

the benefit of present and future generations”.

142

The method of teaching Vietnamese as a second language was developed and written by MOET,

which also organised training for provincial leadership teams of all MTBBE target provinces in 2014.



166

teaching guides, learning aids, etc. Another challenge would be the case of provinces that will demand learning materials in new EMLs. In this case, the know-how already accumulated will be useful, however expenses for developing and printing will be unavoidable. There will be a need for research on new low-cost viable technologies. Meanwhile it must be recognized that this is an important cost in a scaling up process. According to RCEME authorities, the situation regarding learning materials is under control since provinces design their own material. This surely reduces central costs but does not take them out of the picture completely.



167

Qualified teachers As reported, a good number of teachers speaking EMLs is available in some provinces, but not in others. The second group of provinces would have to wait to meet this condition before going into a scaling up. L1 speaking teachers have received their professional training from local teacher training schools and universities. Many of these teachers were able to go into these training institutions thanks to special incentives given by educational authorities; there they were trained to teach according to the mainstream curricula. Some of them learned to teach their respective L1 as a subject. This means, availability of L1 speaking teachers is only the departure point for additional in-service training that will prepare them to apply the MTBBE teaching/learning methodology. MOET is aware of the need that bilingual education be a part of the curricula of the pre-service training of teachers; reforms in this regard are already on the way. Key educational authorities expect this type of training will help to reduce the dependency on foreign resources for the MTBBE. Also, provinces in need of L1 speaking teachers may expect to have a midterm solution for this problem.

Local/DOET financial resources Provinces in Viet Nam have financial resources that they collect from internal revenues and transfers from the central government. Internal revenues generally are insufficient to cover local government expenditures, thus transfers help to cover the imbalances. In 2012, close to three quarters of overall capital expenditure and 56% of recurrent expenditures were devoted to local governments. Most of the spending given to local governments was addressed to education, health care and social security. In addition, provincial authorities are free to determine the fiscal relationship (expenditure assignments and revenue sharing) with districts and communes within their jurisdiction.143 In short, provinces can take their own decisions on how to use their financial resources, as Lào Cai deciding to invest in MTBBE. Some key informants in MOET argued that provinces do not need permission or additional funding to follow the example of Lào Cai.144 However, it is known that the provinces with important proportions of ethnic minority groups are, in general, the poorest ones. This means that a decision like the one taken by the Province of Lào Cai is not an easy one since it will be done at the expense of other social needs and under a context of very limited resources.

143 Truong, 2014). 144 See fifth guideline, section 4.4.2.



168

Apprehension shown by local leadership of two target provinces regarding the scaling up of MTBBE could be a sign that decision makers are aware of the risks (costs) in terms of sustainability of the starting stage of MTBBE. It may advisable to think about policy incentives to facilitate the decision.


REPORT

169

Annex 10 – Summary of findings, conclusions and recommendations Findings Conclusions Recommendations

Design

The AR MTBBE lacks a Project Initiation Document (PID) defining the basis for the management and assessment of its overall success. As a consequence, instead of using a monitoring system based on performance indicators, the monitoring of the AR MTBBE focused on learning indicators such as the number of training workshops and the development, printing and delivery of teaching and learning material. In line with the absence of a PID, the AR MTBBE also lacked a theory of change, laying down what the programme was meant to achieve and the pathway through which it was expected to achieve this. The theory of change was (re)constructed a posteriori by the evaluators in order to facilitate the assessment of performance.

1) The lack of a PID constrains the evaluation of any project since it becomes hard to contrast planned activities versus success as well as to generate performance indicators. Although the MoU between UNICEF and MOET (August 15 2007) and the Implementation Framework of the AR MTBBE offer clues for generating performance indicators, the fact that they lack precision makes it difficult to make accurate judgements about the overall success of the programme, except regarding the indicators of learning outcomes. The lack of a Theory of Change also limited its use as an instrument for planning and monitoring the process from inputs to outcomes/impacts and with this the value added of the programme to the development goals of the country and/or UNICEF.

1) Develop a PID, including a theory of change, for the next programme delivery, clearly defining performance indicators to be used in the management and assessment of its overall success. This should be part of any programme/project initiative. (Responsibility: MOET and UNICEF)


REPORT

170

Findings Conclusions Recommendations

Relevance

Finding #1 The MTBBE approach is relevant and responsive to the socioeconomic, legal and policy context of Viet Nam and UNICEF´s focus on equity. MTBBE translates the Vietnamese legal and policy framework for a more inclusive and equitable educational system, as expressed in the Constitution, Education Law, Education Strategic Development Plan (2011-2020), among other relevant laws and policy frameworks. From the cultural point of view, the relevance of the programme is closely linked to its potential to contribute to the preservation and development of ethnic minority languages and cultural practices and beliefs. Despite the enabling implementing environment attested, a review of Vietnamese key legal and policy

2) The MTBBE approach translates international and Vietnamese legal and policy framework for a more inclusive and equitable educational system and is relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. However, the fact that the existing policies and legal frameworks in Viet Nam are not explicit about the use of EMLs as languages of teaching and learning and the absence of a specific policy on how MTBBE should be implemented may constrain the expansion and scaling up of the programme, as expressed by key stakeholders, in particular those from the local level.

2) Review the existing Vietnamese legal and policy frameworks with the view of strengthening them clearly towards MTBBE. Responsibility: MOET, in coordination with the CEM- National Assembly and with the support of UNICEF 3) Show strong political will to introduce mother tongue learning by establishing a specific policy laying down how the MTBBE approach should be implemented in Viet Nam. Responsibility: MOET

3) The MTBBE approach has proved its potential to respond to educational needs of ethnic minority groups as well as its potential to contribute to the preservation and revival of ethnic minority languages and cultural practices and beliefs.

4) In the current context of curriculum and general education reform in Viet Nam, incorporate MTBBE as one of the solutions to improve quality and equity for ethnic minority children in this country.

Responsibility: MOET leading the advocacy


REPORT

171


instruments suggests the need of revisiting/ strengthening the existing policies and laws clearly towards MTB-MLE. The MTBBE approach has also proved to be relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. By helping ethnic minority children to overcome the language barrier in school, MTBBE can be regarded as one of the platforms than can be used to improve the academic performance of these children and set the ground for official use of EML in other formal setting, including in the health and legal sectors.

However, the evaluation suggests that differences in terms of historical trajectories and language attitudes and practices in Viet Nam may lead to different perceptions about the relevance of the approach.

component, with the support of UNICEF

5) Conduct careful sociolinguistic studies before implementing the MTBBE approach in new ethnic minority groups in Viet Nam. This may give indications about the relevance and the likelihood of success of the programme in those settings. Responsibility: MOET, in coordination with UNICEF in the search of resources and advocacy

4) As international experience has shown, this evaluation also indicates that bilingual education should not just be viewed as a response to the learning challenges faced by ethnic minority children. It has also to be seen from a broader perspective, linked, for example, with efforts for inclusiveness, cultural preservation and improvement of the living conditions of ethnic minorities in Viet Nam. The rationale here is that the same way children perform better in school because a language familiar to them is used as medium of teaching and learning, ethnic minority people in Viet Nam can be better served and participate more effectively in

6) Develop a strategic advocacy/ communication plan to ensure that relevant institutions issue policies on the use of EML in other official arenas, including in basic service delivery sectors such as health, child protection and legal. Responsibility: UNICEF, MOET and other institutions promoting social inclusiveness in Viet Nam


REPORT

172


social, economic and political life if their languages are legitimately used as vehicles of communication in official arenas.


Effectiveness

Finding #2 The programme has been effective in developing and implementing a context-appropriate MTBBE model that has created conditions to improve the learning outcomes of participating EM children and the professional capacity of teachers and education managers. By offering the programme from pre-school to the end of primary education and adopting a flexible approach, the curriculum design allowed the development of both the students’ mother tongues and Vietnamese right from the first day of schooling as well as offered optimal conditions for transfer of knowledge from one language to the other. Nevertheless, there were reports of misapplication of flexible teaching

5) The MTBBE programme was based on an effective curriculum design, one that allowed the development of both the students’ mother tongues and Vietnamese and offered optimal conditions for transfer of knowledge from one language to the other. Taking into account international experience and the results of this programme, it can be said that the design adopted has the potential to produce bilingual and bi-literate students, that is, those who can speak, read and write in both Vietnamese and their mother tongues as well as facilitate learning of content subjects in both languages of schooling and set the conditions for the maintenance of ethnic minority languages.

7) In the scaling up phase, replicate the design and the flexible approach adopted during the pilot stage as they yielded a strong form of bilingual education, which is recommended for the socio-cultural context of Viet Nam. This design should form part of the future policy on MTBBE in Viet Nam. Responsibility: DOETs and MOET

6) The pilot phase of the programme counted with regular in-service training and support that gave MTBBE teachers and education managers the opportunity to improve their professional capacity.

8) As part of the preparation of the scaling up phase, organize a seminar/workshop aimed at analysing and reviewing the teaching-learning methodologies used in the MTBBE to recommend adjustments to


REPORT

173


methodologies as well as some insecurity in the teaching of Vietnamese as a second language, in particular in earlier stages of the programme.

Despite the overall effectiveness of those initiatives, reports of misapplication of flexible teaching methodologies and insecurity in the teaching of Vietnamese as a second language suggest that capacity building should continue to be one of the top priorities of programme implementation in particular at the scaling up phase. Indeed, international experience indicates that capacity building is key for a successful implementation of MTBBE programmes, in particular in contexts where there is no long tradition of using local languages in education, as it is the case with some EMLs in Viet Nam.

make them more responsive and appropriately applied. Responsibility: MOET and UNICEF

7) Teaching and learning of Vietnamese as a second language is often taken as a concurrent approach to MTBBE in Viet Nam. However, as international experience has shown second language teaching-learning methodologies are not necessarily in conflict with the MTBBE approach. Second language teaching-learning methodologies should be applied in MTBBE-MLE as long as they draw on approaches that promote the use of bi/multilingual resources in classroom interaction as a legitimate communicative

9) Adopt multilingual-oriented second language methodologies in the teaching and learning of Vietnamese as part and parcel of the MTBBE approach. This may include the involvement of NGOs promoting the teaching and learning of Vietnamese as a second language in Viet Nam (e.g. Plan International) in training workshops/continuing professional development programmes for MTBBE teachers. Responsibility: MOET leading the process,


REPORT

174


and pedagogical strategy. with the support of UNICEF.


REPORT

175


Impact

Finding #3 The MTBBE has produced factual and perceived impacts on its intended beneficiaries. According to different internal/official internal and evaluations, children learning outcomes have been acceptable in all schools of the three provinces, as compared to national standards. In addition, children display confidence, self-esteem and good communication skills. According to some indicators, girls have benefitted relatively most from the approach. Teachers, school managements, board members and all levels of education managers generally supported the programme and recognize its effect in improving the their competences to successfully apply the MTBBE design and contribute to develop policies and practices promoting the use of EMLs. Parents showed awareness and support to the MTBBE and recognized its benefits in the life of their children.

8) The overwhelming positive academic results obtained by ethnic minority children, with special reference to girls, and its consistency with national standards confirm the pedagogical value of an education programme based on learners’ mother tongues. The familiarity with the language of instruction makes EM children feel at ease in the classroom and creates the conditions for their active participation, which are important preconditions for learning. The positive results achieved in Vietnamese also show that, instead of hindering, MTBBE facilitates the learning of the second language, the language regarded as the gateway to social integration and mobility in Viet Nam. The general conclusion is that MTBBE has the potential to contribute to mind learning outcome gaps between Kinh and EM children and also between boys and girls, thus paving the way towards inclusion and equity in education and in the Vietnamese society at large.

10) Conduct (longitudinal) studies of former MTBBE students’ performance at secondary school in order to assess the educational impact of MTBBE beyond primary education level. The results of those studies will provide further context-specific evidence about the feasibility and validity of the MTBBE for ethnic minority students in Viet Nam. Responsibility: MOET (RCEME/VIES), in coordination with UNICEF in the search of resources and advocacy


REPORT

176


9) As discussed, while recognizing the sociocultural value of using EMLs in formal education, parents and other community members were initially unsure about the pedagogical value of MTBBE, in particular in creating opportunities for effective learning of Vietnamese. The fact that their attitudes and perceptions changed over the course of the programme, especially when they perceived substantial academic results exhibited by participating students, suggests that the achievement of positive educational and sociocultural outcomes may prove to be the best way to advocate for MTBBE. In other words, while drawing on international experiences, advocacy initiatives should involve key actors (students, teachers, parents, education officials, etc.) and use the results from the pilot phase as the main arguments in favour of the MTBBE.

11) Develop a Behaviour Change Communication Strategy (campaign) to raise awareness of parents and communities on the purpose and benefits of the MTBBE and ensure their support and participation as well as to mobilize collective action in support of the programme.

Responsibility: MOET, in collaboration with UNICEF.


Efficiency

Finding #4 Evidence provided by MOET and UNICEF

10) As international experiences have shown and also recognized by key

12) Scale up the MTBBE programme, involving more schools and EM students,


REPORT

177


indicates that monetary and time resources were used in an economically acceptable manner. Stakeholders expressed awareness of the cost vs. results relationship, and most think that the unit costs of the programme are high but compensated by its returns (opportunity costs), particularly its immediate impacts on learning outcomes and teacher training, as well as its contribution to capacity building for policy making.

participants in this evaluation, the expenditure of the pilot programme was higher than the average per pupil expenditure in primary education in Viet Nam because of its pilot nature. The reliance on regular in-service training, the innovative effort placed on material development and the small size of the student population served are the main factors which explain the relatively high costs incurred in the pilot phase. However, the positive results of obtained, including the improvement of learning outcomes of ethnic minority students and change of the mind-set of education managers regarding the role and value of EMLs in education indicate that the MTBBE is a worthwhile investment. In fact, as attested internationally, in a post-pilot phase, the cost of the programme is likely to be lower and similar to that of the mainstream programme in Vietnamese.

thus maximizing the academic and sociocultural benefits of the approach and reducing its costs by capitalizing on economies of scale. Responsibility: MOET (DOETs, RCEME, VIES)

11) The availability of basic and good quality teaching and learning materials is a necessary precondition for a successful implementation of any educational programme. The evidence gathered suggests that this precondition was, in general, fulfilled in the AR MTBBE.

13) Consolidate all reference materials (i.e. teaching guides, grammar books, dictionaries, additional reading and reference books, and visual aids) in the list of basic materials to be produced and supplied to MTBBE schools.


REPORT

178


However, in addition to very basic materials, it is also necessary to furniture bilingual schools with complementary material to aid teaching and learning, in particular in EMLs. As attested internationally, when students, teachers and parents understand that education programmes in low status languages are poorly equipped in terms of teaching and learning aids (and also in terms of quality of the teaching staff) when compared to programmes in the mainstream language, as it is the case of Vietnamese, they may opt out those programmes. This is to be avoided, in particular considering the equity agenda underlying the provision of education in EMLs.

Responsibility: MOET (DOETs, RCEME, VIES)

14) Mobilize independent writers and publishers to produce low-cost and/or no-cost reference materials in EML for MTBBE schools; this may include award offers and promotion of contests and competitions culminating with the publication of the best manuscripts, including on-line publication.

Responsibility: MOET (DOETs, RCEME, VIES)


Sustainability

Finding #5 The AR MTBBE has created basic favourable conditions for future replication and scaling up of the bilingual programme. In addition the socio-political climate is also supportive of the scaling up of the

12) As international experience has shown, in addition to technical preparedness, political will is one of the key factors for a successful and sustainable implementation of MTBBE initiatives. Although, in general, these conditions are in place in Viet Nam, there

15) Develop a strategy for a gradual expansion and scaling up of the MTBBE programme. This should have two goals: (a) the continuity and dissemination of MTBBE in the three pilot provinces and (b) the design and implementation of new


REPORT

179


MTBBE programme to new schools and new ethnic minority groups in Viet Nam. However, for a sustainable scaling up of the MTBBE programme there will be a need to reform the education law to allow EML could be in the role of language of learning, together with supportive policies and permanent dialogue between central and provincial governments.

are some other factors which may threaten the sustainability of the programme if not adequately addressed. These include mainly claims of lack of financial resources to implement the programme in some provinces and the lack of a clear policy on MTBBE in Viet Nam. In fact, the need of financial support for teacher training and material development is generally perceived by stakeholders, especially at the district and provincial levels, as the main challenge for a sustainable replication and scaling up of the programme. Other less voiced constraining factors include the difficulty in managing language diversity in some provinces and the existence of alternative approaches to education of ethnic minority children, especially the teaching of Vietnamese as a second language, usually regarded as less expensive and less demanding.

MTBBE interventions in provinces with characteristics similar to those of the provinces involved in the pilot stage or in other provinces fitting MOET’s criteria.

Responsibility: MOET, in coordination with UNICEF and relevant DOETs

16) Continue to provide financial support for teacher training and development of teaching and learning materials to the experimental provinces for three to five years while they will continue to find other donors to sustain the programme. Responsibility: MOET, UNICEF and other donors of the education sector in Viet Nam.

17) Provide support to teachers for them

to effectively deliver a multi-language

policy by ensuring pre- and in-service

training which lead to the improvement of

their own proficiency in the language of

instruction and in teaching skills in ethnic

minority languages, including early

literacy development in these languages.

Responsibility: MOET


REPORT

180


18) Establish and maintain a dialogue with relevant provincial authorities in order to clarify responsibilities and find solutions for the financial needs of local governments to sustain current MTBBE efforts and results and set the bases for the expansion and scaling up of the programme in those provinces.

Responsibility: MOET and UNICEF

19) Bring language of instruction to the

forefront of dialogue on education sector

funding with the government partners,

partially in relation to both quality and

equity concerns, ensuring the coverage of

MTBBE is highlighted as international

indicator of education quality.

Responsibility: UNICEF and other development partners

20) Mobilize resources to support the

introduction of MTBBE and offer technical

assistant to develop flexible competency

targets, literacy materials and an


REPORT

181


assessment framework in multiple

languages.

Responsibility: UNICEF and other development partners


General contribution to gender and equity

In general terms MTBBE has also contributed to the objectives of UNICEF Viet Nam on gender and equity. MTBBE is per se an equity-oriented policy instrument as soon as it attempts to fill ethnic and gender gaps in education by providing an effective opportunity to ethnic minority children to overcome the language barrier. The efforts in terms of policy made under the MTBBE approach are also addressed to attain equity educational goals in favour of ethnic minority populations.

13) The MTBBE approach has also proved to be relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs. By helping ethnic minority children to overcome the language barrier in school, MTBBE can be regarded as one of the platforms than can be used to improve the academic performance of these children and set the ground for official use of EML in other formal setting, including in the health and legal sectors. As a part of the equity efforts, the MTBBE made special efforts to have good participation of girl students. The programme achieved well this objective since girls came to represent a small

See recommendation No. 6. MTBBE is an important tool for addressing gender and equity issues in Viet Nam (more girls in schools, more girls finishing primary school, better participation of girls and so on). UNICEF should use the results of this programme to show that by using EM languages in other arenas (e.g. child protection, health and legal sectors) more women and EM people in general could attend/participate/benefit more from those services, which is a way of social integration and equity.


REPORT

182


majority of the total enrolment. The evidence indicates that girls’ academic performance was as good as that of boys.



183

Annex 11 – Questionnaires 9.1 Key Informant Interview Questionnaires Four versions of the key informant questionnaire were required for:

KIICM Key Informant Interviews with MOET and other central level managers KIIL Key Informant Interviews with DOET and other provincial and local

authorities KIISP Key Informant Interviews with school principals KIIU Key Informant Interviews with UNICEF management

1. Relevance

To what extent do you know and understand the MTBBE? (1.1.1) KIICM KIIL KIISP

To what extent is the MTBBE relevant and responsive at the national level? (1.1.2.1) KIICM

To what extent is the MTBBE relevant and responsive at the sub-national level? (1.1.2.2) KIIL

To what extent is the MTBBE relevant to national policies? (1.3.4) KIICM

To what extent is the MTBBE relevant to Institutions in the national education system? (1.3.5) KIICM

To what extent is the MTBBE relevant to MET’s Educational Development Strategy 2011-2020? (1.3.6) KIICM

What role has MTBBE played in terms of improving educational policy with respect to the ethnic minorities of the country? (1.4)

KIICM KIIL

To what extent is the MTBBE approach relevant to UNICEF’s renewed focus on equity? (1.5) KIIU

To what extent would you support or not the continuation and expansion of the MTBBE? (1.6) KIICM KIIL KIISP

2. Effectiveness

To what extent is the MTBBE an effective approach to master the Mother Tongue? (2.1.1.1) KIICM KIISP KIIL

To what extent is the MTBBE an effective approach to master the Vietnamese? (2.1.1.2) KIICM KIISP KIIL

To what extent is the MTBBE an effective approach to learn Mathematics? (2.1.2) KIICM KIISP KIIL

To what extent is the MTBBE an effective approach to improve children´s confidence? (2.1.4) KIICM KIISP KIIL

To what extent is the MTBBE an effective approach to improve children´s communication skills? (2.1.5) KIICM KIISP KIIL

To what extent is the MTBBE an effective approach to develop or improve other academic and non-academic skills? Please specify. (2.1.3) (2.1.6)

KIICM KIISP KIIL

In what ways has the MTBBE approach been useful to improve your capacity as an education manager? (2.2.2) KIICM KIISP KIIL



184

Which were the main factors or criteria which you [Central/local/school authority] considered when designing the MTBBE? (2.3.1) (2.3.2)

KIICM KIISP KIIU

Which were the main factors or criteria which you considered when implementing the MTBBE? (2.3.3) KIICM KIISP KIIU

To what extent have the MTBBE materials in the target languages [for teaching, learning and advocacy] been appropriate for their respective purposes? (2.3.4)

KIICM KIIL KIIU

To what extent has the pre-service teacher training met the quantitative and qualitative requirements of the MTBBE program? Is there a deficit in this regard? Are there relevant statistics? (2.3.5)

KIICM KIIL KIIU KIISP

To what extent has in-service teacher training met the quantitative and qualitative requirements of the MTBBE program? Is there a deficit in this regard? Are there relevant statistics? (2.3.6)


To what extent have children´s learning outcomes been assessed? At what levels? What assessment types have been applied? (2.3.7)


Regarding the statistical evidence you know, to what extent is the MTBBE program achieving children´s learning outcomes? (i.e. good command of mathematics, mother tongue and Vietnamese languages: listening comprehension, speaking, vocabulary). (2.4A.1)?

KIICM KIIL KIISP KIIU

To what extent is this statistic evidence good enough to conclude that the MTBBE program is really an appropriate approach to fill the learning gap that affects ethnic minority students? (2.4A.1.1)


Comparing MTBBE to non-MTBBE students, what differences do you perceive in terms of:

- attitudes toward learning? (2.4A.2) KIISP

- self-confidence? (2.4A.3) KIISP

- methods of learning? (2.4A.4) KIISP

- collaboration with school activities? (2.4A.5) KIISP

- communication and collaboration with teachers (2.4A.6) KIISP

Comparing the MTBBE to non-MTBBE teachers, what differences do you perceive in terms of:

- professional capacity (2.4B.2) KIISP

- approaches and attitude towards learning? (2.4B.5) KIISP

- self-confidence? (2.4B.4) KIISP

- in-class teaching and organizing skills to facilitate learning? (4B.1) KIISP



185

- collaboration with school affairs? (2.4B.6) KIISP

- approaches towards other teachers? (2.4B.7) KIISP

- commitment toward community´s needs? (2.4B.10??) KIISP

To what extent has the MTBBE approach had some positive or negative effect:

- on school management? (2.4B.8) KIISP

- on board members? (2.4B.9) KIISP

- on girl student and gender issues at school levels? (2.4A.7) KIISP

- on girl student and gender issues at community levels? (2.4A.8) KIISP

- on parents’ and community participation in school activities? (2.4C.1) parents (2.4C.5) community KIISP

- on parents’ participation in learning and extra-curricular activities? (2.4C.2) KIISP

- on parents’ relationships with schools and community? (2.4C.3) KIISP

- on parents’ acceptance and views about MTBBE? (2.4C.4) KIISP

- on managers’ [all levels] support for and participation in implementation of Action Research Activities? (2.4D.1) KIISP

- on managers’ acceptance of and views about the MTBBE approach and its continuity and scale-up? (2.4D.2) KIISP

3. Efficiency

What has been the monetary cost of teacher training in general and per head? (3.1.1)

KIICM KIIL KIIU

Does some provincial difference exist in this cost? (3.1.2) KIICM KIIL KIIU

To what extent is this cost higher or lower than the training of non-MTBBE teachers? (3.1.3) KIICM KIIL KIIU

What has been the monetary cost of MTBBE materials [teaching, learning, and advocacy]? (3.1.4) KIICM KIIL KIIU

To what extent has this cost been over or under the expected estimates? (3.1.5) KIICM KIIL KIIU

To what extent have trained teachers been appropriately allocated? [or according to needs] (3.1.6) KIICM KIIL KIIU

To what extent have MTBBE materials [teaching, learning, advocacy] been appropriately distributed? (3.1.7) KIICM KIIL KIIU

To what extent has this received sufficient support from higher authorities [province, district, school] to meet the demand in terms of trained and un-trained teachers? (3.1.8)

KIIL KIISP

To what extent has this received sufficient support from higher authorities [province, district, school] to meet the demand in terms of MTBBE materials [learning, teaching and advocacy]? (3.1.9)

KIIL KIISP

4. Sustainability

To what extent is the MTBBE program prepared for continuation once the external support from UNICEF is KIICM



186

withdrawn? Which are the program´s achievements that are better positioned for this change? Which are the achievements more at risk due to the possible change? (4.1)

KIIL KIISP KIIU

To what extent can the MTBBE approach be replicated and scaled up in areas with characteristics similar to existing project sites [e.g. presence of students from one single ethnic group, Mong or J´rai or Khmer in the class?] (4.2)


To what extent is leadership in place for sustainability? (4.3.1) KIICM KIIL KIISP KIIU

To what extent is ownership in place for sustainability? (4.3.2) KIICM KIIL KIISP KIIU

To what extent are financial resources in place for sustainability? (4.3.3) KIICM KIIL KIISP KIIU

To what extent are teaching and learning materials in place for sustainability? (4.3.4) KIICM KIIL KIISP KIIU

To what extent are qualified teachers in place for sustainability? (4.3.5) KIICM KIIL KIISP KIIU

To what extent are local/DOET resources in place for sustainability? (4.3.6)


What could or should be done differently in future replications and/or scaling up? (4.4) KIICM KIIL KIISP KIIU

9.2 Focus Group Discussion Themes Categories FGC Community, Civil Society, Academics FGL DOET, other provincial, schools, local level managers, Board members FGP Parents of MTBBE students FGS MTBBE students FGSN Non-MTBBE students FGT Teachers 1. Relevance

What do you know about the MTBBE Action Research? (1.1.1) FGC FGL FGP



187

FGS FGT FGSN

To what extent is the MTBBE relevant and responsive at the provincial/local level? (1.1.2.1) (1.1.2.2) FGL

How relevant/important is the MTBBE Program for you as parents/community leaders? (1.1.2.2) FGP FGC

2. Effectiveness Is the MTBBE an effective approach to achieve learning outcomes?

Do you like, or not, like the experience of your first school learning in your mother tongue? Why? (2.1.1.1)

FGS

Do you like, or not, learning Vietnamese at school? Why? / FGSN: Would like to be enrolled in the MTBBE program? Why (2.1.1.2)

FGS FGSN

Do you like Mathematics? Is Mathematics easy or difficult to learn? Why? Are you really learning Mathematics at school? (2.1.2)

FGS FGSN

Comparing non-MTBBE students/MTBBE students, how secure do you feel about your progress in learning Mathematics? (2.1.4)

FGS FGSN

How easy or difficult has it been for you to learn at school in the Vietnamese language? (2.1.5) FGSN

Do you like, or not, to speak Vietnamese? Why? What problems, if any, do you have to talk with Vietnamese speakers? (2.1.5)

FGS FGSN

To what extent is the MTBBE an effective approach to learn? How much progress do you observe in children´s learning? (Mother tongue, Vietnamese, Mathematics) (2.1.1.1) (2.1.1.2) (2.1.2)

FGP FGL FGT

Do you observe any other areas in which children are showing progress? Please explain. (2.1.3) (2.1.6) FGP FGL FGT

Do you perceive some learning improvement in other areas? Which areas? Please explain. (2.1.3) (2.1.6) FGS

To what extent has the MTBBE program helped teachers to improve their teaching capacity? (2.2.1) FGT FGL

To what extent has the MTBBE program helped education managers to improve their administrative capacities? (2.2.2)

FGL FGT

Which were the main factors or criteria which the local/school authority considered when approaching and designing the MTBBE? (2.3.1) (2.3.2)

FGL FGT

Which were the main factors or criteria which local/school authorities considered when implementing the MTBBE? (2.3.3)

FGL FGT

What positive or negative impacts did the MTBBE approach have?

Comparing MTBBE to non-MTBBE students, what differences do you perceive in terms of school achievements [grades]? What supporting evidence do you have on this matter? (2.4A.1)

FGL FGT

To what extent is the MTBBE approach improving, or not, students´ attitudes toward learning? In what way is this effect better, the same, or worse than the one in the case of the non-MTBBE program? (2.4A.2)

FGC FGL FGP FGT

To what extent are the issues you learn at school useful for your life? Up to what grade or level [primary, secondary, etc.] would you like to attend school? Why? (2.4A.2)

FGS FGSN

To what extent is the MTBBE approach improving, or not, students´ self-confidence? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.3)

FGC FGL FGP FGT

How secure/self-confident do you feel in terms of how much and how well are you learning Vietnamese at school? Please explain. (2.4A.3)

FGS FGSN

How secure/self-confident do you feel in terms of how much and how well are you learning Mathematics and other matters at school? Please explain. (2.4A.3)

FGS FGSN

To what extent is the MTBBE approach improving, or not, students´ capacity for self-learning? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.4)

FGC FGL FGP FGT

To what extent have you received some instructions or training on self-learning techniques from your teachers? FGS



188

Please mention some examples. (2.4A.4) FGSN

To what extent is the MTBBE approach improving, or not, students´ attitude to collaborate with school activities? In what way is this effect better, the same or worse than the one in the case of non-MTBBE program? (2.4A.5)

FGC FGL FGP FGT

To what extent do you like to collaborate with school activities? Please provide some examples. (2.4A.5) FGS FGSN

To what extent is the MTBBE approach improving, or not, students´ communication and collaboration with teachers? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.6)

FGC FGL FGP FGT

To what extent are you able to communicate with your teachers? (2.4A.6) FGS FGSN

To what extent is the MTBBE approach improving, or not, the treatment to both girl students and gender issues at school level? (2.4A.7)

FGC FGL FGP FGT

How frequently do your teachers organize talks about girls´ rights to education and respect from others? Please explain what you talk about. (2.4A.7)

FGS FGSN

To what extent is the MTBBE approach improving, or not, the treatment to both girl students and gender issues at community level? (2.4A.8)

FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, teachers’ in-class teaching and organizing skills in facilitating MTBBE learning? How is this effect better, the same, or worse that in the case of non-MTBBE teachers?

To what extent is the MTBBE approach improving, or not, MTBBE teachers’ professional capacity? (2.4B.2) FGC FGL FGP FGT

What positive or negative impacts did the MTBBE approach have on teachers’ attitude? (2.4B.3) FGC FGL FGT

What positive or negative impacts did the MTBBE approach have on teachers’ self-confidence? (2.4B.4) FGC FGL FGT

What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards learning? (2.4B.5)

FGC FGL FGT

What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards schools? (2.4B.6)

FGC FGL FGT

What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards other teachers? (2.4B.7)

FGC FGL FGT

To what extent is the MTBBE approach improving, or not, MTBBE school management? (2.4D.8)

FGC FGL FGT

To what extent is the MTBBE approach improving, or not, MTBBE board members’ administrative capacities? (2.4D.9)

FGC FGL FGT

To what extent is the MTBBE approach improving, or not, parents’ participation in school activities? (2.4C.1) FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, parents’ participation in learning and extra-curricular activities? (2.4C.2)

FGC FGL



189

FGP FGT

To what extent is the MTBBE approach improving, or not, parents’ relationships with schools and community? (2.4C.3)

FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, parents’ acceptance and views about MTBBE? (2.4C.4) FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, community’ participation in school activities? (2.4C.5) FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, community’s participation in learning and extra-curricular activities? (2.4C.6)

FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, community’s relationships with schools and parents? (2.4C.7)

FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, community’s acceptance and views about MTBBE? (2.4C.8)

FGC FGL FGP FGT

To what extent is the MTBBE approach improving, or not, managers’ [all levels] support for and participation in implementation of Action Research Activities? (2.4D.1)

FGC FGL FGT

To what extent is the MTBBE approach improving, or not, managers’ acceptance of and views about the MTBBE approach and its continuity and scale-up? (2.4D.2)

FGC FGL FGT

3. Efficiency

To what extent has this district/school received enough MTBBE materials [teaching, learning, and advocacy]? (3.9) FGL FGT

To what extent has this district/school received sufficient support from higher authorities to meet the demand in terms of [trained and un-trained] teachers? (3.8)

FGL FGT

To what extent has this district, school received sufficient support from higher authorities to meet the demand in terms of MTBBE materials [learning, teaching and advocacy]? (3.9)

FGL FGT

4. Sustainability

What could or should be done differently in future replications and/or scaling up? (4.4) FGC FGL FGP FGT

Date post:	03-Feb-2018
Category:	Documents
Upload:	phungquynh
View:	227 times
Download:	3 times

Final Evaluation Report - UNICEF · PDF filefinal evaluation report i evaluation of...

Documents