June 2017
This document was produced at the request of the United States Agency for International Development. It was prepared independently under contract with Checchi and Company Consulting, Inc. for USAID’s Afghanistan “Services under Program and Project Offices for Results Tracking Phase II project.
Data Verification and Quality Assessment Education Management Information System
Afghanistan II
Final Report
i
This report was conducted under Services Under Program and Project Office for Results Tracking Phase II (SUPPORT-II), USAID Contract Number: AID-306- C-12-00012. Assignment Title: Data Verification and Quality Assessment Education Management Information System Afghanistan II
Team Leader: Jehanzaib Khan
Team Members: Patricia Mclaughlin
Abdul Wakeel
Activity Start Date: July 2016
Completion Date: June 2017
Paul DeLucco, Chief of Party
Waheed Ahmadi, Deputy Chief of Party
Checchi and Company Consulting, Inc.
Kabul, Afghanistan
Disclaimer: The views expressed in this report are those of the author and do not necessarily reflect the views of USAID, the Government of the Islamic Republic of Afghanistan, or any other organization or person.
ii
TABLE OF CONTENTS Acronyms and Abbreviations………………………………………………………………………….iv
1.0 Executive Summary: Data Verification and Quality Assessment - EMIS 2……………………….1
1.1 Assessment Purpose and Questions……………………………………………………...1 1.2 Methods and Limitations………………………………………………………………….1 1.3 Key Findings……………………………………………………………………………….2 1.4 Conclusions……………………………………………………………………………….4 1.5 Recommendations………………………………………………………………………...5 1.5.1 For USAID……………………………………………………………………...5 1.5.2 For MOE………………………………………………………………………..6 2.0 Final Report: Data Verification and Quality Assessment - EMIS 2…………………….…………..7
2.1 Background…...…………………………………………………….…………………….7 2.2 Purpose...…………………...……………………………………….……………………7 2.3 Methods………………………………...………………………………………………...8 2.3.1 Survey…………………………………………………………………………..8 2.3.2 Qualitative Interviews …………………………………………………………9
2.4 Sampling …………………………………………………………………………………9 2.4.1 Methodology ……………………………………………………………...........9 2.4.2 Limitations ………………………………………………...………….……….11 2.5 Findings & Conclusions ……..…………………………………………………….……11 2.5.1 Qualitative Findings: Data Collection Process………………………………..11 2.5.2 Quantitative Survey Findings …………………………………………………13 2.6 Recommendations ……………...……………...………………………….………….....33 2.6.1 For USAID…………………...………………………………………………..33
2.6.2 For MOE...……………………………………………………………………..33 Annexes…………………………………..……………………………………………………….......35
Annex I: Statement of Work ………………………………………………………………………...35 Annex II: Data Collection Schedule ………………………………………………………………….38 Annex III: School Lists ………………………………………………………………………………..41 Annex IV: Teachers and Students Attendance & Teachers and Students by Province……………....47 Annex V: Data Collection Instruments ……………………………………………………………...50 Annex VI: Geolocalization..…………………………………………………………………………...64
LIST OF TABLES Table 1: Sample………………………………………………………………………………………….. 10 Table 2: Summary Statistics For School Type, Level, And Building…………………………………..….. 16 Table 3: Summary Statistics for Estimated Total Teachers (Weighted) By Type And Gender…………. 18 Table 4: Summary Statistics for Estimated Total Head Teachers (Weighted) by Type and Gender…… 28 Table 5: Summary Statistics for Estimated Student Enrollment Numbers (Weighted) by School Level and Gender……………………………………………………………………………………………… 28 Table 6: Data Collection Schedule……………………………………………………………………….38 Table 7: Qualitative Interview Schedule: PED Respondents…………………………………...…………39 Table 8: Qualitative Interview Schedule: DED Respondents.…………………...………...……………....39 Table 9: Qualitative Interview Schedule: MoE-EMIS Respondents………………………………………..40 Table 10: List of Closed Schools (Temporarily or Permanently)…………………………………………41
iii
Table 11: List of Schools with Discordant School Level Designations…………………………………....44 Table 12: List of Schools with Permanently Absent Teachers……………………………………………46 Table 13: Total Number of Teachers Present by Attendance Record Vs Actually Present at School…..47 Table 14: Numbers of Teachers by Province……………………………………………………………..47 Table 15: Number of Present Students According to School Register Vs Actually Present at School….48 Table 16: Numbers of Student Enrollment by Province………………………………………………….47
LIST OF FIGURES Figure 1: Schools by Level…………………………………………………………………………………11 Figure 2: Replacement Schools…………………………………………………………………………….11 Figure 3: Schools by Gender………………………………………………………………………………14 Figure 4: Long Term/Permanently Closed Schools………………………………………………………..17 Figure 5: Temporarily Closed Schools…………………………………………………………………….17 Figure 6: Total Teachers by Type and Gender……………………………………………………………19 Figure 7: Total Primary School Teachers by Type and Gender…………………………………………...20 Figure 8: Total Secondary School Teachers by Type and Gender………………………………………...20 Figure 9: Total High School Teachers by Type and Gender………………………………………………20 Figure 10: Overall Sample Total Teachers (National & by School Level)…………………………………20 Figure 11: Teachers by Attendance Register Vs. Actually Present by School Level……………………....21 Figure 12: Overall Teachers by Attendance Register Vs. Actually Present at School …………………....21 Figure 13: Head Teachers by Type and Gender…………………………………………………………..23 Figure 14: Overall Total Sampled Head Teachers………………………………………………………...23 Figure 15 A & B: Comparison of Estimated Male and Female Teacher Totals by Province……………...25 Figure 16: Primary School Students by Gender…………………………………………………....………27 Figure 17: Secondary School Students by Gender………………………………………………………...27 Figure 18: High School Students by Gender………………………………………………………………28 Figure 19: Overall Students (MoE vs Survey)……………………………………………………………...28 Figure 20: Attendance Register vs Actually Present Students…………………………………………….29 Figure 21: Overall Students by Attendance Register Vs. Actually Present Students……………………...29 Figure 22: Female and Male Student Totals by Province from the Survey and MoE Data………………..32
iv
ACRONYMS AND ABBREVIATIONS
ADS Automated Directives System DEC Development Experience Clearinghouse DED District Education Department DVQA Data Verification and Quality Assessment EMIS Education Management Information System FY Fiscal Year MOE Ministry of Education NGO Non-Governmental Organization PED Provincial Education Department SOW Statement of Work Tashkeel USAID
Afghan Civil Service U.S. Agency for International Development
1
1.0 EXECUTIVE SUMMARY Universal public education in Afghanistan has gained significant ground since 2002. Overall, school enrollment increased from less than a million to nine million (World Bank 2016)1. As education expanded, so did the need for accurate data collection in order to facilitate proper allocation of resources and effective decision-making. Recognizing this need, the United States Agency for International Development (USAID) and other development partners supported the Ministry of Education (MoE) to strengthen the reliability of MoE data, collected through the Education Management Information System (EMIS). Findings from the 2015 USAID-commissioned assessment of EMIS cite the absence of independent third-party verification of EMIS data, the dependence of MoE-EMIS on source data collected at the district and school levels where training and procedures remain inadequate, and the lack of a data verification mechanism. In addition, the President’s office and the international donor community raised concerns about the dearth of information about school enrollment, actual attendance of students and teachers, and even the existence of schools. To assess these matters, USAID commissioned this second Data Verification and Quality Assessment (DVQA) of the MoE-EMIS 2016 data. 1.1 Assessment Purpose and Questions This assessment focuses on verifying data collected by the MoE’s EMIS in Afghanistan for 1395. The aim is to assess how reliable EMIS data is, and to identify inconsistencies in EMIS data at the national, provincial, and school levels. The study assessed information about:
• School status (operational), type, location, and structure/building; • Actual students enrolled and attending on the day of the visit (disaggregated by gender); • Number of teachers at the school by contract and taskheel (disaggregated by gender).
1.2 Methods and Limitations This assessment, based largely on a probability sample, relies on quantitative survey data collected from schools. The study draws on 1,067 schools, of approximately 16,000 total schools, stratified by province and school level, i.e., primary, secondary, and high school. The questionnaire was based on the MoE existing data collection form to ensure that survey and EMIS data would be comparable. The survey collected information about; (i) school operational status, type, and structure; (ii) number of teachers by Tashkeel and contract, and number of teachers actually present at school at the time of the visit; and, (iii) student enrollment from grades 1 to 12, number of present students according to school register, and number of students actually found at school. The study also employed qualitative interviews (23) with district and provincial MoE officials focusing on how the EMIS system functions and the challenges faced by MoE officials at district and provincial levels in data collection. The DVQA data collection began on August 20 and ended on October 30, 2016. High risk areas2 in over a dozen provinces were excluded when the sample was being drawn. In addition, during data collection, a large number of sampled schools, approximately 30% of the total,
1 World Bank (2016). Afghanistan Overview. Retrieved from http://www.worldbank.org/en/country/afghanistan/overview 2 The team excluded high risk areas that included neighborhoods, villages, and districts based on security reports and information from local enumerators.
2
had to be replaced for security reasons. Very likely, the schools that could not be surveyed differ from the schools that were surveyed in reporting reliable data, and on other measures, such as teacher and student counts, but this is not possible to verify. In addition, school closure may be more likely in remote or conflict-affected areas. 1.3 Key Findings A. Qualitative Findings: Data Collection Process and Constraints The EMIS data collection process at the school-level varies across provinces. Qualitative data
indicate that school officials fill out the EMIS data collection forms. In some cases, academic officials at the District Education Departments (DEDs) complete the forms during their visits to schools. Generally, DEDs are unable to monitor data collection or provide support to schools in filling out the forms due to a lack of resources.
School officials lack a clear understanding of the EMIS form and certain technical terms used
in it. They find the form too detailed and face difficulties filling it out, particularly the student and teacher data. Some DED officials report similar concerns and an urgent need for capacity building.
Data reported by schools contain discrepancies, including inaccurate information about
teachers by category and student breakdown. If inaccuracies are detected at the DED or Provincial Education Department (PED) levels, clarifications are requested from schools.
Officials interviewed unanimously identify lack of training and capacity gaps at the provincial,
and particularly at the DED and school levels, as significant challenges for improving data quality and process.
B. Survey Findings: School Status, Type, Level, and Building There are small but statistically significant differences in the percentage of boys’ and mixed
schools reported by the two data sets. The MoE data report 3% more boys’ schools and 4% fewer mixed schools than the total survey sample. By school level - i.e. primary, secondary or high school - the MoE reports a slightly higher number of primary girls’ and mixed schools and a lower number of high schools relative to the survey data.
The percentage of schools with boundary walls is significantly lower (9%) in the EMIS data as compared to the survey data. This difference is reflected between the two data sets at the primary and secondary school levels.
Relative to the survey data, the percentage of schools with drinking water availability is reported significantly higher (24%) in the EMIS data at the national and school levels. In contrast, the average number of toilets reported by the MoE data is significantly lower (3.3%) than the average number in the survey data.
The percentage of schools located in rural areas is smaller in the MoE data; on average the MoE reports 7% fewer schools located in rural areas compared to the survey data. This difference is reflected at secondary and high school levels.
A total of 24 schools were closed at the time of the survey, of which 14 have been permanently closed for durations that reportedly range from six months to over a year. Nine of these permanently closed schools are reported open in the MoE data. A majority of these nine are closed due to poor security and the ongoing conflict; two are closed due to
3
nonpayment of rent; and three are closed for unknown reasons. Of the 10 temporarily closed schools, nine were closed for less than a month for extended periods, either for holidays or teacher absence.
C. Survey Findings: Teachers 1. Number of Teachers by Type and Gender
The MoE data reports a significantly higher number of male Tashkeel teachers overall and at
high school level as compared to the survey data; the total estimated numbers of female teachers reported in the two data sets are the same.
In contrast, the numbers of both male and female contract teachers are lower in the MoE data than the survey data. These differences are similarly reflected at primary, secondary and high school levels for both male and female teachers.
Differences also exist between the two datasets in the reported numbers of Ajeer (by Tashkeel) teachers. The number of male Ajeer teachers is higher and the number of female Ajeer teachers is significantly lower in the MoE data than in the survey data.
Once aggregated, the composite and uncategorized teacher numbers reach near parity at the
total sample level. In other words, the MoE and survey data sets record no statistical difference in the total number of teachers. This trend continues at the school level - primary, secondary, high - where the differences are nominal and insignificant.
II. Teachers by Attendance Register vs Actually Present at School More teachers are marked present in attendance registers than the number actually found at
school. These differences correspond to 8% additional male teachers, 6% additional female teachers, and overall 7% additional teachers reported present in the attendance register relative to the headcount numbers. On average, less than 1 additional male and female teachers (or 0.43 and 0.21, respectively) per school were reported present in school registers as compared to the number of teachers actually found at school.
Of the total sampled schools (N=1,067), only 34 (3% of the total sample), comprising 6 high
schools, 11 secondary schools, and seven primary schools, reported one or more teachers who never showed up at school. This number of absent teachers varies from 1 to 4, with the majority of the 34 schools reporting one permanently absent teacher. Of these schools, four were girls only, 10 were boys only, and 20 were mixed schools.
III. Head Teachers by Type and Gender - Tashkeel and Ajeer (by Tashkeel) The number of male Tashkeel head teachers between the two data sets is not significantly
different. In contrast, the number of female Tashkeel and Ajeer (by Tashkeel) head teachers reported by the MoE is significantly lower than the survey data.
There are differences in the composite and uncategorized head teacher number at the total sample level; the MoE number is significantly lower than those recorded in the survey data.
D. Survey Findings: Students I. Student Enrollment by School Level and Gender
4
Small to moderate, but significant, differences exist between the MoE and survey tallies of both girls and boys for every primary class. Cumulatively, the MoE data reports 18% more boys, 12% more girls, and overall 15% more primary school students than the survey data. Similarly, the MoE data includes 15% more boys and 7% more girls at the secondary school level relative to the survey numbers. Overall, the MoE reports approximately 11% more students than the survey data. At the high school level, differences are smaller than at the secondary school level. While the MoE reports 9% more boys than the survey data, there is no difference in female student numbers between the two. Overall the MoE reports 4% more high school students than the survey data.
The total numbers of boys and girls reported in the MoE data are significantly higher -15%
more boys and 9% more girls - as compared to the survey data. The overall MoE and survey totals of students, boys and girls combined, are significantly different, i.e. the MoE reports 12% more students than the survey data.
II. Student Enrollment by Attendance Register Versus Actually Present at School More students are found present at school than marked present in student attendance
registers. The small to moderate differences correspond to approximately 3% additional students in grades 1 and 3, 11% in grade 7, and 12% in grade 10 who were actually present as compared to the attendance record. There were 9.5% more boys and 7% girls actually found at school than reported by school attendance register. This means, on average, that the data from the headcounts report 2.5 additional boys and 1.9 additional girls actually present per school as compared to the school attendance record.
1.4 Conclusions A. Qualitative: Data Collection Process and Constraints Inaccuracies in teacher and student information are not uncommon. The variation in the data collection process at the school level, underdeveloped capacity of school officials, absence of capacity building programs for district and provincial officials, inadequate monitoring at the district and school levels due to budgetary constraints, and limited human resources likely affect both the process and quality of data collection. B. Survey: School Status, Type, Level, and Building Beyond marginal differences, the MoE data about school type and structure does not
significantly vary from the survey data on key indicators. In other words, the MoE reports mostly reliable data on school type and structure.
Contrary to broad speculation, e.g., Buzzfeed 20153, the number of MoE permanently closed
schools is small. Yet, although the number is small, the fact that the MoE data underrepresents the actual number - 09/14 reported by the survey are marked operational in the MoE data - indicates a gap in reporting.
C. Survey: Teachers There are significant discrepancies in the two data sets at the teachers’ subcategory levels,
i.e. teacher type, gender, and school level, with the exception of female Tashkeel teachers. 3 BuzzFeed (2015). Ghost Students, Ghost Teachers, Ghost School. Retrieved from BuzzFeed website: https://www.buzzfeed.com/azmatkhan/the-big-lie-that-helped-justify-americas-war-in-afghanistan?utm_term=.qqlyyMzk1o#.ukepp821V0
5
Qualitative data indicate that variation at the subcategory levels may stem from inaccurate categorization of teacher data at the school or district levels.
Despite the discrepancies in the MoE data across the majority of the subcategories, these
discrepancies almost disappear at the total sample and total teachers by school level. In other words, while the reliability of the MoE data at the subcategory level is questionable, the data is, by and large, consistent at the aggregate level.
Except for the male head teachers (Tashkeel) figures, the MoE head teacher data is less
reliable than the teacher data due to discrepancies in this data present at both the subcategory and composite levels.
Teacher attendance records do not accurately account for all the teachers who are actually
absent from school. The practice of marking teachers present who are actually absent is more common for absent male teachers than for absent female teachers; for primary schools relative to secondary and high schools; and for larger schools compared to smaller ones.
The proportion of schools reporting teachers who never show up at school, i.e. permanently
absent teachers, is small. The practice tends to be more common at the high school level than at the secondary and primary levels.
D. Survey: Students The MoE student enrollment numbers are larger than the survey data for every grade for
boys and for all but grade 12 for girls - overall, the MoE reports 12% more students than the survey data. The discrepancies are more common at the primary school level relative to secondary and high school levels and more common for boys than for girls. Relatively smaller discrepancies in overall girl enrollment numbers as compared to boys indicate that the girls’ data is slightly more reliable than boys.
Student attendance records are inconsistent when compared to the number of students
actually present. Schools generally underreport the number of students present. The practice of inaccurately recording student attendance is more pronounced at the secondary and high school levels than at primary school, and slightly more common for boys than for girls overall.
1.5 Recommendations 1.5.1 For USAID USAID should consider continued support to future independent assessments of the EMIS
data, which may also compare EMIS data between provinces, e.g., secure and insecure, inaccessible, remote. Such assessments, requiring large samples from provinces, could provide valuable insights into solutions for variations in EMIS data quality between regions or provinces.
Future USAID support to the MoE should include capacity-building measures for the MoE-
EMIS at the DED, PED, and EMIS Kabul levels. Such a program might include: a. Training programs specifically focused on planning and EMIS data collection for quality
assurance, sorting, cleaning, analysis, and management. b. Funding or providing technical support to the MoE to develop a strong capacity building
program directed toward EMIS at Kabul, provincial, and district levels. The capacity- building might include conducting needs assessments of school officials, DEDs, PEDs, and
6
MoE-EMIS in Kabul, and developing strategies to address capacity gaps at all levels. c. Supporting the MoE to engage both the EMIS department and its Research and
Evaluation Unit on a regular basis to jointly conduct data quality assessments, EMIS data collection, and analysis.
USAID should also explore ways in which ongoing USAID-funded programs in the country
could provide technical support/training to the EMIS department.
1.5.2 For MoE Beyond data collection for EMIS, the MoE, through its DEDs, should collect data on school
operational status at least twice a year. While the MoE collects EMIS data once a year, the frequency of collecting information on schools’ operational status, if increased, should result in improved monitoring, accountability, and up-to-date and accurate information on functional and nonfunctional schools.
The MoE should explore on a regular basis the causes of long-term and temporarily
unsanctioned school closures, and identify solutions that may include alternative schooling options for affected communities.
The MoE should explore the causes of discrepancies between teacher and student
attendance records, and actual presence of teachers and students, and develop solutions to improve attendance records at school.
The MoE should provide basic training, e.g. filling out questionnaires, census, or EMIS forms,
to individuals primarily responsible for data collection. The MoE should ensure that school Principals or leadership responsible for filling out EMIS forms receive proper training to collect school, student, and teacher information by grades and subcategories accurately.
The MoE should develop a rigorous needs assessment program jointly conducted by both
the EMIS department and the Research and Evaluation Unit to identify capacity gaps in EMIS data collection, management, and analysis at various levels.
The MoE should continue, and if necessary, expand capacity building for DED officials in
planning, collecting, and reporting EMIS data, and in providing support to schools on data collection on a regular basis.
The MoE should simplify the EMIS data collection form, taking into account input from
school, DED, and PED officials.
7
2.0 FINAL REPORT: DATA VERIFICATION AND QUALITY ASSESSMENT (EMIS 2)
2.1 Background Over the last decade and a half, education in Afghanistan has made remarkable gains. Fewer than one million children, almost all of them boys, attended school in Afghanistan before 2002. Access to education for girls was almost nonexistent. Today, the Ministry of Education (MoE) has a network of more than 16,000 schools with an estimated enrollment of nine million children. Of the total enrolled children, nearly 40 percent are female (NYU/ALSE 2016; World Bank 2016; USAID 2017)4. The exponential growth is primarily the result of close collaboration of the MoE with USAID and other donors to build Afghanistan’s education system. As the education system grew, so did the need for accurate data collection and an information management system for meaningful resource allocation and decision making. Although the MoE, working closely with the World Bank and other international donors, has largely been successful in establishing a functional EMIS during the last decade, it still needs to support data collection, the quality of EMIS data, and human resource development. USAID and other development partners have worked with the Ministry of Education to identify ways to strengthen the validity and reliability of the MoE data, principally collected through EMIS. USAID contracted Checchi and Company Consulting, Inc. through the Services under Program and Project Office for Results Tracking Phase II (SUPPORT-II) mechanism to conduct a data verification study in late 2015. Through this initial data verification exercise, USAID and other development partners (DPs) learned valuable lessons and identified key gaps in the current system. One gap is a lack of independent third-party verification of data collected and reported through EMIS. While EMIS has steadily improved, it still relies on source data collected at the district and school levels where training and procedures remain inadequate and ability of officials to travel to schools and verify data remains limited. Efforts are ongoing and USAID is working with other DPs and the MoE to implement assessments and activities to systemically improve MoE capacity to report reliable data. Media have reported on the existence of “ghost” schools, students, and teachers. The President’s office and the international community continue to raise concerns about the dearth of information about school enrollments, actual attendance of students and teachers in schools, and the existence of schools. To assess these conditions accurately, USAID/Afghanistan commissioned a second Data Verification and Quality Assessment (DVQA) of the Ministry of Education’s Education Management Information System. 2.2 Purpose The primary purpose of this assessment is to verify and assess data collected and reported through the Ministry of Education’s Education Management Information System in Afghanistan. More specifically, the DVQA serves a dual purpose: (I) to learn how reliable the EMIS data is, and (II) to identify inconsistencies in EMIS data at national, provincial, and school levels. At the school level, the study assesses information about:
1. School status (operational), type, location, and structure/building;
4 NYU/ALSE (2016). Assessment of Learning Outcomes and Social Effects of Community-Based Education in Afghanistan. Retrieved from http://www.alseproject.com/ World Bank (2016). Afghanistan Overview. Retrieved from http://www.worldbank.org/en/country/afghanistan/overview USAID/Afghanistan (2017) Retrieve from https://www.usaid.gov/afghanistan/education
8
2. Actual students enrolled and attending on the day of the visit (disaggregated by gender); 3. Number of teachers at the school by contract and taskheel (disaggregated by gender).
2.3 Methods 2.3.1 Survey The DVQA used a quantitative survey instrument, or verification tool, based on the existing MoE-EMIS data collection tool. This ensured that the DVQA data were comparable with that of the EMIS and that it used the same pre-tested, existing translations and terms. A total of 70 enumerators, and five regional data monitoring managers received a four-day training in administering the survey instrument using both hard copies and smart phones. The instrument was translated into Pashto and Dari, and pretested before full scale implementation. Data collection began on August 20 and finished on October 30, 2016. (See Annex II for data collection schedule). Enumerators uploaded survey data approximately every day for the regional managers to examine, and if necessary, to contact enumerators, for example, for inconsistent or missing data. In addition, the Kabul-based data verification manager scrutinized uploaded data on a daily basis for data quality assurance. Key indicators used for this verification exercise include: Existence and educational use of schools (whether they are operational); Definition and type of non-existent/closed/ghost schools found in the sample; Number of teachers actually found at the school, disaggregated by gender; Number of teachers employed, as recorded by the school and disaggregated by gender; Number of students actually found at the school, disaggregated by gender; Number of students enrolled, as recorded at the school, disaggregated by gender; GPS coordinates for each school location; The school’s EMIS unique ID number, name, province, level.
The survey instrument consists of three parts: Part A includes questions about school operational status, school type (government or private), school level (primary, secondary or high), school gender, and school structure/building (physical structure, number of rooms, boundary wall). See Annex V for data collection instruments. Part B focused on teachers and head teachers. Using the MoE-EMIS form as a guide, the indicators include teacher type, Tashkeel, contract or Ajeer, and subcategories within teacher type, i.e. gender and primary, secondary, and high school teachers. In addition, teacher data collected from schools included information about present and absent teachers on the day of the survey as well as those who never show up. The number of teachers marked as present in the attendance register is compared with the total number of teachers counted/actually present at school. Part C focused on collecting student enrollment data—total numbers of boys and girls from grades 1 to 12 according to the school records that principals/school officials keep at each school. These student registers located at school were used as the primary source for student data5. If a school had
5 Since all schools (in the sample) keep student attendance registers but not every school keeps a separate enrollment record, the student attendance register was used as the primary source of data collection for students. The survey team collected enrollment data where available. When compared, the number of students by school enrollment records is higher (approximately 5%) than the student register record. This difference is not adjusted for in the analysis. Accounting for the 5% increases the size of difference between the MoE and survey data.
9
more than one shift, data for each shift was collected. Of the total 980 sample schools, 49% are single shift schools, 46% are two-shift schools, and approximately 5% have three shifts. Generally, student enrollment is significantly higher in the first shift than in shifts two and three. The student register data collected by the survey (that also included absent students) is compared with the data provided by the MoE. Since the MoE data was not collapsed by school shifts, the survey data from all shifts was combined for the analysis. In order to assess whether discrepancies exist between student attendance records and the number of the students actually present at school, the survey also focused on collecting student attendance data by counting the number of students present in four selected grades on the day of the visit and comparing these numbers to attendance records. As noted above, records included total students by attendance register, total students absent on the day of the survey, and total permanently absent. In-person headcounts were conducted in the following way: Primary school: All students in grades 1 and 3 were counted; Secondary schools with primary sections: Students in grade 1 in the primary section and
grade 7 in the secondary section were counted; Secondary schools without primary sections: Students were counted in grades 7 and 8; High schools with secondary and primary sections: Grades 1, 7 and 10 were selected for
counting; High schools without secondary or primary sections: Students in grades 10 and 12 were
counted. 2.3.2 Qualitative Interviews In addition to the survey, the study also uses qualitative interviews to collect information from MoE-EMIS Provincial Education Departments (PEDs) and District Education Departments (DEDs). While the survey tool focuses on collecting information at the school level (teachers, students, school buildings, etc.), which is compared to the EMIS data for discrepancies, the qualitative interviews seek to understand how the EMIS system functions, the processes it adopts for gathering and reporting data and challenges faced by PEDs and DEDs in data collection. 2.4 Sampling 2.4.1 Methodology The Data Verification and Quality Assessment (DVQA) draws a probability sample of 1,067 schools, both public and private, stratified by province and by school level resulting in estimates that are generalizable to the population of schools nationwide. The sampling frame was developed from the MoE-EMIS 2015 (Year 1394, Afghan calendar) data set containing primary, secondary, and high schools (approx. 16,000). Other types of schools such as literacy schools, teacher training, technical schools, and madrassas were excluded. First, schools were stratified by geography (34 provinces and Kabul city, for a total of 35 geographic locations) and school level (primary, secondary, high). Second, from each school level stratum within a geography/province, 10 schools were randomly selected, producing a total of 30 schools (approx.) per province. The resulting total number of schools for each school level stratum is: 361 (primary); 350 (secondary); and 356 (high). These totals are large enough to provide reliable estimates at a 95% confidence level at the national level. Stratification by school type allowed for assessing variation in data across the three types of schools.
10
Table 1: Sample
Selecting the same number of schools from each province meant that the sampling fraction varied from province to province simply because of the variation in total number of schools in each province. In order to adjust sampled schools according to their population proportions, in all over- and under-sampled provinces, weights were applied at the analysis stage to yield accurate population estimates. Table I below presents the DVQA sample. The survey sample was merged with the MoE-EMIS data (2016). A school was considered to be matched only if the school/EMIS code, name, province, and level (e.g. primary, secondary, high) were the same in both the survey and the MoE-EMIS data sets. It is important to note that survey data collection coincided with the MoE’s EMIS data collection - from late August to the end of October, 2016. Only matched schools were included in the analysis. Seventeen surveyed schools were removed from the survey sample because of a failure to match them to administrative data. Additionally, only schools designated as operational in both data sets were included in the analysis, resulting in the removal of an additional 24 schools (See Annex III for the list of temporarily/permanently school closed schools). To ensure comparisons made between data sources were as accurate as possible, 56 schools that had different school- level designations in the survey versus the MoE were discarded from the sample (See Annex III for these schools). Seven additional schools with missing level and/or other information were removed, bringing the final analysis sample to 980 schools. The final sample includes 312 primary, 334 secondary, and 334 high schools, as shown in Table 1 above and Figure 1 on the next page.
Primary Secondary High Total
Survey
Number of schools by province 10 10 10 30
Planned total by stratum 361 350 356 1067
Achieved total by stratum 350 364 370 1084
MoE-EMIS and Survey Data Sets Combined
Final matched data set for comparison 312 334 334 980^
A total of 23 Semi-structured interviews were conducted in 10 provinces and Kabul (10 DED; 10 PED; 3 MoE/EMIS).
^Only includes cases that are identical in both data sets.
11
Figure 1: Schools by Level Figure 2: Replacement Schools
It is important to note here that due to poor and varying levels of security, as well as on-going insurgency in many parts of the country, some locations were excluded when drawing the survey sample. Furthermore, a large number of schools (approx. 30%) included in the original sample turned out, during the data collection phase, to be inaccessible due to poor security. These schools were replaced with other schools selected randomly. Additionally, a small number of schools were replaced due to conflicting designated school levels (e.g. primary, secondary or high) versus actual school levels reported during data collection. These schools were also replaced by random selection. Figure 2 above highlights the replacement schools.
2.4.2 LIMITATIONS When the sample was being drawn, high risk areas in many provinces were excluded. During data collection, however, many sampled schools turned out to be located in areas with poor security and were replaced. It is possible that schools that could not be surveyed differ from schools that were surveyed. For instance, it is likely that schools in conflict zones that could not be surveyed report less reliable data than schools in more peaceful regions. These inaccessible schools could also differ from those that were surveyed on other measures, such as having electricity or a school building, and teacher and student counts. School closure may be more likely in remote/conflict affected areas with significant unrest. The sample size (approx. 30 schools) at the provincial level is not large enough to estimate statistically significant differences between provinces. Thus, findings about variation between provinces should be viewed with caution. The assessment team received the 2016 EMIS data from the MoE after significant delay (almost six weeks) as it took more time than expected by the MoE to finish data collection and data cleaning. Please note that given time constraints the MoE-EMIS did not complete the data cleaning process before organizing and providing the teacher data to the assessment team. The partially cleaned data may have added to the variation between the two data sets. 2.5 Findings & Conclusions 2.5.1 Qualitative Findings: Data Collection Process and Constraints The following findings are based on a total of 23 interviews—10 DEDs, 10 PEDs and 3 EMIS Kabul (See Annex II for respondents list). The purpose of the qualitative interviews was to understand how
12
the EMIS system works and ways in which it could be improved. 2.5.1.1 MOE-EDMIS Data Collection Process The MoE collects EMIS data every year around March-April from schools in cold regions and
around September in hot regions. The qualitative interviews indicate that the MoE is moving toward collecting data at one point during the year—mostly likely March/April.
At the instruction of the MoE, the PEDs initiate the data collection process through the
DEDs. The DEDs receive the EMIS data collection forms, which they distribute among the schools in their jurisdiction. According to qualitative interviews, the data collection process at the school level varies across provinces. Generally, the forms are filled out by school officials—principals or head teachers--though a few interviewees reported that the academic officials at the DEDs sometimes fill out the forms themselves during their visit to schools. The schools return the forms to DEDs, who send them to PEDs, where the data is entered and uploaded to the MoE server.
Some provinces conduct basic cross-checking of the data received from districts—comparing
the latest data with the previous year’s data. The EMIS department at the MoE at the central level is primarily responsible for data cleaning.
PEDs use EMIS data to determine distribution of resources, school eligibility for an upgrade,
and sometimes to share with other ministries and non-governmental organizations (NGOs). Additionally, qualitative interview data indicate that the MoE may use EMIS data for strategic planning, resource allocation, and infrastructure development. The use of EMIS data is relatively limited at the district level.
Some officials from DEDs and PEDs reported that some additional questions—mostly
focusing on teachers and teacher qualifications--were added to the EMIS form this year. In addition, the MoE is moving toward integrating the EMIS data with the human resources/teacher payroll data into a single database.
Most of the officials at the district and provincial levels reported high appreciation for the
training provided recently (2016) by the MoE to DED academic officials in data collection (focusing mainly on planning).
Some PED officials reported that they do not receive regular support on EMIS from the
MoE, while others appreciated the MoE’s support—for example, sending master trainers to their provinces to train its officials.
2.5.1.2 Challenges/Constraints Interviewees reported the following challenges/constraints to EMIS data collection and management: School officials lack a clear understanding of the EMIS form and certain terms/indicators used
in it. They find the form too detailed and they face difficulties filling it out, particularly the student and teacher data.
Due to a lack of resources and budgetary constraints, officials report that DEDs are unable
to monitor data collection and provide support to schools in filling out forms. Some DED officials also lack a clear understanding of the EMIS forms and report an urgent need for capacity building.
13
Yet there is no training program for school officials to build their capacity in efficient and effective EMIS data collection. Officials report that capacity development opportunities are almost nonexistent for the EMIS offices at the EMIS Directorate in Kabul, PED and particularly at the DED and school levels. They see these as huge challenges for improving EMIS data quality and process.
Data reported by schools contain discrepancies, including inaccurate information about
teachers by category and student breakdown. However, if inaccuracies are detected at the DED and/or PED levels, these officials may request clarifications/further information from schools.
DEDs do not have separate EMIS offices/sections. When the data arrive from schools, there
are no standard data quality checking processes at the district level. Some officials report that officials cross-check some of the data with the previous year’s data on paper forms. The extent and frequency of data verification is unclear. In rare cases, DED officials might verify the data by visiting schools randomly.
Officials report having inadequate resources at the provincial level such as limited human
resources, transport budget for monitoring, and support visits to districts. Other challenges reported at the provincial and particularly district level include: an
inadequate number of EMIS forms provided to the DEDs, inadequate human resources, lack of electricity, and a lack of computers and poor Internet connection.
The MoE wasn’t able to access and collect data from approximately 1,017 schools in 2016
due to poor security. The majority of the officials interviewed unanimously identify lack of training and capacity
gaps generally at the provincial level and particularly at the DED and school levels as huge challenges for improving EMIS data quality and process.
2.5.1.3 Conclusions from Qualitative Findings While the EMIS data collection process appears generally uniform across provinces and
districts, it varies at the school level. Inaccuracies in teacher and student information are not uncommon. The variation in the data
collection process at the school level, underdeveloped school officials’ capacity, absence of capacity building programs for district and provincial officials, inadequate monitoring at the district and school levels due to budgetary constraints, and limited human resources likely affect both the process and quality of data collection.
There is a consensus among the MoE officials at the district, provincial, and center level that
there should be regular capacity building support in data collection and management. 2.5.2 Quantitative Survey Findings This section provides assessment results based on the comparison between the EMIS II survey and MoE-EMIS data. The key findings are described below. 2.5.2.1 School Status, Type, Level, and Building There are small but statistically significant differences in the percentage of boys and mixed
14
schools reported between the two data sets. The MoE-EMIS data reports 3% more boys schools and 4% fewer mixed schools as compared to the total survey/national sample. By school level—i.e. primary, secondary or high schools— the MoE reports a slightly higher number of primary girls and mixed schools, and a lower number of high schools relative to the survey data. These differences at the school level are small but significant. The mixed school category was created by combining three sub-categories: mixed schools, boys schools with girls’ enrollment, and girls’ schools with boys’ enrollment. Figure 3 below shows overall differences in the number of boys’ and girls’ schools between the two datasets.
The percentage of schools with boundary walls is significantly lower (9%) in the MoE-EMIS
data as compared to the survey data. This difference is reflected between the two data sets at the primary and secondary school levels.
Small but significant differences exist in the average number of classrooms and administrative rooms at the national level, as reported in the two data sets. The number of classrooms in the MoE-EMIS data is smaller than that reported by the survey data at all three school levels. Similarly, the average number of administrative rooms reported for primary and secondary schools is lower in the MoE data set as compared to the survey data. On average, the MoE reports 1.16 fewer classrooms and .3 fewer administrative rooms than the survey data. Relative to the survey data, the percentage of schools with drinking water availability is
reported significantly higher (24%) in the EMIS data at the national and school levels. Similarly, the percentage of toilets reported by the MoE data is significantly lower (3.3%) than the average number in the survey data.
The percentage of schools located in rural areas is smaller in the MoE data—on average the
MoE reports 7% fewer schools located in rural areas than the survey data. This difference is reflected at secondary and high school levels.
At the national/total sample and school levels, the two data sets do not differ in reporting
the average number of government and private schools, average number of schools with electricity, whether the school is in a hot or cold region, and the medium of instruction.
The data for the type of physical structure of school buildings (earthen or concrete) and
school property (government, private, or rented property) are not included in the analysis due to the incompatible categorization of these variables in the two data sets.
There is more variation between the two data sets at the primary school level across various
24%18%
58%
27%19%
54%
0%
20%
40%
60%
80%
100%
Male Female Mixed
Survey MoE
Figure 3: Schools by Gender
15
indicators as compared to secondary and high school levels.
16
Table 2: Summary Statistics for School Type, Level, and Building
Variable Overall Mean/Proportion (SD) Primary Secondary High Survey MoE Survey MoE Survey MoE Survey MoE
School Level .31 .31 .34 .34 .35 .35 School Gender
Boys .24 .27 * .14 .18 .22 .25 .35 .38 Girls .18 .19 .11 .13 * .12 .13 .32 .29
Mixed .58 .54 * .76 .69 * .65 .62 .34 .32 * School Type
Govt .94 .94 .95 .95 .92 .92 .94 .94 Pvt .06 .06 .05 .05 .08 .08 .06 .06
Boundary Wall .59 .50 * .45 .28 * .60 .51 * .72 .69 # of Rooms
Classrooms 9.24 (8.9) 8.08 (8.46) * 4.68 (3.97) 3.55 (4.65) * 8.55 (6.44) 7.62 (7.06) * 14.08 (11.66) 12.65 (10.34)* Administrative 2.36 (2.3) 2.06 (1.89) * 1.02 (1.25) .84 (1.10) * 2.19 (1.60) 1.95 (1.44) * 3.69 (2.89) 3.27 (2.16)
Electricity .27 .27 .17 .14 * .25 .26 .37 .39 Drinking Water .51 .75 * .34 .57 * .52 .79 * .64 .88 *
Number of Toilets 7.70 (7.66) 4.38 (4.77) * 3.38 (5.03) 1.72 (2.70) * 7.15 (6.32) 4.50 (4.65) * 12.07 (8.62) 6.68 (5.33) *** Urbanicity
Rural .78 .71* .81 .75 .80 .72 * .74 .65 * Climate
Hot .21 .20 .22 .20 .20 .18 .21 .20 Cold .79 .80 .78 .80 .80 .82 .79 .80
Medium of Instruction Pashto .29 .29 .33 .31 .31 .30 .26 .26
Dari .68 .68 .65 .64 .67 .67 .72 .72 Both .02 .03 .02 .05 .02 .03 .02 .02
Other <.01 0 0 <.01 Unweighted N 980 980 312 312 334 334 334 334 Weighted N 14,011 14,011 4,389 4,389 4,775 4,775 4,846 4,846 * p < 0.05
17
2.5.2.2 Non-Functional/Closed Schools A. Permanently Closed Schools A total of 24 schools were closed at the time of the survey. Of the total closed schools, 14 are permanently closed (Approx. 1.4% of the total sample schools) for durations that reportedly range from six months to over a year. Nine of these permanently closed schools are reported open in the MoE data, a majority of which are closed due to poor security and on-going conflict (See Figure 4 below). For example, according to local community members, the Afghan army closed one of the schools in Nuristan and is using it as a security post. In Wardak, one of the schools was closed because of threats from the Taliban. Two were closed due to nonpayment of the rent and three were closed for unknown reasons.
B. Temporarily Closed Schools Of the 10 temporarily closed schools, three were closed at the time of survey for extended holidays (e.g. Eid), according to community members, and three were closed due to the absence of teachers. In two cases, teachers were away for training, while the third was absent for over three months. The remaining four schools were reported as temporarily closed for unknown reasons. All of the 10 temporarily closed schools are marked functional in the MoE data. With the exception of the school in which the teacher has been absent for three months, the rest were closed for less than a month. (See Annex III for the list of temporarily/permanently closed schools). Figure 5: Temporarily Closed Schools
Figure 4: Long-Term/Permanently Closed Schools
18
2.5.2.3 Quantitative Survey Conclusions: School Status With the exception of marginal variation between the two data sets, the MoE data about school
type and structure does not significantly vary from the survey data on key indicators. In other words, the MoE reports mostly reliable data on school type and structure.
Contrary to broad speculation (e.g., Buzzfeed 2015)6, the number of MoE permanently/long-
term closed schools is small. Yet although the number of permanently/long-term closed schools is small, the fact that the MoE
data underrepresents the actual number—09/14 reported by the survey are marked operational in the MoE data—indicates a gap in reporting.
2.5.2.4 Quantitative Survey Findings: Number of Teachers by Type and Gender Table 3 below presents differences in estimated teacher numbers (weighted) by teacher type, gender, and school level as reported by MoE-EMIS and survey data. Teacher type refers to teachers by Tashkeel, contract (these are Ajeer teacher who are out of Tashkeel), and Ajeer teachers (who are part of Tashkeel). A. Tashkeel The MoE data reports a significantly higher number of male Tashkeel teachers overall and at the high school level as compared to the survey data; the total estimated numbers of female teachers reported in the two data sets are not different (Figure 6; Table 3). B. Contract (Ajeer – out of Tashkeel) Similarly, the numbers of contract teachers, both male and female, vary between the two data sets. In this category, the numbers of both male and female teachers are lower in the MoE data than the survey data. These differences are similarly reflected at primary, secondary, and high school levels for both male and female teachers.
C. Ajeer (by Tashkeel) Differences also exist between the two datasets in the reported numbers of Ajeer (by Tashkeel) teachers. The number of male Ajeer teachers is higher and the number of female Ajeer teachers is significantly lower in the MoE data than in the survey data. While differences in the two data sets also exist by school level, these differences are not tested for statistical significance due to the low number of teachers in some cases and should be viewed with caution. For example, the comparison of weighted counts for female primary school Ajeer (by Tashkeel) teachers is based on only (unweighted) 18 surveyed teachers as compared to 14 teachers from the MoE data (unweighted counts—not shown here).
6 BuzzFeed (2015). Ghost Students, Ghost Teachers, Ghost School. Retrieved from BuzzFeed website: https://www.buzzfeed.com/azmatkhan/the-big-lie-that-helped-justify-americas-war-in-afghanistan?utm_term=.qqlyyMzk1o#.ukepp821V0
19
Table 3: Summary Statistics for Estimated Total Teachers (Weighted) by Type and Gender
Variable Overall Primary Secondary High Survey MoE Survey MoE Survey MoE Survey MoE
Tashkeel Male 108298 122718* 12034 12971 32683 37697 63579 72071*
Female 82545 82274 6994 6743 20691 20073 54859 55458 Contract (Ajeer-out of Tashkeel)
Male 16871 8724* 3168 753* 5387 3072* 8316 4899* Female 16716 9896* 2792 1770* 6296 3434* 7627 4691*
Ajeer (by Tashkeel)
Male 2986 4661* 956 1260 938 1576 1093 1825 Female 2158 1154 324 180 851 325 983 650
Estimated N 229574 229427 26268 24677 66846 66177 92887 95220 * p < 0.05
D. Overall Differences Figures 7 to 9 show differences by school level (primary, secondary, high) and in overall national/total sample in the number of male and female Tashkeel, contract and Ajeer (by Tashkeel) teachers as reported by the survey and the MoE.
Figure 6: Total Teachers by Type and Gender
20
Discrepancies, by teacher type and gender, described above manifest analogously at the overall primary, secondary and high school levels - see Figures 7 to 9, Table 3. For example, by school level, the MoE numbers of male Tashkeel teachers are higher than the survey data at primary, secondary, and high school levels; the two counts for female Tashkeel teachers are not significantly different by school level. In contrast, the MoE reports lower numbers of male and female contract (Ajeer, out of Tashkeel) teachers; while the number of male Ajeer teachers is higher, the number of female Ajeer teachers is lower in the MoE data than in the survey data.
Interestingly, the composite (and uncategorized) teacher counts reach near parity at the national/total level (Figure 10; Table 3 above). In other words, the MoE and survey data sets record no statistical difference in the total number of teachers. The same trend continues at the school level—primary, secondary and high - where the differences are nominal and insignificant. 2.5.2.5 Teachers by Attendance Register vs Actually Present at School Teacher data collected from schools also included information about present and absent teachers on the
Figure 7: Total Primary School Teachers by Type and Gender
Figure 8: Total Secondary School Teachers by Type and Gender
21
day of the survey and those who never show up at schools. With the school principals’ permission, the enumerators counted every teacher (and student in the selected grades) present in school on the day of the visit. The number of teachers marked as present in the teacher attendance register is compared with the total number counted/actually present at school. T-tests were conducted to determine whether significant differences existed between these two variables for both male and female teachers. Significant differences exist between the number of teachers marked present and the number of
actually present teachers at school at primary, secondary and high levels (Figure 11). At the primary level, approximately 9% more male and 15% more female teachers were marked present in school records than the number of male and female teachers actually found at school. Similarly, approximately 7% more secondary school male teachers and 3% more female teachers were present according to the school attendance record than those counted in the survey. The same trend continues at the high school level: there are 9% more male teachers and 5% more female teachers marked present than actually found at school.
Figure 11: Teachers by Attendance Register vs Actually Present by School Level
Thus, together there are 11% more primary school teachers, 5% more secondary school
teachers, and 7% more higher school teachers marked present in school record than the actual numbers of teachers counted at school (Figure 12).
Overall, more teachers are marked present in attendance registers than the number of teachers
actually found at school (Figure 12). These differences correspond to 8% additional male teachers, 6% additional female teachers and overall 7% additional teachers reported present in the attendance register relative to the headcount numbers (See Annex IV for teachers by attendance register vs actually present). On average, 0.43 additional male teachers per school were reported present in school registers as compared to the number of teachers actually found at school. Similarly, on average 0.21 additional female teachers per school were marked present in teacher attendance registers relative to teachers found at school. These differences are statistically significant for both male and female teachers.
Of the total schools in the survey sample, only 34 (3% of the total sample) consisting of 16 high,
11 secondary, and seven primary, reported one or more teachers who never showed up at
Figure 12: Overall Teachers by Attendance Register vs Actually Present at School
22
school. This number varies from 1 to 4, with the majority of the 34 schools reporting one permanently absent teacher. Among these schools, four are girls only, 10 are boys only, and 20 are mixed schools. (See Annex III for the list of schools with long-term absent teachers)
2.5.2.6 Head Teachers by Type and Gender The survey also collected data from schools on head teachers. As is normally the case in public schools in Afghanistan, the number of head teachers overall, and by school, is considerably lower than the number of regular teachers. In addition, every school does not necessarily have a head teacher. Of the total sampled schools, 678 (35% primary, 69% secondary, 80% high schools) reported having at least one head teacher. Due to the relatively small number of head teachers, it is not possible to compare the head teacher data by school level (primary, secondary, high) and/or calculate differences for statistical significance. Dividing the head teacher data into further subgroups results in values that are too small to compare. Table 4 and figures 13 and 14 below present comparisons of overall number of head teachers and by head teacher type and gender. Table 4: Summary Statistics for Estimated Total Head Teachers (Weighted) by Type and Gender
Variable Overall Survey MoE
Tashkeel & Ajeer (by Tashkeel)
Male 11,382 11,274 Female 4,036 2,604*
Contract (Ajeer-out of Tashkeel)
Male 759 499 Female 571 330
N 16,177 14,377 * p < 0.05 A. Tashkeel and Ajeer (Be Tashkeel) The overall number of male Tashkeel head teachers between the two data sets is not significantly different (Figure 13; Table 4). On the other hand, the number of female Tashkeel and Ajeer (by Tashkeel) head teachers reported by the MoE is significantly lower relative to the survey data. B. Contract (Ajeer-out of Tashkeel) Apparently, there are differences in the total numbers of male and female contract teachers between the two data sets. However, as a majority of the head teachers are by Tashkeel and Ajeer (by Tashkeel), the number of contract head teachers (Ajeer- out of Tashkeel) is too small to calculate differences for statistical significance. Consequently, the figures presented in Table 4 for contract head teachers should be viewed with caution, as they are not indicative of significant differences between the two data sets.
23
As shown in Figure 14 there are differences in the composite (and uncategorized) head teacher number at the total sample level: the MoE number is significantly lower than those recorded in the survey data.
2.5.2.7 Number of Teachers by Province The provincial level findings are neither definitive nor generalizable due to the small number of schools sampled in each province and significant variation in average number of schools and teachers. The average number of teachers per school varies widely by province. For example, in Daykundi, schools have an average of six teachers, while in Kabul City and Helmand schools average over 20 teachers. On average, Wardak, Paktiya, Khost, Paktika, and Zabul all have large male to female teacher ratios. In contrast, Kabul City and Herat employ the highest proportion of female teachers, with schools in these provinces having over four times the number of female teachers relative to other provinces.
The largest absolute differences in the estimated totals for females are in Takhar, Badakhshan,
and Herat with the MoE data estimating 46%, 29%, and 20% fewer teachers respectively as
Figure 13: Head Teachers by Type and Gender
Figure 14: Overall Total Sampled Head Teachers
24
compared to the survey data. In contrast, the MoE numbers for male teachers for Takhar, Badakhshan, and Herat are 31%, 6%
and 11% respectively less than the survey counts. For male teachers, the largest differences exist in Paktika and Kandahar, with the MoE reporting 45% and 33%, respectively, more teachers relative to the survey data.
In Logar, Uruzgan, and Jowzjan provinces differences in the number of male teachers are
minimal. Likewise, differences in the number of female teachers between the two data sets are nominal in Nimroz, Nangarhar and Laghman provinces (See Figures 15 A and 15 B below, and complete table of teachers by province in Annex IV).
25
Figure 15 A & B: Comparison of Estimated Male and Female Teacher Totals by Province
26
2.5.2.8 Conclusions: Quantitative Survey: Teachers by Type, Gender, Location There are significant discrepancies in the two data sets at the teachers’ subcategory levels—i.e.,
teacher type, gender, and school level. The MoE data varies from the survey data across all subcategories of teachers, with the exception of female Tashkeel teachers. Qualitative data (Part A) from the interviews indicates that variation at the subcategory levels may stem from inaccurate categorization of teacher data at the school and/or district levels. In addition, the MoE-EMIS assembled and organized the teacher data for this assessment before the completion of the data cleaning process at the EMIS central level. The partially cleaned data may have added to the variation between the two data sets at the subcategories of the teachers. Finally, overall measurement error either at the MoE or with the survey, or both might have contributed to the variation between the two data sets.
Despite the discrepancies in the MoE data across the majority of the subcategories, these
discrepancies almost disappear at the total sample and total teachers by school levels. In other words, while the reliability of the MoE data at the subcategory level is questionable, the data is, by and large, consistent at the aggregate level.
Except for the male head teachers (Tashkeel) figures, the MoE’s head teacher data is less reliable
than the teacher data due to discrepancies in head teacher data present at both the subcategory and composite levels.
There appear to be differences between the MoE and survey counts of teachers by province.
However, since the number of schools selected in each province is too small to estimate statistically significant variation between provinces, these findings are inconclusive and cannot be generalized.
The teacher attendance record at school does not accurately account for all the teachers that
are actually absent at school. The practice of marking teachers present who actually are absent is more common for male teachers as compared to female teachers, for primary schools relative to secondary and high schools, and for larger schools than smaller ones.
The proportion of schools reporting teachers who never show up at school (permanently absent
teachers) is fairly small. The practice tends to be more common at the high school level than the secondary and primary levels.
27
2.5.2.9 Findings: Student Enrollment by School Level and Gender
Student data collected for the survey included the total number of boys and girls from grades one to twelve. The student data collected by the survey is compared with the EMIS data provided by the MoE. A. Primary School Students Small to moderate but significant differences exist between the MoE and survey tallies of both girls and boys for every primary class (Figure 16; Table 5). Cumulatively, the MoE data reports 18% more boys, 12% more girls, and, overall, 15% more primary school students as compared to the survey data.
B. Secondary School Students The MoE data includes 15% more boys and 7% more girls at the secondary school level relative to the survey numbers. Overall, the MoE figures are significantly higher than the survey numbers, i.e. the MoE reports approximately 11% more students than the survey data (Figure 17, Table 5). Although differences at the secondary level are smaller than at the primary level, these differences are significant. C. High School Students At the high school level, differences are smaller than the secondary school level. While MoE reports 9% more boys than the survey data (the difference is significant), there is no difference in female student counts between the two data sets (Figure 18, Table 5). Overall the MoE reports 4% more high school students than the survey data.
Figure 16: Primary School Students by Gender
Figure 17: Secondary School Students by Gender
28
Table 5: Summary Statistics for Estimated Student Enrollment Numbers (Weighted) by School Level and Gender
Grade Boys Girls Total
Survey MoE Survey MoE Survey MoE Primary 1 602,780 715,605 * 502,879 553,030 * 1,105,658 1,268,635 * 2 642,395 787,658 * 534,844 607,866 * 1,177,238 1,395,524 * 3 589,276 689,301 * 505,037 564,194 * 1,094,313 1,253,495 * 4 559,919 673,425 * 477,556 550,807 * 1,037,475 1,224,231 * 5 503,883 581,873 * 402,673 452,328 * 906,555 1,034,202 * 6 445,786 508,142 * 352,594 383,714 798,380 891,856 * Total 334,4039 395,6004 * 277,5581 3,111,938 * 6,119,620 7,067,942 * Secondary 7 391,148 465,158 * 275,829 304,121 666,976 769,279 * 8 308,660 351,690 * 228,133 235,569 536,793 587,259 *
9 266,465 296,729 * 187,222 198,429 453,687 495,158 * Total 966,273 111,3577 * 691,184 738,119 1,657,456 185,1695 * High 10 181,777 200,418 158,080 158,631 339,857 359,049 * 11 149,448 166,487 137,717 141,434 287,166 307,921 * 12 133,170 138,658 116,518 112,571 249,688 251,229 Total 464,396 505,563* 412,314 412,637 876,710 918,199 * Grand Total 477,4707 557,5144 * 3,879,079 4,262,693 * 8,653,786 9,837,837*
* p < 0.05
Figure 18: High School Students by Gender
Figure 19: Overall Students (MoE VS Survey)
29
2.5.2.10 Overall Differences: Students by Attendance Register Versus Actually Present at
School The total number of boys and girls reported in the MoE data are significantly higher—15% more boys and 9% more girls—as compared to the survey data. The overall MoE and survey totals of students (boys and girls combined) are significantly different, i.e. the MoE reports approximately 12% more students than the survey data (Figure 19). The results below are from grades 1, 3, 7 and 10. Statistical tests could not be conducted for grades 8, 11, and 12 due to relatively few schools in the sample having these grades and/or due to the number of provinces with a single school with the relevant grade. Since standard errors cannot be calculated when there is only one case within a province, these grades are not included in the analysis below.
The headcount data was collected from approximately 498 boys’ and 356 girls’ schools (includes secondary and high schools with a primary section) for grade 1, from 189 boys’ and 176 girls’ schools for grade 3, from 273 boys’ and 218 girls’ schools for grade 7, and from 123 boys’ and 108 girls’ schools for grade 10. Of the 980, the majority of the schools, approximately 92%, were surveyed in the first of half of the day/during the first shift, and students were head counted only in the shift during which the survey was conducted. If a grade--e.g., grade one—had more than one section in a given school, headcounts were collected from all sections. The comparisons for shifts two and three are excluded, as the cases were too small to test for statistical significance. The number of absent students (both temporarily and permanently) was subtracted from the total number of students listed in the attendance register and compared with the headcount data/number of students actually present at school. Statistical tests were conducted to determine whether significant differences existed between these two variables for both male and female students.
Figure 20: Attendance Register vs Actually Present Students
Figure 21: Overall Students by Attendance Register vs Actually Present Students
30
A. Grade 1 Small and insignificant differences exist between grade 1 students listed present according to school attendance register and those actually present. On average, there are 3% more boys and 2% more girls actually present at school than the school record of present students (Figure.20).
B. Grade 3 The number of grade 3 boys present by school attendance records is significantly lower than the
number of students actually found at school. The survey counted 14% more boys in school as compared to the number of present boys in school records (Figure. 20).
While differences exist between the numbers of girls marked present in the school register and
actually present—i.e., headcounts are lower than the number of girls marked present in attendance records—they are fairly small and insignificant.
C. Grade 7 There is a significant difference between the number of students head-counted, and the number of students present by school attendance register (Figure.20). There are 0.9% more boys and 14% more girls actually found at school than the number of boys and girls marked present in school attendance register.
D. Grade 10 Similarly, significant differences exist between grade 10 boys and girls marked present in school
records and those head counted. Approximately 10% more boys and 17% more girls were actually found at school relative to the number of present boys and girls according to school attendance record. These differences are slightly larger than the differences in lower grades (Figure.20).
Overall, more students are actually present at school than marked present in student attendance
registers (Figure. 21). These fairly small to moderate differences correspond to approximately 3% additional students in both grades 1 and 3, 11% in grade 7 and 12% in grade 10 were reported present as compared to the headcount numbers. There were 9.5% more boys and 7% girls actually found at school than reported by school attendance register. This means on average the data from the headcounts report 2.5 additional boys and 1.9 additional girls actually present per school as compared to the school attendance record. (See Annex IV for number of students present according to school attendance register vs actually present).
2.5.2.11 Number of Students by Province As noted above, the provincial level findings are neither conclusive nor generalizable due to the small number of schools sampled in each province and significant variation in average number of schools and students between provinces. The MoE student totals exceed the survey totals for all but six provinces for girls and six
provinces for boys. Provinces with the most similar student counts across the two data sets are Kunar, Ghor, Kabul City, Uruzgan, Daykundi - for both girls and boys; Faryab - for boys; and Parwan - for girls.
31
The largest discrepancies between the MoE and survey data in student numbers are in Paktika
with MoE data showing 87% more boys and 116% more girls; in Kunduz with 119% more boys and 31% fewer girls; and in Farah with 96% more boys and 47% more girls relative to the survey data (Figures 22 A & B).
32
Figure 22: Female and Male Student Totals by Province from the Survey and MoE Data FEMALE MALE
33
2.5.2.12 Conclusions: Student Enrollment & Attendance The MoE student enrollment numbers are 12% larger than the survey data for every grade for boys,
and for all but grade 12 for girls. The discrepancies are more common at the primary school level relative to secondary and high school levels and more common for boys than for girls. In other words, the reliability of MoE student enrollment data is more questionable at the primary and secondary school levels than at the high school level. Relatively smaller discrepancies in overall girl enrollment numbers as compared to boys are indicative of the girls’ data being more reliable than boys.
Although it appears that the MoE over-reports student enrollment numbers, both male and female,
in the majority of provinces, the findings are neither conclusive nor generalizable as the number of schools selected in each province is too small to estimate variation between provinces.
Similar to teachers’ attendance, student attendance record and the number of students actually
present are inconsistent. Schools are generally under-reporting the actual number of students present. The practice of inaccurately recording student attendance is more pronounced at the secondary and high school levels relative to primary school and slightly more common for boys than for girls overall.
2.6 RECOMMENDATIONS 2.6.1 For USAID USAID should consider continued support to future independent assessments of the EMIS data which
may also compare EMIS data between provinces (e.g., secure and insecure/inaccessible/remote). Such assessments would require large samples from provinces and could provide valuable insights into solutions for variations in EMIS data quality between regions/provinces.
Future USAID support to the MoE should consider including capacity building measures for the MoE-
EMIS at the DED, PED, and EMIS Kabul levels. Such a program may include:
a. Training programs specifically focusing on planning and data collation quality assurance, sorting and cleaning data, analysis and management of EMIS data.
b. Funding and/or providing technical support to the MoE to develop a strong capacity building program
directed toward EMIS at Kabul, provincial, and district levels. The capacity building may include conducting needs assessments of school officials, DEDs, PEDs and MoE-EMIS in Kabul and developing strategies to address capacity gaps at all levels.
c. Supporting the MoE to engage both the EMIS department and the Research and Evaluation Unit on a
regular basis to jointly conduct data quality assessments, EMIS data collection, and analysis.
USAID should also consider exploring ways in which ongoing USAID-funded programs in the country could provide technical support/training to the EMIS department.
2.6.2 For MoE Beyond data collection for the EMIS, the MoE, through its DEDs, should collect data on school
34
operational status at least twice a year. While the MoE collects EMIS data once a year, the frequency of collecting information on schools’ operational status if increased should result in improved monitoring, accountability, and up-to-date and accurate information on functional and nonfunctional schools.
The MoE should explore, on a regular basis, the causes of long-term and temporarily unsanctioned
school closures and identify solutions that may include alternative schooling options for communities affected by these closures.
The MoE should explore the causes of discrepancies between teacher and student attendance
records versus actual presence of teachers and students and develop solutions to improve attendance records at school.
The MoE should provide basic training—e.g., filling out questionnaires, census or EMIS forms—to
individuals primarily responsible for data collection. The MoE should ensure that school officials/leaders responsible for filling out EMIS forms receive proper training to collect school, student, and teacher information by grades/subcategories accurately.
The MoE should develop a rigorous needs assessment program jointly conducted by both the EMIS
department and the Research and Evaluation Unit to identify capacity gaps in EMIS data collection, management, and analysis at various levels.
The MoE should continue and if necessary expand capacity building for DED officials in planning,
collecting and reporting EMIS data and in providing support to schools in data collection on regular basis.
The MoE should consider simplifying the EMIS data collection form, taking into account input from
school, DED, and PED officials.
35
ANNEXES
ANNEX I: STATEMENT OF WORK The Afghanistan Education Information Management Systems DQA Background: USAID and other development partners have worked with the MoE to identify ways to strengthen the validity and reliability of the Ministry of Education (MoE) data, principally collected through the Education Management Information System (EMIS). USAID contracted through the SUPPORT II mechanism to conduct a data verification exercise in late 2015. Through this initial data verification exercise, USAID and other development partners (DPs) learned valuable lessons and identified key gaps in the current system. A key gap is a lack of independent third-party verification of data collected and reported though EMIS. While EMIS has steadily improved, it still relies on source data collected at the district and school levels where training and procedures remain inadequate and ability of officials to travel to schools and verify data remains limited. Efforts are ongoing and USAID is working with other DPs and the MoE to implement assessments and activities to systemically improve MoE capacity to report reliable data. Media have reported on the existence of “ghost” schools, students, and teachers. Dearth of information about school enrollments, actual attendance of students, teachers in schools, and the presence of a school continue to be raised as concerns by the President and the international community. Purpose of Activity: USAID/Afghanistan plans to conduct a Data Verification and Quality Assessment (DVQA) of the Ministry of Education’s Education Management Information System (EMIS). The DVQA will be designed to:
1. Describe the overall data quality of EMIS at the national level 2. Describe overall data quality at the provincial level 3. At the school level, document actual students enrolled, attending on the day of the visit, teachers at
the schools by contract and Tashkeel, and status of the school as functioning or non-functioning and the location as a school building or other
Description of Activity: To achieve the stated objectives, Checchi will sample the following groups of schools with the same tool:
1. A sample of schools sufficient to estimate the average difference between the number of registered and actual students and teachers with a 95% confidence interval.
2. At least 30 schools per province. This number will provide provincial level feedback on performance and allow USAID to identify provinces which have the greatest data quality issues.
The total sample will be approximately 1,050. The larger sample, while not purely representative, can be used to perform some comparisons for learning purposes. The verification tool will be based on the existing EMIS tool so that DQA data will be comparable with EMIS and so that tested translations and wording will be used. The key indicators for the verification exercise are: Existence and educational usage of schools, A clear definition and type of ghost schools found in the sample
36
Number of teachers actually found at the school (both qualified and non-qualified as defined by the MoE EMIS) disaggregated by gender
Number of teachers employed, as recorded by the school (both qualified and non-qualified as defined by the MoE EMIS) disaggregated by gender
Number of students actually found at the school, disaggregated by gender Number of students enrolled, as recorded at the school, disaggregated by gender GPS coordinates for each school location (whether matching EMIS records within a certain distance) The EMIS school code
Other key EMIS questions may be retained in the verification tool, but these will be limited to allow the verification team to focus on the above key indicators. The DVQA team will report the discrepancies between the EMIS-reported numbers of teachers and students and the actual numbers in attendance, as well as discrepancies between the EMIS figures and school-based records. GPS coordinates and photos will be taken at each school location. Discrepancies between GPS coordinates taken by the team and recorded by EMIS will also be reported. In parallel with the data verification exercise, the EMIS DVQA team will analyze the EMIS systems and processes for gathering and reporting data. This analysis will describe how the process works at each level: school, district, province and national. The team will conduct a desk study of existing reports and literature, and gather additional data through a series of interviews at the Central level, as well as visits to provinces and districts. The exact locations to be visited will be identified in the work plan. Based on findings from the field and literature, the team will identify weaknesses and vulnerabilities and provide comprehensive, actionable recommendations for improving the process. To complement the report, the Checchi video team will join select EMIS teams in at least two provinces to capture video of how the verification process is carried out and highlight some challenges involved. The video team will create a 2 to 4-minute video that can be shared with USAID and donors when the verification results are presented. Limitations: It is important to note that in the current security environment the verification team will need to replace many selected schools and even districts due to security concerns. In those cases, replacement schools will be visited, based on a randomly selected list. Replacements will be closely tracked and reported so that the effect of replacement can be taken into account when interpreting the results. Deliverables:
1. A work plan which includes the draft verification tool, the final sampling plan of EMIS.
2. A draft report which provides: The findings from the verification exercise, Conclusions and recommendations
1. A final report approved by USAID and uploaded to the Development Experience Clearinghouse
(DEC). The final dataset will also be uploaded, according to current guidelines. Checchi will also present the findings to USAID before the final draft of the report is accepted.
Timeline and Team: The DQA team will consist of international and local expertise as well as enumerators that can visit schools
37
in the various provinces. The team skill mix should include experience in education M&E systems, preferably with familiarity of the Afghanistan Ministry of Education system, statistical sampling, data analysis, management of teams, particularly enumerator teams collecting data simultaneously in various parts of the country. The leadership team of the DVQA team should have extensive experience managing monitoring and verification teams, experience in Afghanistan is preferred. The staffing team should be sufficient to provide supervision and quality control of the verification conducted in the provinces as well as be able to provide all of the statistical analysis required to interpret the results. The verification work will be phased, dependent on a district’s status as a “warm-weather” or “cold-weather” school and enumerators should visit the school while the school is in session and students are attending. The DVQA should begin as early as possible to ensure that both warm weather and cold weather schools are visited. Data collected by the enumerators must be compared to EMIS data. If a school is missing from the current year EMIS, the data will be compared to the data for the most recent year EMIS data is available and documented. A six-day work week is authorized for this assignment. An illustrative example of the level of effort (LOE in days) is provided below: A revised LOE is inserted below. Note: Field work will be started in at least two phases to allow the managers to ensure careful quality checking, particularly in the first weeks of data collection. The second group of provinces will begin approximately 10 days after the first group. The phased approach explains the relatively longer LOE for regional managers. October 3, 2016 Revised LOE per Discussion with COR
Position Remote Prep Travel In-
Country
Report Finalization Remote
Total LOE
Expat Team Leader 4 4 24 20 52
Expat Field Manager 4 71 1 76
Afghan Consultant/Manager 71 71
Afghan Regional Manager (5) 60 days LOE each 300 300
Enumerators (78) 35 days LOE each 2730 2730
Quality Assurance/Admin Staff (1) 75 days LOE 75 75
SUPPORT-II Afghan M&E Specialist
Totals 4 4 3271 21 3304
38
ANNEX II: DATA COLLECTION SCHEDULE
Table 6: Data Collection Schedule (Survey)
# Province Climate # of Schools Est. Start Date
Est. End Date
1 Kabul City Cold 31 20-Aug 10-Sep 2 Wardak Cold 31 25-Aug 22-Sep 3 Paktya Cold 30 25-Aug 22-Sep 4 Kapisa Cold 30 25-Aug 22-Sep 5 Ghor Cold 31 25-Aug 22-Sep 6 Bamyan Cold 30 25-Aug 22-Sep 7 Daikundi Cold 30 25-Aug 22-Sep 8 Kabul Province Cold 31 25-Aug 22-Sep 9 Panjshir Cold 30 25-Aug 22-Sep
10 Parwan Cold 30 25-Aug 22-Sep 11 Uruzgan Hot 30 01-Sep 28-Sep 12 Samangan Cold 30 01-Sep 30-Sep 13 Jawzjan Cold 30 01-Sep 30-Oct 14 Badghis Cold 31 01-Sep 28-Sep 15 Balkh Cold 30 01-Sep 28-Sep 16 Badakhshan Cold 31 01-Sep 28-Sep 17 Herat Cold 32 01-Sep 28-Sep 18 Ghazni Cold 32 01-Sep 28-Sep 19 Zabul Hot 30 24-Sep 30-Oct
20 Helman Hot 30 24-Sep 30-Oct 21 Nuristan Cold 30 24-Sep 30-Oct 22 Laghman Hot 30 24-Sep 30-Oct 23 Logar Cold 30 24-Sep 30-Oct
24 Nimroz Hot 30 24-Sep 30-Oct 25 Farah Hot 30 24-Sep 30-Oct 26 Paktika Cold 31 24-Sep 30-Oct 27 Khost Hot 30 24-Sep 30-Oct 28 Kandahar Hot 30 24-Sep 30-Oct 29 Sarepul Cold 30 24-Sep 30-Oct
30 Baghlan Cold 30 24-Sep 30-Oct
31 Kunduz Cold 30 24-Sep 30-Oct
32 Kunar Cold 31 24-Sep 30-Oct
33 Faryab Cold 31 24-Sep 30-Oct
39
34 Takhar Cold 32 24-Sep 30-Oct
35 Nangarhar Hot 32 24-Sep 30-Oct Total Schools 1067
Table 7: Qualitative Interview Schedule: PED Respondents
# Position/Title District Province Data of Interview
1 Planning, Monitoring & Reporting Officer Wardak Wardak 10/15/2015
2 Manager of Statistics Center Ghazni 10/19/2016 3 EMIS Planning Manager Qali naw Badghis 10/10/2016 4 EMIS Academic Supervisor Mehterlam Laghman 10/09/2016
5 GM-EMIS and Planning (PED) Mehterlam Kapisa 10/04/2016
6 Academic Supervisor Mahmood Agha Logar 10/13/2016
7 Chief of the EMIS and Evaluation Branch Hirat Hirat 10/03/2016
8 GM-EMIS and Planning (PED) Charikar Parwan 10/02/2016
9 Head of General Planning Bazarak Panjshir 9/29/2016
10 EMIS Officer/ M& E or Academic Officer Bamyan Bamyan 10/9/2016
Table 8: Qualitative Interview Schedule: DED Respondents
# Position/Title District Province Data of Interview
1 Monitoring Officer Sayed Abad
Wardak 10/15/2016
2 Manager- Supervision
Jaghori Ghazni 10/19/2016
3 Monitoring and Evaluation Team Members
Qadis Badghis 10/13/2016
4 DED-Director & Academic Supervisor
Qali naw Laghman 10/09/2016
5 GM-EMIS and Planning (PED)
Mahmood Raqi
Kapisa
10/04/2016
6 Academic Supervisor
Mahmood Agha
Logar 10/13/2016
7 EMIS Member Anjil Hirat 10/05/2016 8 DED Director Jabel Saraj Parwan 10/03/2016 9 Head of DED and
Acting EMIS Director
Roka Panjshir 9/30/2016
10 M& E or Academic Officer
Bamyan Bamyan 10/9/2016
40
Table 9: Qualitative Interview Schedule: MoE-EMIS Respondents
# Position/Title Location Data of Interview
1 Director EMIS Kabul City 09/17/2017 2 EMIS Data Analyst Kabul City 09/17/2017 3 EMIS Data Analyst Kabul City 09/17/2017
41
ANNEX III: SCHOOL LISTS
Table 10: List of Closed Schools (Temporarily or Permanently)
#
School Code School Name Province
School Operating Status Reason for
Closure Duration of
Closure Explanation/ General Observation
Survey MoE
1 260300031 Shash Borja Uruzgan Perm Closed Closed Unknown Unknown
The school had no building or any enrollment or teacher record.
2 240200024 Dawi Kandahar Perm Closed Open Security More than a year Unknown security reasons
3
300300028 Want Noristan Perm Closed Open Security More than four months
This school has been closed for more than four months due to on-going conflict. The school was closed at the instructions of the Afghan military. It is being used as a military post.
4
150400045 Lala_Khail Samangan Perm Closed Open Non-payment More than a year
This school is closed and the teachers (two) haven’t show up for over a year apparently due to non-payment of salaries. The school does not have a building.
5 260300004 Ghulam M. Khan Uruzgan Perm Closed Open Security More than 6 months Unknown security reasons
6 260300012 Kondalan Uruzgan Perm Closed Open Security More than 6
months Due to tribal conflicts between two ethnic groups
7 260300018 Shazaman Uruzgan Perm Closed Open Security More than a year Unknown security reasons
8 260300022 Aabparan Uruzgan Perm Closed Open Security More than 6 months Unknown security reasons
9 260300040 Sango Uruzgan Perm Closed Closed Security Unknown Unknown security reasons
10 409000012 Baid Mushk K.
Nazar Jan Wardak Perm Closed Open Security Unknown The school was closed due to the on-going conflict.
11 409000033 Bibi Fatima Wardak Perm Closed Closed Security Unknown Since the school did not have its
42
own building, it was operating in a mosque. The Taliban closed it in 2016.
12 250100011 Malakhyam Tokhi Zabul Perm Closed Closed Unknown More than 6
months
13 250200034 Niswan Kochi_ha Zabul Perm Closed Closed Unknown Unknown
14
231800014 Bedwaan Helmand Perm Closed Open Non-payment Less than 6 months
Apparently, the school was operating in a rented building. According to the community members, the school was closed due to non-payment of the rent to the owner of the building.
15
350900033 Khososi Afghan Kabul City Temp Closed Open Unknown Unknown
This school seems temporarily closed. The local residents did not allow for collecting details about the schools.
16 241500034 Sar Kalay
Nulgham Kandahar Temp Closed Open Absent teachers Less than 3 months This school is temporary closed due to absence of teachers.
17 701000032 Naswan Bala
Deh Paktia Temp Closed Open Holiday Less than a month The school was off because today was between two holidays. (Ashura day and Friday)
18 701000033 Lia Sadatkhail Paktia Temp Closed Open Unknown Less than a month
The School was closed for unknown reasons
19
703000003 Shabak Paktia Temp Closed Open Holiday Less than a month
The school was closed due to Eid holidays. However, it was supposed to be open according to the PED. Students do not come a week after the Eid holidays.
20
727000004 Parangay Paktia Temp Closed Open Holiday Less than a month
Teachers from this school told the students to go home for two weeks after the Eid. The school closure seems to be unauthorized.
21 330800002 Khawak Panjsher Temp Closed Open Unknown Unknown
This school seems temporarily closed, At the time of the visit,
43
there were no teachers, principal or students.
22
330900011 Tankho Panjsher Temp Closed Open Unknown Unknown
This school seems temporarily closed, as there were no teachers, principal or students at the time of the visit
23 404000067 Shaheed Fayaz Wardak Temp Closed Open Teacher training Less than a month This school seems temporarily closed--less than two weeks. The teachers were out for a seminar.
24
404000094 Niswan Ghahwaraqul Wardak Temp Closed Open Teacher training Less than a month
This school seems temporarily closed. There were no teachers in the schools, and the door was locked. According the community members, some teachers are participating in a seminar, while others are in Kabul to take examinations.
44
Table 11: List of Schools with Discordant School Level Designations
# School Code Province School Level
MoE Survey
1 130200005 Baghlan Primary Secondary 2 130200082 Baghlan Primary High 3 161000032 Balkh Primary High
4 280100005 Bamyan Primary High 5 280400065 Bamyan Primary High 6 270100083 Ghor Primary High 7 230100056 Helmand Primary Secondary 8 230100060 Helmand Primary High 9 230100078 Helmand Primary High 10 200100010 Herat Primary Secondary 11 350700046 Kabul City Primary Secondary 12 350800076 Kabul City Primary High 13 320300006 Khost Primary Secondary 14 140100021 Kunduz Primary High 15 701000038 Paktia Primary High 16 307000036 Parwan Primary High 17 310200052 Sarpul Primary Secondary 18 260300009 Uruzgan Primary Secondary 19 260300055 Uruzgan Primary High 20 330400003 Panjsher Secondary Primary 21 340300020 Daikondi Secondary Primary 22 340300040 Daikondi Secondary Primary 23 340700037 Daikondi Secondary Primary 24 210100064 Farah Secondary Primary 25 210700008 Farah Secondary High 26 601000070 Ghazni Secondary High 27 270100175 Ghor Secondary High 28 231800029 Helmand Secondary High 29 170100082 Jawzjan Secondary High 30 350400082 Kabul City Secondary Primary 31 140100035 Kunduz Secondary Primary 32 220100018 Nimroz Secondary Primary 33 220400004 Nimroz Secondary High 34 220400009 Nimroz Secondary Primary 35 220400013 Nimroz Secondary Primary 36 701000021 Paktia Secondary High
45
37 726000008 Paktia Secondary . 38 292000004 Paktika Secondary High 39 307000014 Parwan Secondary Primary 40 120800049 Takhar Secondary High 41 260300038 Uruzgan Secondary Primary 42 110900024 Badakhshan High Secondary 43 160100104 Balkh High Primary 44 340500043 Daikondi High Primary 45 210100115 Farah High Secondary 46 270100272 Ghor High Secondary 47 170100092 Jawzjan High Secondary 48 104000023 Kabul Province High Secondary 49 240200010 Kandahar High Primary 50 300100014 Noristan High Primary 51 701000067 Paktia High Secondary 52 150600007 Samangan High Secondary 53 404000052 Wardak High Primary 54 404000074 Wardak High Primary 55 250600010 Zabul High Secondary 56 251200001 Zabul High Primary
46
Table 12: List of Schools with Permanently Absent Teachers
# School Code Province School Level School Gender # of Permanently Absent Teachers
1 280100014 Bamyan Primary Girls 2 2 280100002 Bamyan Primary Boys 2 3 606000048 Ghazni Primary Boys 1 4 350700049 Kabul City Primary Girls 2
5 112000008 Kabul Province Primary Mixed 1 6 220100045 Nimroz Primary Mixed 1 7 260300060 Uruzgan Primary Boys 2 8 280400057 Bamyan Secondary Mixed 1 9 280500047 Bamyan Secondary Boys 1 10 280100033 Bamyan Secondary Mixed 1 11 280200026 Bamyan Secondary Mixed 1 12 340800016 Daikondi Secondary Mixed 1 13 210700012 Farah Secondary Mixed 2 14 231800023 Helmand Secondary Boys 2 15 170100075 Jawzjan Secondary Mixed 1 16 350500039 Kabul City Secondary Mixed 1 17 120600015 Takhar Secondary Mixed 1 18 260100032 Uruzgan Secondary Boys 1 19 190100019 Badghais High Girls 2
20 190100027 Badghais High Boys 3 21 280400053 Bamyan High Girls 3 22 280400021 Bamyan High Mixed 1 23 280100032 Bamyan High Mixed 4 24 280800012 Bamyan High Mixed 2 25 210100022 Farah High Boys 1 26 607000065 Ghazni High Mixed 1 27 230100004 Helmand High Mixed 1 28 230100013 Helmand High Boys 1 29 231800019 Helmand High Mixed 1 30 200300089 Herat High Boys 1 31 200200066 Herat High Mixed 1 32 351200036 Kabul City High Mixed 1 33 350500092 Kabul City High Mixed 1 34 351000011 Kabul City High Mixed 1
47
ANNEX IV: TEACHERS’ AND STUDENTS’ ATTENDANCE & TEACHERS AND STUDENTS BY PROVINCE Table 13: Estimated Total Number of Teachers Present by Attendance Record vs Actually Present at School
Overall Primary Secondary High Attend
Register Headcounts
Attend Register
Headcounts
Attend Register
Headcounts
Attend Register
Headcounts
Male 79884 73921* 13109 12086* 24637 23086* 42138 38749* Female 54029 51020 6450 5528* 15282 14842* 32298 30650*
Table 14: Estimated Numbers of Teachers by Province
Province Female Male
Survey MoE % Difference Survey MoE % Difference
Badakhshan 5929 4402 -25.75 4937 4600 -6.83 Badghais 890 1145 28.65 2782 2893 3.99 Baghlan 5671 5231 -7.76 3545 4196 18.36 Balkh 6484 5184 -20.05 3444 2688 -21.95 Bamyan 1082 756 -30.13 2536 2106 -16.96 Daikondi 1043 845 -18.98 1068 1143 7.02 Farah 2212 3159 42.81 1790 2359 31.79 Faryab 4139 3475 -16.04 5212 5144 -1.30 Ghazni 2227 3145 41.22 7617 6445 -15.39 Ghor 2465 2037 -17.36 5382 4703 -12.62 Helmand 2421 2614 7.97 4532 4640 2.38 Herat 18907 15208 -19.56 4545 4038 -11.16 Jowzjan 4959 5579 12.50 1748 1738 -0.57 Kabul City 16928 15745 -6.99 3670 3329 -9.29 Kabul Province 2797 2542 -9.12 3743 3560 -4.89 Kandahar 1193 2167 81.64 5417 7705 42.24 Kapisa 669 569 -14.95 2518 2669 6.00 Khost 109 131 20.18 3741 4550 21.63 Kunar 1380 1813 31.38 3820 4713 23.38 Kunduz 6794 5832 -14.16 2225 3968 78.34 Laghman 565 528 -6.55 3712 4268 14.98
48
Logar 721 1113 54.37 3349 3294 -1.64 Nangarhar 2285 2190 -4.16 16017 18469 15.31 Nimroz 1304 1318 1.07 597 621 4.02 Noristan 378 594 57.14 1498 2295 53.20 Paktia 134 112 -16.42 2848 4232 48.60 Paktika 278 162 -41.73 4814 7702 59.99 Panjsher 324 221 -31.79 863 760 -11.94 Parwan 1900 1667 -12.26 3972 3972 0.00 Samangan 1169 1330 13.77 1085 1162 7.10 Sarpul 1325 1118 -15.62 2042 1587 -22.28 Takhar 2279 1129 -50.46 6774 4892 -27.78 Uruzgan 266 145 -45.49 1090 1114 2.20 Wardak 159 98 -38.36 4019 3726 -7.29 Zabul 34 23 -32.35 1200 826 -31.17
Table 15: Estimated Number of Present Students According to School Register Vs. Actually Present at School.
Class Attendance Register
Temp. Absent Perm Absent
Total Present (=Register -
Absent -Perm Absent)
Headcount
Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls
1 307061 307505 60359 58046 43948 46482 202754 202977 208993 207391
3 71420 104269 16340 23121 14329 22371 40751 58777 46811*a 55725
7 190666 123307 38714 22631 42735 30226 109217 70450 119291*b 80915*c
10 108290 78776 19572 17042 24179 19024 64539 42710 70775*d 50077*e
* p < 0.05 a Helmand was removed before conducting a t-test due to their only being one class 3 in the province that was head counted. b Herat removed from the analysis. c Wardak and Kunduz removed before analysis. d Nimroz and Paktika removed before analysis. e Statistical test not conducted because seven provinces had only school where class 10 was head counted.
49
Table 16: Estimated Numbers of Student Enrollment by Province
Province Female Male
Survey MoE % Difference Survey MoE % Difference
1 Badakhshan 234,413 173,123 -26.15 129,381 163,249 26.18 2 Badghais 58,483 574,18 -1.82 95,284 103,105 8.21 3 Baghlan 189,339 202,157 6.77 142,110 145,463 2.36 4 Balkh 236,213 241,730 2.34 231,732 251,605 8.58 5 Bamyan 76,629 80,328 4.83 71,068 76,361 7.45 6 Daikondi 60,457 59,899 -0.92 54,720 52,459 -4.13 7 Farah 33,974 49,918 46.93 60,133 122,436 103.61 8 Faryab 119,583 126,584 5.85 218,830 219,580 0.34 9 Ghazni 98,105 121,699 24.05 318,535 333,926 4.83
10 Ghor 139,214 143,464 3.05 117,407 116,125 -1.09 11 Helmand 114,105 144,435 26.58 216,541 274,696 26.86 12 Herat 477,995 567,888 18.81 267,068 295,501 10.65 13 Jowzjan 107,932 131,834 22.15 102,821 148,540 44.46 14 Kabul City 399,619 402,567 0.74 302,863 291,190 -3.85 15 Kabul Province 92,152 115,585 25.43 182,650 225,294 23.35 16 Kandahar 36,773 78,524 113.54 179,227 161,151 -10.09 17 Kapisa 39,775 58,056 45.96 66,139 77,843 17.70 18 Khost 60,484 79,844 32.01 163,373 179,134 9.65 19 Kunar 98,841 100,988 2.17 128,456 129,892 1.12 20 Kunduz 309,944 213,865 -31.00 118,927 285,173 139.79 21 Laghman 69,982 92,479 32.15 85,545 94,071 9.97 22 Logar 39,219 53,847 37.30 122,439 107,638 -12.09 23 Nangarhar 249,138 331,415 33.02 542,735 667,590 23.00 24 Nimroz 43,493 50,573 16.28 31,486 35,555 12.92 25 Noristan 19,994 23,679 18.43 31,320 36,288 15.86 26 Paktia 45,113 52,517 16.41 64,700 118,121 82.57 27 Paktika 25,833 58,371 125.96 109,144 196,643 80.17 28 Panjsher 12,082 13,038 7.91 15,450 15,666 1.40 29 Parwan 105,728 106,585 0.81 107,410 123,718 15.18 30 Samangan 64,008 81,081 26.67 53,564 57,925 8.14 31 Sarpul 57,560 66,764 15.99 64,239 67,730 5.43 32 Takhar 113,605 120,587 6.15 190,560 205,947 8.07 33 Uruzgan 13,842 13,229 -4.43 34,618 34,497 -0.35 34 Wardak 26,925 38,359 42.47 115,492 118,009 2.18 35 Zabul 8,526 10,263 20.37 38,738 43,022 11.06
50
ANNEX V: DATA COLLECTION INSTRUMENTS Introduction of the DVQA survey, and yourselves to the School Head Master My name is _______ and my colleague’s name is ______________________. We are consultants who provide monitoring and evaluation support for development projects in Afghanistan. Currently, we are conducting this survey to assess and verify the quality of data for MoE and help the MoE improve its Education Management Information System (EMIS). Your school has been selected for this survey. Besides your school, we are surveying a number of schools in every province in Afghanistan. Here is our letter of permission from the MoE. As per the survey protocol and part of this survey we will: -Interview the Principal/Head Master of the school; -Review enrollment and attendance records of teachers and students; -Take the photo of the necessary documents and records for further verification; -Observe and take head counts of sample shifts for both teachers and students; -Take photos of the sign board and school building; We will be at your school for about an hour and need your assistance in locating the classrooms. All information will be strictly confidential and kept for data analysis purposes only. No respondents will be identified by name in this report; therefore, your input will be strictly anonymous. Do we have your permission to proceed? Thank you in advance for your time and cooperation with this survey. Date of interview: __________________ Survey serial#________ Enumerators’ names: (1) ____________________________ (2) _____________________________ Respondent’s name: ____________________________Position/Title________________________ Respondent’s contact number _________________ Part A. School Information
School code/EMIS #: Replacement code#:
School name Replacement school name
Name:
School location 1. ◻ Urban 0. ◻ Rural
Geographic Coordinates Latitude Longitude
Village/Area
District City/District
Province
51
Is the school open and functional? 1. ◻ Yes; 0. ◻ No If the school is open and functional proceed with question 1. If the school doesn’t exist or has been permanently closed or temporarily closed, please skip the following questions and proceed to question number 18.
1 School Level 1. ◻ Primary 2. ◻
Secondary 3. ◻ High
4. ◻ Other (please specify)
2 School Gender
1. ◻ Boys
2. ◻ Girls
3. ◻ Mix
4. ◻ Boys with Girls
5. ◻ Girls with Boys 6. ◻ Separate Shifts for boys and girls
3 School Type 1. ◻ Govt
2. ◻ Private
3. ◻ Other (Please specify below)
4 School Structure/Building
1. ◻ Earthen 2. ◻ Concrete 3. ◻ No building
4. ◻ Other (please specify) 5. ◻ Mix (earthen-concrete)
6. ◻ Tents 5 Boundary Wall 1. ◻ Yes 0. ◻ No
6 Number of Rooms ______Classrooms _____Administrative/Other Rooms
7 School Property
1. ◻ Govt 2. Dedicated Private Land 3. ◻ No Land 4. ◻ Rented House 5. ◻ Under a tree/open area 6. ◻ Provided by Mosque 7. ◻ Other
8 Does the school have electricity? 1. ◻ Yes 0. ◻ No
9 Is drinking water available? 1. ◻ Yes 0. ◻ No
10 Number of Toilets
11 Number of Functional Toilets Boys_____ Girls_____ Both ____
12 School Region Climate 1. ◻ Hot 0. Cold 13 Number of Shifts 1
4 Medium of Instruction 1. ◻ Pashto 2. ◻ Dari 3. ◻ Both 4. ◻ Other
15 Is the school in session today? 1. ◻ Yes 2. ◻ If no, explain below why not
52
No
16 How many days is the school open per week? (Write number of days)
_____days
17
Daily School Hours 1st Shift Starts at ___________ Closes at __________
2nd Shift Starts at ___________ Closes at __________ 3rd Shift Starts at ___________ Closes at __________ School surveyed during shift _________ (write shift number)
18 If the school doesn’t exist or has been permanently closed or temporarily closed, please collect and record the following information:
18a Has there ever been a school in this area/village? 1. ◻ Yes 0. ◻
No
If yes, ask the following questions from 18b to 18i. If no, go to 18e to 18 i
18b What kind of school was it? 1. ◻ Govt 2. ◻ Private 3. ◻ Other
(specify)__________
18c Why was it closed?
◻ Moved to another village/area ◻ There were no teachers ◻ There was no building ◻ Security – e.g., the school was attacked ◻ Other reasons (specify) ______________________ 97. ◻ Don’t know why
18d How long has it been closed?
◻ Less than a month ◻ Less than three months ◻ Less than six months ◻ More than six months ◻ More than a year
18e Is there another school in this area/near this area? 1. ◻ Yes 0. ◻ No
18f If yes, how far is it from here Approximate distance ______km
18g What’s its name?
18h What type of school is it? 1. ◻ Govt
2. ◻ Private
3. ◻ Other (specify)__________
18i School Gender 1. ◻ Boys 2. ◻ Girls 3. ◻ Mix
18j Where is this school located Please visit it and confirm its name and record its school code EMIS #
53
Whether the response to Q18 is Yes or No, please check and verify with at least two additional community members (elders or wakeel) within the village/community/area. Write their names here: Community member_____________________________ Phone # (optional) ___________________ Elder/Wakeel __________________________________ Phone # (optional) ___________________ Elder/Wakeel __________________________________ Phone # (optional) ___________________
Part B: Teacher Information
19. Teachers: Please list number of teachers by qualifications (Assigned to this school)
Qualifications
# of Male Teachers (By Tashkeel)
# of Female Teachers (By Tashkeel)
# of Male Teachers (By Contract)
# of Female Teachers (By Contract)
# of Male Teachers (Ajeer)
#of Female Teachers(Ajeer)
19a Total # of Teachers
19b Shift 1
19c Shift 2
19d Shift 3
19e PhD
19f Masters
19g Bachelors
19h TTC 15 Grade
19i TTC 14 Grade
19j TTC12 Grade
19k TVET 14 Grade
19l TVET 12 Grade
19m IE 14 Grade
19n IE 12 Grade
19o Sports 14 Grade
19p Sports 12 Grade
19q General Education
54
19r Secondary Education
19s Primary Education
197 Private Education
20a # of Teachers Absent Today based on daily attendance records Male ______ Female _______
20b
Of all the listed teachers in your school record, are there any who never show up at school to teach?
1. ◻ Yes 0. ◻ No If yes, how many __________
20c For Q 20c below, ask for principal’s permission to head count all the teacher for that shift Total number of present teachers for surveyed shift according to daily attendance register_____________
20d # of present teachers head counted Male ______ Female _______
21. Head Teachers: Please list number of Head teachers by qualifications (Assigned to this school)
Qualifications
# of Head Male Teachers (By Tashkeel)
# of Head Female Teachers (By Tashkeel)
# of Head Male Teachers (By Contract)
# of Head Female Teachers (By Contract)
21a Total # of Teachers
21b Shift 1
21c Shift 2
21d Shift 3
21e PhD
21f Masters
21g Bachelors
21h TTC 15 Grade
21i TTC 14 Grade
21j TTC12 Grade
21k TVET 14 Grade
21l TVET 12 Grade
21m IE 14 Grade
21n IE 12 Grade
21o Sports 14 Grade
21p Sports 12 Grade
55
22a Of all the head teachers listed in your school record, are there any who never show up at school to teach/work?
1. ◻ Yes 2. ◻ No If yes, how many __________
22b # of head teachers absent today Male ______ Female _______
22c # of present head teachers head counted Male ______ Female _______
Part C. Students Information 23. Shift One: Please list total number of students by class and gender Yearly enrolment record/register available? 1. ◻ Yes; 0. ◻ No Daily Attendance record available? 1. ◻ Yes; 0. ◻ No Note: If enrolment record or attendance registers are not available or incomplete, please specify why they are not at the time of this visit and write the missing number__________________________________ _______________________________________________________________________________________
Class
# of Students by Enrollment Yearly Record
# of Students by
# of Students Absent Today
# of Permanently Absent Students
# of Students Present Today (Head Count)*
Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls
23a 1
23b 2
23c 3
23d 4
23e 5
23f 6
23g 7
23h 8
23i 9
21q General Education
21r Secondary Education
21s Primary Education
21t Private Education
56
23j 10
23k 11
23l 12 *Head Count: If you arrive in the school during the first shift, please select: In Primary Schools: Total two classes—class one and class three In Secondary Schools with primary sections: Total two classes---class one in the primary section and class seven in the secondary section. In Secondary Schools without primary sections: Total two classes—class seven and class eight In High Schools (with secondary and primary sections): Total three classes—class one in the primary section, class seven in the secondary section and class ten in the high section. In High Schools without secondary and primary sections: Total two classes—class ten and twelve Important: Head count to be done only in the shift during which the survey team arrives. Collect student data from all sections of the selected classes
24. Shift Two: Please list total number of students by class and gender Note: If enrolment record or attendance registers are not available, please specify why they are not available at the time of this visit? _______________________________________________________________________________________
Class # of Students by Enrolment Record
# of Students by Attendance Register
# of Students Absent Today
# of Permanently Absent Students
# of Students Present Today (Head Count)*
Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls
24a 1
24b 2
24c 3
24d 4
24e 5
24f 6
24g 7
24h 8
24i 9
24j 10
24k 11
57
24l 12 *Head Count: If you arrive in the school during the second shift, please select: In Primary Schools: Total two classes—class one and class three In Secondary Schools with primary sections: Total two classes---class one in the primary section and class seven in the secondary section. In Secondary Schools without primary sections: Total two classes—class seven and class eight In High Schools (with secondary and primary sections): Total three classes—class one in the primary section, class seven in the secondary section and class ten in the high section. In High Schools without secondary and primary sections: Total two classes—class ten and twelve Important: Head count to be done only in the shift during which the survey team arrives. Collect student data from all sections of the selected classes
25. Shift Three: Please list total number of students by class and gender Note: If enrolment record or attendance registers are not available, please specify why they are not available at the time of this visit? _______________________________________________________________________________________
Class # of Students by Enrolment Record
# of Students by Attendance Register
# of Students Absent Today
# of Permanently Absent Students
# of Students Present Today (Head Count)*
Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls
25a 1
25b 2
25c 3
25d 4
25e 5
25f 6
25g 7
25h 8
25i 9
25j 10
25k 11
25l 12
58
*Head Count: If you arrive in the school during the third shift, please select: In Primary Schools: Total two classes—class one and class three In Secondary Schools with primary sections: Total two classes---class one in the primary section and class seven in the secondary section. In Secondary Schools without primary sections: Total two classes—class seven and class eight In High Schools (with secondary and primary sections): Total three classes—class one in the primary section, class seven in the secondary section and class ten in the high section. In High Schools without secondary and primary sections: Total two classes—class ten and twelve Important: Head count to be done only in the shift during which the survey team arrives. Collect student data from all sections of the selected classes
26 Has the MoE EMIS data been collected for this year? 1. ◻ Yes 2. ◻ No
27 If yes, approximately when was the MoE EMIS data collected this year?
Approximately (month/year)___________________
28 Whose responsibility is it to fill out the EMIS form in your school? Title ___________________________
29 How many times do you fill out the EMIS data form per year?
General comment and observation:
PED/DED Qualitative Questionnaire Date of interview: __________________ Province__________________________ District _________________________________ Interviewer names: (1) ________________________ (2)_____________________________ Respondent’s name:________________________ Position/Title_________ Length of time in position_______________________________ Respondent’s contact number ________________________________ ----------------------------------------------------------------------------------------------------------------------------
# Questions Prompts
59
1
Could you describe your responsibilities within the MoE? How long have you served in this position? Could you describe your role in EMIS data collection process?
2
Could you describe how your office (PED/DED) collected EMIS data in 1394 (2015)?
Process/steps of EMIS data collection at the provincial, district and school levels
3
What changes did you observe in EMIS data collection in 1395 (2016)? In your understanding, why were these changes introduced? In your opinion, will these changes help improve the data collection process and the quality of EMIS data? Please explain how?
New changes in the process of EMIS data collection at the center, provincial, district and school level
4
When/what time of the year does the MoE collect data from schools in hot and cold regions? How often is the EMIS data collected from the schools? In what form (paper/digital) does the data from schools and district arrive at your office? How often do you share EMIS data with the MoE?
5
Please describe the role your office plays in data entry Who enters the data? How long does it take? What system are in place in your office to ensure data quality?
6
What kind of support does your office receive from the MoE for EMIS data collection and data entry?
Training Financial resources
7
What kind of support does your office provide to schools for EMIS data collection? Who collects data from schools and how do you assign the EMIS data collection responsibilities to the school officials/teachers? Are the same officials responsible for collecting data/filling out the EMIS form every year? What kind of training do these schools officials receive to be able
60
to fill out the EMIS data collection form? What kind of challenges do schools face when filling out the EMIS form? (EMIS form structure; volume of data required to complete the EMIS form etc.)
8
Please describe the types of EMIS training you and your staff received in the past. How often are these trainings offered? Please describe ways in which these trainings have affected your staff’s skills.
Data collection Quality check Data entry Data cleaning
9
In your opinion, what are some of the capacity gaps in your office that, if addressed, could further improve the data collection process and the quality of EMIS data? How can these capacity gaps be addressed?
Limited human resource Inadequately trained staff Constant Staff transfer within MoE
10
In your opinion, how can the MoE improve the data collection process at the school level?
11
Please describe how often your office uses the EMIS data? Is it used for decision-making? How? Examples. Is it used for strategic planning and budget allocation? How? Examples
12
What recommendations/suggestions do you have to improve the use of the EMIS data at: The Center Provincial level District level
13
What would you recommend for the overall improvement in EMIS data collection process and quality of data? How might the MoE implement these recommendations? What kind of support does your office need from the MoE?
61
14.
How do you track dropouts? Please describe the total, annual and daily student registers to track attendance in each school.
15 When do Principals receive/ review the data that is collected by the DED?
62
MOE- EMIS Kabul Qualitative Questionnaire
Date of interview: __________________ District__________________________
Province _________________________________
Interviewer names: (1) ________________________ (2)_____________________________
Respondent’s name:________________________ Position/Title_________
Respondent’s contact number ________________________________
----------------------------------------------------------------------------------------------------------------------------
# Questions Prompts
1 Could you describe your role within the MoE? How long have you served in this position? Could you describe your role in EMIS data collection process?
2
Could you describe how the MoE collected EMIS data until 1394 (2015)? Could you tell us if the process of data collection varies from province to province? If yes, how and why? When/what time of the year does the MoE collect data from schools in hot and cold regions?
Process of EMIS data collection at the center, provincial, district and school levels
4
What changes have been introduced to EMIS data collection in 1395 (2016)? Could you tell us why these changes were introduced? Could you tell us how they may affect the data collection process and the quality of EMIS data? Please describe how the MoE checks data quality during the data collection?
New changes in the process of EMIS data collection at the center, provincial, district and school level
5
Please describe the process of data cleaning once the MoE completes data collection? Who cleans the data? When? How long does it take to clean the data?
6
In your opinion, does the EMIS department (Kabul) have qualified/trained staff for designing data collection and cleaning processes? Please explain? Does the EMIS staff receive any training in EMIS data collection and data cleaning? How often? Who trains them?
7
In your opinion, what are some of the capacity gaps in the EMIS department that, if addressed, could further improve the data collection process and the quality of EMIS data? How can these capacity gaps be addressed?
Limited human resource Inadequately trained staff Constant Staff transfer within MoE
63
8
Who (officials) is responsible to oversee EMIS data collection at PED and DED levels? How are these officials identified? How often do these officials change/are transferred within the MoE?
Any variation by provinces
9
In your opinion, what are some of the capacity gaps in PEDs and DEDs that, if addressed, could further improve the data collection process and the quality of EMIS data? How can these capacity gaps be addressed?
Limited human resource Inadequately trained staff Constant Staff transfer within MoE
10
Who is responsible for data collection at the school level and how does the MoE identify these officials? Are these officials trained in EMIS data collection? Who trains them?
PED/DED officials School officials—principals, administrators, teachers
11 In your opinion, how can the MoE improve the data collection process at the school level?
12 In many countries, EMIS data is publicly available. Is this true in MoE’s case? If no, why not, please explain?
13
Please describe how the MoE uses the EMIS data? Is it used for decision making? How? Examples. Is it used for strategic planning and budget allocation? How? Examples
14
Please describe ways in which PEDs and DEDs use/are expected to use EMIS data? For what purpose? How often?
15
What recommendations/suggestions do you have to improve the use of the EMIS data at: The Center Provincial level District level
16
What would you recommend for the overall improvement in EMIS data collection process and quality of data? Can the MoE implement these recommendations? What kind of support does the MoE need from donors?
17
Please tell us what kind of challenges the MoE faces in collecting EMIS data collection What kind of challenges does the MoE face in using EMIS data for various purposes?
64
ANNEX VI: GEOLOCALIZATION
The following maps show the locations, by province, of the 1,067 schools surveyed.
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
U.S. Agency for International Development
1300 Pennsylvania Avenue, NW
Washington, DC 20523
