QUANTIFIED PARTICIPATORY ASSESSMENT OF THE IMPACT OF … · 2019-06-07 · 1 IMaCS Pilot Assessment...

transcript

World Bank Institute

Final Report

September 2007

UNICEF

September 2008

Third Party Assessment of GOI-UNICEF Quality Package for Primary Education (2003-2007)

Final Report

ICRA Management Consulting Services Limited

In consortium with

Pragmatix Research and Advisory Services Private Limited

IMaCS, Pragmatix Third Party Assessment of GOI-UNICEF Quality Package for Primary Education (2003-07)

UNICEF-India Country Office

Third Party Assessment September 2008 Page | II

TABLE OF CONTENTS

ABBREVIATIONS/ACRONYMS .................................................................................................................... VI

ACKNOWLEDGEMENTS ............................................................................................................................ VIII

EXECUTIVE SUMMARY ................................................................................................................................ IX

1. INTRODUCTION ....................................................................................................................................... 1

BACKGROUND .................................................................................................................................................... 1 SCOPE OF THE ASSIGNMENT ................................................................................................................................ 2 METHODOLOGY .................................................................................................................................................. 3 QUANTIFIED PARTICIPATORY ASSESSMENT – DISTRICT LEVEL ........................................................................... 4 REVIEW OF STATE/DISTRICT STRATEGIES FOR RAISING QUALITY OF PRIMARY EDUCATION ................................. 6 LEARNING ACHIEVEMENT ASSESSMENT .............................................................................................................. 6 SAMPLE .............................................................................................................................................................. 7 REPORT STRUCTURE ........................................................................................................................................... 7

2. ANDHRA PRADESH ................................................................................................................................. 8

INTRODUCTION ................................................................................................................................................... 8 BACKGROUND .................................................................................................................................................... 8 THE QUALITY PACKAGE IN AP ........................................................................................................................... 8 KEY SURVEY DATA ............................................................................................................................................. 9

Schools, teachers and students ...................................................................................................................... 9 SCHOOL FACILITIES .......................................................................................................................................... 10 CLASSROOM ENVIRONMENT ............................................................................................................................. 12

Classroom environment for Class 1 students .............................................................................................. 12 Classroom environment for Class 2 students .............................................................................................. 13

TEACHING METHODS ........................................................................................................................................ 14 Teaching methods in class 1 ....................................................................................................................... 14 Teaching methods in class 2 ....................................................................................................................... 14

TRAINING AND ITS EFFECTIVENESS ................................................................................................................... 15 Types of training and teachers trained ....................................................................................................... 16 Quality of training ....................................................................................................................................... 17

SUPPORT FOR TEACHERS ................................................................................................................................... 18 Local Support .............................................................................................................................................. 18 Support from cluster resource centres and peers ........................................................................................ 19

OTHER COMMUNITY PERSPECTIVES .................................................................................................................. 20 LEARNING ACHIEVEMENT ASSESSMENT ............................................................................................................ 21 CONCLUDING OBSERVATIONS ........................................................................................................................... 23

3. BIHAR ....................................................................................................................................................... 25

INTRODUCTION ................................................................................................................................................. 25 BACKGROUND .................................................................................................................................................. 25 THE QUALITY PACKAGE IN BIHAR ................................................................................................................... 25 KEY SURVEY DATA ........................................................................................................................................... 26

Schools, teachers and students .................................................................................................................... 26 School facilities ........................................................................................................................................... 27

CLASSROOM ENVIRONMENT ............................................................................................................................. 29 Classroom environment for class 1 students ............................................................................................... 29 Classroom environment for class 2 students ............................................................................................... 30

Third Party Assessment September 2008 Page | III

SUPPORT FOR TEACHERS ................................................................................................................................... 34 Local support .............................................................................................................................................. 34 Support from cluster resource centres and peers ........................................................................................ 35

OTHER COMMUNITY PERSPECTIVES ..................................................................................................... 36 LEARNING ACHIEVEMENT ASSESSMENT ............................................................................................................ 37 CONCLUDING OBSERVATIONS ........................................................................................................................... 39

4. MADHYA PRADESH .............................................................................................................................. 41

INTRODUCTION ................................................................................................................................................. 41 BACKGROUND .................................................................................................................................................. 41 THE QUALITY PACKAGE IN MP ........................................................................................................................ 41 KEY SURVEY DATA ........................................................................................................................................... 42

SUPPORT FOR TEACHERS .......................................................................................................................... 49 Local support .............................................................................................................................................. 49 Support from cluster resource centres and peers ........................................................................................ 50

OTHER COMMUNITY PERSPECTIVES ..................................................................................................... 51 LEARNING ACHIEVEMENT ASSESSMENT ............................................................................................................ 53 CONCLUDING OBSERVATIONS ........................................................................................................................... 55

5. ORISSA ...................................................................................................................................................... 57

INTRODUCTION ................................................................................................................................................. 57 BACKGROUND .................................................................................................................................................. 57 THE QUALITY PACKAGE IN ORISSA .................................................................................................................. 57 KEY SURVEY DATA ........................................................................................................................................... 58

TEACHING METHODS ................................................................................................................................. 62 Teaching methods in class 1 ....................................................................................................................... 62 Teaching methods in class 2 ....................................................................................................................... 63

SUPPORT FOR TEACHERS .......................................................................................................................... 65 Local support .............................................................................................................................................. 65 Support from cluster resource centres and peers ........................................................................................ 66

OTHER COMMUNITY PERSPECTIVES ..................................................................................................... 67 LEARNING ACHIEVEMENT ASSESSMENT ............................................................................................................ 68

6. STAKEHOLDER ASSESSMENT........................................................................................................... 73

INTRODUCTION ................................................................................................................................................. 73 HOW SOUND IS THE QP CONCEPTUALISATION? ................................................................................................. 73 HOW‘S QP DOING? HAS IT HELPED IMPROVING QUALITY? SIX DIMENSIONS OF QP? ........................................ 73 HOW DO TEACHERS AND CONCERNED ADMINISTRATORS REGARD THE QP INITIATIVE? .................................... 74 WHAT ROLE HAVE THE CRCS/BRCS PLAYED? ................................................................................................. 74 QP VIS-À-VIS SSA TRAINING ............................................................................................................................ 74 LEARNING ACHIEVEMENT OF CHILDREN? ......................................................................................................... 75

Third Party Assessment September 2008 Page | IV

WHAT IS THE OVERALL IMPACT OF THE QP? .................................................................................................... 75 ADOPTING QUALITY AS THE ONLY WAY FOR ACHIEVING UEE .......................................................................... 75

7. CONCLUSIONS ....................................................................................................................................... 76

INTRODUCTION ................................................................................................................................................. 76 SCHOOLS, TEACHERS AND STUDENTS ............................................................................................................... 76 SCHOOL FACILITIES .......................................................................................................................................... 77 CLASSROOM ENVIRONMENT ............................................................................................................................. 77 TEACHING METHODS ........................................................................................................................................ 78 TEACHER TRAINING .......................................................................................................................................... 78 TEACHER SUPPORT ........................................................................................................................................... 79 LEARNING ACHIEVEMENT ................................................................................................................................. 79 CONCLUDING OBSERVATIONS ........................................................................................................................... 81

8. RECOMMENDATIONS .......................................................................................................................... 83

SUGGESTIONS FOR FUTURE EVOLUTION OF QP ................................................................................................. 83 Teachers ...................................................................................................................................................... 83 TLMs ........................................................................................................................................................... 84 Implementation plan ................................................................................................................................... 84

ANNEXE 2: QUANTIFIED PARTICIPATORY ASSESSMENT ................................................................. 88

ANNEXE 3: FINAL FIELD FORMATS .......................................................................................................... 94

1. FOCUS GROUP DISCUSSION WITH SCHOOL TEACHERS ......................................................... 94

2. SCHOOL OBSERVATION ................................................................................................................... 101

3. CLASSROOM OBSERVATION ........................................................................................................... 103

4. FOCUS GROUP DISCUSSION WITH PARENTS & COMMUNITY ............................................. 106

ANNEXE 4: STAKEHOLDER ASSESSMENT QUESTIONNAIRE ......................................................... 110

ANNEXE 5: LIST OF SCHOOLS .................................................................................................................. 113

ANNEXE 6: LEARNING ABILITY TESTS USED IN THE ASSESSMENT ............................................ 117

Third Party Assessment September 2008 Page | V

Abbreviations/Acronyms

ADEPTS Advance Education Performance through Teachers Support

AP Andhra Pradesh

AS Academic Support

BRC Block Resource Centre

CE Classroom Environment

CLIP Children Learning improvement Programme

CLAP Children Learning Accelerated Program

CLASP Children Learning After School Programme

CRC Cluster Resource Centre

CP Community Participation

DIET District Institute of Education and Training

DO Drop Outs

DPC District Project Coordinator

DRC District Report Card

EDI Education Development Index

EPO (UNICEF) Education Project Officer

FGD Focus Group Discussion

GOI Government of India

LA Learning Achievement

MDG Millennium Development Goal

MP Madhya Pradesh

MGML Multi-Grade Multi-Level

NPEGEL National Programme for Education of Girls at Elementary Level

NUEPA National University of Education, Planning and Administration

OPEPA Orissa Primary Education Programme Authority

QIP Quality Improvement Program

QP Quality Package

QPA Quantified Participatory Assessment

PTA Parent-Teachers Association

SC Scheduled Castes

ST Scheduled Tribes

SCERT State Council for Education, Research and Training

SE School Environment

SIEMAT State Institute for Education Management and Training

SLM Student Learning Material

SMC School Management Committee

Third Party Assessment September 2008 Page | VI

Abbreviations/Acronyms

SSA Sarva Shikha Abhiyan

SWASTHH (Plus) School Water Sanitation Towards Health and Hygiene

TLM Teaching-Learning Material

ToR Terms of Reference

ToTs Training of Trainers

TT Teacher Training

UEE Universal Elementary Education

UNICEF-ICO United Nations Children Fund – India Country office

VEC Village Education Committee

Third Party Assessment September 2008 Page | VIII

Acknowledgements

This study was commissioned by United Nations Childrens Fund, India Country Office (UNICEF, ICO) for conducting an impact assessment of UNICEF‘s interventions on the addressing the quality of primary education through a ―quality package‖ devised in cooperation with the national and state government departments. The assessment was guided from UNICEF, ICO by Dr. Samphe LhaIungpa, Ms. Chetana Kohli, Ms. Sumaira Chowdhury and Ms. Kiran Bhatty. ICRA Management Consulting Services Limited (IMaCS), a New Delhi-based management consulting firm in consortium with Pragmatix Research and Advisory Services Private Limited, was selected to carry out the review for Bihar, based on methodology developed by UNICEF earlier. The consultants visited Andhra Pradesh (Chittoor district), Bihar (Vaishali district), Madhya Pradesh (Bhind district) and Orissa (Ganjam district) to hold discussions with the respective government departments at both state and districts. Further, the consultants also conducted a field-based qualitative participatory assessment in selected districts. Funding for this study was supported by UNICEF. We would like to express our thanks to all the participating government departments and UNICEF‘s state offices (in particular the education project officers) for their cooperation and support.

Third Party Assessment September 2008 Page | IX

EXECUTIVE SUMMARY

BACKGROUND

The Sarva Shiksha Abhiyan (SSA) is the Government of India‘s (GOI‘s) national programme for universal

elementary education (UEE). Launched in 2001, the programme has sought to ensure that all children in India

completed five years of primary school by 2007 and elementary school by 2010. While SSA improvement in the

quality of education is also a key focus area for SSA, it has not been able to take up this issue as a primary

concern, given the high repetition rates, dropout rates and dismal learning achievement in primary schools.

UNICEF‘s education programme (2003-07) supported SSA by focusing on improving the quality of primary

education so as to make learning more attractive and effective. Thus, UNICEF India conceptualised a holistic,

gender-sensitive, quality education package (QP) for implementation in primary schools in 2003. QP entailed a

multi-pronged approach, involving strategies for the facilitation of across-the-board improvements in the entire

curricular package. QP, therefore, included teaching-learning materials, classroom transactions, teacher training,

assessment and examinations, while simultaneously building community-school linkages and dealing with

infrastructure and environment issues. The main premise of QP was that working on any one of these factors

without addressing inter-related issues would not lead to any significant qualitative improvement in learning

outcomes. Thus, a holistic, broad-based approach was developed to ensure meaningful, sustainable

achievements, while stemming the waste of human resources and the incidence of student dropouts at the

primary level.

Further, QP also set in place a set of standards, norms and practices that addressed six critical dimensions of

quality: school and classroom infrastructure standards to support a more interactive pedagogy; teacher

orientation; development of child friendly teaching learning materials and processes; and community

engagement and support to quality education. UNICEF‘s raison d'etre of developing QP and demonstrating its

effectiveness was that QP could eventually be replicated and scaled up across the country through the SSA for a

wider and sustainable impact.

QP was implemented jointly by GOI and UNICEF in 13 states and 25 districts in the country from 2003

onwards. Each state team adopted QP differentially depending on the prevalent system of education in the state

and its districts. Thus, any exercise to compare the progress in adopting QP by different UNICEF-supported

states was constrained by the varying levels of their educational systems.

UNICEF commissioned ICRA Management Consulting Services (IMaCS) in consortium with Pragmatix

Research and Advisory Services Private Limited in November 2007 to undertake a third party assessment of QP

in 4 of the 13 QP states. The states were chosen in such a way that they would not only represent geographical

coverage of QP, but also reflect a cross-section of different QP typologies that existed across the spectrum

(initial-intermediate-mature scale). Based on the above criteria, and lessons from the pilot assessment carried

out in Rajasthan1, the states chosen for the study were Andhra Pradesh (AP), Bihar, Madhya Pradesh (MP) and

Orissa. The first and the last states were regarded as ‗intermediate‘ while the middle two were deemed ‗initial‘.2

PURPOSE

The objective of the assignment was to analyse the effectiveness of QP and evaluate how learning outcomes

were affected by the six QP components:

I. Development and supply of essential teaching-learning materials (TLMs)

II. In-service teacher training (TT)

III. Academic support to improve active learning, continuous, supportive student assessment and rational

class management (AS)

IV. Improvement of classroom environment (CE)

V. Improvement of school environment and facilities (SE)

VI. Promotion of community participation (CP)

In particular, this assessment was also configured to analyse how each different state has been able to implement

QP, given its differing contextual environment and the differential abilities of the key stakeholders (including

the multiple agencies involved in implementing QP and SSA). In all states, QP TLMs were specific to classes 1

and 2.

1 IMaCS Pilot Assessment in Tonk district, Rajasthan, June 2007

2 None of the states implementing QP were deemed ‗mature‘

Third Party Assessment September 2008 Page | X

METHODOLOGY

Impact assessment of an input-based (quality) education programme is not a straightforward exercise as there

are a plethora of interacting variables that can affect the output/outcomes (quality of learning) in the classroom.

For instance, Hanushek3 points out: ―Class sizes have fallen, qualifications of teachers have risen, and

expenditures have increased. Unfortunately, little evidence exists to suggest that any significant changes in

student outcomes have accompanied this growth in resources devoted to schools.‖

For QP (in the larger context of SSA) the ―output variables‖ included enrolment, retention, attendance, drop outs

and students learning achievement. However, for the purpose of the impact assessment both enrolment and

attendance were skewed by the mobilisation drives and the provision of mid-day meal scheme under SSA;

retention was also compromised by varying automatic promotion policies of the state governments. The only

two output indicators that were deemed to be relevant for understanding the impact of QP and guiding future

policies were: improvement in the dropout rates and incremental gains in learning achievement (LA) in QP

schools (vis-à-vis non-QP schools). Further, as the only accurate gauge for the former is a cohort study, only

learning achievement was chosen as the output indicator for this impact assessment study.

Further, as baseline data for the chosen districts was not available, the impact assessment had to devise ordinal

scoring as per quality standards/benchmarks in order to delineate the impact of the quality education package on

QP-schools in comparison with a control group of non-QP schools. A specifically devised methodology in two

distinct modules was used to understand the impact of QP. In the first module, a Quantified Participatory

Assessment (QPA) was undertaken at the district level to understand in qualitative and quantitative terms how

QP schools were performing vis-à-vis non-QP schools and as per UNICEF‘s multidimensional inputs. During

the second module, an overall analysis of the policy environment at the state-level and the role of different

stakeholders at both the district and state levels were done.

QPA collected qualitative and quantitative information from 141 schools in the 4 districts (of which 103 were

QP schools and 38 were non-QP) to ascertain how QP schools were performing vis-à-vis non-QP schools in

terms of the six QP dimensions and chosen output indicator—learning achievement of school children. The

learning achievement was tested in language (local) and mathematics for children of Classes 3 and 5, since these

were the classes for which UNICEF-commissioned learning assessment tests were available. Field teams

comprising of 5 to 6 members spent a day in each school, observing school and classroom environments and

facilities, discussing the quality of teacher training with teachers, academic support and the use of TLMs, and

meeting community representatives to discuss community participation in school activities. The assessment was

completed between February and March 2008, after which the data was collated, cleaned and analysed. The

findings have been reviewed by UNICEF‘s State Education Program Officers (EPOs).

KEY FINDINGS

Although UNICEF‘s quality package is well designed and sequenced, it was resisted by the underlying system

of education, which often looked at the package (barring notable exceptions) as an additional entity. This burden

was further compounded by the implementation vacuum that existed for QP; for instance, even a lag in printing

of QP materials such as readers, could (and did) derail key QP dimensions.

Overall, from an analysis of stakeholders reflections in conjunction with district-level QPA findings about QP, it

is apparent that conceptualising quality in terms of six separate dimensions, UNICEF has changed the policy

and programme context in how quality can be addressed in the field, not only in the UNICEF-supported-districts

but in the entire state. This is true in both comparatively advanced (AP and Orissa) and lagging QP states (MP

and Bihar). However, as there are different degrees of difficulties in implementing the different dimensions,

there has been a noticeable focus on infrastructural elements such as TLMs, CE and SE (in order of priority).

Generally, across all the states, the other three dimensions have generally been neglected (TT, AS and CP).

The key findings of the assessment have been grouped in two categories: Performance of input variables (six

dimensions of QP); and impact of QP on the output variable (learning achievement).

3 Hanushek, E.A. 2003. ―The Failure of Input-Based Schooling Policies‖, The Economic Journal, 113, (485),

F64-F98.

Third Party Assessment September 2008 Page | XI

1. Six Dimensions of QP

I. TLMs: Innovative TLMs have been one of the beneficial corollaries of QP and all states sampled have

developed them in their local language. Wherever used optimally, they have resulted in making the

classroom more inclusive and participatory. However, as the school assessment shows that low level of

TLM usage (inferred or observed) implies that the teachers are not really conversant with the TLMs and

therefore use them quite mechanically with a tendency to repeat them again and again.

QPA data shows that the proportion of schools that used TLMs and student-learning materials (SLMs4)

was comparatively higher in Orissa and AP, but the proportions were generally lower in others especially

Bihar and MP (Table A).

Table A: Teaching methods in surveyed schools in the four states (%)

AP Bihar MP Orissa

QP Non-QP QP Non-QP QP Non-QP QP Non-QP

Use of TLMs Class 1* 50 10 16 0 48 50 72 50

Use of TLMs Class 2* 42 50 8 0 45 38 84 30

Use of SLMs Class 1* 31 40 28 30 7 13 8 0

Use of SLMs Class 2* 43 60 8 11 41 51 84 50

*QPA ordinal scoring: Proportion of schools at benchmark and above

II. TT: As the prime agency for QP training has been Rishi Valley (Chittoor), there has been convergence

among different states/districts in (selecting and) sending their master trainers (teachers) there to ensure

swift start of QP initiatives. Post-training in Rishi Valley, however, experiences of different states have

been mixed. For instance, in Bihar, because of an unfortunate combination of factors, the trained master

teachers were not utilised at all. However, in general, according to the teachers, BRCs and CRCs

consulted QP trainings were indeed valuable and at a much higher level (and value) from the in-service

training.

QPA data shows that all QP teachers in AP and MP received language training, while all QP and non-QP

teachers in MP received training in mathematics. In general, many more teachers from QP schools than

non-QP schools received training, although Bihar and MP showed contrary trends. (Table B). Further, the

quality of training as reported by teachers was also consistently higher in QP schools indicating that the

advent of QP had a positive impact on the trainings of QP teachers.

Table B: Teaching methods in surveyed schools in the four states (%)

AP Bihar MP Orissa

Regular teachers trained 13 5 13 5 100 77 53 0

Para teachers trained 0 0 NA NA 88 100 22 0

Teachers trained in language 100 60 76 0 100 0 88 20

Teachers trained in mathematics 85 20 48 60 100 100 76 40

Teachers trained in TLMs 73 0 40 70 96 100 4 0

III. AS: In the absence of a professional system (or ladder) for the teachers, the idea of academic support is

appealing. However, building this system on the basis of CRCs and BRCs does not seem to be the right

strategy as fundamentally they are from the same stock (or in the same soup) as the teachers (trainees).

As a result, the BRCs/CRCs have largely focussed on administrative affairs and less to a providing

4 For analysis it is useful to differentiate the UNICEF-supported learning materials into two categories: TLMs

and SLMs

Third Party Assessment September 2008 Page | XII

academic guidance. In effect, BRCs/CRCs although trained repeatedly have not become the cornerstones

of the academic support system as yet.

QPA data shows that more schools in Orissa received academic support than schools in other states, and

in general, QP schools received more support, and teachers‘ perceptions were ‗rosier‘ than that of

communities, in all states (Table C). Interpreting this from the context of the overall assessment, ordinal

data only shows that a system of support was functional in all the states; however, as stated above, it was

far from ideal.

Table C: Teaching methods in surveyed schools in the four states (%)

AP Bihar MP Orissa

Academic support from CRC

Coordinator* 54 40 4 30 37 13 68 100

Academic support from peer

group of teachers* 88 90 12 10 4 0 79 90

IV. CE: After TLM‘s, the next area of focus in implementation was CE, as it provided a tangible (and visible)

way to differentiate QP interventions from the others that have taken place over the years. Stakeholder

analysis suggests that in order to make the classroom brighter and more child-friendly, implementation of

QP became overly concerned with the supply of equipment (including classroom furniture, which

because of constraints of space was not relevant everywhere).

QPA data shows that more schools in Orissa and MP had classrooms for Class 1 and 2 children that

provide adequate space, ventilation and lighting (Table D). It was not always the case, however, that QP

schools have the better classroom facilities. While this is the case in AP and Orissa, non-QP schools in

Bihar and MP did better on several of these criteria.

Table D: Facilities in surveyed schools in the four states (%)

AP Bihar MP Orissa

Space, ventilation, lighting Class 1* 19 70 12 20 75 43 75 70

Space, ventilation, lighting Class 2* 38 40 4 0 44 37 72 40

Blackboards and seating Class 1* 12 0 4 10 74 0 12 0

Blackboards and seating Class 2* 20 0 0 0 41 51 12 0

Reading corner Class 1* 20 0 0 0 8 0 8 0

Reading corner Class 2* 20 30 0 0 41 51 8 0

V. SE: In improving the schools‘ environment, QP focused on addressing the lack of recreation facilities and

on innovations such as introducing a garden kit and even ‗building as learning aid (BALA)‘. Although

QP had hoped to harness the synergies between its programming and other sections in UNICEF such as

Child‘s Environment (SWASTHH), however, evidence from the ground in the sampled districts showed

that this seldom happen.

QPA data shows that schools in Bihar and Orissa have the largest number of rooms and teaching rooms,

with AP and MP having the smallest (Table E). Further, QP schools in AP have the highest proportion of

schools with working and accessible drinking water facilities and toilet facilities for girls, followed by

Bihar, Orissa and MP. In general, QP schools perform better than non-QP schools in terms of facilities;

however, this may not be attributable to QP directly. The better state of infrastructure at QP school was

more likely to be an outcome of the pro-activeness of the concerned district/state agency in an UNICEF-

supported school.

Third Party Assessment September 2008 Page | XIII

Table E: Facilities in surveyed schools in the four states

AP Bihar MP Orissa

QP Non-

Average number of rooms 4 3 7 5 3 5 9 10

Average number of teaching rooms 3 2 5 4 2 2 5 6

Proportion of schools with working drinking

water facilities (%) 71 29 63 25 56 19 59 19

Proportion of schools with working and

accessible girls toilets (%) 46 21 38 6 50 7 68 24

Proportion of schools with ‗above

benchmark‘ Watsan facilities* (%) 4 0 20 40 7 0 44 20

benchmark‘ security and cleanliness* (%) 96 90 28 20 71 25 84 80

benchmark‘ Recreation and sports* (%) 31 40 28 30 41 25 44 20

VI. CP: Despite the fact that during conceptualisation, QP emphasised the importance of engaging the

community, this was rarely observed during implementation. The absence of parents (community)

deprives QP of critical support. This is true in all states except in Bihar where parents have been

intrinsically involved in ―Project Sankalp‖ which is focussed on getting students to (or back to) school

and then keeping them (attend) there.

QPA data shows that, in general, QP schools received more support, and teachers‘ perceptions were

‗rosier‘ than that of communities, in all states (Table F).

Table F: Community support in surveyed schools in the four states (%)

AP Bihar MP Orissa

Local support from SDMC

etc. (teachers view)* 42 30 84 80 25 0 84 80

Local support from SDMC

etc. (parents‘ view)* 27 20 80 80 21 0 88 70

Local support from PTA,

GBM etc. (teachers view)* 35 40 0 0 55 0 68 60

Local support from PTA,

GBM etc. (parents‘ view)* 23 20 0 0 62 0 68 60

From QPA data, QP seems to have had a marginally positive effect on the primary schools surveyed in terms of

school and classroom infrastructure, teaching methods, teacher training and community support. However, the

impact of improved support facilities on the learning achievement levels of the students was quite limited.

2. Learning Achievement

The impact on learning achievement is affected critically by the baseline conditions in different states and

districts. In states like AP and Orissa, which may be classified as ‗intermediate‘ on an initial-intermediate-

mature scale, QP has made a positive difference to teaching in Classes 1 and 2. Class 3 children in QP schools

also generally perform better in learning achievement tests compared with their counterparts in non-QP schools,

with the difference being statistically significant in some (but not all) cases. In states like Bihar and Madhya

Pradesh, which may be classified as ‗initial‘ on the same initial-intermediate-mature scale, the roll-out of QP has

not made a significant difference either in the six QP dimensions or in the learning achievement of children.

Third Party Assessment September 2008 Page | XIV

Frequently, non-QP schools performed better on one or more dimension and there were no statistically

significant differences in the learning achievement test results of children.

QP notwithstanding, committed teachers continue to be the key factor in Indian primary school education. In a

rural landscape, where community environment, teacher training and academic support are roughly the same,

some schools provide better teaching services largely because of dedicated teachers. QP has made a difference

in schools where such teachers have seized the opportunity presented by QP, whether it is the TLMs and SLMs,

sports kits, reading corners, infrastructure funds or teacher training. Where these have been provided, these

teachers have made full use of them. Where they have not been provided, and classrooms are over-crowded and

under-furnished, committed teachers have continued to be innovative with local materials for TLMs and with

the skill and resources they have at their disposal to teach the children in their care. Yet, even the most

committed of teachers can be demoralised and turn cynical in the face of poor school and classroom

infrastructure, ineffective teacher training, and insufficient academic support.

QP, however, has made little difference in learning achievement in cases where the teachers are not committed,

irrespective of the other factors. In such schools, although QP has improved water, sanitation and other

infrastructure and also provided for better quality of teacher training and local and academic support to teachers,

there is no commensurate impact on children‘s learning abilities yet, and non-QP schools are performing almost

as well.

CONCLUSIONS

The conclusions have been categorised according to the benchmarks for impact assessment—relevance,

effectiveness, efficiency, sustainability and impact—and are presented below.

Relevance: Because of UNICEF‘s initiative, quality of primary education has become essential (no longer ―just

desirable‖) to both the discourse and the implementation of primary education across the country. At the district

level (instrumentally relevant administrative unit), however, there is a single-minded focus on ensuring access in

quantitative terms. Thus, QP has not been able to demonstrate the relevance of focussing on quality to achieve

The infrastructural aspects of QP have been more readily adopted by SSA. However, innovations such as the

learning ladder based on cards, which help the learner to take an independent path of learning, have been

successively diluted by the SSA authorities, for instance, in AP. The erstwhile system has integrated the TLMs

developed by the Chittoor district but only by diluting their innovative design (and thereby reducing the

potential for improving quality).

Hence, QP was relevant conceptually and has helped to shape the policy discourse on both why and how the

quality of primary education in the country should be augmented. However, this relevance was diluted during

implementation in most of the districts/states and as a result contrary to QP‘s conceptualisation, quality itself

seems to have been compartmentalised (into different dimensions) and the focus on integrating all the

dimensions together has been lost.

Effectiveness: Despite having well-established offices in the states as well as good working relationships with

the key state departments, UNICEF was not effective in rolling QP as it lacked the critical support from

teachers. For successful implementation, the teachers must own the package and treat it as a broad canvas or a

framework to facilitate learning and also innovate in the classroom.

Efficiency: IMaCS‘ earlier study5 for UNICEF for designing the analytical framework for cost-benefit analysis

was stymied by the lack of financial data within UNICEF regarding the cost of QP; programme costs are not

easy to disentangle from support costs. As a result, this impact assessment was not able to determine the

efficiency of QP. Further, it was difficult to estimate the costs.

Sustainability: Given the fragmentation witnessed during implementation, the sustainability of QP without

UNICEF support, as envisaged earlier, is not possible. QP itself has not yet demonstrated its full potential in a

cost-effective manner; however, various dimensions of the package such as the TLMs (and even TT) have been

replicated by the SSA. This certainly can be constituted as a success for the package. However, from the point

5 Development of TOR for Cost Benefit Analysis for Quality Package for UNICEF, IMaCS, June 2007

Third Party Assessment September 2008 Page | XV

of conceptualisation the incorporation of only some parts (dimensions) of the package instead of the whole

package can also be viewed as a failure.

Impact: QP has demonstrated a new approach to education to both teachers and students, so much so that there

is a demand for it to be introduced into higher classes. However, QP‘s impact has been limited because of feeble

implementation; particularly due to its lack of ability to engage teachers effectively.

LESSONS LEARNED

Teachers are the main variable, but have not been engaged effectively by QP: Having failed to engage the

teacher on all the integrated dimensions, QP has not been able to differentiate itself from the supply-driven

approaches. It has thereby fallen short of its goal of instituting a (support) system for delivering quality in

the classroom.

Supplying more or better TLMs does not increase their use: QP intervention has only improved the supply

of TLMs, but has not affected their usage in class by teachers. TLM use, however, has not been a focus area

for QP implementation.

Need better implementation planning for a successful QP: Managing the complex interfaces required for

the success of QP across varying interests (and abilities) of different stakeholders (especially parents and

communities) needs a logical framework or result-based management approach culminating in a time-

bound implementation matrix.

RECOMMENDATIONS

Overall

UNICEF should invest greater energy and efforts in engaging and involving the state governments and

agencies in implementing QP in its true spirit.

Clarify the entire QP implementation system so that it is easily understood by all stakeholders, particularly

community representatives, parents and teachers.

Reconstruct QP to embrace the whole school and do not restrict to just a few grades.

Provide special training to teachers and all key stakeholders in quality management in education, supported

by relevant and high-quality training material on quality management in primary and elementary education.

Turn QP from a supply-driven to a demand-driven programme, by involving teachers and other

stakeholders effectively from design to implementation and monitoring.

Create a more prominent and inspiring role for mothers in implementing QP in schools, to foster greater

participation from parents.

Make QP dynamic, so that it can respond quickly to changes in policy and educational technologies.

Teachers

Involve teachers more in QP design and implementation, to build ownership and foster innovation.

Train and orient leaders of teacher organisations to participate more effectively in QP design and

implementation.

Re-design teachers‘ capacity building programmes to improve personal effectiveness of teachers (soft

skills) first, followed by competence in instruction and content.

Develop illustrated and attractive advocacy literature for teachers in place of conventional and uninspiring

teacher manuals.

Build new mechanisms for on-the-job mentoring of teachers in QP.

Create institutional self-evaluation toolkits for teachers, as this is not only a more powerful tool of

organisational diagnosis, but it also fosters ownership of findings among the teachers.

Complement QP package with an ICT kit, to increase its adoption and effective use.

Strengthen the SLM inputs as these are directly used by students.

Third Party Assessment September 2008 Page | XVI

CRC and BRC

Identify competent and trained staff for the CRCs and BRCs, drawn from among teacher educators in

universities, colleges of education and DIETs, who can effectively mentor teachers, support QP

implementation, and generate resource groups in states.

Free CRC and BRC coordinators from routine administrative work, so that they can perform their roles and

discharge their responsibilities effectively.

The third party assessment of QP shows that only concerted effort by the communities, teachers and the

educational system can ensure that a system of primary education can deliver quality (with access) at the village,

district and state levels. Therefore, the way forward for UNICEF to support the government programme (SSA),

would be to facilitate (implement) quality interventions bottom-up in the future, that is, to concentrate on the

teacher as the prime vector of quality and ensure that all things (aspects) that hinder the teacher‘s ability to teach

are minimised.

Third Party Assessment September 2008 Page | 1

1. INTRODUCTION

Background

In the decade of 1991-2001, the country had recorded tremendous progress in education. The increase in literacy rate from 52% to 65% was the highest ever since independence. Significant progress was made in ensuring access to schools for 90% of children (6 -11 years) within a kilometre distance and attendance rates increased for children (6-14 years) from 68% in 1992-93 to 79% in 1998-99. However, poor quality of learning, completion rates and transition still continued to be a major challenge for the country.

Around the same time, the 93rd Constitutional amendment made elementary education a fundamental right of every child aged 6-14 years. Sarva Shiksha Abhiyan (SSA) was launched by the government with the objective of meeting its commitment to universalizing quality elementary education. The tenth five year plan called for continued focus on improving quality of education and learning achievements, strengthening the education system‘s capability and community involvement in managing education.

UNICEF recognized the need (and opportunity) to define parameters of quality along with the norms already laid out by SSA. It was considered that these would become the benchmarks for all educational interventions and could be used to review progress towards goals. While enrolment was one major achievement, the larger challenge was of retaining children in schools and providing an education that both parents and children could value- in-itself; and also perceive as a good investment.

Further, UNICEF also understood that the issue of 'quality of education' would need to be addressed from various different dimensions in order for SSA to integrate with its roll-out. Therefore, UNICEF adopted a multi-pronged strategy that included:

Positive demonstrative role for quality improvement by developing and implementing a holistic quality package in collaboration with the stakeholders, which included the aspects of teacher development, curriculum development, assessment, and improving the learning environment in schools and making learning meaningful.

Building capacities of village education committees for participation of community in management of education and making schools and community more accountable for achieving universal elementary education (UEE).

Networking with counterpart government departments both at the centre and the state coupled with intersectoral linkages with UNICEF programmes of Health, Nutrition, and Child Environment through a district-based focus would be adopted for achieving programme objectives.

The first task that was set out was developing quality specifications for different aspects of school education. The process was set rolling by first identifying how such quality specifications are developed and how these over a period of time become benchmarks of quality. The key learning‘s that emerged were:

There were two aspects to quality – having the right things in place and doing things right.

Standards comprised of two components – non-negotiable elements/interventions and quality specifications.

A participatory process was required at all levels for evolving a common understanding and agreement on quality specifications.

Standards provided precise guidelines for execution.

Based on the learning‘s it became evident that it would be useful and essential to distinguish between having the right components for quality education and right kind of design and delivery of these. It would not be enough to do the right thing – it needed to be done right. This leads to two important questions:

Were the inputs of the right quality?

Were they being implemented properly?

To address these two questions, quality specifications for the different components of quality initiative were evolved. The specifications are categorized into two levels – essential and effective. Essential levels of specifications were identified as the starting point on which to build upon for reaching the effective levels of quality specifications.

UNICEF and state and district SSA partners rolled out a Quality Package (QP) in 2003. The QP was a set of standards, norms and practices which addressed six critical dimensions of quality, school and classroom infrastructure standards to support a more interactive pedagogy, teacher orientation, development of child friendly teaching learning materials and processes, and community engagement and support to quality education. All these dimensions together constituted QP which could be progressively adopted by SSA. This additional ―quality package‖ for primary schools was in various stages of implementation in 25 districts and 13 major states in the country.

QP also set in place a set of standards, norms and practices that addressed six critical dimensions of quality: school and classroom infrastructure standards to support a more interactive pedagogy; teacher orientation; development of child friendly teaching learning materials and processes; and community engagement and support to quality education. UNICEF‘s raison d'etre of developing QP and demonstrating its effectiveness was that QP could eventually be replicated and scaled up across the country through the SSA for a wider and sustainable impact. The main premise of QP was that working on any one of these factors without addressing inter-related issues would not lead to any significant qualitative improvement in learning outcomes. Thus, a holistic, broad-based approach was developed to ensure meaningful, sustainable achievements, while stemming the waste of human resources and the incidence of student dropouts at the primary level.

Scope of the assignment

UNICEF commissioned ICRA Management Consulting Services Limited (IMaCS) in consortium with Pragmatix Research and Advisory Services Private Limited in November 2007 to undertake a third party assessment of QP in 4 of the 13 QP states6. The states were chosen in such a way that they would not only represent geographical coverage of QP, but also reflect a cross-section of different QP typologies that existed across the spectrum (initial-intermediate-mature scale). Based on the above criteria, and lessons from the pilot assessment carried out in Rajasthan7, the states chosen for the study were Andhra Pradesh (AP), Bihar, Madhya Pradesh (MP) and Orissa. The first and the last states were regarded as ‗intermediate‘ while the middle two were deemed ‗initial‘.

6 Further, a detailed terms of reference for this assignment is appended in Annexe 1.

7 IMaCS Pilot Assessment in Tonk district, Rajasthan, June 2007

The scope of the assignment (See Annexe 1 for terms of reference) was to analyse the effectiveness of the QP package to evaluate how the output variables behave with differing strengths of the UNICEF investments in the 6 input variables such as:

I. Development and supply of essential teaching-learning materials (TLMs)

II. In-service teacher training (TT)

III. Academic support to improve active learning, continuous, supportive student assessment and rational class management (AS)

IV. Improvement of classroom environment (CE)

V. Improvement of school environment and facilities (SE)

VI. Promotion of community participation (CP)

In particular, this assessment was also configured to analyse how each different district (state) has been able to implement the quality package, given its differing contextual environment and differential abilities of the key stakeholders (including the multiple agencies involved in implementing QP/SSA). However, comparing progress in adopting QP by different UNICEF-supported districts/states is constrained by the differing levels of their education system.

Methodology

Impact assessment of an input-based (quality) education programme is not a straightforward exercise as there are a plethora of interacting variables that can affect the output/outcomes (quality of learning) in the classroom. For instance, over the last two decades, there has been an increased emphasis on providing better infrastructure – school and classroom – to primary schools across the world. However, numerous micro and macro level research done since then has not been able to establish a clear relationship between increased investment in inputs (resource) and benefits such as gains in learning achievement. As Hanushek8 observes

“Class sizes have fallen, qualifications of teachers have risen, and expenditures have

increased. Unfortunately, little evidence exists to suggest that any significant changes

in student outcomes have accompanied this growth in resources devoted to schools.”

Therefore, the overall methodology of impact assessment was as per UNICEF‘s methodology devised during an earlier study9. For QP (in the larger context of SSA10) the ―output variables‖ included enrolment, retention, attendance, drop outs and students learning achievement. However, for the purpose of the impact assessment both enrolment and attendance were skewed by the mobilisation drives and the provision of mid-day meal scheme under SSA; retention was also compromised by varying automatic promotion policies of the state governments. The only two output indicators that were deemed to be relevant for understanding the impact of QP and guiding future policies were: improvement in the dropout rates and incremental gains in learning achievement (LA) in QP schools (vis-à-vis non-QP schools). Further, as the only accurate gauge for the former is a cohort study, only learning achievement was chosen as the output indicator for this impact assessment study.

8 Hanushek, E.A. 2003. ―The Failure of Input-Based Schooling Policies‖, The Economic Journal, 113, (485),

F64-F98. 9 Development of TOR for Cost Benefit Analysis for Quality Package for UNICEF, IMaCS, June 2007

10 As QP was dovetailed with the much larger SSA programme, the issues of segreagting the impact of the

former could only be addressed with a control group (as described in the next sub section).

Further, one the shortcoming of planning and implementation of QP was that there no baseline from which to measure changes in all the QP parameters. This lack of comparative data made it very difficult to attribute a positive change to the inputs of QP. Therefore, for the chosen districts, the impact assessment had to devise ordinal scoring as per quality standards/benchmarks in order to delineate the impact of the quality education package on QP-schools in comparison with a control group of non-QP schools.

The assignment was conducted in two separate mutually reinforcing modules. In the first module a Quantified Participatory Assessment (QPA),11

was done at the district level to understand in qualitative and quantitative terms how QP schools were performing vis-à-vis non-QP schools and as per UNICEF‘s multidimensional inputs. During the second module, an overall analysis of the policy environment at the state-level and the role of different stakeholders at both district- and state-level

Quantified participatory assessment – District level

The methodology used for school-level assessment was the QPA (For detailed note, see Annexe 2) which collected both quantitative and qualitative information on all assessment issues. An abridged version of the Learning Assessment Tests for Class 3 and 5, developed by Educational Initiatives Pvt. Ltd, Ahmadabad for UNICEF, was used with the key proviso — an equal number of ‗easy‘ and ‗difficult‘ questions were selected. A simplified workflow process for QPA is given in Figure 1.1.

Figure 1.1: QPA workflow

Five training workshops were held to finalize the QPA field formats. The first in Gurgaon with the four state coordinators developed a draft format, which was again discussed in detail in four state-level workshops with the field staff, state coordinators, and UNICEF consultants and staff, wherever available.12 The field formats were pilot-tested in at least two schools in each state, and the findings were incorporated in the final field formats used in the survey. The final formats used in the field are in Annexe 3.

During the actual survey, a full day was spent in each school, with prior appointment, to carry out the following activities:

Focus group discussion with teachers

School observation

Classroom observation

11 For a detailed note on the methodology see Annexe 2. 12 The UNICEF Education Program Officer (EPO) in Bihar and two consultants participated in the workshop in Bihar, while the EPOs in AP, MP and Orissa were briefed on the survey and workshop.

Visit QP district; interview personnel; review education

documents and data on primary schools (QP & non-QP); hold discussions with stakeholders

With the participation of UNICEF and state officials undertake a rigorous training of the QPA field investigators and conduct pilot assessments to orient the investigators

and to collect design-related feedback from the field

Conduct QPA; and compile analytical report of primary education activities being implemented in the selected QP

districts and differentiate the performance of QP and non-QP schools and across other categories such as social

exclusion; share draft with district, state and UNICEF

Learning assessment of children in Classes 3 and 5, for Language and Mathematics

Focus group discussion with the community

A database was created using Microsoft ACCESS and all the information collected was entered into the database and cleaned before analysis.

An example of descriptive ordinal scoring is given in Box 1.1.

Box 1.1: An example of the ordinal scoring QPA methodology

The assessment of recreation and sports facilities in the schools surveyed in AP is described as an example. The situation on the ground was captured using a 5-point ordinal score where a score of 0 was given to the ‗worst case‘ situation, benchmark was scored at 50, and the ideal situation at 100. Scores of 25 and 75 described ‗below benchmark‘ and ‗above benchmark‘ situations, respectively. Being a fully-flexible ordinal scoring system, situations were also scored ‗in between‘ the main nodes at 0, 25, 50, 75 and 100. The descriptions of the indicators used were developed during the training workshop and fine-tuned during pre-testing (Table 1.1).

Table 1.1: Ordinal scoring formats to assess recreation and sports facilities in schools

Scores Options Score

0 There is a no outdoor playing area for school children

25 There is space available for outdoor play area but no sports kit

50 Benchmark: Space for playing area is available and toys such as skipping ropes and balls are also available

75 In addition, there are safe and functional outdoor play facilities such as swings, slides and ropes; adequate sports material (football, soft ball, cricket kit etc.) is available for all the children;

100 Ideal: In addition, everyday there is a specific time for every class for sports

Reason for score

The proportion of schools where a particular situation was found (e.g., benchmark, or ideal or below benchmark) are given separately for of QP and non-QP schools in Figure 1.2. The findings show that non-QP had better facilities than QP schools, as 40% of the 10 non-QP schools assessed scored above the benchmark score of 50 (describing a situation where ‗space for playing area is available and toys such as skipping ropes and balls are also available‘) compared to only 31% of QP schools. Also 31% of QP schools reported the ‗worst case‘ scenario, i.e., where ‗There is a no outdoor playing area for school children‘, compared to only 10% of non-QP schools.

Figure 1.2: Recreation and sports facilities in surveyed schools in AP

Non-QP

Worst case Below benchmark Benchmark to Excellent Excellent to Ideal Ideal case

Review of state/district strategies for raising quality of primary education

To better understand the policy and implementation context (constraints) at the state and district level a senior team undertook the state and district stakeholder assessment (For final questionnaire see Annexe 4). Essentially, this was conducted by directly interacting with state- and district- level stakeholders including state and non-state agencies/actors. The team visited all the four selected UNICEF state office s as well as the offices of the state-SSA, SCERT, DIET, SIEMAT to interview key informants and collect relevant information. Also, with the facilitation of UNICEF and SSA offices, key stakeholders were invited to the UNICEF offices (or the team went to the district) for a consultation to discuss the impact of the QP on primary education. A simplified workflow process followed by the team for stakeholder consultation is given in Figure 1.3.

Figure 1.3: Stakeholder workflow

Learning achievement assessment

As the key outcome indicator of the assessment was ―learning achievement‖, revised versions of the learning achievement tests developed for UNICEF Education Initiatives (EI) Pvt. Ltd., Ahmadabad, were used for class 3 Language and Mathematics and class 5 Language and Mathematics, in the four states of Andhra Pradesh, Bihar, Madhya Pradesh and Orissa. The reason why tests were administered to children in classes 3 and 5 were partly because class 3 children would have benefited from two years of QP-based teaching (i.e., in classes 1 and 2), and partly because these were the only classes for which EI had devised tests. The EI language tests used were in the local language, i.e., in Hindi in Bihar and Madhya Pradesh, Oriya in Orissa and in Telugu in Andhra Pradesh.

The EI tests were comprehensive and based on the detailed contents syllabus of these classes. But each test of 30 questions took 1.5 hours to administer. Also, the questions did not have any explicit difficulty levels, although there were questions of different difficulty levels within the test. Further, since correct answers of difficult and easy questions were scored the same in the EI scoring system, an equal weight was given to both difficult and easy questions. Thus, if there were a greater proportion of easy questions, they scores would give a better result – even if children were unable to answer the difficult questions.

The EI tests were therefore amended in two important ways for the present assessment. First, they were shortened to just 10 questions each. Second, they were separated into difficulty levels, with each pedagogical concept (e.g., addition or reading comprehension or writing) having an easy question and a difficult version. There is also an increase in difficulty levels from questions 1 to 10. The scoring system, however, was unaltered. The shortening of the test meant that children, especially those in class 3, needed only half an hour to complete each test – instead of 1.5 hours as in the original EI tests. And the separation into an equal number of easy and difficult questions means that it was easy to

Meet UNICEF Education Personnel at ICO; discuss QP;

collect available information

Meet UNICEF State Field Office and SSA/SCERT/DIET; interview personnel; collect available communication strategy/plan documents, review. Facilitate state-level

experience sharing consultation

Compile analytical report of primary education activities being implemented in the state and in the selected QP

district; share draft with UNICEF, State

identify good performance in answering the more difficult questions. The actual tests used in the assessment are in Appendix 8.

Sample

QPA collected qualitative and quantitative information from 141 schools in the 4 districts (of which 103 were QP schools and 38 were non-QP) to ascertain how QP schools were performing vis-à-vis non-QP schools in terms of the six QP dimensions and chosen output indicator—learning achievement of school children (Table 1.2). The learning achievement was tested in language (local) and mathematics for children of Classes 3 and 5, since these were the classes for which UNICEF-commissioned learning assessment tests were available. Field teams comprising of 5 to 6 members spent a day in each school, observing school and classroom environments and facilities, discussing the quality of teacher training with teachers, academic support and the use of TLMs, and meeting community representatives to discuss community participation in school activities. The assessment was completed between February and March 2008, after which the data was collated, cleaned and analysed. The findings have been reviewed by UNICEF‘s State Education Program Officers (EPOs).

Table 1.2: schools surveyed in different states and districts

State District

Number and type of schools surveyed

QP Non-QP Total

1 Andhra Pradesh Chittoor 26 10 36

2 Bihar Vaishali 25 10 35

3 Madhya Pradesh Bhind 27 8 35

4 Orissa Ganjam 25 10 35

TOTAL 103 38 141

The final list of schools13 was selected from the list provided by the state UNICEF Education Program Officers and the State Education Department authorities in the districts surveyed.

Report structure

The next four sections (Section 2-5) detail the findings of the impact assessment of for the four states, Andhra Pradesh, Bihar, Madhya Pradesh and Orissa. These sections begin with general sample details, and then present the main findings on school and classroom infrastructure, teacher training and its quality, teaching-learning materials and their use, student learning ability and community participation, for that particular state. Section 6 present the findings that emerged by interacting with the key stakeholders in the four states about how UNICEF‘s approach to quality had affected the discourse (and implementation) of quality education initiatives in the state. The findings across all four states and presents with concluding observations are presented in Section 7 while Section 8 outlines recommendations with a view to the future evolution of UNICEF‘s QP.

The list of schools visited is given in Annexe 5

2. ANDHRA PRADESH

Introduction

The chapter begins with an overview of primary school education in Andhra Pradesh (AP). The next section reviews the sample of schools, students and teachers surveyed in AP, followed by the main findings on school and classroom infrastructure, classroom teaching, training and its effectiveness, community perspectives and student‘s learning achievement. The final section presents concluding observations on the findings of the survey.

Background

Andhra Pradesh (AP) is the fourth largest state in India by area and population, and is the largest state in southern India. As per the education development index (EDI) devised by National University of Education Planning and Administration (NUEPA), AP was ranked 12 among 35 states. Among the states chosen for the impact assessment AP had the highest EDI ranking.

As per the Annual Status of Education Report (Rural) 2007, Chittoor was the best ranked district in terms of out of school children and learning levels in class 1-2 and 3-5. As per the District Report Card (DRC), Chittoor, 2007, the district had 4221 primary schools and 662 upper primary with primary schools out of which over 95% and 91% were rural schools.

According to UNICEF while the state is on track to achieve universal primary enrolment, the realisation of the critical Millennium Development Goals (MDGs) viz., universal completion of elementary education remains a concern for the state given the poor ―quality‖ of teaching-learning being transacted in the majority of state schools. The equity challenges lies in reaching out to the scheduled castes (SCs) and scheduled tribes (STs) groups comprising 16 and 7 per cent of the total population (Census 2001).

The Quality Package in AP

In terms of typologies for QP, the four surveyed districts were differentiated by conceptualisation and implementation of QP on the "initial-intermediate-mature" scale. Chitoor was undoubtedly the most forward of these districts, although referred as intermediate, it was probably at the cusp of turning mature.

The preparation for implementing the quality package in Andhra Pradesh started in early 2003-04. Chitoor was chosen as a pilot district due to four reasons: initiative by J Krishnamurthi Foundation to support multi-grade multi-level (MGML) education in the districts bordering Tamil Nadu; legacy of an earlier successful Janshala programme; proximity (and influence) of Rishi Valley Institute for Educational Resources (RIVER) who pioneered one of the most influential MGML "School in a Box" kits; and of having one of the better district institutes of education (DIET) in the state. This combination helped Chitoor

in preparing the district for the adoption of innovative MGML materials and techniques which were one of the cornerstones of UNICEF's QP.

Despite supply gaps for the newly conceptualized MGML materials including graded readers and a teachers manual, which delayed the actual implementation of QP to early 2005, the focus of QP interventions in Chitoor was inside the classroom – TLM, CE, AS and TT (in order of priority). In addition, due to the Community Participation Act (1998), even the community was involved and received daylong training regarding QP – what are they supposed to do (monitor) and what to expect.

Key survey data

A total of 36 rural schools were surveyed, 28 of which were QP schools and 10 were non-QP schools (Table 2.1).

Table 2.1: Schools of different types surveyed in Chittoor, AP

Schools surveyed

Numbers

QP Non-QP Total

Primary 19 9 28

Primary with upper primary 7 1 8

Total 26 10 36

The main findings concerning teachers and students are given below.

Schools, teachers and students

There were more teachers in QP schools: There were 3.0 teachers per QP school compared to only 1.7 teachers in non-QP schools.

Most teachers were regular and male: More than 90% of teachers were regular and around two-thirds of the teachers were male in both QP and non-QP schools (Table 2.2). These were largely in line with the trends prevailing in the district (and the state) as per the district and state report cards (2007). In terms of significance, the less dependence on para-teachers reflects not only state policy but also points to a functioning system of recruitment of regular teachers.

Table 2.2: Teachers in schools surveyed in Chittoor, AP

QP Non-QP Total

Regular teachers as a proportion of all teachers 91% 94% 91%

Proportion of regular teachers who are female 37% 31% 36%

Proportion of para teachers who are female 43% 100% 50%

More students per teacher in non-QP schools: On the average, there were 42 students per teacher in non-QP schools, compared to only 22 students per teacher in QP schools (Figure 2.1). Although the non-QP schools did not meet the SSA norms or the district average (23 as per DRC Chittoor, 2007), QP schools were as per design specifications that call for a much lower student teacher ratio in classrooms. As a result, teachers in Chittoor‘s QP schools should have been, in principle, been able to devote significantly larger time per student.

Figure 2.1: Students enrolled in the schools surveyed in Chittoor, AP, 2007-08

Number of students

per school

Number of students

per teacher

QP schools surveyed Non-QP schools surveyed Total schools surveyed

Student enrolment grew, but there was no pattern to changes in enrolment: The number of children enrolled during 2007-08 in classes 1 – 5 in the 26 QP schools during 2007-8 was 2% higher than in 2006-07, while it was 8% higher in the case of non-QP schools surveyed. Changes in enrolment from 2007 to 2008, however, were not uniform across classes or across boys and girls, in QP or non-QP schools.

High student absenteeism: On the day of the survey, an average of 16% of enrolled students in QP schools was absent, compared to 12% in non-QP schools (Table 2.3). Further, there was no systematic variation in absenteeism across classes, or across boys and girls.

Table 2.3: Student absenteeism on day of the school survey in Chittoor, AP

Class 1 Class 2 Class 3 Class 4 Class 5

Average Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls

QP schools 17% 14% 29% 22% 11% 13% 0% 14% 22% 14% 16%

Non-QP schools 17% 14% 11% 25% 14% 13% 17% 0% 13% 0% 12%

Teachers are not regular: Only 11% of the regular teachers and 9% of the para-teachers in QP schools achieved the benchmark of 140 days or more in the last academic year. The situation was even more dismal in non-QP schools where only 3% of the regular teachers and 5% of the para-teachers achieved the same benchmark. These dismal statistics, in view of the multiple interventions by SSA and QP to specifically address this, shows how difficult it is to change the practice of teacher absenteeism in the district (state and country!).

School facilities

Non-QP schools are smaller with only 2 rooms on average used for teaching: The surveyed QP schools have an average of 4 rooms, of which 3 were used for teaching, while the surveyed non-QP schools are smaller and have an average of 3 rooms, of which 2 were used for teaching. The other room was most commonly used as the office (Figure 2.2).

Figure 2.2: Facilities in schools surveyed in Chittoor, AP

Proportion of rooms used for teaching

Proportion of schools with working drinking water

facilities

Proportion of schools with separate girls toilets that are

working AND available for use

Non QP schools surveyed QP schools surveyed

Better drinking water and toilet facilities in QP schools: These were assessed both by asking teachers and subsequently through independent observation (using ordinal scoring). Overall, as per teachers, there was a significant difference between the facilities in QP and non-QP schools surveyed (Figure 2.3). However, assessment using the QPA ordinal scoring methodology showed that water and sanitation facilities were much poorer than what the teachers had mentioned in their assessment for instance, only 1 QP school (4% of the sample) had adequate and safe drinking water facilities, with separate toilets for girls, which were clean and in use. No school, however, provides sanitation facilities as per School Water Sanitation Towards Health and Hygiene (SWASTHH Plus)14 norms or soap for hand washing (score of 100).

Figure 2.3: School facilities in QP and non-QP schools in Chittoor, AP

Non-QP

More QP schools secure and clean: On the average QP schools were better than non-QP schools (Figure 2.3) as only 4% were below benchmark compared to 40% in non-QP schools.

There is at least one latrine for 40 teachers, one latrine for 40 girls, and one latrine for 80 boys; and at least

one urinal for 50 boys.

This difference can possibly be attributed to the additional emphasis in QP schools on hygiene-related communication and interventions. One non-QP school (10%) scored zero because it was a very old school (built in 1933) where the walls were in a very bad condition, floors were cracked and the classrooms were not clean. Others scoring below benchmark were also in a dilapidated condition, with monkeys, leaks during rain and damaged windows and dirty classrooms. In the schools scoring above benchmark, classrooms were largely swept and tidy, with progression towards lockable doors, well-maintained exteriors, compound walls and a lockable gate.

More non-QP schools have recreation and sports facilities: As Figure 2.3 shows, 40% of non-QP schools have scores ranging from ‗benchmark‘ to ‗excellent‘, compared to only 31% in QP schools. This contrary finding shows that UNICEF did not focus on supplying the sports kits to all QP schools. The schools that scored zero either did not have land for children to play outdoor games or the land available was full of trees or undulating. In some cases, the school premises were used to thresh and dry agricultural produce or filled with building material. One non-QP reported that because games were not possible, children preferred reading books!

Classroom environment

Three issues that make important contributions in creating an environment conducive to teaching were assessed: the adequate space, ventilation and lighting, blackboards and seating, and space and books for a reading corner. As in the case of school facilities, the options to assess classroom environment are detailed in Annexe 3. Separate assessments were made for classes 1 and 2, and these findings are presented in Figures 2.5 and 2.6.

Classroom environment for Class 1 students

Most QP schools were crowded and poorly lit: Surprisingly, as far as space, ventilation and lighting in class 1 are concerned non-QP schools were much better than QP schools, with 70% above benchmark (Figure 2.4). Only 5 out of 26 QP schools had enough space, light and ventilation. In the other schools, classrooms were small, cramped and poorly ventilated. In many cases, they were congested with children from both classes 1 and 2, and study material.

Figure 2.4: Classroom environment in QP and non-QP schools in Chittoor, AP: Class 1

Non-QP

Readin

Most schools have blackboards, but children sat on the floor: A small number of model schools in the district were supplied with colourful moulded tables and chairs, and as a result, a small proportion (16%) of QP schools achieved the benchmark (Figure 2.4). In most QP schools scoring below benchmark, children were sitting on the floor without mats, as was the case in all the non-QP schools surveyed. As the classrooms have limited space, despite the low classroom student ratios in the district, the introduction of furniture often resulted in over-crowding, and made access around the classroom difficult. On the other hand, blackboards were functional and in use in all schools, but extra writing space for children were only in the above-benchmark scoring QP schools.

Classrooms too crowded for ‘reading corners’: Although QP schools perform slightly better than non-QP schools for the ‗reading corner‘ (Figure 2.4), however as this intervention had not been standardised, the few successes were primarily due to teachers initiative. As mentioned earlier, most classrooms are over-crowded and therefore there is no place for a reading corner. Only in a small minority of cases is there sufficient space for a reading corner, and in one case (which received a score of 100), a separate room was being used as a library!

Classroom environment for Class 2 students

Most schools had multiple classes in one room: In most cases, even in QP schools, there was only one room for all the classes and to store school material! Classes 1 to 3 were crammed into a single room, sitting on the floor, sometimes without place for the teacher to move about in the classroom and without sufficient space for children to do their deskwork.

Teaching methods

QP interventions in AP were limited to class 1 and 2 as it was primarily focused on strengthening these foundational grades. Three issues were explored in the context of the methods of teaching: the use of teaching learning materials (TLMs), student learning materials (SLMs) and the methods used in classroom teaching. These were assessed using the QPA ordinal scoring system (See Annexe 3)

Teaching methods in class 1

TLMs were better used in QP schools: Both the quality of TLMs15 and their use was better in Class I in the QP schools surveyed with 50% of the schools scoring above the benchmark level, and no school is in the ‗worst case‘ situation (Figure 2.5). This finding shows that QP‘s efforts to promote learning with TLM‘s have been quite successful, given the fact that TLM‘s were only provided to schools in 2005-06.

Figure 2.5: Classroom teaching in QP and non-QP schools in Chittoor, AP: Class 1

Non-QP

In all schools with less-than-benchmark scores, TLMs were either not displayed, or even if displayed, were not being used in regular classroom teaching by the teacher, who continues to teach using the blackboard and textbooks (the traditional method). Children were unable to recognise the TLMs or explain them! Good use of SLMs16 in QP schools: Although SLMs were supplied to QP schools, the survey found that they were insufficient for the number of children in the class (Figure 2.5) and therefore were largely not given to students to use on their own (to reduce conflict over sharing!). In some non-QP schools, due to teacher‘s initiative, some SLMs (similar to UNICEF supplies) were seen. The presence of these SLMs shows that these materials have the potential to become sustainable across all schools.

As often classes 1 to 3 were crammed into a single room, sitting on the floor, sometimes without place for the teacher to move about in the classroom and without sufficient space for children to do their deskwork, the findings of class 2 (wherever it was separate) largely mirror the overall findings of class 1.

In QP schools, in addition to the SSA TLM grants, UNICEF had provided TLMs as well as training 16

SLMs included supplies such as the learning ladder and other student-centered learning materials

(mathematics kit)

Training and its effectiveness

This section presents findings on teachers trained, type of trainings, quality of training, and suggestions from teachers on how the training could be improved (see Box 2.1). Note that even trainings in the same subject were conducted in different batches, by different resource persons, in varying locations.

Box 2.1: Impressions on teacher training from the ground: Chittoor, AP

Initially when the QP kit was introduced, trainings were given for a period of 15 days as part of the Pettalo Badi (School in a Box) government training programme. Teachers trained under this programme were given more material than teachers trained subsequently. Thereafter, Sneha bala training was introduced, but only for 5 days and there were no follow-up trainings. Teachers felt that the time given was not sufficient, particularly, for mathematics, and that they could not understand certain concepts clearly. They also felt that they were not given sufficient materials for hands-on experience. Trainings were generally only in Telugu. Focused training on preparation and using TLMs was not conducted separately for teachers; it was included as part of Sneha bala training. But, 8 teachers in Chittoor district were trained in TLM preparations related to music, drawing and learning materials on subject. They, in turn, trained other teachers on these aspects at the Mandal-level during CRC meetings. Similarly, several TLMs already been developed for Mathematics and Telugu were used by resource persons in the demonstration classes.

Table 2.4: Kind of training given to teachers in different academic years, Chittoor, AP

Training Name Period of

implementation Purpose

Quality Improvement Programme (QIP).

2003-04 How to help children reach minimum level in Mathematics and Telugu, by introducing the child to only easy words for identification (sarala padala gurthinpu)

Children Learning improvement Programme (CLIP)

2004-05 Development of skills in Mathematics and Language (Dharalanga chaduvadam and Rayadam)

Children Learning Accelerated Program (CLAP)

2005-06 2006-07 2007-08

All subjects are part of this training, which was designed by DIET

Village profile training Nov 2007 To get the skills of acquiring basic village data

Development of English language skills

2006-07 Improvement in English language skills

As a part of the CLAP, training on all the subjects is being given. Two competencies on each subject are developed. The following competencies are developed in various subjects:

A. Telugu Reading with understanding and Writing on own B. Mathematics Oral Mathematics and Problem solving C. E.V.S-1 Subject understanding and Map reading and recognition D. E.V.S-2 Subject understanding and Drawing pictures E. English Reading and Speaking

The officials at mandal, district and state levels monitor schools with reference to the progress on the above competencies. Teachers are not satisfied with CLAPs as it covers large number of subjects. They preferred the QIP and CLIP. Teachers mentioned that no CLAP training was done from January 2007, and instead a new training program called ‗Children Learning After School Programme‘ (CLASP) has been started.

Types of training and teachers trained

Larger proportion of regular teachers received training in QP schools: Only regular teachers received training, as per a government order in AP. Although the overall proportions were quite low in both QP schools (13%) and non-QP schools (5%), more teachers in QP than non-QP schools received training (Figure 2.6). A larger proportion of female teachers received training in both QP and non-QP schools. Further, in schools with more than average number (3) teachers, it was often the case that some junior teacher(s) received training on the same subject more than once!

Figure 2.6: Proportion of teachers trained in schools surveyed in Chittoor, AP

More teachers trained in language and mathematics, and more from QP schools: All QP school teachers who went for training had received language and literacy training, while this proportion was only 60% for teachers trained in non-QP schools (Table 2.4). Also, a much higher proportion of QP school teachers received training in mathematics (85%) than in non-QP schools (20%). This shows that the add-on 3 –day UNICEF QP training is having a positive impact on the participation of teachers in the in-service teacher training programme.

Table 2.5: Type of training reported in QP and non-QP schools surveyed

Type of training QP Schools Non-QP Schools

Language and Literacy 100% 60%

Mathematics 85% 20%

Preparation of TLMs 73%

Use of TLMs 19%

Refresher training NA

Others 96% 100%

Almost all teachers given training in ‘other’ topics: Most teachers who had gone for training were also trained in ‗other‘ subjects, such as sanitation, wall painting, library creation and maintenance, women‘s rights, adolescent girls, life skills, environment, health, education of handicapped, puppetry, paper crafts and anaemia. These trainings were designed to use school teachers as socio-economic vectors in rural communities.

0% 50% 100%

Proportion of permanent male

teachers who received training

Proportion of permanent female

Proportion of all permanent

QP Schools surveyed Non-QP Schools surveyed

Quality of training

This issue was analysed using QPA ordinal scoring questions and the options to assess are detailed in Annexe 3 and the performance is graphed below (Figure 2.7). It must be noted that even trainings in the same subject were conducted in different batches, by different resource persons, in varying locations.

Figure 2.7: Quality of training received by teachers in schools surveyed in Chittoor, AP

Non-QP

Language

Language trainings rated better by QP school teachers: Teachers in 54% of QP schools reported benchmark and above scores (Figure 2.7). In the lowest scoring QP schools, the teacher trained was not teaching classes 1 or 2 or had been transferred subsequent to the training. Trained teachers across both QP and non-QP schools felt that the duration of the training course was too short and too much was crammed into a few days, which made it difficult to understand the content adequately. Moreover, as there were no refresher and follow-up trainings, the lacuna in understanding persisted and constrained practice.

Mathematics training rated poorly by non-QP school teachers: 75% of trained teachers in non-QP schools rated it ‗worst case‘. Even in QP schools, more than 50% of teachers who received maths training scored it below benchmark. The reasons proffered for this state of affairs included insufficient time given to understand the methods for maths teaching, trainers were not good, not enough practice was given during the training. Even teachers who rated it at benchmark said that the material given was difficult to follow, and that refresher or follow-up trainings were necessary.

More than half the teachers in QP schools rated training in preparing and using TLMs as below benchmark: In QP schools, teachers rating the TLM training below benchmark opined that the duration of the training was too short and hands-on training was not received. Thus, these teachers were not using TLMs. On the other hand, teachers who had scored the TLM training above benchmark were preparing and using TLMs on their own initiative and did not seem to depend on the training alone. Not surprisingly, most of the teachers in non-QP schools had not received such training.

Support for teachers

Local Support

To address the manifold challenges teaching (and managing) primary school students in rural areas, teachers require support from the local community, the parent-teachers association (PTA) and the local officials of the state education department. In AP the local school is supported by a school management committee (SMC). Usually all teachers are members, along with a few parents, representatives from the local village Panchayat and is presided over by the Sarpanch. These support linkages for teachers were explored using QPA ordinal questions (see Annexe 3) by asking the parents and members of the communities during a focus group discussion organised at the end of the school visit by the field team and are presented in Figure 2.8.

Figure 2.8: Local support to teachers in schools surveyed in Chittoor, AP

Non-QP

School

(Teach

School

(Teach

School committees are generally ineffective, work better with local ‘champion’: Across, both QP and non-QP schools, teachers generally tended to give higher scores than community members (including parents) for local-level support in terms of the quality of meetings of school organisations (Figure 2.8). This is probably due to the fact that the SMC meetings happen in the school. In the above-benchmark situations, there is usually a leader (e.g. Sarpanch) who was active and convened regular meetings where, in some cases, decisions were taken on issues such as constructing a new toilet, resolving the drinking water problem, mid-day meals, and getting extra land for the school. There was no perceptible difference in the functioning‘s of SMC across both QP and non-QP schools. This largely reflects UNICEF‘s inability to focus on one of the QP dimensions: ―promotion of community participation‖.

19% 58% 15% 8%

Few schools have good PTAs: There was no systematic difference regarding PTAs across both QP and non-QP schools. In schools scoring above benchmark, parents meet regularly, discuss issues such as the mid-day meal and student performance and take decisions regarding the school. In the best case non-QP school, parents have gone on strike for a new school building and submitted a memorandum to the District Collector. In all below benchmark schools, no PTA meetings were held and if organised, very few parents come. Where parents come, it is only to discuss the performance of their ward, not to discuss general school issues. Again this reflects UNICEF‘s lack of support for facilitating the critical support of the parents (and community) for QP schools

Support from cluster resource centres and peers

The CRCC is supposed to visit schools regularly, observe classroom teaching, give feedback to the classroom teacher, give demonstration lessons as and when required, and join the education inspector and officials of the local DIET/SCERT in a joint inspection of a panel of teachers over time.

Good support from CRCCs to QP schools: In general, the CRCCs appear to have given more support to QP schools. Further, 20% of non-QP schools reported ‗worst case‘ scores, implying that the CRCCs do not visit the school. In the other below benchmark cases, the CRCC visits as a formality, on his way to other places, spends less than an hour with the teachers, but does not offer guidance. The preference of CRCCs for QP schools shows that the trainings provided by UNICEF had probably established reciprocal linkages among QP teachers and CRCCs.

Figure 2.9: Academic support & responses to QP kit, in schools surveyed in Chittoor, AP

Non-QP

Teachers

Community & Parents

Support

support

Strong academic peer support from other teachers: A high percentage (88-90%) of both QP and non-QP schools reported above benchmark scores (Figure 2.9). Most schools reported that regular meetings were organised by the CRC largely pertaining to regular government programmes of CLIP and CLASP, but not usually Snehabala (QP).

Most parents were not aware of the QP kit: From Figure 2.9 it is evident, that most of the parents were unaware of QP kits and this was corroborated from the teachers who were aware of this discrepancy (evidently even more than the parents and the community). In the above benchmark cases, all parents were aware of the QP kit and that their children were being taught something different even from the private schools.

Parents unaware of QP kits reinforces that QP did not focus on engaging the community effectively.

Other community perspectives

Girl children felt safe coming to school: Contrary to the general perception, neither the parents nor the teachers professed any systematic or major concern about girls‘ safety in coming to school or caste-based discrimination in schools. This lack of concern can probably be explained by the fact that a majority of schools in Chittoor district are in hamlets17 which are both limited in extent as well as caste-segregated.

The QP kit was not the major reason for children coming to school: In both QP and non-QP schools, the major reason cited was ‗to learn‘, followed by ‗mid-day meal scheme‘. In QP schools ‗new learning material‘ was also cited as a major reason by teachers and community (Table 2.6).

Table 2.6: Main reasons for children coming to school in Chittoor, AP

Reasons for attending school QP Schools Non-QP Schools

Teachers Community Teachers Community

Mid-day meal 22% 22% 29% 20%

New learning material 33% 17% 4% 0%

To learn 36% 36% 32% 40%

To play 5% 13% 18% 10%

Because parents tell 4% 6% 7% 20%

Not to be a nuisance at home 0% 7% 11% 10%

DRC 2007 reports that around 58% of primary schools were single classroom schools, which was in sync with

the lower number of students per school in Chittoor. This low student population implies that schools were

more likely to have been established in hamlets.

QP‘s universe was the government primary schools whose demography has changed to include both girls and the disadvantaged over the last decade. It is only to that extent QP addressed the issue of exclusion and there was no special initiative for addressing social exclusion in primary education. Three key issues were assessed: (1) learning achievement in language and mathematics in classes 3 and 5; (2) scores on easy and difficult questions; and (3) performance of SC/ST students. Learning achievement test is in Annexe 6

Class 3 Mathematics and Language: QP students outperformed non-QP students but these differences are statistically significant only for mathematics (Figure 2.10).18 Further, a larger proportion of both QP and non-QP students correctly answered the easier question in each pair of questions (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10).19

Figure 2.10: Language and mathematics achievement of class 3 students in Chittoor, AP

Class 3: SC/ST students: The sample was disaggregated by socially disadvantaged groups (such as SC and ST) to determine if they were doing better in QP schools despite no attempt by QP to address them specifically. The findings for class 3 language and mathematics show that SC/ST children in QP and non-QP schools generally had a lower proportion of correct answers to each question (Figures 2.11 and 2.12). However, the differences between SC/ST and non-SC/ST students were not ‗statistically significant‘.20 Statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

18 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers is not statistically significant (using t-tests assuming equal variances. However, the difference is statistically significant (only 5.5% probability of error) if the first two question results are dropped. 19 The 10-question test had five pairs of questions, one easy (odd numbered question) and one more difficult (even numbered question), See the section on Methodology in Chapter 1. 20

Using t-tests assuming equal variances, the probability of error that non-SC/ST students do better in the language test (one-tailed test) is around 30% in QP schools and 33% for non-QP schools. It is 37% for mathematics test differences in QP schools and 15% for non-QP schools.

65%63%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

I II I II I II I II I II

3 Maths QP 3 Maths Non-QP

35%38%

84% 84%

17%19%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

3 Lang QP 3 Lang Non-QP

Figure 2.11: Language achievement of SC/ST and non-SC/ST class 3 students in Chittoor, AP

Figure 2.12: Mathematics achievement of SC/ST and non-SC/ST class 3 students in Chittoor, AP

Class 5 Mathematics and Language: QP students generally outperform non-QP students (Figure 2.13). These differences are statistically significant for language tests suggesting that the better performance by QP over non-QP school students. In contrast, differences are not statistically significant for mathematics.21

Figure 2.13: Language and mathematics achievement of class 5 students in Chittoor, AP

21 For the 10 language questions, the difference across QP and non-QP schools in the proportions of correct answers is statistically significant However, for the 10 mathematics questions, the difference of correct answers is not statistically significant (using t-tests assuming equal variances), i.e., the probability of error is more than 10% if the difference is asserted.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

3 Maths QP: Non-SC/ST 3 Maths QP: SC/ST

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

3 Maths non-QP: Non-SC/ST 3 Maths non-QP: SC/ST

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

3 Lang QP: Non-SC/ST 3 Lang QP: SC/ST

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

3 Lang non-QP: Non-SC/ST 3 Lang non-QP: SC/ST

44%41%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

34% 33%

20%18%

35%33%

67%70%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Generally, a larger proportion of QP and non-QP students correctly answered the easier question in each pair of questions (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10).

Class 5 SC/ST students: SC/ST children in QP and non-QP schools performed worse than non-SC/ST children generally (Figures 2.14 and 2.15). In the case of language in QP schools, and mathematics in non-QP schools, the worse performance was statistically significant.22 Statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

Figure 2.14: Language achievement of SC/ST and non-SC/ST class 5 students in Chittoor, AP

Figure 2.15: Mathematics achievement of SC/ST and non-SC/ST class 5 students in Chittoor, AP

Concluding observations

The concluding observations have been categorised according to the benchmarks for impact assessment—relevance, effectiveness, efficiency, sustainability and impact—and are presented below.

Relevance: In Chittoor, there was already a good degree of cognisance about the need for primary education to be more innovative and focussed on quality due to the legacy of earlier initiatives. Further, the influence of Rishi Valley located within the district and also from Nali Kali from adjacent Karnataka was also palpable. In this sense, QP was the

For the 10 language questions, better performance by non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers, is not statistically significant (using t-tests assuming equal variances), But improved performance by non-SC/ST students in the 10-question mathematics test in QP schools is statistically significant, the probability of error being 4.7%.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

perfect initiative for Chittoor as it integrated many aspects of quality primary education under the rubric of its six dimensions.

Chittoor became the mother-district for QP as most of the other state initiatives began with a training of master trainings in Rishi Valley with in-the-field observation of the Chittoor initiative (if not a model). Innovations such as the learning ladder based on cards, which help the learner to take an independent path of learning were successfully developed and implemented in the classrooms. Even, the erstwhile system has integrated the TLMs developed by the Chittoor district however by diluting their innovative design (and thereby reducing the potential for improving quality).

Chittoor was also able to benefit from the classroom infrastructure provided by UNICEF due to the low student teacher/classroom ratios that prevail in the district. However, on the whole, given the facilitative environment that existed in the district, QP has not as yet become mature as an initiative as the inputs provided to the teacher have not been used or monitored in a supportive manner.

Effectiveness: Chittoor has been one of the favoured districts for educational reforms (as outlined above). In this sense, UNICEF already has well established linkages in the district to phase the implementation of QP effectively. However, the effectiveness of QP was affected bythe two failings of the implementation were related to not bringing the teacher as a partner and giving lower priority to involving the community. Despite demand for introduction of QP tools into other primary classes (3-5) from a significant proportion of teachers and the community, QP remained limited to only class 1 and 2.This factor severely limited the effectiveness (and impact) of the initiative.

Efficiency: IMaCS‘ earlier study23 for UNICEF for designing the analytical framework for cost-benefit analysis was stymied by the lack of financial data within UNICEF regarding the cost of QP; programme costs are not easy to disentangle from support costs. As a result, this impact assessment was not able to determine the efficiency of QP. Further, it was difficult to estimate the costs.

Sustainability: Given the fragmentation witnessed during implementation, the sustainability of QP without UNICEF support, as envisaged earlier, is not possible. QP itself has not yet demonstrated its full potential in a cost-effective manner; however, various dimensions of the package such as the TLMs, MGML, TT and even ‗reading corners‘ have been replicated by the SSA. This certainly can be constituted as a success for the package.

Impact: QP demonstrated a new approach to education to both teachers and students, so much so that there is a demand for it to be introduced into higher classes. However, QP‘s impact has been limited because of feeble implementation; particularly due to its lack of ability to engage teachers effectively.

Development of TOR for Cost Benefit Analysis for Quality Package for UNICEF, IMaCS, June 2007

3. BIHAR

Introduction

The chapter begins with an overview of primary school education in Bihar. The next section reviews the sample of schools, students and teachers surveyed in Bihar, followed by the main findings on school and classroom infrastructure, classroom teaching, training and its effectiveness, community perspectives and student‘s learning achievement. The final section presents concluding observations on the findings of the survey.

Background

Bihar is one of the poorest states in India As per the education development index (EDI) devised by National University of Education Planning and Administration (NUEPA), Bihar was ranked 33 among 35 states. Among the states chosen for the impact assessment Bihar had the lowest EDI ranking.

As per the Annual Status of Education Report (Rural) 2007, Vaishali district was better than the average district in the state in terms of out of school children and learning levels in class 1-2 and 3-5. As per the District Report Card (DRC), Vaishali, 2007, the district had 1217 primary schools and 398 upper primary with primary schools out of which over 95% and around 91% were rural schools.

According to UNICEF while the state is on track to achieve universal primary enrolment, the realisation of the critical MDGs viz., universal completion of elementary education remains a concern for the state given the poor ―quality‖ of teaching-learning being transacted in the majority of state schools. The equity challenges lies in reaching out to the scheduled castes (SCs) and scheduled tribes (STs) groups comprising 16 and 7 per cent of the total population (Census 2001).

The Quality Package in Bihar

In terms of typologies for QP, the four surveyed districts were differentiated by conceptualisation and implementation of QP on the "initial-intermediate-mature" scale. Vaishali was undoubtedly an average of these districts, and was tagged as intermediate.

The two most essential requirements for quality education are teachers and classrooms. Interventions on quality improvement, community mobilization and disparity reduction to achieve universal elementary education cannot have much impact, if these two non-negotiable elements were absent. This was, in fact, the case in Bihar when QP was introduced in late 2005.

For instance, as per state data, in 2004-05, Bihar was behind all states in both these indicators – teachers and classrooms. PTR was as high as 60 in Bihar as against the national norm of 40 and the shortage of teachers was over 115,000. The student per classroom (SCR) was 90, highest in the country. The area per student in classroom was about 2 square feet in the state compared to the national norm of 8-10 square feet per student.

In short, Bihar was grappling with the basic issue of access in terms of physical infrastructure and human resources in schools. Quality improvement, therefore, cannot be the only mantra for education in Bihar. Although quality improvement can lead to improvement in participation, enrolment, retention and equity, it cannot address the access issues. In addition, to the above, the selected QP district was abruptly changed from Nalanda to Vaishali and the actual training of teachers in Rishi Valley was frittered away by this and the mass transfer of teachers that happened in 2007. Nevertheless, there were some innovations in other dimensions such as Bal Sansad in which students were empowered to participate in the school and quite effectively to motivate community participation in the school.

In this context, the introduction of QP in the state may not only have been mistimed but also not relevant as it failed to disprove the notion that quality is, in fact, necessary to solve the issue of access (and equity).

Key survey data

Table 3.1: Schools of different types surveyed in Vaishali, Bihar

Number of surveyed schools

Numbers

QP Non-QP

Primary 12 7 19

Total 25 10 35

The main findings concerning teachers and students are given below.

More teachers in non-QP schools: On average, there are 10.7 teachers in non-QP schools, compared to 9.6 teachers in QP schools. These numbers were significantly greater than the data reported in DRC Vaishali, 2007 – 3.3 in primary and 8.6 in upper primary schools.

There are no para-teachers and most teachers are male: Following a government order, all school teachers have been made regular government employees. Moreover, around two-thirds of teachers in both QP and non-QP schools surveyed were male.

Table 3.2: Teachers in schools surveyed in Vaishali, Bihar

More students per school in non-QP schools: On the average, there were 39 students per teacher in non-QP schools, compared to only 35 students per teacher in QP schools (Figure 3.1). Once again, the surveyed schools seem to be much better than the DRC reported average of 62 and 69 in primary and primary with upper primary schools respectively.

QP Non-QP Total

Figure 3.1: Students enrolled in the schools surveyed in Vaishali, Bihar, 2007-08

Student enrolment grew in QP schools: The number of children in all classes enrolled in the 25 QP schools during 2007-8 was 2% higher than in 2006-07, while it was 3% lower in the case of non-QP schools surveyed. However, this difference is not statistically significant and cannot be attributed to QP.

Very high student absenteeism: On the day of the survey, an average of 43% of enrolled students in QP schools were absent, compared to 38% in non-QP schools, although there is no systematic variation in absenteeism across classes, or across boys and girls. This level of absenteeism indicates that coming to school is not a well-established norm among the children in the district.

Table 3.3: Student absenteeism on day of the school survey in Vaishali, Bihar

QP schools 44% 37% 45% 41% 51% 45% 37% 32% 54% 43% 43%

Non-QP schools 26% 13% 38% 44% 54% 58% 30% 32% 50% 39% 38%

Few teachers attend regularly; more do so in QP schools: 37% of teachers in QP schools worked 140 days or more in the last academic year, however only 16% of the non-QP teachers did so. Although the difference between QP and non-QP in this regard is appreciable, the statistics, in view of the multiple interventions by SSA and QP to specifically address this, are still dismal and require a judicious mix of incentives and effective monitoring to change for the better.

School facilities

QP schools are bigger on average with 7 rooms per school: QP schools surveyed had an average of 7 rooms, of which 5 were used for teaching, while non-QP schools had 5 rooms on average, of which 4 were used for teaching (Figure 3.2).

Number of students

per school

Number of students

per teacher

Figure 3.2: Facilities in schools surveyed in Vaishali, Bihar

Much better drinking water and toilet facilities in QP schools: These were assessed both by asking teachers, and subsequently through independent observation using ordinal scoring. According to the teachers, much more QP schools surveyed had better working drinking water facilities and separate and working toilets for girls (Figure 3.2). However, the independent assessment showed that the water and sanitation facilities were actually better in non-QP schools than QP schools, with 40% of the former scoring above benchmark compared to 20% of the latter (Figure 3.3). This divergent finding shows the difference between access and usage (which was surprisingly better in non-QP schools). In all the below benchmark schools, there were either no drinking water supply within the school premises – because of which toilets were not functional – or it was located far from the school, outside the boundary wall, or the water was not potable. Even where water supply was available and the toilet was clean and in use, there were no separate toilets for girls.

Security and cleanliness was poor in both QP and non-QP schools: Though more QP schools achieved above benchmark than non-QP schools; more non-QP schools achieved excellent to ideal scores (Figure 3.3). In the below-benchmark cases, school premises (including the corridors) were dirty, classrooms were not swept and cleaned or the doors and windows were broken and could not be locked. Only in few above-benchmark cases classrooms were clean and secure, school premises neat and tidy, and a lockable gate to the premises.

facilities

Proportion of schools with separate girls toilets

that are working AND available for use

QP schools surveyed Non-QP schools surveyed

Figure 3.3 School facilities in QP and non-QP schools in Vaishali, Bihar

Non-QP

Recreation and sports facilities were poor in both QP and non-QP schools: Most of the schools scored below benchmark, i.e. there was no space for children to play, or, if there was space, there was either no material (or it was locked up in the school cupboard) for the children to play with.

Three issues that make important contributions in creating an environment conducive to teaching were assessed: the adequate space, ventilation and lighting, blackboards and seating, and space and books for a reading corner. As in the case of school facilities, the options to assess classroom environment are detailed in Annexe 3. Separate assessments were made for classes 1 and 2, and these findings are presented in Figures 3.4 and 3.5.

Classroom environment for class 1 students

Most schools crowded and poorly lit: As far as space, ventilation and lighting in class 1 are concerned non-QP schools perform a shade better than QP schools, with 20% reporting at least the benchmark situation (Figure 3.4). Only in 2 cases each in QP and non-QP schools, were classrooms sufficiently large, well-lit and airy for class 1 children – and these seemed luxurious compared to the situation in other schools!

Figure 3.4: Classroom environment in QP and non-QP schools in Vaishali, Bihar: Class 1

Non-QP

Readin

Seating was poor in both QP and non-QP school; children sat in the open: Most schools scored below benchmark scores, i.e. children were sitting on mats in the open, or on durries brought from home, and only rarely did the school provide benches and portable blackboards for classes conducted in the open. When classes were conducted indoors, children from classes 1 and 2 were put into one room, leaving little room for each child to work comfortably. Only in the few schools reporting ‗above benchmark‘ scores were there tables, chairs, blackboards and sufficient space per child.

No space for reading corners, hardly any books: Only in a few QP schools there was a ‗reading corner‘; further, this QP initiative was not replicated (like in other districts/states) by a non-QP school. In addition, there was hardly any space for books and no books even when there was space!

Classrooms crowded with children from multiple classes: The situation was similar to that of Class 1: A majority of the classes were held out in the open or in classrooms crowded with children of Classes 1 and 2.

Teaching methods

Three issues were explored in the context of the methods of teaching: the use of teaching learning materials (TLMs), student learning materials (SLMs) and methods used in classroom teaching. These were assessed using the QPA ordinal scoring system (see Annexe 3).

Poor use of TLMs, situation a bit better in QP schools: With students sitting in the open often enough, it was not surprising that only 16% of QP schools and none of the non-QP schools were above the benchmark. In some non-QP schools, TLMs were available in the school office, but were not being used in class. Only in 4 QP schools (16% of the sample) were TLMs displayed and in use (Figure 3.5).

SLMs24 hardly being used: SLMs were not available in around 50% of QP schools. In the rest of the QP schools, they were either kept locked or they had been damaged as students were left to use them on their own, without guidance from teachers. The overall situation is again constrained by class 1 being held either in the open or with class 2.

Figure 3.5: Classroom teaching in QP and non-QP schools in Vaishali, Bihar: Class 1

Non-QP

The situation was similar to that of Class 1: A majority of the classes were held out in the open or in classrooms crowded with children of Classes 1 and 2.

This section presents findings on teachers trained, type of trainings, quality of training, and suggestions from teachers on how the training could be improved (see Box 3.1). Note that even trainings in the same subject are conducted in different batches, by different resource persons, in varying locations.

Box 3.1: Impressions on Teacher Training from the ground: Vaishali, Bihar

There are multiple trainings provided either by the Bihar Education Project of the Government of Bihar or UNICEF. Trainings provided by the Government of Bihar are broadly 51 days induction trainings, UJALA I, II & III and separate trainings on Mathematics and Languages. All these trainings were compulsory. Currently, newly recruited teachers are provided with a 51-day induction training which is further broken in to three parts: the first 25 days comprise residential training at block level; then teachers are posted in to their respective schools for 20 days of practical training; and

SLMs included supplies such as student-centered learning materials (mathematics kit)

then, they are finally called back to the training centre for a 5 days final review and feedback

session.

Ten-day Training of Trainers (ToTs) courses for conducting the induction trainings are organised at the district level. Teachers who have undergone ToTs at the district level were however apprehensive that the training may not be sufficient to prepare them for conducting the 51-day training module for new teachers.

UJALA I, II & III trainings are of 10 days each. UJALA I was basically designed for teachers of classes I & II, UJALA II for classes III, IV & V and UJALA III for classes VI, VII & VIII, i.e., for middle school teachers. These trainings were compulsory and every teacher had to undergo these trainings one by one. In practice, however, these trainings are not regular and many teachers have taken only one

such training.

It is also important to note that in Bihar separate trainings on TLMs were not provided at any point of time. Instead, they were in-built into all training programmes. Teachers mentioned that on the

average one day out of the five day training module was spent on TLMs.

UNICEF sponsored trainings were provided on the MGML, Sankalp and Meena Manch. When the MGML concept was first conceived, a 7-day training programme was organised at Nalanda in February 2006. Initially clusters were selected and every school under the cluster with more than two teachers was selected, and within these clusters, two teachers per school were selected. It was compulsory for selected teachers to attend this training, and after the training was completed, the concerned schools were provided with the MGML kit. However, teaches mentioned that many schools did not have MGML-trained teachers because trained teachers had been transferred to other schools. In the second phase of the MGML training, these teachers were given 2-3 days on-site support by the Block Resource Person of UNICEF to improve their teaching abilities.

Sankalp training are organised at the block and district levels and comprise two types: 1-day training on survey and 2-day training on accelerated learning. At the block level, school core teams

are trained whereas at the district level, block core teams are given training.

Trainings on building Meena Manch and Bal Sansads have also been provided by UNICEF. During the assessment, teachers mentioned that the Meena Manch has created awareness on sanitation, prohibition of child marriage, importance of adult education and bringing school girls back to schools. Almost every school was found to have Meena book series.

Very few teachers received training: Less than 15% of male teachers and around 5% of female teachers in the surveyed schools received training in the year preceding the assessment. Of these, more male teachers from QP schools received training (Figure 3.8). In the context of the multiple trainings organised by BEP or by UNICEF, this result is quite surprising and indicates that there were ―designated trainees‖ among the teachers who attended most of the trainings.

Figure 3.6: Proportion of teachers trained in schools surveyed in Vaishali, Bihar

QP teachers underwent more training: On the average, a larger proportion of QP school teachers received training across language and literacy; but, surprisingly more non-QP teachers received training in mathematics, preparation and use of TLMs (Table 3.4). Further, almost all teachers in surveyed QP schools had received QP training from UNICEF.

Table 3.4: Type of training reported in QP schools in Vaishali, Bihar

Language and Literacy 76%

Mathematics 48% 60%

Preparation of TLMs 40% 70%

Use of TLMs 44% 60%

UNICEF (QP) 96% NA

Refreshers 4%

Others 92% 100%

Almost all teachers given training in ‘other’ topics: Almost all teachers who had gone for training reported being trained in ‗other‘ subjects, including sanitation, wall painting, women‘s rights, adolescent girls, life skills, environment, health, education of the handicapped, puppetry, paper crafts and anaemia. These trainings were designed to use school teachers as socio-economic vectors in rural communities

Quality of training

This issue was analysed using QPA ordinal scoring questions and the options used are detailed in Annexe 3. It must be noted that even trainings in the same subject were conducted in different batches, by different resource persons, in varying locations.

Language trainings rated much better by teachers from QP schools: Teachers from 25% of the QP schools surveyed reported benchmark and above scores, compared to non-QP schools, where only 11% reported ‗above benchmark‘ situations (Figure 3.7). In the worst cases (zero scores), either no trainings were held or when held the teachers did not get to use TLMs or provided with TLMs for their own use. In effect, the essential purpose of TLM trainings which was to enable teachers to make TLMs was not understood, except for a few committed teachers.

Mathematics training poorly by teachers from both QP and non-QP schools: A majority of teachers in both QP schools (57%) and non-QP schools (67%) reported that no trainings were held at all.

Figure 3.7: Quality of training received by teachers in schools surveyed in Vaishali, Bihar

Non-QP

Language

Majority of teachers not trained in preparing and using TLMs: Teachers in 70% of QP schools and 90% of non-QP schools reported below benchmark scores for training in preparing and using TLMs. As there were no separate TLM trainings these findings were not unexpected.

Support for teachers

Local support

To address the manifold challenges teaching (and managing) primary school students in rural areas, teachers require support from the local community, the parent-teachers association (PTA) and the local officials of the state education department. In AP the local school is supported by a school management committee (SMC). Usually all teachers are members, along with a few parents, representatives from the local village Panchayat and is presided over by the Sarpanch. These support linkages for teachers were explored using QPA ordinal questions (see Annexe 3) by asking the parents and members of the communities during a focus group discussion organised at the end of the school visit by the field team and are presented in Figure 3.8.

Figure 3.8: Support to teachers in schools surveyed in Vaishali, Bihar

Non-QP

School

(Teach

School

(Teach

School committees working well, better in non-QP schools: In general, local level support to non-QP schools was better than to QP schools (Figure 3.8). Teachers, however, tended to give higher scores than community members (including parents) for local-level support in terms of the quality of meetings of school organisations.

PTAs disbanded by government order: In Vaishali district, according to a district administration order, PTAs were disbanded when the survey was conducted.

Good support from CRCCs to QP schools but quality of support better in non-QP schools: Most QP school teachers (80%) felt that CRCC provided academic support and monitored their schools, while only 20% of the non-QP teachers thought so. Regarding quality of support, however, 30% of non-QP schools reported above benchmark scores while only 4% of QP schools reported such scores (Figure 3.9). In the worst cases in both QP and non-QP schools, the CRCC does not visit the school, or comes for only about half an hour or so, as a formality to be fulfilled.

Figure 3.9: Academic support to teachers & responses to QP kit in Vaishali, Bihar

Non-QP

Teachers

Community & Parents

Support

support

Academic peer support from other teachers poor: Teachers in around 90% of both QP and non-QP schools gave below benchmark scores for academic peer support from other teachers (Figure 3.9). In most of the cases, however, teachers reported that although they met fairly regularly, only administrative issues were discussed.

Most parents were not aware of the QP kit: In general, QP kits were not supplied in Bihar – even critically required MGML kits given multiple classes in one classroom is a norm in the district were not supplied. In fact, a lot of teachers were unaware of QP kits!

OTHER COMMUNITY PERSPECTIVES

Girl children feel safe coming to school: Community representatives, comprising parents, Panchayat members, and school committee members, had no major or systematic concern about girls‘ safety in coming to school. Similarly, there was little concern about caste-based discrimination in schools. This finding was quite baffling and maybe the community members attending the focus group discussion organised by the field team were not entirely representative.

The major reason for children coming to school was the mid-day meal: In both QP and non-QP schools, teachers felt that the major reason was ‗the mid-day meal scheme‘, followed by ‗to learn‘ (Table 3.5).

Table 3.5: Main reasons for children coming to school, Vaishali Bihar

Mid-day meal 35% 32% 37% 30%

To learn 30% 22% 26% 33%

To play 10% 6% 7% 7%

QP‘s universe was the government primary schools whose demography has changed to include both girls and the disadvantaged over the last decade. It is only to that extent QP addressed the issue of exclusion and there was no special initiative for addressing social exclusion in primary education. Three key issues were assessed: (1) learning achievement in language and mathematics in classes 3 and 5; (2) scores on difficult and easy questions; and (3) performance of SC/ST students. Learning achievement test is in Annexe 6

Class 3 Mathematics and Language: Non-QP students performed better than QP students in language, but the QP students performed better in mathematics. Neither of these differences were statistically significant (Figure 3.10).25 There is no consistent pattern of higher proportions of correct answer for easy (odd numbered questions) and lower proportions for difficult (even numbered questions) across both QP and non-QP schools

Figure 3.10: Language and mathematics achievement of class 3 students in Vaishali, Bihar

Class 3: SC/ST and non-SC/ST students: The sample was disaggregated by socially disadvantaged groups (such as SC and ST) to determine if they were doing better in QP schools despite no attempt by QP to address them specifically. There was no clear trend of worse performance by class 3 SC/ST children in QP and non-QP schools in the language and mathematics tests. In the language tests, they perform slightly better in QP schools and slightly worse in non-QP schools, but these differences were not statistically significant (Figures 3.11 and 3.12).26 Statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

25 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers is not statistically significant (using t-tests assuming equal variances). 26

For the 10 language and mathematics questions, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers is not statistically significant (using t-tests assuming equal variances).

62% 62%

11%13%

9% 10%

78% 78%

15% 14%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Figure 3.11: Language achievement of SC/ST and non-SC/ST class 3 students in Vaishali, Bihar

Figure 3.12: Mathematics achievement of SC/ST and non SC/ST class 3 students in Vaishali, Bihar

Class 5 Mathematics and Language: QP students generally outperform non-QP students, except for question 4 in language and question 1 in mathematics, but these differences were not statistically significant (Figure 3.13).27 A larger proportion of QP and non-QP students correctly answered the easier question in each pair of questions.

Figure 3.13: Language and mathematics achievement of class 5 students in Vaishali, Bihar

27 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers is not statistically significant (using t-tests assuming equal variances), i.e., the probability of error is more than 10% if the difference is asserted.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

45% 44%

62%60%

36% 35%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Class 5 SC/ST and non-SC/ST students: A lower proportion of SC/ST children in QP and non-QP schools get many of the questions correct compared to non-SC/ST children, but these differences are not statistically significant (Figures 3.14 and 3.15).28 Further, statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

Figure 3.14: Language achievement of SC/ST and non-SC/ST class 5 students in Vaishali, Bihar

Figure 3.15: Mathematics achievement of SC/ST and non- SC/ST class 5 students in Vaishali, Bihar

Relevance: In the context of the challenges that exist in most of the districts (including Vaishali) in Bihar, QP was not the ideal initiative for the state. Vaishali district still has to address the basic enrolment, attendance and out-of-school children, and has not been able to focus on quality (aka QP) at all.

If the earlier selected QP district – Nalanda – was not changed to Vaishali, then there was possibly a better chance to focus on issues of quality. This was due to the the legacy of interventions in Nalanda – most notably primary education enhancement project (PEEP)

For the 10 language and mathematics questions, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers is not statistically significant (using t-tests assuming equal variances), i.e., the probability of error is more than 10% if the difference is asserted.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

changed and due to the wastage of QP‘s effort in the district, for instance, even the training of master trainers in Rishi Valley had been done before the QP district was changed to Vaishali.

Effectiveness: Although, UNICEF had well established linkages in the district the focus was on enrolment drives and prohects such as ‗Sankalp‘. Ostensibly, QP was rolled out to the schools (model plus) but the effectiveness of the endeavour was low due to the context of the district and the state.

Efficiency: IMaCS‘ earlier study29 for UNICEF for designing the analytical framework for cost-benefit analysis was stymied by the lack of financial data within UNICEF regarding the cost of QP; programme costs are not easy to disentangle from support costs. As a result, this impact assessment was not able to determine the efficiency of QP. In addition, there were cost implications from moving the QP district from Nalanda to Vaishali, which probably further compounded issues of efficiency.

Sustainability: In Bihar‘s context, the sustainability of QP without (and even with) UNICEF support, does not seem possible.

Impact: A small community of teachers (mostly committed) have been impacted (energised) by the introduction of to some of the QP materials such as TLMs. However, on the whole, the impact of QP in Vaishali has been quite limited.

4. MADHYA PRADESH

Introduction

The chapter begins with an overview of primary school education in Madhya Pradesh (MP). The next section reviews the sample of schools, students and teachers surveyed in MP, followed by the main findings on school and classroom infrastructure, classroom teaching, training and its effectiveness, community perspectives and student‘s learning achievement. The final section presents concluding observations on the findings of the survey.

Background

MP is the most central state and the 2nd largest in India with a total geographical area of 308,000 sq. Km. The state is predominantly rural with 52,143 villages and 313 community development blocks spread over all 48 districts. Three-quarters of the state‘s population live in rural areas with 37% living below poverty line making MP as the 3rd poorest Indian state after Orissa and Bihar. As per the education development index (EDI) devised by National University of Education Planning and Administration (NUEPA), MP was ranked 29 among 35 states.

As per the Annual Status of Education Report (Rural) 2007, Bhind was among the worst ranked district in terms of out of school children and learning levels in class 1-2 and 3-5. As per the District Report Card (DRC), Bhind, 2007, the district had 1756 primary schools and 67 upper primary with primary schools out of which around 95% and 82% were rural schools.

MP has witnessed steady progress in education sector. The state has been able to almost achieve access to primary and upper primary schools. The net enrolment at primary level has increased to 95% and there has been decrease in the out of school children. Despite the gains in enrolment and access, the major challenges are lack of quality education and large disparities in enrolment, attendance and completion among the disadvantaged groups. It is important to note that nearly 35% of the population of the state comprises of SCs (15%) and STs (20%).

The Quality Package in MP

In terms of typologies for QP, the four surveyed districts were differentiated by conceptualisation and implementation of QP on the "initial-intermediate-mature" scale. Bhind was undoubtedly the most backward ward of these districts, although still referred as intermediate.

In MP, the quality education initiative was implemented in collaboration with Rajya Shiksha Kendra. The main purpose is to demonstrate quality learning outcomes and school environment through development and implementation of interventions. It is also hoped that this would provide the model for up scaling in the state for improving quality of education. In MP quality education initiative was implemented in collaboration with Rajya

Shiksha Kendra and was the mainstay of UNICEF‘s support to state government for achieving the objectives set out in SSA well as National Programme for Education of Girls at Elementary Level (NPEGEL). The programme focus was on developing a model for gender sensitive quality education interventions.

In MP, QP was started at the end of year 2005. 25 schools per districts were selected as model schools for implementation of QP from each of the four QP districts Bhind, Guna, Jhabua and Shivpuri. Around 50 teachers from the selected schools were trained at Rishi Valley and after that preparation of tools begun. In MP, however, for nearly a year when the tools were being developed, UNICEF‘s EPO post was vacant, which led to some delays and thus the work on the ground began sometime in August 2005. The focus of QP interventions in Bhind was CE, CP, SE, TLM, TT and AS (in order of priority).

UNICEF was instrumental in introducing the concept of quality standards in the state as well as rolling out Advancement of Education Performance through Teacher Support (ADEPTS) in the state.

Key survey data

Table 4.1: Schools of different types surveyed in Bhind, MP

Schools surveyed

Numbers

QP Non-QP Total

Primary 27 6 33

Total 27 8 35

The main findings on teachers and students are given below.

There are more teachers in non-QP schools: There were only 1.7 teachers per QP school compared to 2.5 in non-QP schools. According to the DRC of Bhind district, 2007, the average number of teachers in primary school was 2.2, thereby placing QP schools below the district average.

Most teachers were male; most male teachers were regular: Overall, 78% of the teachers surveyed across QP and non-QP schools were male (Table 4.2). Only 6% of all regular teachers were female while the same proportion for para teachers was 73%. Further, as the state has decided to let the teachers cadre fade away, future recruitment of teachers will be through the contractual (para teacher) route.

Table 4.2: Teachers in schools surveyed in Bhind, MP

QP Non-QP Total

More students in non-QP schools: There were 89 children enrolled per non-QP school, on average, compared to only 62 children per QP school. However, student per teacher in both QP and non-QP schools was the same (Figure 4.1).

Figure 4.1: Students enrolled in the schools surveyed in Bhind, MP, 2007-08

Unsustainably high student absenteeism: On the day of the survey, an average of 48% of enrolled students in QP schools were absent, compared to 59% in non-QP schools, although there was no systematic variation in absenteeism across classes, or across boys and girls. The unsustainably high level of student absenteeism directly correlates with the extremely low levels of teacher attendance (Table 4.2 & Table 4.3).

Table 4.3: State-wise student absenteeism on day of the school survey in Bhind, MP

QP schools 50% 60% 50% 50% 50% 50% 33% 50% 40% 44% 48%

Non-QP schools 67% 50% 25% 50% 69% 75% 56% 50% 67% 78% 59%

Teachers are not regular: Only 7% of permanent teachers in QP schools and 2% in non-QP schools worked 140 days or more in the last academic year. District Bhind has a famous law and order (dacoit) problem that makes it a district worth avoiding at all costs for the regular teachers (and maybe even the para teachers).

School facilities

Non-QP schools were larger on average, but number of rooms used for teaching was the same as QP schools: On average non-QP schools had 5 rooms while QP schools had 3 rooms. However, on the average, the numbers of rooms used for teaching in both were the same (2) in both QP and non-QP schools (Figure 4.2). So, on balance the surveyed schools had matching infrastructure (in use). The unused rooms in the non-QP schools were essentially being used for storage of assorted supplies, cooking mid-day meal, etc.

Better drinking water and toilet facilities in QP schools: These were assessed both by asking teachers, and subsequently through independent observation using QPA ordinal scoring techniques. Overall, as per the teachers‘ perceptions, a much larger proportion of QP schools had working drinking water facilities and girls‘ toilets (Figure 4.2). In addition, the QPA assessment also showed that the water and sanitation facilities were better in QP

Number of students

per school

Number of students

per teacher

schools than non-QP schools, (Figure 4.3). Nevertheless, the difference was much smaller than what the QP teachers had reported in Figure 4.2.

Figure 4.2: Facilities in schools surveyed in Bhind, MP

Many more QP schools secure and clean: 71% of QP schools were above benchmark, compared to 25% in non-QP schools (Figure 4.3). In the schools scoring below benchmark, classrooms were dirty or without a lockable door, some were being used to store fuel wood and grain, or classes were being held outside in the verandah. In contrast, the classrooms in almost all QP schools were reported to be clean and well-swept.

Figure 4.3: School facilities in QP and non-QP schools in Bhind, MP

Non-QP

More QP schools had recreation and sports facilities: These are better in QP schools (41% above benchmark) compared to non-QP schools (25% above benchmark). In the near and above benchmark cases, space was available as were indoor games such as carom boards, puzzles and snakes-and-ladders, and outdoor games material such as rings,

facilities

QP schools surveyed Non-QP schools surveyed

racquets, footballs, and other balls. Only 1 QP school scored 75 as it had slides, swings, ropes and balls, besides a specific games period during the day, however no sports kit!

Three issues that make important contributions in creating an environment conducive to teaching were assessed: the adequate space, ventilation and lighting, blackboards and seating, and space and books for a reading corner. As in the case of school facilities, the options to assess classroom environment are detailed in Annexe 3.

The majority of QP schools had classrooms with sufficient space, air and light: Around 75% of the QP schools surveyed reported at least the benchmark situation compared to just 43% in non-QP schools (Figure 4.4). Further, two QP schools achieved a perfect score as well.

Figure 4.4: Classroom environment in QP and non-QP schools in Bhind, MP: Class 1

Non-QP

Readin

Most QP schools had adequate blackboards and seating: The availability of blackboards and seating was distinctly better in QP schools, with 74% reporting above benchmark scores compared to 0% in non-QP schools. This is particularly revealing as in terms of inputs, all the schools had durries provided by SSA and in only one QP school sampled, UNICEF had provided chowkis for children of classes 1 and 2. Also, the classrooms in the district were not large enough for introduction of chowkis in an effective manner.

Hardly any ‘reading corner’ facilities in either QP or non-QP schools: More than 90% of both QP and non-QP schools were below benchmark. In the best QP and non-QP schools books were available but were stored in the office cupboard and produced only ‗on demand‘.

As there were multi-grade situations were fairly common among class 1 and 2, the findings of independent assessments in class 1 and class 2 were nearly identical.

Overall, the situation was quite similar to the one prevailing in class 1.

Teaching methods

As QP in Bhind (and MP) was limited to class 1 and class 2 only an independent assessment was conducted in both class 1 and 2. Three issues were explored in the context of the methods of teaching: the use of teaching learning materials (TLMs), self learning materials (SLMs) and the methods used in classroom teaching. These were assessed using the QPA ordinal scoring system (Annexe 3).

TLMs were better used in QP schools: 41% of QP schools surveyed reported above benchmark scores, compared to 13% of non-QP schools (Figure 4.5). In above benchmark QP schools, teachers were using the TLMs; children were aware and familiar with them and could correctly explain their use. In one QP school, the teacher had prepared and was using locally-relevant TLMs in class even scored a perfect score.

Figure 4.5: Classroom teaching in QP and non-QP schools in Bhind, MP: Class 1

Non-QP

SLMs 30 were being used in around half the QP schools: Although SLMs were supplied to QP schools, only 48% of the QP schools reported better than benchmark scores. In the QP schools scoring below benchmark, SLMs were damaged and hence not being used or children were not being guided by the teacher in their use.

(mathematics kit)

Although assessment was conducted in both class 1 and class 2, the findings of the assessment were similar and in a lot of cases, class 1 and 2 were found together in one room.

This section presents findings on teachers trained, type of trainings, quality of training, and suggestions from teachers on how the training could be improved.

Box 4.1: Impressions on teacher training from the ground: Madhya Pradesh

In, MP since 2007, the state government started a new training programme called Advance Education Performance through Teachers Support (ADEPTS), wherein all teachers at the block level received training on different subjects (14 days). These were conducted by trainers who had received a 15-day TOT at the district level, and a ten-day training at state level from officials of the DIET, Education Department, and representatives of the Ministry of HRD, GOI.

According to teachers, attendance of the 14-day block-level summer training, involving all the subjects and given by district-level trainers was compulsorily. However, they felt that the quality of training and arrangements made were not good enough for effective learning.

Figure 4.6: Structure of government training in MP

10 days ToT on

ADEPTS

14 days ToT on

ADEPTS

15days ToT on

ADEPTS

Most teachers received training: All regular male and female teachers in QP schools and all female teachers in non-QP schools received training (Figure 4.7).

Figure 4.7: Proportion of teachers trained in schools surveyed in Bhind, MP

All QP school teachers received training in language but none of the non-QP school teachers did: Apart from language, QP and non-QP teachers had similar statistics with respect to training (Table 4.4)

Table 4.4: Type of training reported in QP schools in Bhind, MP

Mathematics 100% 100%

Use of TLMs 93% 88%

UNICEF (QP) 22% NA

Refreshers 52% 25%

Others 44% 13%

Less than half the QP school teachers and very few non-QP school teachers given training in ‘other’ topics: This reveals that in Bhind, either the role of school teachers as social and other messaging nodes was limited (essentially due to the limited possibility of teacher staying back in the school after hours).

Quality of training

This issue was analysed using QPA ordinal scoring questions and the options to assess are detailed in Annexe 3, and the performance is graphed below (Figure 4.8). It must be noted that even trainings in the same subject were conducted in different batches, by different resource persons, in varying locations.

Language trainings rated better by teachers from QP schools: Teachers from 44% of the QP schools reported benchmark and above scores, compared to 13% in non-QP schools (Figure 4.8). Only in some of the QP schools that scored benchmark and above did

0% 50% 100%

teachers say that TLMs were provided, while in other cases teachers said they had purchased TLMs out of SSA funds.

Mathematics training rated poorly by teachers from non-QP schools: Teachers from 78% of QP schools reported higher-than-benchmark scores, compared to only 13% of non-QP school teachers (Figure 4.8). In some of the QP schools scoring benchmark and above, teachers reported that TLMs were provided by UNICEF or purchased out of school (SSA) funds; refresher training was provided; and they used TLMs in class.

Figure 4.8: Quality of training received by teachers in schools surveyed in Bhind, MP

Non-QP

Language

Training in preparing and using TLMs rated satisfactory by QP school teachers: Teachers in 52% of QP schools rated it above benchmark, compared to just 13% of non-QP school teachers (Figure 4.8). In some QP schools where TLMs had been provided, training was not satisfactory.

SUPPORT FOR TEACHERS

Local support

To address the manifold challenges teaching (and managing) primary school students in rural areas, teachers require support from the local community, the parent-teachers association (PTA) and the local officials of the state education department. These support linkages for teachers were explored using QPA ordinal questions (see Annexe 3) by asking the parents and members of the communities during a focus group discussion organised at the end of the school visit by the field team and are presented in Figure 4.9.

Poor support from school committees: In general, local level support to QP schools was better than to non-QP schools with 25% of QP schools scoring above benchmark while none of the non-QP schools did so (Figure 4.9). Teachers, however, tended to give higher scores than community members (including parents) for local-level support in terms of the quality of meetings of school organisations. In the majority of other cases, meetings were fixed regularly (even if only to mark it in the register), but only 2-3 members attend, and discuss mid-day meal and financial issues, not the quality of teaching.

Figure 4.9: Support to teachers in schools surveyed in Bhind, MP

Non-QP

School

(Teach

School

(Teach

Better support from PTAs of QP Schools: Distinctly stronger local support was reported from PTAs of QP schools compared to non-QP schools, with 62% of QP schools scoring above benchmark and no non-QP school doing so (Figure 4.9). In the schools scoring below benchmark, the PTA either did not meet or only came on special days (15th August and 26th January) when food was served in the school. Overall the significant difference in level of support given to QP (as compared to non-QP schools) can be directly attributed to how QP assisted in building capacities of PTAs in developing and implementing school development plans as per the Peoples Education Act formulated by the state. UNICEF developed a set of three training modules for capacity building of PTAs on their roles and responsibilities and trained over 2000 PTAs in 4 UNICEF supported districts (Bhind, Guna, Jhabua and Shivpuri).

Figure 4.10: Academic support to teachers & responses to QP kit in Bhind, MP

Non-QP

Teachers

Community & Parents

Support

support

Poor CRCC support to schools: Teachers from 37% of QP schools and only 1 non-QP school (13% of sample) reported benchmark-and-above scores (Figure 4.10). Teachers in two non-QP schools said that the CRC did not pay regular visits or came only as a ‗postman‘ (to deliver circulars from the education department), while those in the remaining non-QP schools said that the CRC visited once a month but only for around half an hour, and only rarely did they (mostly ‗he‘) observe classes and gave feedback to the teachers.

No academic peer support from other teachers due to government order: Peer-to-peer support was non-existent in all QP and non-QP schools as the teachers reported that all such meetings had been stopped following a government order from the education commissioner in October 2007.

Most parents unaware of QP kits: Over 90% of parents were unaware of this kit; only in two schools were parents aware or had seen the TLMs.

Girl children were mostly absent; parents felt safe sending them to school: Overall, no systematic worries about girls‘ safety or caste-based discrimination in schools were reported in the focus group discussion organised by the field team in the village(s). However, the dismal absenteeism rate among girls (compared to boys) implies that safety is a big concern in reality.

The QP kit was not the major reason for children coming to a QP school: In both QP and non-QP schools, teachers felt that the major reason was ‗to learn‘, followed by ‗the mid-day meal scheme‘, and ‗new learning material‘ (Table 4.5).

Table 4.5: Main reasons for children coming to school in Bhind, MP

Mid-day meal 23% 14% 29% 0%

To learn 36% 43% 37% 100%

To play 8% 14% 17% 0%

Class 3 Mathematics and Language: There was no statistically significant difference between the performance of class 3 children in the language and mathematics tests conducted in QP schools and non-QP schools in MP (Figure 4.11).31 The pattern of higher proportions of correct answer for easy (odd numbered questions) and lower proportions for difficult (even numbered questions) was apparent for most of the pairs of questions asked (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10).

Figure 4.11: Language and mathematics achievement of class 3 students in Bhind, MP

Class 3: SC/ST and non-SC/ST students: The sample was disaggregated by socially disadvantaged groups (such as SC and ST) to determine if they were doing better in QP schools despite no attempt by QP to address them specifically. There was no statistically significant trend of worse performance by class 3 SC/ST children in QP and non-QP schools in the language and mathematics tests (Figures 4.12 and 4.13).32. In the language tests in QP schools, however, SC/ST students gave a higher proportion of wrong answers to all questions except question 1, and indeed when this question is removed from the analysis, there was a statistically significant difference between SC/ST and non-SC/ST students performance. Statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

31 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers is not statistically significant (using t-tests assuming equal variances). 32

For the 10 language and mathematics questions, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers is not statistically significant (using t-tests assuming equal variances). However, if the language question 1 is removed from the analysis, the differences between SC/ST and non-SC/ST test results for class 3 language are statistically significant at 93% confidence level (using a t-test assuming equal variances).

50%48%

20%18% 18%

58% 58%

13%15%

50%48%

20%18% 18%

58% 58%

13%15%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Figure 4.12: Language achievement of SC/ST and non-SC/ST class 3 students in Bhind, MP

Figure 4.13: Mathematics achievement of SC/ST and non SC/ST class 3 students in Bhind, MP

Class 5 Mathematics and Language: Non-QP students generally outperformed QP students in class 5 language, while the performance was mixed in mathematics. However, once again these differences were not statistically significant (Figure 4.14).33 Also, a large proportion of students in QP and non-QP schools correctly answered the easier question in each pair of questions (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10).

Figure 4.14: Language and mathematics achievement of class 5 students in Bhind, MP

33 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers were not statistically significant (using t-tests assuming equal variances).

44% 40% 37% 31% 52% 44% 83% 76% 42% 24%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

40%37%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Class 5 SC/ST and non-SC/ST students: A lower proportion of SC/ST children in QP and non-QP schools answered the questions correct compared to non-SC/ST children, but these differences were not statistically significant (Figures 4.15 and 4.16).34 Statistically the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP

Figure 4.15: Language achievement of SC/ST and non-SC/ST class 5 students in Bhind, MP

Figure 4.16: Mathematics achievement of SC/ST and non- SC/ST class 5 students in Bhind, MP

Relevance: In MP, four QP districts Bhind, Guna, Jhabua and Shivpuri were selected for QP and work on the ground began sometime in August 2005.The focus of QP interventions in Bhind was CE, CP, SE, TLM, TT and AS (in order of priority), however, as the time elapsed since implementation of QP began was too small, Bhind district has been tagged as ‗initial‘.

The implementation of QP was also affected by the larger order governance changes at the states (and education department in particular). In Bhind, the issues of law and order and the availability of teachers (very high absenteeism rate) make the implementation of

For the 10 language and mathematics questions, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers is not statistically significant (using t-tests assuming equal variances.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

any initiative difficult. In some QP schools, as argued across this document, some teachers were able to use the QP materials as a springboard to innovate and change things for the better – may be in some cases to above the ‗effective‘ level.

Effectiveness: The effectiveness of QP was limited by primarily two factors: First, the absence of UNICEF‘s EPO during the critical time of drafting of materials delayed the implementation, for instance, even in 2007 workbooks developed by DIETs (Indore and Dhar) have not been field-tested or printed! Second, the level of administrative turnover in the district and the state was very high resulting in low capacity to understand and implement QP.

Sustainability: Given the level of maturity and the fragmentation witnessed during implementation, the sustainability of QP without UNICEF support, is not possible. QP itself has not yet demonstrated its full potential in a cost-effective manner; however, various dimensions of the package such as the MGML have been appreciated by SSA.

Impact: QP demonstrated a new approach to education to both teachers and students. However, QP‘s impact has been limited because of feeble implementation; particularly due to its lack of ability to engage teachers effectively. Some other impacts included:

In MP (and Bhind) due to the 2002 Act regarding PTA‘s, there has some work done on involving communities in affairs of the primary schools. This has also been seized by QP and due to UNICEF‘s intervention, a phase wise capacity building of PTAs has been initiated instead of the ―one-time training‖.

Further, UNICEF was instrumental in introducing the concept of quality standards in the state as well as rolling out ADEPTS in the state.

5. ORISSA

Introduction

The chapter begins with an overview of primary school education in Orissa. The next section reviews the sample of schools, students and teachers surveyed in Orissa, followed by the main findings on school and classroom infrastructure, classroom teaching, training and its effectiveness, community perspectives and student‘s learning achievement. The final section presents concluding observations on the findings of the survey.

Background

Since 2000, Orissa has experienced stable political environment and the per capita income has increased almost four-fold in last 14 years, from Rs 1,325 in 1992 to Rs 5,067 in 2006. However, there are disparities within the state with the coastal areas being generally more developed. Poverty is more prevalent in rural areas (48%) than in urban areas (43%). It is estimated that 47% of the state‘s 38 million people live below poverty line. Poverty among the SC and especially the ST population is strikingly higher than among other population groups. ST population represents 22% of the population but constitute more than 40% of the poor.

As per the education development index (EDI) devised by National University of Education Planning and Administration (NUEPA), Orissa was ranked 26 among 35 states. As per the Annual Status of Education Report (Rural) 2007, Ganjam district was better than the district state average in terms of out of school children and learning levels in class 1-2 and 3-5. As per the District Report Card (DRC), Ganjam, 2007, the district had 2281 primary schools and 779 upper primary with primary schools out of which over 91% and around 95% were rural schools.

The state government has, in the last 5 years, progressed towards meeting the objectives of the National Policy on Education and the recent constitutional amendment to make education a fundamental right through the SSA, a flagship programme of the Government. While the gross enrolment rate for grade I-V has increased from 109% to 117%, the net enrolment rate has gone up from 86 % to 94 % with a transition rate from primary to upper primary of 76 to 82%. During the 2005-06, there has been a decline in gender gap with Gender Parity Index (GPI) of 0.92 in 2003-04 to 0.93 in 2005-06 at the primary level. The literacy rate is 64%. Social group-wise, literacy rate is lowest among scheduled tribes (18%) and scheduled castes (30%) than in general population. Sex-wise literacy rate is 52% in adult male population as compared to 29% among adult women. Both male and female literacy rates are lowest in southern region of the state. There is also significant disparity between rural and urban Orissa.

The Quality Package in Orissa

In terms of typologies for QP, the four surveyed districts were differentiated by conceptualisation and implementation of QP on the "initial-intermediate-mature" scale.

Ganjam was undoubtedly among the better of these districts, clearly on the intermediate-level on the above mentioned scale.

QP implementation in Orissa started in 2005, but in earnest only in 2006. The SCERT, supported by UNICEF had taken a lead in the QP process. However, OPEPA, and through them, the SSA, a critical link in the potential replication process was involved to a much lesser degree for the want of facilitation of coordination between these agencies. This has led to a situation where the state has not been able to leverage the gains it has made in during QP pilot implementation.

The focus of QP interventions in Ganjam was inside the classroom –TT, CE, SE, CP and TLM (in order of priority). Further, the project also focussed on learning continuum resource and particularly in enhancing child participation, the programme helped in giving a voice to children to provide regular feedback to the district administration, notably TSC and SSA, through a child reporter‘s blog site.

With respect to the situation in Ganjam, it was apparent that the success of QP owed a lot to the efforts of the quality monitor (funded by UNICEF in the district) and the reigning DPC (SSA) in the district. As such, if QP implementation is thus dependent on such personality driven leadership, and if implementation is to be replicated, a leadership development process needs to be initiated to lead to improve chances of successful implementation.

QP was scaled-up under SSA in Koraput, Ganjam & Puri in all blocks in the 2007-08 academic session demonstrating government and policy interest in ownership of the approach and strategy.

Key survey data

Of the 35 rural schools surveyed, 25 were QP schools and 10 were non-QP schools (Table 5.1).

Table 5.1: Schools of different types surveyed in Ganjam, Orissa

Schools surveyed

Numbers

QP Non-QP Total

Primary 6 1 7

Total 25 10 35

There are more teachers in non-QP schools: On average, there were 6.2 teachers per QP school compared to 6.9 teachers per non-QP school, which is much more than the average for the district as per DRC, 2007: 2.7 for primary and 5.6 for primary with upper primary schools

Around two-thirds of teachers were regular; and of them over two-third were male: Overall, the proportion of regular to para teachers is 2:1, which is higher than all QP districts surveyed. Further, around half of the para teachers were female as opposed to

one- third of the regular teachers. The recruitment policy for para teachers seems to be more gender-neutral36 (Table 5.2).

Table 5.2: Teachers in schools surveyed in Ganjam, Orissa

QP Non-QP Total

Very few teachers are regular, more in QP schools: Only 16% of teachers in the QP schools surveyed worked 140 days or more in the last academic year, compared to 8% of teachers in non-QP schools. But nearly 50% of non-permanent teachers, however, had attended regularly in QP schools, although this proportion was only 14% in non-QP schools.

Students per teacher and per school were a bit lower in QP schools: QP schools had slightly less children per teacher (23) compared to QP schools (26). This compared quite favourably with the district average student per teacher ratio (DRC, 2007) of 39 in primary and 50 in primary with upper primary schools. However, changes in enrolment from 2007 to 2008, however, were not uniform across classes or across boys and girls, in QP or non-QP schools.

Falling student enrolment in QP and non-QP schools: The number of children in all classes enrolled in the 25 QP schools during 2007-8 was 5% lower than in 2006-07, while it was 13% lower in the case of non-QP schools surveyed.

High student absenteeism: On the day of the survey, an average of 22% of enrolled students in QP schools was absent, compared to 29% in non-QP schools. There were no absentees among girls in QP schools in classes 1, 2 and 4, and consistently less than non-QP schools. This probably indicates that the gender sensitive components of QP were rolled out properly in the district.

Table 5.3: Student absenteeism on day of the school survey in Ganjam, Orissa

QP schools 33% 0% 33% 0% 39% 24% 27% 0% 35% 26% 22%

Non-QP schools 33% 33% 29% 32% 32% 35% 29% 24% 21% 26% 29%

School facilities

Fairly large schools, but only half the rooms used for teaching: Most, schools had 9 – 10 rooms, on average, of which only 5-6 were being used for teaching (Figure 5.1). Not only does this indicate that the surveyed schools had lesser students per classroom than the norm, it also indicates a less than effective usage of the space by the teachers (and head teachers in particular).

One of the reasons for this are that para teachers, informally or formally, are helpers (with lower qualifications

than the regular teachers)

Figure 5.1: Teaching and non-teaching rooms in surveyed schools in Ganjam, Orissa

facilities

QP schools surveyed Non QP schools surveyed

Better drinking water and toilet facilities in QP schools: These issues were assessed both by asking teachers, and subsequently through independent observation (using ordinal questions, see below). Teachers over-estimated the available facilities but this was corroborated by ordinal scoring methodology, which showed that 50% of QP schools scored above the benchmark, compared to 20% of non-QP schools (Figure 5.2). In the above benchmark schools, which included only 2 non-QP schools, there were several water points, separate, clean and functional toilets for girls that were available and being used by children, and places for children to wash their hands.

Figure 5.2: School facilities in QP and non-QP schools in Ganjam, Orissa

Non-QP

Security and cleanliness were good in most schools: 84% of QP schools scored above benchmark, compared to 80% in non-QP schools (Figure 5.2). In the above benchmark schools, which were the majority, the interiors and exterior were kept clean and tidy, doors are in working condition, there is a good boundary wall with a lockable gate. Some of the best schools had fans and flowering gardens, done with or without the help of the local community.

Recreation and sports facilities in QP schools were much better than non-QP schools: 44% of QP schools scored above the benchmark compared to only 20% of non-QP schools (Figure 5.2). In the above benchmark schools, there was both space for playgrounds, sports materials and the best ones also had specific games periods. Footballs, volleyballs, skipping ropes, swings and slides were the common outdoor games material, while carom and toys were available for indoor games.

Three issues that make important contributions in creating an environment conducive to teaching were assessed: the adequate space, ventilation and lighting, blackboards and seating, and space and books for a reading corner. As in the case of school facilities, the options to assess classroom environment are detailed in Annexe 3.

Most schools had large, well-ventilated classrooms: Overall, 70-75% of the schools surveyed had had sufficient space, ventilation and lighting – often exceeding the norms of 6 square feet per child and two windows per room (Figure 5.3). QP schools perform slightly better than non-QP schools, with 75% reporting at least the benchmark situation, compared to 70% of non-QP schools.

Figure 5.3: Classroom environment in QP and non-QP schools in Ganjam, Orissa: Class 1

Non-QP

Readin

Most schools have functional blackboards but children generally sat on the floor: QP schools perform better than non-QP schools, with 48% reporting above benchmark scores compared to 10% in non-QP schools. In general, children sat on floors, with or without mats, and in most QP schools they are using the table supplied by UNICEF. However, by the introduction of tables in these schools, the rooms were overcrowded with insufficient space for each child!

No ‘reading corner’ but reading habits were cultivated in some QP schools: Almost all QP schools reported having books, but no space to keep them. They were usually kept in

almirahs in the school office, a register is maintained and children issue books to take home to read. Only a few schools had adequate space for a reading corner. Thus, even without the physical space, the reading habit was being cultivated in QP schools.

As there were sufficient and often separate classrooms for class 1 and class 2, a separate assessment of class 2 was also conducted. However, in general the findings mirror the findings of class 1.

TEACHING METHODS

QP interventions in Orissa were also limited to class 1 and 2. In this context, assessment was conducted in class 1 and 2. Three issues were explored in the context of the methods of teaching: the use of teaching learning materials (TLMs), self learning materials (SLMs) and the methods used in classroom teaching. As in the previous section, these were assessed using the QPA ordinal scoring system (See Annexe 3)

Use of TLMs was much better in QP schools: The use of TLMs in class I was better in the QP schools surveyed, since 80% of schools are above benchmark compared to 40% in non-QP schools (Figure 5.4). In non-QP schools, however, 20% of schools were in the ‗worst case‘ situation, i.e., ‗TLMs not displayed or visible and children do not know about TLMs‘, and no school has a score above the benchmark. In the below benchmark QP schools, TLMs were present in the school but were either not being used in class or had not been used sufficiently by the teacher for the children to correctly explain them on their own.

Figure 5.4: Classroom teaching in QP and non-QP schools in Ganjam, Orissa: Class 1

Non-QP

Worst case Below benchmark Benchmark to Excellent

Excellent to Ideal Ideal case

SLMs37 widely used in QP schools: 84% of QP schools scored above the benchmark. In the below benchmark cases, SLMs were either not present or damaged or teachers were reluctant to use them.

A separate assessment of teaching methods in class 2 was also conducted. However, in general the findings mirror the findings of class 1.

This section presents findings on teachers trained, type of trainings, quality of training, and suggestions from teachers on how the training could be improved. It must be noted that even trainings in the same subject are conducted in different batches, by different resource persons, in varying locations.

Box 5.1: Impressions on Teacher Training from the ground: Ganjam, Orissa

Trainings were being conducted both at block and district level. District level trainings were meant for mostly for teacher trainers, while block level trainings were for teachers. The Orissa Primary Education Programme Authority (OPEPA) stipulates 20 days training program for teachers under SSA. In practice, however, the 10 monthly meetings at the CRC were also accepted as ‗training days‘, and so teachers need to undergo a formal 10-day training to satisfy the OPEPA stipulation. A teacher who has undergone one particular training cannot go for the same training next year, but can go for the next level of trainings (like UNMESH- I, UNMESH-II, English, UDAYA-I, UDAYA-II, etc.). The 20-days training were compulsory for all regular teachers for every year but not for para-teachers. The QP provides an additional 7 days of training for Class I and Class II teachers, the only two classes for which QP is provided currently. Out of this 5 days are for 'zero mile stone' training and two days are for mathematics and self learning. From 2008, a training program for class III teachers has started, but it is being designed for 5 days (3 + 2).

Around half the permanent teachers received trainings, only in QP schools: Despite, the emphasis on training, only 50% of the regular teachers and 22% of para teachers in QP schools in Ganjam received training, while none of the teachers in the non-QP schools had gone for training (Figure 5.5). This may reflect the gap in coordination between the state SCERT (nodal agency for UNICEF‘s QP and for training) and OPEPA, the nodal agency for programme implementation of SSA in Orissa.

(mathematics kit)

Figure 5.5: Proportion of teachers trained in schools surveyed in Ganjam, Orissa

Most QP school teachers received training in language and mathematics: A larger proportion of QP school teachers received training in language and literacy and mathematics (Table 5.4).

Table 5.4: Type of training reported in QP schools in Ganjam, Orissa

Mathematics 76% 40%

Use of TLMs 4% 0%

UNICEF (QP) 88% NA

Refreshers 8% 10%

Very few QP school teachers and no non-QP school teacher received training in using TLMs: Only teachers from 1 QP school (4% of sample) reported receiving training in preparing and using TLMs (Table 5.4).

Almost all teachers given training in ‘other’ topics: All teachers who had gone for training received training in ‗other‘ subjects, including sanitation, wall painting, library creation and maintenance, women‘s rights, adolescent girls, life skills, environment, health, education of handicapped, puppetry, paper crafts and anaemia. These trainings were designed to use school teachers as socio-economic vectors in rural communities.

Quality of training

This issue was analysed using QPA ordinal scoring questions and while the options used are detailed in Annexe 3, the performance is graphed below (Figure 5.6). It must be noted that even trainings in the same subject were conducted in different batches, by different resource persons, in varying locations.

0% 50% 100%

Figure 5.6: Quality of training received by teachers in schools surveyed in Ganjam, Orissa

Non-QP

Language

Language trainings quality rated satisfactory by less than half of QP school teachers: Teachers from 40% of QP schools reporting benchmark and above scores, compared to only 10% in non-QP schools (Figure 5.9). In the lowest scoring cases, teachers said that trainings were not properly organised, the resource persons were not up to the mark.

Mathematics training rated satisfactory by around half the QP school teachers: Teachers in 52% of QP schools scored these above benchmark compared to just 10% of teachers in non-QP schools. Schools scoring zero were those where teachers had not received training.

TLM training not satisfactory: Trainings in the use of TLMs were rated above benchmark by teachers in only 24% of QP schools and 10% of non-QP schools. The poor performance is largely because there were no separate trainings on TLMs. In all below benchmark cases, teachers mentioned that some training was given at the cluster level, and in these meetings teachers share their knowledge on TLMS and their use, but that this was not sufficient and no TLMs are provided. Many teachers reported buying TLMs out of SSA funds.

SUPPORT FOR TEACHERS

Local support

To address the manifold challenges teaching (and managing) primary school students in rural areas, teachers require support from the local community, the parent-teachers association (PTA) and the local officials of the state education department. Usually all teachers are members, along with a few parents, and representatives from the local village Panchayat. These committees take day-to-day management decisions, and decisions on larger issues are either taken for approval to the General Body (GB) Meeting of the larger group of parents and community, or discussed in the latter forum. These support linkages for teachers were explored using QPA ordinal questions, which are detailed in Annexe 3, and graphed in Figure 5.7.

Figure 5.7: Support to teachers in schools surveyed in Ganjam, Orissa

Non-QP

School

(Teach

School

(Teach

Good support from school committees in general: Local level support to QP and non-QP schools good and was almost the same (Figure 5.7). Surprisingly, the community tended to give higher scores than teachers for local-level support in terms of the quality of meetings of school organisations. In the majority of cases in both QP and non-QP schools, the school committees were active, meetings are regular and well-attended, school performance on the quality of education is reviewed and monitored and decisions are taken to develop the school. The key issues discussed even in non-QP schools were the mid-day meal situation, sanitation, separate library room, land for the school, as well as financial and human resources for school development.

PTAs not very active, better in QP schools: A majority of schools (68% QP and 60% non-QP) score above benchmark, as PTA meetings were held regularly and well attended, decisions were taken to influence the VEC, parents are conscious and interested in school affairs and monitor and discuss issues such as attendance of children, sanitary facilities and academic performance.

Good support from cluster resource centre coordinators: In general, the CRCCs were visiting both QP and non-QP schools and providing support to teachers (Figure 5.8). Further, the same proportion of QP and non-QP schools were in the ‗excellent to ideal‘ category. However, the nature of academic support was more administrative in nature.

Academic peer support from other teachers much better in non-QP schools: Teachers in 90% of QP schools reporting above benchmark scores, compared to 79% in non-QP schools (Figure 5.8). All schools reported regular cluster level meetings, and the lowest scores were only because practical and hands-on demonstrations were not being provided. In general, academic sharing was good and appreciated as being useful by teachers.

Poor response by parents to QP kits: Around 80% of parents were unaware of these kits (Figure 5.8). Teachers‘ perception of the extent of this awareness was roughly the same as that of the community and parents.

Figure 5.8: Academic support to teachers & responses to QP kit in Ganjam, Orissa

Non-QP

Teachers

Community & Parents

Support

support

Girl children felt safe coming to school: Contrary to the general perception, neither the parents nor the teachers professed any systematic or major concern about girls‘ safety in coming to school or caste-based discrimination in schools in the focus group discussion organised by the field team in the village(s).

The QP kit was not the major reason for children coming to school: In both QP and non-QP schools, teachers felt that the major reason was ‗to learn‘ (Table 5.5).

Table 5.5: Main reasons for children coming to school in Ganjam, Orissa

Mid-day meal 24% 18% 25% 17%

To learn 27% 32% 36% 33%

To play 16% 11% 14% 17%

Class 3 Mathematics and Language: QP school children did significantly better in language than non-QP school children, while there was no statistically discernable difference between the performance in the mathematics test between QP and non-QP schools children (Figure 5.9).38

Figure 5.9: Language and mathematics achievement of class 3 students in Ganjam, Orissa

The pattern of higher proportions of correct answer for easy (odd numbered questions) and lower proportions for difficult (even numbered questions) was not very clear in each pair of questions (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10). In the language test, for example, the pattern was broken for question pairs 1 & 2 and 7 & 8 in QP schools and for questions 1 & 2 for non-QP schools. For the mathematics test, the pattern was broken in question pairs 3 & 4 and 9 & 10 in both QP and non-QP schools.

Class 3: SC/ST and non-SC/ST students: The sample was disaggregated by socially disadvantaged groups (such as SC and ST) to determine if they were doing better in QP schools despite no attempt by QP to address them specifically. In QP schools, SC/ST students generally performed poorer and this difference was even statistically significant in the case of mathematics (Figures 5.10 and 5.11).39 However, statistically the sample

38 For the 10 mathematics test questions, the difference across QP and non-QP schools in the proportions of correct answers was not statistically significant at the 90% confidence level (using a t-test assuming equal variances). But this difference was statistically significant for the language test results, with a less than 2% chance of error (with t-tests conducted assuming both equal and unequal variances). 39

For the 10 language and mathematics questions in non-QP schools and language questions in QP schools, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers was not statistically significant (using a t-test assuming equal

83%86%

43%45%

31% 31%

54% 55%

21% 20%

22% 22%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

56% 56%

13%15%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

Figure 5.10: Language achievement of SC/ST and non-SC/ST class 3 students in Ganjam, Orissa

Figure 5.11: Mathematics achievement of SC/ST and non SC/ST class 3 students in Ganjam, Orissa

Class 5 Mathematics and Language: QP students generally outperformed non-QP students, except for question 4 in language and question 1 in mathematics, but these differences were not statistically significant (Figure 5.12).40 Again, except for a few exceptions, a larger proportion of QP and non-QP students correctly answered the easier question in each pair of questions (e.g., questions 3 & 4, 5 & 6, 7 & 8, and 9 & 10). The pattern is broken in question pairs 1 & 2 in language, and 5 & 6 in QP schools in mathematics.

variances). The difference was, however, statistically significant (with less than 5% chance of error) in the case of mathematics achievement in QP schools (using a t-test assuming equal variances). 40 For the 10 language and mathematics questions, the difference across QP and non-QP schools in the proportions of correct answers was not statistically significant (using a t-test assuming equal variances).

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Figure 5.12: Language and mathematics achievement of class 5 students in Ganjam, Orissa

Class 5 SC/ST and non-SC/ST students: A lower proportion of SC/ST children in QP and non-QP schools answered many of the language questions correctly compared to non-SC/ST children, but these differences were not statistically significant (Figures 5.13 and 5.14).41 Figure 5.13: Language achievement of SC/ST and non-SC/ST class 5 students in Ganjam, Orissa

However, SC/ST students performed distinctly poorer in the mathematics test for class 5, in both QP and non-QP schools. Moreover, this difference was statistically significant (Figure 5.14). 42 Statistically, however, the sample was insufficient to compare whether SC/ST did better under QP schools. Even through qualitative information and discussions at the field-level, it was not apparent whether SC/ST students would do better under QP.

For the 10 language questions, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers was not statistically significant (using a t-test assuming equal variances. 42

For the 10 mathematics questions, however, the difference across SC/ST and non-SC/ST children in QP and non-QP schools, in terms of proportions of correct answers was statistically significant (using a t-test assuming equal variances).

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

39% 39%

31%29%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

49%51%

19% 19%

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Figure 5.14: Mathematics achievement of SC/ST and non- SC/ST class 5 students in Ganjam, Orissa

Relevance: In Ganjam, given earlier intervention by UNICEF and other developmental interventions on primary education, QP was perceived as the corrective to the predicament of the district where children were coming to school but were not learning. The focus of QP interventions in Ganjam was inside the classroom –TT, CE, SE, CP and TLM (in order of priority). Further, the project also focussed on learning continuum resource (aka ladder of learning) and particularly in enhancing child participation.

Ganjam was also able to benefit from the classroom infrastructure provided by UNICEF due to the low student teacher/classroom ratios that prevail in the district. However, on the whole, given the facilitative environment that existed in the district, QP has not as yet become mature as an initiative as the inputs provided to the teacher have not been used or monitored in a supportive manner.

Effectiveness: UNICEF has well established linkages in the district to phase the implementation of QP effectively. However, the effectiveness of QP was affected by the failings of a fragmented and fitful implementation framework and planning. Responding to demand for introduction of QP tools into other primary classes (3-5) from a significant proportion of teachers and the community, QP in Ganjam has tried to develop and introduce QP materials in class 3-5.In time this factor alone, may enhance the effectiveness (and impact) of the initiative.

Sustainability: Given the fact that QP has scaled-up under SSA in Koraput, Ganjam & Puri in all blocks in the 2007-08 academic session demonstrates government‘s interest as well as ownership of QP‘s paradigm – Both approach and strategy.

Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10

Impact: QP demonstrated a new approach to education to both teachers and students, so much so that there is a demand for it to be introduced into higher classes. QP has had an impact (on SSA) on the policy level, and also at the school level (class 3 language).

6. Stakeholder Assessment

Introduction

The stakeholder assessment was carried out separately from the school assessment. The main focus was to understand from key stakeholders how UNICEF‘s approach to quality had affected the discourse (and implementation) of quality education initiatives in the state/ district across different platforms (including private schools).

How sound is the QP conceptualisation?

One of the key successes of QP was the production of higher quality of textual (and TLMs) materials. The graded readers can be chalked out as one of QP‘s significant successes. This has also successfully migrated to the SSA domain, however, there is some concerns whether the readers developed by the system are actually ―graded‖.

One of the key failures of the conceptualisation of the QP programme has been the development of the system for academic support. Although, given the lack of professional system (or ladder) for the teachers, the idea of academic support is appealing; however, the same lack of professionalism becomes a roadblock because the CRCs and BRCs are from the same stock (or in the same soup) as the teachers (trainees).

The second failure, not of conceptualisation, but of implementation has been the nearly ubiquitous roll-out of the QP initiative without engaging the community. The absence of parents (community) robs QP of critical (differentiating) support which can often help in making a particular initiative successful, provided it is rooted in the community (bottom-up). This was true in all states except in Bihar which despite (or because) being on the initial rungs of ensuring enrolment and attendance on the QP ladder.

How’s QP doing? Has it helped improving quality? Six dimensions of QP?

Overall, from analysis of stakeholder‘s reflections about QP, it is apparent that by pioneering the materialisation of quality in terms of six separate dimensions, UNICEF has changed the policy and programme context in how quality is being addressed in the field. This was true in both comparatively advanced (AP and Orissa) and lagging QP states (MP and Bihar). However, as there were different degrees of difficulties in implementing the different dimensions, there was a noticeable focus on infrastructural elements such as TLMs, CE and SE (in order of priority). Generally, across all the states the other three dimensions were conversely neglected as follows TT, AS and CP.

TLMs were developed multiple times at different times, and wherever used optimally they did result in making the classroom more inclusive and participatory. However, as the school assessment bears out their usage (inferred or observed) reveals that teachers were not really conversant with them and use them quite mechanically with a coupled tendency to repeat them again and again. Therefore, their usage was quite low and the (negative) correlation between their use and performance of teachers (through student‘s learning levels) held.

How do teachers and concerned administrators regard the QP initiative?

According to the stakeholders consulted, it was quite easy to blame the teachers for the failure of the system and reformist offshoots such as Janshala and even QP. However, this was not true as the number of constraints that teachers have to transcend in daily matters, especially in conforming to administrative fiats (and forms)

The teachers interviewed during the school assessment, were attracted to QP as a person (even a parent) but not necessarily as a teacher. Obviously, there were many exceptions that regard QP as something that was required (and have even practised some of QP‘s ways in their classroom on their own stead)

As a result the most attractive part of the package, for a majority of them was worksheets and stationery supplies (paper and coloured pencils). This reveals that the teacher identified with these student learning supplies as they made their job a little bit easier. However, this alone was probably not as per QP‘s purpose, design or intent.

TLMs were often found more stored rather than used in the classroom. As a rule of thumb in classrooms where the benchmarks for space were not maintained, TLMs end–up being a burden and wherever the teachers tried to use (demonstrate) them they became more time consuming. This experience can be inverted for classrooms where there was adequate place (to use TLM‘s optimally) such as in AP where enrolments are generally low.

In classrooms, where TLMs were ‗hanged‘ (and not changed routinely) to create an appearance of child-friendly environment, TLMs can be labelled more accurately as clutter as it became difficult to see the class from end-to-end.

What role have the CRCs/BRCs played?

The BRCs/CRCs were largely beholden to administrative affairs and much less to providing academic guidance. In effect, as per comments of some stakeholders, the BRCs/CRCs have become ―black holes‖, i.e. although trained to facilitate a plethora of times; they have not been successful as master-trainers or become good facilitators.

As alluded to earlier, given their bearings (as teachers), they were seldom able to provide concurrent M&E and moreover their reports did not seem to be used to guide the system but to fulfil the requirements of someone else‘s MIS (even DISE).

Further, it seems that as CRCs/BRCs were chosen by consent by the teacher body, often some of the best teachers (leaders) became CRCs/BRCs at cost to themselves (administrative) and the students (learning).

QP vis-à-vis SSA training

As the prime agency for QP training has been Rishi Valley (Chitoor), there has been convergence among different states/districts in (selecting and) sending their master trainers (teachers) there to ensure a swift start to QP initiatives. However, in Bihar this trip to Rishi Valley ended up being wasted due to multiple reasons: 1) lack of material support to the resource group after they came back from training); 2) transfer of the QP district from Nalanda to Vaishali; and 3) inability to transcend the logistical impediments of meeting as the resource group was from different blocks of the district without a common meeting place (At present, attempts to resuscitate DIETs are being made)

Some states such as Orissa have redesigned the earlier 3-day QP training into a 3+5 day programme, where the first 3 days are used to orient teachers in creating an environment to make children comfortable with QP processes — and attend (and learn?) the school more regularly. The remaining 5 days are used for general orientation, pedagogy, improvements in learning environment, techniques for math, language and environmental science.

Learning achievement of children?

The QP schools across the 13 states, and in particular, across the four states of AP, Bihar, MP and Orissa, inhabit a varied typology in space, time, community and past histories (earlier educational projects). This complexity has been taken into account in the sampling undertaken by the school assessment, however, due this differing landscape, QP is best assessed as a comparison between the states/districts own benchmarks instead of any comparative yardsticks.

Echoing these concerns, as apparent from the school assessment, the record of QP is mixed. For instance, QP schools perform better in AP and Bihar for Class 3, while non-QP schools perform better in Orissa and MP. In Class 5, children from non-QP schools in MP and AP do better than those in QP schools in MP and Orissa.

Further, there was a general and perceptible (not significant) difference between scores of SC/ST and non-SC/ST children in all classes, all states, and for all subjects. Although, for the QPA conducted in four states, the samples for QP and non-QP schools were different (35 Vs 10), there seems to be no indication that SC/ST (and other excluded) groups do marginally better under QP.

What is the overall impact of the QP?

Due to UNICEF‘s initiative, quality of primary education has become essential (no longer ―just desirable‖) to both the discourse and the implementation of primary education in the country and state. However, at the district (instrumentally relevant administrative unit) the relevance of focussing on quality as the only way to achieve UEE is still required.

The MGML aspects of QP have been more readily adopted by SSA. However, innovations such as the learning ladder based on cards which help the learner to take an independent path of learning have been successively diluted (content-wise) by the SSA authorities, for instance, in AP. From the erstwhile system of 1400 cards, the number has been whittled down to 72 and directly integrated with the textbook (by becoming activities for the learner/class)

Adopting quality as the only way for achieving UEE

Over six decades since independence, India as a nation has been striving to attain UEE, however, despite the plethora of systems and persistent efforts of everyone concerned; this simple goal still eludes us. In further efforts, one needs to focus on achieving UEE by continually increasing the quality of primary education, without this all efforts will undoubtedly fail.

7. Conclusions

Introduction

The main findings across the four states are briefly reviewed in this chapter, and some observations made about the effectiveness of the QP package.

The schools surveyed in the four states had widely varying characteristics. In general, AP and MP had the lowest number of teachers per school and students per school, while Bihar and Orissa had much larger numbers for these characteristics (Table 7.1). Crucially, however, schools in AP and Orissa had the lowest numbers of student per teacher. Undoubtedly, due to these differing attributes, inter-state comparisons should be used as indicative only (not statistically significant!)

Table 7.1: General characteristics of surveyed schools in the four states

AP Bihar MP Orissa

Teachers/school 3.0 1.7 9.6 10.7 1.7 2.5 6.2 6.9

Children/school 64 72 341 421 62 89 143 180

Children/teacher 22 42 35 39 36 36 23 26

Change in enrolment 2% 8% 2% -3% 0% 3% -5% -13%

Absenteeism 16% 12% 43% 38% 48% 59% 22% 29%

(Regular) Teachers with >140 days attendance

11% 3% 37% 16% 7% 2% 16% 8%

(Para) Teachers with >140 days attendance

9% 5% NA NA 18% 9% 46% 14%

Absenteeism among students on the day of the survey was highest in MP and Bihar, lower in Orissa, and lowest in AP. Coupled to the generally high levels of absenteeism across all districts (states) sampled was the problem of teachers‘ absenteeism. For instance, the proportion of regular teachers with more than 140 days attendance in the past academic year in descending order were in Bihar, Orissa, AP and MP respectively.

If smaller class sizes (i.e., children per teacher), rising enrolment, lower absenteeism and greater teacher attendance are taken as pre-requisites for a ‗well-performing‘ school, then schools in AP were the ‗best‖, followed by schools in Orissa.

School facilities

Schools in Bihar and Orissa had the largest number of rooms and teaching rooms, and AP and MP had the smallest (Table 7.2). Also schools in AP had the highest the proportion of schools with working and accessible drinking water facilities and toilet facilities for girls, followed by Bihar, Orissa and MP. It is apparent from the difference across QP and non-QP from table 7.2, that the former schools generally had better facilities. However, in many districts (states) the story became quite similar in nearly all states once ―use of facilities‖ is used as a sorting variable based on ordinal scoring.

Table 7.2: Facilities in surveyed schools in the four states

AP Bihar MP Orissa

QP Non-QP

Average number of rooms 4 3 7 5 3 5 9 10

Average number of teaching rooms 3 2 5 4 2 2 5 6

Proportion of schools with working drinking water facilities

71% 29% 63% 25% 56% 19% 59% 19%

Proportion of schools with working and accessible girls toilets

46% 21% 38% 6% 50% 7% 68% 24%

Proportion of schools with ‗above benchmark‘ Watsan facilities*

4% 0% 20% 40% 7% 0% 44% 20%

Proportion of schools with ‗above benchmark‘ Security and cleanliness*

96% 90% 28% 20% 71% 25% 84% 80%

Proportion of schools with ‗above benchmark‘ Recreation and sports*

31% 40% 28% 30% 41% 25% 44% 20%

More schools in Orissa and MP had separate classrooms for class 1 and 2 children that provided adequate space, ventilation and lighting (Table 7.3). However, it was not always the case, that QP schools have the better classroom facilities. While this was the case in AP and Orissa, non-QP schools in Bihar and MP did better on several of these criteria.

Table 7.3: Facilities in surveyed schools in the four states

AP Bihar MP Orissa

Space, ventilation, lighting Class 1* 19% 70% 12% 20% 75% 43% 75% 70%

Space, ventilation, lighting Class 2* 38% 40% 4% 0% 44% 37% 72% 40%

Blackboards and seating Class 1* 12% 0% 4% 10% 74% 0% 12% 0%

Blackboards and seating Class 2* 20% 0% 0% 0% 41% 51% 12% 0%

Reading corner Class 1* 20% 0% 0% 0% 8% 0% 8% 0%

AP Bihar MP Orissa

Reading corner Class 2* 20% 30% 0% 0% 41% 51% 8% 0%

Teaching methods

The proportion of schools that used TLMs and SLMs was comparatively higher in Orissa and AP, but the proportions were generally lower in others especially Bihar and MP (Table 7.4). In general, more QP schools used TLMs and had better teaching methods, although the proportion of schools varied across the surveyed districts (states). This was probably correlated by the increased number of trainings the QP teachers (compared to non-QP teachers) had undergone for preparing and using TLMs.

Table 7.4: Teaching methods in surveyed schools in the four states

AP Bihar MP Orissa

Use of TLMs Class 1* 50% 10% 16% 0% 48% 50% 72% 50%

Use of TLMs Class 2* 42% 50% 8% 0% 45% 38% 84% 30%

Use of SLMs Class 1* 31% 40% 28% 30% 7% 13% 8% 0%

Use of SLMs Class 2* 43% 60% 8% 11% 41% 51% 84% 50%

Teacher training

All QP schools and most non-QP schools in MP had teachers who had received training in the last academic year (Table 7.5). All QP teachers in AP and MP received language training, while all QP and non-QP teachers in MP and more than 75% of QP teachers in AP and Orissa received training in mathematics. In general, many more teachers from QP schools than non-QP schools received training, although Bihar and MP showed contrary trends.

Table 7.5: Teaching methods in surveyed schools in the four states

AP Bihar MP Orissa

QP Non-QP

% permanent teachers trained 13% 5% 13% 5% 100% 77% 53% 0%

% non-permanent teachers trained 0% 0% NA NA 88% 100% 22% 0%

% teachers trained in language 100% 60% 76% 0% 100% 0% 88% 20%

% teachers trained in mathematics 85% 20% 48% 60% 100% 100% 76% 40%

AP Bihar MP Orissa

QP Non-QP

% teachers trained in TLMs 73% 0% 40% 70% 96% 100% 4% 0%

Quality of training: language* 54% 0% 25% 11% 44% 13% 40% 10%

Quality of training: mathematics* 42% 0% 9% 0% 78% 13% 52% 10%

Quality of training: TLMs* 46% 13% 30% 10% 52% 13% 24% 10%

Teacher support

More schools in Orissa received local and academic support than schools in other states, and in general, QP schools received more support and teachers‘ perceptions were ‗rosier‘ than that of communities, in all states (Table 7.6).

Table 7.6 Teaching methods in surveyed schools in the four states

AP Bihar MP Orissa

QP Non-QP

Local support from SDMC etc. (teachers view)*

42% 30% 84% 80% 25% 0% 84% 80%

Local support from SDMC etc. (parents‘ view)*

27% 20% 80% 80% 21% 0% 88% 70%

Local support from PTA, GBM etc. (teachers view)*

35% 40% 0% 0% 55% 0% 68% 60%

Local support from PTA, GBM etc. (parents‘ view)*

23% 20% 0% 0% 62% 0% 68% 60%

Academic support from CRC Coordinator*

54% 40% 4% 30% 37% 13% 68% 100%

Academic support from peer group of teachers*

88% 90% 12% 10% 4% 0% 79% 90%

Learning achievement

The (slightly) better school facilities, classroom environment, teacher training and local support enjoyed by QP schools did not translate clearly into improved performance by children of QP schools. Given that the QP package was implemented only for class 1 children from about two years ago in these states, the best outcome indicators were the language and mathematics ability tests for class 3 children. Generally, more QP school students gave correct answers to the test questions, than non-QP school students. But this improved performance was statistically significant only in the case of mathematics in AP and language in Orissa (Table 7.7). In other cases, such as MP and Orissa, although QP

school students had generally got a larger proportion of answers correctly than students in non-QP schools, these differences were not deemed to be ‗statistically significant‘. However, this does suggest that a performance gap was emerging between QP and non-QP school students.

Table 7.7 Learning ability in surveyed schools in the four states

Andhra Pradesh

Bihar Madhya Pradesh Orissa

Class 3 Language

Similar Similar Similar QP better

Class 3 Mathematics

QP better Similar Similar Similar

Class 3 SC/ST

Similar Similar Similar Maths in QP

Class 5 Language

QP better Similar Similar Similar

Class 5 Mathematics

Similar Similar Similar Similar

Class 5 SC/ST

Lang better in QP Maths better in non-QP

Similar Lang better in QP Maths better in QP Maths better in non-QP

SC/ST children performed worse than non-SC/ST children generally, and with statistical significance in the case of Mathematics in QP schools in Orissa. This suggests that there are some systemic factors underlying poor performance by SC/ST children in Orissa.There were more statistically defensible differences in class 5, particularly among SC/ST and non-SC/ST learning ability, but these cannot be attributed to the QP package.

The impact on learning achievement is affected critically by the baseline conditions in different states and districts. In states like AP and Orissa, which may be classified as ‗intermediate‘ on an initial-intermediate-mature scale, QP has made a positive difference to teaching in Classes 1 and 2. Class 3 children in QP schools also generally perform better in learning achievement tests compared with their counterparts in non-QP schools, with the difference being statistically significant in some (but not all) cases. In states like Bihar and Madhya Pradesh, which may be classified as ‗initial‘ on the same initial-intermediate-mature scale, the roll-out of QP has not made a significant difference either in the six QP dimensions or in the learning achievement of children. Frequently, non-QP schools performed better on one or more dimension and there were no statistically significant differences in the learning achievement test results of children.

QP notwithstanding, committed teachers continue to be the key factor in Indian primary school education. In a rural landscape, where community environment, teacher training and academic support are roughly the same, some schools provide better teaching services largely because of dedicated teachers. QP has made a difference in schools where such teachers have seized the opportunity presented by QP, whether it is the TLMs and SLMs, sports kits, reading corners, infrastructure funds or teacher training. Where these have been provided, these teachers have made full use of them. Where they have not been provided, and classrooms are over-crowded and under-furnished, committed teachers have continued to be innovative with local materials for TLMs and with the skill and resources they have at their disposal to teach the children in their care. Yet, even the most committed of teachers can be demoralised and turn cynical in the face of poor school and classroom infrastructure, ineffective teacher training, and insufficient academic support.

QP, however, has made little difference in learning achievement in cases where the teachers are not committed, irrespective of the other factors. In such schools, although QP has improved water, sanitation and other infrastructure and also provided for better quality of teacher training and local and academic support to teachers, there is no commensurate impact on children‘s learning abilities yet, and non-QP schools are performing almost as well.

Relevance: Because of UNICEF‘s initiative, quality of primary education has become essential (no longer ―just desirable‖) to both the discourse and the implementation of primary education across the country. At the district level (instrumentally relevant administrative unit), however, there is a single-minded focus on ensuring access in quantitative terms. Thus, QP has not been able to demonstrate the relevance of focussing on quality to achieve UEE.

The infrastructural aspects of QP have been more readily adopted by SSA. However, innovations such as the learning ladder based on cards, which help the learner to take an independent path of learning, have been successively diluted by the SSA authorities, for instance, in AP. The erstwhile system has integrated the TLMs developed by the Chittoor district but only by diluting their innovative design (and thereby reducing the potential for improving quality).

Hence, QP was relevant conceptually and has helped to shape the policy discourse on both why and how the quality of primary education in the country should be augmented. However, this relevance was diluted during implementation in most of the districts/states and as a result contrary to QP‘s conceptualisation, quality itself seems to have been compartmentalised (into different dimensions) and the focus on integrating all the dimensions together has been lost.

Effectiveness: Despite having well-established offices in the states as well as good working relationships with the key state departments, UNICEF was not effective in rolling QP as it lacked the critical support from teachers. For successful implementation, the teachers must own the package and treat it as a broad canvas or a framework to facilitate learning and also innovate in the classroom.

Efficiency: IMaCS‘ earlier study44 for UNICEF for designing the analytical framework for cost-benefit analysis was stymied by the lack of financial data within UNICEF regarding the cost of QP; programme costs are not easy to disentangle from support costs. As a result, this impact assessment was not able to determine the effectiveness of QP. Further, it was difficult to estimate the costs.

Sustainability: Given the fragmentation witnessed during implementation, the sustainability of QP without UNICEF support, as envisaged earlier, is not possible. QP itself has not yet demonstrated its full potential in a cost-effective manner; however, various dimensions of the package such as the TLMs (and even TT) have been replicated by the SSA. This certainly can be constituted as a success for the package. However, from the

point of conceptualisation the incorporation of only some parts (dimensions) of the package instead of the whole package can also be viewed as a failure.

Impact: QP has demonstrated a new approach to education to both teachers and students, so much so that there is a demand for it to be introduced into higher classes. However, QP‘s impact has been limited because of feeble implementation; particularly due to its lack of ability to engage teachers effectively

8. Recommendations

Based on the contextual understanding of the primary education system in all the four states and districts, there are the three central themes that QP will have to address so that it can become instrumental in providing a cost-effective solution to addressing the quality (and quantity) vacuum that exists in the classroom. TEACHERS: ―Changing the way teachers teach remains a central, unanswered,

problem (and if solved, solution) of the future of QP (primary education”45.

Without any doubt, the teacher is the ―master‖ variable and is the key to unlocking the potential of the 6 dimensions of the package. Without engaging the teacher on all the integrated dimensions, QP has not been able to differentiate itself from the supply-driven approaches. It has thereby has fallen short of its goal of instituting a (support) system for delivering quality into the classroom.

TLMS: QP intervention by UNICEF has only succeeded in changing the supply of TLMs; not their usage in class by teachers.

Very often the field and also in the mind of different stakeholders, the quality package has been repackaged into a single dimension TLM. Although, this identification reveals a need on the ground, it also renders the overall package ineffective. In addition, as per the assessment, TLMs are optimally used in the class only if the teacher is motivated/committed, which cannot be simply by providing TLMs and undergoing training.

IMPLEMENTATION PLAN: To deliver quality in the classroom without a robust implementation plan is extremely risky

UNICEF has well-established state offices and has also extended its reach to the districts. However, managing the complex interfaces required for the success of QP across varying interests (and abilities) of different stakeholders (especially parents and communities) needs a logical framework or result-based management approach culminating in a time-bound implementation matrix. Expecting success without this in place is misguided.

Suggestions for future evolution of QP

The quality package needs to be a dynamic in responding to the changes in policy and also in educational technologies that can enable its purpose significantly. Therefore, in order to help the evolution of QP further, some suggestions as per the themes outlined are given below:

Teachers

Success of QP is locked in teachers‘ willingness and competence to implement the new teaching-learning approaches using TLM. This is the weakest spot. Capacity building of teachers follows the most conventional format emphasizing on competence development alone. Programmes of teachers‘ capacity building need to be reconstructed aiming at improving personal effectiveness of teachers (soft skills) followed by competence in instruction and related other areas. .

Development of TOR for Cost Benefit Analysis for Quality Package for UNICEF by IMaCS, June 2007

QP is comprehensive, however, not necessarily comprehendible by teachers, BRC/CRC officials and community leaders. QP should be spelt out in small and implementable components. It should be decongested and the delivered.

Teachers and Headmasters of QP schools do not necessarily demonstrate their understanding about the relevance and importance of various kits in the QP, e.g. BIG, Sports, etc. Illustrated attractive advocacy literature should be developed for teachers (not the conventional uninspiring teachers manuals) introducing various kits.

CRC/BRC personnel being recruited from among the primary teachers lack source-credibility and acceptability among the fellow teachers. Mentoring is a skill of higher order. For implementation of QP, a team of mentors should be identified/selected and trained in mentoring skills. Such mentors need not be full-time; they should preferably be selected from amongst the teacher educators in the universities, colleges of education and DIETs. On one hand, this will provide support to QP implementation, on the other hand it will generate a resource group in each State.

The CRC/BRC functions, most often, as another conventional unit of the government, often busy in collecting data and feeding the state. They lack perception about their role in implementing QP. The CRC/BRC coordinators should be freed from the routine work of the State governments, and allowed to perform their role for which these were created.

School evaluation instrument developed as a component of the toolkit is currently used by the CRC and BRC coordinators. A copy of the formats filled up by the coordinators is given to the school that often store it in the files without reflecting and deriving strategies for change. Instead, teachers can be involved in institutional self-evaluation, in addition to the current practice. Institutional self-evaluation works as a more powerful tool of organizational diagnosis. Also it provides an opportunity of ownership among the teachers.

At the conceptual level, QP is a well designed package for quality intervention in primary schools. However, it also needs to be complemented with an ICT Kit; both ICT in primary and elementary education is also favoured in the SSA.

The components of QP that approaches students directly are doing much better than other components. QP should strengthen direct inputs to students (SLMs).

Implementation plan

As implementation of QP is largely in the domain of state governments, hence, successful implementation is largely dependent upon the interest and competence of the state system and its officials. UNICEF should invest greater energy and efforts in engaging and involving the State governments and agencies in implementing QP in its true spirits.

QP has been designed by experts and pedagogues. Despite its soundness, it lacks teacher involvement. For successful implementation, the teachers must own it. It is possible to build up teacher ownership if QP is treated as a broad canvas or a framework that facilitates a teacher to innovate and find a space for himself or herself.

QP is supply driven; it faces resistance from teachers, and teacher‘s organizations in some states. They suspect it to be another effort to increase their workload. For success, the supply driven paradigm must be reconstructed to emerge as demand driven. Advocacy and training of leaders of Teachers Associations are recommended.

Successful implementation of such an innovative package for quality improvement requires mentoring, rather than monitoring as is prevalent in the existing format. Mechanism of QP implementation should build new mechanism of on-the-job mentoring of teachers in the QP contexts.

Quality is a holistic phenomenon. Quality education abhors islands of quality efforts (grades I and II) in oceans of indifference. QP should be reconstructed to embrace the whole school – upto Vth or VIIIth grades depending upon whether the school is a primary or an elementary school.

Quality management is an effortful endeavour. It cannot be achieved by default, by improving quality of teaching and learning material and programmes of capacity building of teachers. Quality managers in the schools, clusters, blocks and districts should be provided special training in quality management in education. Training packages should be designed and supported by the relevant high-quality training material on quality management in primary and elementary education.

The role of parents and community leaders envisaged in QP is at best marginal. Since there are a large number of parents, mothers in particular, who have some amount of education should be more pro-actively involved in quality improvement in primary education. QP should figure out a more prominent and inspiring role for mothers in implementing QP in schools.

The third party assessment of QP shows that only concerted effort by the communities, teachers and the educational system can ensure that a system of primary education can deliver quality (with access) at the village, district and state levels. Therefore, the way forward for UNICEF to support the government programme (SSA), would be to facilitate (implement) quality interventions bottom-up in the future, that is, to concentrate on the teacher as the prime vector of quality and ensure that all things (aspects) that hinder the teacher‘s ability to teach are minimised.

ANNEXE 1: Terms of Reference

The overall purpose of the study is to analyze the effectiveness of the QP package by using the model developed by the agency (in a previous project) to evaluate how the output variables behave with differing strengths of the UNICEF investments in the 6 input variables.

1. Development and supply of essential teaching-learning materials (TLMs)

2. In-service teacher training (TT)

3. Academic support to improve active learning, continuous, supportive student

assessment and rational class management (AS)

4. Improvement of classroom environment (CE)

5. Improvement of school environment and facilities (SE)

6. Promotion of community participation (CP)

In particular the study will estimate the capital and running costs of the inputs and assess them in terms of the incremental benefits in education outputs derived from the investment in each of the inputs. The benefits to be tested by the model are in terms of achievements in the following outputs: a) Raising the benchmarks and norms around the quality of school and classroom

infrastructure among key education stakeholders;

b) Costs involved in making the shift from essential to the effective level of quality;

c) Measuring the impact of these norms on levels of learning using assessment tests;

d) Affecting the Teaching/Learning processes both observed and conceptual with respect

to teacher knowledge and practices;

e) Measuring relevance and use of QP teacher training;

f) The availability and critical use of reasonable quantum of Teaching /Learning Materials

and their impact on:-

Enrolment – both GER and particular NER;

Attendance of children, especially girls both pre-mid-day meal and post-mid-day

Retention – in the framework of a no detention policy at primary grades, as

measurable through cohort analysis;

Improvement in drop-out rate (DO) in QP schools (vis-a-vis non-QP school);

g) Learning Assessments – incremental gains in learning achievements (LA) in QP schools

(vis-à-vis non-QP schools).

h) The effectiveness of QP training and its differentiation from standard SSA training

i) Perception of training by Teachers: quantum as well as quality.

j) Value of QP training in the context of overall- in-service training.

k) Children's achievements measuring the 8 intelligences and well being indicators.

l) Community linkages and partnerships to sustain QP

m) To assess to what the extent the inception of QP has facilitated increased participation

of the community

n) To assess the gaps in community participation

o) Appropriateness of the scope of reform and action - classrooms, schools, cluster,

block, district? The roll-out strategies need to be evaluated to see if the reforms

are/can be implemented at just the class I and II levels or for the school as a whole

and then beyond to cluster, block and district levels too.

p) Gender and social inclusion indicators, track record

q) Measure impact of Classroom environment (CE), School environment (SE), TLMs and

community participation on girls‘ education/participation

r) Measure impact of Classroom environment (CE), School environment (SE), TLM and

community participation on other socially excluded such as SC/STs or minorities.

s) Key lessons learned with respect to integration of inputs with least costs. How the

output variables behave with differing strengths of the UNICEF investments in the 6

input variables.

ANNEXE 2: Quantified Participatory Assessment

1. INTRODUCTION

Several methods have been developed in the recent past to address this issue of generating numbers from participatory activities.46 The Methodology for Participatory Assessment (MPA) 47 was developed in the late 1990s to assess the sustainability of 88 water supply and sanitation projects in 15 countries and used participatory tools to bring out information and then translated this into numbers using a scoring system.48 The MPA continues to be used as a ‗comparative evaluation tool in large domestic water projects and programs‘.49 The Quantified Participatory Assessment (QPA) was developed from the Methodology for Participatory Assessment (MPA) and used in India in a variety of development projects since 1999 (James, 2003a).50 Apart from the expansion from the water and sanitation sector to other sectors, notably watershed development, poverty alleviation, rural livelihoods and water resources, the QPA added several other features to the MPA, including peer review of scores, documentation of reasons for scores, use of an MS ACCESS database to store and analyse information, several rounds of stakeholder meetings and a detailed action planning report. The QPA was also the basis of the modification of the MPA in Nepal to the NEWAH Participatory Assessment (NPA) by the Gender and Poverty (GAP) Unit of the national NGO, Nepal Water and Health (NEWAH), in Kathmandu, Nepal.51 The NPA adapted the MPA to suit the geographical, socio-economic and ethnic reality of Nepal, modified the scoring systems to include benchmarks in a flexible 0 – 100 scale, developed additional tools to elicit information on health, hygiene and sanitation issues, and collected additional qualitative information using case studies (James et al., 2003a, 2003b, 2003c). Qualitative Information Appraisal (QIA) is a generic methodology, developed from the experiences with the MPA, QPA and NPA, which goes beyond the constraints of the term ‗Assessment‘. The QIA is designed for use in both one-time assessments for baseline, mid-term and overall project impact assessments, as well as for continuous monitoring as part of a project‘s regular monitoring and evaluation system.

46 See, for instance, Chambers (2003). 47 The MPA was developed by Christine van Wijk (van Wijk, 2003) for a Participatory Learning and Action (PLA) project that was a multi-disciplinary and multi-country assessment exercise looking at the factors underlying the sustainability of water supply and sanitation projects (Dayal et al., 1999, Gross et al., 2001). 48 The scoring system is detailed in James (2000 and 2001) and in Dayal et al. (1999). 49 Wijk, 2001, p. 2. The revised MPA is described in Mukherjee and van Wijk (2003) while experiences with using the MPA are in van Wijk and Postma (2003), Postma at al., (2003), van Wijk et al., (2002), Paudyal et al. (2002). 50 This work was done by AJ James who did the statistical analysis of the MPA data for the initial PLA study coordinated by Rekha Dayal of the Water and Sanitation Program. See also, James (2002, 2003b, 2003c, 2003d), James and Kaushik (2002), James et al., (2002), James and Snehalata (2002a and 2002b). 51 For an account of the pilot MPA and the problems experienced in the field see Paudyal et al. (2002). See James et al., (2003a and 2003b) for a description of the creation of the NPA, and James et al., (2003c) for the details of one application in Nepal.

2. APPLICATIONS

The QPA has been applied in several applications within India and outside (see Table).

Funding source Location Project Focus Area Sample size Year

Water & Sanitation Program (World Bank)

Global Participatory Learning and Action (PLA) global study of the World Bank‘s Water and Sanitation Program

Impact assessment of RWSS projects

88 projects; 15 countries

1997-9

European Community

India Doon Valley Integrated Watershed Management Project

Social & environmental impact

16 villages 1999-2000

DFID India India APRLP Water Resources 106 habitations 2001-2

DFID India India WIRFP Rural Livelihoods 45 villages 2002-3

World Bank India Rajasthan District Poverty Initiatives Project

Project Processes

14 villages, 2 districts

2001-2

World Bank India Analytical and Advisory Activity (AAA) on Urban Public Health in Tamil Nadu

Performance of Essential Public Health Functions

26 ULBs 2002-3

Asian Development Bank

Nepal Community-based Water Supply and Sanitation project preparation

Water Supply, Sanitation & Hygiene

5 regions 2003

Asian Development Bank

Sri Lanka & Vietnam

Evaluation of ADB-funded national Water Supply and Sanitation projects

Sri Lanka 104 sub-projects Vietnam WSS 20 villages; 350 households

UNICEF India Independent Evaluation of the Child‘s Environment Programme (CEP)

117 villages 2004

Nepal Water for Health (NEWAH)

Nepal GAP Evaluation Water Supply, Sanitation & Hygiene

15 villages 2003-4

Tamil Nadu Water and Drainage Board

India Change Management Pilots Evaluation

200 habitations 2005-6

Uttaranchal Livelihood Project in the Himalayas (ULIPH)

India Baseline survey Rural Livelihoods 140 villages 2007

UNICEF India Impact of Quality Package on Education Quality

Education 18 schools 2007

3. KEY COMPONENTS OF THE METHOD

From past experience in applying this methodology (in India, Nepal, Sri Lanka and Vietnam), the following arrangements have been found to be optimal:

I. Inception Meeting

A brief meeting (one-day) to clarify the issues to be assessed, the background information available and the logistical arrangements.

II. Methodology and Planning Workshop

This is a vital part of the assessment, where the assessment team discuss and finalise the issues to be assessed, the indicators to be used, the ordinal scales, and thus the QPA field formats. This usually has role plays, mock interviews and field testing to make sure the assessment team practise and develop their PRA and facilitation skills, which is one of the key determinants of the success of the QPA field assessment. This workshop can take from 10 – 14 days depending on the complexity of the issues to be addressed.

Field testing: Although this is usually carried out at the end of the Methodology and Planning Workshop activity, it deserves a special mention. Two rounds of field testing are needed, the first to identify the problems to be rectified in the field formats, and the second to make sure the revised formats are suitable for the survey. Given the size of the assessment team being trained (36 field staff + 3 field coordinators+ 1 field supervisor+2 Research Associates), usually 2 villages are needed for each round of field testing – making a total of 4 villages. None of these villages should be part of the actual survey.

III. Field Assessment

Informing villages about the assessment: Prior information is usually needed for the meetings and focus group discussions – except where it is apprehended that villages may be ‗dressed up‘ for the assessment. If so, the village is informed only a day or two in advance.

Village assessment schedule: In accordance with ‗good practice‘ in participatory assessments, the assessment usually starts with a meeting with village officials (headman, patwari, VAO, etc.), elders, teachers and key informants – to inform them about the purpose of the assessment, to get basic information about the village, and to plan the various focus group discussions (FGDs). Thereafter, a transect walk and social mapping is carried out (to check ‗unserved households‘, etc.), also a water system review. Subsequently, either in the afternoon or evening, FGDs can be held with those who have received training from the project/TWAD Board, women‘s groups, etc. Finally, a village meeting is held to inform them about the basic findings of the assessment. Compliance with international ‗good practice‘ is vital for the validity of the participatory assessment.

Assessment time: Assessments take 1 – 4 days per village, depending on the complexity of the field formats. The minimum time is 1 day per village. It is best to have the team debriefing and data entry the very next day, so that field teams remember details of discussions and verify the scores. Entering data in the latter part of the same day will minimise errors and avoid the fatigue (and hence errors) of mass

data entry at the end of the assessment. This gives a maximum rate of 3 villages per week (with 1 day off), at which rate, 10 2-person teams can cover 100 villages in 20 days.

Field teams: While field teams have been between 4 – 6 people per village, the ideal combination is a 4-member field teams which can split into two 2-member teams in the field. The minimum, however, is 2-persons per team. Gender balanced teams are highly desirable. To complete 100 villages in 2 weeks, at the rate of 3 villages per week per team will require 18 teams, or 36 field staff.

Field coordinators: Field-level coordination is essential for quality control, especially to check the nature of facilitation during FGDs and to ensure validation of information provided in the FGDs. They are also useful for trouble-shooting field-level problems, including logistics. In addition to the field supervisor, a minimum of 3 Field Coordinators would be necessary for a 100-village assessment. Focus group discussions: Each FGD takes between 1-2 hours, and more than 2 hours tests participants‘ patience and could yield biased responses. These have basically to give participants the ‗freedom and space‘ to present their own views, feelings and must adhere to good practice of facilitation (e.g., no leading questions, no prompting, opportunities for all participants to express their views, etc.).

IV. Database, Data Cleaning and Analysis

Database: An ACCESS database is usually created for data entry, so that the computer format matches the paper format exactly and thus minimises data entry errors.

Data cleaning: Even after careful data entry, there is need to ‗clean‘ the data, usually in a joint meeting with the field teams, lasting up to 5 days, depending on the number of villages surveyed and the number of issues covered in the field formats. Basically, this involves scanning the scores and reasons for scores entered in the database, identifying data gaps (e.g., Reason for Score not filled out), and doing some basic calculations (e.g., COUNT, MAX, MIN) to check possible data entry errors. Having the field team at this point is useful for quick cross-verification.

Data analysis: This basically involves generating frequency histograms and user-friendly graphs to present the findings as clearly and intelligibly as possible. This should take around 3 days after data cleaning.

V. Report Writing

Pulling together the methodology, presenting the main findings, and mentioning the quality control efforts of the survey are the key aspects of the report writing exercise, which should take around 6 days in total.

REFERENCES Chambers, Robert (2003) ‗Participation and numbers‘ in PLA Notes, 47, August.

Dayal, Rekha, Christine van Wijk, and Nilanjana Mukherjee (1999) Methodology for Participatory Assessments: with Communities, Institutions and Policy Makers, Water and Sanitation

Program (Washington), and International Resource Centre for Water and Sanitation (Delft, the Netherlands): New Delhi.

Deshingkar, Priya and James. A. J. (2001) ‗PRA: Some Concerns from the Field‘ in IFAD, ANGOC and IIRR, Enhancing Ownership and Sustainability: A resource book on participation, International Fund for Agricultural Development (IFAD), Asian NGO Coalition for Agrarian Reform and Rural Development (ANGOC) and International Institute for Rural Reconstruction (IIRR).

Gross, Bruce, Wijk, C. van, and Mukherjee, Nilanjana (2001) Linking Sustainability with Demand, Gender and Poverty: A study in Community-Managed Water Supply Projects in 15 Countries. World Bank Water and Sanitation Program and IRC International Water and Sanitation Centre, New Delhi.

James, A. J. (2000). ‗MPA: A New Methodology for Participatory Assessment‘ Waterlines, October 2000.

James, A. J. (2001). ‗Enhancing the ―Assessment‖ in Participatory Assessments‘, in IFAD, ANGOC and IIRR, Enhancing Ownership and Sustainability: A resource book on participation, International Fund for Agricultural Development (IFAD), Asian NGO Coalition for Agrarian Reform and Rural Development (ANGOC) and International Institute for Rural Reconstruction (IIRR).

James, A. J. (2002). ‗Quantified Participatory Assessments for the Water Resources Audit of the Andhra Pradesh Rural Livelihoods Project: Kalyandurg Mandal in Anantapur District‘, report submitted to DFID India.

James, A. J. (2003a). ‗Quantified Participatory Assessment: Capturing Qualitative Information in Large-Scale Development Projects‘. Unpublished.

James, A. J. (2003b). ‗Quantified Participatory Assessment of the Impacts of the Western India Rain fed Farming Project‘, IFFDC project area, report submitted to Atkins, UK.

James, A. J. (2003c). ‗Quantified Participatory Assessment of the Impacts of the Western India Rain fed Farming Project‘, GVT project area, report submitted to Atkins, UK.

James, A. J. (2003d). ‗PIMEDD Self-Assessment System For Public Health Functions in Urban Local Bodies In India: Report of a Pilot Assessment in Tamil Nadu, submitted to the World Bank, New Delhi.

James, A. J. and Rakesh Kaushik. (2000). System for Integrated Monitoring Assessment and Learning (SIMAL) for the Rajasthan District Poverty Initiatives Project (DPIP), report submitted to the World Bank, New Delhi.

James, A. J. and Rakesh Kaushik. (2002). Piloting Quantified Participatory Assessments in the Rajasthan District Poverty Initiatives Project (DPIP), report submitted to the World Bank, New Delhi.

James, A. J., and M. Snehalata. (2002a). ‗Quantified Participatory Assessments for the Water Resources Audit of the Andhra Pradesh Rural Livelihoods Project: Dhone Mandal in Kurnool District‘, report submitted to DFID India.

James, A. J., and M. Snehalata. (2002b). ‗Women, Water And Livelihoods: Engagement With Policy In Andhra Pradesh‘, in Women's Empowerment Policy & natural resources - What progress? , Report of Conference organised by the Planning Commission, Government of India, and Overseas Development Group, University of East Anglia, UK, and funded by the Department for International Development, Government of the United Kingdom, New Delhi, 31 May 2001.

James, A. J., Michelle Moffatt and Raju Khadka. (2003a). ‗Evolving the NEWAH Participatory Assessment (NPA)‘, A Case Study Prepared for the IRC International Water and Sanitation Centre, Delft, Netherlands

James, A. J., Raju Khadka, Dipendra Shahi and Jennifer Appave. (2003c). ‗Evaluating the impact of NEWAH's gender and poverty approach using the NEWAH Participatory Assessment: A Report of the Assessment of 15 Communities in 5 Development Regions, submitted to Nepal Water for Health (NEWAH), Kathmandu, Nepal.

James, A. J., Raju Khadka, Michelle Moffatt and Corine Otte. (2003). ―From MPA to NPA in Rural Nepal‖, unpublished, IRC International Water and Sanitation Centre, Delft, Netherlands.

James, A. J., Vineet Pangtey, Pratibha Singh, and Keith Virgo. (2002). ―Bringing People‘s Perceptions to Project Management Desktops: A Quantified Participatory Assessment of the Doon Valley Project‖, Impact Assessment and Project Appraisal.

James, A. J., Leonie Postma and Corine Otte (2003) ―A Qualitative Information System for Large-Scale Development Projects‖, unpublished, IRC International Water and Sanitation Centre, Delft, Netherlands.

Moffatt, Michelle, Laxmi Paudyal and A. J. James. (2002). ‗Linking demand, gender and poverty for sustainability‘, Paper presented at the 28th WEDC Conference on Sustainable Environmental Sanitation & Water Services, Kolkata, India.

Moffatt, Michelle and Umesh Pandey. (2003). forthcoming

Moffatt, Michelle and Raju Khadka. (2002). ‗A Gender and Poverty Approach in Practice‘, Paper presented at the 28th WEDC Conference on Sustainable Environmental Sanitation & Water Services, Kolkata, India.

Mukherjee, Nilanjana and Christine van Wijk (2003) Sustainability Planning and Monitoring in community water supply and sanitation, A Guide on the Methodology for Participatory Assessment (MPA) for Community-Driven Development Programs, Water and Sanitation Program, Washington and IRC International Water and Sanitation Centre, Delft, Netherlands.

NEWAH (2002), ‗NEWAH Participatory Assessments: A Brief Note‘, unpublished. December.

NEWAH (2003), ‗Consolidated Report of the Socio-Economic Survey for the Project Preparation Technical Assistance of the Community Based Water Supply and Sanitation Project‘, submitted to ARD Pvt. Ltd., Nepal Water for Health, Kathmandu, April.

Postma, Leonie, Christine van Wijk and Corine Otte (2003), ‗Participatory quantification in the water and sanitation sector‘, in PLA Notes, 47, August.

Ramamohan Rao, M.S., C.H. Batchelor, A. J. James, R. Nagaraja, J. Seeley and J. A. Butterworth (eds.) (2003), Andhra Pradesh Rural Livelihoods Programme Water Resources Audit: Phase I Report, Andhra Pradesh Rural Livelihoods Programme and Department for International Development, Government of the UK.

Van Wijk, Christine (2001) The Best of Two Worlds? Methodology for participatory assessment of community water services, Technical Paper Series No. 38, IRC International Water and Sanitation Centre, Delft.

Van Wijk, Christine and Leonie Postma (2003) ‗MPA: A new methodology for participatory monitoring‘, Waterlines.

Van Wijk, Christine, Kumala Sari and the Pradipta Paramitha Team, Nina Shatifan, Ruth Walujan, Ishani Mukherjee, Richard Hopkins (2002) Flores revisited: Sustainability, hygiene and use of community-managed water supply and sanitation and the relationships with project approaches and rules, Water and Sanitation Program – South East Asia, Jakarta and IRC International Water and Sanitation Centre, Delft, Netherlands.

WS Atkins (2000) Impact on Social Equity and Household Livelihoods Study, Technical Assistance Report, K. Rai and C. Kunwar, Watershed Management Directorate, Dehradun, Uttaranchal, India.

WS Atkins (2000) Socio-Economic and Environmental Impact Study, Technical Assistance Report, A.J. James, Watershed Management Directorate, Dehradun, Uttaranchal, India.

ANNEXE 3: Final Field Formats

Quantified Participatory Assessment of the

Impact of the UNICEF-GOI Quality Package

Fill in one sheet for each school

START TIME AM/PM

1. FOCUS GROUP DISCUSSION WITH SCHOOL TEACHERS

1.1. School and survey details

Name of State

Name of District

Name of Village

Name of School

Name of Field Investigator 1

Name of Field Investigator 2

Date of Survey

1.2. General details Circle the right answer

Is it a Quality Package School? YES NO

Type of school

1 Primary YES NO

2 Upper Primary YES NO

3 Other (specify)

YES NO

1.3. Availability of Teaching Learning Material (TLMs) Circle the correct answer

Type of Teaching Learning Material

Is it in the school?

Was it provided by

UNICEF?

Is it being used?

Is it in useable condition?

MGML Kit/Materials YES NO YES NO YES NO YES NO

Sports kit YES NO YES NO YES NO YES NO

Gardening kit YES NO YES NO YES NO YES NO

Hygiene kit YES NO YES NO YES NO YES NO

Supplementary Reading Material YES NO YES NO YES NO YES NO

Self-learning Material YES NO YES NO YES NO YES NO

Type of Teaching Learning Material

Is it in the school?

Was it provided by

UNICEF?

Is it being used?

Is it in useable condition?

Stationery YES NO YES NO YES NO YES NO

Worksheets/Workbooks YES NO YES NO YES NO YES NO

Water & salt testing kits YES NO YES NO YES NO YES NO

Desks/Chairs/Chowkis YES NO YES NO YES NO YES NO

Blackboards YES NO YES NO YES NO YES NO

Mirrors YES NO YES NO YES NO YES NO

Globe, Maps and Charts YES NO YES NO YES NO YES NO

Others (specify)

YES NO YES NO YES NO YES NO

1.4. Teacher training and academic support

Was any training conducted for school teachers? YES NO

Was training conducted by UNICEF for school teachers? YES NO

Did trained teachers teach Class III and V last year (2007-08)? YES NO

Did the CRC provide any academic support to teachers last year (2007-08)? YES NO

Did the CRC monitor schools last year (2007-08)? YES NO

Did the CRC interact with SDMC/VEC/VSS last year (2007-08)? YES NO

1.5. Number of school teachers and school staff

Number of permanent teachers

Number who received

QP training?

Number of Non-permanent

teachers

Number who received

QP training?

Female

1.6. Type of teacher training received

Type of training Yes/No Number of teachers trained

1 Training in language (and literacy) YES NO

2 Training in maths (and numeracy) YES NO

3 Training to prepare and use TLMs YES NO

5 Refresher training YES NO

6 UNICEF training on QP (specify)

1 YES NO

2 YES NO

3 YES NO

Type of training Yes/No Number of teachers trained

7 Other (specify type of training)

1 YES NO

2 YES NO

3 YES NO

4 YES NO

1.7. Quality of Teacher Training: Language

0 Training was not satisfactory at all (because badly organised, bad trainers, poor quality material, etc.)

25 Training was satisfactory; but TLMs not provided OR, if provided have not been used by the teacher

50 Benchmark: Training was satisfactory, TLMs were provided and have been used by the teacher once

75 In addition, teachers have attended refresher (follow-up) trainings

100 Ideal: In addition, teachers have prepared their own locally-relevant lessons and educational materials and are using TLMs regularly

Reason for score

1.8. Quality of Teacher Training: Maths

Reason for score

1.9. Quality of Teacher Training: Preparing and using TLMs

Reason for score

1.10. Suggestions for improving the effectiveness of teacher training

Actions Rank

1 Better trainers

2 More practical and hands-on training

3 Better atmosphere for asking questions and clarifying doubts

4 More convenient timings for training

5 Better logistics (place of stay, food, etc.)

6 Other (specify)

Comments

1.11. Nature of SDMC/VSS/VEC meetings

0 No members come even when meeting is organized

25 Only few members (around 3) come for meetings

50 Benchmark: Quorum is met for one meeting in a year

75 Quorum is met for more than one meeting in a year, and decisions are taken on school issues

100 Ideal: In addition, committee is regularly monitoring quality of education in the school

Reason for score

1.12. Nature of PTA or General Body Meetings

100 Ideal: In addition, PTA or GB is regularly monitoring quality of education in the school

Reason for score

1.13. Nature of meetings of other CBOs involved in school (e.g., MTA)

Reason for score

1.14. Nature of academic support from Cluster Resource Centre (CRC) Coordinator

0 No visits by CRC Coordinator to the school (section)

25 During the visit, the CRC Coordinator observes at least two classrooms using an observation schedule as per CRC toolkit.

50 Benchmark: CRC Coordinator makes regular visits (once in a month) and gives at least 2½ hours during every visit

75 In addition, during visits the CRC Coordinator observes classroom teaching, gives feedback to the teacher and gives demonstration lessons according to need

100 Ideal: In addition, the CRC Coordinator, Education Inspector and DIET/SCERT liaison officers do a panel supervision of each teacher‘s classroom once a year

Reason for score

1.15. Academic sharing by teachers

0 No academic sharing is done among primary school teachers

25 Teachers meet at cluster level to discuss teaching issues, but these are not regular and not much is discussed in these meetings

50 Benchmark: Cluster meetings are held regularly and effective teaching strategies are developed to address problems faced in class

75 In addition, teachers get to see practical demonstration of effective teaching strategies and practice these themselves

100 Ideal: Peer-to-peer sharing evolves into a locally relevant resource group for teachers, amply supported by head teachers and the CRC

Reason for score

1.16. Reasons for children attending school

Reason Rank

1 Because of mid-day meal scheme

2 Because of new Learning Material (e.g., QP material)

3 To learn

4 To play

5 Because their parents tell them to come

6 Not to be a nuisance at home!

7 Others (specify)

1.17. Number of primary school children in Classes III and V

Number of children

Class I Class II Class III Class IV Class V

Boys Girls Boys Girls Boys Girls Boys Girls Boys Girls

2006-07 Enrolled

2007-08 Enrolled

2007-08 Attending*

*On the day of the visit

1.18. Attendance of teachers working since 1 April 2007

Permanent Teachers

Non-permanent Teachers

Number of teachers with more than 140 days attendance in the current academic year (2007-08)

Ask these questions only for UNICEF QP Schools

1.19. Positive and negative features about the QP Kit

Positive features Rank

Negative features Rank

1.20. Response of parents to QP Kit

0 Parents have not been told about the QP kit and/or are unaware of it

25 Parents have been told and/or some parents are aware about the QP kit

50 Benchmark: All parents have been shown the QP kit (e.g., when they come to school) OR all parents are aware of it

75 In addition, some parents ask about the kit

100 Ideal: All parents know about the QP kit and ask about when the kit is to be changed, replaced or improved

Reason for score

End time AM/PM

2. SCHOOL OBSERVATION

START TIME AM/PM

2.1. Availability of facilities

Number of rooms in the school building

Number of rooms used for teaching

Does the school have drinking water facilities? Yes No

Does the school have working drinking water facilities?

Does the school have a separate girl‘s toilet? Yes No

Is it in working condition? Yes No

Is it available for use? Yes No

2.2. School Environment – Water and Sanitation

0 There are no safe drinking water or toilet facilities available within the school compound

25 There are safe drinking water and toilets facilities available within the school, but no separate toilet for girls

50 Benchmark: There is a separate toilet for girls. All the toilets conform to SWASTHH norms. In addition, all toilet, water and drinking water facilities are fully functional and maintained in hygienic conditions

75 In addition, there is place for washing hands that is separate from the toilets and the drinking water facility.

100 Ideal: Toilets are available as per SWASTHH Plus norms52 and are used by everyone. In addition, soap is available at the hand washing facility.

Reason for score

2.3. School Environment –Recreation and sports

0 There is a no outdoor playing area for school children

25 There is space available for outdoor play area but no sports kit

There is at least one latrine for 40 teachers, one latrine for 40 girls, and one latrine for 80 boys; and at least one urinal for 50 boys.

50 Benchmark: Space for playing area is available and toys such as skipping ropes and balls are also available

75 In addition, there are safe and functional outdoor play facilities such as swings, slides and ropes; adequate sports material (football, soft ball, cricket kit etc.) is available for all the children;

100 Ideal: In addition, everyday there is a specific time for every class for sports

Reason for score

2.4. School Environment – Security and Cleanliness

0 The classroom is not clean or there is no door for locking a classroom (with TLMs)

25 The classroom is clean and there is a door for locking a classroom (with TLMs)

50 Benchmark: The school interior is well swept and the doors are in working condition

75 In addition, the school exterior is well kept and the school building has a lockable gate

100 Ideal: Classrooms, interior and exterior of the schools are well maintained (with repairs and replacement) with the support of the PTA or community or Panchayat

Reason for score

End time AM/PM

3. CLASSROOM OBSERVATION START TIME AM/PM

Fill one sheet for Class I and another sheet for Class II

3.1. Manner in which Teacher Learning Materials (TLMs) and Self Learning Materials (SLMs) are being used in the classroom

0 TLMs not displayed or visible and children do not know about TLMs

25 TLMs are displayed or visible but the teacher follows a traditional teaching method usually

50 Benchmark: There are signs that the teacher uses TLMs regularly in class (wear and tear, etc.); children are familiar with TLMs and are able to correctly explain their use

75 In addition, worn-out TLMs have been replaced and children are also encouraged to use SLMs

100 Ideal: In addition, the teacher has developed additional TLMs and SLMs and children are using SLMs

Reason for score

3.2. Enhancing Learning Achievement

0 Teacher does not take class OR is not interested in teaching

25 The teacher teaches, TLMs are present in the class but the teacher does not use TLMs

50 Benchmark: The teacher teaches using given TLMs in the class

75 In addition, the teacher has improved TLMs on his own initiative (using SSA TLM grant)

100 Ideal: In addition, the teacher pays special attention to problem children and helps them to improve

Reason for score

3.3. Use of Student Learning Materials

0 SLM material not received or locked; students don't use SLM materials

25 SLM material either damaged or incomplete; students use SLM materials occasionally but the teacher has not helped the students

50 Benchmark: SLM material received; being used with the help of teacher

75 In addition, students use SLMs extensively on their own and SLMs show evidence of wear and tear; one student can explain how far he or she has reached in the ‗learning ladder‘

100 Ideal: In addition, all students can explain how far he or she has reached in the ‗learning ladder‘

Reason for score

3.4. Classroom Environment – Space, Ventilation and Lighting

0 There is no covered space for conducting classes

25 There is at least 6 square feet of space per child where they can sit and work.

50 Benchmark: In addition, each classroom has at least two windows that allow in light and air.

75 In addition, there is 12 square feet or more for the children to sit and work.

100 Ideal: In addition, the primary school has adequate and well-maintained covered space and, adequate windows for natural lighting and ventilation.

Reason for score

3.5. Classroom Environment – Reading Corner

0 There is no space available for a reading corner or library

25 There is space available for a reading corner or a library; but it is not being used (e.g., locked)

50 Benchmark: There is space available for a reading corner or a library and it is being used. There is one copy of each book for every 8 children

75 In addition, there are ‗age-appropriate‘ books for every 8 children per month (on rotation basis within cluster)

100 Ideal: In addition, The library is open all day during the school hours and is being used by the children and maintained by the teacher and children.

Reason for score

3.6. Classroom Environment – Blackboard & Seating

0 Children sitting on floor (without mat): the blackboard is not functional.

25 There is a clean mat to sit on and there is enough working space / table / chowki upon which TLMs can be placed when in use; but over-crowded blackboard available but not functional or too small

50 Benchmark: There is a chowki for every four children. In addition, there is at least one functional blackboard per classroom, measuring at least 6 by 4 feet

75 In addition, there is extra wall writing space that the children can use, and there are other blackboards placed at their level.

100 Ideal: In addition, The blackboards can be easily seen by children (even from the back row). The chowkis are being used, and when not used, they are stored neatly, for easy access when needed

Reason for score

End time AM/PM

NUMBER OF CHILDREN TAKING THE LEARNING ASSESSMENT TEST

CLASS III CLASS V

4. FOCUS GROUP DISCUSSION WITH PARENTS & COMMUNITY

START TIME AM/PM

4.1. Nature of SDMC/VSS/VEC meetings

Reason for score

4.2. Nature of PTA or General Body Meetings

100 Ideal: In addition, PTA or GB is regularly monitoring quality of education in the school

Reason for score

4.3. Nature of meetings of other CBOs involved in school (e.g., MTA)

Name of CBO: ____________________________________________

Reason for score

4.4. How safe do girl children feel when they come to school?

0 All girl children do not feel safe, and do not come to school

25 Some girl children are not coming to school because they do not feel safe or comfortable in school

50 Benchmark: Most girl children come to school regularly, but some are staying back fearing safety

75 Most girl children are coming to school regularly, others come but irregularly due to safety concerns

100 Ideal: All girl children feel perfectly safe about coming to school and come to school.

Reason for score

4.5. How safe do SC/ST girl children feel when they come to school?

0 All girl children do not feel safe, and do not come to school

25 Some girl children are not coming to school because they do not feel safe or comfortable in school

50 Benchmark: Most girl children come to school regularly, but some are staying back fearing safety

75 Most girl children are coming to school regularly, others come but irregularly due to safety concerns

100 Ideal: All girl children feel perfectly safe about coming to school and come to school.

Reason for score

Ask these questions only for Schools with UNICEF QP

4.6. Response of parents to QP Kit

0 Parents have not been told about the QP kit and/or are unaware of it

25 Parents have been told and/or some parents are aware about the QP kit

50 Benchmark: All parents have been shown the QP kit when they come to school or all parents know about the QPA kit

75 In addition, some parents ask about the kit on subsequent visits

100 Ideal: All parents know about the QP kit and ask about when the kit is to be changed, replaced or improved

Reason for score

4.7. Reasons for children of Class I and II attending school

Reason Rank

1 Because of mid-day meal scheme

2 Because of new QP material

3 To learn

4 To play

5 Because their parents tell them to come

6 Not to be a nuisance at home!

7 Others (specify)

4.8. Improvements perceived in children of Class I and II by parents after QP

Improvements

4.9. What problems do parents see in the primary school? What suggestions do they have for improvement?

Problems Suggestions

ANNEXE 4: Stakeholder Assessment Questionnaire

Does UNICEF’s quality package lead to educational quality in primary schools?

1. What‘s the status of universal elementary education in the state?

Enrolment

Drop-out rates

Academic performance

2. What‘s the status of school (& class) infrastructure in the state?

Habitation specific

Teacher availability and attendance, etc.

RAISING benchmarks and norms around the quality of school and classroom infrastructure (for UNICEF and SSA)

3. What are the states‘ initiatives in promoting primary education of the ―excluded‖ communities?

Linguistic

Minorities

4. What are the states‘ initiatives in improving quality of elementary schooling?

Teacher training

Teacher support – Head teacher and CRC/BRC

Monitoring

Community participation in school management

5. How‘s QP doing? Has it helped improving quality? Six dimensions of QP?

I. In-service teacher training (TT) II. Academic support to improve active learning, continuous, supportive student

assessment and rational class management (AS) III. Improvement of classroom environment (CE) IV. Improvement of school environment and facilities (SE) V. Promotion of community participation (CP) VI. Development and supply of essential teaching-learning materials (TLMs)

Which components/dimensions are replicable?

Why? Why not?

6. How sound is the QP conceptualization?

What‘s missing in conceptualization

What‘s not working in reality

7. Have you ―seen‖ the teaching-learning materials (TLMs) in the QP?

How effective are these?

How good are the designs and product development?

How does they relate to the learning needs of students, local situation and as a tool for the teachers to improve learning?

8. How do teachers and concerned administrators regard the QP initiative?

Is the package relevant/replicable? What works (and doesn‘t work)

How are the TLMs used in the classroom?

Do they clutter the room? Where are they stored? Do the chowkis reduce space in the classroom? EXPLAIN: the usefulness of TLM (QP).

9. What role have the CRCs/BRCs played?

Do they provide academic support?

Do they conduct M&E? Are their reports used (useful)?

Don‘t they lead to another layer in the teachers‘ hierarchies

10. QP vis-à-vis SSA training

What‘s the value of QP training in the context of overall- in-service training? Is it effective? If yes, then state three reasons ―why‖.

Is the duration (3 days) of training adequate?

How should it be done?

11. Learning achievement of children?

Are they learning better with QP?

Do girls (or other excluded groups) do better under QP

If yes, then how? Does QP cater to the ―8 intelligences‖

If no, then why?

12. What is the overall impact of the QP?

Has quality of primary education become essential (no longer ―just desirable‖)

Which aspects of QP package are scalable through SSA? How should they be sequenced?

13. Adopting QP – Costs plus

Does quality of primary education come at a price? Is that affordable by the system?

For EPOs plus53:- Costs involved in making the shift from essential to the effective level of quality as per UNICEF‘s /GoI‘s guidelines

Senior SSA officials – SPD minus

ANNEXE 5: List of Schools

ANDHRA PRADESH: Chittoor District

Village School QP?

1 Basavaiahpalem Mandal Parishad Primary School QP 1

2 Chinna Raju Kuppam Mandal Parishad Primary School QP 2

3 Chinna singamala Mandal Parishad Primary School QP 3

4 Elkatur Mandal Parishad Upper Primary School QP 4

5 KBR Puram Mandal Parishad Primary School QP 5

6 Koppedu Mandal Parishad Upper Primary School QP 6

7 Koppedu DW Mandal Parishad Primary School QP 7

8 Munagalapalem Mandal Parishad Primary School QP 8

9 Papanaidupeta Mandal Parishad Primary School QP 9

10 Parlapally Madal Parishad Upper Primary School QP 10

11 Penkulapadu Mandal Parishad Primary School QP 11

12 Pillamedu Mandal Parishad Primary School QP 12

13 Pulla Reddy Kandriga Mandal Parishad Primary School QP 13

14 Rachagunneri Mandal Parishad Upper Primary School QP 14

15 Rachapalem Mandal Parishad Primary School QP 15

16 Ramasamudram Mandal Parishad Primary School QP 16

17 RV Kandriga Mandal Parishad Primary School QP 17

18 Sadasivapuram Mandal Parishad Primary School QP 18

19 SBV Puram Mandal Parishad Primary School QP 19

20 Shivanadhapuram Mandal Parishad Primary School QP 20

21 Thirumala Kuppam Mandal Parishad Primary School. QP 21

22 TMV Kandriga Mandal Parishad Upper Primary School QP 22

23 Urandur Mandal Parishad Primary School QP 23

24 Venkatapuram Mandal Parishad Upper Primary School QP 24

25 VS Puram Mandal Parishad Primary School QP 25

26 West Varattur Mandal Parishad Upper Primary School QP 26

27 Beerakuppam Mandal Parishad Primary School Non-QP 1

28 Gajulamandyam Mandal Parishad Primary School Non-QP 2

29 Illathur Mandal Parishad Primary School Non-QP 3

30 Indira nagar Mandal Parishad Primary School Non-QP 4

31 Mahaveerapuram Mandal Parishad Primary School Non-QP 5

32 Nagalapuram Mandal Parishad Upper Primary School Non-QP 6

33 Pannuru Mandal Parishad Primary School Non-QP 7

34 S V Nagar Mandal Parishad Primary School Non-QP 8

35 Sadasiva Shankara Puram Mandal Parishad Primary School Non-QP 9

36 SN Puram Mandal Parishad Primary School Non-QP 10

BIHAR: Vaishali District

Village School QP?

1 Amritpur Govt Middle School Amritpur QP 1

2 Attahaulapur Hindi Middle School Attahulapur QP 2

3 Attaullahpur Urdu Midddle School Attaullahpur QP 3

4 Balha Primary School Balha QP 4

5 Balua Basanta Govt Primary School Balua Basanta QP 5

6 Bharopur Dyodi Govt Middle School Bharopur Dyodi QP 6

7 Harivanshpur Primary School, Harivanshpur QP 7

8 Husaina Raghav SRAS Middle School, Husaina Raghav QP 8

9 Jahanabad Govt Middle School Jahanabad QP 9

10 Jahanabad Ghadma (Basanta) Primary School, Basanta QP 10

11 Kanchapura Primary School Kanchapura QP 11

12 Mayel Govt Primary School Mayel QP 12

13 Mile Pakri Middle School Mile Pakri QP 13

14 namidih middle school namidih QP 14

15 Packoli Adarsh Govt Middle School Packoli QP 15

16 Pakdi Nayatola Primary School Pakdi Nayatola QP 16

17 Panapur Dharampur Middle School Panapur Dharampur QP 17

18 Parshurampur Primary School Parshurampur QP 18

19 Pomiya Dehati Govt Primary School Pomiya Dehati QP 19

20 Prasidh nagar Government Middle School Prasidh nagar QP 20

21 Raimapur Primary school Raimapur QP 21

22 Rajasan Government Primary School Rajasan QP 22

23 Rjasan Primary School Rjasan QP 23

24 SaidpurGanesh Primary School SaidpurGanesh QP 24

25 Salempur UMS vidyalaya (Middle School) QP 25

26 Daulatpur Rel Middle School, Daulatpur Non-QP 1

27 Dhobaghatti Govt Middle School Dhobaghatti Non-QP 2

28 Dmaich Primary School Dmaich Non-QP 3

29 Gouda Primary School Gouda Non-QP 4

30 Hariharpur Shankar Govt. Primary School Hariharpur Shankar Non-QP 5

31 Indira Awas Dmaich Primary School Indira Awas Dmaich Non-QP 6

32 Mandai Deeh Govt Primary School, Mandai Deeh Non-QP 7

33 pahetia primary school, pahetia Non-QP 8

34 Rahimapur pethia primary school,Rahimapur pethia Non-QP 9

35 Thathan Buzurg Primary School Thathan Buzurg Non-QP 10

MADHYA PRADESH: Bhind district

Village School QP?

1 Angsouli GPS, Angsouli QP 1

2 Bagharai GPS (Girls), Bagharai QP 2

3 Baghora GPS, Baghora QP 3

4 Bagulari GPS, Bagulari QP 4

5 Bhaggu Ka Pura GPS, Bhaggu Ka Pura QP 5

6 Degwa GPS,Degwa QP 6

7 Dhamsa Ka Pura GPS, Dhamsa Ka Pura QP 7

8 Dhansa G GIRLS PS,Dhansa QP 8

9 Gigrkhi GPS,Gigrkhi QP 9

10 Habibpura GPS Habibpura QP 10

11 Hanumantpura Govt. Primary School, Hanumantpura QP 11

12 Jagannath Ka Pura GPS, Jagannath Ka Pura QP 12

13 Jaitpura GPS, Jaitpura QP 13

14 Jhankri GPS,Jhankri QP 14

15 Jogian ka pura EGS, Jogian ka pura QP 15

16 Kerwas Govt. Primary School, Kerwas QP 16

17 Khuman Ka Pura GPS, Khuman Ka Pura QP 17

18 Lahchurakapura GPS,Lahchurakapura QP 18

19 Mahavir Singh Ka Pura EGS, Mahavir Singh Ka Pura QP 19

20 Mukutsingh Ka Pura GPS, Mukutsingh Ka Pura QP 20

21 Mungadpura EGS,Mungadpura QP 21

22 Nirpura Government Primary School ,Nirpura QP 22

23 Patokhi Ka Pura GPS, Patokhi Ka Pura QP 23

24 Pawai GPS, Pawai QP 24

25 Pipahdi GPS (Boys), Pipahdi QP 25

26 Pithampura GPS,Pithampura QP 26

27 Sarva Govt Girls Primary School, Sarva QP 27

28 Aenthar GPS, Aenthar Non-QP 1

29 Barohi GPS, Barohi Non-QP 2

30 Daboha GPS (Boys), Daboha Non-QP 3

31 Imleda Govt. Primary School, Imleda Non-QP 4

32 Kheria Bagh GPS, Kheria Bagh Non-QP 5

33 Kiratpura GPS, Kiratpura Non-QP 6

34 Kuthonda Govt. Primary School, Kuthonda Non-QP 7

35 Pachera GPS (girls), Pachera Non-QP 8

ORISSA: Ganjam District

Village School

1 AMBAPUA PUPS,Ambapua QP 1

2 Badabaranga UP School, Badabaranga QP 2

3 Ballidi Ballidi UP School QP 3

4 Banka Project UP School, Banka QP 4

5 Biripur Project UP School Biripur QP 5

6 Brahmahatya Berhampur Nodal UP School, B.Berhampur QP 6

7 Chikitipentha Mahatap Nodal UP School QP 7

8 Chikitipentha (GPS) Girls Primary School, Chikitipentha QP 8

9 Chtradhepa Chtradhepa PS QP 9

10 Dhumchhai NUPS QP 10

11 G.Nuagoan Nodal Upper Primary School QP 11

12 Jagannathpur Jagannathpur project UP school QP 12

13 Jilliba UP School,Jilliba QP 13

14 K.Kardakana Nodal Upper Primary School QP 14

15 Kamakulamanipur Primary School, Kamakulamani pur QP 15

16 Kotilingi UP School, Kotilingi QP 16

17 Madhabandhapur Madhabandhapur UP School QP 17

18 Marudi Jagannathpur Primary school QP 18

19 Norendrapur Mendri Sahi UP School QP 19

20 Pritatali Project upper primary school, Pritatali QP 20

21 Punanda Govt. UP School,Punanda QP 21

22 Rampalli Project UP School, Rampalli. QP 22

23 Surla Primary School Surla Junction QP 23

24 Takarada Nodal UP School,Takarada QP 24

25 Taraipattapur Traipattapur UP School QP 25

26 Aska NAC Ex BD Girls PUPS Non-QP 1

27 Bamakai PUPS Non-QP 2

28 Gothagan Project UP school, Gothagan Non-QP 3

29 Jakara Nodal Kalua UPS Jakara Non-QP 4

30 Jillundi Primary School Jillundi Non-QP 5

31 Kalasandhapur U.P School, Kalasandhapur Non-QP 6

32 Khirapanka Manipadia UP School Non-QP 7

33 Kukudakhandi Nodal UP school, Kukudakhandi Non-QP 8

34 Narayanpur Narayanpur Primary School Non-QP 9

35 Polrajpur Upper primary School Non-QP 10

ANNEXE 6: Learning Ability Tests used in the Assessment

The Hindi versions of the abbreviated tests used in the field assessment are presented separately for Class 3 Language, Class 3 Mathematics, Class 5 Language and Class 5 Mathematics.

xq.koRrk f’k{kk dk;ZØe dk izHkko vkadyu

1- dkSu&lh isafly lcls cM+h gS\ ml ij fu’kku ¼√½ yxkvksA

2- lcls yach yM+dh ij fu’kku ¼√½ yxkvksA

3- vxj ,d caMy esa 10 isafly gSa] rks uhps ds fp= esa dqy feykdj fdruh isaflysa gksxh\

mRrj% _______

vof/k% 30 fefuV d{kk% rhljh ¼xf.kr½

Ldwy dk uke%

ftys dk uke% CykWd dk uke%

Nk=@Nk=k dk uke% lkekftd Js.kh% SC ST OBC Gen

4- mRrj fy[kks%

5- mRrj fy[kks%

6- mRrj fy[kks% 10 x 6

mRrj% _________

7- la[;kvksa ds Øe ¼iSVuZ½ dks ns[k] NwVh bqbZ la[;k,¡] [kkyh txgksa esa fy[kkA

3] 6] 9] 12] __, __

8- fp= ns[kdj fn[kk, x, flikfg;ksa dks fxuus dk lgh rjhdk dkSu&lk gS\ lgh fodYi

ij fu’kku yxkvksaA

v- 10 $ 5

c- 10 x 5

l- 5 $ 10

n- 10 x 10

9- oxZ ds vanj fu’kku ¼√½ yxkvksA

bl iqLrd ds iUus fdl vkdkj esa gS\ ml ij fu’kku

¼√½ yxkvksA

iz’u 1 ls 2 ds fy,] eSa tk 'kCn cksywa] ml lgh ¼√½ fu’kku yxkvksA

1 v- fnu c- rhu l- fny n- tku

2 v- NRrhl l- NRrjh l- NIij n- Nyuh

3 fp= dks ns[kdj lgh okD; ij fu’kku ¼√½ yxkvksA

v- ;g [kM+h gSA

c- og [kk jgh gSA

l- os dke dj jgs gSaA

n- og jks jgk gSA

4 fp= dks ns[kdj lgh okD; ij fu’kku ¼√½ yxkvksA

v- bl fp= esa pkj Qy gSA

c- pkjksa i'kq [kk jgs gSA

l- fp= esa [kjxks’k 'ksj vkSj Å¡V ds chp gSA

n- Å¡V xk; ds lkFk [kM+k gSA

vof/k% 30 fefuV d{kk% rhljh ¼Hkk"kk½

Ldwy dk uke%

uhps nh xbZ iafDr;ksa dks i<+dj iz’u la[;k 5 ls 6 rd ds lgh mRrjksa ij fu’kku ¼√½ yxkvksA

ikB’kkyk dh ?kaVh cthA lHkh cPps cLrs mBkdj] vius ?kj py fn,A 'khyk dh ek¡ mls ?kj

ys tkus ds fy, vkbZ FkhA tc os yksx ?kj igq¡ps rks ns[kk] mldh lgsyh jkuh mldk bartkj

dj jgh FkhA jkuh us 'khyk ls dgk fd vkvks] ge [ksy [ksyrs gSaA

5 ikB’kkyk dh ?kaVh ctus ij ’khyk dgk¡ xbZ\

v- [ksy ds eSnku esa c- cxhps esa l- vius ?kj n- pkph ds ?kj

6 jkuh dkSu Fkh\

v- 'khyk dh cfgu c- 'khyk dh Vhpj l- 'khyk dh ek¡ n- 'khyk dh lgsyh

iz’u 7 ls 8 ds fy,] eSa tks 'kCn ;k okD; cksyw¡] mls /;ku ls lqudj fy[kksA

____________________________________

9 fp= ds ckjsa esa ,d okD; fy[kksaA

_________________________

10 fp= ds ckjsa esa nks okD; fy[kksaA

__________________________

1- vadks esa fy[kks%

3 lSdM+k $ 7 bdkbZ =_________

2- 1] 7] 0] 8 vadks ls cuus okyh lcls cM+h la[;k fy[kksA

¼fdlh Hkh vad dk iz;ksx nqckjk u djsaA½

mRrj% _________×

3- gy djks%

4- loky dks lgh djus ds fy, [kkyh fMCcksa sea lgh la[;k fyf[k,A

vof/k% 30 fefuV d{kk% ikapoh ¼xf.kr½

Ldwy dk uke%

5- gy djks%

6- ,d isM+ ij 22 fpfM+;k¡ cSBh FkhA 3 fpfM;k¡ mM+ xbZ] ckn esa ,d vkSj fpfM+;k¡ vkbZ vkSj isM+

ij cSB xbZA vc isM+ ij fdruh fpfM+;k¡ gS\

mRrj% _________

7- gy djks%

8- gy djks%

3 0 6 2

9- gy djks% 360 ÷ 32

HkkxQy% _________ 'ks"kQy%_________

10- jke yM~Mwvksa dh 6 FkSfy;k¡ ysdj vk;kA mlus yM~Mwvksa dks vius 9 nksLrksa esa cjkcj&cjkcj

ck¡V fn;kA izR;sd nksLr dks 2 yM~Mw feysaA crkvksa izR;sd FkSyh esa fdrus yM~Mw Fks\

mRrj% _________

iz’u 1 ls 2 ds fy,] eSa tk 'kCn ;k okD; cksywa] mls /;ku ls lqudj fy[kksA

1 __________________

2 __________________

3 Åij fn, x, fp= ds ckjsa esa 2 okD; fy[kksA

1. __________________

2. __________________

vof/k% 30 fefuV d{kk% ik¡poh ¼Hkk"kk½

Ldwy dk uke%

4 Åij fn, x, fp= ds ckjsa esa 4 okD; fy[kksA

1. ____________________________________

2. ____________________________________

3. ____________________________________

4. ____________________________________

iz’u 5 ls 6 ds fy,] lgh 'kCn pqudj [kkyh txg esa fy[kksA

5 xk; vkSj cdjh dh nkSM+ esa xk; vkxs fudy xb] cdjh &&&&&&&& jg xbZA

6 esjs ikl ,d dsyk gS] rqEgkjs ikl fdrus &&&&&&& gSa\ ¼dsyks@dsys½

iz’u 7&8 ds fy,] eSa tSlk dgw¡] oSlk djksA

uhps nh xbZ iafDr;ksa dks i<dj iz’u 9 vkSj 10 ds lgh mRrjksa ij fu’kku ¼√½ yxkvksaA

D;k rqeus dHkh e/kqeD[kh ns[kh gS\ ;fn rqe mls /;ku ls ns[kksxs rks rqEgsa mlds iSjksa

ij dbZ NksVs& NksVs cky fn[kkbZ nsaxsaA mu ckyksa esa dqN /kwy ds d.k lh fn[kkbZ

iM+us okyh pht fpidh jgrh gSA ;s /kwy ds d.k ijkx dgykrs gSaA tc e/kqeD[kh

Qwyksa ij 'kgn bdV~Bk djus tkrh gS rks ;s d.k mlls fpid tkrs gSaA e/kqeD[kh

Qwy& Qwy vkSj ikS/ks&ikS/ks mxkus esa lgk;d gksrs gSA bl izkdj e/kqeD[kh dks [kkuk

rks feyrk gh gSa vkSj lkFk esa u, ikS/ks Hkh mx tkrs gSaA

9 ;s NksVs&NksVs cky dgk¡ gksrs gS ftuesa ^ijkx* fpids gksrs gSa\

v- Qwyksa ij c- [kkus ij l- ijkx ij n- e/kqeD[kh ds

iSjksa ij

10 ijkx D;k djrs gSa\

v- ;g ikS/kksa dk

[kkuk gSA

c- ;g dsoy /kwy

l- u, ikS/ks

mxkus esa lgk;d

gksrs gSaA

n- e/kqeD[kh dks

[kkuk izkIr djkrs

QUANTIFIED PARTICIPATORY ASSESSMENT OF THE IMPACT OF … · 2019-06-07 · 1 IMaCS Pilot Assessment...

Documents