Post on 01-Jan-2017
transcript
An assessment of students’ competency levelsin Grades 3-9 in Northern and Eastern Provinces, March 2010
An assessment of students’ competency levels
in
Northern and Eastern Provinces,
March 2010
Acknowledgements
UNICEF would like to acknowledge the role of the Northern and Eastern Provincial Ministries
of Education in conducting this large scale assessment in 2010. We thank in particular the
Northern Provincial Secretary of Education, Mr. L. Illangovan, Mr. Rasiah, former Provincial
Director of Education (PDE) and Mr. Rathakrishnan, Assistant Provincial Director of
Education (APDE) of the Northern Province; and Mr. Thandayuthapani, former PDE and Mr.
Murugupplilai, former Deputy Director of Education, Development (DDE (Dev) of the Eastern
Province for their sustained commitment to this undertaking and their close involvement
with the design and conduct of the assessment.
UNICEF would also like to thank the officers from the Primary Department of the National
Institute of Education (NIE), and the officers and teachers from the Northern and Eastern
Provinces who contributed to the design and finalisation of the assessment tools. Finally,
UNICEF would like to thank the author of this report, Dr. David Carroll, who generously
shared his considerable expertise while based in the Eastern Province.
This assessment was funded by the Government of the Netherlands as part of its generous
support to UNICEF’s Education in Emergencies and Post-Crisis Transition (EEPCT) program in
Sri Lanka.
Contents
Executive Summary ............................................................................................................... 1
Background ............................................................................................................................ 3
Planning the Testing Programme .......................................................................................... 5
Proposed Strategy for Developing Test Content ............................................................... 5
Proposed Strategy for Test Delivery .................................................................................. 6
Numbers to Be Tested ........................................................................................................... 6
Northern Province .............................................................................................................. 7
Eastern Province ................................................................................................................ 7
Reference Group ................................................................................................................ 8
Plan for Testing in Schools ..................................................................................................... 9
Test Design in Tamil and Mathematics ................................................................................. 9
Assessment of Basic Competencies ..................................................................................... 10
Guide to Assessment for Grade 1 and 2 Teachers .............................................................. 11
Test Analysis ........................................................................................................................ 11
IRT Analysis ...................................................................................................................... 11
Setting Placement Criteria ............................................................................................... 12
Outcome – Administration and Scoring .............................................................................. 13
Results of the Reference Group: Tamil and Mathematics Tests ......................................... 13
IRT Scaling ........................................................................................................................ 14
Item Results ..................................................................................................................... 18
Results of the Reference Group: Assessment of Basic Competencies (ABC) ...................... 20
Results by Grade .............................................................................................................. 21
Frequency Distributions ................................................................................................... 21
Reading Achievement and Problem Solving .................................................................... 22
Achievement on ABC by School ....................................................................................... 23
Intra-Class Correlation on the ABC .................................................................................. 24
Analysis of the Target Group – Tamil and Mathematics ..................................................... 24
Setting Cut Scores ................................................................................................................ 30
Diversity in Performance between Schools ......................................................................... 36
Analysis of the Target Group – Basic Competencies ........................................................... 40
Exclusions ......................................................................................................................... 40
Overall Mean of Included Cases ...................................................................................... 40
Grade-wise Mean and SD of Included Cases ................................................................... 41
Frequency Distributions of Included Cases ...................................................................... 42
Achievement by School .................................................................................................... 44
Conclusions and Recommendations ................................................................................... 45
Annexes ................................................................................................................................... 49
Annex 1: Mathematics Item Map – Item Prefix Deleted ................................................. 50
Annex 2: Tamil Item Map – Item Prefix Deleted ............................................................. 51
Annex 3: Supplementary Tables for Assessment of Basic Competencies ....................... 52
Annex 4: Computing the intra-class correlation .............................................................. 56
Annex 5: Summary of Design of System of Placement Tests in a Given Subject ............ 58
Annex 6: ABC Tests .......................................................................................................... 59
Annex 7 Assessment Guide for Grade 1 and 2 Teachers ................................................. 73
Acronyms and Abbreviations
ABC Assessment of Basic Competencies
ALP Accelerated Learning Programme
CFA Cease Fire Agreement
EFA Education for All
ELC Essential Learning Competencies
GoSL Government of Sri Lanka
IDP Internally Displaced Person
IRT Item Response Theory
LTTE Liberation Tigers of Tamil Eelam
NIE National Institute of Education
PDE Provincial Department of Education
TLS Temporary Learning Shelter
1
Executive Summary
This report documents the development, delivery, scoring and processing of a set of tests of
Tamil and mathematics for placing displaced and returned learners in the Accelerated
Learning Programme (ALP). The planning of the programme, and the strategy adopted for
developing, delivering and processing the tests, is described in section 2. Section 3 outlines
the numbers to be tested, and the selection of a “reference group”, a group of non-displaced
learners who formed the basis of the standard setting. The purpose of this group was to set a
standard which reflected what the learners tested could reasonably be expected to have
achieved, had they not been displaced. Section 4 outlines the broad time requirements for
the testing, while section 5 describes the broad structure of the tests in terms of the
outcomes assessed.
Section 6 describes the Assessment of Basic Competencies (ABC), a test of basic literacy and
numeracy administered to secondary-level test takers; and section 7 outlines the procedure
proposed for grade 1 and 2 learners. In the event, this latter procedure was not adopted; it
was assumed that all displaced persons of grade 1 and 2 age would need to participate in the
ALP Foundation Programme (an assumption supported by the testing of higher grades).
Section 8 outlines briefly the planned procedure for setting a standard and analysing the
results, based on pooling the Reference Group data from all grades, and using Rasch one-
parameter IRT. Section 9 describes the outcome of the testing, in terms of administration and
scoring.
Section 10 gives the results of the analysis of the Reference Group in Tamil and mathematics,
on a single scale covering grades 3 to 9. It shows the gradient of achievement from year to
year, which in mathematics shows a reasonably steady increase (with the exception of grade
4); but in Tamil shows a sharp drop from grade 5 to grade 6. It also shows that the diversity in
achievement is much greater than the increase in achievement from year to year. The quality
of the tests is shown to have been good, and their difficulty appropriate to the achievement
of the Reference Group.
Section 11 outlines the results of the ABC, individually and by school. Overall, reading ability
is not a major problem in the Reference Group; although there is a minority, which may be as
large as one in five of the test takers, who clearly do have a problem with basic skills. Since
they are studying in grades 6 to 9, this should not be; and it would be desirable if some
attention were paid to improving the basic skills of this minority, both before they enter grade
6 and (for those already in grades 6 to 9) during their secondary education. The section also
shows that there are some schools where a lack of basic skills is quite widespread.
Sections 12 through 15 report the results of the “main” testing. The results showed a sharp
disparity in achievement between the Reference Group schools and the schools in Eastern
Province selected for possible inclusion in the ALP. It also shows considerable variability
2
within this group of schools, however, suggesting that the delivery of the ALP may need to
take account of the performance of schools as well as individual learners. In Northern
Province, the overall level of achievement was comparable with that in the Reference Group;
but there was a significant achievement deficit associated with any form of displacement –
about three years’ learning for those still in IDP camps, or in schools which had restarted, and
about half that for those who had been displaced, but found places in schools which had
continued to function normally. The impact of displacement was also greatest amongst the
younger learners. Again, there was considerable diversity in average achievement from
school to school, especially in Eastern Province. Although the data for the ABC test are less
complete than those for the Tamil and mathematics tests, the results show significant
numbers of learners who lack the required level of literacy and numeracy.
3
Background
Since the introduction of free universal education in 1945 and its independence in 1948, Sri
Lanka’s well-established education system has been distinguished by progressive policies and
impressive achievements in primary enrolment and gender parity leading to a comparative lead
in the region towards the realization of the Millennium Development Goals, namely Goal 2
(Basic Education) and Goal 3 (Gender Equality). Although Sri Lanka’s overall statistics look
promising, they obscure substantial human development challenges and regional discrepancies
in the provision, utilization and outcomes of education particularly in areas of former conflict.
After the abrogation of the Cease Fire Agreement (CFA) between the Government of Sri Lanka
(GoSL) and the Liberation Tigers of Tamil Eelam (LTTE) in January 2008, the government
intensified the war in LTTE-controlled areas of the country. The ensuing fluidity of the military
situation resulted in the frequent and unexpected displacement of civilians and the rapid
deterioration of public institutions and facilities in the conflict areas. Children were particularly
vulnerable under these conditions and the stability and quality of their education was at severe
risk.
In late 2008 and early 2009, a succession of LTTE military setbacks gradually brought LTTE-
controlled areas in the North under government control and culminated in the military defeat of
the LTTE in mid-May 2009. This period of intense conflict generated over 280,000 Internally
Displaced Persons (IDPs) adding to the existing caseload of nearly 300,000 IDPs. At the height of
displacement in 2009, approximately 260,000 IDPs1 were residing in the IDP camps, transit sites
and host schools of Vavuniya. The remaining IDPs were housed under similar circumstances in
the districts of Trincomalee, Jaffna and Mannar.
As a result of advocacy by the education cluster to consider education as a right to which all
children should be entitled despite their circumstances, the Sri Lankan Army had made it a
priority by May 2009 to secure spaces for schools within the new camps. In Jaffna, cluster
advocacy and negotiation enabled many IDP students to attend schools outside of the camps
and others were able to join classes in host schools thus promoting a sense of normalcy and
gradual healing in the aftermath of the protracted conflict. In total, approximately 80,000
students representing 95 per cent of the school-aged population (5-14 years) attended school in
IDP camps.
In October 2009, after nearly six months of mostly sporadic releases of IDPs from the camps, the
Government accelerated the process increasing the number of people returning to 2,000 per
day. By the end of February 2010, over 174,000 people had been released from the camps and
returned to their districts of origin.
1 Reported by Office for Coordination of Humanitarian Affairs, 31 August 2009
4
Conditions for Testing in Gamini School
During the height of the IDP influx into camps the Ministry of Education, in collaboration with
UNICEF, supported the Vavuniya Zonal Education Office to develop and administer a rapid
assessment of IDP children in Vavuniya South and Mannar districts. The assessment indicated
that many children were at or below the minimum learning competency for their age group and
would require a specific program to help them catch up on education lost during the conflict.
Although the security restrictions within the camps were not conducive to large scale
interventions at that time, from the end of 2009 UNICEF has assisted the Ministry of Education,
the National Institute of Education
and the Northern and Eastern
Provincial Departments of Education
(PDE) to implement key activities for
an Accelerated Learning Programme
(ALP) to address the learning needs of
these students.
The ALP is intended to be a temporary
school-based intervention to help
students rapidly reach their age-
appropriate learning competency thus
promoting grade 1-9 retention. The
design of the ALP will allow those
students who have fallen behind in
education to enter the programme at
the appropriate designated
competency level regardless of age.
They will exit the programme once
they have reached their age-appropriate learning competency and transition back to the
corresponding grade level in the formal system. The design of the ALP ensures that older
students will be able to sit for Ordinary level and Advanced level examinations with minimal
disruption.
A programme of placement testing was therefore needed. This could have a dual purpose. The
primary purpose was to determine who would be able to benefit from entering ALP, and at what
level. On the basis of their Placement Test results, students may either remain in their original
grade in mainstream education, or be placed within ALP at a level which reflects their actual
achievement, and gives them an opportunity to develop essential competencies that they lack.
The second purpose of testing was as a planning tool, giving guidance on the demand for ALP in
different locations, and in particular the requirement for ALP classes within specific schools.
5
Planning the Testing Programme
It was agreed in advance that the testing would take place in Tamil and mathematics only, at
grades 3 through 9. Students in grades 1 and 2 would not be tested, but assessed by their
teachers. A brief guide would be provided to these teachers, in case they did not feel able to
determine independently which students were in need of entering the ALP. In addition to the
subject-based tests, there would be a test of basic literacy and numeracy for learners in
secondary education only, to identify those who are in particular need of training in basic skills.
Initial meetings were held with the senior managers of the two provincial education ministries,
to determine how the tests would be developed, administered, marked and analysed. The basic
plan as agreed was as follows:
Proposed Strategy for Developing Test Content
Test items would be developed as quickly as possible, using a five-step process:
(i) Develop an agreed set of outcomes for grades 3-9
The ALP team would work with the NIE experts to agree a set of outcomes on which the
items would be based, reflecting any elisions the ALP team may seek to introduce in the
ELCs, subject to the agreement of the NIE specialists.
(ii) Develop draft items
Draft items would be developed at a workshop with subject specialists from Northern
and Eastern Provincial Departments of Education. A core group of the international
testing consultant plus at least one national specialist in each subject would review and
revise the items, and select those which may be suitable for further use.
(iii) Pilot items
Limited pilot testing of items was planned, to assure their quality, using “traditional”
item analysis2.
(iv) Review items for estimated difficulty
A group of expert judges (mainly teachers) will review items using (e.g.) a modified
Angoff rating procedure.
(v) Select items for instruments
Using this information, a good set of items will be selected for each grade. A test for a
given grade will be assembled out of the items selected for the preceding two grades.
2 In the event, the items were not in fact piloted.
6
Item types will need to be considered carefully – and in particular, the extent to which
multiple-choice can be used. Answers to constructed-response questions will be kept as
brief as possible, with as little scope for variation as possible. Where (i.e., in Tamil)
there has to be an extended response, it will be carefully structured.
Proposed Strategy for Test Delivery
As noted above, testing would take place in Tamil and mathematics only, with a test of basic
competencies for grades 6 through 9. In Tamil and mathematics, there would be a test in each
subject covering the outcomes for the previous two grades, plus a basic literacy test common to
all (derived from an outside source). Tests would be taken by all students in participating
schools, not only the recently-returned.
There would be three main stages in the testing:
(i) Administer tests in schools
The tests would be administered in and by the schools, with some quality assurance by
the zone and UNICEF. Marking would be done in and by the schools, using a template
supplied as part of the manual of instructions for test administration.
Results would be tabulated by the schools, used for allocation to ALP or mainstream
schooling following an agreed procedure, and also tabulated and further analysed
centrally for programme planning purposes.
(ii) Centrally analyse a sample of test data
In addition to the scoring of tests by schools, a sample of item-level test data from
Tamil-medium schools less affected by the conflict would be gathered and analysed to
lay down a baseline, and also to investigate general levels of achievement in Tamil-
speaking areas.
(iii) Develop items for post-testing
Post-testing is required both for exit from the programme and for impact assessment. A
procedure would be developed for writing items and assembling tests; however the
procedure may not be fully implemented due to shortness of time, and it may be
necessary to find a national institution to complete the development, administration
and reporting of end-of-level tests.
Numbers to Be Tested
The original plan had been to test only those learners who were in the IDP camps; in the event,
the senior managers of the two Provincial Ministries were convinced that the problems were
much more widespread. Also, in the period while the testing was being planned, the
7
Government of Sri Lanka instituted a major programme of returning IDPs to their home areas.
Northern Province
The Northern PDE management felt that the damage to the Province was so extensive that only
isolated areas such as parts of Vavuniya South Zone could be considered to have been spared
major trauma. They therefore requested that all learners in Tamil medium education in the
relevant grades be tested.
The exact numbers and locations of the learners to be tested were in this case difficult to
determine. In some areas, many, if not most, schools had been closed during the conflict, and
severely damaged. Many students had been displaced, and were either attending TLSs or not
attending school at all. The Provincial Department of Education was able to provide numbers
from before the conflict, which were taken as an upper limit on the total number likely to
present for testing; but because so many families were in the process of returning to their
previous homes, or being resettled, it was not able to be very specific about their locations.
Thus, it was agreed to use the most recent figures the Provincial Department of Education could
obtain, and to give Zonal managers an extra supply of test booklets for distribution as necessary,
and to give the largest numbers to the Zonal managers in the areas of greatest displacement.
The basic strategy proposed was as follows:
Zone Notes
Jaffna 100 schools plus 200 booklets per grade to zone
Valikamam 130 schools plus 200 booklets per grade to zone
Thenmaradchchi 56 schools
Vadamaradchchi 67 schools
Island 50 schools plus 200 booklets per grade to zone
Mannar 69 schools
Madhu 0 schools; 900 booklets per grade to zone
Kilinochchi 25 schools
Mullaitivu 8 TLSs plus 500 booklets per grade to zone
Thunukkai 30 schools
Vavuniya North 0 schools; 1000 booklets per grade to zone
Vavuniya South 71 schools
Eastern Province
In mid-2006 military offensives in the Eastern Province displaced more than 170,000 people who
previously lived in areas controlled by the Liberation Tigers of Tamil Eelam (LTTE). Over 44,000
families were displaced from Batticaloa, and a further 7,428 families from
8
Trincomalee. By the third quarter of 2007 the entire Eastern Province had been brought under
the control of the Government and a resettlement process was initiated, and the Provincial
Department of Education, Eastern Province therefore had a more stable idea both of which
schools needed to be tested, and of the numbers of students attending the schools. Testing was
limited to parts of six zones, as follows:
Zone Division Notes
Trincomalee Kuchchaveli 2 schools only
Thampalahamam 2 schools only
Mutur Eechchilampattu
Mutur 11 schools only
Kalkudah Koralaipattu
Koralaipattu N
Batticaloa Manmunai N
Manmunai W
Thirukkovil Thirukkovil
Paddirippu Manmunai SW
Porathivupattu
The planned total numbers to be tested were therefore in the region of 140,000, rather than the
85,000 originally envisaged; and the populations to be tested were dispersed across hundreds of
schools rather than being contained in a limited number of IDP camps. These two factors
greatly increased the challenge of delivering the testing programme.
Reference Group
For purposes of setting a standard, it was proposed to evaluate the performance of the
displaced students against the actual performance of a Reference Group of learners from
mainstream schools who have not suffered displacement. This Reference Group was not
intended to reflect the national standard of achievement, because achievement studies carried
out during the conflict have consistently showed Northern and Eastern Provinces as the two
lowest-achieving in the nation. Instead, the Reference Group was intended to reflect the typical
performance of Tamil-medium students in Northern and Eastern Provinces, who had not
suffered significant ill-effects from the conflict, as being more typical of what the displaced
students might have been expected to achieve if they had not been displaced.
Although part of one zone of Northern Province – Vavuniya South – was not so severely affected
by conflict as the remainder of the province, and it would therefore have been possible in
principle to find non-displaced students there to form part of the Reference Group,
9
this was not in fact done, for two main reasons. First, the Northern PDE had asked for all
schools in Northern Province to be included in the main testing. Second, the Northern PDE was
still coping with massive disruption across the whole province; and Vavuniya South Zone as a
whole had significant numbers of IDPs, not merely in camps but also in many schools. The PDE
would therefore have struggled to find suitable schools, and the effort involved would have
been disproportionate to the benefit. This was not the case with Eastern Province, and
therefore, the Reference Group was chosen from Eastern Province only. Two divisions which
were considered by the Provincial Department of Education to have been less affected by the
conflict were chosen to comprise the Reference Group – Eravur (Batticaloa Central Zone) and
Manmunai South Eruvilpattu (Paddirippu Zone). It was proposed that the data from these two
divisions would be analysed at the item level using item response theory (IRT), and scaled score
tables produced.
Plan for Testing in Schools
The plan was for placement testing to be completed within one school day, and the duration of
the tests in any grade to be related to the length of typical final examinations for the grade. The
planned duration of the testing was as follows:
Grades 1-2: Guided assessment by teacher.
Grade 3: 30 minutes’ testing time in each subject (total approximately 1 hour plus time
for administration etc.).
Grade 4: 45 minutes’ testing time in each subject (total approximately 1½ hours plus time
for administration etc.).
Grade 5: 1 hour’s testing time in each subject (total approximately 2 hours plus time for
administration etc.).
Grades 6-9: 1½ hours’ testing time in each subject plus basic skills test (total approximately
3½ hours plus time for administration etc.).
Test Design in Tamil and Mathematics
Given the purposes set for it, the ALP testing programme was required both to distinguish
between learners who can remain in mainstream schooling, and those who need to enter ALP,
and also to direct learners to the level within ALP that is most appropriate for them.
This was addressed by making the tests modular. The test taken by a learner in a given grade
contains items based on the content of each grade within any level the learner may enter, as
follows:
10
Test
Grade
Competencies by Key Stage and Grade
Found. KS1 KS2 KS3 KS4 KS5
1 2 3 4 5 6 7 8
1
2
3
4
5
6
7
8
9
Thus, learners in a particular grade would in general be tested on competencies from the
previous three or four grades (grade 9 is not included because learners had not covered it at the
time of testing). The items within the placement test for a given grade were grouped in blocks
according to the grade they test, and these blocks are arranged within the test paper in
ascending order of grade. This means that while the test a learner takes depends on the grade
the learner is in, there is a considerable amount of common content in tests for different grades.
Assessment of Basic Competencies
There was substantial anecdotal evidence that lack of basic skills was a problem in at least some
secondary schools. Therefore, it was agreed to administer a test of basic literacy and numeracy,
at or below the grade 4 level, to all students tested in grades 6-9.
A simple test of basic competencies was developed, based on the model of the ABC test used in
Bangladesh to assess basic competencies in the 1999 “Education Watch” survey, and the EFA
2000 assessment. The test assessed basic reading, writing and numeracy. The test was however
reduced in scope by comparison with original ABC test, both because of the requirements of
administering it as a group test, and because the “life skills” element of the original ABC test was
deemed irrelevant in the present context. A draft test was developed in English, covering
reading comprehension, dictation, letter writing, and arithmetic using the four basic operations,
including word problems and mental arithmetic. This English draft was reviewed by the NIE, and
some changes were made. The resulting final draft was translated into Tamil, and reviewed
again before being finalised.
The text of the Assessment of Basic Competencies, in English and Tamil, is given in annex 6
below.
11
Guide to Assessment for Grade 1 and 2 Teachers
A brief and simple guide for teachers on how to determine whether a grade 1 and 2 student
belongs in the foundation level of the ALP or in mainstream schooling was prepared. A set of
key competencies were identified in foundation level, and grade 1 in Tamil and mathematics.
Simple assessment tasks were then developed linked to these competencies, with guidelines on
how to assess whether the competency has been mastered, a scoring template and cut-off
scores.
The text of the guide is given in annex 7 below. It should be noted that the final text of this
document exists only in Tamil language, as significant sections of it relate to the Tamil language,
and so were never translated into English.
Test Analysis
The modular design of the Tamil and mathematics tests in principle allows for two basic
approaches to analysis and standard-setting. It is possible to score each module separately, and
give each student a score on each of the two preceding key stages in primary education and, in
the case of secondary education up to four of the preceding grades, setting the standard in
terms of average performance of the actual test takers of the relevant grade. Alternatively,
since all the tests have substantial amounts of common content, it is possible to combine the
data from all seven grades of the Reference Group into a single dataset, calibrate that dataset
using IRT, and on that basis define a single scale covering all seven grades.
Although the first approach is initially attractive, as giving a better indication of exactly what
teaching any individual has missed, it is in practice less reliable than the “global” IRT scaling,
because the module scores are based on far fewer items than scores from the whole test. It is
also in fact less informative, as well as much more labour-intensive. IRT scaling allows scores
from different grades to be put on the same scale. This means that scaled scores for all tests,
regardless of grade, are expressed in the same units, and so scaled scores achieved on different
tests can be directly compared, and any score on any grade can be interpreted in terms of
where it would stand in the rank order of any other grade.
IRT scaling is also administratively convenient, because results can be reported based on the
entire range of achievement, not the achievement a single grade.
IRT Analysis
The data for all seven grades were merged into a single data file. This involves giving each item
a unique ID, based in this case on the grade for which it was originally written, and its position in
the first test in which it appears. The data files for individual grades were then arranged so that
the items were in the same order in each, and items which did not appear in a given grade were
replaced by blank columns. Thus, each item appeared in the same column of each data file in
12
which it appeared. The files for the different grades could then be merged smoothly, resulting
in a single data file covering all seven grades, where each column contained information about
the same item, regardless of grade. ID information was also included – province, zone, division,
school and grade.
Where an item was answered correctly, the relevant column was marked one (or more in the
case of a partial-credit item). Where the question was answered wrongly, the column was
marked zero. Where an item was “not attempted” by a test-taker, the column was also marked
zero, even though the test taker had written nothing. Where an item was “not presented” to a
test taker, i.e., the item was not in the test for the relevant grade, or the relevant page had been
left blank by the printer, the relevant column was left blank. This allowed the software to
distinguish which items were included in a given test, and which were not. It also meant that
test takers whose papers were incomplete because of a printing error or some other kind of
problem could be included in the analysis – they would still receive an estimated achievement
based on the items they answered, although it might be less reliable than one based on the
whole test.
The data set was then calibrated as a whole using WINSTEPS software. Item difficulties and
goodness-of-fit statistics were computed based on all the test takers who were presented with
an item, regardless of grade. Person achievement estimates were based on all the items a test
taker was presented with, regardless of grade. All reported statistics were based on this
analysis. A separate score table was however computed for each grade separately (although the
achievement estimates are on the same scale regardless of grade, so achievement estimates
derived from tests for different grades can be compared directly. These score tables can be
used to include future test takers in the placement process.
Setting Placement Criteria
Only the Reference Group was calibrated using Rasch one-parameter IRT. The purpose of this
was to give a standard of comparison for candidates for the ALP. Those learners who fail to
reach a specific level on the test for their grade can be placed in the ALP. Those who are placed
in the ALP can be placed in a specific level according to the achievement of the tested group as a
whole, regardless of age.
In addition to taking account of the full range of learner achievement in setting cut-off scores for
placement in given levels of the ALP, placement in the programme can take account of
provision. All available test data for the ALP candidates were analysed and tabulated by zone,
division and school. The proportion of test takers achieving a given level on the test was also
tabulated. This additional distributional information can be used at local level to decide which
schools should have an ALP section, or which should be turned over wholly to the ALP – and to
identify available places within an appropriate distance for individuals seeking to enter the ALP.
13
Outcome – Administration and Scoring
Testing was originally scheduled for February 2010. In the event, this proved not to be feasible,
and testing took place over an extended period during March 2010, finally being completed by
April 2nd 2010.
The scoring and data entry had two main stages:
a) Immediately after the testing was completed, the booklets for the Reference Group were returned to UNICEF in Trincomalee, and entered to computer there.
b) For the remainder of the data, where a school or a zone was able to enter data locally, this was done. These data were then checked centrally and if necessary, re-entered.
Data entry for the Reference Group was completed early in April, and analysis of these data was
completed by mid-April. Entry of the main data proved more challenging, and was not
completed until mid-September 2010.
Results of the Reference Group: Tamil and Mathematics Tests
Tables 1 and 2 below give the mean raw scores by grade in mathematics and Tamil respectively.
In Tamil, the mean raw score ranges from about 60% up to around 75%, suggesting both that
the tests as developed were appropriate to the ability of the learners, and that the learners in
general are achieving an acceptable level of competence.
Table 1: Mean Raw Scores by Grade – Mathematics
Grade N Mean SD % Correct
3 1702 21.35 6.32 76.24
4 1705 39.53 16.70 54.16
5 1668 63.09 20.50 67.84
6 1457 52.10 23.79 58.54
7 1498 58.73 24.87 55.93
8 1638 17.87 10.70 38.85
9 1599 21.08 11.58 39.04
In mathematics, however, the tests were on average slightly more difficult, and the mean scores
on the grade 8 and 9 tests were well below 50%, suggesting that the content is more
challenging, especially in grades 8 and 9 – and to the extent that the tests measure as intended
the most important competencies, that even in the Reference Group, learners are struggling to
achieve the desired outcomes in the higher grades.
14
Box 1 – Progression in Mathematics
Table 2: Mean Raw Scores by Grade – Tamil
Grade N Mean SD % Correct
3 1741 27.13 9.44 73.33
4 1773 38.22 12.09 74.95
5 1664 68.66 21.74 73.83
6 1613 66.63 29.88 65.97
7 1549 90.15 39.11 59.31
8 1641 72.12 32.34 61.64
9 1532 88.98 33.73 60.53
Individual item difficulties were not estimated grade-wise, for reasons which will be clear from
the account of the scaling procedure given in section 2 below.
IRT Scaling
As noted in section 8 above, the modular design of the Tamil and mathematics tests, with
substantial common content from grade to grade, allowed all Reference Group data, from all
seven grades, to be combined into a single dataset, and calibrated using Rasch one-parameter
item response theory (IRT). This led to definition of a unified scale of estimated achievement
covering all seven grades, with score tables being produced for each grade-wise test yielding
achievement estimates on this common scale.
Since this “global” IRT scaling allows scores from different grades to be put on the same scale,
scaled scores for all tests,
regardless of grade, are
expressed in the same units,
scaled scores achieved on
different tests can be directly
compared, and any score on
any grade can be interpreted in
terms of where it would stand
in any other grade. This means
that the amount of progress the
average learner makes from
grade to grade can be
estimated, and the
achievement of any individual
within the target group can be
related to
15
the distribution of achievement in any other grade, to determine where they should be placed;
and the average achievement of any group can be estimated, to determine the starting point for
instruction.
Individual results are expressed as estimated achievement, on a scale with a notional mean of
zero, and standard deviation of one. In practice, because of the very wide range of achievement
to be scaled, the mean is not quite zero in either case; and because the scores are not normally
distributed, but rather widely spread across the score range, with many more high and low
scores than would typically be found in a single-year group, the standard deviation is much
larger than one. The grand mean and standard deviation of the achievement estimates is given
in Table 3 below.
Table 3: Mean Estimated Achievement for the Reference Group
Subject N Min. Max. Mean SD
Tamil 11513 -7.43 6.74 .7534 1.96
Mathematics 11267 -8.20 6.38 .1105 1.88
The progression in mean mathematics achievement from grade to grade can be seen from Table
4 below, and Box 1.
Table 4: Mean Achievement Estimates by Grade, Mathematics
Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
Mean -0.41 -0.82 0.03 0.12 0.25 0.58 1.03
SD 2.04 1.58 1.70 1.90 1.82 1.88 1.68
Min. -8.00 -7.51 -8.21 -6.81 -6.82 -4.73 -4.74
Max. 2.79 4.10 5.44 4.86 5.22 6.38 6.13
With the exception of grade 4, there is a roughly regular increase in mean achievement from
grade to grade. It should be noted that there is progression in achievement in grades 8 and 9,
thus indicating that any problem there may be with test difficulty in grades 8 and 9 (as
suggested above) is not due to low achievement amongst grade 8 and 9 learners, but either a
steep gradient in the curriculum, or test items that were unexpectedly difficult.
The progression from grade to grade in mean Tamil achievement, however, is rather different,
as can be seen from Table 5 below, and Box 2. There is regular progression within a stage, in
that mean achievement increases from grade 3 to grade 4, and grade 4 to grade 5, and again
from grade 6 to grade 7, to grade 8, to grade 9. However, between grade 5 and grade 6, there is
a sizeable drop in mean achievement. Mean achievement at grade 6 level is in fact estimated to
be lower than at grade 3 level.
16
Box 2 – Progression in Tamil
Table 5: Mean Achievement Estimates by Grade, Tamil
Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
Mean 0.47 0.65 1.33 0.12 0.56 0.94 1.22
SD 2.18 1.96 2.01 1.77 1.81 2.00 1.56
This drop is counter-intuitive; development in language achievement between grade 3 and
grade 6, or between grade 5 and grade 9, is very significant. But it is very difficult to explain.
There is little reason to ascribe it
to the measurement process,
since there is significant overlap
of content between the grades 5
and 6 tests. It is more likely to
be caused by some dramatic
change in the curriculum
between primary and secondary
education, or to be an artefact
of the curriculum content, not
matching well the actual
development of children’s
language, and so constraining
the measurement of growth.
Table 6 above and Chart 1 show the progression in mathematics achievement from grade to
grade at ten percentile levels, plus the two quartiles. Apart from the unexpectedly high
measured achievement in the top half of the ability range in grade 1, the progression is rather
regular, with the isopercentile lines increasing slowly, and being roughly parallel from grade to
grade.
Table 6: Percentile Achievement Estimates by Grade, Mathematics
Percentiles Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
10 -2.97 -2.80 -2.08 -2.47 -2.14 -1.79 -0.99 20 -2.17 -2.06 -1.45 -1.39 -1.13 -0.77 -0.39
25 -1.75 -1.74 -1.09 -1.01 -0.78 -0.57 -0.05
30 -1.53 -1.50 -0.85 -0.71 -0.50 -0.21 0.26
40 -1.05 -1.12 -0.36 -0.23 -0.05 0.25 0.54
50 -0.50 -0.75 0.04 0.33 0.40 0.68 1.07
60 0.22 -0.38 0.41 0.80 0.88 1.07 1.44
70 0.73 -0.03 0.93 1.32 1.35 1.59 1.91
75 0.73 0.23 1.14 1.54 1.65 1.96 2.14
80 1.52 0.51 1.37 1.74 1.89 2.26 2.49
90 2.79 1.06 2.15 2.35 2.36 2.85 3.20
17
Chart 1: Percentile Achievement Estimates by Grade, Mathematics
-4.00
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
Grade
Mean
Sco
re
10th
20th
25th
30th
40th
50th
60th
70th
75th
80th
90th
Table 7 and Chart 2 below show the progression in Tamil achievement from grade to grade at
the same percentile and quartile levels. Again, the isopercentile lines are roughly parallel; and
they all show a sharp drop in mean achievement between grade 5 and grade 6, after which the
steady growth in achievement resumes, although for most groups it does not reach, even by
grade 9 level, the average achievement of the grade 5 learners.
Table 7: Percentile Achievement Estimates, Tamil
Percentiles Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
10 -2.38 -1.87 -1.19 -2.54 -1.91 -1.64 -0.57
20 -1.27 -0.89 -0.12 -0.93 -0.54 -0.18 0.38
25 -0.80 -0.57 0.18 -0.54 -0.20 0.21 0.59
30 -0.37 -0.24 0.48 -0.21 0.08 0.50 0.77
40 0.14 0.38 0.93 0.26 0.56 0.99 1.08
50 0.53 0.67 1.39 0.68 0.93 1.31 1.38
60 1.00 1.22 1.78 0.96 1.26 1.63 1.67
70 1.64 1.72 2.38 1.22 1.59 1.99 1.99
75 2.11 1.72 2.55 1.35 1.77 2.17 2.18
80 2.11 2.05 2.98 1.52 1.96 2.37 2.37
90 2.87 3.25 3.69 1.74 2.43 2.93 2.99
18
Box 3 – Mathematics Test Information Function
This unusual pattern is unlikely to be an artefact of the test. Even if the test items developed for
grades 6 through 9 were unusually difficult, the IRT analysis should compensate for this.
Moreover, fifteen items from the grade 5 test, as well as nine additional items from the grade 4
test, also appeared in the grade 6 test. This should be sufficient common content to ensure
good linking. Therefore, this finding needs to be investigated in more depth. As a first step,
testing should be repeated with a group of students from outside Northern and Eastern
Provinces. If the same pattern emerges, more in-depth investigation should be undertaken of
the test itself, as well as the curriculum and instructional materials, to identify possible reasons
for the achievement gap.
It is also worth noting that in Tamil language the gap between the bottom decile and the
remainder of the Reference Group appears to be relatively large. This may suggest a need to
give more attention to the achievement of the lowest achievers in Tamil language.
Chart 2: Percentile Achievement Estimates by Grade, Tamil
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9
Grade
Mean
Scale
d S
co
re
10th
20th
25th
30th
40th
50th
60th
70th
75th
80th
90th
Item Results
Because of the method of analysis chosen, grade-
wise item analysis has little meaning, and was
not undertaken. The results cited are therefore
for the Reference Group as a whole, rather than
for individual grades. Table 8 gives the total
number of items scaled, and the number with
19
Box 4 – Tamil test Information Function
raw difficulties below 20%, and in each band of ten percentage points above 20%.
A potentially more fruitful way of looking at item difficulty, however, is in relation to the
achievement of the test takers. Annexes 1 and 2 below contain the item maps for the two tests.
In an item map, the left-most column gives units of standard deviation, the second column gives
numbers of test takers with estimated achievement in that band, and the right-hand column
gives the item names. In the present case, the left-most
digit in the item name gives the grade, and the
remainder the item number (with some adjustments to
ensure that each item has a unique ID). Both persons
and items are estimated on the same scale, as an
essential element in the IRT calibration process. Thus,
an item map allows for comparison of the design of the
test with the ability of the test takers. Ideally, there
should be items in each row of the item map for which
there are test takers, and the number of items in any
row of the item map should be greatest at or near the
points across the ability range at which decisions are to
be made. Thus, in the present case, the distribution of
items should be roughly rectangular, because decisions
are being taken at several points across the achievement range.
The items in the mathematics map cover the achievement range well; although there are rather
more items below average ability than
above, this is not cause for concern
because in this case more decisions will be
taken below average achievement than
above. There is also a broad tendency for
the items testing the primary grades to
cluster below average achievement, and
the items testing the secondary grades to
cluster above average achievement. This
can be seen also in the test information
function given in Box 3, where there is
evidence of a relatively broad peak, with
relatively good information across the
range of achievement from -4 to +4.
In Tamil language, the match between items and test takers is not so good. The test takers tend
to cluster above average achievement, while there are more items below than above average
achievement. The items for any given grade are also much more widely distributed across the
Table 8: Raw Item Difficulties
Overall %
Correct
No. of Items in
Tamil Maths
<20 17 28
20 to 29.99 4 13
30 to 39.99 14 18
40 to 49.99 28 17
50 to 59.99 51 22
60 to 69.99 43 27
70 to 79.99 68 21
80 to 89.99 49 23
90 up 12 8
Total 286 177
20
difficulty range than in mathematics. There are many secondary-level items below average
difficulty, and many primary-level items above average difficulty. This again is suggestive that
the progression in achievement in Tamil language is much less clear than in mathematics. This is
reflected in a somewhat narrower information function (Box 4), especially below the overall
average achievement.
In general, the quality of the items in both subjects appears to be good. In mathematics, only
two items (M1009341 and M1004241) are outside the acceptable range on the “Infit” statistic,
and then only by a very small amount. No items have negative point-measure correlations, and
only six (M1007531, M1003021, M1009341, M1009331, M1008371 and M1007541) have point-
measure correlations below 0.30. In Tamil, only five items (T1081801, T1061802, T1081604,
T1081702 and T1050306) are outside the acceptable range on the “Infit” statistic, and only the
first of those could be called “severely misfitting”. No items have negative point-measure
correlations, and only six (T1092705, T1092205, T1072703, T1091905, T1081104 and T1050306)
have point-measure correlations below 0.30.
The reliability of the tests was also good. The overall Cronbach Alpha (equivalent to KR-20)
reliability of the person raw scores was estimated as 0.95 in mathematics, and 0.92 in Tamil
language.
Results of the Reference Group: Assessment of Basic Competencies (ABC)
The results for the ABC test were reported separately for reading, writing and numeracy. Mean
scores for the student population as a whole are given in Table 9 below.
Table 9: Overall Mean Scores on the ABC Test
Reading Writing Number Total
Mean
Score Mean %
Mean
Score Mean %
Mean
Score Mean %
Mean
Score Mean %
4.70 78.29 3.66 73.20 12.04 80.27 20.40 78.46
The average performance of the students in the Reference Group, in excess of 70% correct on all
sections, can be defined as acceptable.
Mean item difficulties are given in annex 3, Tables A3-1 to A3-4. No single item was
exceptionally difficult. Only two items were answered correctly by fewer than 70% of test-
takers – the closing part of the letter, and the word problem involving subtraction.
21
Results by Grade
Variation in mean ABC score from grade to grade is given in Table 10 below. The amount of
variation from grade to grade was limited – only the grade 9 results were significantly different
from (i.e., higher than) the others.
Table 10: Mean Scores on the ABC Test
Reading Writing Number
Grade Mean SD Mean SD Mean SD
6 4.51 1.936 3.52 1.661 11.68 3.438
7 4.64 1.837 3.58 1.560 11.85 3.420
8 4.56 2.013 3.60 1.682 11.80 3.488
9 5.18 1.470 3.96 1.381 12.94 2.832
Frequency Distributions
As can be seen from Tables 11 and 12, in spite of the high overall mean scores, a significant
minority of secondary-level students, even within the Reference Group, still have clearly
inadequate levels of reading and writing skill.
Table 11: Frequency Distribution – Reading
Score Frequency Valid % Cum. %
0 468 7.4 7.4 1 128 2.0 9.4
2 299 4.7 14.1
3 475 7.5 21.6
4 653 10.3 31.9
5 840 13.3 45.2
6 3475 54.8 100.0
Total 6338 100.0
Table 12: Frequency Distribution – Writing
Score Frequency Valid % Cum. %
0 489 7.7 7.7
1 343 5.4 13.1
2 568 9.0 22.1
3 846 13.4 35.4
4 1304 20.6 56.0
5 2786 44.0 100.0
Total 6336 100.0
22
In reading, about one student in three gets a maximum of two of the three questions correct;
and about one in four gets only one correct. Similarly, in writing, about one student in three
scored 3 or fewer out of 5, while one in five achieved one or zero marks. Students achieving at
this level of literacy are unlikely to be able to learn successfully in secondary education.
In number, as can be seen from Table 13, numbers of items, and scores, are higher.
Table 13: Frequency Distribution – Number
Score Frequency Valid % Cum. %
0 45 .7 .7
1 40 .6 1.4
2 40 .6 2.0
3 61 1.0 3.0
4 94 1.5 4.5
5 118 1.9 6.4
6 117 1.9 8.3
7 171 2.8 11.1
8 210 3.4 14.5
9 264 4.3 18.7
10 392 6.3 25.1
11 445 7.2 32.3
12 731 11.8 44.1
13 629 10.2 54.2
14 938 15.2 69.4
15 1894 30.6 100.0
Total 6189 100.0
Reading Achievement and Problem Solving
There is also some indication that ability to read and understand could have had some impact
on number performance. The test contained simple items testing the four operations, as well as
word problems testing addition and subtraction, dictated numbers and mental arithmetic. The
results for these items are crosstabulated in Tables 14 and 15 below.
Table 14: Percent Passing Related Addition Items
Word Problem –
Addition
Wrong Right
Simple
Addition
Wrong 3.5% 3.2%
Right 14.6% 78.8%
23
Table 15: Percent Passing Related Subtraction Items
Word Problem –
Subtraction
Wrong Right
Simple
Subtraction
Wrong 6.8% 2.7%
Right 27.7% 62.9%
In each case, the great majority of the test takers pass both items. However, there is in each
case a group of test takers who can do the simple addition or subtraction task, but who cannot
solve the word problem. In the case of the addition problem, this group is almost 15% of the
test takers; but in the case of the subtraction problem, it is more than one in four of all test
takers. The proportion failing to do simple addition or subtraction at all is much smaller; and the
proportion solving the word problem correctly but failing to solve the simple addition or
subtraction is around 3% - small enough to suggest that this may be due mainly to carelessness.
This result gives some indication of the possible impact of a failure of basic literacy on
achievement, and also of its likely extent.
Achievement on ABC by School
Tables 16, 17 and 18 show the distribution of school mean scores in each sub-scale. In reading,
if the minimum acceptable score is set at 4 (i.e., two questions correct out of three), then nine
schools out of thirty have an average score which is below this, suggesting that there is real
cause for concern about literacy levels in those schools. Similar results are found in writing,
where six schools have a mean below three out of five, and number, where six schools fail to
achieve a mean of eleven out of fifteen. The specific schools in question are not identified in
this report, but will be identified in the report to the Provincial Department of Education.
Table 16: School Mean Scores, Reading (Max. 6)
Mean Schools
Below 3 1 3.0 to 3.99 8
4.0 to 4.99 8
5.0 up 13
Total 30
Table 17: School Mean Scores, Writing (Max. 5)
Mean Schools
Below 2.5 1 2.5 to 2.99 6
3.0 to 3.99 13
4.0 up 10
Total 30
24
Table 18: School Mean Scores, Number (Maximum 15)
Mean Schools
Below 10 2 10.0 to 10.99 4
11.0 to 11.99 11
12.0 to 12.99 7
13.0 up 6
Total 30
Intra-Class Correlation on the ABC
The computation of the intra-class correlation is described in annex 4 below. The intra-class
correlation for the ABC test is very low, at 0.177. This suggests that the diversity of performance
of the students within the schools in the Reference Group is rather similar to the performance of
the group as a whole – i.e., that in general there are few very good and very weak schools with
respect to this test. That is to say, there is relatively low inequality in performance between
schools. However, the fact that the test is taken by four grades is likely to reduce the intra-class
correlation, possibly overstating the degree of equality of outcome between schools.
Analysis of the Target Group – Tamil and Mathematics
The total number of records in the data set is given in Table 19 below.
Table 19: Number of Records
Province N %
Eastern 23012 19.0
Northern 97914 81.0
Total 120926
The number of valid records was rather lower, due to non-completion of tests (for various
reasons) and out-of-range scores (mainly due to non-compliance with scoring guidelines). Valid
Ns for the target group (i.e., the number of test-takers who returned a test paper, which was
then marked and given a mark within the range of possible marks) is given in Table 20. In total,
there were about 109,000 valid responses.
Table 20: Overall Mean Scaled Achievement Estimates
Subject N Min. Max. Mean SD
Tamil 109219 -7.43 6.74 .4243 2.06
Mathematics 109186 -8.20 6.38 -.1831 2.06
25
Table 20 also gives the grand mean achievement estimate for each subject. In each case, the
grand mean was about a quarter of a (Reference Group) standard deviation lower than for the
Reference Group (Reference Group: .7534 in Tamil, .1105 in mathematics). This is roughly
equivalent to one year’s growth in achievement.
Table 21 shows the estimated mean achievement, by subject and province. In each subject,
Northern Province achieves a higher mean score than Eastern Province by more than half a
(pooled) standard deviation; but this may not be comparing like with like, because the Northern
Province test takers include a number of test takers from schools which had continued to
operate throughout the conflict.
Table 21: Mean Scaled Person Measure by Subject and Province
Province Mathematics Tamil
Mean SD N Mean SD N
Reference Group 0.11 1.88 11267 0.75 1.96 11513
Eastern Selected -1.14 1.99 21374 -0.64 2.15 21387
Northern 0.05 2.01 87812 0.68 1.95 87832
Comparing the performance of the Reference Group (drawn from schools in Eastern Province
not affected by conflict) with the selected schools in Eastern Province does confirm that the
schools selected for testing by the Eastern PDE were drawn from those in greatest need. Since
the average growth in mathematics achievement from one grade to the next in the Reference
Group is about 0.265, the difference in grand mean of about 1.25 in mathematics, and about 1.4
in Tamil in Eastern Province suggests that the target schools in Eastern Province are in general
about four years behind the Reference Group.
In order to explore in greater depth the impact of the conflict on learners’ achievement in
Northern Province, test-takers were divided into four groups:
those who were not displaced, and appear to have continued to attend school throughout the conflict;
those who having been displaced had moved to schools elsewhere that continued to function normally;
those who having been displaced had at the time of testing returned to schools which had been restarted; and
those who at the time of testing were still in IDP camps.
The available information about the status of the test takers was incomplete, partly because at
the time of testing many schools were in the process of restarting, and so were not included in
the list of restarted schools received from Northern PDE. It also appears that some teachers in
26
the IDP camp schools put the name of the school the students had originally come from rather
than the IDP camp school on test papers. These schools and individuals were of necessity
excluded from this analysis (although not the overall analysis) because their status was not
confirmed. In addition, some functioning schools did not provide reliable information about
which of their students were from outside the pre-conflict community. These were also
excluded from the analysis.
Table 22 below gives the numbers and average performance of test-takers known to be in each
category. Since the analysis relates to a subset only, somewhat more than half of the total, it can
only be offered for guidance.
Table 22: Scaled Person Measure by Type of Displacement (Northern Province)
Return Code Maths Tamil
Mean SD N Mean SD N
Not Displaced 0.13 1.97 30492 0.74 1.96 29760
IDP in Regular School -0.30 2.01 6724 0.44 1.96 6708
Restarted School -0.76 2.02 7563 -0.18 2.05 7150
IDP at Time of Testing 0.09 2.30 3600 -0.13 1.87 3536
Total -0.07 2.04 48379 0.49 2.00 47154
The mean achievement estimates for the test takers in Northern Province who have not
suffered displacement are much higher than for other groups, with the sole anomalous
exception of mathematics amongst test takers still in IDP camps.
The associated graphic (Box 5) expresses this as a deviation from the mean achievement of the
Reference Group (GM). Learners who have not been displaced achieve at almost the exactly
same level as the Reference Group. Those who were displaced, but found places in schools
Box 5: Northern Province - Deviations from Grand Mean
-1.0000
-0.8000
-0.6000
-0.4000
-0.2000
0.0000
0.2000
Not
Displaced
IDP in Reg.
Sch.
Restarted
School
Still IDP
Type of Student
Scale
d A
bil
ity
Math Deviation from GM
Tamil Deviation from GM
27
Box 6: Progression in Mean Mathematics Achievement by Province
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
3 4 5 6 7 8 9
Grade
Me
an
Ac
hie
ve
me
nt
Eastern
Northern
which had continued to function, achieve at a level about 0.4 below the Reference Group mean
in mathematics, and 0.3 below in Tamil. Those in restarted schools achieve at a level about 0.9
below the Reference Group mean. Those still in the camps achieve at a similar level in Tamil to
those in restarted schools; but in mathematics at a level similar to the Reference Group.
This shows that displacement by conflict is strongly related to learning achievement. As the
regression in Table 27 below confirms, the average gain in mathematics from year to year is
about 0.265, so a typical displaced person in a regular school may be defined as being roughly
one and a half grades behind a typical learner who has not suffered displacement, while a
typical learner who is still displaced, or in a newly-restarted school, may be considered to be
more like three grades behind.
The ALP is designed to cover two years’ learning in any one year. The actual amount of time any
individual will spend in the ALP will depend on their actual achievement; but this suggests that a
typical displaced person in a regular school may need to spend about one year in the ALP, while
a typical learner who is still displaced, or is in a newly-restarted school, may need about two
years.
Tables 23 and 24, and Boxes 6 and 7, show the progression in mean subject achievement from
grade to grade within each province as a whole. As Box 6 shows, the progression in
mathematics within Northern Province taken as a whole is somewhat greater than in Eastern
Province – the difference in mean achievement in grade 3 is 0.72, and in grade 9 is 1.44.
28
Table 23: Mathematics Achievement by Grade and Province
Grade Eastern Northern
Mean SD N Mean SD N
3 -1.36 2.27 3518 -0.64 2.06 11344 4 -1.57 1.78 3590 -0.92 1.73 14682
5 -1.36 1.86 3463 -0.28 1.90 12785
6 -1.22 1.88 2917 0.05 1.96 13426
7 -1.27 1.98 2708 0.31 1.85 11979
8 -0.59 1.95 2748 0.86 1.86 12039
9 -0.24 1.85 2345 1.20 1.84 11642
Table 24: Tamil Achievement by Grade and Province
Grade Eastern Northern
Mean SD N Mean SD N
3 -0.66 2.46 3527 0.32 2.22 11604 4 -0.42 2.20 3497 0.68 2.12 13631
5 -0.48 2.13 3550 0.90 2.01 12597
6 -1.18 2.05 2924 0.13 1.80 13657
7 -0.95 1.96 2689 0.63 1.78 12178
8 -0.61 2.15 2797 1.04 1.86 12538
9 -0.17 1.76 2352 1.12 1.61 11678
Table 27 below gives the regression by province, showing an average increase per grade in
mathematics achievement of 0.184 in Eastern Province, .352 in Northern
As Box 7 shows, the differential gain is somewhat less in Tamil, although it is still greater in
Northern Province, taken overall. Table 29 gives the regression for each province separately,
showing a minimal gain overall in Eastern Province (0.1 per grade), and a somewhat larger gain
in Northern Province (0.99) per grade.
Box 7: Progression in Mean Tamil Achievement by Province
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
3 4 5 6 7 8 9
Grade
Me
an
Ac
hie
ve
me
nt
Eastern
Northern
29
In Tamil language, when grades 3-5 and 6-9 are treated separately, and the regression is done
by stage, as in Tables 30 and 31, there is stronger progression within a stage than overall; but
there is still less progression from grade to grade in primary than in secondary, and amongst the
target group than the Reference Group.
Tables 25 and 26 and Boxes 8 and 9 show the progression in achievement within Northern
Province, broken down by type of displacement.
Table 25: Mathematics Achievement by Grade and Type of Displacement
Not Displaced IDP/Regular IDP/Restarted Still IDP
Grade Mean SD Mean SD Mean SD Mean SD
3 -0.50 2.01 -0.86 2.15 -1.97 2.01 -2.13 2.73
4 -0.73 1.68 -1.37 1.66 -1.95 1.77 -1.37 1.90
5 -0.02 1.85 -0.57 1.76 -1.13 1.68 -1.24 1.67
6 0.08 1.98 -0.33 1.90 -0.86 1.89 0.04 1.96
7 0.35 1.88 -0.01 1.74 -0.56 1.83 0.02 1.75
8 0.79 1.88 0.49 1.79 0.30 1.82 1.57 2.06
9 1.23 1.82 1.25 1.69 0.59 1.63 1.55 1.90
In mathematics, the basic pattern is similar across all three types of displacement. The
achievement deficit is largest in grades 3 and 4, and declines from grade 5 onwards. The deficit
is smaller in secondary education, and in grades 8 and 9 it is very small. In general, the deficit
Box 8: Northern Province - Progression in Mathematics Achievement by Type of Displacement
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
3 4 5 6 7 8 9
Grade
Mean
Ach
ievem
en
t
Not Displaced
IDP in Regular School
Returned IDP in Restarted School
IDP at Time of Testing
30
Box 9: Northern Province - Progression in Tamil Achievement by Type of Displacement
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
3 4 5 6 7 8 9
Grade
Me
an
Ac
hie
ve
me
nt
Not Displaced
IDP in Regular School
Returned IDP in Restarted School
IDP at Time of Testing
amongst displaced persons studying in regular schools is less than those studying in restarted
schools.
The results in Tamil language are similar, and if anything easier to interpret. The deficit in
achievement is much larger in the primary than the secondary grades. In fact, it is so large in the
early grades that emergency action appears to be indicated. The achievement deficit amongst
displaced learners in regular schools almost disappears in the secondary stage. Amongst the IDP
learners in the restarted schools, it remains around 0.5, equivalent to at least a grade and an
half of study. These results are confirmed by the regressions given in Table 28 below.
Table 26: Tamil Achievement by Grade and Type of Displacement
Not Displaced IDP/Regular IDP/Restarted Still IDP
Grade Mean SD Mean SD Mean SD Mean SD
3 0.55 2.10 0.09 2.28 -1.08 2.18 -1.41 2.42
4 0.88 2.05 0.06 2.09 -0.57 2.18 -0.89 2.36
5 1.03 1.90 0.58 1.81 0.09 1.93 0.22 1.93
6 0.06 1.86 -0.08 1.76 -0.68 2.08 0.06 1.66
7 0.53 1.92 0.41 1.69 -0.14 1.97 -0.39 1.59
8 1.04 1.90 1.08 1.88 0.38 1.83 0.28 1.58
9 1.14 1.68 1.20 1.53 0.61 1.57 0.01 1.55
31
Setting Cut Scores
The original plan for setting cut scores was to base them on the progression in mean score – if a
learner’s score was at or below the mean for a particular grade, this would indicate that the
learner was functioning at or below the expected level for that grade, regardless of their age. In
practice, however, although the measured increase in achievement from grade to grade is
reasonable in terms of effect size, it is relatively small by comparison with the diversity of
achievement within a grade.
Table 27: Mathematics Regression
Reference Group
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -1.466 0.053 -27.777 0.000
Grade 0.265 0.008 0.285 31.556 0.000
Dependent Variable: Estimated Maths Measure (Reference Group)
Eastern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -2.193 .041 -53.119 .000
Grade .184 .007 .181 26.906 .000
Dependent Variable: Estimated Maths Measure (Reference Group)
Northern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -2.042 .020 -100.30 .000
Grade .352 .003 .343 108.172 .000
Dependent Variable: Estimated Maths Measure (Reference Group)
In mathematics, for example, as Table 27 shows, the average gain in achievement from year to
year is around 0.265. But as Table 28 below shows, the standard deviation of the Reference
Group achievement estimates within a grade varied between 1.58 and 2.04. This means that
the average growth in achievement from grade to grade (effect size) represented only between
0.13 and 0.168 standard deviations. This means that there is not a very strong pattern of
32
growth from grade to grade, and as a result there is a very large overlap in performance
between grades.
Table 28: Mathematics Regression by Type of Displacement
Not Displaced
Model Unstandardized Coefficients Standardized Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -1.715 0.034 -49.967 0.000
Grade 0.313 0.006 0.309 56.745 0.000
Dependent Variable: Estimated Maths Measure (Reference Group)
IDP in Regular School
Model Unstandardized Coefficients Standardized Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -2.465 0.067 -36.613 0.000
Grade 0.377 0.011 0.384 34.093 0.000
Dependent Variable: Estimated Maths Measure (Reference Group)
Returned IDP in Restarted School
Model Unstandardized Coefficients Standardized Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -3.606 0.068 -53.071 0.000
Grade 0.467 0.011 0.451 43.933 0.000
Dependent Variable: Estimated Maths Measure (Reference Group)
IDP at Time of Testing
Model Unstandardized Coefficients Standardized Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -4.196 0.120 -34.876 0.000
Grade 0.662 0.018 0.525 37.035 0.000
Dependent Variable: Estimated Maths Measure (Reference Group)
The reason why this is not a good basis for placing learners in the ALP is immediately clear from
specific examples. The 50th percentile in the Reference Group for grade 3 was an achievement
estimate of -0.49, so we can say that a learner that achieves an estimated achievement of -0.49
is functioning at the third grade level. The difficulty with this interpretation is that in the
Reference Group, about 57% of the grade 4, 37% of the grade 5, and 35% of the grade 6 are
achieving at this level. Although there are clearly different reasons for their under-achievement,
offering the ALP to returning IDPs who achieve this level, while denying it to such a large
proportion of non-displaced learners achieving at the same level, might be considered
inappropriate.
33
Table 29: Mathematics Mean Achievement
by Grade (Reference Group)
Grade Mean Std. Dev. N ES
3 -0.410 2.036 1702 0.130
4 -0.818 1.581 1705 0.168
5 0.030 1.701 1668 0.156
6 0.125 1.901 1457 0.139
7 0.342 1.745 1498 0.152
8 0.579 1.883 1638 0.141
9 1.030 1.680 1599 0.158
Total 0.110 1.884 11267
The situation is more complex in Tamil language because of the fall in estimated achievement in
the Reference Group between grade 5 and grade 6 (see Box 2 above). As Table 30 shows, the
average gain across grades is in fact very small indeed.
Table 30: Overall Tamil Regression
Whole Reference Group
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) .327 .057 5.772 .000
Grade .072 .009 .074 7.957 .000
Dependent Variable: Estimated Tamil Measure (Reference Group)
Eastern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) -.695 .045 -15.320 .000
Grade .010 .007 .009 1.341 .180
Dependent Variable: Estimated Tamil Measure (Reference Group)
Northern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
T Sig.
B Std. Error
(Constant) .092 .021 4.366 .000
Grade .099 .003 .099 29.577 .000
Dependent Variable: Estimated Tamil Measure (Reference Group)
The practical difficulty created by the fall in achievement between the end of primary and the
beginning of secondary can be illustrated by concrete examples. The 50th percentile of the
grade 3 Reference Group has an estimated achievement of 0.5292. At grade 4, 46% of the
34
Reference Group is functioning at or below this level; and while in grade 5 this falls to 31%, in
grade 6 it rises again to 47%. In fact, the average estimated achievement of the 50th percentile
of the grade 7 Reference Group is 0.9297, about the same as the average estimated
achievement of the 40th percentile of the grade 5 Reference Group. An approach based on the
estimated achievement of an earlier grade will therefore clearly not be very viable in Tamil
language, certainly if it involves placement across stages.
To some extent, this problem is resolved if the data are analysed as separate stages, primary
and secondary, as Tables 31 and 32 below show.
Table 31: Tamil Regression (Primary Stage)
Reference Group
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -.895 .143 -6.250 .000
Grade .428 .035 .167 12.154 .000
Dependent Variable: Estimated Tamil Measure
Eastern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -.876 .110 -7.956 .000
Grade .089 .027 .032 3.291 .001
Dependent Variable: Estimated Tamil Measure
Northern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -.530 .056 -9.484 .000
Grade .291 .014 .109 21.425 .000
Dependent Variable: Estimated Tamil Measure
35
Table 32: Tamil Regression (Secondary Stage)
Reference Group
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -2.074 .153 -13.546 .000
Grade .371 .020 .225 18.359 .000
Dependent Variable: Estimated Tamil Measure
Eastern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -3.236 .130 -24.822 .000
Grade .335 .017 .183 19.269 .000
Dependent Variable: Estimated Tamil Measure
Northern Province
Model Unstandardized Coefficients Standardized
Coeff. (Beta)
t Sig.
B Std. Error
(Constant) -1.830 .053 -34.450 .000
Grade .341 .007 .211 48.369 .000
Dependent Variable: Estimated Tamil Measure
Analysed in this way, the amount of growth in achievement from grade to grade within a stage is
larger than the average for mathematics. As Table 33 below shows, the effect size is also 0.2
standard deviations or greater in each case (the criterion commonly set for a “medium” effect).
Table 33: Tamil Mean Achievement
by Grade (Reference Group)
Grade Mean Std. Dev. N ES
3 0.47 2.18 1741 0.20
4 0.65 1.96 1773 0.22
5 1.33 2.01 1664 0.21
6 0.12 1.77 1613 0.24
7 0.56 1.81 1549 0.24
8 0.94 2.00 1641 0.21
9 1.22 1.56 1532 0.27
Total 0.75 1.96 11513
36
Even so, however, placement within the ALP cannot realistically be based on relating the
performance of a given test taker to the average performance of learners in a particular grade,
because the performance of learners within a grade is so diverse. For example, if the average
performance of a grade 4 learner is 0.2 standard deviations less than the average performance
of a grade 5 learner, that still means that around 42% of grade 5 learners will achieve a score
lower than that of the average grade 4 learner.
Therefore, placement will need to be based on consideration internal to a specific grade. This
will entail setting a specific achievement level in the Reference Group (e.g., the bottom quartile
boundary) below which an individual in the target group (but not in the Reference Group) will be
placed in the ALP, presumably at a point where they have one year in the ALP; and perhaps a
second point below that initial cut score which will determine which individuals will spend two
years in the ALP. This would be consistent with the broad achievement findings – that the
average IDP learner within a still-functioning school probably needs about one year of ALP, while
the average IDP in a restarted school or camp needs two years.
Setting cut scores may also depend not only on objective level of achievement but also on
considerations of provision. For example:
The reported proportion of IDPs in still-functioning schools tends to be quite low; it may not in all cases be viable to put together ALP classes in such schools, and in some cases, the most appropriate response may therefore be to identify IDP learners in need of special attention and offer them remedial classes either during or after school hours.
In restarted schools, there may be a need for two or three tracks – e.g.: “regular” (i.e., learners in grade-appropriate classes); “ALP+1” (i.e., learners spending one year in the ALP); and “ALP+2” (i.e., learners remaining in the ALP for two years). To what extent this is viable in any given school will depend on the number of learners in a grade. In many schools, it may not be feasible to track the learners into separate streams in this way. A reduced range of provision (i.e., assigning all learners in a given grade to the same level or programme) may be the only option.
There is also the problem of learners returning during the school year (although this may not be such a big problem in 2011 as it was in 2010).
In IDP camps, the basic situation is similar; but there is a real question whether any provision at all is viable within the camp, since the inhabitants are in the process of being resettled.
Diversity in Performance between Schools
In order to explore in greater depth the diversity in performance between schools, the
performance of the schools was analysed. Ideally, such an analysis would take account of the
school type – whether it was a restarted school, an IDP camp, or a school which had continued
to operate. However, because data were not available for all schools, such an analysis would
have excluded some schools or learners. Only two schools in Eastern Province gave information
37
about whether test takers were displaced or non-displaced learners, so these two schools were
not treated separately. Similarly, while the Northern Province schools fall naturally into groups
– those that continued to operate and received IDPs; those that have restarted; and those that
serve IDP camps, the available information was incomplete. Therefore, the different types of
schools were not treated separately for purposes of analysis.
A second challenge for analysis was finding a way to represent school performance. At the
summary level, it would be sufficient to compute intra-class correlations; but this would have
given no useful information about individual schools. To make statements about individual
schools, it was necessary to analyse a list of schools.
In principle, it would have been possible to compute a school mean based on all test takers; but
since some schools do not have secondary grades, while some others do not have primary
grades, and some have all grades, school means would not be strictly comparable. However,
computing means for individual grades would unduly complicate the analysis and interpretation.
Therefore, each test-taker, regardless of grade, was coded, based on whether or not their
achievement was within the bottom quartile range of the Reference Group for that grade. The
cut scores used for each quartile were as given in Table 34 below.
Table 34: Reference Group Cutting Scores for Bottom Quartile
Gr3 Gr4 Gr5 Gr6 Gr7 Gr8 Gr9
Maths -1.75 -1.74 -1.09 -1.01 -0.78 -0.57 -0.05
Tamil -0.80 -0.57 0.18 -0.54 -0.20 0.21 0.59
The proportion of all the test takers in each school falling into that group was then computed,
and the schools were classified according to that proportion. On each subject (maths and
Tamil), the schools were then divided into four groups as follows:
Grouping by Proportion in Reference Group Bottom Quartile
Group Definition
Group 1 25% or fewer of learners in RG bottom quartile
Group 2 25.01-50% of learners in RG bottom quartile
Group 3 50.01-75% of learners in RG bottom quartile
Group 4 75.01% or more of learners in RG bottom quartile
38
Schools were then classified into four levels of achievement deficit, as follows:
Definitions of Levels of Achievement Deficit
Deficit Definition
None 25% or fewer of learners are in RG bottom quartile in
at least one subject
Mild 25.01-50% of learners are in RG bottom quartile in
both subjects
Moderate 50.01-75% of learners are in RG bottom quartile in at
least one subject
Severe 75.01% or more of learners are in RG bottom quartile
in at least one subject
The rationale for this classification is that Reference Group schools will vary in the proportion of
their learners who score in the bottom quartile. In a notional average school, 25% of the
learners will fall into this group; but many schools will have more, so schools with a higher
proportion of learners in the bottom quartile in one subject were not considered to have a
problem. Schools where more than 25% of learners in both subjects were in the bottom quartile
would by definition be below average, and so were defined as having an achievement deficit,
but one so mild as not to need attention. However, schools where more than 50% of learners
tested in the bottom quartile in at least one subject were defined as having an achievement
deficit which requires attention, and so were defined as having a moderate level of achievement
deficit. Finally, schools where at least 75% of learners in at least one subject tested in the
bottom quartile in at least one subject were defined as having a severe level of achievement
deficit. It is assumed that schools with a moderate or severe level of achievement deficit would
most likely want to include most or all of their students in the ALP.
In Eastern Province, as Table 35 below shows, 28 of the 170 schools tested (just over 16%)
showed evidence of a severe achievement deficit, including 13 where at least 75% of learners
were in the bottom quartile in each subject. Sixty-five schools, or about 38%, showed a
moderate level of achievement deficit, meaning that 93 schools, or about 55%, could be
expected to participate in the ALP in a major way.
39
Table 35: Levels of Achievement Deficit by Province
Code Eastern Northern
Deficit Number % Number %
1 Group 1 on both 16 9.41 212 34.64 None
2 Group 1 and 2 15 8.82 100 16.34
3 Group 2 on both 46 27.06 201 32.84 Mild
4 Group 2 and 3 26 15.29 46 7.52 Moderate
5 Group 3 on both 39 22.94 37 6.05
6 Group 3 and 4 15 8.82 10 1.63 Severe
7 Group 4 on both 13 7.65 6 0.98
Total 170 612
In Northern Province as a whole, there were many fewer schools showing signs of “moderate”
or “severe” learning deficit. Amongst the “restarted” schools and IDP camps, the picture was
somewhat different – although as Table 36 shows, even these groups did not show the very high
proportions of schools with high levels of learning deficit found amongst the target group in
Eastern Province.
Table 36: Levels of Achievement Deficit amongst IDPs
Restarted Camp Deficit
Achievement Number % Number %
Group 1 on both 15 15.15 1 16.67 None
Group 1 and 2 13 13.13 1 16.67
Group 2 on both 31 31.31 2 33.33 Mild
Group 2 and 3 15 15.15 2 33.33 Moderate
Group 3 on both 16 16.16 0 0.00
Group 3 and 4 4 4.04 0 0.00 Severe
Group 4 on both 5 5.05 0 0.00
Total 99 6
The results for the IDPs in the camp are very surprising, given that feedback from teachers
suggested that large numbers should fall within the “severe” category. It was not however
possible to investigate what had happened, so long after the event.
40
Analysis of the Target Group – Basic Competencies
There were a lot more problems with the assessment of basic competencies. A lot of schools
never received copies, and many of those gave us round-figure estimates. Of those that did
receive tests, some did not understand what they were for. A significant minority used their
own scoring system. So in doing the analysis, we divided them into three groups – those who
scored within the marking scheme; those that did not; and no data.
Administration and scoring of the Assessment of Basic Competencies (ABC) faced several
difficulties. A significant proportion of schools did not receive copies of the test sheets. Some of
these schools did not return results; some of them estimated test-takers’ basic competencies,
usually on a scale of 0-100. Some schools received, administered and scored the ABC, but
submitted only total scores. Many of these, but not all, submitted corrected data when
requested.
Exclusions
Cases with missing data were excluded from the analysis, as were those with only total scores,
and those with out-of-range values. Table 37 below shows the numbers of valid records,
together with the numbers in each excluded category.
Table 37: Data Received for the ABC Test
Frequency Percent Valid %
Valid Data 60649 50.2 89.8
Invalid Data 6888 5.7 10.2
Total with Data 67537 55.8 100.0
Missing Data 53389 44.2
Grand Total 120926 100.0
Further analysis will be based only on those cases with valid data – which, as Table 37 shows,
includes only about half the data received. The ABC results are therefore indicative rather than
fully representative.
Overall Mean of Included Cases
As Table 38 below shows, for each sub-test, the overall mean for the test-takers from Eastern
Province is well below that for the Reference Group from Eastern Province. The overall mean
for the test-takers from Northern Province is comparable to that for the Reference Group.
41
Table 38: Province-wise Overall Mean Scores on the ABC Test
Reading Writing Number
Score % Score % Score %
Eastern 2.89 48.20 2.98 59.52 8.53 56.90
Northern 4.52 75.31 3.80 75.97 12.14 80.91
Reference Group 4.70 78.29 3.66 73.20 12.04 80.27
As Table 39 below shows, the mean score on each test for test-takers who were still in IDP
camps at the time of testing is markedly lower than for other groups. The other two IDP groups
fare rather better; those studying in regular schools achieve at a level comparable with those
who have not been displaced; while those in restarted schools achieve at a level only slightly
lower.
Table 39: Northern Province: Mean Scores by Displacement Status
Reading Writing Number
Mean SD Mean SD Mean SD
Not Displaced 4.14 1.85 3.67 1.42 12.01 3.46
IDP in Reg. School 4.53 1.80 3.74 1.46 11.82 3.64
Returned/Restarted 4.12 2.18 3.23 1.70 11.03 3.85
IDP at Testing 2.69 1.87 2.53 1.94 10.32 4.82
Grade-wise Mean and SD of Included Cases
Similarly, as Tables 40 and 41 below show, there were no major differences in achievement from
grade to grade. Average performance tended to be higher in the higher grades – in grade 9,
achievement was about one-third of a standard deviation higher than in grade 6. The gradient
of improvement was clearer in Northern Province than in Eastern Province.
Table 40: Grade-Wise Mean Scores on the ABC Test - Eastern
Reading Writing Number
Grade Mean SD Mean SD Mean SD
6 2.75 1.84 2.84 1.63 8.09 4.24
7 2.78 1.90 2.82 1.70 8.47 4.38
8 3.09 1.90 3.01 1.62 8.24 4.26
9 2.99 1.89 3.31 1.58 9.58 3.97
42
Table 41: Grade-Wise Mean Scores on the ABC Test - Northern
Reading Writing Number
Grade Mean SD Mean SD Mean SD
6 4.20 1.95 3.50 1.58 11.43 3.83
7 4.48 1.80 3.62 1.47 11.87 3.54
8 4.62 1.77 3.98 1.35 12.50 3.28
9 4.84 1.66 4.14 1.26 12.87 2.95
Frequency Distributions of Included Cases
Tables 42, 43 and 44 below give the frequency distributions of scores in reading, writing and
number. By comparison with the Reference Group, there are many more test takers in the
target schools in Eastern Province who are achieving very low scores in reading and writing –
suggesting that there may in fact be a larger literacy problem in the secondary grades in these
schools than is indicated by the main testing. Overall, the situation in Northern Province is less
severe, and more similar to the performance of the Reference Group. That is not to say that
there is no problem; a very sizeable minority have not reached an acceptable level in basic
literacy. The same general pattern applies in number, with more than half the test takers in
Eastern Province not achieving an acceptable minimum standard, while performance in
Northern Province is only slightly lower than in the Reference Group.
Table 42: Frequency Distribution – Reading
Score Eastern Northern Reference Group
Freq. Cum. % Freq. Cum. % Freq. Cum. %
0 652 7.1 1283 4.2 468 7.4 1 1420 22.5 1384 8.7 128 9.4
2 3290 58.2 3042 18.6 299 14.1
3 694 65.7 2007 25.1 475 21.6
4 804 74.4 4509 39.8 653 31.9
5 793 83.0 3713 51.8 840 45.2
6 1566 100.0 14807 100.0 3475 100.0
Total 9219 30745 6338 Table 43: Frequency Distribution – Writing
Score Eastern Northern Reference Group
Freq. Cum. % Freq. Cum. % Freq. Cum. %
0 857 9.3 1367 4.5 489 7.7
1 1185 22.2 1468 9.2 343 13.1
2 1586 39.4 3028 19.1 568 22.1
3 1602 56.7 4433 33.5 846 35.4
4 1670 74.8 6264 53.9 1304 56.0
5 2319 100.0 14171 100.0 2786 100.0
Total 9219 30731 6336
43
Table 44: Frequency Distribution – Number
Score Eastern Northern Reference Group
Freq. Cum. % Freq. Cum. % Freq. Cum. %
0 270 2.9 250 0.8 45 .7 1 444 7.7 226 1.6 40 1.4
2 397 12.0 295 2.5 40 2.0
3 371 16.1 434 3.9 61 3.0
4 455 21.0 419 5.3 94 4.5
5 516 26.6 519 7.0 118 6.4
6 550 32.6 606 9.0 117 8.3
7 557 38.6 741 11.4 171 11.1
8 679 46.0 953 14.5 210 14.5
9 596 52.4 1201 18.4 264 18.7
10 764 60.7 1624 23.7 392 25.1
11 701 68.3 1927 30.0 445 32.3
12 872 77.8 3081 40.1 731 44.1
13 1068 89.4 3319 50.9 629 54.2
14 450 94.3 5112 67.6 938 69.4
15 529 100.0 9940 100.0 1894 100.0
Total 9219 30647 6189
Charts 3, 4 and 5 show the frequency distribution of scores in Northern Province in reading,
writing and number, broken down by type of displacement.
Chart 3: Northern Province - Performance on ABC Reading
Test
0.00
10.00
20.00
30.00
40.00
50.00
60.00
0 1 2 3 4 5 6
Score
Perc
en
t A
ch
ievin
g S
co
re Not Displaced
IDP in regular school
Returned IDP in restarted
school
IDP at time of testing
Chart 3 shows a large proportion, more than 20%, of test takers who were still in the IDP camps,
and a somewhat smaller proportion of those who were in the “restarted” schools, but still more
than 10%, scoring zero on the reading test. This is in marked contrast to the non-displaced test
takers, and those who were IDPs but in regular schools. A similar pattern is found in writing, but
44
even more pronounced, in that almost 30% of test takers in the IDP Camps scored zero. A
similar, but less pronounced, pattern is found in number. These figures should be interpreted
with caution, because the data are incomplete; but it does seem clear that there is a significant
problem of poor literacy skills amongst the test takers in the camps and the restarted schools.
Chart 4: Northern Province - Performance on ABC Writing Test
0.00
10.00
20.00
30.00
40.00
50.00
0 1 2 3 4 5
Score
Perc
en
t A
ch
ievin
g
Sco
re
Not Displaced
IDP in regular school
Returned IDP in restarted
school
IDP at time of testing
Chart 5: Northern Province - Performance in ABC Number Test
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Score
Perc
en
t A
ch
ievin
g
Sco
re
Not Displaced
IDP in regular school
Returned IDP in restarted
school
IDP at time of testing
Achievement by School
Tables 45, 46 and 47 show the distribution of school mean scores for reading, writing and
number, with the Reference Group distribution for purposes of comparison.
Table 45: School Mean Scores, Reading (Max. 6)
Mean Schools % Ref Gp %
Below 3 87 23.90 3.33
3.0 to 3.99 46 12.64 26.67
4.0 to 4.99 131 35.99 26.67
5.0 up 100 27.47 43.33
Total 364
45
Table 46: School Mean Scores, Writing (Max. 5)
Mean Schools % Ref Gp %
Below 2.5 59 16.25 3.33
2.5 to 2.99 52 14.33 20.00
3.0 to 3.99 164 45.18 43.33
4.0 up 88 24.24 33.33
Total 363
Table 47: School Mean Scores, Number (Maximum 15)
Mean Schools % Ref Gp %
Below 10 115 31.77 6.67
10.0 to 10.99 54 14.92 13.33
11.0 to 11.99 68 18.78 36.67
12.0 to 12.99 69 19.06 23.33
13.0 up 56 15.47 20.00
Total 362
It is clear that there is major disparity of outcome from school to school. There is a large group
(at least a quarter of the schools) where every test taker achieved full marks, or almost full
marks. On the other hand, there are a similar number of schools where many or most students
have a level of literacy (and to a lesser extent numeracy) well below that required for them to
carry out basic writing tasks. This will inevitably handicap their learning. It is important to
remember that the ABC test is not intended to discriminate in this way. It is intended to
represent a minimum level of skill which every secondary-stage learner should possess.
Therefore, the fact that at least a quarter of all schools have such a low level of basic skill is real
cause for concern.
Conclusions and Recommendations
The test development process appears to have been quite successful, in spite of the lack of
opportunities for piloting. The tests in Tamil and mathematics were at an acceptable level of
difficulty for the reference group, and worked well, although the tests proved to be a little more
difficult than anticipated for the higher grades, especially in mathematics. The assessment of
basic competencies also appears to have worked well, especially in showing up individuals who
had reached the secondary grades, but lacked the basic literacy required to learn successfully,
and identifying schools where this problem was particularly prevalent.
46
The main testing, scoring and data entry were handled at local level, and the quality and
timeliness of the work of the schools was very variable, especially within Northern Province.
This is understandable, however, given the very poor situation in most areas of Northern
Province; but it resulted in delay, and also significantly affected the proportion of tests that
could confidently be included in the analysis. The analysis has taken a somewhat conservative
approach in terms of what was included and excluded; and numbers of test takers in camps and
restarted schools could certainly be increased by more detailed study of the lists and data files,
if time and resources permitted.
The analysis technique used, whereby the items were pooled, and analysed as a single data set
using Rasch one-parameter IRT, proved exceptionally powerful, because it placed all
measurements on the same scale, meaning that it was possible to have a clear picture of both
progression in achievement from grade to grade and diversity within a grade, and their relative
importance. This means that this data set is exceptionally informative, and it threw up some
striking findings. Most notable among these is the sharp drop in mean achievement in Tamil
from grade 5 to grade 6. This is not simply an artefact of the composition of the Reference
Group. It applies across the whole range, and suggests a need for review of the curriculum and
instructional materials. The reasons for it need to be explored in depth by the NIE.
However, the result of the scaling is in one sense disappointing, in that it shows that the gain in
achievement from one grade to the next is much smaller than the diversity of achievement
within one grade. This means that the approach originally proposed, setting the cut-off score
for admission to the ALP in terms of the average achievement of earlier grades was not feasible,
because the overlap in achievement between grades was too great. In practice, therefore, it is
proposed that the cut-off score should be set in relation to achievement within a given grade. It
is proposed that this be done by defining those who achieve scores in the bottom quartile of the
Reference Group as being in need of support of some kind from the ALP. Ideally, this support
may take different forms, depending not only on individual need, but local circumstances.
The results showed a sharp disparity in achievement between the Reference Group schools and
the schools in Eastern Province which were selected for possible inclusion in the ALP. Although
it seems possible that not all schools selected by Eastern PDE for possible inclusion in the ALP
were conflict-affected (some may reflect the impact of the tsunami at the end of 2004), the fact
remains that this group of schools performed strikingly poorly. Even within this group of
schools, however, there is considerable variability, with about 55% of the schools showing
moderate or severe achievement deficit, and 45% showing relatively little deficit. Those schools
where the overall achievement deficit is relatively small are likely to have relatively few learners
who need support, and so may not be able to put together a “critical mass” for participation in
the ALP. It may not be cost-effective for these schools to establish ALP classes, and unless their
ALP candidates can travel to another school, they may need to be given support by means of
additional teaching either outside school hours or by being taken out of some lessons to
47
participate in a tailored remedial programme. In the schools where the achievement deficit is
more severe, it may be possible to run more than one ALP stream alongside regular classes,
depending on the severity of the school’s achievement deficit and the size of the school. In
smaller schools where the achievement deficit is severe, this may not be feasible, and it may be
necessary for the entire school to be designated as an ALP class. This is not cause for concern.
Going through the ALP is unlikely to harm learners who are already achieving at or near an
acceptable level.
In Northern Province, the overall level of achievement was comparable with that in the
Reference Group. This is surprising, and certainly reflects a number of factors. Perverse as it
may seem, one significant factor is the proportion of learners who had not been displaced. The
Northern PDE asked for all learners to be tested, unlike the Eastern PDE, which selected those it
believed to be in greatest need. Because by no means all schools supplied information about
which learners were displaced and which were not, it is not possible to fully exclude non-
displaced test takers from the analyses. Another factor was the presence of data from schools
which did not administer the tests rigorously, or did not follow the scoring instructions
scrupulously. Where possible, these scripts have been excluded; but many have no doubt got
through the net. This is regrettable; and it is to be hoped that schools in Northern which did not
conduct the test scrupulously will re-test under better conditions, now that the benefit of the
testing is clearer.
The patterns of achievement shown by the results give strong support for the establishment of a
programme such as the ALP. On average, there was a significant achievement deficit associated
with any form of displacement. Those learners who were still in the IDP camps, or in schools
which had been evacuated and subsequently restarted, showed an overall level of learning
deficit amounting to about three years’ learning time, by comparison with the Reference Group.
Those who had been displaced, but found places in schools which had continued to function
normally showed about half this level of achievement deficit.
As might be predicted, the impact of displacement was greatest amongst the younger learners.
In both subjects, there was a consistent pattern of relatively high levels of learning deficit
amongst displaced learners in grades 3-5, and a rather lower level of learning deficit in grades 6-
9. This can be seen clearly in the steeper regression slopes for the target groups compared with
the Reference Group, and the more severe kinds of displacement within the target group. It
suggests that repairing the damage done by displacement at a time when the basic learning
skills are being developed and consolidated learners in grades 3-5 should be given priority in the
ALP.
There was also very considerable diversity in average achievement from school to school,
especially in Eastern Province, where more than 16% of schools showed “severe” deficit, and
18% showed “none”. The disparity was less pronounced in Northern Province; but because the
number of schools tested was so much larger, the number of schools with “severe” or
48
“moderate” deficit was still substantial. The fact that low achievement is not evenly distributed
across schools, but is concentrated in some, while others have few problems, suggests that the
strategy for introducing ALP will need to respond as much to the condition of the school as of
the individuals tested. Those schools where the problems are particularly severe may need to
become “ALP schools”, while those where the problems are less acute may be able to focus on
individual learners. It might also be productive for the PDE to focus more attention on the
limited number of schools with severe problems, with the aim of helping them to prepare and
implement an improvement plan.
By contrast with the two subject tests, the ABC test is purely criterion-referenced, and aimed at
assessing a very minimal level of basic literacy and numeracy, a level so low that without it a
learner is almost certainly unable to function in a classroom environment. Therefore every, or
almost every, test taker should have achieved almost full marks. Although the data for the ABC
test are less complete than those for the Tamil and mathematics tests, there is cause for
concern about standards of basic literacy in grades 6-8 among the schools tested. About two
out of three test takers in Eastern Province, and one out of four in Northern Province, achieve
no more than three marks out of six on the reading test; and roughly two out of five in Eastern
Province, and one out of five in Northern province, achieve no more than two out of five on the
writing test. Learners who achieve scores of this order cannot be said to have the basic literacy
required to function in secondary education. The lack of numeracy skills is perhaps less
pervasive, but still significant. On the test of number, which is basically a very simple arithmetic
test, more than one in four in Eastern Province achieve five or fewer out of fifteen. This level of
achievement suggests a need for action to assure that those entering grade 6 have a basic
standard of literacy, as well as remedial action to help those already in secondary school who
lack the required level of literacy.
Therefore, the ALP placement testing, although it struggled with significant obstacles, has
produced very strong evidence of the need for action to help the learners who were displaced
by the recent conflict to reach an acceptable standard, and given clear guidance about the kind
of help needed, and how best to target it. It has also produced a tool, in the form of the set of
placement tests, which can be used to place learners who were not tested earlier in the year, or
whose results are compromised in some way, for example by problems with administration or
scoring.
49
Annexes
Annex 1: Mathematics Item Map – Item Prefix Deleted
Annex 2: Tamil Item Map – Item Prefix Deleted
Annex 3: Supplementary Tables for Assessment of Basic Competencies
Annex 4: Computing the intra-class correlation
Annex 5: Summary of Design of System of Placement Tests in a Given Subject
Annex 6: ABC Tests
Annex 7 Assessment Guide for Grade 1 and 2 Teachers
50
Annex 1: Mathematics Item Map – Item Prefix Deleted
Persons <more> Items <rare> 7 .# # # # .
6 . . . .
5 . 9331 . 8301 9411 . T 8371 . 9351
4 . 7531 8381 8382 9401 .# T 7501 .# 7431 7551 9311 9391 9442 .# # 7541 9312 9431
3 .# # # 7421 7481 8311 8401 9361 9372 .# # # # 4212 6291 7471 7511 8281 9422 .# # # # 7411 8261 8321 9341 9421 .# # # # # # S 4211 8361 9381
2 .# # # # # # S 6311 8391 9321 9452 .# # # # # # # # 6361 6371 9371 9451 .# # # # # # # # # # 6252 6261 7441 .# # # # # # # # # # 6381 7521
1 .# # # # # # # # # 4103 4181 4183 4204 6221 6391 6392 7491 .# # # # # # # # # # #
#
4191 4241 5262 6251 6301 6401 7451 .# # # # # # # # # # #
#
4153 4182 4203 6281 7522 .# # # # # # # # # # # 4143 4202 6321 6341 7461
0 .# # # # # # # # # # # M M 4243 5263 5266 5272 5941 6241 .# # # # # # # # # # # 3032 3033 4172 4173 4213 4215 5281 6331 9441 .# # # # # # # # # # #
#
3061 3062 4104 4122 4152 4163 4174 5264 5265 5273 5302
5933 6231 6232 .# # # # # # # # # # 4102 4105 4112 4162 4192 4214 4244 5261 5282
-1 .# # # # # # # # # 4121 5301 .# # # # # # # # 3082 3101 4133 4161 4201 4234 5303 5911 6271 6352 .# # # # # # # 4151 4221 4223 4232 4233 5931 .# # # # # # S 3081 3094 4101 4142 4222 4242 5271 6351
-2 .# # # # 4132 5922 6211 .# # # # # # S 3052 3054 3072 4113 4171 4216 4231 5292 .# # # 4111 4141 5923 .# # # 3013 3051 3053 3093 5921
-3 .# # 3012 3043 3071 3092 4131 5291 5932 .# # 3011 5901 .# # # T 3044 .#
-4 . 3042 3091 . 4901 . T 3023 3041 . 3022
-5 . . . 3021 .
-6 . . .
-7 .# Persons <less> Items <frequ>
EACH '# ' IS 50.
51
Annex 2: Tamil Item Map – Item Prefix Deleted
Persons
<more>
Items <rare> >5 ..# # # # #
5 . .# . T 91905
4 . 92705 .# # 91903 91904 .# 72703 81104 81502 92703 .# # # T 72002 81101 92004 92205 92502 92704
3 .# # 72802 92501 .# # # # # 72701 81605 92702 .# # # # # # S 92103 .# # # # # 72404 72801 81003 81603 92101 92104
2 # # # # # # # #
# # # #
72004 81005 81602 81604 92102 .# # # # # # # #
# # # #
51303 72003 72202 72401 81103 81601 91902 92201 92701 .# # # # # # # #
# #
S 72405 72504 81004 81102 81304 81501 81503 92401 .# # # # # # # #
# # #
41003 51603 51704 51705 61802 72301 72503 72505 72604 72702 81001 81105 81303 81505
92301 92305 1 .# # # # # # # #
# # #
50805 51601 51604 51605 51703 61403 72001 72501 72803 81405 81705 92001 92303 92304
92601 .# # # # # # # #
# # #
M 30803 30804 40802 51304 51402 51403 51404 51405 51502 51701 60603 61304 61305 61404
61710 72203 72502 81702 81801 92005 92302 92402 .# # # # # # # #
#
30802 40801 51503 51602 51702 60601 60602 61301 61302 61303 61401 61709 61801 61804
72603 81002 91901 .# # # # # # # # 30103 30801 51504 51505 51804 61105 61708 61803 92202 92203 92204 0 .# # # # # # M 41002 51302 51401 61505 61706 61707 72104 72201 72302 81305 81404 .# # # # # # # 30102 30603 30604 31003 41102 61101 61102 61103 61104 61504 61705 61901 72005 72102
80904 81204 81302 81402 81504 .# # # # # 30505 31002 41103 41403 51805 61005 61405 61501 61603 61704 72103 72105 72403 81201 81202 81203 81703 81704 92802 92803 92805 .# # # # 30504 30702 40603 41101 41301 41302 41303 51501 61604 61805 61904 61905 72101 72402 80901 80905 81401 92003 92801 92804
-1 .# # # S 30101 30701 30903 31001 40602 51801 51802 51803 61202 61402 61601 61702 61902 61903
62002 62003 72602 81301 81403 92002 .# # # 30502 30503 30601 30602 30902 40601 41001 41201 41202 50804 51902 61502 61602 61605
61703 62004 62005 .# # S 30404 30405 30901 40605 41402 41502 50803 51301 51905 61001 61004 61201 61503 72601
80902 80903 81701 .# # 41501 50801 61002 61701 .# # 30402 41401 50802 51903 51904 62001 .# 30403 30501 40604 40702 40703 40902 50306 61003 .# 40701 40901 51901 .# T 30303 30401
-3 .# . T 30304 .# 30202 30203 30305 60301 60302 . 30302
-4 . 30301 . 30201 . .
-5 . <-5 …..#
Persons
<less>
Items <frequ> EACH '# ' IS 66.
52
Annex 3 – Supplementary Tables for Assessment of Basic Competencies
Table A3-1: Descriptives by Grade, with Confidence Intervals
Gr. N Mean S. D. S. E.
95% CI for Mean Lower Upper
Reading 6 1521 4.51 1.936 .050 4.42 4.61 7 1576 4.64 1.837 .046 4.55 4.73
8 1841 4.56 2.013 .047 4.46 4.65
9 1400 5.18 1.470 .039 5.10 5.26
Total 6338 4.70 1.859 .023 4.66 4.75
Writing 6 1520 3.52 1.661 .043 3.44 3.61
7 1576 3.58 1.560 .039 3.50 3.65
8 1841 3.60 1.682 .039 3.53 3.68
9 1399 3.96 1.381 .037 3.88 4.03
Total 6336 3.66 1.592 .020 3.62 3.69
Number 6 1449 11.68 3.438 .090 11.51 11.86
7 1572 11.85 3.420 .086 11.68 12.02
8 1773 11.80 3.488 .083 11.64 11.96
9 1395 12.94 2.832 .076 12.79 13.09
Total 6189 12.04 3.356 .043 11.96 12.13
Table A3-2: Item Percent Correct, Reading/Writing
Task % Corr.
Reading Comprehension 1 85.80 Reading Comprehension 2 78.26
Reading Comprehension 3 71.10
Dictation 77.49
Letter – Salutation 74.37
Letter – Body 74.76
Letter – Sign-off 63.83
Table A3-3: Item Percent Correct, Number (1)
Task % Corr.
Dictated Number 1 77.36 Dictated Number 1 72.95
Dictated Number 1 71.60
Simple Addition 1 93.53
Simple Addition 2 90.76
Simple Subtraction 1 90.76
Simple Subtraction 2 80.66
Simple Multiplication 1 88.70
Simple Multiplication 2 77.98
Simple Division 1 77.65
Simple Division 2 71.29
53
Table A3-4: Item Percent Correct, Number (2)
Task % Corr.
Word Problem – Addition 82.06 Word Problem – Subtraction 65.53
Mental Arithmetic – Addition Problem 78.01
Mental Arithmetic – Subtraction Problem 78.99
Table A3-5: School Mean Scores, Reading
Reading
Total N Mean S. D. S. E. 95% CI for Mean
Lower Upper
Eravur 1 325 5.50 1.014 .056 5.39 5.61 2 472 5.42 1.168 .054 5.31 5.52
3 277 4.57 2.018 .121 4.33 4.81
4 86 5.27 1.582 .171 4.93 5.61
5 309 5.12 1.295 .074 4.97 5.26
6 310 5.63 .896 .051 5.53 5.73
7 182 4.98 1.512 .112 4.76 5.20
8 93 3.94 2.240 .232 3.47 4.40
9 287 5.44 1.315 .078 5.29 5.60
Manmunai S 1 208 5.40 1.475 .102 5.20 5.61
2 597 5.22 1.325 .054 5.12 5.33
3 185 3.61 2.179 .160 3.29 3.93
4 414 3.59 2.146 .105 3.39 3.80
5 212 3.75 2.075 .142 3.46 4.03
6 117 3.68 1.911 .177 3.33 4.03
7 237 3.05 2.522 .164 2.73 3.37
8 201 5.11 1.514 .107 4.90 5.32
9 256 4.20 1.742 .109 3.98 4.41
10 203 4.53 1.784 .125 4.29 4.78
11 154 5.00 1.495 .120 4.76 5.24
13 118 5.33 1.110 .102 5.13 5.53
14 342 4.57 1.841 .100 4.38 4.77
15 85 4.48 2.158 .234 4.02 4.95
16 83 2.82 2.807 .308 2.21 3.43
17 117 4.32 2.322 .215 3.90 4.75
18 78 3.90 2.166 .245 3.41 4.39
19 166 5.03 1.416 .110 4.81 5.25
20 95 4.19 2.340 .240 3.71 4.67
22 75 5.36 .864 .100 5.16 5.56
23 54 3.65 2.208 .300 3.05 4.25
Total 6338 4.70 1.859 .023 4.66 4.75
54
Table A3-6: School Mean Scores, Writing
Writing Total N Mean S. D. S. E. 95% CI for Mean Lower Upper
Eravur 1 325 4.63 .878 .049 4.53 4.73
2 472 4.62 .789 .036 4.55 4.69
3 277 3.61 1.537 .092 3.42 3.79
4 86 4.65 .823 .089 4.47 4.83
5 309 4.55 .819 .047 4.46 4.64
6 310 4.59 .943 .054 4.48 4.70
7 181 3.34 1.610 .120 3.11 3.58
8 93 2.31 1.444 .150 2.01 2.61
9 287 3.98 .991 .059 3.87 4.10
Manmunai S 1 208 4.02 1.251 .087 3.85 4.19
2 597 3.73 1.296 .053 3.62 3.83
3 185 2.76 1.635 .120 2.52 2.99
4 414 2.44 1.809 .089 2.26 2.61
5 211 3.19 1.824 .126 2.94 3.44
6 118 3.50 1.723 .159 3.19 3.81
7 237 2.76 1.861 .121 2.52 3.00
8 201 4.37 1.222 .086 4.20 4.54
9 256 2.67 1.789 .112 2.45 2.89
10 203 3.12 1.638 .115 2.90 3.35
11 154 4.23 1.135 .091 4.05 4.41
13 118 3.56 1.533 .141 3.28 3.84
14 342 3.35 1.629 .088 3.17 3.52
15 85 2.89 1.456 .158 2.58 3.21
16 83 2.77 1.915 .210 2.35 3.19
17 117 3.38 1.960 .181 3.03 3.74
18 78 3.38 1.614 .183 3.02 3.75
19 165 3.10 1.628 .127 2.85 3.35
20 95 4.26 1.552 .159 3.95 4.58
22 75 4.16 1.091 .126 3.91 4.41
23 54 3.52 1.501 .204 3.11 3.93
Total 6336 3.66 1.592 .020 3.62 3.69
55
Table A3-7: School Mean Scores, Number
Number N Mean S. D. S. E. 95% CI for Mean
Lower Upper
Eravur 1 325 13.37 2.133 .118 13.13 13.60
2 472 12.97 2.291 .105 12.77 13.18
3 273 11.33 3.014 .182 10.97 11.69
4 85 13.75 2.017 .219 13.32 14.19
5 307 13.78 1.776 .101 13.58 13.97
6 310 12.16 2.453 .139 11.89 12.44
7 182 10.96 4.153 .308 10.35 11.56
8 93 10.41 3.831 .397 9.62 11.20
9 286 12.20 2.337 .138 11.93 12.47
Manmunai S 1 208 12.51 3.150 .218 12.08 12.95
2 596 13.11 2.738 .112 12.89 13.33
3 127 11.14 3.445 .306 10.54 11.75
4 412 11.47 3.722 .183 11.11 11.83
5 211 10.47 4.031 .278 9.92 11.02
6 116 11.49 3.523 .327 10.84 12.14
7 236 8.91 3.616 .235 8.45 9.37
8 201 12.44 3.394 .239 11.97 12.91
9 186 10.56 4.366 .320 9.93 11.20
10 203 11.12 3.934 .276 10.58 11.67
11 154 11.51 3.152 .254 11.00 12.01
13 116 13.33 2.505 .233 12.87 13.79
14 341 12.82 3.214 .174 12.48 13.16
15 84 11.11 3.624 .395 10.32 11.89
16 83 9.77 4.235 .465 8.85 10.70
17 117 12.11 3.669 .339 11.44 12.78
18 76 11.55 3.235 .371 10.81 12.29
19 165 11.85 3.704 .288 11.29 12.42
20 95 11.88 3.458 .355 11.18 12.59
22 75 13.31 2.342 .270 12.77 13.85
23 54 11.31 3.791 .516 10.28 12.35
Total 6189 12.04 3.356 .043 11.96 12.13
56
Annex 4 – Computing the intra-class correlation
The Intra-class Correlation (ICC) is a measure of homogeneity among analytical units, for
example, between schools in a national educational system. Students and schools are treated
as sources of variance. The school variance component can further be broken down to
investigate disparities between the groupings of schools defined by the stratification applied
in national sample designs. This allows assessment of the efficiency of the school-level
stratifications in reducing the standard errors of the survey estimates. The ICC is therefore
used by sample survey statisticians in deriving efficient sample designs and sample sizes for
hierarchical populations – that is, where the data are hierarchical in nature, with students
within classes, and classes within schools.
A low ICC, say less than 0.25, indicates relatively small variations in performance between
schools. As the ICC increases, it reveals ever-increasing variation in the performance of
schools, with some achieving very high, and others very low, levels of performance.
For the sampling statistician, a system with a low ICC implies a sample design that focuses
more on the within-school component, sampling fewer schools but more students within each
school. As the ICC increases, the focus shifts to sampling more schools, and perhaps fewer
students within schools.
As a population attribute, variability between schools offers a measure of equity, or disparity,
of learning opportunity. In systems with a low ICC, schools perform at roughly equivalent
levels, whereas a high ICC indicates disparity of learning opportunity, with some schools
performing well, while others perform poorly. The ICC therefore incidentally allows a
researcher to address policy concerns related to equity and disparity of learning
opportunities.
The ICC simply expresses the between-school variance as a proportion of the sum of the two
variance components, as described in the following equation:
where B2 is the between-school variance, and w
2 the within-school variance. These
quantities can be derived from a single-level analysis of variance (Neter & Wasserman3, p.
442). In that case, the ICC can be computed as:
3
Neter, John, & Wasserman, William, “Applied Linear Statistical models”, Irwin, 1974
57
where MSB and MSW are defined in the ANOVA table of figure A2-1. In an unbalanced ANOVA,
the quantity n′ must be used. The quantity n′ is estimated as follows (op. cit., p. 528):
where n is the number of sampled schools, and ni is the number of sampled students in school
“i”. This quantity can be interpreted as the average ni in the case of an unbalanced ANOVA. In
this case, n′ = 210.6802.
The ANOVA table below presents the general case with unequal student sample sizes within
schools The ANOVA table presents the general case with unequal student sample sizes within
schools.
Table A2-1: ANOVA Table for ABC Test, Total Score
SSQ df MSQ F Sig.
Between Schools 37499.284 29 1293.079 46.562 .000
Within Schools 170931.807 6155 27.771
Total 208431.090 6184
The ICC is therefore (1293.079-27.771)/(1293.079+210.6802*27.771) = 0.177118
Table A2-2: ANOVA Table for ABC Test, Subscale Scores
SSQ df MSQ F Sig.
Reading Between Groups 3658.156 29 126.143 43.594 .000
Within Groups 18252.970 6308 2.894
Total 21911.126 6337
Writing Between Groups 3331.602 29 114.883 56.897 .000
Within Groups 12732.647 6306 2.019
Total 16064.249 6335
Number Between Groups 8326.703 29 287.128 28.809 .000
Within Groups 61384.206 6159 9.967
Total 69710.909 6188
58
Annex 5 – Summary of Design of System of Placement Tests in a Given Subject
Test for … Duration Content Outcome
Grade 1 (KS1) No written test Guided assessment by teacher Placed as teacher recommends
Grade 2 (KS1) No written test Guided assessment by teacher Placed as teacher recommends
Grade 3 (KS2) 30 minutes Grade 1 & 2 “Pass” stays in grade 3 “Fail” goes into ALP Level1 “No or Minimal Attempt” goes into ALP Foundation Level
Grade 4 (KS2) 45 minutes Part 1, 20 minutes Part 2, 25 minutes
Part 1: Grade 1 & 2 Part 2: Grade 3
“Pass” part 2 – stays in grade 4 “Fail” part 2, “Pass” part 1 – goes into ALP Level 2 “Fail” part 2 & part 1 – goes into ALP Level 1
Grade 5 (KS3) 60 minutes Part 1, 30 minutes Part 2, 30 minutes
Part 1: Grade 1 & 2 Part 2: Grade 3 & 4
“Pass” part 2 – stays in grade 5 “Fail” part 2, “Pass” part 1 – goes into ALP Level 2 “Fail” part 2 & part 1 – goes into ALP level 1
Grade 6 (KS4) 90 minutes Part 1, 45 minutes Part 2, 45 minutes
Part 1: Grade 3 & 4 Part 2: Grade 5
“Pass” part 2 – stays in grade 6 “Fail” part 2, “Pass” part 1 – goes into ALP Level 3 “Fail” part 2 & part 1 – goes into ALP level 2
Grade 7 (KS4) 90 minutes Part 1, 30 minutes Part 2, 30 minutes Part 3, 30 minutes
Part 1: Grade 3 & 4 Part 2: Grade 5 Part 3: Grade 6
“Pass” part 3 – stays in grade 7 “Fail” part 3, “Pass” part 2 – goes into ALP Level 4 “Fail” part 3 & part 2, “Pass part 1” – goes into ALP level 3 “Fail” all 3 parts – goes into ALP level 2
Grade 8 (KS5) 90 minutes Part 1, 45 minutes Part 2, 45 minutes
Part 1: Grade 5 Part 2: Grade 6 & 7
“Pass” part 2 – stays in grade 8 “Fail” part 2, “Pass” part 1 – goes into ALP Level 4 “Fail” part 2 & part 1 – goes into ALP level 3
Grade 9 (KS5) 90 minutes Part 1, 30 minutes Part 2, 30 minutes Part 3: 30 minutes
Part 1: Grade 5 Part 2: Grade 6 & 7 Part 3: Grade 8
“Pass” part 3 – stays in grade 9 “Fail” part 3, “Pass” part 2 – goes into ALP Level 5 “Fail” part 3 & part 2, pass part 1 – goes into ALP level 4 “Fail” all 3 parts – goes into ALP level 3
59
Annex 6: ABC Tests
Test Administrator’s Guide
1. Reading Skills
Tell test takers to read the passage on their test papers. Tell them that after they have finished
reading it, they will have to answer some questions in the spaces under the passage on their answer
sheet. Pause to allow them to read the passage (allow 2-3 minutes). Ask each question in turn.
Speak at a slow normal speed, and clearly. Pause after each question to give the test takers time to
write the answer.
1.1 What does Uday Chandran cultivate in his land?
1.2 Where does he save money?
1.3 How many members are there in his family?
When you have read all three questions, tell the test takers that you will read the questions once
more so they can check their answers. Read the questions again, at normal speed. Then go round
the group of test takers to check that they have understood the task.
2. Writing Skills
Task 2.1: Dictate the following sentence and ask the child to write it.
Tell the test takers that you are going to read a sentence, and they must write it down in the space
on the answer sheet. Tell them that you will read the sentence twice.
Read the sentence once, slowly and clearly.
Our country is very beautiful
Then pause to give the test takers time to write. Go round the group and check that they are writing
in the right place. When they appear to have finished, tell them you will read the sentence again so
they can check their work. Read the sentence a second time, at slow normal speed.
Task 2.2: Ask the child to write a letter to his/her father or to any relative that he/she passed the
examination. Tell them to write one or two sentences only in the space provided. Go round the
group and check that they know what to do.
60
3. Number Skills
Task 3.1: Tell the test takers that you are going to say some numbers, and they must write them
down in the spaces on the paper. Dictate the following numbers with pauses between them, and
ask the test takers to write them down.
Number
5
67
208
Then tell the test takers that they should answer questions 3.2 to 3.6 on their answer sheets, working
as quickly as possible
Task 3.2: Add
i 50 ii 36
+35 +47
Task 3.3: Subtract
I 68 ii 72
-25 -34
3.4 Multiply
i 4 ii 12
x 2 x 6
3.5 Divide
i 2 8 ii 4 48
61
Task 3.6
a. Rajah had Rs. 35. His uncle gave him Rs. 50. How much money does Rajah have now?
b. Kamala has a collection of 72 beads. 34 of them are green and the rest are blue. How many blue
beads has she?
Go around the room to check that the test takers are answering in the correct places.
Task 3.7: Mental Arithmetic
When the test takers appear to have finished, tell them that you will read the last two questions
aloud, and they have to write down the answers in the spaces on their answer sheet. Tell them that
this part is mental arithmetic.
Read the following sums one by one and ask the child to write the answer.
Question
a) In the market you have bought fish for Rs. 75, potatoes for Rs. 25, and green chilis for Tk. 10. How
much money have you spent?
b) You had Rs. 30. You bought a notebook and pencils for Rs. 15. How much money do you have
left?
Read the question once, then pause. Then read the question a second time.
When the test takers appear to have finished, collect in their answer sheets
62
Assessment of Basic Competencies – Student Sheet
Name of Student ……………………….. Name of School ………………………
1. Read this Passage
Uday Chandran is a good farmer. He grows paddy, potatoes and chilis. He uses this to feed his
family. He sells some and saves in the bank. His son and daughter study in the school. Uday
Chandran with his wife and two children is a small family. A small family is a happy family.
1.1……………………………………………………..
1.2……………………………………………………..
1.3……………………………………………………..
2. Writing Skills
2.1……………………………………………………..
Task 2.2:
63
3. Number Skills
3.1:
a. ……………………………………………………..
b. ……………………………………………………..
c. ……………………………………………………..
3.2: Add
i 50 ii 36 +35 +47
3.3 Subtract
i 68 ii 72 -25 -34
3.4 Multiply
i 4 ii 12 x 2 x 6
3.5 Divide
i 2 8 ii 4 48
3.6a Rajah had Rs. 35. His uncle gave him Rs. 50. How much money does Rajah have now?
……………………………………………………..
3.6b Kamala has a collection of 72 beads. 34 of them are green and the rest are blue. How many
blue beads has she?
……………………………………………………..
3.7a……………………………………………………..
3.7b……………………………………………………..
64
Assessment of Basic Competencies – Scoring Guide
1. Reading Skills
1.1 He cultivates paddy, potatoes and chilis.
1.2 He saves money in the bank.
1.3 There are four members in his family.
Score each: Correct = 2; Incorrect = 1; Can’t answer = 0
2. Writing Skills
Task 2.1: Dictation
Score: Correct = 2; Partially correct = 1; Can’t answer = 0
Task 2.2: letter writing
Mark Point Code
Salutation Can write = 1; Can’t write = 0
Message communication Can write = 1; Can’t write = 0
Sincerely Can write = 1; Can’t write = 0
3. Number Skills
Score all number skills items: correct = 1; not correct = 0
Task 3.1: Dictated numbers
a. Five
b. Sixty-seven
c. Two-hundred and eight
65
Task 3.2: Adding
i. 85
ii. 83
Task 3.3: Subtract
i. 43
ii. 38
Task 3.4: Multiply
i. 8
ii. 72
Task 3.5: Divide
i. 4
ii. 12
Task 3.6: Word Problems
a. Rs. 65
b. 37
Task 3.7: Mental Arithmetic
a. Rs. 110
b. Rs. 15
Score the three sections – reading, writing and number skills – separately, and record the three
scores on the student’s sheet.
Totals:
Reading: 6
Writing: 5
Number: 15
66
mbg;gil jpwikfSf;fhd fzpg;gPL - guPPl;ir epu;thfj;jpdu; topfhl;b
1. thrpg;G jpwik
guPl;ir nra;gtu;fis guPl;irf;jhspy; cs;s ge;jpia thrpf;Fk;gb nrhy;yTk;. NkYk; mtu;fs; thrpj;J Kbe;jgpd; Nfl;fg;gl;l Nfs;tpfSf;F nfhLf;fg;gl;l ,;lq;fspy; gjpyspf;f Ntz;Lk; vdTk; $wTk;. mth;fs; Fwpj;j ge;jpia thrpg;gjw;F Neuk; toq;fTk;. (2-3 epkplq;fs;). xt;nthU Nfs;tpiaAk; Kiwahf Nfl;fTk;. mikjpahfTk; nkJthfTk; njspthfTk; fijf;fTk;. guPl;rhu;j;jpfs; gjpyspf;f $batifapy; xt;nthU Nfs;tpfSf;F gpd;Dk; epWj;jp thrpf;fTk;. 1.1 cjar;re;jpud; jdJ fhzpapy; vt;tifahd gapu;fis gapupLfpwhu;? 1.2 mtu; gzj;ij vt;thW Nrkpf;fpwhu;? 1.3 mtUila FLk;gj;jpy; vj;jid mq;fj;jtu;fs; ,Uf;fpwhu;fs;? %d;W Nfs;tpfisAk; thrpj;J Kbj;j gpd;du; rup gpio ghu;f;f cjTk; nghUl;L ,d;ndhU Kiw mtw;iw kPz;Lk; thrpg;gjhf mwptpf;fTk;. kPz;Lk; Nfs;tpfis nkJthf thrpf;fTk;. NkYk; mtu;fs; jq;fsJ nraw;ghl;bid KOikahf tpsq;fpf;nfhz;bUf;fpwhu;fsh vd mwpe;J nfhs;Sk; nghUl;L guPl;rhj;jpfis tyk; tuTk;. .
2. vOj;J jpwik. 2.1 nrhy;tnjOjyhf vOJk;gb gps;isia mwpTWj;jp; fPo;tUk; thf;fpaj;ij thrpf;fTk;. ePq;fs; xU thf;fpaj;ij thrpf;fg; NghtjhfTk; mjid cupa ,lj;jpy; vOJk;gbAk;
guPl;rhj;jpfsplk; nrhy;yTk.; NkYk;; me;j thf;fpakhdJ ,U Kiwfs; thrpf;fg;gLk; vdTk; mwptpf;fTk;. xU Kiw mt;thf;fpaj;ij nkJthfTk; njspthfTk; thrpf;fTk;.
“vkJ ehL kpfTk; mofhdJ”
thrpj;J Kbj;jJk; guPl;rhj;jpfs; vOtjw;fhd mtfhrj;ij toq;fTk;. gpd;du; mtu;fs; cupa ,lj;jpy; vOjpAs;shu;fshntd mwpe;J nfhs;s mtu;fis tyk; tuTk;. mtu;fs; vOjp Kbj;Jtpl;ljhf njd;gl;lnjd;why; rup gpio ghu;gjw;fhf kPz;Lk; xU Kiw thrpg;gjhf mwptpf;fTk;. ,uz;lhtJ KiwahfTk; thf;fpaj;ij nkJthf thrpf;fTk;.
2.2. fbjk;. jhd; guPl;irapy; rpj;jp mile;Jtpl;lij njuptpj;J jdJ je;ijf;Nfh my;yJ NtW cwtpdUf;Nfh fbjk; vOJk;gb gps;isaplk; nrhy;yTk;. mjw;nfd toq;fg;gl;l ,lj;jpy; khj;jpuk; xd;W my;yJ ,uz;L trdq;fis vOJk;gb nrhy;yTk;. mtu;fs; jhq;fs; nra;a Ntz;ba nraw;ghl;bid KOikahf tpsq;fpf;nfhz;bUf;fpwhu;fsh vd;gjid mwpe;J
nfhs;Sk; nghUl;L guPl;rhj;jpfis tyk; tuTk;;.
3. fzpjj; jpwd; 3.1 ePq;fs; rpy ,yf;fq;fis $wg; NghtjhfTk; mitfis guPl;ir jhspy; cupa ,lj;jpy; vOJk;gbAk; guPl;rhu;j;jpfsplk; nrhy;yTk;. fPo; Fwpg;gplgl;l ,yf;fq;fis ,yf;fq;fSf;F ,ilNa ,ilntsp tpl;L nrhy;yTk;.
67
,yf;fq;fs;
5
67
208
3.2 njhlf;fk; 3.6 tiuahd tpdhf;fSf;F kpf tpiuthf tpilj; jhspy; tpilaspf;Fk; gb guPl;rhj;jpfsplk; $wTk;. 3.2 $l;ly;;
i. 50 ii 36
+ 35 + 47
3.3.fopj;jy;
i 68 ii 72
- 25 - 34
3.4 ngUf;fy;
i 4 ii 12
x 2 x 6
3.5 tFj;jy;
i 2 8 ii 4 48
3.6 a). uhIh &gha; 35 itj;jpUe;jhu;. mtUila khkh mtUf;F &gha; 50 nfhLj;jhnud;why;
jw;nghOJ uhIhtplk;; vt;TsT gzk; cs;sJ?
b. fkyh 72 kzpfis Nru;j;J itj;jpUe;jhs;. mitfspy; 34 gr;ir epwkhdit>
kpFjpaidj;Jk; ePy epwkhdit vdpy; mtsplk; cs;s ePy epw kzpfspd; vz;zpf;if vj;jid?
guPl;rhj;jpfs; cupa ,lq;fspy; tpilaspf;fpwhu;fsh vdgjid Rw;wp ghu;f;fTk;.
3.7 kdf;fzpjk;
guPl;rhj;jpfs; tpilaspj;J tpl;ljhf njupe;jhy;> ,Wjpahd ,uz;L tpdhf;fisAk; rj;jkhf thrpf;fg;NghtjhfTk;> mtw;wpw;F tpilfis cupa ,lq;fspy; vOJk;gbAk; $wTk;. NkYk; ,J kdf; fzpj gFjp vd;Wk; $wTk;.
68
fPo; tUk; tpdhf;fis xd;wd; gpd; xd;whf thrpf;fTk;. gps;isfis tpilaspf;Fk;gb
$wTk;. tpdhf;fs;: m) ePq;fs; re;ijapy;; 75 &gha;F kPDk;> 25 &gha;F fpoq;Fk;> 10 &gha;F gr;ir kpsfhAk; nfhs;tdT nra;fpwPu;fs; vdpy; ePq;fs nkhj;jkhf nryT nra;j njhif vd;d? M) cq;fsplk; 30 &gha; ,Ue;eJ. mjpy; 15 &gha;F Fwpg;G Gj;jfk; xd;iwAk; ngd;rpy;fisAk; nfhs;tdT nra;jPu;fnsd;why;> cq;fsplk; kpFjpahf ,Uf;Fk; njhif vd;d?
tpdhit xU Kiw thrpj;J rpwpJ Neuk; jhkjpj;J kPz;Lk; ,uz;lhk; Kiwahf thrpf;fTk; guPl;rhj;jpfs; tpilaspj;J Kbj;jpUg;ghu;fs; vdpd;> tpilj;jhs;fis mtu;fsplkpUe;J ngw;Wf;nfhs;sTk;.
69
mbg;gil jpwikfSf;fhd fzpg;gPL
khztdpd; ngau;: ----------------------------------- ghlrhiyapd; ngau;:-------------------------------
1. ,g; ge;jpia thrpf;fTk;: cjare;jpud; xU jpwikahd tptrhap. mtu; ney;> cUisf;fpoq;F> kpsfha; Nghd;wtw;iw gapupLthu;. ,tw;iw jdJ FLk;gj;jpw;fhf gad;gLj;Jthu;. rpytw;iw tpw;gid nra;J gzj;ij tq;fpapy; Nrkpg;ghu;. mtUila kfDk;> kfSk; ghlrhiyapy; fy;tp gapYfpwhu;fs;. kidtp> ,U gps;isfs; cl;gl cjare;jpudpd; FLk;gk; xU rpW FLk;gk;. xU rpwpa FLk;gk; re;Njhrkhd FLk;gkhFk;..
1.1 -------------------------------------------------------------------------------------------------------
1.2 -------------------------------------------------------------------------------------------------------
1.3 -------------------------------------------------------------------------------------------------------
2. vOj;J jpwd;
2.1 -------------------------------------------------------------------------------------------------------
2.2.
70
3. vz;fs; jpwd;
3.1 m) ------------------------------------------------------------------------------------------------------------ M) ------------------------------------------------------------------------------------------------------------
3.2 $l;Lf
i. 50 ii 36 + 35 + 47
3.3.fopf;Ff
i 68 ii 72 - 25 - 34
3.4 ngUf;Ff
i 4 ii 12 x 2 x 6
3.5 tFf;Ff
i 2 8 ii 4 48
3.8 a. uhIh &gha; 35 itj;jpUe;jhu;. mtUila khkh mtUf;F &gha; 50 nfhLj;jhnud;why;
jw;nghOJ uhIhtplKs;s KOj;njhifAk; vt;TsT.?
-------------------------------------------------------------------------------------------------------
b. fkyh 72 kzpfis Nru;j;J itj;jpUe;jhs;. mitfspy; 34 gr;ir epwkhfTk;
kpFjpahdit ePy epwkhfTk; ,Ue;jd vdpy; mtsplk; ,Uf;Fk; ePy epw kzpfspd; njhif vd;d?
-------------------------------------------------------------------------------------------------------
3.7 m………………………………………………………………….
3.7 M ………………………………………………………………..
71
mbg;gil jpwikfSf;fhd fzpg;gPL –
1. thrpg;G jpwd;
1.1 mtd; ney;> cUisf;fpoq;F> kpsfha; Nghd;wtw;iw gapupLfpwhu;. 1.2 mtd; jdJ gzj;ij tq;fpapy; Nrkpf;fpwhu;. 1.3 mtDila FLk;gj;jpy; ehd;F mq;fj;jtu;fs; cs;sdu;.
Gs;spfs;.: rup = 2, gpio = 1, tpilaopf;ftpy;iy = 0
2. vOj;J jpwd;
2.1 nrhy;tnjOjy;
Gs;spfs;: rhp = 2XusT rhp = 1, tpilaopf;ftpy;iy = 0
2.2 fbjk; vOJjy;
Fwpg;G FwpaPL
tpspg;G vOj KbAk; = 1, vOj ,ayhJ = 0
jfty; vOj KbAk; = 1, vOj ,ayhJ = 0
cd;ikAs;s vOj KbAk;= 1, vOj ,ayhJ = 0
;
3. vz;fs; jpwd;
vz;jpwDf;fhd KOg;Gs;spfs; : rhp = 1, gpio = 0
3.1 nrhy;tnjOjy;
m) Ie;J
M) mWgj;jp VO ,) ,UEhw;wp vl;L
3.2 $l;Lf
1. 85 2. 83
3.3 fopf;Ff
1. 43 2. 38
3.4 ngWf;Ff
72
1. 8 2. 72
3.5 tFf;Ff
1. 4 2. 12
3.6
m) 65 &gha; M) 37
3.7 kzf; fzpjk;
m) 110 &gha; M) 15 &gha;
thrpg;Gj; jpwd;> vOj;Jj; jpwd;> vz;jpwd; Mfpa Kd;W gFjpfSf;Fk; jdpj;jdpahf khztu;fSf;F fpilj;j Gs;spfis khztu; gjpT ml;ilapy; gjpjy; Ntz;Lk;.
73
Annex 7 – Assessment Guide for Grade 1 and 2 Teachers
1 , 2 juq;fspy;; khztu;fis mDkjpf;Fk; Jupj fw;wy; epfo;r;rp jpl;lk;
Mrpupau; ifE}y;
1. gpd;dzp
2010 khu;r; khjj;jpy; tlf;F fpof;F khfhzq;fspy; cs;s gy khztu;fspd; Nju;r;rp kl;lk; kjpg;gPL nra;Ag;gl;lJ. ,lg;ngau;Tfspdhy; jkJ ghlrhiy ehl;fspy; fw;wy; nraw;ghLfis ,oe;J taJf;Fk; tFg;Gf;Fk; Vw;w Nju;r;rp
kl;lk; ,y;yhj khztu;fis ,dq;fz;L Jupj fw;wy; epfo;r;rpj; jpl;lj;jpy; ,izj;J Njitahd Nju;r;rpkl;lj;ij miltjw;F cjTtNj ,jw;fhd fhuzkhFk;.
3 – 9 tFg;G / juq;fSf;F khztu;fis mtu;fs; ngw;w Gs;spfspd; mbg;gilapy; Nru;j;Jf; nfhs;sg; gLthu;fs;. 1 , 2
juk; / tFg;G khztu;fs; guPl;rpf;fg; glkhl;lhu;fs;. gjpyhf mtu;fis Mrpupau;fs; kjpg;gPL nra;thu;fs;. Mrpupau;fNs
Jupj fw;wy; epfo;r;rpj; jpl;lj;jpy; ahu;> ve;j kl;lj;Jf;Fr; Nru;j;Jf; nfhs;sg; gLthu;fs; vd;gijj; jPu;khdpg;ghu;fs;.
khztiu juk; 1 y; Nru;j;Jf; nfhs;Sjy;.
khztu; mbg;gil kl;lj;Jf;Fupa Kf;fpa Nju;r;rp milTfisg; ngw;Ws;shuh vd;gij kjpg;gpl ,izg;G-1 gad;gLj;jTk;.
rupahf Nju;r;rp milTfisg; ngw;W Kbj;jpUe;jhy; ,izg;G-2 gbtj;jpy; „√.‟
rup „milahsk; ,lTk;. Nju;r;rpailatpy;iy my;yJ cjtp Njit vdpd; „ x „milahsj;ij ,lTk;.
mbg;gil kl;lg; guPl;irapy; 8 Kf;fpa Nju;r;rpfspy; %d;W‟ x‟ Nju;r;rpapd;ik milahsq;fisg; ngw;wpUe;jhy; mtiu mbg;gilj; juj;jpNyNa Nru;j;Jf; nfs;sNtz;Lk;.
khztiu juk; 2 y; Nru;j;Jf; nfhs;Sjy;.
khztu; Kjyhe;ju kl;lj;Jf;Fupa Kf;fpa Nju;r;rp milTfisg; ngw;Ws;shuh vd;gij kjpg;gpl ,izg;G-3 gad;gLj;jTk;.
rupahf Nju;r;rp milTfisg; ngw;W Kbj;jpUe;jhy; ,izg;G – 4 gbtj;jpy; √.
„rup „milahsk; ,lTk;. Nju;r;rpailatpy;iy my;yJ cjtp Njit vdpd; „ x „milahsj;ij ,lTk;.
Kjyhe;ju kl;lg; guPl;irapy; 20 Kf;fpa Nju;r;rpfspy; Ie;J njhlf;fk; gj;J tiu ‟ x‟ Nju;r;rpapd;ik milahsq;fisg; ngw;wpUe;jhy; mtiu Kjyhk; juj;jpNyNa Nru;j;Jf; nfs;sNtz;Lk;.
74
Kjyhe;ju kl;lg; guPl;irapy; 20 Kf;fpa Nju;r;rpfspy; gj;Jf;F Nky; ‟ x‟ Nju;r;rpapd;ik milahsq;fisg; ngw;wpUe;jhy; mtiu mbg;gilj; juj;jpNyNa Nru;j;Jf; nfs;sNtz;Lk;.
75
,izg;G 3: juk; 1f;fhd Nju;r;rp kjpg;gPl;L topfhl;ly;.
gFjp 1 fzpjk;
Nju;r;rp : fw;gtuhy; kjpg;gPL
1. 1 njhlf;fk; 9 tiu vOjKbAk;
,yf;fq;fSk; mtw;wpd; FwpaPl;Lg; ngau;fisAk; 1 – 9 tiu vOJf. xU gps;isf;F ehd;D ml;ilfisf; nfhLf;fTk;. mjidg; ghu;j;J thrpj;J> jkJ nfhg;gpapy; vOjTk; nrhy;Yq;fs;. mjid mtjhdpAq;fs;. vj;jid ,yf;fq;fis rupahf vOjpAs;sdu; vd;gijf; FwpAq;fs;.
2. 1 njhlf;fk; 10 tiu nghUl;fis vz;zTk; vOjTk; KbAk;.
Gps;isaplk; rpW fw;fisf; nfhLq;fs; mtw;iw vz;Zk;gb $Wq;fs;. rupahf vz;Zfpwhu;fsh vd;W mtjhdpj;Jf; FwpAq;fs;.
3. 10 f;Ff; fPo; ,uz;L Nrhbg; nghUl;fis $l;l KbAk;
,uz;L fw;Ftpaiy Mf;Fq;fs;. vj;jid fw;fs; cs;sd vd;W gps;isfsplk; NfSq;fs;.
4. vJ ePsk;> vJ fl;il vd %d;W nghUl;fis xg;gpl;Lg; ghu;j;Jf; $wKbAk;.
xU khztuplk; msTNfhy; xd;iwf; nfhLq;fs;. tFg;giwapy; cs;s tpj;jpahrkhd nghUl;fis msf;fr; nrhy;Yq;fs;. (thq;F> Nkir> Nghd;wit) mtu;fs; msf;Fk;NghJ mtjhdpAq;fs;. vJ ePsk;. vJ mfyk; vdf; Nfl;L vOjr; nrhy;Yq;fs;. rupahd tpilfisf; ftdpj;J FwpAq;fs;.
5. ePsj;ij mstpl KbAk; xU khztuplk; msTNfhy; xd;iwf; nfhLq;fs;. tFg;giwapy; cs;s nghUl;fis msf;fr; nrhy;Yq;fs;. (nl];f;> thq;F> Nkir> Nghd;wit) mtu;fs; msf;Fk;NghJ mtjhdpAq;fs;. vJ ePsk;. vJ mfyk; vdf; Nfl;L vOjr; nrhy;Yq;fs;. rupahd tpilfisf; ftdpj;J FwpAq;fs;.
6. &gha; 1, &gha; 2 , 5 rjk; Mfpatw;iw ,dq;fz;L NtWgLj;j KbAk;.
Gps;isaplk; &gha; 1, &gha; 2 , 5 rjk; Mfpatw;iwf; fye;J nfhLq;fs;. &gha; 1, &gha; 2 , 5 rjk; Mfpatw;iwf; fz;Lgpbf;fr; nrhy;Yq;fs;. rupahfj; njupe;Js;shuh vd mtjhdpj;Jf; FwpAq;fs;.
7. ehzaq;fisg; gad;gLj;jp gzg;gupkhw;wk; %yk; tpahghuk; nra;aKbAk;.
nghUl;fspd; glq;fisAk;> mtw;wpd; tpiyfisAk; nfhz;l gl;baiy khztuplk; nfhLq;fs;. tpiyfs; ehzaj;jpy; (&gha; 1, &gha; 2 , 5 rjk;) Fwpg;gplg; gl;bUf;f Ntz;Lk;. ,uz;L %d;W nghUl;fisf; fhl;b tpiy Nfl;fTk;. ,Utif ehzaj;ijf; nfhLj;J me;jg; nghUSf;F ,J NghJkh vdf; Nfl;fTk;. tpil rupahdjh vdf; Fwpj;Jf; nfhs;f.
8. ,U gupkhz tbtq;fisf; nfhz;L ,yFthd cUtq;fisr; nra;aKbAk;.
xU rPuhd tbtq;fs; jug;gl;Ls;sd. mLj;jLj;J tuNtz;ba tbtq;fs; vd;d vd;W tpdhTq;fs;. rupahfr; nrhy;fpwhu;fsh vd mtjhdpj;J Fwpg;gpLf.
76
gFjp 2 – jkpo; nkhop
Nju;r;rp : fw;gtu; kjpg;gPL
1. #oypy; cs;s xypfis ,dq;fz;L NtWgLj;jp mwptu;
glq;fs; xl;ba ,uz;L ml;ilfisf; nfhLq;fs; xU ml;ilia ePq;fs; ifapy; vLj;J cau;j;jpg; gpbAq;fs;. gps;is glj;jpy; cs;s nghUSf;F Vw;g xypia vOg;Gthu;.(
kzpapd; glk;> G+idapd; glk;)> khztu; rupahfr; nrhy;fpd;whuh vd mtjhdpj;Jg; gjpAq;fs;. gpiofs; Vw;Wf; nfhs;sg;gl khl;lhJ.
2. nrhw;fs;> nrhw;njhlu;fspd; xypNgjq;fis ,dq;fz;L NtWgLj;jp mwptu;
,uz;L nrhw;njhlu;fs; vOjpa ml;ilfis khztuplk; toq;Fq;fs;. xU nrhw;njhliu ePq;fs; $Wq;fs;. khztu; rupahd ml;ilia vLj;Jf; fhl;b thrpg;ghu;. ,t;thW ehd;F jlitfs; nra;Aq;fs;. gpiofs; Vw;Wf; nfhs;sg;gl khl;lhJ.
3. rpy xj;j nrhw;fSf;Fs; Nru;e;jpUf;Fk; tpj;jpahrkhd nrhw;fis ,dq;fhZtu;.
gpd; tUk; nrhw;fis vOJf. xU gps;isf;F xU ml;il nfhLq;fs;. mjpy; vOjpAs;sij thrpf;Fk;gb $Wq;fs;. rupahf tpj;jpahrkhd nrhw;fis ,dq;fz;L thrpf;fpwhuh vd;W mtjhdpAq;fs;. Kjy; ml;ilia thrpg;gjpy; f];lk; ,Ue;jhy; ,uz;lhtJ ml;iliaf; nfhLj;J thrpf; tplTk;. Rupahf thrpj;jhy; rup vdg; gjpAq;fs;.
,y;iyahapd; cjtp Njit ‘x ‘ vdg; gjpAq;fs;
miy - mis – mio
fis – fiy - fio
4. nfhLf;fg;gl;l tbtj;jpDs; epwe;jPl;Ltu;
xU tiuglj;ij khztuplk; nfhLq;fs;.( ahid my;yJ khk;gok;) mtu;fis
mg;glj;Jf;F epwe;jPl;lr; nrhy;Yq;fs;. NfhLfSf;Fs; epwep jPl;Lfpwhuh vdgijf; ftdpj;Jg; gjpAq;fs;.
5. nrhw;fSf;F Vw;w glj;jpid tiutu;.
Mrpupau; xU khztiu gf;fj;J miwapDs;
khztuplk; xU kuj;ij tiuAkgb $Wq;fs;. xU gwit kuj;jpy; ,Ug;gJ Nghy; tiuAk;gb $Wq;fs;. khztupd; tiujiy mtjhdpj;J mJ milahsq;fhzf; $bajhf
cs;sjh vdgijg; gjpAq;fs;
ngupa fsk;
ngupa fyk;
nts;isf; nfhf;F
nty;yf; fl;b
77
Nju;r;rp : fw;gtu; kjpg;gPL
6. mwpTWj;jYf;F Vw;wthW nraw;gLtu;.
khztuplk; nghJthd %d;W nghUl;fisf; nfhLq;fs;.( gid tpij> Nghj;jy; %b>
ngd;rpy; Nghd;wit) ngd;rpiy vLf;Fk;gb $Wq;fs;. Xu Nghj;y; %biaj; jUkhW NfSq;fs;. gid tpijfia ngd;rpYf;Fk;> Nghj;jy; %bf;Fk; ,ilapy; itf;Fk;gb
$Wq;fs;. khztdpd; gpujpgypg;gpid mtjhdpAq;fs;. ,t;thW gy;NtW nraw;ghLfs; nfhLf;fyhk;. gpiotpl;lhy; Fwpg;gpLq;fs;.
7. njhlu;ghly; eltbf;iffSf;fh mtjhdkhfr; nrtpkLg;gu;.
xU khztiu gf;fj;J miwapDs; epw;Fk;gb $Wq;fs;. ,d;DnkhU khztiu mtUf;Fj; njupAk;gb epw;f tpLq;fs;. Kd;Df;F epw;gtiu mire;J Nghyr; nra;ar; nrhy;Yq;fs;. ,g;NghJ miwapy; ,Uf;Fk; khztuplk; Kd;dhy; epwgtu; vd;d nra;fpwhu;? vd
tpdTq;fs;. gy;NtW tpdhf;fis tpdtyhk;. miwapy; ,Ug;gtu; cq;fs; tpdhTf;Fg; gjpy; $Wtij mtjhdpj;J gjpAq;fs;. miwapDs; epw;Fk; khztu; mtuplk;
8. ,yFthd nra;jpfis rupahfg; gfpu;e;J nfhs;tu;.
Mrpupau; khztuplk; fhiyapy; nra;j Ntiyfs; gw;wp ciuahlyhk;.- vg;NghJ gLf;ifia tpl;L vOe;jhu;> ghlrhiyf;Fg; Nghtjw;F vg;gbj; jahuhthu;> fhiyapy; vjid cz;ghu;? Nghd;w tpdhf;fis tpdtyhk;. khztu; rhjhuzkhfr; ruskhf tpilaspf;fpwhuh vd;W mtjhdpAq;fs;. mg;gbj; jLkhwpdhy; vjtp Njit vdf; Fwpgg;plyhk;.
9. nrtpkLj;jij my;yJ ghu;j;jij $Wthu;fs;.
mz;ikapy; njhiyf;fhl;rpapy; ghuj;j jq;fSf;Fg; gpbj;j tplak; xd;iwf; NfSq;fs;. myy;J khztUf;Fg; gpbj;j fijnahd;iwf; $Wk;gb NfSq;fs;. mtu; $Wk;NghJ xU rPuhfr; nrhy;fpwhuh vd;W mtjhdpj;Jf; FwpAq;fs;.
10. nrhw;njhlu;fisAk; trdq;fisAk; cuj;J thrpg;gu;.
gpd;tUd Nghd;w nrhw;njhlu;fisAk;> trdq;fisAk; ml;ilapy; vOJf.
rpd;df; FUtp
rpd;df; FUtp gwf;FJ.
vdJ eha;f;Fl;b
vdJ nts;isg; gR.
nrhw;njhlu;> trdq;fs; vOjpa ml;ilia khztuplk; nfhLj;J thrpf;Fk;gb $Wq;fs;. Kjy;Kiw thrpf;f KbahJ jLkhwpdhy; NtnwhU ml;iliaf; nfhLq;fs;. rupahf thrpj;jhy; rup vdg; gjpAq;fs;. ,y;iyahapd; cjtpNjit vdf; Fwpg;gplTk;.
78
Nju;r;rp : fw;gtu; kjpg;gPL
11. rpWtu; gj;jpupiffs;> fijg; Gj;jfq;fis thrpg;gu;.
(jkpo; 2> ghlk; 8> gf;fk;13) juk;-1 f;fhd fijg; Gj;jfj;jpy; ,Ue;J fPo; fhZk;
fijiaf; nfhLq;fs;.
vy;yhk; ed;ikf;Nf
xU kdpjd; gazk; nra;jhd;. fLikahd ntapyhf ,Ue;jJ. fisg;gpdhy; Myku epoypy; ,Ue;jhd;. mz;zhu;e;J Mykuj;ijg; ghuj;jhd;. mjd; ,iyfs; ngupjha; ,Ue;jd.
Goq;fs; kpfr; rpwpjha; ,Ue;jd. “,e;jg; ngupa kuj;Jf;F ,e;jr; rpd;dg; goq;fsh?
Ntbf;if.” Vd;W nrhy;yp ,Ue;jhd;. mg;gbNa gLj;J cwq;fpdhd;. gok; xd;W mtdJ
new;wpapy; tpOe;jJ. mtd; jpLf;Fw;W vOe;jhd;. new;wpiaj; jltpdhd;. nkypjha; nehe;jJ.
“flTNs Mykuj;jpd; gok; ngupjha; ,Ue;jhy; vdJ fjp vd;dthapUf;Fk;.? ,iwtd;
vy;yhk; mwpe;Nj gilj;jpUf;fpwhd;. mtd; filg;G vy;yhk; ed;ikf;Nf” vd;W vOe;J
nrd;whd;.
,f;fijia thrpf;Fk;gb $Wq;fs;. thrpf;Fk;NghJ fijia tpsq;fpf; nfhs;fpwhuh ,y;iyah vd;gij mtjhdpAq;fs;.( thrpf;Fk;NghJ jLkhw;wk;> ntWg;G>gpiotpLjy;>Mfpad voyhk;. MdhYk; thrpf;Fk; gps;isf;F jhd; thrpg;gJ tpsq;Ffpwjh vd;gijj;jhd; ghu;f;f
Ntz;Lk;)
12. Mrpupau; nrhy;tij nrtpkLj;J vOJthu;fs;
nrhy;tnjOJjy;: ePq;fs; nrhy;tijg; gps;isaplk; vOJk;gb $Wq;fs;. nkJthf fPo;fhZk; nrhw;fisg; Nghd;w nrhw;fisj; njspthff; $Wq;fs;. rw;W epWj;jptpl;L khztu; jhk; vOjptw;iw gurPypg;gjw;fhf jpUg;gpf; $Wq;fs;. khztu;fs; gpiofs; tpl;bUf;fpwhu;fsh vdg; ghUq;fs;.
xU gpio kl;Lk; Vw;Wf; nfhs;syhk;.
MLk;. Kl;Lk;> je;jJ> tpis> jtis> fpsp
79
,izg;G 4: ngWNgw;W mwpf;if – Kjyhe;ju Nju;r;rp kl;lk;
gFjp 1: fzpjk;
kjpg;gPL
khztu; ngau;: nkhj;jk 1
njhlf;fk; 9
tiu vOjKbAk
1 njhlf;fk; 1
0 tiu nghUl;fis
vz;zTk; vOjTk; KbAk;.
10
f;Ff; fPo; ,uz
;L Nrhbg; nghUl;fis
$l;l KbAk;
vJ ePsk;> vJ fl;il vd %d;W
nghUl;fis xg;gpl;Lg; ghu;j;Jf;
$wKbAk;.
ePsj;ij mstpl KbAk;
&gha;
1, &
gha;
2 ,
5 rjk; Mfpatw;iw
,dq;fz;L NtWgLj;j KbAk;.
ehz
aq;fisg; gad;gLj;jp gzg;gupkhw;wk;
%yk; tpahghuk; nra;aKbAk;.
,U gupkhz
tbtq;fisf; nfhz
;L
,yFthd
cUtq;fisr; nra;aKbAk
1
2
3
4
5
6
7
8
9
10
Correct ( ✓ )
Needs help ( x )
80
gFjp 2 : jkpo; nkhop
kjpg;gPL
khztu; ngau;: nkhj;jk #oypy; cs;s xypfis ,dq;fz;L
NtWgLj;jp mwptu;
nrhw;fs;> nrhw;njhlu;fspd;
xypNgjq;fis ,dq;fz;L NtWgLj;jp
mwptu;
rpy xj;j nrhw;fSf;Fs; Nru;e;jpUf;Fk;
tpj;jpahrkhd
nrhw;fis
,dq;fhZ
tu;.
nfhLf;fg;gl;l tbtj;jpDs;
epwe;jPl;Ltu;
nrhw;fSf;F Vw;w glj;jpid tiut
u;.
mwpTWj;jYf;F Vw;wthW
nraw;gLtu;.
njhlu;ghly; eltbf;iffSf;fh
mtjhd
khfr; nrtpkLg;gu;.
,yFthd
nra;jpfis rupahfg;
gfpu;e;J nfhs
;tu;.
.
nrtpkLj;jij my;yJ ghu;j;jij
$Wthu;fs;.n
. nrhw;njhlu;fisAk; trdq;fisAk;
cuj;J thrpg;gu
rpWtu; gj;jpupiffs;> fijg;
Gj;jfq;fis thrpg;gu;.
Mrpupau; nrhy
;tij nrtpkLj;J
vOJthu;fs
1
2
3
4
5
6
7
8
9
10
Correct ( ✓ )
Needs help ( x )