Date post: | 24-Dec-2015 |
Category: |
Documents |
Upload: | melvin-stone |
View: | 217 times |
Download: | 2 times |
National Assessment Findings National Assessment Findings (Grade 4 & 8)(Grade 4 & 8)
National Education Assessment System (NEAS)National Education Assessment System (NEAS)
Ministry of Education (Pakistan)Ministry of Education (Pakistan)
South Asia Regional Conferenceon
Quality Education for All
(October 24, 2007)
22
Vision & Mission StatementVision & Mission Statement
VisionVision
To provide fair and valid Assessments to promote To provide fair and valid Assessments to promote
the quality of children's learning in Pakistanthe quality of children's learning in Pakistan
MissionMissionTo carry out fair and valid National Assessments To carry out fair and valid National Assessments
along with related activities with the overall along with related activities with the overall
objectives of enhancing quality, equity & access to objectives of enhancing quality, equity & access to
educationeducation
33
Purpose of National Assessment Purpose of National Assessment in Pakistanin Pakistan
Monitoring the performance of the education systemMonitoring the performance of the education system
Providing feed back to improve the quality of services in Providing feed back to improve the quality of services in education system (curriculum, textual materials, education system (curriculum, textual materials, teachers’ training and delivery)teachers’ training and delivery)
Setting and disseminating realistic standards of students Setting and disseminating realistic standards of students achievementachievement
Identifying correlates of achievementIdentifying correlates of achievement
Providing information to policy makers for effective Providing information to policy makers for effective interventionsinterventions
3
44
National Assessment ScheduleNational Assessment Schedule
LevelLevel : : Grade 4 and 8Grade 4 and 8Subjects: Languages, Maths, General Science and Social StudiesSubjects: Languages, Maths, General Science and Social Studies
YearYear Grade and SubjectGrade and Subject Test TypeTest Type
20042004 GradeGrade 4 (Mathematics, Languages 4 (Mathematics, Languages Pilot TestingPilot Testing
20052005Grade 4 (Mathematics, Languages)Grade 4 (Mathematics, Languages)
GradeGrade 4 (General Science, Social Studies) 4 (General Science, Social Studies)
Large Scale TestingLarge Scale Testing
Pilot TestingPilot Testing
20062006
Grade 4 (Mathematics, Languages, General Grade 4 (Mathematics, Languages, General Science, Social Studies)Science, Social Studies)
GradeGrade 8 (Mathematics, Languages) 8 (Mathematics, Languages)
Large Scale TestingLarge Scale Testing
Pilot TestingPilot Testing
20072007Grade 8 (Mathematics, Languages)Grade 8 (Mathematics, Languages)
GradeGrade 8 (General Science, Social Studies) 8 (General Science, Social Studies)
Large Scale TestingLarge Scale Testing
Pilot TestingPilot Testing
20082008Grade 4 (Mathematics, Languages)Grade 4 (Mathematics, Languages)
Grade 8 (General Science, Social Studies)Grade 8 (General Science, Social Studies)
Large Scale TestingLarge Scale Testing
Large Scale TestingLarge Scale Testing 4
55
National Sample of NEASNational Sample of NEAS
NEAS uses a two-stage stratified random cluster sample NEAS uses a two-stage stratified random cluster sample for G-4 and G-8.for G-4 and G-8.
Explicit strata are :Explicit strata are :• Gender: male-female – two explicit strata.Gender: male-female – two explicit strata.• Location: urban-rural – two explicit strata.Location: urban-rural – two explicit strata.• Province/Area: four provinces and four areas – eight explicit Province/Area: four provinces and four areas – eight explicit
strata.strata.
Total proposed explicit strata: 2x2x8=32. Total proposed explicit strata: 2x2x8=32.
The implicit strata are school type and districtThe implicit strata are school type and district
Contd…
66
National Sample of NEASNational Sample of NEAS
The basic sampling frames are taken from the NEMIS The basic sampling frames are taken from the NEMIS database.database.
Within a stratum, all schools are included, with selection Within a stratum, all schools are included, with selection probability proportional to size (PPS).probability proportional to size (PPS).
Clustering occurs only at the school level. A maximum Clustering occurs only at the school level. A maximum of 20 and minimum 6 students are selected from each of 20 and minimum 6 students are selected from each school.school.
In some rural areas, up to 2% of students in the smallest In some rural areas, up to 2% of students in the smallest
schools are eliminated from frames, and additional schools are eliminated from frames, and additional schools are sampled.schools are sampled.
77
Geographical Distribution of NEAS Geographical Distribution of NEAS Primary Sampling UnitsPrimary Sampling Units
88
Schools Sampled – Planned and Schools Sampled – Planned and Achieved Achieved
Province/AreaProvince/Area
Grade-4 (2006)Grade-4 (2006)Planned Achieved Planned Achieved
% %
Grade-8 (2007)Grade-8 (2007)Planned Achieved Planned Achieved
%%
BalochistanBalochistan 110110 8181 100100 100100
NWFPNWFP 100100 9999 101101 100100
PunjabPunjab 240240 9696 226226 100100
SindhSindh 165165 9393 125125 100100
AJKAJK 7070 98.598.5 6767 98.598.5
FANAFANA 6969 9090 6262 100100
FATAFATA 7272 7878 6969 9191
ICTICT 3030 100100 2020 100100
Total Achieved SchoolsTotal Achieved Schools 856856 9292 770770 9999
The outcome for achieved school sample at G-8 level(99%) is better than G-4(92%)
99
Students Sampled – Planned and Students Sampled – Planned and AchievedAchieved
Province/AreaProvince/AreaGrade-4 (2006)Grade-4 (2006)
Planned Achieved %Planned Achieved %Grade-8 (2007)Grade-8 (2007)
Planned Achieved %Planned Achieved %
BalochistanBalochistan 22002200 56.556.5 20002000 9494
NWFPNWFP 20002000 8282 20202020 9999
PunjabPunjab 48004800 8282 45204520 9898
SindhSindh 33003300 6363 25002500 9090
AJKAJK 14001400 6969 13401340 8989
FANAFANA 13801380 5757 12801280 8787
FATAFATA 14401440 5050 12001200 6868
ICTICT 600600 9797 400400 9797
Total Achieved SchoolsTotal Achieved Schools 1712017120 7070 1526015260 9393
Students’ participation rate at G-8 level is higher than G-4
1010
National Assessment InstrumentsNational Assessment Instruments
Basic competencies in the following core subjects are Basic competencies in the following core subjects are assessed at both grades levelsassessed at both grades levels
• LanguageLanguage• MathematicsMathematics• Science Science • Social StudiesSocial Studies
The focus of NEAS assessment is to determine the The focus of NEAS assessment is to determine the extent which students have acquired reading, writing, extent which students have acquired reading, writing, problem solving, and critical thinking skills, and not the problem solving, and critical thinking skills, and not the rote learning from textbooks rote learning from textbooks
1111
Development of National Assessment Development of National Assessment InstrumentsInstruments
The instruments used for national assessment were based on The instruments used for national assessment were based on National CurriculumNational Curriculum
These instruments were developed through a long process of These instruments were developed through a long process of consultation among national and provincial subject experts consultation among national and provincial subject experts therefore, they reflect broad national consensus on both the therefore, they reflect broad national consensus on both the structure and contentstructure and content
The instruments are based on assessment The instruments are based on assessment FrameworksFrameworks covering three ability strands in each subject. These covering three ability strands in each subject. These frameworks are reviewed refined and updated on regular basis frameworks are reviewed refined and updated on regular basis to reflect the desired goals of assessmentto reflect the desired goals of assessment
These instruments undergo rigorous expert review as well as These instruments undergo rigorous expert review as well as empirical and psychometric evaluation before acquiring the empirical and psychometric evaluation before acquiring the final shapefinal shape
Several Several spot/pilot testsspot/pilot tests are conducted in active collaboration are conducted in active collaboration with provincial/area assessment centres to ascertain the with provincial/area assessment centres to ascertain the reliability of the NEAS assessment instrumentsreliability of the NEAS assessment instruments
1212
Students’ Achievement in National Students’ Achievement in National AssessmentAssessment
Students’ achievement is assessed on the basis of their Students’ achievement is assessed on the basis of their scores. scores.
These scores provide a measure of how well the students can These scores provide a measure of how well the students can answer items (questions) based on the National Curriculum. answer items (questions) based on the National Curriculum.
Scaled scores are used to prevent item selection having an Scaled scores are used to prevent item selection having an undue influence on student scores. undue influence on student scores.
The scale used in NEAS is similar to TIMSS, SAT, TOEFL etc. The scale used in NEAS is similar to TIMSS, SAT, TOEFL etc. The range of scaled scores is between The range of scaled scores is between 0-1000.0-1000.
The achievement scales are constructed so that a student The achievement scales are constructed so that a student achieving 50% correct marks receives a achieving 50% correct marks receives a scaled scorescaled score of of 500500 and the and the Standard DeviationStandard Deviation of the scale is set at of the scale is set at 100.100.
These scores are used to report trends over time as well as the These scores are used to report trends over time as well as the average and relative performance of different groups of average and relative performance of different groups of students. students.
1313
National Scores of Grade 4 & Grade 8National Scores of Grade 4 & Grade 8
Subject Subject Year/ Year/ Grade Grade
Scaled Mean (500)Scaled Mean (500)
Score Range (0 -1000) Score Range (0 -1000)
Significance of Significance of DifferenceDifference
MathematicsMathematics2005 2005
Grade 4Grade 4
421421 Mathematics score Mathematics score significantly higher significantly higher than Language than Language score score
LanguageLanguage 369369
General Science General Science 2006 2006
Grade 4Grade 4
467467 Social Studies Social Studies score significantly score significantly higher than General higher than General Science scoreScience score
Social StudiesSocial Studies 496496
MathematicsMathematics2007 2007
Grade 8Grade 8
457457 Language score Language score significantly higher significantly higher than Mathematics than Mathematics scorescore
LanguageLanguage 512512
1414
Achievement of Competencies-Maths (G-4)Achievement of Competencies-Maths (G-4)
Difficult
Average
Easy
Measurement, Procedural Knowledge, and Practical Reasoning: Mutual conversion of units of length; Adding/subtracting expressions using problems from every day life involving units of length, mass and capacity.Geometry Procedural Knowledge : Finding perimeter of a square and rectangle.
Number sense, Practical Reasoning, and Information Handling : reading and interpreting a line graph Measurement Conceptual Understanding: relationship among units of length; Geometry, Conceptual Understanding: measuring and drawing line segments in MM and CM;
Number sense; conceptual understanding, procedural knowledge, : Concept of an improper, mixed and equivalent fractions. The highest common fraction of two or more than natural numbers.
1515
Reading for literary experience: Example: Poems and storiesReading for information: Example: essay General understanding : Global knowledge of text, providing title of text, explaining purpose or theme of text
Reading to perform a task: Reading to accomplish something , using purpose and structure of text; Example: Time TableKnowledge of Language and Vocabulary: Know and distinguish noun, verb, and adjectives; understand the use of idioms
Narrative writing: Use of creativity and power of observation, for example writing a storyPersuasive Writing: Using factual information, reasoning and comparison, developing and organizing ideas, using language effectivelyExample: Letter to Father
Difficult Average Easy
Achievement of Achievement of Competencies-Urdu Competencies-Urdu
(G-4)(G-4)
1616
Content Domains achieved in LanguageContent Domains achieved in Language
Subtest Subtest
Number of Number of ItemsItems Scaled MeanScaled Mean
ReadingReading 4646 520520
GrammarGrammar 1111 507507
WritingWriting 66 484484
Language Total Language Total 6363 512512
• Scores are significantly higher on reading as compared to
other domains of Language
• Students got the lowest scores on writing domain
1717
Location–Wise Achievement ScoresLocation–Wise Achievement Scores
LocationLocation
Scaled Scaled Mean Mean
(Grade-4)(Grade-4) (Grade-8)(Grade-8)
MathsMaths LanguageLanguage General General ScienceScience
Social Social StudiesStudies
MathsMaths LanguageLanguage
RuralRural 418418 360360 465465 491491 471471 503503
UrbanUrban 430430 399399 475475 513513 439439 525525
DifferenceDifference n. sn. s Highly Highly SignificantSignificant
n. sn. s SignificantSignificant SignificantSignificant SignificantSignificant
1818
Gender-Wise PerformanceGender-Wise Performance
GenderGender
Scaled Scaled Mean Mean
(G – 4) 2005(G – 4) 2005 (G – 4) 2006(G – 4) 2006 (G – 8) 2007(G – 8) 2007
MathsMaths LanguageLanguageGeneral General ScienceScience
Social Social StudiesStudies
MathsMaths LanguageLanguage
BoysBoys 420420 350350 459459 489489 464464 503503
GirlsGirls 424424 405405 479479 506506 446446 529529
DifferenceDifference n. sn. sHigh High
SignificantSignificant SignificantSignificant n. sn. s SignificantSignificant SignificantSignificant
1919
Earthquake effects on Students’ Earthquake effects on Students’ Achievement (Grade 4)Achievement (Grade 4)
Did you have to move to a new Did you have to move to a new place after the October 2005 place after the October 2005 earthquakeearthquake %%
Scaled ScoresScaled Scores
MathsMaths UrduUrdu ScienceScience S. StudiesS. Studies
YesYes 77NoNo 9393
356356400400
355355386386
430430471471
455455500500
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
Earthquake significantly affected students’ achievement in all the four Earthquake significantly affected students’ achievement in all the four subjects.subjects.
Students having to migrate after earthquake scored significantly lower on all Students having to migrate after earthquake scored significantly lower on all the four subjects the four subjects
Achievement in Relation to Achievement in Relation to Background FactorsBackground Factors
2121
Background Data GatheringBackground Data Gathering
During every cycle of assessment, background data are During every cycle of assessment, background data are gathered by questionnaire and analysis of fieldworkers’ gathered by questionnaire and analysis of fieldworkers’ reportsreports
Questionnaires are given to:Questionnaires are given to:• students (including a section for parents), students (including a section for parents), • head teachers and head teachers and • teachersteachers
Conditions, facilities etc. in schools are observed and Conditions, facilities etc. in schools are observed and recorded by Test Administratorsrecorded by Test Administrators
Background data are correlated with learning achievement to Background data are correlated with learning achievement to identify needs for improvement and factors associated with identify needs for improvement and factors associated with achievementachievement
2222
Background Data Gathering FrameworkBackground Data Gathering Framework
Background data are classified into five areas following Background data are classified into five areas following the Craig-Heneveld Framework:the Craig-Heneveld Framework:
• The StudentsThe Students• Supporting Inputs from Outside the SchoolSupporting Inputs from Outside the School• Teaching-Learning ProcessTeaching-Learning Process• Enabling Conditions Enabling Conditions • School ClimateSchool Climate
2323
The StudentsThe Students
Data is gathered from students about:Data is gathered from students about:
Socio-economic background (e.g. family size, Socio-economic background (e.g. family size, father’s/mother’s education and occupation)father’s/mother’s education and occupation)
Factors in the home environment related to achievement Factors in the home environment related to achievement (e.g. help in homework, space for reading, need to work (e.g. help in homework, space for reading, need to work outside)outside)
Attitudes towards educationAttitudes towards education
2424
Example of Student Background Example of Student Background Question:Question:
VariableVariable % %Scaled ScoresScaled Scores
MathsMaths UrduUrdu General General ScienceScience Social StudiesSocial Studies
YesYes 2323 NoNo 7777
383383403403
370370389389
455455473473
481481503503
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
Do you work outside the home to earn money? Do you work outside the home to earn money?
ConclusionConclusion:: about 23% of Grade 4 students reported having to work about 23% of Grade 4 students reported having to work outside the home. Their achievement is significantly lower than others’. outside the home. Their achievement is significantly lower than others’.
2525
Supporting Inputs from OutsideSupporting Inputs from Outside
Background Information is gathered from different Background Information is gathered from different sources about:sources about:
Parent and community support (e.g. frequent school-Parent and community support (e.g. frequent school-parent meetings, financial and other support, parental parent meetings, financial and other support, parental involvement in homework) involvement in homework)
Effective support from the education system (e.g. Effective support from the education system (e.g. supervision, availability of teaching materials and supervision, availability of teaching materials and supplementary materials)supplementary materials)
Adequate material support (e.g. facilities, furniture in Adequate material support (e.g. facilities, furniture in school, school budget for supplies)school, school budget for supplies)
2626
Use of Blackboard and AchievementUse of Blackboard and Achievement
Use of Use of BlackboardBlackboard%%
Scaled ScoresScaled Scores
MathsMaths UrduUrdu G. ScienceG. Science S. StudiesS. Studies
YesYes9797NoNo0303
39 939 9350350
384384341341
469469431431
497497457457
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
Teachers’ use of black board significantly increased Teachers’ use of black board significantly increased students’ achievement in all the four subjects students’ achievement in all the four subjects
Ninety-seven percent teachers reported they Ninety-seven percent teachers reported they dailydaily use use black boardsblack boards
2727
Multi-grade Teaching and Multi-grade Teaching and AchievementAchievement
Multi-grade Multi-grade Teaching %Teaching %
Students’ Scaled ScoresStudents’ Scaled ScoresMaths UrduMaths Urdu
Teachers’ Scaled ScoresTeachers’ Scaled Scores Maths UrduMaths Urdu
YesYes 21 21NoNo 78 78
442442466466
495495516516
616616644644
644644661661
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
Lower frequency of Lower frequency of Multi-grade Teaching is linked with increased Multi-grade Teaching is linked with increased performance for both students and teachersperformance for both students and teachers
Twenty- one percent teachers reported they have to teach more than on Twenty- one percent teachers reported they have to teach more than on classes in one periodclasses in one period
2828
Rewards and Punishment in SchoolRewards and Punishment in School
RewardReward % %Scaled ScoresScaled Scores
MathsMaths UrduUrdu General General ScienceScience S. StudiesS. Studies
Yes 85Yes 85NoNo 14 14
403403
372372
390390
354354
474474
440440
504504
464464
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
PunishmentPunishment % %
FrequentlyFrequently .03.03Occasionally 38Occasionally 38NeverNever 58 58
351351388388405405
345345376376391391
444444458458476476
452452490490505505
DifferenceDifference SignificantSignificant SignificantSignificant SignificantSignificant SignificantSignificant
Student who Never got punished by their teachers performed significantly better Fifty-eight Student who Never got punished by their teachers performed significantly better Fifty-eight percent students reported they never got punished. percent students reported they never got punished.
Only .03 % students reported frequent punishmentOnly .03 % students reported frequent punishment Rewarding students’ performance increased achievement all the subjectsRewarding students’ performance increased achievement all the subjects
2929
PTA/ SMC in School: PTA/ SMC in School: Head Teachers’ ReportHead Teachers’ Report
0
10
2030
40
50
60
7080
90
100
PTA/SMCSchool
Need ForPTA/SMC in
Schools
Funding ofPTA/SMCfrom Govt.
CopperativeRole of
PTA/SMC
Pe
rce
nt
Yes
No
• Around 80% head teachers reported presence of PTA/ SMCs’ in School
•Presence, need, funding, or role of PTA/SMC did not increase students performance in any of the NEAS tested school
•Only 39% head teachers reported getting funds from Govt.
•Ninety-seven percent head teachers reported need for SMCs in Schools
3030
Strengthening Links between NEASStrengthening Links between NEASand the Educational Planning Processand the Educational Planning Process
NEAS can Support Educational Planning by:NEAS can Support Educational Planning by:
• providing data about the actual conditions in schools; andproviding data about the actual conditions in schools; and
• indicating the possible impact of policy actions on student indicating the possible impact of policy actions on student achievementachievement..
Educational Planners can make NEAS more Effective by:Educational Planners can make NEAS more Effective by:
• proposing topics to be added to background data gathering; proposing topics to be added to background data gathering; andand
• suggesting relationships to be explored.suggesting relationships to be explored.
3131
The Presentation SummaryThe Presentation Summary
NEAS has assessed the achievement of Grade 4 & 8 NEAS has assessed the achievement of Grade 4 & 8 students in four subjects: students in four subjects: Urdu, Mathematics, Science and Social studies.Urdu, Mathematics, Science and Social studies.
The assessment reveals large differences in average The assessment reveals large differences in average performance among provinces and between students performance among provinces and between students from differing backgrounds.from differing backgrounds.
These differences in average performance can be traced These differences in average performance can be traced back to differences in student and family characteristics, back to differences in student and family characteristics, school processes and inputs. school processes and inputs.
3232
Institutionalization of NEAS and Institutionalization of NEAS and Future Vision of National Assessment Future Vision of National Assessment in Pakistan in Pakistan
Institutionalization of NEAS:Institutionalization of NEAS:
Government of Pakistan declared NEAS as a permanent institution of the Government of Pakistan declared NEAS as a permanent institution of the Ministry of Education, from July 2007Ministry of Education, from July 2007
Future Vision of NEAS:Future Vision of NEAS:
Recommendations by National NEAS Select Committee:Recommendations by National NEAS Select Committee:
• NEAS future planning till 2020 comprises National Assessment at Grade 4, NEAS future planning till 2020 comprises National Assessment at Grade 4, 5, 8, 10 and 12 5, 8, 10 and 12
• Private schools are included in NEAS sample on Pilot basis in 2008 Private schools are included in NEAS sample on Pilot basis in 2008 National AssessmentNational Assessment
• Listening and speaking skills of student assessment would be part of Listening and speaking skills of student assessment would be part of NEAS future activitiesNEAS future activities
• Pakistan can explore the possibility to participate in the next TIMSS in 2012Pakistan can explore the possibility to participate in the next TIMSS in 2012
3333
Thank You…Thank You…