Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | millicent-hart |
View: | 213 times |
Download: | 1 times |
THE STATE OF READINESS national senior certificate (NSC) examination and annual national assessment (ANA)
31 AUGUST 2010
Portfolio Committee on Basic Education
Presented by: Nkosinathi Sishi, PhDChief Director: Educational Measurement, Assessment and Public ExaminationsRSA
PURPOSE OF THE SOR
The state of readiness visit in 2010 focused on the following key areas:
1. The readiness of the PEDs to administer the 2010 NSC examination.
2. The status of the monitoring and moderation systems relating to School Based Assessment (SBA) at the Grade 10, 11 and 12 levels.
3. The State of Readiness of the PEDs to administer the ANA in Grade 3, 6 & 9.
4. Readiness of learners to take the assessment
International comparability of QUALITY, STANDARDS, Systems, and practices
Strategic goal:
1.Credible educational measurement, assessment run with integrity2.Quality practices
Strategic objectives1.National assessment and measurement of high international esteem2.Valid and fair educational judgments 3.National historical learner records4.Integrity of Site-based assessment5.Recognition of prior learning6.Credible qualifications, certification and accreditation7.Educational excellence 8.NQF Principles
PrinciplesFair Assessment
Developmental Assessment
1. Assessment should be developmental rather than judgemental,2. Credibility of site-based assessment remains a big challenge.
Fit for Purpose Assessment
1. Valid Assessment, Reliable Assessment , Integration, Relevance 2. Credibility and Standards3. Flexibility and Recognition of Prior Learning,4. Legitimacy, Access and Redress5. Articulation and Portability.
3
Methodology
1. Hypothesis2. Self-evaluation3. Meetings and interviews 4. Site visits: Provincial offices, printing and distribution
centres, districts5. Documents analysis6. Sampling7. Plenary 8. Shortcomings identified in the 2008 and 2009
monitoring process
Specific Aim
1. Establishing system preparation for the 2010 NSC examination
2. Combat readiness for the Annual National Assessment
3. Credibility and integrity of Site-based Assessment4. Preparation of learners for the 2010 national
assessment and public examinations
6
MAKING VALID EDUCATION ASESSMENT JUDGEMENTS
Operational ObjectivesOperational Objectives1. High quality question papers are set and moderated2. Candidates are accurately registered & resulted on the
examination computer system 3. Analytical reports are generated4. Question papers are accurately printed, securely packed and
distributed to examination venues5. All candidates including special needs catered for6. Answer scripts accurately marked and moderated7. All examination processes are conducted with integrity8. Prioritisation of security and accountability in the distribution of
scripts and the collection of answer books
Operational Objectives
1. SBA is completed in accordance with policy prescripts and moderated.
2. All examination and SBA marks are accurately captured.3. All examination irregularities are appropriately managed.4. All candidates are accurately and timeously resulted.5. Effective feedback provided to schools.6. All eligible candidates receive certificates.
National Senior Certificate Examinations 2010
• School-based assessment is ongoing throughout the year.• Practical examinations in Visual Art and Design began on 16 August
2010.• Practical examinations in Computer Applications Technology and
Information Technology to be conducted from 13 to 15 October 2010• The first written examinations on 25 October 2010:– Information Technology Theory, and – Computer Application Technology Theory paper.
• The last examinations to be written on 3 December 2010:– Religion Studies Paper 2– Civil Technology, and – Hospitality Studies
Province Full Time Enrolments 2009 Full Time Enrolments 2010
Eastern Cape70 622 70 852
Free State30 462 29 094
Gauteng101 268 94 546
KwaZulu Natal139 965 131 379
Limpopo93 287 93 806
Mpumalanga55 735 55 885
North West31 395 29 585
Northern Cape10 765 10 399
Western Cape46 861 46 927
Total580 360 562 473
Province Part Time Enrolments 2009
Part Time Enrolments 2010
Eastern Cape3 516 11 141
Free State919 2 022
Gauteng8 278 22 852
KwaZulu Natal12 577 19 990
Limpopo7 834 9 529
Mpumalanga2 989 4 684
North West1 029 2 672
Northern Cape494 1 281
Western Cape1 715 6 047
Total39 351 80 218
2010: No of candidates
Fulltime 562 473
Part-time 80 218
Total 642 691
2009: No of candidates
Fulltime 580 360
Part-time 39 351
Total 619 711
Integrated Examination Computer System
1. The examination computer system is the backbone of the administration of the NSC - the system is still being enhanced.
2. All candidates have been registered on the Integrated Examination Computer System (IECS).
3. Registration data has been audited by the DBE. 4. A third schedule of Entries has been sent to all schools to ensure
accurate registration of all candidates.5. At the end of August 2010, mark sheets will be printed.6. Letters of admission to the examination will be sent to schools in
September 2010.7. In 2010 a full parallel run on the old system will be conducted to
improve confidence in the processing of results8. Both systems are maintained and supported by the State
Information Technology Agency (SITA)
13
Question Papers1. Security surrounding all processes relating to preparation of
question papers is satisfactory.2. All question papers are set by the Department of Basic
Education to ensure a national standard.3. 260 question papers have been set by panels of examiners and
moderated subject specialists.4. The question papers have been moderated by Umalusi to ensure
appropriate standard5. The first batches of question papers have been handed to
PEDs for printing and packing6. Final handover of question papers is 10 September 2010.7. Question papers are being brailled for blind candidates and
adapted for deaf candidates.
Printing, Packing and Distribution
1. All staff working in the printing and packing sections have security clearance and have signed confidentiality agreements.
2. PEDs have received the first two sets of question papers printing.3. Question papers will be distributed to examination centres on the
morning of the examination, except where extreme distances are involved.
15
Preparation for the examinationPreparation for the examination
1. Invigilators who supervise the writing of exams will be trained across all PEDs in September.
2. The principal is usually the Chief Invigilator.3. Detailed regulations govern the conduct of exams and the
managing of irregularities – DBE has issued a Guideline document to assist this process
4. Markers and chief markers have been appointed/ will be appointed in September 2010.
5. Marking centres have been finalized and markers will be trained before marking commences.
16
Answer scripts
1. Each answer script is the product of a candidate’s 12 years at school and his/her performance is encapsulated in the script.
2. In 2010, emphasis is being placed on the return of answer scripts, and management of scripts during marking process:– All scripts to be returned to the district office the same day– Scripts to enjoy the same attention and security as the question
paper. – All scripts correctly labelled with exam number and subject.
• Scanning of scripts to be introduced by all PEDs as soon as possible. – All scripts to be checked at each stage of the process.– Implementation of sanctions where Chief Invigilators do not
manage the return of scripts correctly.
17
Examination Centres
1. All examination centres are audited to ensure adequate resourcing.
2. All examination centres are monitored, with a special focus on examination centres with a history of irregularities.
3. Centres that do not satisfy national criteria are de-registered and alternative centres are arranged. – This includes private centres.
18
MarkingMarking
1. The quality of marking will be the focus in 2010.2. Marking centres have been established across all PEDs based on
the prescripts contained in the Policy.3. Markers have been appointed based on the PAM criteria.4. Training of markers is ongoing and specialised training sessions
will be organised prior to the marking sessions.5. National Marking Guideline discussions will be hosted in Pretoria to
ensure standardisation of marking across all PEDs 6. Internal moderators to be appointed for each subject at each
marking centre.7. DBE will appoint external moderators to evaluate the marking at
each centre.
19
School Based Assessment (SBA)School Based Assessment (SBA)
1. School based assessment constitutes 25% of the final promotion mark.2. SBA is moderated at school, district and provincial level.3. PEDs monitor the collection of marks from schools quarterly.4. DBE issued a Guideline document to assist PEDs with the collection,
control and capture of SBA marks to address shortcomings experienced in previous examinations, such as outstanding marks.
5. SBA marks are currently being moderated by the PEDs and capture will be completed by 15 November.
20
Processing and Release 2010
1. Marking will be completed by 16 December across all PEDs.2. Mark Capture will be completed by 18 December .3. A technical team will prepare proposals in respect of adjustments
for the pre-standardization meeting – 21 December. 4. Pre-standardization meetings will be held on 22 & 23 December. 5. Umalusi standardisation meeting to be held on 24 December. 6. Results will be thoroughly checked by DBE, PEDs and Umalusi on
28 – 30 December.7. Umalusi approval meeting to be held on 31 December 2010.8. Results will be released on 6 January 2011.
21
Analysis and FeedbackAnalysis and Feedback
1. Detailed analysis of results per school, per district, per province, per subject will be made available.
2. Reports from chief markers/internal moderators will be consolidated for distribution to schools.
3. Outcomes of the Umalusi standardisation meetings will be made available to curriculum specialists.
4. Workshops will be conducted with teachers and subject advisors.
5. Under-performing schools will be brought to account and improvement plans will be designed.
6. Curriculum specialists to develop subject improvement plans.
Overview of State of Readiness1. The monitoring exercise was severely impacted on by the public service
strike - not possible to visit district offices and schools in North West & Northern Cape.
2. It was established that schools are filtering candidates in respect of:1. progression to Grade 12 and2. the subjects that they may offer.
3. School Assessment Irregularities Committees are not yet functional. 4. The collection and reconciliation of answer books from examination centres
the exams needs attention. 5. At several district and circuit offices storage facilities were inadequate.6. In the electronic age, security of examinations now also means securing
electronic copies of the question papers – this needs attention.7. Varying SBA moderation practices within and across the provinces do not
give confidence that all candidates treated in a fair & equitable manner. 8. DBE is engaged in the process of developing guidelines for moderation
which should resolve the current plethora of practices.
23
Progress in Mpumalanga
1. DBE is managing the NSC, SC and ABET examinations in the province and working collaboratively with the regional staff.
2. DBE staff are deployed to the province.3. Certain functions have been decentralised to regions while DBE will
manage centralised functions. 4. Focus is on establishing improved systems and processes and
building staff capacity.5. Supplementary, Senior Certificate and June ABET Level 4
examinations have been successfully administered.6. All question papers to be printed and stored in Pretoria, and will
delivered on a weekly basis to the regional offices or nodal areas and daily to the examination centres.
7. Scripts will be collected from schools daily
24
the black box
1. McKinsey report: quality of an education system cannot go beyond the effectiveness of class room interaction
2. Low impact of teacher human capital means our measures are inadequate
3. School organization
NSC CHALLENGES NSC CHALLENGES
1. Five week world cup recess may have impacted on the teaching and learning momentum.
2. The teacher strike has resulted in a school shut down.3. NSC policy is still not fully understood by learners, parents and
educators.4. Security of question papers.5. Introduction of a new examination computer system.6. Moderation of the quality and standard of SBA needs attention.7. Conduct of examinations at private centres is not always
satisfactory.
2010 ANA STATE OF READINESS2010 ANA STATE OF READINESS
Awareness:1. On the whole provinces ,districts and schools visited are aware of the
continued implementation of ANA in 2010.2. Plans are afoot to strengthen advocacy around awareness especially in
districts and schools.
Communication strategy:1. Advocacy around ANA needs to be strengthened and monitored
1. its purpose2. administration 3. capturing, analysis, reporting, and 4. post assessment improvement plans and support
2. Communication and flow of information needs to be improved to ensure a consistent message communicated at all levels.
2010 ANA STATE OF READINESS2010 ANA STATE OF READINESS
Statistics:
Learner statistics must be verified before sending to DBE– provinces will be requested to deal with shortages.
Storage of test papers in districts:
Inadequate space to store the large quantities of Gr 1 – 6 test papers in districts.
PEDs will need to rent warehousing space in some districts.
RecommendationsFacilitate and maintain rigorous moderation systems for SBA.
1. Develop a policy for the moderation of SBA2. Monitor provincial moderation systems.3. Moderate sample assessment tasks and learner evidence
in the case of Life Orientation – July and October 2010.4. Moderate SBA tasks and learner evidence.
Recommendations
Provide diagnostic feedback to schools.
1. Develop comprehensive analysis (qualitative and quantitative) of data.
2. Detailed reports made available to provinces, districts and schools.
3. Conduct post test analysis of ANA and public examinations.
Overview1. filtering candidates in respect of progression to Grade 12. It
was also evident in visits to schools that restrictions were placed on certain learners in respect of the subjects that they may offer.
2. the establishment and functionality of the School Assessment Irregularities Committee is neglected
Overview
• PEDs need to focus on the collection and reconciliation of answer books from examination centres to marking centres and storage as adequate systems are not in place in all provinces to ensure that all scripts are accounted for.
• Generally storage facilities at Provincial offices are excellent but at several district/regional offices and circuit offices storage facilities were well below the expected norms and standards for a secure storage facility and provinces must improve storage facilities as a matter of urgency.
Overview
• Security is always a concern in examinations but in the electronic age, security of examinations is fast moving away from merely securing physical copies of the question papers. There is now a need to secure the electronic copies of the papers.
• Moderation practices vary across the provinces, and even within provinces. The Department of Basic Education is engaged in the process of developing guidelines for moderation which should ensure that each candidate is being treated in a fair manner.
Overview
• One province has been requested to visit their printing and packing procedures to improve security in this area.
• The Annual National Assessments will be printed nationally this year. Provinces indicated that this would alleviate many problems. The key area of concern at the moment is that schools provide accurate statistics in respect of learner numbers and the Language of Learning and Teaching.
CONCLUSION
• Based on the information that was made available to the monitoring team, it can be concluded that the Provincial Departments of Education have made significant strides towards readiness for the NSC examinations
• However in respect to the annual national assessment, learner preparation has been disrupted by the strikes and it is recommended that these are shifted to February 2011. There are educational reasons to support that such tests are in future conducted earlier than later in the year
ConclusionConclusion1. Learner preparation: Registration, irregularities, subject changes,
item analysis2. Stratification of the learner attainment strategy3. System administration4. Validation of standards5. Credibility of Site-Based Assessment6. Logistics regarding the ANA7. Analysis of ANA results8. Item writing, banking and discrimination9. Promotion requirements10. Release of results11. Top candidates