Introduction

1. The design of Assessment Instruments must be based on the following assessment principles: validity, reliability, impact, practicability and fairness

2. Validity is the extent to which test is assessing what it intended to assess.

3. Validity is the mainstay in the modern conception of assessment and evidence of validity should be visible in all processes, setting of papers, marking process and grading of learners performance and resulting

4. Reliability is the ability of the test to yield consistent results and consistent demonstration of a particular skill by learners confirms the validity of the skill possess by learners.

Focus for 2013 and Beyond

a) Finalise Policy on the Quality Assurance of SBA.

b) Systems Audit focussing on support of SBA implementation.

c) Moderation of assessment tasks

d) Moderation of learner evidence in selected districts, across PEDs.

e) Development of Assessment exemplars.

f) Moderation of common tests/exams set for the Preparatory

examinations.

g) Common Assessment Task for Life orientation.

Quality Assurance Approach

Systems Audita) The system audit was conducted in which the team of moderators audited

the provincial and district system to ascertain its potential to support school based assessment

b) DBE moderators met the curriculum planners and subject advisors of the sampled districts.

c) The audit focussed on all policy documents, reports, guidelines, curriculum materials, instruments of moderation and quality assurance of assessment tasks

d) The audit was followed by the moderation of learner evidence and assessment tasks

e) 20 schools were sampled from two districts

Quality Assurance Approach Methodology

a) A stratified purposive sampling technique was followed and schools performing within the range of underperforming (0% – 60%) and good performing (80% - 100%) were sampled

b) Two instruments were used:Systems Audit instrument: - Location of the SBA, - the relationship between the exams and curriculum,- training programme for subject advisors and teachers,- feedback, - common tests

Moderation Instrument - Content coverage, - cognitive levels,- language and technical aspects

Quality Assurance Approach

c) The following subjects were sampled namely: Accounting, Mathematics, Physical Sciences, Life Sciences, Geography, English First Additional Languages, History and Life Orientation. d) 68 Moderators were appointed, rigorously trained and deployed across provinces.e) Judgement is based on the set criteria as exemplified by the system audit document and moderation instrumentf) The judgement is generally based on a group (subject experts) observation, discussion and judgement based on consensusg) Findings can therefore be considered to be valid, reliable, dependable and credible

Moderation Instrument (criteria)a) Questions are clear, concise, precise, intelligible, and worked using language

appropriate to the range of candidates for whom the test or Task is intended.

b) Repetition of questions from previous examinations is avoided.

c) Appropriate curriculum content which is adequately covered within the test/task.

d) Appropriate distribution of learning outcomes and assessment standards

e) Correlation between item difficulty, time allocations and mark allocations

f) Compliance with content and construct validity

g) Appropriate distribution of cognitive levels

h) Language level is in keeping with the vocabulary and comprehension levels of Grade 12

learners

Moderation Instrument (criteria)i) Questions or tasks are free from subject error from an academic point of view, e.g.

historic or scientific facts.

j) Bearing in mind the different national, ethnic and religious backgrounds of candidates,

questions and/or source material will not cause offence due to inappropriate subject

matter and/or Language

k) Instructions on the assessment tasks and tests were clear and precise.

Quality Assurance Findings Support systems provided by the district and provinces1. In some provinces the evidence points to the fact that there is little

interaction between subject advisors and teachers2. The quality assurance of SBA moderation in some provinces has revealed

that there is a lack of a functional, collaborative partnership between the directorates of Curriculum and Examinations and Assessment.

3. Some provinces are more advantaged than others regarding the number of subject advisors

4. In some provinces subject advisors are overloaded with both GET and FET workload and are not able to provide effective support.

5. There is generally very limited opportunities for the training of teachers across provinces

6. There is limited training of subject advisors in provinces especially in relation to assessment issues

Quality Assurance FindingsCompliance 1. There is general compliance to the requirements of SAG and subject

policy guideline documents.2. The learner evidence and teacher files are mostly neatly organised

Moderation – pre-moderation of learner evidence a) Provincial moderation where it did take place was meaningful and

constructive.b) There is evidence of moderation and pre-moderation across provincesc) Prevalence of inaccurate and inconsistent marking of learner tasks.d) Poor marking process resulting in the inflation of marks -inaccurate

marking guideline, and misapplication of rubricse) Poor moderation of learner evidence which manifests into the so-called

shadow markingf) Lack of quality pre-moderation or quality pre-moderation of tasks

Quality Assurance FindingsQuality of Assessment Tasks

1. In terms of tests and exams the format, layout and presentation of the question papers are generally neat in few instances

2. Evidence of post moderation in some form is present in most provinces. In most cases this reflected school, district and provincial levels of moderation.

3. The content coverage in most cases is within the scope of the grade 12 syllabus.

4. The quality of assessment tasks and tests generated at school level are of a poor standard.

Quality Assurance FindingsQuality of Assessment Tasks

5. The technical quality of papers is very poor: cut and paste from previous papers, no sequencing of the number of questions, no mark allocations, poor formatting noted

6. Over dependence on past year papers was becoming an unhealthy trend. This dependence stifles creatively and originality that is expected from teachers.

7. Investigation tasks in Maths and Physical Sciences was poorly done. Some provinces deviated from the required norm.

Quality Assurance FindingsFeedbacka) There were no examiners’ reports that accompanied assessment tasksb) The history of the paper was only provided by examiners that set the

common tests and for tests and tasks set at school level only few schools managed to submit the history of the paper.

c) The quality of the comments on the papers moderated were not appropriate and led to the production of poor papers and also lulled teachers into a false sense of security

d) Inaccurate feedback to learners, due to inconsistent marking, flawed marking tools and shadow marking

e) Reports emanating from centralised moderation are not accurate and serve as barriers in the development of teachers

Proposed Intervention - 2014Focus

(a) Urgent need for the national training of subject advisors on SBA .

(b) Development of Training Programme to be implemented by Subject Advisors.

(c) Development of Assessment Exemplars (Alternative forms of assessment)

Proposed Intervention - 2014Suggested Approach

1. Three subject advisors per province, per subject to be nominated as SBA champions in

the province.

2. Provincial SBA Champions to be exposed to a two week intense training programme

nationally, coordinated by the DBE.

3. Focus on seven key subjects in 2013 (Mathematics, Physical Science, Life Sciences,

Accounting, History, Geography and Economics) and incrementally phased in to include

other subjects and GET.

4. Training Programme will focus on:

(a) Design of high quality SBA assessment tasks.

(b) Develop a Training Manual for use in the training of subject advisors.

(c) Development of assessment exemplars

Assessment for Learning 1. Assessment for learning needs to be driven by the curriculum and teacher

development sections and there is a need to keep teachers within the classroom and support them within the school rather outside the school.

2. Authentic assessment for learning considers assessment as a service of learning, active engagement of learners and learning as a process and not a product.

3. These central tenets of assessment for learning cannot be achieved through centralised moderation of assessment tasks and learner evidence

4. Therefore curriculum section needs to be strengthened and need to ensure that subject advisors are capacitated with the requisite knowledge of their subject to assist teachers with teaching methodologies, assessment strategies and the teaching of higher order skills such as critical thinking and problem solving.

5. Classroom assessment activities that are driving teaching and learning needs to be the main focus of curriculum and teacher development

6.

Conclusion

The focus on FET teacher training is highlighted in Action Plan 2014 (Goal 16: Improve the professionalism, teaching skills, subject knowledge and computer literacy of teachers throughout their entire careers), and is still largely focused on ensuring compliance with Curriculum Policy and Assessment requirements.

The systems audit indicates there is rich opportunity for the teacher training to build the skills of teachers in the design and development of School Based Assessment Tasks, tests and the overall management of assessment