Monitoring SQA Qualifications over time

Monitoring SQA Qualifications over time

Dr Rob van Krieken

Another bridge in Arnhem

ContentContent

Why is SQA monitoring over time?

What are the standards SQA is monitoring?

SQA’s monitoring programme

Implementation

Reports and actions

Why is SQA monitoring over time?Why is SQA monitoring over time?

to avoid

‘creep’ within a stable system

shock when the system changes

Example of “creep”Example of “creep”

Year 1 Year 2

Cut-off % Pass Cut-off % Pass

49 86% 49 87%

50 80 % 50 81 %

51 72% 51 73%

Example of “creep”Example of “creep”

Year 2 Year 3

Cut-off % Pass Cut-off % Pass

49 87% 49 88%

50 81 % 50 82 %

51 73% 51 74%

Example of “creep”Example of “creep”

Year 3 Year 4

Cut-off % Pass Cut-off % Pass

49 88% 49 89%

50 82 % 50 83%

51 74% 51 75%

Example of “creep”Example of “creep”

Year 1 Year 4

Cut-off % Pass Cut-off % Pass

49 86% 49 89%

50 80 % 50 83%

51 72% 51 75%

System changesSystem changes

Vocational qualifications are reviewed systematically at the end of their duration

O-grade to Standard Grade (1984 – 1990s)

SCE Higher – Revised Higher (1992 - )

(Revised) Higher - Higher (Still) (2000)

CSYS – Advanced Higher (2001)

New National Qualifications (2014-2016)

What are the standards SQA is monitoring?What are the standards SQA is monitoring?

‘Minimum level of demand’

– Laid down in specifications (“standards”)

– Exemplified in assessments/marking/boundary

Within the educational context of

– SCQF levels, similar qualifications

– Quality and quantity of teaching and learning

– Cohort

Monitoring programmesMonitoring programmes

Different programmes for National Courses and Vocational Qualifications (HN and SVQ)

Sampling by subject area and level

Cycle depends on qualification / level

Sample Higher National DiplomasSample Higher National Diplomas2014 • 2015 2016 2017 2018 2019

Beauty Therapy Administration and

Information Technology

Early Education and Childcare: Childhood Practice

Social Care Social Service

Hospitality Management

Computing: Software Development

Travel and Tourism

Nautical Science

Early Education and Childcare

Beauty Therapy Administration

and Information Technology

Social Care 

Hospitality Management

Computing: Software Development

Social Service

Travel and Tourism

Nautical Science

C&ACollect and archive

C&CCollect and Compare

Sample Higher National QualificationsSample Higher National Qualifications

2013 2014 2015 2016 2017 2018 2019 2020

English C&A C&C C&C

Mathematics C&A C&C C&C

Art and Design C&A C&C

Business Management C&C

Chemistry C&A C&C

Computing C&C

Engineering Science         C&C     C&C

French C&A C&C

Geography C&A C&C

RMPS C&A       C&C      

Implementation: Panels, materials, and tasksImplementation: Panels, materials, and tasks

Panel:

– 3 subject experts

Materials:

– Specification

– Assessments, Marking Instructions

– Evidence

– Judgements

Task:

– Questionnaire

QuestionnaireQuestionnaire

1 Educational Context:

Are there any general changes that may have influenced the general ability of candidates in one of the years, in particular the way they meet the demands made by the syllabus and exam?

QuestionnaireQuestionnaire

2 Unit Specification or Standards and Guidance

2.1 Are there differences in the Unit Specifications and Guidance that affect the level of demand?

2.2 Were the Unit Specification Standards in 2011 on the whole more / no more / less demanding than before?

QuestionnaireQuestionnaire

3 Assessment

3.1 Are there differences in Assessment practice that affect the level of demand? Please tick the relevant boxes and add if necessary.

3.2 Was the Assessment in 2011 on the whole

more / no more / less demanding than before?

QuestionnaireQuestionnaire

4 Quality and Judgement of Evidence

4.1 Are there differences in the quality of the evidence and in the way this has been

judged? Please tick the relevant boxes and add if necessary.

QuestionnaireQuestionnaire

5 Overall judgement

A reasoned overall judgement of the level of demand of the qualification as a whole, taking into account standards, assessments, and evidence.

Reports and actionsReports and actions

Overall report

Summary of conclusions

Actions SQA undertakes

Issues and answers 1Issues and answers 1

Too much change

Overall judgement

Quantity and type of evidence

Panel membership

No looking Backward