International comparisons in senior secondary assessments

Post on 01-Nov-2014

766 views 0 download

Tags:

description

Presentation at 37th IAEA Annual conference, Manila

transcript

International comparisons in senior secondary assessments

Dennis OppossPresented at the 37th IAEA Annual Conference, Manila27 October 2011

What is Ofqual?

We maintain standards, improve confidence and distribute information about qualifications, examinations and assessments used in England.

We recognise organisations that deliver qualifications and monitor their activities.

Ofqual’s reliability programme

Two-year research study

International comparisons work

Six-year research programme

Investigate the demand of assessments taken internationally in comparison to those taken in England

International comparisons work

The first of these studies focuses on assessments available to senior secondary learners intending to progress to university in England

The two types of studies

PISA, TIMSS, PIRLS

A special test designed to be a benchmark against itself

Test written through international collaboration to assess a common understanding of ‘quality’

Sample of all students

What Ofqual is looking at

What students do as part of their normal school assessment

Assessments written in the context of each system, reflecting their values, curriculum and purpose

Assessments for students looking to university

Existing studies

‘On their own, cross-sectional international comparisons such as PISA cannot identify cause-and-effect relationships between certain factors and educational outcomes, especially in relation to the classroom and the processes of teaching and learning that take place there’.

OECD 2011

Senior secondary assessment

Chemistry

English (national language or main language of tuition)

History

Mathematics

Other systems

International Baccalaureate

The ACT, USA

Cambridge Pre-U

Cambridge International

A levels

Senior secondary assessment

Some assessments have the specific purpose of selecting students for higher education (for example a matriculation examination)

Others are assessments of educational achievement that in addition are used to identify students for higher education (of which A levels are one)

Senior secondary assessment

The demand of each assessment was judged in relation to its appropriateness in preparing learners for entry to honours level degree study in England

Assessment model level analysis

Reviewers consider

what learners needed to know and be able to do to succeed in the assessment

the nature of the subject matter to be covered

accessibility to the range of learners taking the assessment.

Assessment instrument level analysis

CRAS analysis is used as a basis for organising the thoughts of reviewers with regard to specific features at question, task or question paper level.

CRAS analysisCategory What is it? Scoring

1 2–3 4

C Complexity

The complexity of each component operation or idea and the links between them

Simple operations (i.e. ideas/steps)

No comprehension, except that required for natural language

No links between operations

Synthesis or evaluation of operations

Requires technical comprehension

Makes links between operations

R Resources

The use of data and information All and only the data/ information needed is given

Learner must generate all the necessary data/information

A Abstractness

The extent to which the learner deals with ideas rather than concrete objects or phenomena

Deals with concrete objects

Avoids need for technical terms

Highly abstract

Requires use of technical terms

S Strategy

The extent to which the learner devises (or selects) and maintains a strategy for tackling and answering the question

Strategy for answer is given

No need to monitor strategy

No selection of information required

No organisation required

Learner needs to devise their own strategy

Learner must monitor the application of their strategy

Learner must select content from a large, complex pool of information

Learner must organise how to communicate response

CRAS analysis

Category: Resources

What is it?

The use of data and information.

Scoring:1 All and only the data/ information needed is given.

4 Learner must generate all the necessary data/information.

CRAS analysis

Category: Abstractness

What is it?

The extent to which the learner deals with ideas rather than concrete objects or phenomena.

Scoring:1 Deals with concrete objects.

Avoids need for technical terms.

4 Highly abstract.Requires use of technical terms.

Emerging findings

Independent research and extended essays

Internally set assessment

Multiple-choice questions

Mark schemes

Predictability

Emerging findings – Mathematics

Pure mathematics or applications

Levels

New technology

Mechanics

Emerging findings – Chemistry

Balance of coverage

Challenging topics

Mathematical content

Emerging findings – English What is English?

What is a text?

Types of assessments

Emerging findings – History What is history?

citizenship? study skills and concepts? rote learning facts and figures?

Emerging findings – Summary

Layers of breadth and depth

Assessment variety

Producing the best A levels

Future studies – 16+,11+

Criteria for selection of systems

Subjects: mathematics and English; others?

Getting in touch

www.ofqual.gov.uk

dennis.opposs@ofqual.gov.uk