AQA’s Centre for Education Research and Policy

AQA is a leading provider of education assessment and support. We are the largest awarding body of A-levels and GCSEs in the UK. We operate as a social enterprise and invest significant resources in research that helps ensure learners have the opportunity to fulfil their potential.

Education policy Research evidence provides the foundation for everything we do at AQA. It informs our contribution to education policy debates and how we run our organisation. For example, in the public sphere, we have consistently advocated the introduction of international benchmarking to drive up standards, while internally we have drawn on research to facilitate the introduction of technology into assessment.

Recent examples of our work

The value of AS-levels: Increasing attainment and curriculum breadth or ‘dumbing down’ the gold standard? Debra Malpass, 2011

Has the AS qualification achieved its stated aims following its introduction as part of the Curriculum 2000 reforms? This paper reviews the evidence of whether AS-levels have achieved their primary aims of broadening the curriculum and increasing attainment by providing a more gradual transition between GCSEs and A-levels. It discusses the future of AS-levels and considers whether the qualification should be retained in its current form, reformed to address its critics or replaced entirely.

Experiences of teaching and learning in A-levels – is stretch and challenge needed? Jo-Anne Baird, Anthony Daly, Suzanne Chamberlain and Michelle Meadows, 2011

In interviews and focus groups, students and teachers agreed that the Curriculum 2000 A-levels were demanding, but not in an educationally beneficial way. They reported highly strategic approaches to teaching and learning, with a strong emphasis upon studying past question papers and mark schemes. Students and teachers welcomed the stretch and challenge policy and there were some indications that its introduction could have the intended effects on learning.

The impact of re-sitting on the fairness of grading. Chris Wheadon, 2010

One consideration in allowing re-sitting is that exams are not perfectly reliable instruments; on any occasion a candidate may perform slightly better or worse, depending on a number of chance factors related to their preparation and the exam content. From this perspective, it may seem fair to allow candidates to re-sit to compensate for this inherent unreliability. This work informs policy by estimating the impact of re-sitting on grading reliability.

Aptitude tests versus school exams as selection tools for Higher Education and the case for assessing educational achievement in context. Neil Stringer, 2008

This paper discusses the pros and cons of aptitude tests as tools for university selection. It focuses on ‘school-proofness’ claims, concluding that it is invalid to assess suitability for university using measures purportedly immune to study, and unfair to allocate opportunities according to qualities acquired by chance. Claims that aptitude tests can be ‘school-proof’ and have the power to predict academic achievement, relative to achievement tests, are found wanting.

For more information about any of the projects in this area, please contact:

Alison WoodHead of Education Policye-mail: awood@aqa.org.uk Tel: +44 (0)1483 556 415

Research themes

AQA’s Centre for Education Research and Policy

The Centre for Education Research and Policy is at the heart of AQA’s commitment to evidence-based policy and practice.

Our Centre is a multi-disciplinary research facility. Its credentials have been established over the last forty years. While we maintain the highest standards of academic rigour, our work is grounded in the practical realities of education assessment and teacher and learner support. This ensures that our research leads to recommendations that are relevant and can be implemented.

Our research provides robust evidence on which to base policy because:

we employ a range of quantitative and qualitative methodologies, rather than a single approach, which ensures that our research findings are credible and valid

the conclusions drawn in any study stay within the limitations of the data

the strengths and weaknesses of any study are made explicit.

Our work is rigorously reviewed by a prestigious committee of national and international university researchers, chaired by Professor Jannette Elwood of Queen’s University, Belfast. We disseminate our work in academic books and journals, at national and international conferences, in executive summaries for policy makers and through newsletters and magazine articles for practitioners.

For more information about our work, please contact:

Dr. Michelle MeadowsHead of the Centre for Education Research and Policye-mail: mmeadows@aqa.org.uk Tel: +44 (0)161 958 3780

“ Research evidence provides the foundation for everything we do at AQA.”

Research to support teaching and learning AQA is driven by our commitment to helping teachers develop their pupils to their full potential. Our support for teachers and learners is underpinned by a robust programme of research. This expertise in learning and pedagogy informs the development of our products and services, as well as the evaluation of the outcomes they deliver.

Recent examples of our work

“I’ll tell you what really stresses me out …” A-level students’ experiences of preparing for and taking exams. Suzanne Chamberlain, 2010

Students talked about pre-exam anxiety (eg revision, mock exams) and exam-day anxiety (eg time available to complete the exam, uncertainty about questions). Most believed that anxiety aided their performance, sharpening their minds and increasing their focus. Few reported experiencing anxiety that impaired their functioning. Nonetheless, pre-exam anxiety could have a negative impact on students’ ability to prepare and interventions are being considered.

Students’ approaches to learning and their performance in the Extended Project pilot. Anthony Daly and Anne Pinot de Moira, 2010

In this evaluation of the Extended Project Qualification, achievement was linked to intrinsic motivation and the extent to which learners took a deep learning strategy, searching extensively for information and building a comprehensive knowledge base. This is clearly a desirable attribute for lifelong learning and indicates that the qualification encouraged students to be innovative and creative about their learning. Furthermore, engagement with the project appeared independent of ability, lowering barriers to participation.

Using the theory of planned behaviour to understand student choice at A-level. Rachel Taylor, 2010

Recent trends in A-level uptake have led to concern that students are opting for ‘easier’ subjects. This is an in-depth study of the attitudes and beliefs that underpin students’ intentions to study Media Studies and Physics. As well as future career and study plans, the key beliefs related to the views of ‘significant others’. That is the extent to which students believed that their teachers, parents and friends would approve of their choice. This work gives useful clues as to how students’ behaviour could be influenced and how potential barriers to subject uptake might be removed.

Law and Psychology Admissions Tutors’ perspectives of how A-levels prepare students for university study. Anthony Daly, 2007

The extent to which A-levels prepare students for university is often publicly debated. This study sought the views of university admissions tutors for Law and Psychology. While most believed that A-levels provided a good basis, a significant minority shared doubts as to the validity of particular A-levels, such as Law. This work served as an impetus to dialogue between AQA and Higher Education.

For more information about any of the projects in this area, please contact:

Dr. Ian Stockford Head of Education Research e-mail: istockford@aqa.org.uk Tel: +44 (0)161 957 3353

StandardsOne of AQA’s priorities is to maintain comparable standards over time, between awarding bodies and, where appropriate, between qualifications. Part of our work is to provide the best possible technical support to the setting of grade boundaries. However, we have a much broader and deeper programme of research into the technical issues that influence standards and innovative approaches to the setting and maintenance of standards.

Recent examples of our work

Setting and maintaining GCSE and GCE grading standards: the case for Contextualised Cohort-Referencing. Neil Stringer, 2010

In order to maintain standards over time examiners scrutinise candidate work whilst bearing in mind the difficulty of the question papers relative to those of the past. They are asked to set grade boundaries such that the standard at key grades is carried forward. Evidence of the enormous cognitive complexity of this task is reviewed, and the need for good technical data to support examiners’ judgements made clear. An innovative form of cohort-referencing that is sensitive to changes in exam entry patterns is suggested.

Awarding GCSE And GCE - Time To Reform The Code Of Practice? Ben Jones, 2010

The process by which qualification standards are maintained as specified by the statutory code of practice has changed little since A-levels started in 1951, despite critical research evaluations, technological developments and the availability of mass, candidate-level data. It is expensive, opaque and not conducive to rigorous monitoring and evaluation. Evidence demonstrates that strict adherence to this code provides no guarantee of the alignment of standards. In short, it is argued that the process is in need of a substantial reform.

“ Helping teachers develop their pupils to their full potential.”

Online or face-to-face? An experimental study of examiner training. Suzanne Chamberlain and Rachel Taylor, 2010

Traditionally examiners have been trained to mark at face-to-face meetings, facilitated by the senior examining team. Technological advances promote the opportunity for online training and removes some of the potential barriers to teachers’ participation. AQA’s online training facility was rigorously evaluated using operational data, but an experiment was needed to control for potential confounds. This experimental study, comparing marking reliability following face-to-face and online training, showed that both training modes had a comparable positive effect on marking reliability. Issues in persuading examiners of the benefits of the system are also discussed.

Contemporary validity theory and the assessment context in England. Neil Stringer, 2008

This review of contemporary validity theory argues that those responsible for qualifications in England take an outdated approach to test validity. The quality of qualifications could benefit from explicit evaluation of validity, especially during development. The literature contains examples of the types of evidence required to satisfy validity concerns and guidance on how to gather it, on which awarding bodies may draw. The involvement of regulatory authorities in qualification development means, however, that responsibility for validity cannot lie exclusively with the awarding bodies, and a coordinated approach to validation is required.

For more information about any of the projects in this area, please contact:

Dr. Chris WheadonHead of Assessment Researche-mail: cwheadon@aqa.org.uk Tel: +44 (0)1483 556 413

Fears For Tiers: Are Candidates Being Appropriately Rewarded For Their Performance In Tiered Examinations? Chris Wheadon and Anton Béguin, 2010

In tiered GCSEs it is essential that standards between the tiers are aligned such that a grade C candidate should not be under or over-rewarded according to the tier entered. This is particularly challenging as research shows that examiners setting grade boundaries are likely to over-reward candidates taking more accessible (foundation tier) exams, and under-reward those taking more demanding (higher tier) exams. This paper explores the use of Item Response Theory, a statistical method, to equate tier level standards to support examiner judgements.

Principles of standard setting. Lesley Meyer, 2009

A beginners’ guide to standard setting - the process of establishing grade boundaries for an exam, dividing the distribution of candidates’ performances into categories eg grade A* through to grade G. In ongoing exams, the aim is to maintain standards between years, between awarding organisations and between cognate subjects, generally in that order. There is debate each summer about public exam standards. Looking at the principles behind the standard setting process gives some indication as to why this is the case – things are not as straightforward as is assumed.

For more information about any of the projects in this area, please contact:

Ben JonesHead of Standardse-mail: bjones@aqa.org.uk Tel: +44 (0)161 958 3779

Assessment QualityAQA’s range of qualifications is designed to provide high quality measurement of pupils’ skills, knowledge and understanding. Using psychometrics - the theory and technique of education measurement - this research programme provides the evidence base for continuous improvement of the quality of our assessments.

Recent examples of our work

Test anxiety, heart rate and performance in A-level French speaking mock exams. Anthony Daly, 2010

While research suggests that a small degree of anxiety can be motivating, debilitating test anxiety can disrupt mental processes. As such, anxiety can complicate the measurement process and threaten assessment validity. This study explores the relationship between test anxiety, heart rate and performance in A-level French students undertaking a mock speaking test. There was a positive relationship between heart rate and performance on the test, but self-reported test anxiety was associated with lower levels of general academic performance. Further research will evaluate this link further, with the view to informing possible strategies to reducing any debilitating effects of test anxiety.

Numbering nested questions. Sofia Parkinson and Neil Stringer, 2010

This experimental study investigated the impact of different question numbering systems on pupil performance. Controlling for ability, the type of numbering system (alphanumeric or sequential) used had no effect on performance. Analysis of pupils’ rubric infringements found no evidence of systematic error caused by either system. Pupils identified strengths and weaknesses in both systems but expressed no strong preference.

AQA’s Centre for Education Research and Policy

Copyright © 2011 AQA and its licensors. All rights reserved.The Assessment and Qualifications Alliance (AQA) is a company limited by guarantee registered in England and Wales (company number 3644723) and a registered charity (registered charity number 1073334). Registered address: AQA, Devas Street, Manchester M15 6EX. aqa.org.uk

“ Our research provides the evidence base for developing policy and continuously improving assessments.”