Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | sophia-gordon |
View: | 223 times |
Download: | 0 times |
Too much testing, too little learning – getting the balance right
Tim Oates
Group Director
Assessment Research & Development
Criteria relating to assessment – ‘Cambridge Approach’
Reliableconsistent measurement
Validmeasures precisely what it claims to measure
Sound construct basemeasures something consistent with curriculum aims
Consequential validitythe uses to which the assessment is put are technically and ethically sound
Beneficial impactthe full range of effects are beneficial
Utilitycost, resource
Problems in UK assessment
Underdeveloped formative assessmentAssessment dominating curriculum thinking Relentless transformation into high stakes Creep in functionEscalation of purposes
http://www.cambridgeassessment.org.uk/Images/109848-cambridge-approach.pdf
Or google ‘Cambridge Assessment Cambridge Approach
In assessment, the concept of ‘construct’ is vital
Multiply three digit numbers Understands and is inventive with metaphorReads a wide range of books for pleasureDiffusion across a membraneUnderstand the concept of percentage and calculate pc Use the concept of inequality to analyse social relationsUnderstands conservation of massMeasures accurately to quantify oxidationVerbal reasoningExternalising behaviourDiagnoses malignant melanomaLands with one engine on fire and undercart locked in ‘up’ position
Distributing responsibility for producing public goods of education
• Discipline-specific knowledge, skills and understanding in broad range of disciplines
• Orientation to learning, ‘learning to learn’
• Physical and mental well-being
• Personal and social identity
• Personal capitals (Bynner et al)
• Social capitals (Schuller et al)
• Cultural capitals
• Moral, civic and political understanding, including international awareness
• Facility in technology
Distributing production of public goods across the key elements of the curriculum
• Taught curriculum – subjects• Taught curriculum – cross curriculum elements • ‘Taught’ curriculum – extra-curriculum elements
• Extra-curriculum elements – guided (school trips etc) and unguided (student clubs etc)
• Institutional participation – student councils etc, learner voice• Support elements – IAG etc
• Ethos – values and value-driven practices• Culture – lived experience of the institution
Distributing responsibility for different facets of the curriculum
Aims Central Government
Content Government Agency (eg Ofsted)
Pedagogy Local Government/intermediate bureaucracy
Assessment Governing Bodies
Evaluation Head teachers
Middle tier management
Teachers and assistants
Other staff
Pupils
Parents
Other organisations – health organisations, police etc
Exam boards and assessment organisations
High Expectations
Low Support High Support
Low Expectations
The Zeitgeist and context in 2007
Performativity culture – increasing elaboration of accountability measures – refinement and extension
High support and high expectations
Very high level of centralised innovation and direction through non-statutory instruments
Constant underlying shift in the locus of control
A reduction of ‘curriculum thinking’ to ‘qualifications thinking’ – ‘syllabus to ‘specification’ – assessment begins to dominate
You need restriction in any system, and some have improved by carefully locating restriction in the right place (eg selection to teacher training in Finland, plus dirigiste implementation of comprehensive schooling) – did we, are we, restricting the right things?
Re-conditioning the Zeitgeist
Density of assessment versus the weight of assessment
The recommendation to remove ‘levels’
The purpose of assessment: a focus on constructs
Shifting the sense of who is responsible for what
The ‘powerful knowledge’ thesis
Empirical evidence on personal and social capitals
Transnational comparisons of the focii of national curricula and the implicit and explicit imperatives in education
‘Could do better’ Cambridge Dec 2010
Assessments – what do we have?
1National curriculum tests and tasks - which most pupils in State schools are expected to take
2Public examinations - which most pupils are expected to take
3Optional tests and progress tests - which schools can elect to enter pupils for
4Tests other than national curriculum tests (available from commercial companies) - which many schools choose to use
5In-course or school-based assessments - which are set throughout a pupil's provision
Detail
KS11 maths test1 reading test1 spelling test1 writing task
KS23 maths (non-calculator; calculator; mental arithmetic); 3 English (reading, writing, spelling & handwriting); 2 science
KS31 reading & writing1 Shakespeare2 Science3 maths
GCSEAverage number taken = 9.5Mode of 2 papers per subject (some have more, others fewer)
Advanced level AS in four subjects, for many one of the three units is coursework. Mode approx 10 papers.
A2 as for AS, for three subject mode approx 8
Number of tests/ exams
KS1 4
KS2 8
KS3 7
GCSE 19
AS 10
A2 8
Total 56
2002Daily Mail 105
Trends to 2002
•Modularisation in A level
•Increasing elaboration of National Assessments (eg mental maths)
•Refinement in the form of the national tests
•Enforcement of League Table measures (performance tables and targets)
Trends from 2002
•Increase in the battery of national assessments (eg ICT KS3)
•Development of new test forms (Single Level National Tests)
•State-initiated formative assessment (Assessing Pupil Progress)
•Shift from external testing in KS1 to teacher assessment
DetailEarly Years
Reported as a profile of 13 scales, using a score of 0-9, covering cognitive, social, physical and emotional development
Yr 1 (age5) Phonics screening check Statutory test, based on reading 40 words and non-words, taken in one week window in June.
KS1 Yr 2 (age7) statutory teacher assessment – levels for reading, writing, speaking and listening; overall level for mathematics; and a level for each attainment target in Science. P-scales for pupils with special educational needs.
KS2 Yr 6 (age 11) English reading test Level 3-5 (plus Eng level 6 reading test); English grammar, punctuation and spelling test Level 3-5 (plus Eng level 6 test); Mental maths test; Maths test A; Maths test B; (plus level 6 paper 1, paper 2)
KS3 Yr 7, 8, 9, Optional tests Eng, Maths; yr 9 Science; Yr 9 (Age 14) – statutory teacher assessment in core and non-core subjects (13 subjects) – scale 1-8 in 10 subjects.
Modular GCSE Average number of GCSE or equivalent taken = 10.9Mode of 2.5 papers per subject (some have more, others fewer) plus controlled assessment
Advanced level AS in four subjects, for many one of the three units is coursework. Mode approx 10 papers.
A2 as for AS, for three subject mode approx 8
Number of tests/examinations
EYFS 0
KS1 1
KS2 5 (7 for higher ability)
KS3 0
GCSE 27
AS 10
A2 8
Total 51 (56 – 2002)
2012
Trends to 2012
Rise in shift to GCSE ‘equivalent’ qualifications, attributed to schools striving to meet national targets
Crisis in the practical administration of National Tests
Cessation of expansion of National Testing – abandonment of development of KS3 ICT tests
Increase in attention to formative assessment including introduction of Assessment of Pupil Performance (oriented to determining ‘levels’ through formative assessment processes)
Exploration of innovations in marking (eg rank ordering /paired comparison for national tests in English)
Reduction in National Tests (removal of KS3 tests, reduction of KS2 tests from Sci Eng Math to Eng and Math only)
Elaboration of targets and measures including introduction of EBac (English Baccalaureate) as a school performance measure
Trends from 2012
Introduction of phonics screening test
Linear qualifications promoted and modular examinations discouraged or abandoned
Contracting model for GCSE-level qualifications (move to single board per subject)
Abandonment of APP as a national initiative
Greater HE involvement in design and operation of A Levels encouraged
2014 Benchmarking – a means of measuring progression (Bew recommendation)Phonics check KS2 tests
Nick Gibb 2010 – ‘..In Primary, apart from KS2 and the phonics check, I have no interest in the assessment which is done…’
Be very careful to read this in the right way – it confirms the professionalism of teachers and deliberate contraction of the role of the State
High autonomy in formative assessment
Constructs are all-important
The National Curriculum 1995
Chemical Reactions
i. that when chemical reactions take place, mass is conserved;j. that virtually all materials, including those in living systems, are
made through chemical reactions;k. to represent chemical reactions by word equations;l. that there are different types of reaction, including oxidation and
thermal decomposition;m. that useful products can be made from chemical reactions, including
the production of metals from metal oxides;n. about chemical reactions, e.g. corrosion of iron, spoiling of food, that
are generally not useful;o. that energy transfers that accompany chemical reactions, including
the burning of fuels, can be controlled and used;p. about possible effects of burning fossil fuels on the environment.
Science - key stage 3Materials and properties
National Curriculum 2007
Chemical and Material BehaviourIn their study of science, the following should be covered:
a. chemical change takes place by the rearrangement of atoms in substances;
b. there are patterns in the chemical reactions between substances;
c. new materials are made from natural resources by chemical reactions;
d. the properties of a material determine its uses.
The National Curriculum – School Curriculum distinction
The ‘powerful knowledge’ thesis
What’s wrong with a knowledge-based curriculum?
2010
Levels
3 contrasting, co-existing models
1 the score on a compensation-based test2 best fit 3 threshold
Poor construct integrityContradictions between school and State Poor communication with parents Undue pace – expectations of Ofsted Labelling – contrary to TGAT
The lessons from transnational comparisons
High attainment, high equity and high enjoyment is possible
Don’t assess everything which moves
High density formative assessment is valuable to pupil (externalisation of inner processes) teacher (information on progress, misconception and concept development) and parent (how can I best help my child)
‘Has Alex developed a sufficient understanding of conservation of mass to move to next segment of the learning progression?’ – hence year by year in Primary
Spiral curriculum – application of concepts – Singapore Different models of ability and progression – Stigler and Stevenson
The ACHIEVE experience
ECERTS-EECERTS-R
An ideal?
High intensity and density
Formative function
High levels of apparent duplication and redundancy
Resource-intensive
Highly regulated with high accountability
Evidence-driven
Med School #2Written examinations (short answer, extended matching, multiple choice, modified essay questions)Computer-based tests including data interpretation testsShort in-course examinationsWritten accounts of problems and casesPoster, clinical and other presentationsLogbook and workbook recordingDirect observation of practice including graded performanceClinical examinations – OSCEs Formative assessments in courseStudent-selected componentsPortfolio
Med School #4Module assessment pattern Written examinations (short answer, extended matching question) Practical assessment of communication and clinical skillsCase presentationsCase reports (from family studies) Patient portfolio Research project Personal development portfolio
Med School #7On line examinations (extended matching item, extended response, short answer, multiple choice, data interpretation, drag and drop graphical questions)Written assignments (essay, case reports) Verbal reports and presentationsOn line ‘progress test’OSCEsRecord of Clinical Experience (RoCE)Portfolio Workplace-based assessment tools for specific skills and knowledgeDirect Observation of Procedural Skills (DOPS)Mini-Clinical Evaluation Exercise (Mini-CEX)Case-based discussion (CBD) Ward simulation exercise (filmed) Formative assessment
Med School #19Modular assessment pattern Written examinations and assessments of performance during rotationsOSCEsFormative assessment of clinical skillsFormative assessment of clinical knowledgeSummative written assessment of laboratory medicinePortfolio
Clinicians, lecturers, tutors and students all emphasised that there was ‘…no real difference between learning and assessment….’
Student: ‘…it is all so intensive…you are reading all evening and night just to get through the mountain of material, and you could be asked a question on an obscure bit of it at any time and in any part of the course; an exam, a question on the ward, by another student…’
Student: ‘….you know you’re going into an OSCE and you know there’s no time to think…you prepare for it, of course, but it could call on knowledge from any part of your course, from any part of your experience…’
Living in a levels-free world
Wroxham does it Finland, Singapore do itSoft landing as use decays Focus on deep, secure learning of key constructsImplement learning progressions (Schmidt and Prawat)
What we may have from 2014consultation 11 Oct close
Year by year statement of contentEach school publishing its school curriculum and assessment schemeLevels no longer used
Assessment modelStatutory tests at KS1 M&E – discussion of precise form Phonics screening check end of Yr1 – with cut score Statutory tests at KS2 M&E (scale score and decile reporting) KS2 reported against prior attainment measure – baseline assessment
Non-modular GCSE conditioned by accountability measures Non-modular A level VQs
Let’s not fool ourselves
We have one of the most diverse systems in the world(institutional forms, local structures, size of schools, school transfer, ideas about education….)
This was true in 2000 and remains trueMany of the structural shifts began years ago (GM, Academies)
What should we restrict and how? State role in National Curriculum, in assessment
My ideal Educationally focussed measurement
Potent and valid formative assessmentDiagnostic assessment – CAT, PIPS, VESPARCH
High density, low weightHigh autonomy in selection and use (assessment schemes)A pull down bank of items
Independent measurement for monitoring national standards
A switch to high equity and high attainment through attainment measures not progress measures
No regression to thresholds (the grade D phenomenon)
Professionalisation of assessment expertise – use of local collaborative mechanisms for development and promotion of good practice – getting the right unit of collaboration