www.gu.se
SQA SeminarGlasgow, 8 April 2009
Sweden’s experiences in curriculum and assessment development
Gudrun Erickson
University of Gothenburg, SwedenDepartment of Education
www.gu.se
Outline
• The Swedish school system
• Curriculum: current system – discussions – (proposed) changes
• Assessment: current system – discussions – (proposed) changes
• Collaborative approaches to national assessment
• National and international experiences
• Concluding remarks – Common concerns
• DISCUSSION
www.gu.se
The Swedish school systemhttp://www.skolverket.se/sb/d/190
• Highly decentralized; main responsibilities at local level
• National level (e.g.): Education Act; Curricula; Syllabi for subjects, including grading criteria;
National tests
• A goal and criterion referenced system
• Preschool; 9-year compulsory school; upper secondary education; adultadult education; universities… (http://skolnet.skolverket.se/polopoly/utbsys-eng/)
• > 98% of students continue to three-year upper secondary education
• Development dialogues; individual development plans; written reports; formalgrading (3/4 point scale), from school year 8
• Teachers responsible for grading
• Extensive system of advisory national testing and assessment
www.gu.se
The Swedish National Agency for Education
Director-generalDeputy director general
National development
National evaluation
Analysis and research
Finance
Human resources
ICT
Quality development.
Advisory council
Education Administration
School improvement
Government funding
Education statistics
Result analysis
Adult education
Upper secondary school
Preschool and Compulsory school
Assessment and testing
Director-general´s office
Legal secretariatInformation service
Information
Internal audit
www.gu.se
Syllabi and local planning
Nationallevel
Locallevel
Whythis subject?
Whatdirection?
How?(Very limited
guidance)
When?(Limited
guidance)
Why this to reachgoals?
What (exact)subject matter?
How will we work?
When to teach what?
www.gu.se
Balancing act
Equity / EqualityLocal independenceFlexibility Quality assurance
www.gu.se
Discussions & (Proposed) Changes• Increasing clarity of curricula and syllabi, including indications of
essential content areas
• Increased quality control
• More grade levels (from 3/4 to 6 + “no basis for grading”)
• Earlier start of formal grading (from school year 8 down to 6)
• Earlier national tests; national tests in a wider range of subjects
• Individual written documentation from school year one
Ongoing revision of national curricula and syllabi
www.gu.se
Aims of the current national testing/assessment system
• Enhance individual educational achievement;
• Clarify curricular subject goals and indicate clearly strengths and weaknesses in individual learner profiles;
• Concretize curricular subject goals and grading criteria;
• Enhance equity and fairness in assessment and grading;
• Provide statistics for local and national analyses of educational achievement
Advisory function
www.gu.se
The national assessment systemshould not:
• Determine the precise choice of content and methodsTo be done in collaboration between teachers and students
• Function as examinationsTeachers to award grades based on the assessment of students’ accumulated work;Consequently, the national tests have an advisory function; however, to what extent is not defined
www.gu.se
National assessment materials• Formative materials / Diagnostic materials,
including models for self and peer assessmentA widening range of subjects
• National tests: Summative subject tests (mandatory)Extensive materials, including guidelines
An increasing number of ‘core’ subjects (Sw, Maths, Eng, Science…)
• Electronic assessment bank (formative & summative materials) A widening range of subjects
The Swedish National Agency for Education commissions different university departments in the country to take responsibility for test
development (and research)
www.gu.se
Reactions to national tests• Students usually positive to varied, authentic, “different” tasks/tests
• > 95 % of teachers positive to the national testsfunction, content, level of difficulty, guidelines, individual proficiency profiles, layout…
• [Teacher] discussions, e.g., about workload; weight/importance of test results; adaptation/accommodation; standards…
• [Political] discussions, e.g., about number of tests; aim(s) of national tests; role of national test; stability over time; reliability…
www.gu.se
Test grade: national subject tests (2008)year 9 – upper secondary education
% “Not pass”/ Pass Pass with Pass withFail distinction special
distinction
Swedish9 3% 41% 45% 11%B 11% 46% 34% 9%
Mathematics9 17% 48% 26% 10%A 23% 45% 23% 9%
English9 4% 35% 43% 18%A 6% 39% 43% 11%
www.gu.se
Final Grade & national Test Grade– end of compulsory school (year 9) – 2008
FG<TG FG=TG FG>TG
Swedish 6 % 79 % 15 %
Mathematics 2 % 74 % 24 %
English 7 % 84 % 9 %
www.gu.se
Can teachers’ ratings be trusted?• Joint study (2008) by the Swedish NAE and four university departments
responsible for test development; to be published shortly on the NAE website
• Random sampling: 100 teacher-rated grade 9 tests of Swedish, Mathematics and English (oral subtests not included, since recording is not a requirement)
• Re-rating by three independent raters; 10-point scale
• To some extent, varying results between subjects (higher correlations for Mathematics and English than for Swedish)
• Example: EnglishListening & Reading comprehension (50/50 constructed and selected response): r = > .99; Written production (“essay”): r = .86-.93; Generalizability coefficient: .85 [Analyses of oral test results in similar studies show comparable results]
www.gu.se
Collaborative approaches to national assessment
• Political level (NA) – Universities
• University level (national and international)
• Test development (researchers from different disciplines, teacher trainers, teachers, students…)
• Local test administration
• Rating by teachers
• Reporting by teachers (results and reactions)
• Data collection – Analyses – Research
• Public reporting / Dissemination
www.gu.se
A collaborative test development process (FL)
• Analyses of literature, research, curricula, examples of tests• Development of items and tasks based on common, explicit
principles and specifications• Piloting with a limited number of students > Adjustment of tasks• Large scale pre-testing in randomly selected classes in the country
(n ≈ 400 students/task); systematic collection of feedback from all participating teachers and students
• Analyses of test results and feedback (a "q+q” approach)• Selection/Sequencing of tasks; Standard setting and Benchmarking• Nation-wide test administration• Collection of data / Analyses / Research > Reporting (publicly
available on the web)
www.gu.se
Contributors and contributions • Teachers and Teacher trainers
Development groups; Task/Item construction; Mini trials – Pre-testing (administrating, observing, analysing, discussing, reporting): Selection, composition, sequencing, standard setting; Rating and benchmarking; Reporting and responding after administration of tests…
• Students of ‘all’ agesProviding information and sharing their views in interviews and (regular) questionnaires in connection with pre-testing; comments on, e.g, relevance; perceived level of difficulty; content; vocabulary; clarity of instructions; time, speed of speech; Retrospective, task related self-assessment…
• ResearchersAspects of language; Gender; Quantitative properties of item and test data; Teachers’ handling of constructed response; Progression in tasks and tests; Dimensionality; Rater introspection; Test-taker and teacher feedback…
www.gu.se
Current discussions• Number/Type of aims for national assessment system?
• Clarifying/Strengthening the role of national tests?
• Formative and Diagnostic materials – where, how and from/by whom?
• Effects and implementation of computer assisted testing and assessment?Aspects of validity? Linear – Sequential – CAT..?
• Increasing standardization - if yes, concerning what, and how?
• Clearer emphasis on aspects of reliability (equity)?
• How to avoid narrowing the curriculum - negative impact/washback?
• Emphasizing common basic principles for all types of assessment !
www.gu.se
Good [language] testing and assessmentaccording to approximately 1 400 teenagers in 10 European countries
Breadth and VariationFocus on ”usefulness” / communication
Learning potentialClarity
FairnessEnough timeChallenge
(Erickson & Gustafsson, 2005)www.ealta.eu.org/resources.htm
www.gu.se
EALTAwww.ealta.eu.org
European Association for Language Testing and Assessment
• Broad membershipTeachers, Teacher trainers, Developers of large scale testing systems
• Non-commercial• Individual membership free
• Founded with support from the EU
• Resources page; Discussion list for membersAnnual conferences (June 2009: Turku, Finland)
797 individual members (41 European countries); 81 associate members(27 countries, 5 continents); 51 institutional members; 16 expert members
www.gu.se
EALTA Guidelines for Good Practice in Language Testing and Assessment
Adopted in 2006; Currently translated into 34 languagesTranslations freely available on the web
Address the activities of the three membership categoriesvia questions
Initial emphasis on basic principles for all types of assessment:
TRANSPARENCYRespect for students/examinees, responsibility, fairness, reliability, validity,
collaboration among the parties involved
www.gu.se
Concluding remarks – Common concerns• Creating a reasonable balance between local and central initiative,
interpretation and responsibility
• Maintaining high standards of validity as well as reliability
• Aiming for positive impact on learning and teaching
• Creating a reasonable balance, and maintaining a close cooperation, between R & D – research and development / researchers and test
developers
• Further developing and elaborating methods of collaboration with wide groups of stakeholders
• Bridging the gap between formative and summative assessment – emphasizing common, basic principles