Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | peyton-sayer |
View: | 213 times |
Download: | 0 times |
Designing an Assessment Designing an Assessment System for the Race to System for the Race to
the Topthe Top
Edward HaertelSchool of EducationStanford University
RTTT Public Meeting, General AssessmentDenver, ColoradoDecember 1, 2009
1
What’s Different This What’s Different This Time?Time?In the absence of well-defined and demanding standards, education in the United States has gravitated toward de facto national minimum expectations, with curricula focusing on low-level reading and arithmetic skills and on small amounts of factual material in other content areas. Most current assessment methods reinforce the emphasis on these low-level skills and processing bits of information rather than on problem solving and critical thinking. The adoption of world-class standards would force the Nation to confront today's educational performance expectations that are simply too low.
NCEST Report, 1992
2
New IdeasNew IdeasCommon Standards
◦fewer, clearer, higher?Aligned assessments of individual
growthExplicit attention to range of
assessment types / formats, content sampling, item release plans, etc.
21st century technology◦computer-based testing, longitudinal
growth models, Evidence-Centered Design, …
3
Old IdeasOld IdeasIf assessments are aligned to
standards, then growth in test scores will track intended schooling outcomes
Essentially all valued schooling outcomes can be captured by test performances of◦students working alone◦without “smart workplace” resources◦answering clear questions with clear
answers4
Old IdeasOld IdeasOne assessment system will “do it
all”◦track progress
of students / schools / …
◦inform decisions about graduation / college readiness /
career readiness / college course placement
◦support improvement in teaching / learning / instructional programs
◦determine effectiveness of schools / principals / teachers
5
6
Theories of Action are, if
anything, less clear than in
1992A Theory of Action is a chain of logical propositions that “connects the dots,” explaining step by step, in commonsense terms, how the assessment system, accountability system, and related reforms will lead to improved learning outcomes
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkAligning tests and curricula to
common standards is not sufficient to create a coherent system
7
Content Standards
Curriculum &
Instruction
Assessments
?
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkAligning tests and curricula to
common standards is not sufficient to create a coherent system
8
Content Standards
Curriculum &
Instruction
Assessments
?
11th Grade History /11th Grade History /Social StudiesSocial Studies
US11.11.2.
Discuss the significant domestic policy speeches of Truman, Eisenhower, Kennedy, Johnson, Nixon, Carter, Reagan, Bush, and Clinton (e.g., education, civil rights, economic policy, environmental policy).
9
Item Testing US11.11.2Item Testing US11.11.2
10
High School BiologyHigh School Biology
BI6.f
Students know at each link in a food web some energy is stored in newly made structures but much energy is dissipated into the environment as heat. This dissipation may be represented in an energy pyramid.
11
Item Testing BI6.fItem Testing BI6.f
12
Rethinking AlignmentRethinking Alignment
It is not enough for each item to be somehow connected to one or more standards
Assessments, standards, and curriculum and instruction must all be considered together ◦Curriculum-embedded formative
assessments◦Portfolios incorporating well-specified,
externally scorable components
13
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkTesting need not imply that
students work alone, without references or resources, on small tasks with one right answer◦Well-structured problems
(Frederiksen, 1984)
◦Content-lean, process-constrained tasks(Baxter & Glaser, 1998)
◦Sequestered Problem Solving(Bransford & Schwartz, 1999)
14
Rethinking What It Means Rethinking What It Means to Take A Testto Take A TestA comprehensive assessment system
must also include◦ ill-structured problems with multiple solutions
that are better or worse, not right or wrong◦content-rich, process-open tasks requiring
flexible, adaptive application of coherent knowledge structures to solve novel problems
◦ tests of preparation for future learning – measuring skill in approaching new problems, “knowing with” as well as “knowing how” or “knowing that”
15
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkA single form of testing cannot
satisfy the full set of RTTT design requirements◦Sole reliance on external, on-demand
tests will fall short, even if constructed-response items are included
◦Sole reliance on external, on-demand tests will fall short, even if interim / benchmark assessments are included
16
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsA comprehensive assessment system
must use multiple forms of evidence◦curriculum-embedded assessments◦extended projects◦a range of student performances
including finding, organizing, and evaluating information analyzing and synthesizing data planning and carrying out investigations solving unfamiliar, open-ended problems written reports and oral presentations
17
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsDifferent forms of assessment,
organized in different ways, may be required for◦improving instructional decision
making in the classroom◦tracking student growth and
predicting college or career readiness
◦accountability
18
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsStandards, assessments, curriculum,
and instruction must be aligned and coherent◦horizontal links across content areas◦vertical links across grade levels
Assessment at the classroom or school level may employ matrix sampling◦different students answer different
questions to provide better group-level information
◦some assessment system components may not yield individual scores
19
SummarySummaryWe must RETHINK Alignment
◦not just test-to-standards, but instead a coherent, simultaneous consideration of standards, assessments, curriculum and instruction, and I might add, teacher preparation
20
SummarySummaryWe must RETHINK what it means to
take a test◦not just sequestered problem solving,
with tidy questions and routine problems, but also complex tasks with multiple solutions judged by multiple
criteria sometimes worked out over an extended time sometimes involving collaboration sometimes requiring students to figure out
what information they need and where to find it
21
SummarySummaryWe must RETHINK multiple
indicators◦not just a broader mix of item formats◦not just interim plus end-of-year tests◦some portion of classroom assessment
must be sufficiently documented and structured so as to be incorporated into comprehensive assessment systems to meet multiple needs
22
23
Thank you