Oregon’s Alternate Oregon’s Alternate Assessment: Past, Present, Assessment: Past, Present,
and Future Tenseand Future Tense
Oregon Department of Oregon Department of EducationEducation
Dianna Carrizales, PhD
Office of Student Learning and PartnershipsOffice of Assessment and Instruction
In conjunction withUniversity of Oregon: Behavioral Research and Teaching
Group
In the Beginning In the Beginning
• Accepting the Accepting the NCLBNCLB challengechallenge– Meaningful assessment of students with Meaningful assessment of students with
significant cognitive disabilitiessignificant cognitive disabilities
• Defining the population Defining the population – Functional life skills Functional life skills – Some academic skills Some academic skills
The Evolution of a System The Evolution of a System
• 2001-20052001-2005 • 2006-20072006-2007
10 CIM
8
7
6
5
4
3
10 CIM
8
7
6
5
4
3
Extended CLRAS
Exte
nd
ed
A
ssessm
en
t
Scaff
old
ed
A
dm
inis
trati
on
Accom
mod
ate
d A
ssessm
en
t
Gen
era
l A
ssessm
en
t
Targ
ete
d A
ssessm
en
t
Juri
ed
Assessm
en
t
Accom
mod
ate
d A
ssessm
en
t
Mod
ified
Assessm
en
t
Extended K - 2
Change and RefinementsChange and Refinements
• Accepting the Accepting the peer review challengepeer review challenge– Grade level contentGrade level content– Comparability of formsComparability of forms– Technical documentationTechnical documentation– Sound alternate achievement standards Sound alternate achievement standards
based on research-based and based on research-based and documented proceduresdocumented procedures
• General AssessmentGeneral Assessment
• General Assessment with General Assessment with accommodationsaccommodations
• Extended AssessmentExtended Assessment– ElementaryElementary– MiddleMiddle– HighHigh
• Scaffold Administration option of Scaffold Administration option of Extended AssessmentExtended Assessment
Current Conclusion Current Conclusion (2007-(2007-2008) 2008)
• 2007-20082007-2008
10 CIM
8
7
6
5
4
3
Exte
nd
ed
A
ssessm
en
t
Scaff
old
ed
A
dm
inis
trati
on
Accom
mod
ate
d A
ssessm
en
t
Consider General Assessment (with Consider General Assessment (with or without accommodations) ifor without accommodations) if
• Student:Student:– Performs at or around grade levelPerforms at or around grade level– Has academic difficulties that primarily surround reading but Has academic difficulties that primarily surround reading but
may be average or close to average in other subject areasmay be average or close to average in other subject areas– Has academic difficulties in areas other than reading that are Has academic difficulties in areas other than reading that are
“mild to moderate” and can typically be addressed by using “mild to moderate” and can typically be addressed by using simplified languagesimplified language
– Is reading within two to three grades of his or her enrolled Is reading within two to three grades of his or her enrolled levellevel
• Instruction:Instruction:– Is primarily general curriculum instruction (but may also use a Is primarily general curriculum instruction (but may also use a
specialized curriculum in some areas)specialized curriculum in some areas)
Consider Extended Assessment Consider Extended Assessment ifif• Student:Student:
– Performs well below grade levelPerforms well below grade level– Is significantly below grade level in readingIs significantly below grade level in reading– Has academic difficulties that are generalized (to all subject Has academic difficulties that are generalized (to all subject
areas) and are significantareas) and are significant– Benefits from specialized individual supportsBenefits from specialized individual supports
• Instruction:Instruction:– Is primarily a specialized curriculum or Is primarily a specialized curriculum or – From general curriculum must be significantly reduced in From general curriculum must be significantly reduced in
breadth, depth, and complexity breadth, depth, and complexity
Consider Scaffolded Administration Consider Scaffolded Administration of Extended Assessment if:of Extended Assessment if:
• Student:Student:– Performance is significantly impacted by a disability– Does not read– Has academic, mobility, and receptive and expressive
language difficulties that are generalized and significant– Relies on individual and significant supports to access reduced
content materials
• Instruction:Instruction:– Is from a specialized curriculum and has functional Is from a specialized curriculum and has functional
components components and/orand/or– Includes academic goals that are significantly reduced in Includes academic goals that are significantly reduced in
depth, breadth, and complexity from grade level contentdepth, breadth, and complexity from grade level content
Participation by Subject Participation by Subject (2006-(2006-2007)2007)
Subject Elementary Middle/High
Total number of students participating per subject area
Total number of elementary
participants per subject
Percentage of elementary
participants by subject
Total number of middle/high
participants per subject
Percentage of middle/high
participants by subject
Reading 5444 3178 58%* 2266 42%
Mathematics 4720 2483 53% 2237 47%
Writing 2174 1006 46% 1168 54%
Science 1562 552 35% 1010 65%
*Read: 58% of students taking an Extended Reading Assessment were elementary students.
Participation by Gender Participation by Gender (Reading(Reading
exemplarexemplar))
Gender Elementary Middle/High
N % N %
Female 1077 34%* 827 36%
Male 2101 66% 1439 64%
*Read: 34% of the students taking Elementary Extended Reading are female.
2006-2007 Extended 2006-2007 Extended Assessment Participation by Assessment Participation by DisabilityDisability• ElementaryElementary
– Mental Retardation (25.5%)Mental Retardation (25.5%)– Specific Learning Disabilities Specific Learning Disabilities
(20.75%)(20.75%)– Autism Spectrum Disorders Autism Spectrum Disorders
(16.5%)(16.5%)– Communication Disorder Communication Disorder
(12.25%)(12.25%)
• Middle/HighMiddle/High– Mental Retardation (42%)Mental Retardation (42%)– Autism Spectrum Disorder Autism Spectrum Disorder
(18%)(18%)– Specific Learning Disabilities Specific Learning Disabilities
(12.25%)(12.25%)– Other Health Other Health
Impairments(9.5%)Impairments(9.5%)
Extended Assessment Extended Assessment DescriptionDescription
• Performance assessmentPerformance assessment• Individually administeredIndividually administered• Response options or selected response Response options or selected response
(not multiple choice)(not multiple choice)• Scaffold vs. standard administrationsScaffold vs. standard administrations
– Direct instruction principlesDirect instruction principles– ““Multiple choice” optionsMultiple choice” options– Accommodations research Accommodations research
• Based on grade level contentBased on grade level content
Development of Grade-Level Development of Grade-Level Items for Oregon’s Extended Items for Oregon’s Extended AssessmentAssessment• Grade-level content standardsGrade-level content standards
– Reduction in Complexity (one slide)Reduction in Complexity (one slide)– Reduction in breadth (one slide)Reduction in breadth (one slide)– Reduction in depth (one slide)Reduction in depth (one slide)
• http://www.ode.state.or.us/teachlearn/real/newspaper/http://www.ode.state.or.us/teachlearn/real/newspaper/
• Field and pilot testingField and pilot testing– Item level piloting (with individual students)Item level piloting (with individual students)– Population piloting (filming and piloting)Population piloting (filming and piloting)
Reduction in ComplexityReduction in Complexity• Development:Development:
– Select words with the lowest number of Select words with the lowest number of syllables syllables
– Reduce number wordsReduce number words– Monitor clause structure Monitor clause structure – Allow opportunities for modeling Allow opportunities for modeling – Provide more examples when possibleProvide more examples when possible– Test rules rather than exceptionsTest rules rather than exceptions– Monitor sequencing to avoid confusionMonitor sequencing to avoid confusion
• Administration:Administration:– Appropriate pacingAppropriate pacing– Performance-neutral praise statementsPerformance-neutral praise statements
Reduction in DepthReduction in Depth
• Evaluate:Evaluate:– Alignment between depth of knowledge in Alignment between depth of knowledge in
content standards and depth of knowledge in content standards and depth of knowledge in assessment itemsassessment items
– Representation of skill-sets required by each Representation of skill-sets required by each item item • Recognition and reproductionRecognition and reproduction
• Skill or conceptSkill or concept
• Strategic thinkingStrategic thinking
• Extended thinkingExtended thinking
Reduction in BreadthReduction in Breadth
• Creation of grade bands by typical Creation of grade bands by typical developmental divisions developmental divisions – Elementary, grades 3 – 5Elementary, grades 3 – 5– Middle, grades 6 – 8Middle, grades 6 – 8– High, grades 9 – 12 (10)High, grades 9 – 12 (10)
• Vertical alignment of grade level Vertical alignment of grade level content standardscontent standards– Standards that span the Standards that span the
breadth of the grades breadth of the grades assessed assessed
• IdenticalIdentical
• ProgressiveProgressive
Grade 2 Grade 3 Grade 4
Read, write, order, model, and compare whole numbers less than 100
Read, write, order, model, and compare whole numbers less than one thousand.
Read, write, order, model, and compare whole numbers to one million, common fractions, and decimals to hundredths.
ProgressiveProgressive
IdenticalIdenticalGrade 4 Grade 5 Grade 6
Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.
Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.
Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.
Elementary Reading Elementary Reading ExampleExample
Item Content Prompts
Say: Here are some words with other words that are related.
1 Show me happy. Show me the opposite of happy.
[1 = either answer correct / 2 = both answers correct]
2 Show me over. Show me the opposite of over.
[1 = either answer correct / 2 = both answers correct]
3 Show me sea/ see . (Note: either word is correct as both sound the same). Show me another word that sounds the same. [1 = either answer correct / 2 = both answers correct]
4 Show me go. Show me another word that means go.
[1 = either answer correct / 2 = both answers correct]
5 She raised her hand so her teacher would call on her. Show me another word that means raised.
[1 = reached out her hand / 2 = lifted her hand]
Test DescriptionTest Description
• Content prompts: items reflecting content Content prompts: items reflecting content of the test based on grade-level standardsof the test based on grade-level standards
• Prerequisite skills: items taken as a lead-in Prerequisite skills: items taken as a lead-in to the test to determine level of supportto the test to determine level of support
• Access level: Permission of supports (other Access level: Permission of supports (other than accommodations) that can be than accommodations) that can be provided during the performance provided during the performance assessmentassessment
• Scaffold administration: to administer the Scaffold administration: to administer the version with additional prompts, version with additional prompts, supportive language, and graphics supportive language, and graphics
Standard SettingStandard Setting
• Book-marking selectedBook-marking selected
• 32 teachers, 3 days, board-approved32 teachers, 3 days, board-approved
• Standards set by gradeStandards set by grade
• Rigorous outcomesRigorous outcomes
• Unanswered Mathematics questions remainUnanswered Mathematics questions remain– DifficultyDifficulty– PopulationPopulation– Administration/trainingAdministration/training
Extended Assessment Reading Extended Assessment Reading OutcomesOutcomesTable 1: Extended Reading Impact by Grade, 2006-2007
Grade
Does Not Yet
MeetNearly
Meets Meets ExceedsMeets or
Exceeds
3 19% 18% 36% 26% 62%
4 29% 15% 32% 24% 56%
5 30% 18% 35% 17% 52%
6 30% 15% 31% 24% 54%
7 32% 18% 38% 12% 50%
8 38% 19% 31% 13% 43%
10 39% 24% 25% 13% 37%
Extended Assessment Extended Assessment Mathematics OutcomesMathematics OutcomesTable 2: Extended Mathematics Impact by Grade, 2006-2007
Grade
Does Not Yet
MeetNearly
Meets Meets ExceedsMeets or
Exceeds
3 28% 36% 24% 12% 36%
4 52% 12% 15% 21% 36%
5 54% 17% 19% 10% 28%
6 63% 23% 12% 3% 15%
7 69% 15% 14% 2% 16%
8 77% 13% 7% 2% 10%
10 78% 13% 9% 0% 9%
Extended Assessment Writing Extended Assessment Writing OutcomesOutcomes
Table 3: Extended Writing Impact by Grade, 2006-2007
Grade
Does Not Yet
MeetNearly
Meets Meets ExceedsMeets or
Exceeds
4 33% 27% 34% 6% 40%
7 67% 11% 16% 6% 22%
10 66% 3% 23% 9% 31%
Extended Assessment Science Extended Assessment Science OutcomesOutcomes
Table 4: Extended Science Impact by Grade, 2006-2007
Grade
Does Not Yet
MeetNearly
Meets Meets ExceedsMeets or
Exceeds
5 32% 23% 37% 8% 45%
8 33% 48% 11% 9% 20%
10 40% 40% 13% 7% 21%
Training structure Training structure
• 2007-20082007-2008– 8 regional trainings to train trainers (Oct-Nov)8 regional trainings to train trainers (Oct-Nov)– Held at host sites across the stateHeld at host sites across the state– 300 (2006-2007) assessors will be re-trained 300 (2006-2007) assessors will be re-trained
on the assessment, decision-making principles, on the assessment, decision-making principles, and web-training system to train others in their and web-training system to train others in their home-districtshome-districts
– Self-contained web-training siteSelf-contained web-training site– Data-base of trainersData-base of trainers– Training, self-testing, proficiency component Training, self-testing, proficiency component
monitored at a state level monitored at a state level
ChallengesChallenges
• Instructional content vs. accountabilityInstructional content vs. accountability
• Difficulty levelDifficulty level
• Working with the field toward a new Working with the field toward a new mode of thinking about assessing mode of thinking about assessing students with significant cognitive students with significant cognitive disabilitiesdisabilities
• Who is/is not being tested with this Who is/is not being tested with this assessment? assessment?
Lessons learnedLessons learned
• Expectations can be increasedExpectations can be increased• Teachers are open to research-based change that Teachers are open to research-based change that
is results orientedis results oriented• Biggest changes to the assessment since last year: Biggest changes to the assessment since last year:
– Use of Prerequisite SkillsUse of Prerequisite Skills• Biggest changes to training since last year: Biggest changes to training since last year:
– ShorterShorter• The four “P’s” The four “P’s”
– ProcessProcess– PopulationPopulation – ParallelismParallelism – Peer-reviewPeer-review
New directionsNew directions
• Possibly scaling to general Possibly scaling to general assessmentassessment
• Reducing the number of items while Reducing the number of items while increasing the depth and breadth of increasing the depth and breadth of standards coveragestandards coverage
• ReportingReporting