Goal Setting in a Multi-Tier Model
Mark R. Shinn, Ph.D.Professor and School Psychology ProgramNational Louis University, Skokie, [email protected]://markshinn.org
January 9th, 2013
1 of 6 members of Technical Review Panel, National Center for Student Progress Monitoring, USDE/OSEP2003-2007
Editor and Contributor to 2 Major Texts on CBM
Author of More than 75 Refereed Journal Articles and Book Chapters on the Topic of CBM, Progress Monitoring, and Screening
My Area of Expertise
Mark R. Shinn, Ph.D. Serves as a Paid Consultant for Pearson Assessment for their AIMSweb product that provides CBM assessment materials and organizes and report the information from 3 tiers, including RTI. He provides technical support and training.
Mark R. Shinn, Ph.D. Serves as a Consultant for Cambium/Voyager/Sopris for their Vmath product, a remedial mathematics intervention but has no financial interests. He helped them develop their progress monitoring system.
Mark R. Shinn, Ph.D. Serves as a Consultant for McGraw-Hill Publishing for their Jamestown Reading Navigator (JRN) product and receives royalties.He helped them develop their progress monitoring system.
Mark R. Shinn, Ph.D. Serves as a Member of the National Advisory Board for the CORE (Consortium on Reaching Excellence) and receives a stipend for participation. He provides training and product development advice.
DisclosureDisclosure
Background Reading on CBM and Decision Making In Multi-Tiered Model/RtI
Espin, C.A., McMaster, K., Rose, S., & Wayman, M. (Eds.). (2012). A measure of success: The influence of Curriculum-Based Measurement on education. Minneapolis, MN: University of Minnesota Press.
Will Be Available in
• pdf format
• iBook format
Presentation is Based on the Following White Paper in PREPARATION
A “glossy” and official Pearson version will be finished soon and sent to you.
Shinn, M.R. (2012). Measuring general outcomes: A critical component in scientific and practical progress monitoring practices. Minneapolis, MN: Pearson Assessment.
References on CBM and Goal Setting
Bateman, B.D. (2007). From gobbledygook to clearly written annual IEP goals. Verona, WI: Attainment Company.
Bateman, B.D., & Linden, M.A. (2006). Better IEPs: How to develop legally correct and educationally useful programs (4th ed.). Verona, WI: Attainment Company.
Fuchs, L.S., & Shinn, M.R. (1989). Writing CBM IEP Objectives. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children. (pp. 132-154). NY: Guilford.
Shinn, M.R. (2003). AIMSweb™ Training Workbook Progress Monitoring Strategies for Writing Individualized Goals in General Curriculum and More Frequent Formative Evaluation. Bloomington, MN: Pearson, Inc.
Shinn, M.R. (2010). Building a scientifically based data system for progress monitoring and universal screening across three tiers including RTI using Curriculum-Based Measurement. In M. R. Shinn & H. M. Walker (Eds.), Interventions for achievement and behavior problems in a three-tier model, including RTI (pp. 259-293). Bethesda, MD: National Association of School Psychologists.
Shinn, M.R., & Shinn, M.M.. (2000). Writing and evaluating IEP Goals and making appropriate revisions to ensure participation and progress in general curriculum. In C. F. Telzrow & M. Tankersley (Eds.), IDEA Amendments of 1997: Practice guidelines for school-based teams. (pp. 351-381). Bethesda, MD: National Association of School Psychologists.
Accessing Reading Materials
markshinn.org1. Click on the Downloads for
Professionals Icon
2. Click on the Presentations and Handouts Folder
3.Click on AIMSweb Goal Setting in Multi-Tiered Model (Sponsored by Pearson) 2013 Folder
8
Big Ideas
1. A Key Element in Scientifically Based Progress Monitoring is Quality Goals. For Students Who Are Discrepant or At Risk for Failure in Basic Skills, Goals Must Be Statements that Reduce the Achievement Gap!
2. CBM Progress Monitoring is Based on General Outcome Measurement, Where a Few Observable and Measurable Goals Are Used to Measure a Number of Important or Big Things.
3. There are 5 “families” of Progress Monitoring
• Most Frequent PM to IEP Goals for Students Who Receive Special Education
• Very Frequent PM for Students with Severe Achievement Discrepancies Who Receive Tier 3 Intervention
• Frequent PM for Students with At Risk Who Receive Tier 2 Intervention
• Benchmark PM for ALL Students to Ensure Growth and Development
• Most Frequent PM for Students Being Evaluated for Potential Special Education as Part of RtI
4. The Goal Setting Practices Are Similar, Although Not Identical Among the “Families” with Most Practices Tied to a Standard PM Protocol and Some Being Individualized
5. The Biggest Points of Confusion Remain Establishing the Criterion for Acceptable Performance (CAP)
Mark’s 5 Family SummaryFamily Time
FrameGoal
MaterialCAP
OutcomePM
FrequencyIEPs (Individualized Approach)
IEP “Anniversary” Date
Individualized to Reduce the Gap
Significantly Reduces the Gap, Preferably LN
1-2 Times Per Week
Tier 3(Standard Protocol)
End-of School-Year Expected Grade Level Significantly Reduces the Gap, Preferably LN
1 Time Per Week
Tier 2(Standard Protocol)
End-of School-Year Expected Grade Level Reduces the Gap, Preferably LN
Benchmark Assessment Monthly or Bi-Weekly or Weekly
Tier 1(Standard Protocol)
End-of School-Year Expected Grade Level AYP and Passing High Stakes Tests
Benchmark Assessment
RtI as SE Eligibility(Standard Protocol)
Time Limited to Avoid Wait to Fail
Expected Grade Level Significantly Reduces the Gap, Preferably LN
1 Time Per Week
Key Vocabulary to Support Understanding
Curriculum-Based MeasurementShort, standardized basic skills measures validated as general outcomes measures (GOM).
General reading skill or ability:
R-CBM: Oral reading
Maze: Silent reading
General mathematics skill or ability:
M-COMP: General mathematics computation skills
M-CAP: General math concepts and application skills
General writing skill or ability:
WE-CBM: General written expression skills
General spelling skill or ability:
S-CBM: General written expression skills
General Outcome Measurement
GOM assesses progress on a standard and equivalent measure the same way over time.
Think: Testing “small” to make statements about something “big” (i.e.,very important)!
1 min. standardized sample of oral reading (R-CBM) allows statements to be made
about general reading ability
Key Vocabulary
Standard Protocol Goal Setting:
Consistent Process including Time Frame, Goal Material, and Frequency of Progress Monitoring is the SAME for
Students within a “Family” (e.g., Tier 2) of PM
Individualized Goal Setting:
When a Time Frame, Goal Material, and Frequency of Progress Monitoring is DIFFERENT for Individuals
Based on the Severity of the Achievement Discrepancy (IEP Goals)
Key Vocabulary
Time FrameWHEN the Goal is to be Reached
Grade-Level MaterialThe Assessment Material in Which the Student is EXPECTED to Be Successful NOW
Goal MaterialThe Assessment Material in Which the Student is EXPECTED to Be Successful At The End of Intervention and in Which Progress Will Be Measured
Present Level of Performance (PLOP)The Assessment Material in Which a Student is Currently Successful
Criterion for Acceptable Performance(CAP)
How “Successful” Performance will be Judged
Norm-Based Approaches to CAP
Choosing to Define Success by How Other STUDENTS Perform
Standards-Based Approach to CAP
Choosing to Define Success by a Prediction of Passing Success on a State Standards Test
Norm-Based CAPS
Local Norm-Based
Define Success by How Others in the School or Community Perform Using Percentiles and Corresponding Raw Scores
National Norm-Based
Define Success by How Others in the AIMSweb National Norm Data Base Perform Using Percentiles and Corresponding Raw Scores
National Norm-Based Rates of Improvement (ROI)
Define Success by How Others in the AIMSweb National Norm Data Base Perform Using Weekly Rates of Improvement
Standards-Based CAPS
Success Probability: Students scoring at or above the target score have the designated probability (80% or 50%) of success.
AIMSweb users who entered 2010 state-test scores for either reading or mathematics into the AIMSweb system. For reading, there were a total of 32,002 students from 20 states.
A Grade 4 student who reads 67 WRC has a greater than 50% chance of passing the state standards test at the beginning of the year
A Grade 4 student who reads 136 WRC has a greater than 80% chance of passing the state standards test at the end of the year
Ready?
Lots of “Complicated” Information That Requires Thinking!
WHAT’S THE “SAME” REGARDLESS OF PM FAMILY?
Goal Format
<Student> <Behavior> <CAP> <Goal Material> <Time Frame>
<John> <Will Read> <115 Words Correctly (WRC) with 3 or fewer errors> <from a randomly selected Grade 4 Standard Reading Passage> <by the end of the 2013 school year>
<John> <Will Earn a score of greater than 35 points> <on a randomly selected Grade 5 Mathematics Applications Probe> <in 1 Year when his IEP expires>
Beginnings
Goal Format Can Vary<Time Frame><Student> <Behavior> <CAP> <Goal Material>
<By the end of the 2013 school year> <John> <Will Read> <115 Words Correctly (WRC) with 3 or fewer errors> <from a randomly selected Grade 4 Standard Reading Passage>
<Goal Material> <Student> <Behavior> <CAP><Time Frame>
<When given a randomly selected Grade 4 Standard Reading Passage> <John> <Will Read> <115 Words Correctly (WRC) with 3 or fewer errors> <By the end of the 2013 school year>
FIRST STEP: Individualized Progress Monitoring Goals (SE IEPs)
Why?
Observable, Measurable Goals Are Required by IDEA
Most Educators Practice “Procedural Compliance” Rather than Addressing the Intent of the Law
Establishing High Quality Goal Setting and Progress Monitoring in Special Education Creates “Legitimacy” for PM and Accelerates PM Across Tiers and Programs
Start By Abandoning Old Goals
• Student will perform spelling skills at a high 3rd grade level.
• Student will alphabetize words by the second letter with 80% accuracy.
• Student will read words from the Dolch Word List with 80% accuracy.
• Student will master basic multiplication facts with 80% accuracy.
• Student will increase reading skills by progressing through the reading program with 90% accuracy as determined by teacher-made fluency and comprehension probes by October 2013.
• Student will be a better reader.
• Student will read aloud with 80% accuracy and 80% comprehension.
• Student will make 1 year's gain in general reading from K-3.
• Students will read 1 story per week.
XIndividualized Goal Setting Strategies
1. Determine the Present Level of Performance (PLOP) based on Survey-Level Assessment (SLA)
2. Know the Time Frame for the Goal (typically the “anniversary date”--1 year.
3. Determine the Level of Curriculum That Defines Success and Reduces the Gap
4. Define the Criterion for Acceptable Performance (CAP)
A SLA for Grade 6 Student to Write Individualized Goals
Present Level of Performance (PLOP)
Expected Level of Performance
Achievement Level of Average Students in Fall
PLOP in Grade 6
Median of Grade 6 Benchmark Scores
or
3 Individually Administered Grade 6 Passages
24 WRC
A Survey Level Assessment to Write Individualized Goals
PLOP
Expected Level of Performance
Potential Goal (and PM) Material
A Survey Level Assessment to Write Individualized Goals
Goal Material and CAP of 90 WRC
Every Goal Should Be Considered by Advantages and Disadvantages
Advantage(s) of Grade 3 Disadvantage(s) of Grade 4
• Likely More Growth That Student Has Made in the Past
• Doesn’t Reduce the Gap At All
Advantage(s) of Grade 4 Disadvantage(s) of Grade 4
• Likely More Growth That Student Has Made in the Past
• Significantly Reduces the Gap
• Gap Will Still Be Significant and Likely Still Require Intensive Intervention
X X
Every Goal Should Be Considered by Advantages and Disadvantages
Advantage(s) of Grade 5 Disadvantage(s) of Grade 5
• Likely More Growth That Student Has Made in the Past
• Significantly Reduces the Gap and Special Education Less Likely to Be Needed
• Requires a More Intensive Intervention and “Sacrifice”
Advantage(s) of Grade 6 Disadvantage(s) of Grade 6
• Reduces the Gap and Special Education Not Likely to Be Needed
• Requires a Very Intensive Intervention and “Sacrifice”
WORDS = GRAPH OF EXPECTED RATE OF PROGRESS
AIM LINE Expected Rate of Progress to Significantly Reduce the Gap
Ginny will read aloud 95 WRC with 3 or fewer errors when given a randomly selected Grade 4 reading passage by June 1, 2013
Short-Term Goal (ROI)= 2.4 WRC
DEFINING SUCCESS (CAP)SHOULD REQUIRE THINKING
Norm-Based Approaches to CAP
Local Norms
National Norms
National Rates of Improvement (ROI)
Standards-Based Approach to CAP
Prediction of Passing Success State Standards Test
Mark’s CAP Biases
CAP Advantage Disadvantage Comment
Local Norms(LN)
What Teachers and Parents Know Best; Best Method to Judge LRE
Lack of Confidence for Use
Preferred Method
National Norms(NN)
Comfort Level If NN = LN It Doesn’t MatterIf NN> LN or NN< LN then Poor Decisions
Use LN = NN or IF You Don’t Have Alternatives
National ROI Norms
Easy to UnderstandMath Makes It Easy
Removes ThinkingMay Inadvertently Not Reduce the Gap
Use ROI to Cross Validate How Much to Expect
Standards-Based Link to High Stakes Test
Everyone’s Focus Most High Stakes Tests’ Standards Are Not High
Value-Added, And for Tier 1 Typical Students
Norm Differences
National NormsLocal Norms 33
137 153 165 107 125 139
Implications of ROI
34
I Worry that Teams May Not Think and Just Select Average Growth
Ginny will read aloud 95 WRC with 3 or fewer errors when given a randomly selected Grade 4 reading passage by June 1, 2013
Short-Term Goal (ROI)= 2.4 WRC
“ STANDARD PROTOCOL” PROGRESS MONITORING
FOR TIERS 1, 2, 3 AND SPECIAL EDUCATION ELIGIBILITY
What’s Different in Standard Protocol Progress Monitoring
No Survey Level Assessment*
Time Frame and Goal Material Are The SAME Regardless of Tier or Purpose
Time Frame: End-of School Year
Goal Material = Expected Grade
(e.g., Grade 4 student, Goal Material = Grade 4)
TIER 3 STANDARD PROTOCOL PROGRESS MONITORING
37
Like Individualized Progress Monitoring for IEPs, BUT...
Progress is Monitored in Goal Material
No SLA
Progress Monitored 1 Time Per Week
Actual Rate of Progress is Exceeding Goal
Family Time Frame Goal Material
CAP Outcome
PM Frequency
Tier 3(Standard Protocol)
End-of School-Year Expected Grade Level Significantly Reduces the Gap, Preferably LN
1 Time Per Week
TIER 3 EXAMPLE
38
Goal in Grade 4 Should Significantly Reduce the Gap
Performs at 2nd Percentile
Locally
Team Decided 2nd (57 WRC) to 15th (130 WRC) By End of the
Year
Rationale for the Goal
Advantage(s) of Performing at the 15th
Percentile
Disadvantage(s) of 15th Percentile
• Significantly Reduces the Gap and Tier 3 Less Likely to Be Needed
• Student’s Educational Needs May Be Met By Tier 2 Intervention
• Student Still Below Average and Needs Support
My Bias is that Tier Students (Less Discrepant) Are Progress Monitored LESS Frequently
Family Time Frame
Goal Material
CAP Outcome
PM Frequency
Tier 2(Standard Protocol)
End-of School-Year Expected Grade Level Reduces the Gap, Preferably LN
Benchmark Assessment Monthly or Bi-Weekly or Weekly
TIER 2 STANDARD PROTOCOL PROGRESS MONITORING
41
Progress is Monitored in Goal Material
Student Performing at
13th Percentile Locally
Goal in Grade 4 Should Significantly Reduce the Gap
Team Decided 13th (96 WRC) to 25th (140 WRC) By End of
the Year
Rationale for the Goal
Advantage(s) of Performing at the 15th
Percentile
Disadvantage(s) of 15th Percentile
• Significantly Reduces the Gap and Tier 2Less Likely to Be Needed
• Student’s Educational Needs May Be Met By Core Instruction
• Requires an Effective Tier 2 Intervention
43
Family Time Frame
Goal Material
CAP Outcome
PM Frequency
Tier 1(Standard Protocol)
End-of School-Year Expected Grade Level AYP and Passing High Stakes Tests
Benchmark Assessment
Progress is Monitored in Goal Material
Goal in Grade 2 Improve Like Other Students
Goal Should Enable AYP
Goal of 90 WRC is Improving About the Same Rate as Other Students AND Means > 80% of Passing State Test
Student Performing at 34th Percentile
Locally
Standards-Based CAPS
44
Goal Setting for Special Education
Fall Entry into Tier 3PLOP of 40 WRC = 3rd PercentileEnd-of-Year Goal of 110 WRC = 15th PercentileRationale: Demonstrate RtI Resulting in Significant Reduction of Gap and Potential Service inTier 2
RTI for Eligibility
Grade-Level Materials
After Sufficient Number of Weeks Showing Lack of Response to Appropriate Instruction
Significantly Reduce the Gap
Mark’s 5 Family SummaryFamily Time
FrameGoal
MaterialCAP
OutcomePM
FrequencyIEPs (Individualized Approach)
IEP “Anniversary” Date
Individualized to Reduce the Gap
Significantly Reduces the Gap, Preferably LN
1-2 Times Per Week
Tier 3(Standard Protocol)
End-of School-Year Expected Grade Level Significantly Reduces the Gap, Preferably LN
1 Time Per Week
Tier 2(Standard Protocol)
End-of School-Year Expected Grade Level Reduces the Gap, Preferably LN
Benchmark Assessment Monthly or Bi-Weekly or Weekly
Tier 1(Standard Protocol)
End-of School-Year Expected Grade Level AYP and Passing High Stakes Tests
Benchmark Assessment
RtI as SE Eligibility(Standard Protocol)
Time Limited to Avoid Wait to Fail
Expected Grade Level Significantly Reduces the Gap, Preferably LN
1 Time Per Week
