Response to Intervention | RTI | RTI Resources | Intervention … · 2009-03-17 · progress on...

RTI Toolkit: A Practical Guide for Schools

‘But Is This Student Benefiting From Special Education Services?’: Reliable Methods to Measure Student Academic Progress Jim Wright, Presenter

17 March 2009 Technical Assistance Meeting for Committee on Special Education Chairpersons Crowne Plaza Resort Lake Placid, NY

Jim Wright 364 Long Road Tully, NY 13159 Email: [email protected] Website: www.interventioncentral.org

Response to Intervention

www.interventioncentral.org

Finding RTI-Ready Measures toAssess and Track Student Academic Skills

Jim Wrightwww.interventioncentral.org



Monitoring Student Academics: Curriculum-Based Measurement


www.interventioncentral.org 3

Source: National Research Council. (2002). Helping children learn mathematics. Mathematics Learning Study Committee, J. Kilpatrick & J. Swafford, Editors, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

5 Strands of Mathematical Proficiency

1. Understanding

2. Computing

3. Applying

4. Reasoning

5. Engagement

5 Big Ideas in Beginning Reading

1. Phonemic Awareness

2. Alphabetic Principle

3. Fluency with Text

4. Vocabulary

5. ComprehensionSource: Big ideas in beginning reading. University of Oregon. Retrieved September 23, 2007, from http://reading.uoregon.edu/index.php

Models in Reading & MathResponse to Intervention


Curriculum-Based Evaluation: Definition

“Whereas standardized commercial achievement tests measure broad curriculum areas and/or skills, CBE measures specific skills that are presently being taught in the classroom, usually in basic skills. Several approaches to CBE have been developed. Four common characteristics exist across these models:

• The measurement procedures assess students directly using the materials in which they are being instructed. This involves sampling items from the curriculum.

• Administration of each measure is generally brief in duration (typically 1-5 mins.)

• The design is structured such that frequent and repeated measurement is possible and measures are sensitive to change.

• Data are usually displayed graphically to allow monitoring of student performance.”

SOURCE: CAST Website: http://www.cast.org/publications/ncac/ncac_curriculumbe.html


www.interventioncentral.orgSOURCE: CAST Website: http://www.cast.org/publications/ncac/ncac_curriculumbe.html



Curriculum-Based Measurement: Advantages as a Set of Tools to Monitor RTI/Academic Cases

• Aligns with curriculum-goals and materials• Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific performance levels for specific

tasks• Uses standard procedures to prepare materials, administer, and

score• Samples student performance to give objective, observable ‘low-

inference’ information about student performance • Has decision rules to help educators to interpret student data and

make appropriate instructional decisions• Is efficient to implement in schools (e.g., training can be done quickly;

the measures are brief and feasible for classrooms, etc.)• Provides data that can be converted into visual displays for ease of

communicationSource: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.

Jim Wright, Presenter www.interventioncentral.org 2



Standards for Judging Academic Measures for RTI (National Center for Student Progress Monitoring)



RTI Literacy: Assessment & Progress-Monitoring (Cont.)

To measure student ‘response to instruction/intervention’ effectively, the RTI Literacy model measures students’ reading performance and progress on schedules matched to each student’s risk profile and intervention Tier membership.

• Benchmarking/Universal Screening. All children in a grade level are assessed at least 3 times per year on a common collection of literacy assessments.

• Strategic Monitoring. Students placed in Tier 2 (supplemental) reading groups are assessed 1-2 times per month to gauge their progress with this intervention.

• Intensive Monitoring. Students who participate in an intensive, individualized Tier 3 reading intervention are assessed at least once per week.

Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge.



Example of Curriculum-Based Assessment Reading Probe



DIBELS Reading Probe: Benchmark 2.1



57 WPM



Assessing Basic Academic Skills: Curriculum-Based Measurement

Reading: These 3 measures all proved ‘adequate predictors’ of student performance on reading content tasks:– Reading aloud (Oral Reading Fluency): Passages

from content-area tests: 1 minute.– Maze task (every 7th item replaced with multiple

choice/answer plus 2 distracters): Passages from content-area texts: 2 minutes.

– Vocabulary matching: 10 vocabulary items and 12 definitions (including 2 distracters): 10 minutes.

Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.





Mathematics: Single-skill basic arithmetic combinations an ‘adequate measure of performance’for low-achieving middle school students.Websites to create CBM math computation probes:

Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.

•www.interventioncentral.org

•www.superkids.com




Writing: CBM/ Word Sequence is a ‘valid indicator of general writing proficiency’. It evaluates units of writing and their relation to one another. Successive pairs of ‘writing units’make up each word sequence. The mechanics and conventions of each word sequence must be correct for the student to receive credit for that sequence. CBM/ Word Sequence is the most comprehensive CBM writing measure.Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.




Interpreting Data: The Power of Visual Display



Creating CBM Monitoring Charts



Sample Peer Tutoring Chart Response to Intervention


Sample Peer Tutoring Chart



Single-Subject (Applied) Research Designs“Single-case designs evolved because of the need to understand patterns of individual behavior in response to independent variables, and more practically, to examine intervention effectiveness. Design use can be flexible, described as a process of response-guided experimentation…, providing a mechanism for documenting attempts to live up to legal mandates for students who are not responding to routine instructional methods.” p. 71

Source: Barnett, D. W., Daly, E. J., Jones, K. M., & Lentz, F.E. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38, 66-79.



Single-Subject (Applied) Research Designs: Steps“The basic methods [of single-case designs] are• selecting socially important variables as dependent measures or target

behaviors• taking repeated measures until stable patterns emerge so that participants

may serve as their own controls (i.e., baseline)• implementing a well-described intervention or discrete intervention trials• continuing measurement of both the dependent and independent

variables within an acceptable pattern of intervention application and/or withdrawal to detect changes in behavior and make efficacy attributions

• graphically analyzing the results to enable ongoing comparisons of the student’s performance under baseline and intervention conditions, and

• replicating the results to reach the ultimate goal of the dissemination of effective practices.”

Source: Barnett, D. W., Daly, E. J., Jones, K. M., & Lentz, F.E. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38, 66-79.




Formative Assessment: Donald: Grade 3Response to Intervention


Formative Assessment: Donald: Grade 3



IEP Goal Statements for CBA/CBM



Writing CBM Goals in Student IEPs (Wright, 1992)

Source: Wright, J. (1992). Curriculum-based measurement: A manual for teachers. Retrieved on September 4, 2008, from http://www.jimwrightonline.com/pdfdocs/cbaManual.pdf












IEP Goals for CBA/CBM: READING

Reading In [number of weeks until Annual Review], when given a randomly selected passage from [level and name of reading series] for 1 minute

Student will read aloud

At [number] correctly read words with no more than [number]decoding errors.



IEP Goals for CBA/CBM: Written Expression

Written Expression

In [number of weeks until Annual Review], when given a story starter or topic sentence and 3 minutes in which to write

Student will write

A total of:[number] of words or[number] of correctly spelled wordsor[number] of correct word/writing sequences



IEP Goals for CBA/CBM: Spelling

Spelling In [number of weeks until Annual Review], when dictated randomly selected words from [level and name of spelling series ordescription of spelling word list] for 2 minutes

Student will write

[Number of correct letter sequences]



Formative Assessment: Essential Questions…5. How does the school check up on progress

toward the goal(s)?The school periodically checks the formative assessment data to determine whether the goal is being attained. Examples of this progress evaluation process include the following:

– System-Wide: A school-wide team meets on a monthly basis to review the frequency and type of office disciplinary referrals to judge whether those referrals have dropped below the acceptable threshold for student behavior.

– Group Level: Teachers at a grade level assembles every six weeks to review CBM data on students receiving small-group supplemental instruction to determine whether students are ready to exit (Burns & Gibbons, 2008).

– Individual Level: A building problem-solving team gathers every eight weeks to review CBM data to a student’s response to an intensive reading fluency plan.

Sources: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge.

Shinn, M. R. (1989). Curriculum-based measurement: Assessing special children. New York: Guilford.


RTI-Ready Methods to Monitor Student Academics

Reading: Phonemic Awareness / Alphabetics

Initial Sound Fluency : 3 minutes Administration: 1:1 Description: The student is shown 4 pictures, each depicting an object that begins with a different letter sound. The examiner gives the student a letter sound and asks the student to select the picture of the object that begins with that letter sound. The process is repeated with new sets of pictures until the time Where to get materials: DIBELS https://dibels.uoregon.edu/

Phoneme Segmentation Fluency 1 minute Administration: 1:1

Description: The student is read a list of words that contain from 2 to five phonemes. For each word, the student is asked to recite all of the phonemes that make up the word. Where to get materials: DIBELS https://dibels.uoregon.edu/

Nonsense Word Fluency 1 minute Administration: 1:1 Description: The student is shown a list of nonsense words of 2 to 3 letters in length. For each word, the student is to read the word or give the sounds that make up the word. Where to get materials: DIBELS https://dibels.uoregon.edu/

Letter Naming Fluency 1 minute Administration: 1:1 Description: The student is presented with a list of randomly arranged letters. The student names as many letters as possible. Where to get materials: DIBELS https://dibels.uoregon.edu/

Letter Sound Fluency 1 minute Administration: 1:1 Description: The student is presented with a list of randomly arranged letters. The student gives the sounds of as many letters as possible. Where to get materials: www.interventioncentral.org

Word Identification Fluency 1 minute Administration: 1:1 Description: The student is presented with a list of words randomly selected from a larger word list (e.g., Dolch Wordlist). The student reads as many words as possible. Where to get materials: • Easy CBM http://www.easycbm.com • Intervention Central http://www.interventioncentral.org (Dolch wordlists)


Reading: Fluency

Oral Reading Fluency : 1 minute Administration: 1:1 Description: The student reads aloud from a passage and is scored for fluency and accuracy. Passages are controlled for level of reading difficulty. Where to get materials: • DIBELS https://dibels.uoregon.edu/ • AimsWeb http://www.aimsweb.com/ • Easy CBM http://www.easycbm.com • Intervention Central http://www.interventioncentral.org (Use the OKAPI page to create

customized ORF passages)

Reading: Basic Comprehension

Maze Passages : 1-3 minutes Administration: Group Description: The student is given a passage in which every 7th word has been removed. The student reads the passage silently. Each time the student comes to a removed word, the student chooses from among 3 replacement words: the correct word and two distractors. The student circles the replacement word that he or she believes best restores the meaning of the text. Where to get materials: • AimsWeb http://www.aimsweb.com/ • Intervention Central http://www.interventioncentral.org (Use the Maze Passage Generator

page to create customized Maze passages)

Multiple-Choice Reading Comprehension

: Unknown Administration: Group

Description: The student is given a passage to read. The student then answers a series of standardized comprehension questions based on the text. Where to get materials: • Easy CBM http://www.easycbm.com

Math: Early Math Fluency

Quantity Discrimination Fluency : 1 minute Administration: 1:1 Description: The student is given a sheet with number pairs. For each number pair, the student must name the larger of the two numbers. Where to get materials: • AimsWeb http://www.aimsweb.com/ • Intervention Central http://www.interventioncentral.org (Numberfly Early Math Fluency Probe

Creator)


Missing Number Fluency : 1 minute Administration: 1:1 Description: The student is given a sheet containing numerous sets of 3 or 4 sequential numbers. For each number series, one of the numbers is missing. The student must name the missing number. Where to get materials: • AimsWeb http://www.aimsweb.com/ • Intervention Central http://www.interventioncentral.org (Numberfly Early Math Fluency Probe

Creator)

Number Identification Fluency : 1 minute Administration: 1:1 Description: The student is given a sheet with numbers in random order. The student gives the name of each number. Where to get materials: • AimsWeb http://www.aimsweb.com/ • Intervention Central http://www.interventioncentral.org (Numberfly Early Math Fluency Probe

Creator)

Oral Counting Fluency : 1 minute Administration: 1:1 Description: The student counts aloud as many words in sequence as possible, starting from zero or one. Where to get materials: • The student does not require materials for this assessment. The examiner can make a sheet

with numbers listed sequentially from 0-100 to record those numbers that the student can recite in sequence.

Math: Computation

Math Computation Fluency : 2 minutes Administration: Group Description: The student is given a worksheet with single-skill or mixed-skill math computation problems. The student works independently to complete as many problems as possible. The student receives credit for each correct digit appearing in his or her answer. Where to get materials: • AimsWeb http://www.aimsweb.com/ • Intervention Central http://www.interventioncentral.org (Math Worksheet Generator) • SuperKids http://www.superkids.com/aweb/tools/math/ (This website allows you to create

math computation worksheets for more advanced areas such as fractions, percentages, decimals, and more)


Math: Applied Problems Math Concepts & Applications : 6-8 minutes Administration: Group

Description: Students are given assessment booklets with a mix of applied problem types appropriate to that grade level. (Assessments are available for grades 2-6). A mix of applied problems is included in each assessment, sampling the typical math curriculum for the student’s grade (e.g., money skills, time-telling, etc.) Where to get materials: • MBSP: Monitoring Basic Skills Progress: Basic Math Kit – Second Edition developed by Drs.

Lynn & Dough Fuchs, Vanderbilt University. Available through Pro-Ed: http://www.proedinc.com/

Math: Vocabulary Math Vocabulary Probes (Howell,

2008) : 5 minutes Administration: Group

Description: Students are given a math vocabulary probe consisting of 20 vocabulary items. There are two versions commonly used: (1) The sheet contains vocabulary terms on one side of the sheet and the definitions of those terms—in scrambled order—on the other. The student connects term to its correct definition; (2) The sheet contains only definitions. The student must read each definition and write the correct corresponding vocabulary term. Where to get materials: • Math vocabulary probes are developed by the school. Teachers create ‘vocabulary pools’ that

contain the key vocabulary items to be included in probes. From that larger pool, vocabulary items are randomly sampled to create individual probes.

Writing CBM Writing : 4.5 minutes Administration: Group

Description: The student is given a worksheet with a ‘story starter’ (an introductory sentence that provides the student with a topic on which to write). The student is given 60 to 90 seconds to plan a story and 3 minutes to produce a writing sample. CBM Writing probes can be scored for total words written, number of correctly spelled words, and number of ‘correct writing sequences’ (a more comprehensive measure of the student’s mastery of the mechanics and conventions of writing). Where to get materials: • Many schools make their own story starters, which are simple to produce. • AimsWeb http://www.aimsweb.com/ References Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford Howell, K. W. (2008). Best practices in curriculum-based evaluation and advanced reading. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 397-418). Bethesda, MD: National Association of School Psychologists.


Evaluate the ‘RTI Readiness’ of Your School’s Academic Measures Directions. Use the questionnaire below to evaluate the ‘RTI readiness’’ of any academic measure. Note that questions on the form are hierarchically organized: If items earlier in the survey are endorsed ‘no’, the measure probably cannot be used for more advanced applications that appear later in the survey. Use the table Interpreting the Results of This Survey below to identify the appropriate uses for your measure in the RTI problem-solving process..

_ Item # Rating Item YES NO

Background: Validity. 1. Content Validity. Does the measure provide meaningful information about the

academic skill of interest? Y N

2. Convergent Validity. Does the measure yield results that are generally consistent with other well-regarded tests designed to measure the same academic skill?

Y N

3. Predictive Validity. Does the measure predict student success on an important future test, task, or other outcome?

Y N

Background

Baseline: Reliability. 4. Test-Retest/Alternate-Form Reliability. Does the measure have more than one

version or form? If two alternate, functionally equivalent versions of the measure are administered to the student, does the student perform about the same on both?

Y N

5. Interrater Reliability. When two different evaluators observe the same student’s performance and independently use the measure to rate that performance, do they come up with similar ratings?

Y N

Baseline

Benchmarks & Goal-Setting 6. Performance Benchmarks. Does the measure include benchmarks or other

performance criteria that indicate typical or expected student performance in the academic skill?

Y N

7. Goal-Setting. Does the measure include guidelines for setting specific goals for improvement?

Y N

Goal-Setting

Progress-Monitoring and Instructional Impact 8. Repeated Assessments. Does the measure have sufficient alternative forms to

assess the student weekly for at least 20 weeks? Y N

9. Equivalent Alternate Forms. Are the measure’s repeated assessments (alternative forms) equivalent in content and level of difficulty?

Y N

10. Sensitive to Short-Term Student Gains. Is the measure sensitive to short-term improvements in student academic performance?

Y N

11. Positive Impact on Learning. Does research show that the measure gives teachers information that helps them to make instructional decisions that positively impact student learning?

Y N

Progress-Monitoring

Jim

Interpreting the Results of This Survey of Your Academic Measure: • YES to Items 1-3. Background. The measure gives valid general information about the student’s academic skills

and performance. While not sufficient, the data can be interpreted as part of a larger collection of student data. • YES to Items 4-5. Baseline. The measure gives reliable results when given by different people and at different

times of the day or week. Therefore, the measure can be used to collect a current ‘snapshot’ of the student’s academic skills prior to starting an intervention.

• YES to Items 6-7. Goal-Setting. The measure includes standards (e.g., benchmarks or performance criteria) for ‘typical’ student performance (e.g., at a given grade level) and guidelines for estimating rates of student progress. Schools can use the measure to assess the gap in performance between a student and grade level peers—and also to estimate expected rates of student progress during an intervention.

• YES to Items 8-11. Progress Monitoring. The measure has the appropriate qualities to be used to track student progress in response to an intervention.

Name of Measure: ________________________________________________________________________________

Wright, Presenter www.interventioncentral.org 12

Comparing Reading Measures for ‘RTI Readiness’

Background: Validity Baseline: Reliability Goal-Setting Progress-Monitoring

Content Validity.

Convergent Validity

Predictive Validity

Test-Retest/

Alternate Form Reliability

Interrater Reliability

Performance Benchmarks

Goal-Setting

Repeated Assessments

Equivalent Alternate

Forms

Sensitive to Short-Term

Student Gains

Positive

Impact on Learning

Name of Measure 1 2 3 4 5 6 7 8 9 10 11

School: _________________________________ Date: _______________ Person(s) Completing Ratings: __________________________________________

Phonemic Awareness/Alphabetics Fluency With Text Vocabulary Comprehension

Directions: Use this form to compare reading measures in your school for qualities of ‘RTI readiness’. Put an ‘X’ in a column if the measure has that measurement quality. (Consult the form Evaluate the ‘RTI Readiness’ of Your School’s Academic Measures for a more detailed description of each measurement quality.)


Comparing Math Measures for ‘RTI Readiness’

School: _________________________________ Date: _______________ Person(s) Completing Ratings: __________________________________________

Understanding Computing Applying Reasoning Engagement

Directions: Select one of the proficiency strands (NRC, 2002) of math assessment above. Use this form to compare math measures in your school for qualities of ‘RTI readiness’. Put an ‘X’ in a column if the measure has that measurement quality. (Consult the form Evaluate the ‘RTI Readiness’ of Your School’s Academic Measures for a more detailed description of each measurement quality.)

Background: Validity Baseline: Reliability Goal-Setting Progress-Monitoring

Content Validity.

Convergent Validity

Predictive Validity

Test-Retest/

Alternate Form Reliability

Interrater Reliability

Performance Benchmarks

Goal-Setting

Repeated Assessments

Equivalent Alternate

Forms

Sensitive to Short-Term

Student Gains

Positive

Impact on Learning

Name of Measure 1 2 3 4 5 6 7 8 9 10 11

National Research Council. (2002). Helping children learn mathematics. Mathematics Learning Study Committee, J. Kilpatrick & J. Swafford, Editors, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.


Date post:	03-Aug-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Response to Intervention | RTI | RTI Resources | Intervention … · 2009-03-17 · progress on...

Documents