Overview of Progress Monitoring Training Session

Part of a training series developed to accompany the Part of a training series developed to accompany the AIMSweb AIMSweb Improvement SystemImprovement System..

Purpose is to provide the background information and data collection Purpose is to provide the background information and data collection procedures necessary to organize and conduct frequent, direct and procedures necessary to organize and conduct frequent, direct and continuous measurement of students’ skill improvement.continuous measurement of students’ skill improvement.

Designed to accompany: Designed to accompany:

• Administering and Scoring of Reading Curriculum-Based Measurement for Use in General Outcome Measurement Training Workbook.

• AIMSweb Progress Monitor - Strategies for Writing Individualized Goals in General Curriculum and More Frequent Formative Evaluation

• Standard Progress Monitoring Reading Assessment Passages.

• AIMSweb Progress Monitor Web-Based Software.

• Training Video.

Big Ideas About Frequent Formative Evaluation Using General Outcome Measures and the Progress Monitoring Program

One of the most powerful interventions that schools can use is systematic and frequent formative evaluation.

Benchmark Assessment is not enough for some students because they may be in ineffective programs too long.

The solution is to write individualized goals and determine a feasible progress monitoring schedule.

The core of frequent progress monitoring is Survey-Level Assessment, goal setting using logical educational practices, and an analysis of student need and resources for determining progress monitoring frequency.

Formative Assessment

Process of assessing student achievement Process of assessing student achievement duringduring instruction to instruction to determine whether an instructional program is effective for individual determine whether an instructional program is effective for individual students.students.

• When students are progressing, keep using your instructional

programs.

• When tests show that students are not progressing, you can change your instructional programs in meaningful ways.

• Has been linked to important gains in student achievement (L. Fuchs, 1986) with effect sizes of .7 and greater.

Systematic formative evaluation requires the use of:

Standard assessment tools…

That are the same difficulty and that are…

Given the same way each time.

More Severe Achievement Problems and/or More Resource Intensive Programs Require More Frequent Formative Evaluation

Benchmark Testing (3 - 4 x Per Year) is not enough for some students.

With Very Low Performers, Not Satisfactory to Wait This Long!

Programs That are More Resource Intensive…

Title I, English Language Learning, Special Education

Should monitor student outcomes more frequently than the Benchmark Testing schedule.

Formative Evaluation of Vital Signs Requires Quality Tools

Technical adequacy (reliability and validity);

Capacity to model growth (able to represent student achievement growth within and across academic years);

Treatment sensitivity (scores should change when students are learning);

Independence from specific instructional techniques (instructionally eclectic so the system can be used with any type of instruction or curriculum);

Capacity to inform teaching (should provide information to help teachers improve instruction);

Feasibility (must be doable).

Formative Evaluation is Impossible without Clear Goals

Goals Make Progress Decisions Easier

Current Goal Setting Practices Are Unsatisfying

Do you like these IEPs?I do not like these IEPsI do not like them Jeeze LouiseWe test, we checkWe plan, we meetBut nothing ever seems complete.Would you, could youLike the form?I do not like the form I seeNot page 1, not 2, not 3Another changeA brand new boxI think we allHave lost our rocks!

Improving the Process of Setting Goals for Formative Evaluation

Set a few, but important goals.

Ensure goals are measurable and linked to validated formative evaluation practices.

Base goal setting on logical educational practices.

Few But Important Goals

Goal Smorgasboard! • Student will perform spelling skills at a high 3rd grade level.• Student will alphabetize words by the second letter with 80% accuracy.• Student will read words from the Dolch Word List with 80% accuracy.• Student will master basic multiplication facts with 80% accuracy.• Student will increase reading skills by progressing through Scribner with

90% accuracy as determined by teacher-made fluency and comprehension probes by October 199x.

• To increase reading ability by 6 months to 1 year as measured by the Woodcock Johnson.

• Student will make one year's growth in reading by October 1990 as measured by the Brigance.

• Student will be a better reader.• Student will read aloud with 80% accuracy and 80% comprehension.• Student will make one year's gain in general reading from K-3.• Students will read 1 story per week.

Improving the Process of Setting Goals for Formative Evaluation

Set a few, but important goals.

Ensure goals are measurable and linked to validated formative evaluation practices.

Base goal setting on logical educational practices.

Reduce the Number of Goals to a Few Critical Indicators

Reading In (#) weeks (Student name) will read (#) Words Correctly in 1 minute from randomly selected Grade (#) passages.

Spelling In (#) weeks (Student name) will write (#) Correct Letter Sequences and (#) Correct Words in 2 minutes from randomly selected Grade (#) spelling lists.

Math Computation In (#) weeks (Student name) will write (#) Correct Digits in 2 minutes from randomly selected Grade (#) math problems.

Written Expression In (#) weeks (Student name) will write (#) Total Words and (#) Correct Writing Sequences when presented with randomly selected Grade (#) story starters.

Ensure the Goals are Measurable and Linked to Validated Formative Evaluation Practices

Goals should be based on quality tests like CBM.

Based on validated practices such as how often, how many samples, etc.

Base Goal Setting on Logical Educational Practices

Leonardo currently reads about 15 words correctly from Grade 5 Leonardo currently reads about 15 words correctly from Grade 5 Standard Reading Assessment Passages. He reads Grade 1reading Standard Reading Assessment Passages. He reads Grade 1reading passages successfully; 68 correct words per minute, which is how passages successfully; 68 correct words per minute, which is how well end of first grade students read this material.well end of first grade students read this material.

Goal Setting Strategies

Current Performance Information based on Survey-Level Assessment (SLA).

Know the Time Frame for the Goal.

Determine a Future Performance Level.

Conducting a Survey Level Assessment

Students are tested in successive levels of general curriculum, beginning with their current expected grade placement, until a level at which they are successful is determined.

Reading Survey-Level Assessment for Kate, A Third-Grader

Grade ReadingAssessmentPassages

Passage 1(WRC/E)

Passage 2(WRC/E)

Passage 3(WRC/E)

Median(WRC/E)

FallPerformance

Level

3 35/6 37/5 50/8 37/6 At RiskReader

2 60/4 58/3 42/7 58/4 AverageReader

Setting the Time Frame, Goal Level Material, and Criterion

Time Frame

End of Year (At Risk or Grade-Level Expectations)

In 18 Weeks…

Annual IEP Goals (Special Education)

In 1 year… (or) In 32 Weeks…

Setting the Goal Material

Logical Task--• Matching or Not Matching Expected Grade Placement

• Title 4 Fourth Grader--Grade 4 Material?

• Grade 4 Special Education Student--Grade 4 Material?

When Grade-Level Expectations Are Not Appropriate

Consider the Severity of the Discrepancy

Consider the Intensity of the Program

Determining the Criterion for Success

Local Benchmark Standards.

Linkage to High Stakes Tests.

Normative Growth Rates.

Developing Your Own Sample of Standards.

Benchmark Standards Advantages and Disadvantages

Advantages Disadvantages

Easily Understood Uncomfortable, Especially in Low Achievement Environments

Can Indicate When Student No Longer Needs Specialized Instruction

Issues of “Equity”

Determining Least Restrictive Environment (LRE)

Linkage to High Stakes Standards Advantages and Disadvantages

Advantages Disadvantages

Reduces Problems of Equity when Local Achievement is Low

Need Linkage to High Quality High Stakes Test

Increases Usefulness of High Stakes Tests

Linkage Must Be Established Empirically

Helps Specify Long-Term Targets (What Grade 2 Student Needs to Read to Be Successful on Grade 6 Test)

Adoption of Assumption that the Attainment of the Target Guarantees Passing High Stakes Test

Normative Growth Rates

Criterion for Success = Score on SLA + (Grade Growth Rate times # of Weeks)

Score on SLA (30)+ (Ambitious Grade Growth Rate (2.0) times # of Weeks (32)

Or

30 + (2.0 * 32) or 30 + 64 = Annual goal of 94 WRC

Growth Rate Standards Advantages and Disadvantages

Advantages Disadvantages

Easily Understood May Underestimate What Can Be Attained with High Quality Instruction

Developing Your Own Sample of Standards

Developing a Small Local Norm

Benchmark Standards Advantages and Disadvantages

Advantages Disadvantages

Same Advantages as Benchmark Standards

Same Disadvantages of Benchmark Standards

Small Sample Size

How Frequently to Assess?

Balancing IDEAL with FEASIBLE

The End