OVERVIEW OF CURRICULUM-BASED OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL MEASUREMENT AS A GENERAL

OUTCOME MEASURE OUTCOME MEASURE

Mark R. Shinn, Ph.D.

Michelle M. Shinn, Ph.D.

Lisa A. Langell, M.A., S.Psy.S.

AIMSweb in a Picture and a Sentence

AIMSweb is a 3-tier Progress Monitoring System based on direct, frequent and continuous student assessment which is reported to students, parents, teachers and administrators via a web based data management and reporting system for the purpose of determining response to instruction.

CBM is a GOM Used for Scientific Reasons Based on Evidence

ReliableReliable and validvalid indicator of student achievement

Simple, efficientSimple, efficient, and of shortshort duration to facilitate frequent

administration by teachers

Provides assessment information that helps teachers plan better helps teachers plan better

instructioninstruction

Sensitive to the improvementSensitive to the improvement of students’ achievement over time

Easily understoodEasily understood by teachers and parents

Improves achievementImproves achievement when used to monitor progress

Things to Always Remember About CBM

Designed to serve as ““indicatorsindicators”” of general reading achievement: CBM probes don’t measure everything, but measure the importantimportant things.

Standardized testsStandardized tests to be given, scored, and interpreted in a standard standard wayway

ResearchedResearched with respect to psychometric properties to ensure accurate measures of learning

Skill Areas Currently Assessable via AIMSweb:

• Early Literacy[K-1 benchmark, Progress Monitor (PM) any age]

– Letter Naming Fluency– Letter sound fluency– Phonemic Segmentation Fluency– Nonsense Word Fluency

• Early Numeracy (K-1 benchmark, PM any age)– Oral Counting– Number identification– Quantity discrimination– Missing number

• Oral Reading (K-8, PM any age)• MAZE (Reading comprehension); (1-8, PM any age)• Math Computation (1-6, PM any age)• Math Facts (PM any age)• Spelling (1-8, PM any age)• Written Expression (1-8, PM any age)• Early Literacy and Oral Reading—Spanish (K-8)

Big Ideas of Benchmark (Tier 1) Assessment

Benchmarking allows us to add Benchmarking allows us to add systematicsystematic Formative Evaluation to Formative Evaluation to current practice.current practice.

For Teachers (and Students)For Teachers (and Students)• Early Identification of At Risk StudentsEarly Identification of At Risk Students• Instructional PlanningInstructional Planning• Progress MonitoringProgress Monitoring

For ParentsFor Parents• Opportunities for Communication/InvolvementOpportunities for Communication/Involvement• AccountabilityAccountability

For AdministratorsFor Administrators• Resource Allocation/Planning and SupportResource Allocation/Planning and Support• AccountabilityAccountability

Benchmark (Tier 1):Benchmark (Tier 1): 3x per year3x per year

Strategic Monitoring (Tier 2):Strategic Monitoring (Tier 2): 1x per month for select students1x per month for select studentsat risk for educational difficultiesat risk for educational difficulties(Optional)(Optional)

Progress Monitor (Tier 3):Progress Monitor (Tier 3): Intensive assessment with Intensive assessment with adjustable frequency that adjustable frequency that matches needmatches need

Managing Data after Assessment is Easy:

• After completing assessment, quickly type data in system.

• AIMSweb instantly generates multiple reports for analysis and various decision-making purposes.

A few of the many reports available appear here:

SSAAMMPPLLEESS

Tier 1: BenchmarkTier 1: Benchmark

Three times per year for all students.Three times per year for all students.

Fall: SeptemberFall: SeptemberWinter: JanuaryWinter: JanuarySpring: MaySpring: May

For Teachers: Classroom Report

Box & Whiskers Graphs (box plots): A Brief Explanation

AIMSweb commonly uses box plots to report data.

This chart will help familiarize yourself with box plots:

Consider bell-curve. Consider bell-curve. Box plots are somewhat Box plots are somewhat similar in shape and representation.similar in shape and representation.

7575thth percentile percentile

Median (50Median (50thth percentile) percentile)

2525thth percentile percentile

9090thth percentile percentile

1010thth percentile percentile

outlieroutlier

Average range Average range of population of population includedincluded in in

sample.sample.

Below Average Below Average RangeRange

Above Average Above Average RangeRange

Report Beginning of Year Status

Individual Report: Student

Know When Things are Working

Have Data to Know When Things Need Changing

Data to Know that Changes Made a Difference

Identifying At Risk Students

Tier 2: Tier 2: Strategic MonitorStrategic Monitor

(Monthly)(Monthly)

Provides option to increase assessment frequency for select students who have been identified as at-risk in the Benchmark process – or for all students ifdesired.

At-a-Glance Views of Student Ranking & Growth

Follow student progress over time.

Compare Sub-group Trends:

Compare a School to a Composite

Compare Yearly Improvement Progress by 50Compare Yearly Improvement Progress by 50thth Percentiles Percentiles

Compare Average Student Compare Average Student Performance by Benchmark & Across YearsPerformance by Benchmark & Across Years

View Growth Trends by Grade and Benchmark Period;View Growth Trends by Grade and Benchmark Period;

View by Comparison Group & Across YearsView by Comparison Group & Across Years

Compare Average Performance by Sub-GroupCompare Average Performance by Sub-Group

Compare by:Compare by:

• Service CodeService Code

• EthnicityEthnicity

• ELL/ESLELL/ESL

• Meal StatusMeal Status

• Various targetsVarious targets

School School Reporters may Reporters may also view data also view data

by Grade Level:by Grade Level:

School Reporters May View By Grade Level Data: School Reporters May View By Grade Level Data: Options Shown BelowOptions Shown Below

View Data by Scores View Data by Scores and Percentile Rank and Percentile Rank

per Grade Level within per Grade Level within a school or by specific a school or by specific

ClassroomClassroom

Convert Tables to Convert Tables to Chart formatsChart formats

View by:View by:

National Normative Data

National Normative Data

Small Sampling of District-Level Reports

Customer-Level Reports (Multi-District)

Compare Your School to Various Composite Groups, By Grade:Compare Your School to Various Composite Groups, By Grade:

Progress Monitoring

Strategies for Writing Individual Goals in General Curriculum and More Frequent Formative Evaluation

Mark Shinn, Ph.D.Lisa A. Langell, M.A., S.Psy.S.

Big Ideas About Frequent Formative Evaluation Using General Outcome Measures and the Progress Monitoring Program

One of the most powerful interventions that schools can use is systematic and frequent formative evaluation.

Benchmark Assessment is not enough for some students because they may be in ineffective programs too long. (3 mos +)

The solution is to write individualized goals and determine a feasible progress monitoring schedule.

The core of frequent progress monitoring is:1. Survey-Level Assessment2. Goal setting using logical educational practices3. Analysis of student need and resources for determining progress

monitoring frequency.

With Very Low Performers, Not Satisfactory to Wait This Long!

Formative Evaluation of Vital Signs Requires Quality Tools

Technical adequacy (reliability and validity);

Capacity to model growth (able to represent student achievement growth within and across academic years);

Treatment sensitivity (scores should change when students are learning);

Independence from specific instructional techniques (instructionally eclectic so the system can be used with any type of instruction or curriculum);

Capacity to inform teaching (should provide information to help teachers improve instruction);

Feasibility (must be doable).

Thinking About A Student’s Data

Sample Student:

Melissa Smart3rd grade student

Progress Monitor

8

77

92

110

34

50

Melissa Smart

Formative Evaluation—Is simply data enough?

Formative Evaluation: Is data and a goal enough?

Formative Evaluation: Are data, goals & trends enough?

Formative Evaluation is Impossible without all data:Goals Make Progress Decisions Easier

Need Shift to Few But Important Goals

Often Ineffective Goal Smorgasboard! • Student will perform spelling skills at a high 3rd grade level.• Student will alphabetize words by the second letter with 80% accuracy.• Student will read words from the Dolch Word List with 80% accuracy.• Student will master basic multiplication facts with 80% accuracy.• Student will increase reading skills by progressing through Scribner with

90% accuracy as determined by teacher-made fluency and comprehension probes by October 2006.

• To increase reading ability by 6 months to 1 year as measured by the Woodcock Johnson.

• Student will make one year's growth in reading by October 2006 as measured by the Brigance.

• Student will be a better reader.• Student will read aloud with 80% accuracy and 80% comprehension.• Student will make one year's gain in general reading from K-3.• Students will read 1 story per week.

Reduce the Number of Goals to a Few Critical Indicators

Reading In (#) weeks (Student name) will read (#) Words Correctly in 1 minute from randomly selected Grade (#) passages.

Spelling In (#) weeks (Student name) will write (#) Correct Letter Sequences and (#) Correct Words in 2 minutes from randomly selected Grade (#) spelling lists.

Math Computation In (#) weeks (Student name) will write (#) Correct Digits in 2 minutes from randomly selected Grade (#) math problems.

Written Expression In (#) weeks (Student name) will write (#) Total Words and (#) Correct Writing Sequences when presented with randomly selected Grade (#) story starters.

Conducting a Survey Level Assessment

Students are tested in successive levels of general curriculum, beginning with their current expected grade placement, until a level at which they are successful is determined.

Conducting a Survey Level Assessment

John5th grader:

5th grade passage

26/12

John4th grade passage

49/7

John3rd grade passage

62/4

Base Goal Setting on Logical Educational Practices

Example of PLEP statementExample of PLEP statement::

John currently reads about 26 words correctly from Grade 5 Standard John currently reads about 26 words correctly from Grade 5 Standard Reading Assessment Passages. He reads Grade 3 reading passages Reading Assessment Passages. He reads Grade 3 reading passages successfully; 62 correct words per minute with 4 errors, which is how successfully; 62 correct words per minute with 4 errors, which is how well beginning 3well beginning 3rdrd grade students read this material. grade students read this material.

Response To Intervention Software (RTI)

The End