Overview of Curriculum-Based Measurement (CBM) and AIMSweb® Mark R. Shinn, Ph.D. Michelle M. Shinn,...

Overview of Overview of CurricuCurriculum-Based Measurement (CBM) lum-Based Measurement (CBM)

and AIMSweb®and AIMSweb®

Mark R. Shinn, Ph.D.

Michelle M. Shinn, Ph.D.

Lisa A. Langell, M.A., S.Psy.S.

Evaluations to Inform Teaching—Summative & Formative Assessment

SumSummative Assessmentmative Assessment: Culmination measure. Mastery assessment. Assessment after instruction. Pass/fail type assessments which summarize the knowledge students learn. Typical summative assessments include:

• End of chapter tests• High-stakes tests (e.g., State assessments)• GRE, ACT, SAT, GMAT, etc. tests• Driver’s license test• Final Exams.

FormFormative Assessment:ative Assessment: Process of assessing student achievement frequently during instruction to determine whether an instructional program is effective for individual students.InformsInforms::

• When students are progressing, continue using your instructional programs.

• When tests show that students are not progressing, you can change your instructional programs in meaningful ways.

Summative AssessmentSummative Assessment:Characterized as assessment ofof learning.

Formative AssessmentFormative Assessment: Characterized as assessment forfor learning. (Citation: http://en.wikipedia.org/wiki/Summative_assessment)

Summative Summative assessmentassessment tells you what happeneded.

Formative Formative assessmentassessment tells you

what’s happeninging.

Summative & Formative Assessment

http://en.wikipedia.org/wiki/Summative_assessment

http://en.wikipedia.org/wiki/Summative_assessment

Evaluations to Inform Teaching—Diagnostic Assessment

Diagnostic Assessments:Diagnostic Assessments: Measures that indicate specific skill strengths and those areas needing improvement. Results may indicate skill areas needing intervention/instruction. Programming may then address students’ needs.

Examples:• Criterion-referenced assessments• Cognitive assessments• Rating scales• Norm-referenced, standardized

assessments• Tests may be based on the assessment of

cognitive skills, academic skills, behavior, health, social-emotional wellbeing, etc.

http://www.sxc.hu/browse.phtml?f=download&id=866529

So, Where Does So, Where Does Curriculum-Based Measurement (CBM) fit?Curriculum-Based Measurement (CBM) fit?

• Summative?

• Formative?

• Diagnostic?

What is Curriculum-Based What is Curriculum-Based Measurement (CBM)?Measurement (CBM)?

• CBM is a formform of Curriculum-Based Assessmentof Curriculum-Based Assessment (CBA).

• CBM is the method of monitoring student progressmethod of monitoring student progress through direct, continuous assessment of basic skills.

• CBM is used to assess skills such reading, spelling, mathematics, and written language.

• CBM probes require about 1 to 4 minutesrequire about 1 to 4 minutes to complete, depending on the skill being measured.

• Student performance is scored for speed and accuracy to determine determine proficiencyproficiency.

• Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress datagiven frequently to provide continuous progress data.

• The results are charted to provide for timely evaluationcharted to provide for timely evaluation based on hard data.

Origins of CBM

• CBMCBM was initially developed more than 20 years ago by Stanley Deno and others at the University of Minnesota Institute for Research on University of Minnesota Institute for Research on Learning Disabilities Learning Disabilities to develop a reliable and valid measurement system for evaluating basic skills growth

• CBM is supported by 30 years of school-based research30 years of school-based research

• CBM is endorsed by the United States Department of EducationUnited States Department of Education as a method for assessing student progress.

• Starting in the area of reading, researchers have expanded to investigate additional academic areas over the years. This includes in-depth research and ultimately the publication of additional measures in literacy, mathematics, and written languageliteracy, mathematics, and written language

• Supporting documentation can be found in 100’s of articles, 100’s of articles, chapters, and books available within the professional literature chapters, and books available within the professional literature describing the use of CBM to make a variety of important educational decisions

Advantages of CBMAdvantages of CBM

• DirectDirect measuremeasure of student performance Deno, S.L. (1985). Curriculum-based measurement: the emerging alternative. Exceptional Children. 52(3):219-32.

• Correlates strongly with “best practices”“best practices” for instruction and assessment

• Correlates strongly with research-supported methodsresearch-supported methods for assessment and interventionassessment and intervention

• FocusFocus is on repeated measures of performancerepeated measures of performance(This cannot be done with most norm-referenced and standardized tests due to practice effect or limited forms.)

Common Characteristics of General Outcome Measures

CBM involves the same kind of evaluation technology as other CBM involves the same kind of evaluation technology as other professions.professions.

PowerfulPowerful measures that are:

• SimpleSimple

• AccurateAccurate

• EfficientEfficient indicators of performance that guide and inform a variety of decisions decisions

• GeneralizableGeneralizable thermometer that allows for reliable, valid, cross thermometer that allows for reliable, valid, cross comparisons of datacomparisons of data

General Outcome Measures (GOMs) from Other Fields

Medicine measures height, weight, temperature, and/or blood pressure.

Department of Labor measures the Consumer Price Index.

Wall Street measures the Dow-Jones Industrial Average.

Companies report earnings per share.

McDonald’s® measures how many hamburgers they sell.

CBM is Used for Scientific Reasons CBM is Used for Scientific Reasons Based on EvidenceBased on Evidence

• ReliableReliable and validvalid indicator of student achievement

• Simple, efficientSimple, efficient, and of shortshort duration to facilitate frequent

administration by teachers

• Provides assessment information that helps teachers plan helps teachers plan

better instruction better instruction

• Sensitive to improvementSensitive to improvement of students’ achievement over time

• Easily understoodEasily understood by teachers and parents

• Improves achievementImproves achievement when used to monitor progress

Things to Always Remember About CBM

• CBMs are designed to serve as ““indicatorsindicators”” of general reading achievement: CBM probes don’t measure everything, but they measure the important thingsimportant things

• Standardized testsStandardized tests to be given, scored, and interpreted in a standard waystandard way

• ResearchedResearched with respect to psychometric properties to ensure | accurate measures of learning

Items to Remember (continued)

• Are sensitivesensitive to improvement in brief intervalsbrief intervals of time

• Also can tell us howhow students earned their scores (offers opportunity to gather qualitative information)

• Designed to be asas short as possibleshort as possible (2-4min) to ensure its “do-ability”“do-ability”

• Are linkedlinked toto decision makingdecision making for promoting positive achievement and Problem-SolvingProblem-Solving

What is AIMSweb®? What is AIMSweb®?

• AIMSweb®AIMSweb® uses CBM.

• AIMSweb® is a scientifically basedscientifically based, formative assessment system that 'informs''informs' the teaching and learning process:

– Educators only have control over the quality of the instruction and the fidelity with which it is implemented.

– Take care not to automatically assume that a lack of progress is always due to lack of effectiveness of the educational programming/intervention.

– There may be times when a student’s lack of progress may be attributable to other causes, e.g., transient internal or familial stressors, physical illness, poor attendance.

– Addressing programming to include such things as:– Attendance plans– Behavior Plans– Mental health / physical health concerns

• AIMSweb® provides continuous student performance datacontinuous student performance data and reports improvement to parents, teachers, and administrators

• AIMSweb® enablesenables evidence-based evaluationevidence-based evaluation and data-driven instruction. data-driven instruction.

Three-Tiered Assessment ModelThree-Tiered Assessment Model

Tier 1:BENCHMARKBENCHMARKUniversal Screening for all students

Tier 2:STRATEGIC MONITORSTRATEGIC MONITORMonthly monitoring for students who are mild to moderate risk for failure

Tier 3: PROGRESS PROGRESS MONITORMONITORIntensive monitoring towards specific goals for students at significant risk for failure

1

2

3

Three-Tiered Assessment Model: Three-Tiered Assessment Model: BenchmarkBenchmark

Tier 1:BENCHMARKUniversal Screeningfor all students

Tier 2:STRATEGIC MONITORMonthly monitoring for students who are at mild to moderate risk for failure

Tier 3:PROGRESS MONITORIntensive monitoring towards specific goals for students at risk for failure1

2

3

1

BENCHMARK : Universal ScreeningBENCHMARK : Universal Screening

• Universal ScreeningUniversal Screening

• Identification of students at-risk for Identification of students at-risk for failure failure

• Program evaluation across all Program evaluation across all students students

• 1-4 minutes to complete 1-4 minutes to complete AIMSweb® assessments AIMSweb® assessments

• Some AIMSweb® assessments Some AIMSweb® assessments are individually administered, while are individually administered, while

others may be done as a group. others may be done as a group.

1

Measures Currently Available via AIMSweb®:Measures Currently Available via AIMSweb®:

• Early Literacy[K-1 benchmark, Progress Monitor (PM) any age]

– Letter Naming Fluency– Letter sound fluency– Phonemic Segmentation Fluency– Nonsense Word Fluency

• Early Numeracy (K-1 benchmark, PM any age)– Oral Counting– Number identification– Quantity discrimination– Missing number

• Oral Reading (K-8, PM any age)• MAZE (Reading comprehension); (1-8, PM any age)• Math Computation (1-6, PM any age)• Math Facts (PM any age)• Spelling (1-8, PM any age)• Written Expression (1-8, PM any age)

All students in an academic curriculum are “benchmarked” three times per year across any/all of these assessment areas.

Optional R-CBM Activity: Learning the Process of Benchmark Data Collection and Analysis:

Short Case Studies

67 / 2 85 / 8 74 / 9

1 min. 1 min. 1 min.1 min. 1 min. 1 min.

Why use Median vs. Average?

Averages are susceptible to outliers when dealing with small number sets.Median Score is a statistically more reliable number than average for R-CBM.

Data: Get the MEDIAN score for student’s 3 passages: CORRECT WORDS

67 / 2 85 / 8 74 / 9

1 min. 1 min. 1 min.1 min. 1 min. 1 min.

Why use Median vs. Average?

Averages are susceptible to outliers when dealing with small number sets.Median Score is a statistically more reliable number than average for R-CBM.

Data: Get the MEDIAN score for student’s 3 passages: ERRORS

MEDIAN SCORE: 74/8

(Report in AIMSweb®)NOTE: R-CBM is the only measure for which

the median score is calculated.

BENCHMARK DATA ACTIVITYBENCHMARK DATA ACTIVITY

• Your Certified AIMSweb® Trainer (CAT) will assign you a number, from 1 to 4

• Open up your workbook to:

• Calculate the Median Score for the student (number) you were assigned

WRC = Words Read CorrectERRORS = Number of Errors

• Wait for all trainees to finish the exercise. Your trainer will confirm accuracy momentarily.

WRC/ErrorsExample: 64/2

(All identifying information and scores are fictitious.)

54 2



98 9



79 12



48 12


Once data are collected, it is easily entered into AIMSweb®’s web-based software.

Simply type in the scores! (pictured at left)

Dozens of reports are then instantly available.(Estimated time: 3-5 minutes.)

Easy Score EntryEasy Score Entry


AIMSweb as a Program Evaluation Tool: AIMSweb as a Program Evaluation Tool: Benchmark 3x/year for Universal Screening—Benchmark 3x/year for Universal Screening—AllAll KidsKids


Michael Martin:Michael Martin: A student with Benchmark Data that indicates he is performing significantly behind peers at his school.

Fall Benchmark Data for Michael MartinFall Benchmark Data for Michael Martin

Martin, Michael: Grade 5


Box & Whiskers Graphs (box plots): Box & Whiskers Graphs (box plots): A 3-Step ExplanationA 3-Step Explanation

AIMSweb commonly uses box plots to report data.

AIMSweb’s Box plots are AIMSweb’s Box plots are somewhat similar in somewhat similar in shape and representation shape and representation as to a vertical bell curve.as to a vertical bell curve.

7575thth percentile percentile

Median (50Median (50thth percentile)percentile)




Above 90Above 90thth percentilepercentile

Average range of Average range of population population includedincluded

in sample.in sample.

Below Average Below Average RangeRange

Above Average Above Average RangeRange

Below 10Below 10thth percentilepercentile

Michael Martin

Target Line

**

*

**

*

**In relation to user-defined comparison group

1 3

2

Average Range (middle 50%)

AIMSweb® “Box & Whiskers GraAIMSweb® “Box & Whiskers Graphs”phs” / Comparison Groups: / Comparison Groups: Michael MartinMichael Martin

AIMSweb Comparison Group Choices/Options:

• All students in grade-level at student’s school (pictured left)

• All students in grade-level across student’s school district

• All students in grade-level across multiple districts within account

• All students in grade-level within student’s state*

• All students in grade-level nationally (Aggregate Norms)*

(Comparison group includes all students for whom data are reported using AIMSweb® web-based software)


DiscussionDiscussion: Consider Michael’s R-CBM performance in : Consider Michael’s R-CBM performance in relationship to different AIMSweb® Comparison Groupsrelationship to different AIMSweb® Comparison Groups

Grade 5:Michael’s SchoolMichael’s School

Grade 5: Michael’s DistrictMichael’s District

Grade 5: National National Aggregate NormsAggregate Norms


Discussion: SAMDiscussion: SAMPLE STUDENTPLE STUDENT

Grade 5:Luis’s SchoolLuis’s School

Grade 5: Luis’s Luis’s DistrictDistrict


AIMSweb National Aggregate Norm TableAIMSweb National Aggregate Norm Table

AIMSweb Aggregate Norm TableAIMSweb Aggregate Norm Table

AIMSweb District vs. Aggregate AIMSweb District vs. Aggregate Norm Table ComparisonNorm Table Comparison

Michael appears to be performing Michael appears to be performing below expectationsbelow expectationswhen compared to all three comparison groups.when compared to all three comparison groups.

Grade 5:Michael’s SchoolMichael’s School

Grade 5: Michael’s DistrictMichael’s District



• Consider modifying instructional program(s)

• Consider increasing frequency of assessment to assess efficacy of alternate program(s)

• Review data regularly to assess progress

An Introductory Look at Additional An Introductory Look at Additional Benchmark DataBenchmark Data

Individual Report: 33rdrd Grade Student Grade Student

QUESTIONS:

• What does report suggest about Viviana’s progress?

• What does report suggest about the school’s progress for its 3rd grade students?

• What if you saw this pattern in only one school’s 3rd grade within your district?

• What if you saw this pattern across most or all 3rd grade groups in your district?

Data to Know When Things are WorkingWorking

QUESTIONS:

• What does report suggest about Jamie Connor’s progress?

• What does report suggest about the school’s progress for 3rd grade students?

• What if you saw this pattern in only one school’s 3rd grade within your district?

• What if you saw this pattern across most or all 3rd grade groups in your district?

Data to Know that Things Went WellThings Went Well

QUESTIONS:

• What does report suggest about Heather A’s progress?

• SPRINGSPRING: Compared to Grade 3 peers at her school, is Heather performing in the: - Well Above Average Range? - Above Average Range? - Average Range? - Below Average Range? - Well-Below Average Range?

Have Data to Know When ThingsThings Need ChangingNeed Changing

QUESTIONS:

• What does report suggest about U. Boardman’s progress?

• What are possible reasons why U. Boardman might not be making progress?

• What might happen if nothing is done to address U. Boardman’s needs?

• Without this type of visual data, collected at each Benchmark period, do you believe U. Boardman’s stagnation would have been quickly noticed otherwise?

Data to Know that ChangesChanges Made a DifferenceMade a Difference

QUESTIONS:

• What does report suggest about U. Boardman’s progress by spring?

• What are possible reasons why U. Boardman might be making progress?

• What does this report suggest about the progress Grade 3 students made from winter to spring?

• Could program changes that impacted U. Boardman simultaneously positively impact the whole class?

• How could this report be used for parent conferences? Grade level team meetings? Other ideas?

Identifying Students At-Risk for FailureAt-Risk for Failure

QUESTIONS:

• What does report suggest about Lindsey’s progress (Spring)?

• What are possible reasons why Lindsey might not be making progress?

• Based on this report, is Lindsey’s instruction assisting her in closing the performance discrepancy between her school’s Grade 5 peers?

• How would this report be helpful if Lindsey were not currently receiving Tier 2 support? Tier 3? Special Education / Special Programs?

• How would this report still be helpful if Lindsey was already receiving Special Education?

Three-Tiered Assessment Model: Three-Tiered Assessment Model: Strategic MonitorStrategic Monitor

TIER 1 BENCHMARKUniversal Screening

TIER 2STRATEGIC MONITORMonthly monitoring for students who are questionable or of concern

TIER 3PROGRESS MONITORIntensive monitoring towards specific goals for at-risk students

2

AIMSweb as a Program Evaluation Tool: AIMSweb as a Program Evaluation Tool: Schools May Schools May Strategic MonitorStrategic Monitor Monthly for students at Monthly for students at Mild to ModerateMild to Moderate Risk Risk


Tier 2: Strategic MonitorTier 2: Strategic Monitor

Strategic Monitor:Strategic Monitor:(1x/month)

Provides option to increase assessment frequency from three times per year to once per month for select students.


Example of Strategic Monitor report containing monthly data collected over a full school year

Strategic Monitoring: October (Lindsey Hunter)Strategic Monitoring: October (Lindsey Hunter)

(All identifying information and scores

are fictitious.)

(All identifying information and scores

are fictitious.)

Strategic Monitoring: November (Lindsey Hunter)Strategic Monitoring: November (Lindsey Hunter)

Intensive monitoring of individualized goals for Intensive monitoring of individualized goals for students at-risk of failurestudents at-risk of failure

TIER 3: Progress Monitor

3

3

Progress Monitor (Tier 3):Progress Monitor (Tier 3): Intensive assessment with adjustable Intensive assessment with adjustable

frequency that matches needfrequency that matches need


3

BenchmarkBenchmark(Tier 1):(Tier 1): 3x per year3x per year

Strategic Monitoring Strategic Monitoring (Tier 2):(Tier 2): 1x per month for select 1x per month for select studentsstudentsat risk for educational at risk for educational difficulties difficulties

Progress MonitorProgress Monitor(Tier 3):(Tier 3): Intensive assessment with Intensive assessment with adjustable frequency that adjustable frequency that matches needmatches need


At-a-Glance Views of Student At-a-Glance Views of Student Ranking & GrowthRanking & Growth

• Follow student progress over time.

• Sort by Service Code and enhance ability to conduct differential instruction, track progress by group type, and assess improvement.

At-a-Glance Views of Student At-a-Glance Views of Student Ranking & GrowthRanking & Growth

Compare Sub-group Trends:Compare Sub-group Trends:

Measure growth of:

• General Education

• Title 1

• Special Education

• ELL/ESL

• Meal Status groups

• Compare with your custom-set targets

• View weekly growth rates by group type

Compare a School to a CompositeCompare a School to a Composite

Many More Reporting Options AvailableMany More Reporting Options Available

Finally…

AIMSweb®’s Progress Monitoring and data reporting system involves AIMSweb®’s Progress Monitoring and data reporting system involves testing using simple general, testing using simple general, RESEARCHEDRESEARCHED outcome measures. outcome measures.

It provides an It provides an ONGOINGONGOING data base that reports progress and data base that reports progress and feedback for teachers, administrators, and parents, enabling everyone feedback for teachers, administrators, and parents, enabling everyone to make decisions about the growth and development of students’ to make decisions about the growth and development of students’ basic skills.basic skills.

Your dataYour data, via AIMSweb®, is , via AIMSweb®, is professionally managedprofessionally managed by staff in a by staff in a process that communicates that process that communicates that YOU are are in chargein charge of student of student learning.learning.

The End

Date post:	26-Mar-2015
Category:	Documents
Upload:	alexander-clark
View:	222 times
Download:	0 times

Overview of Curriculum-Based Measurement (CBM) and AIMSweb® Mark R. Shinn, Ph.D. Michelle M. Shinn,...

Documents