Date post: | 28-Nov-2014 |
Category: |
Education |
Upload: | ohio-education-research-center |
View: | 287 times |
Download: | 2 times |
© 2013, Battelle for Kids. All Rights Reserved.
VALUE-ADDED INFORMATION
IN PRACTICE
Jamie Meade, Managing Director,
Strategic Measures, Battelle for Kids
December 12, 2013
© 2013, Battelle for Kids. All Rights Reserved.
Session Objectives
Provide value-added information relevant to
the practitioner’s perspective
What should teachers and principals know and
understand about SAS® EVAAS® value-added
reporting?
How may educators use SAS EVAAS value-
added information to inform practice and
impact student academic progress?
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Measures in Ohio
For more than 10 years, Battelle for Kids has
provided support for professional learning and
fostered collegial dialogue for understanding
and using value-added measures.
Battelle for Kids continues to advocate for the
use of value-added measures to inform
practices and support student learning.
© 2013, Battelle for Kids. All Rights Reserved.
Ohio’s Value-Added Models
Generated by SAS® Analytics: Customers in 135 countries
More than 65,000 business, government
and university sites
SAS customers or their affiliates
represent 90 of the top 100 companies
on the 2012 FORTUNE Global 500® list
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Information in Practice:
Building Awareness
Understanding the Difference
Achievement Measures
Progress/Growth Measures
© 2013, Battelle for Kids. All Rights Reserved.
Achievement and Growth:
Understanding the Difference
Achievement
Growth
© 2013, Battelle for Kids. All Rights Reserved.
Measuring Growth is Important for
ALL Students
Proficient
Grade
3rd
4th
5th
6th
7th
8th
© 2013, Battelle for Kids. All Rights Reserved.
Why can’t we simply compare OAA scaled scores
from one year to the next to measure growth?
Level 3rd
4th
5th
6th
7th
8th
Advanced 447-521 452-551 439-548 448-575 458-574 459-537
Accelerated 429-446 432-451 424-438 429-447 436-457 432-458
Proficient 400-428 400-431 400-423 400-428 400-435 400-431
Basic 378-399 377-399 382-399 378-399 378-399 379-399
Limited 240-377 239-376 245-381 235-377 268-377 288-378
Spring 2013 OAA Math Scaled Score Ranges
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Information in Practice:
SAS EVAAS MRM Model
Mutlivariate Response Model (MRM) Grades 4–8 Reading and Math
Ohio Achievement Assessments
© 2013, Battelle for Kids. All Rights Reserved.
SAS®
EVAAS®
Value-Added:
Two Models in Ohio
Mean Gain Approach (MRM)
4–8 Reading and Math (OAA) Reports
Predicted Mean Approach (URM)
5 & 8 Science (OAA) School and District Reports
Extended Testing
ODE RttT Mini-Grant
BFK SOAR
© 2013, Battelle for Kids. All Rights Reserved.
SAS®
EVAAS®
MRM Model*
Uses grades 3–8 Reading and Math OAAs
Compares the average growth of students in
the most recent year to the average growth of
students in 2010 (state’s baseline year).*
Growth expectation is defined as maintaining
placement in the distribution of NCE scores
from one year to the next.*
*conceptual level
© 2013, Battelle for Kids. All Rights Reserved.
Raw Score Scaled Score NCE
Sample
Raw Score
Range
Sample
Scaled Score
Range
Normal Curve
Equivalent
NCE
52
0
551
247
99
1
© 2013, Battelle for Kids. All Rights Reserved.
Scaled Scores Converted to NCEs in
2010 Baseline Year
Example
2010 Scaled Scores
Rank Ordered
551
247
Normal Curve Equivalent (NCE)
99
1
Conversion Values Are Fixed/Frozen
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Terminology
Normal Curve Equivalent (NCE)
The NCE is similar to a percentile rank in that scores are derived from scaled scores and ranked based upon performance.
A significant difference between percentile rank and NCE is that an NCE scale is an equal interval scale.
© 2013, Battelle for Kids. All Rights Reserved.
Normal Curve Equivalent (NCE)
1 10 20 30 40 50 60 70 80 90 99
1 5 10 20 30 40 50 60 70 80 90 95 99
Distribution
of Scores
Normal Curve
Equivalents
PercentileEquivalents
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Terminology
Baseline Score
Group of students’ prior year mean NCE
Example: Spring 2012 OAA mean NCE
Observed Score
Group of students’ new/most recent mean NCE
Example: Spring 2013 OAA mean NCE
© 2013, Battelle for Kids. All Rights Reserved.
Basic, Conceptual Example
Scaled scores are converted to NCEs
394 = 46 Student 1 430 = 59
402 = 50 Student 2 417 = 54
384 = 42 Student 3 400 = 49
394 = 46 Student 4 390 = 44
410 = 52 Student 5 425 = 57
Mean Baseline = 47.2
Grade 6
Baseline
Grade 7
Observed
Mean Observed = 52.6
A basic measure of the growth for this group is 5.4 NCEs
Growth = Mean Observed – Mean Baseline
Growth = 52.6 – 47.2 = 5.4 (Mean NCE Gain)
© 2013, Battelle for Kids. All Rights Reserved.
Teacher Value-Added Report
Note: Battelle for Kids is utilizing visual representations of
copyrighted EVAAS® Web reporting software from SAS in this
presentation for instructional purposes.
© 2013, Battelle for Kids. All Rights Reserved.
Statistical Terminology
Standard ErrorAll measures of student learning contain error.
Standard error is a measure of the
uncertainty.
In the EVAAS teacher value-added report, the
size of the standard error is influenced by
N size (size of the student group)
Missing scores
© 2013, Battelle for Kids. All Rights Reserved.
Value-Added Terminology
Growth Index
Since the size of the standard error (degree of
certainty) will vary across teachers, their
estimated gain must be standardized to include
both the estimate and the degree of certainty
(standard error).
Divides a teacher’s estimated gain by the
associated standard error.
© 2013, Battelle for Kids. All Rights Reserved.
Teacher Value-Added
Progress Table
© 2013, Battelle for Kids. All Rights Reserved.
Levels of Teacher Value-Added Effects
Most
Effective
Teacher's index: 2 or greater
Students are making substantially more progress than the state
growth standard.
Above
Average
Teacher's index: equal to or greater than 1, but less than 2
Students are making more progress than the state growth
standard.
Average
Teacher's index: equal to or greater than -1, but less than 1
Students are making the same amount of progress as the state
growth standard.
Approaching
Average
Teacher's index: equal to or greater than -2, but less than -1
Students are making less progress than the state growth standard.
Least
Effective
Teacher's index: less than -2
Students are making substantially less progress than the state
growth standard.
© 2013, Battelle for Kids. All Rights Reserved.
Teacher Diagnostic Graph
© 2013, Battelle for Kids. All Rights Reserved.
Key Considerations:
Systemic Programs, Delivery Models, Structures,
Services, etc.
Access to Professional Learning
Curriculum
Instruction
Assessment
Using Value-Added to Inform Practice
© 2013, Battelle for Kids. All Rights Reserved.
Link/Roster Verification Process
Ohio’s Journey:
Spring
2012
60%
of Ohio’s
LEAs
Participated
Spring
2011
30%
of Ohio’s
LEAs
Participated
Spring
2013
100%
of Ohio’s
LEAs
Participated
© 2013, Battelle for Kids. All Rights Reserved.
What is Roster Verification?
A process to capture the instructional linkage
between teachers and students.
Teachers verify three questions for each class
roster:
1. I taught these students
2. During these months
3. For __% of instruction (sole or shared)
© 2013, Battelle for Kids. All Rights Reserved.
Why Do We Link?
Student schedule files do not always capture
an accurate connection between teachers and
the students they teach.
Roster Verification provides teachers and
principals with a solution to review, modify,
and verify class rosters to ensure the
instructional linkage between teachers and
students is accurate.
@BattelleforKids
facebook.com/battelleforkidsorg
youtube.com/battelleforkids
BattelleforKids.org
Jamie Meade
Thank You!