Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | ingrid-kane |
View: | 23 times |
Download: | 0 times |
Linking Instructional Practices to Linking Instructional Practices to Value-added Student LearningValue-added Student Learning
Minneapolis Public SchoolsMinneapolis Public Schools
November, 2001November, 2001
www.mpls.k12.mn.us/REA/www.mpls.k12.mn.us/REA/
Minneapolis Public SchoolsMinneapolis Public SchoolsMeasuring Up Against High StandardsMeasuring Up Against High Standards MPS developed standards-based reading and
math tests - Northwest Achievement Levels Tests (NALT). All students participate Sensitive to growth Predictive of high stakes outcomes
Developed indicators in concert with stakeholders who are internal and external to the district.
Indicators of school performance are based on multiple indicators. 33 for elementary and middle schools 24 for high schools
Linking Assessments to High Linking Assessments to High Standards:Standards: Choose items for the assessment bank which Choose items for the assessment bank which
directly link to the standards. Use your directly link to the standards. Use your teacher experts, curriculum specialists, and teacher experts, curriculum specialists, and assessment personnel to create the bankassessment personnel to create the bank
““All Means ALL” - create levels of the All Means ALL” - create levels of the assessment which allow for 98% of students assessment which allow for 98% of students being assessedbeing assessed
Find out where the State benchmarks locate Find out where the State benchmarks locate on the scale (linking studies)on the scale (linking studies)
Measure continuous progress towards the Measure continuous progress towards the Standard Standard
Criteria for IndicatorsCriteria for Indicators
Meyer (1996)Meyer (1996) Indicators must measure things that matter Indicators must measure things that matter
or are valued by society (MCA, MBST).or are valued by society (MCA, MBST). Indicators must be able to avoid being Indicators must be able to avoid being
“corrupted.”“corrupted.” inclusion v. exclusion criteriainclusion v. exclusion criteria
Indicators must be able to measure the Indicators must be able to measure the unique contribution schools “add” for each unique contribution schools “add” for each measured outcome.measured outcome.
Northwest Achievement Levels Test Validity Northwest Achievement Levels Test Validity Coefficients with Minnesota State TestsCoefficients with Minnesota State Tests
Grade/Measure Grade 3ReadingMCA
Grade 3MathMCA
Grade 5ReadingMCA
Grade 5MathMCA
Grade 8ReadingMBST
Grade 8 MathMBST
Grade 2 NALTa (n/a) .80(n= 3219)
Grade 3 NALTb .87(n=3785)
.87(n=3640)
Grade 4 NALTa .85(n= 3153)
.86(n=3097)
Grade 5 NALTb .88(n=3533)
.89(n=3484)
Grade 7 NALTc .82(n= 2608)
.88(n= 2631)
aNALT April, 1998 correlation with MCA March, 1999 for same subject area (predictive validity)bNALT April, 1999 correlation with MCA March, 1999 for same subject area (concurrent validity)cNALT April, 1998 correlation with MBST February, 1999 for same subject area (predictive validity)
Linkage with State StandardsLinkage with State Standards
NALT Scale Score Which Predicts Passage of theMinnesota Basic Standards Test (MBST) in 8th Grade
Grade Reading Scale Score Reading Scale Score
3 190 1994 197 2065 204 2146 209 2187 213 2258 218 232
True gain model: Growth True gain model: Growth CurvesCurves
Class of 2005 Longitudinal Reading Gains
190
200
210
220
230
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11
Grade 12
NA
LT
Sc
ale
Sc
ore
(R
as
ch
Mo
de
l)
High Standard
Basic Standard
Class of 2005 (n=2006)
Improving Student Performance Improving Student Performance One Student at a TimeOne Student at a Time
180
190
200
210
220
230
240
250
Grade 2
Grade 4
Grade 6
Grade 8
Grade 10
Grade 12
Josh (AI)
Fahim (AA)
High Standard
Basic Standard
Correlation of Poverty and Math NALT ScoresCorrelation of Poverty and Math NALT Scores
R2 = 0.7901
30
40
50
60
70
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percent free or reduced price lunch
NA
LT M
ath
Ave
rage
NC
E
School P = 44.4 nce
Distribution of school average mathematics gain Distribution of school average mathematics gain scores on the 1999 NALT by free or reduced price scores on the 1999 NALT by free or reduced price
lunch percentages.lunch percentages.
R2 = 0.0074
-3-2-10123456789
10
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percent free or reduced price lunch
NA
LT
Mat
h N
CE
Gai
n
School P = 6.3 nce gain
Percent of Students Making About One Percent of Students Making About One Year Growth or More on NALT Math Year Growth or More on NALT Math
by Racial/Ethnic Categoryby Racial/Ethnic Category
51% 48%53% 53%
60%58% 56%
66% 63% 64%
0%
20%
40%
60%
80%
100%
AmericanIndian
AfricanAmerican
AsianAmerican
HispanicAmerican
WhiteAmerican
Per
cent
age
of S
tude
nts
2000
2001National Norm = 57%
Elementary/Middle QPA Elementary/Middle QPA 33 Indicators33 Indicators
37%
33%
6%
6%
6%
6%6%
Reading
Math
Writing
Behavior
Climate
Attendance
Programs
Special Recognition SchoolsSpecial Recognition Schools www.mpls.k12.mn.us/REA/www.mpls.k12.mn.us/REA/ Meeting Standards 2 years in a row:Meeting Standards 2 years in a row:
Armatage, Bancroft, Barton, Armatage, Bancroft, Barton, Bethune, Bryn Mawr, Bethune, Bryn Mawr, Burroughs, Dowling, Downtown Burroughs, Dowling, Downtown Open, Emerson, Ericsson, Open, Emerson, Ericsson, Field, Hale, Howe, Keewaydin, Field, Hale, Howe, Keewaydin, Kenny, Kenwood, Lake Harriet Kenny, Kenwood, Lake Harriet Upper & Lower, Lincoln, Lind, Upper & Lower, Lincoln, Lind, Longfellow, Loring, Lyndale, Longfellow, Loring, Lyndale, Marcy, Mill City, Pillsbury, Marcy, Mill City, Pillsbury, Seward, Tuttle, Waite Park, Seward, Tuttle, Waite Park, WindomWindom
Meeting Standards first yearMeeting Standards first year Cooper, Hamilton, Northrop, Cooper, Hamilton, Northrop,
Wenonah, WillardWenonah, Willard
““Beat the Odds” Beat the Odds” High ELL, PovertyHigh ELL, Poverty Bancroft, Bethune, Bryn Bancroft, Bethune, Bryn
Mawr, Cooper, Hamilton, Mawr, Cooper, Hamilton, Lincoln, Jenny Lind, Lincoln, Jenny Lind, Longfellow, Lyndale, Longfellow, Lyndale, Tuttle, WillardTuttle, Willard
Characteristics of “Needs Characteristics of “Needs Improvement Schools” in MPSImprovement Schools” in MPS One school in this category two years in One school in this category two years in
a row:a row: Student Stability 78% Student Stability 78% Staff Stability 33% (14/21 new staff)Staff Stability 33% (14/21 new staff) Three principals in three yearsThree principals in three years Low parent choice Low parent choice Higher Poverty (84%)Higher Poverty (84%) Large ELL Population (32%)Large ELL Population (32%)
Characteristics of Characteristics of “Distinguished School” in MPS“Distinguished School” in MPS One school in this category two years in One school in this category two years in
a row:a row: Student Stability 94% Student Stability 94% Staff Stability 87% (3/23 new staff)Staff Stability 87% (3/23 new staff) One principals in three yearsOne principals in three years High parent choice (waiting list)High parent choice (waiting list) Lower Poverty (40%)Lower Poverty (40%) Moderate ELL Population (20%)Moderate ELL Population (20%)
Structured interviews with 3 Structured interviews with 3 Schools that “Beat the Odds”Schools that “Beat the Odds” Process: Interviewed 16 teachers at 3 sites (Lincoln, Lyndale,
Sanford). The teachers were selected by the principal of each site and represented teachers early in their careers, mid-career teachers and veterans.
Key Factors: Staff Stability Administrative Cohesion Unity of purpose Collective Decisionmaking School investment in instruction (mentors, reduced class size) Administrator as instructional leader (principal observations) Direct Instruction/ Mastery Learning Extended class day
Value-added ModelValue-added Model Primary question is thisPrimary question is this, “, “How can an equitable How can an equitable
accountability system be put in place when there is a accountability system be put in place when there is a large degree of variability in the way student large degree of variability in the way student characteristics are distributed among our schoolscharacteristics are distributed among our schools?”?” English language learnersEnglish language learners Special education programsSpecial education programs Race/ethnicityRace/ethnicity PovertyPoverty
MPS relies on an empirical- and literature-based MPS relies on an empirical- and literature-based model to make predictions, not set expectations! model to make predictions, not set expectations!
Schools are acknowledged/rewarded for “Schools are acknowledged/rewarded for “Beating Beating the Oddsthe Odds.”.”
Value-added Teacher EffectsValue-added Teacher Effects
Post-test reading score=Post-test reading score= Pretest reading scorePretest reading score + Free or reduced price lunch+ Free or reduced price lunch + Racial/ethnic code+ Racial/ethnic code + Neighborhood poverty concentration+ Neighborhood poverty concentration + Lives with both parents code+ Lives with both parents code + Limited English Proficiency status+ Limited English Proficiency status + Special Education status+ Special Education status + Teacher effects+ Teacher effects
Teachers who “beat the odds” in Teachers who “beat the odds” in second grade reading reported: second grade reading reported:
more use of small group instructionmore use of small group instruction more development of word attack skillsmore development of word attack skills more individual student oral readingmore individual student oral reading more guidance during initial practicemore guidance during initial practice more explicit and direct phonics more explicit and direct phonics
instructioninstruction more use of systematic motivation more use of systematic motivation
strategiesstrategies
Current Study: Value-added Math Current Study: Value-added Math and Reading by Grade by School for and Reading by Grade by School for
Students of ColorStudents of ColorEarly Reading (k-3) Reading 3-6 Reading 6-8 Groups Early Math (k-3) Math 3-6 Math 6-8
BARTON X X AA, Hisp X XBETHUNE X AA
BURROUGHS X X AA, Hisp X XDOWLING X AA XDOWNTOWN OPEN AA XEMERSON X Hisp X XHALE X AA,Asian XHOWE X AA X XJEFFERSON X Hisp
KENWOOD Asian XLINCOLN X AA X XLONGFELLOW X AA,AI X XLORING X X AI, Asian
LYNDALE X X AA, Hisp, Asian X XMARCY X AA XOLSON X AA
PILLSBURY AA,AI XSANFORD X AA
SEWARD X AI XSHERIDAN X X AI
WENONAH X AA
Measurement RecommendationsMeasurement Recommendations
Measure students on standards-based Measure students on standards-based assessments as soon as they enter schoolassessments as soon as they enter school
Continuously monitor student progress Continuously monitor student progress towards the standardstowards the standards
Produce individual student growth curvesProduce individual student growth curves Use data to monitor the effectiveness of Use data to monitor the effectiveness of
interventionsinterventions Identify schools and teachers who make Identify schools and teachers who make
exceptional progress with “at risk” studentsexceptional progress with “at risk” students
Intervention RecommendationsIntervention Recommendations Involve exceptional teachers in staff development Involve exceptional teachers in staff development
(modeling,video tape, etc.)(modeling,video tape, etc.) Give incentives for exceptional teachers to work in Give incentives for exceptional teachers to work in
the highest need schoolsthe highest need schools Place staff development in the schools with “hands Place staff development in the schools with “hands
on” modeling and supporton” modeling and support Identify teaching practices that correlate with Identify teaching practices that correlate with
success and replicate themsuccess and replicate them Reward groups of teachers (e.g. schools) who Reward groups of teachers (e.g. schools) who
“beat the odds”“beat the odds”