Measuring Charter Quality

Eric Paisner, NAPCS Anna Nicotera, NAPCS

Lyria Boast, Public Impact

Who is evaluating schools?

State Education Agencies (SEAs)Local Education Agencies (LEAs)Charter School AuthorizersCharter OrganizationsPrivate/media organizations (e.g. Great Schools, US News and World Reports)

Why measure school quality?

Hold schools accountable for resultsIdentify schools for support, intervention, or closure Inform students, parents, and communitiesProvide a consistent set of metrics that policymakers and community members can use to compare school performance

Trends

Summative ratingsMultiple measuresStudent growth modelsExpanded proficiency metricsCollege and career readiness measuresStudent and parent engagement

Multiple Measures

Summative Ratings

Summative Ratings

Multiple Measures

What is included in most rating systems?

Rating systems of all types generally include data related to five broad categories:Student GrowthProficiencySubgroup PerformanceCollege and Career Readiness (high schools)Student and Parent Engagement

Wisconsin – Sample 2011-12 School Report Card

Student GrowthStudent growth models assess how much students are learning each year.In 2011-12, 22 states used growth models to evaluate schools. The most common growth models used to evaluate schools are:

Student Growth Percentiles Value-added analysis Value tables

Student Growth

Growth models require two or more years of student-level assessment resultsIt is important to ask whether “typical” growth is “adequate” to bring students to grade level.Growth can be difficult to assess for high school students when there are no annual assessments

ProficiencyNCLB AYP designations report the percentage of students meeting or exceeding proficiency.Additional methods used to assess proficiency include:

Comparison to district or state performanceTargets for advanced proficiencyEvaluation of students at different proficiency

levels – achievement indexControls for differences in student population

A New Approach to School MeasurementAll schools will be assigned a composite index score between 1 and 100.

Measure DefinitionElementary

/ Middle Schools

High Schools

Absolute Percent Proficient

How many students have attained proficiency or better? 30 points 30 points

Progress To 2017 Target

Is the school approaching its 2017 targets? 10 points 10 points

Achievement GapsIs the school serving all students, including

those with disabilities and English Learners? 30 points 30 points

Percent of Students at Distinction Level

How many students have attained distinction? 5 points 5 points

Growth Are all students making progress? 25 points n/a

HS Graduation RatesIs the school reaching its graduation-rate

goals? n/a  20 points

High School Scaled Score Is the school improving annually?

n/a  5

TOTAL100

possible points

100 possible points

Wisconsin Composite Index

Example of Proficiency Index

The Louisiana School Performance Score (SPS) includes an index score based on how many students are in each proficiency level.

Subgroup Performance

New approaches include:

Creation of consolidated “supergroups” to avoid double-counting students that belong to more than one subgroup

Focus on lowest-performing students instead of students in demographic subgroups

Use of “achievement gap” metrics that calculate gaps between different student groups

College and Career Readiness

Availability of postsecondary data continues to improve across states Common data points include:

Extended grad rates Diploma quality Advanced coursework College readiness exams

Industry certification College remediation College attendance Dual credits

Student and Parent Engagement

Some rating systems have incorporated measures of engagement, including:

Parent and student surveysStudent retention ratesStudent attendance rates

Engagement measures can be difficult to quantify in meaningful ways.

Challenges of cross-state comparison

Absence of a national rating system Different assessments and proficiency

benchmarks in each state Range of growth models used in different

states; access to student-level data required to calculate student growth

Collection and access to college and career readiness data inconsistent across states

Discussion Topics

Is the time right for a national measure of charter school quality?What data elements should be part of a national measure of charter school quality?What are the biggest hurdles to developing and implementing a national measure of charter school quality?

Discussion