Research and Public Policy

Evans School of Public Affairs

April 30, 2013

Annie Pennucci

Associate DirectorWashington State Institute for Public Policy

www.wsipp.wa.gov

UpcomingExamplesThe ResearchBackground

Washington State Institute for Public PolicyWashington State Institute for Public Policy

Created by the 1983 Legislature

Mission: carry out non–partisan research on projects assigned by the Legislature or the Institute’s Board of Directors

Senator Mike CarrellSenator Karen FraserSenator Jeanne Kohl-Welles Representative Cary CondottaHouse Republican VacancyHouse Democratic VacancyKen Conte, House StaffRichard Rodger, Senate Staff

David Schumacher, OFM Director Gubernatorial Appointee VacancySandra Archibald, Univ. of WALes Purce, The Evergreen State Col.Robert Rosenman, WA State Univ.Rodolfo Arévalo, Eastern WA Univ.

Board of DirectorsSenator Mark Schoesler, Co-Chair

House Democratic Vacancy, Co-Chair

Slide 2 of 22

UpcomingExamplesThe ResearchBackground

WSIPP

Capitol

Washington State Institute for Public Policy

Slide 3 of 22

UpcomingExamplesThe ResearchBackground

1983 legislative intent

Apply expertise of academia to needs of state policymakers

Connect The Evergreen State College (based in Olympia) with state government

Goal: better informed policy making

Washington State Institute for Public Policy

Slide 4 of 22

UpcomingExamplesThe ResearchBackground

Current role

Applied research

Legislature assigns and funds most studies

Reports are easy to understand and accessible to policymakers

Non-partisan staff

Washington State Institute for Public Policy

Slide 5 of 22

UpcomingExamplesBackground The Research

Three Types of Legislative Assignments to WSIPP

1. What Works? Benefit-Cost Analysis (research reviews)

2. Outcome evaluations of specific Washington programs (primary research)

3. Miscellaneous

Are there evidence-based policy options that improve public outcomes, at less cost?

Slide 6 of 22

UpcomingExamplesBackground The Research

We systematically analyze all, credible evaluations of real world ways to improve key public outcomes.

1a. What Works? What Doesn’t?

CrimeEducation, Early Ed. Child Abuse & Neglect Substance AbuseMental HealthDevelopmental Disabilities

Teen BirthsEmploymentPublic AssistancePublic HealthHousing

What does the weight of the evidence tell us?

Slide 7 of 22

UpcomingExamplesBackground The Research

1b. What’s Cost-Beneficial?

For programs and policies determined to “work”

How much does it cost?

What are the long-term benefits in monetary/fiscal terms?

Which are the best investments from the state’s perspective?

What is the risk of success/failure?

Slide 8 of 22

UpcomingExamplesBackground The Research

2. Outcome Evaluations

Do Washington programs & policies achieve their intended effects?

Comparison group research

Administrative and survey data

Outcomes-focused

Slide 9 of 22

UpcomingExamplesBackground The Research

3. Miscellaneous

Often qualitative (e.g., describe a program or process or survey stakeholders regarding their views)

Review other states’ policies in comparison with Washington’s

Key Institute role = independence, objectivity

“Other duties as assigned”

Slide 10 of 22

UpcomingThe ResearchBackground Examples

Slide 11 of 22

Example #1: How Does Class Size Impact Student Outcomes?

Report to the 2013 Legislature

Test scores and high school graduation

Literature review (meta-analysis)

• Supplemented by WSIPP analysis of state-level data

Cost estimation

Risk analysis

UpcomingThe ResearchBackground Examples

Slide 12 of 22

How to find studies:

Keywords in combination (class size, evaluation, K-12, outcomes, public schools, effective, and so on)

Databases (ProQuest and other academic journal sources, Google Scholar)

Previously published reviews/meta-analyses. Comb reference lists.

Organization databases (e.g., the Campbell Collaboration, What Works Clearinghouse)

Literature Search

UpcomingThe ResearchBackground Examples

Slide 13 of 22

Literature Review

Screening studies:

Is the study an evaluation or descriptive?

Does the evaluation address the research question?

Is there a valid comparison group?

What outcomes are measured?

UpcomingThe ResearchBackground Examples

Slide 14 of 22

An effect size (or in the case of class size, an “elasticity”)

Represents the difference in outcomes between the experimental and comparison groups

Tells us how much change we might expect in an outcome if a program or policy is implemented

Must be interpreted within the context of the policy environment

Meta-analysis

UpcomingThe ResearchBackground Examples

Slide 15 of 22

0 1 2 3 4 5 6 7 8 9 10 11 12-0.5%

+0.0%

+0.5%

+1.0%

+1.5%

+2.0%Individual StudiesInstitute Summary

School Grade Where Class Size Is Reduced

Ch

ang

e in

Ou

tco

mes

*The measured outcomes include student test score gains, high school graduation, and dropout rates.The chart plots the 77 credible effects we found in the research literature. Our summary line is a weighted average.

How a 10% Decrease in Class SizeAffects Student Outcomes*

UpcomingThe ResearchBackground Examples

Slide 16 of 22

Per-Student Benefits and Costs of Reducing Class Size by One Student

Grade Costs Avg. Benefits Avg. NetValue

K $198 $2,302 $2,1041 $198 $1,218 $1,0212 $198 $725 $5283 $198 $578 $3814 $179 $422 $2435 $179 $366 $1876 $179 $347 $1687 $162 $358 $1968 $162 $336 $1759 $160 $306 $146

10 $160 $301 $14111 $160 $378 $21812 $160 $353 $193

UpcomingThe ResearchBackground Examples

Slide 17 of 22

Chance that benefits outweigh costs

0%

50%

100%

K 1 2 3 4 5 6 7 8 9 10 11 12Grade Where Class Size Is Reduced

1st grade: 94%

Close to 50%

Economics & Risk: Reducing Class Size by One Student

UpcomingThe ResearchBackground Examples

Slide 18 of 22

Benefits Per Individual

Crime reduction $5,485 Lower CJ / victim costs

Main Source of Benefits

Test scores & graduation rates $13,266 Increased earnings

Special education reduction $1,081 Lower K-12 costs

Lower K-12 costsGrade repetition reduction $307

Lower CW / victim costs Fewer out-of-home placements $557 Lower health care system costs $756 Lower Medicaid enroll.

Total Benefits Per Individual $22,693

Cost Per Individual

Benefits Per Dollar of Cost

$7,489

$3.06

Nearly 100% chance

of benefits exceeding costs

Less child abuse & neglect $1,240 Lower CW / victim costs

Example #2: Early Childhood Education (low income)2011 dollars

UpcomingThe ResearchBackground Examples

Slide 19 of 22

WSIPP “Consumer Reports” ListsEvidence-based policy options ranked by return on investment

ExampleThe ResearchBackground Upcoming

Slide 20 of 22

Current Legislative Proposals

In the 2013 session:

Reduce class size in K-3 (or K-1)

Expand early childhood education for low-income children by 10%

Assign WSIPP additional reviews to identify evidence-based options for K-12 education funding increases

ExampleThe ResearchBackground Upcoming

Slide 21 of 22

Current WSIPP Projects

Innovative K-12 Public Schools in Washington State

State Need Grant (Student Outcomes)

Outcomes of Juveniles Tried in Adult Courts

I-502 Evaluation: Costs and Benefits of the Legalization of Cannabis in Washington State

Impacts on Crime from Different Approaches to Policing

ExampleThe ResearchBackground Upcoming

Slide 22 of 22

Potential* Upcoming WSIPP Projects

How can the state control Medicaid costs?

Which components of early childhood education programs are evidence-based?

Develop a risk assessment tool for individuals involuntarily committed for mental illness

Compile an inventory of evidence-based approaches to substance abuse intervention

*Depends on 2013 legislative session