Slide 1
Capturing the Consequences of Ongoing Welfare Reform:
The Case of California’s CalWORKs
NAWRS Conference
Atlanta, GA August 25, 2015
Image via Barbara Helgason / Fotolia Image Via: Vedran Vukoja AKA vepar5 /Fotolia
Slide 2
Introduction
Name Title Role on Project Lynn Karoly, Ph.D. Senior Economist RAND PI
Diana Lavery, M.A. Project Associate RAND Project Director
Name Title Role on Project Johannes (Hans) Bos, Ph.D. Vice President and Program Director AIR PI
Lori Turk-Bicakci, Ph.D. Senior Researcher AIR Project Director
CalWORKs The California Work Opportunity and Responsibility to Kids (California’s TANF program)
California Senate Bill (SB) 1041
Instituted significant reforms to CalWORKs
Evaluation of the SB 1041 Reforms to California’s CalWORKs Program Conducted by RAND and American Institutes for Research (AIR)
Slide 3
CalWORKs Policy SB 1041 Represents a Significant Change
• Twin goals of legislation – Engage participants in work, education, and service activities early – Provide more flexibility in activity options
• Reforms to key components of CalWORKs – 48 month CalWORKs WTW time clock changed to 24 months of CalWORKs
WTW followed by 24 months of TANF – Decreased weekly work requirements for single parents – Adults with former young child exemption need to be re-engaged by county – Reinstated Cal-Learn for pregnant/parenting teens – Other (financial incentives, earned income disregard, etc.)
Slide 4
Current CalWORKs Policy Comparison to Other States
• Among highest income thresholds to qualify for TANF • Among more restrictive states due to time-limit policy • One of the few of states that provides benefits
indefinitely for children in need • Relatively high benefit levels and one of the most
generous financial work incentives • Alignment with many other states through elimination of
the core requirement during the first 24 months on aid
Slide 5
Challenges in Evaluating SB 1041 • Context in California in 2012:
– Improving economy – Affordable Care Act – Other competing efforts that county staff had to
accommodate
• Difficult to attribute changes to SB 1041
Slide 6
RAND and AIR’s Approach to Evaluating Policy Change What impact has SB 1041 had?
• Mixed methods – Interviews, surveys, fieldwork, administrative data collection
• Multi-level – CalWORKs participants, caseworkers, county leadership, state leadership, other stakeholders
Slide 7
The CalWORKs SB 1041 Evaluation
• Contract period July 2014 through Dec 2017
• Full state evaluation, with more intensive examination in six focal counties
• Yearly reports will – examine implementation of SB 1041 – assess impact of SB 1041 – recommend changes to the program
Slide 8
The CalWORKs SB 1041 Evaluation Research Questions
• Has SB 1041 provided greater flexibility in the services and activities available to participants?
• Has SB 1041 helped to remove barriers to participants’ employment?
Slide 9
The CalWORKs SB 1041 Evaluation Research Questions
• Has SB 1041 provided greater flexibility in the services and activities available to participants?
• Has SB 1041 helped to remove barriers to participants’ employment?
• Has SB 1041 accelerated participants’ paths to self-sufficiency?
• Has SB 1041 improved the well-being of participant children and their families?
Slide 10
The CalWORKs SB 1041 Evaluation Research Questions
• Has SB 1041 provided greater flexibility in the services and activities available to participants?
• Has SB 1041 helped to remove barriers to participants’ employment?
• Has SB 1041 accelerated participants’ paths to self-sufficiency?
• Has SB 1041 improved the well-being of participant children and their families?
• How have the SB 1041 changes affected CalWORKs and related programs at the county level?
• Has SB 1041 affected California’s TANF annual Work Participation Rates?
Slide 11
The CalWORKs SB 1041 Evaluation Research Questions
• Has SB 1041 provided greater flexibility in the services and activities available to participants?
• Has SB 1041 helped to remove barriers to participants’ employment?
• Has SB 1041 accelerated participants’ paths to self-sufficiency?
• Has SB 1041 improved the well-being of participant children and their families?
• How have the SB 1041 changes affected CalWORKs and related programs at the county level?
• Has SB 1041 affected California’s TANF annual Work Participation Rates?
What improvements to CalWORKs Program are recommended?
Slide 12
Evaluation Design: Mixed Methods with Five Study Components
Process Study
County Operations Impact Study
Participant Tracking Study
Participant Status Study
Participant Impact Study
4
2
3
1
5
Slide 13
Evaluation Framework SB1041 DIRECTIVES
BY THE STATEIMPLEMENTATION
BY COUNTIESPARTICIPATION BY
PARTICIPANTS
Eligibility requirements Determine eligibility Participate in activities; receive assistance
Services and support Administer services and support Time limit reached
Benefits Deliver benefits Employment secured
OUTCOMES FOR FAMILIES & CHILDREN
★ Stable, gainful employment
★ Economic self-sufficiency
Process StudyProcess & County Operations Impact
Studies
Participant Status &Tracking Studies
Participant Impact Study
Slide 14
Data Collection
Study Component
State Key Informant Interviews
Focal County
Interviews and Focus
Groups
All–County Survey (ACS)
Adminis-trative Data
Participant Longitudinal
Survey (CalSES)
Current Population
Survey (CPS)
1. Process Study 2. County
Operations Impact Study
3. Tracking Study
4. Participant Status Study
5. Participant Impact Study
Slide 15
Data Analysis • Qualitative Analysis
– Document review of state and county communication – Identify key themes, challenges, and successes at the county level with
insights from site visits
• Quantitative Analysis – Data snapshots – Interrupted time series analysis with multiple entry cohorts – Hierarchical linear model (HLM) regression framework
• Triangulation
Slide 16
For More Information: First Evaluation Report: Evaluation of the SB 1041 Reforms to the California’s CalWORKs Program Available at the RAND Website: http://www.rand.org/pubs/research_reports/RR919.html
Future reports will be available by December of 2015, 2016, and 2017.
http://www.rand.org/pubs/research_reports/RR919.htmlhttp://www.rand.org/pubs/research_reports/RR919.html
Slide 17
Discussion • What methods have you used to document
implementation of a new program? • What methods have you used to assess impact of
policy change?
Slide 18
Slide 19
APPENDICES
Slide 20
RAND and AIR Project Team Name Title Role on Project
Lynn Karoly, Ph.D. Senior Economist RAND PI
Robert Bozick, Ph.D. Senior Sociologist RAND Co-PI
Lois Davis, Ph.D. Senior Policy Researcher RAND Co-PI
Diana Lavery, M.A. Project Associate RAND Project Director
Beverly Weidmer, M.A. Survey Director RAND Survey Director
Name Title Role on Project
Johannes (Hans) Bos, Ph.D. Vice President and Program Director AIR PI
Sami Kitmitto, Ph.D. Principal Researcher AIR Co-PI
Lori Turk-Bicakci, Ph.D. Senior Researcher AIR Project Director
Cheryl Graczewski, M.A. Senior Researcher AIR Deputy Project Director
Gabriele (Gaby) Fain, M.A. Principal Researcher AIR Qualitative Team Lead
Slide 21
Historical Overview of California’s Welfare to Work Policy
• 1998: CalWORKs Implemented – California’s TANF program; “work first” model
• 2005: SB 1104 – Tightened participation requirements
• 2011: SB 72 – Tightened the time limit from 60 to 48 months, reduced income
disregard, suspended Cal-Learn
Slide 22
California’s Welfare to Work Program before SB 1041
0
2
4
6
8
10
12
14
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1,000,000
July
-94
Janu
ary-
95Ju
ly-9
5Ja
nuar
y-96
July
-96
Janu
ary-
97Ju
ly-9
7Ja
nuar
y-98
July
-98
Janu
ary-
99Ju
ly-9
9Ja
nuar
y-00
July
-00
Janu
ary-
01Ju
ly-0
1Ja
nuar
y-02
July
-02
Janu
ary-
03Ju
ly-0
3Ja
nuar
y-04
July
-04
Janu
ary-
05Ju
ly-0
5Ja
nuar
y-06
July
-06
Janu
ary-
07Ju
ly-0
7Ja
nuar
y-08
July
-08
Janu
ary-
09Ju
ly-0
9Ja
nuar
y-10
July
-10
Janu
ary-
11Ju
ly-1
1Ja
nuar
y-12
Unem
ployment rate (%
)
Cas
eloa
d
Month and year
SB 1104 implemented
SB 72 implemented CalWORKs implemented
Unemployment rate
Caseload
PRWORA passage
January-13 SB 1041
Implemented
Slide 23
Key SB 1041 Policy Changes: Time Clock
Policy Pre–SB 1041 Post–SB 1041 Time Clock Structure 48 months of CalWORKs WTW
assistance, services, and activities
48 months of support in two consecutive periods: (1) 24 months of CalWORKs WTW services and
activities, followed by (2) 24 months of TANF work activities
Time Clock Initiation n.a. • Those starting the program on or after January 1, 2013 will follow the SB 1041 time clock structure
• Those in the program prior to January 1, 2013 with fewer than 24 months accumulated will follow the SB 1041 time clock structure
• Those in the program prior to January 1, 2013 with more than 24 months accumulated will follow the pre-SB 1041 48-month time clock structure
Slide 24
Key SB 1041 Policy Changes: Weekly Work Activity Requirements
Policy Pre–SB 1041 Post–SB 1041 Weekly Total Work Activity Requirements
• Single-parent: 32 hours • Two-parent: 35 hours
• Single-parent, child < six: 20 hours • Single-parent, child ≥ six: 30 hours • Two-parent: 35 hours
Weekly Core Work Requirements
• 20 of the 32 hours of the weekly work requirement have to be in core activities which include employment, on-the-job training, and vocational educational training
• No core requirement during the 24 month CalWORKs service period
• During the 24 month TANF service period, 20 hours of the weekly work requirement for single-parents and 30 hours of the weekly work requirement for two-parents have to be in core activities which include employment, on-the-job training, and vocational educational training
Slide 25
Key SB 1041 Policy Changes: Young Child Exemption and Cal-Learn
Policy Pre–SB 1041 Post–SB 1041 Young Child Exemption
• Short-term exemption: One adult per family excused from WTW activities if caring for one child under two years of age or two children under six years of age
• One lifetime exemption: One adult per family excused from WTW activities if caring for child under two years of age
• Adults exempt from the pre-SB 1041 young child exemption rules are not required to participate until the county re-engages them
Cal-Learn Program suspended Program reinstated April 1, 2013: Pregnant/parenting teens will no longer be under WTW rules, but under special Cal-Learn rules
Slide 26
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Cas
eloa
d
Year
Two parent
Zero parent
All other family types
TANF timed out Safety Net
CalWORKs Caseload by Family Type
Slide 27
CalWORKs Caseload Process, Effective January 1, 2013
Slide 28
Illustrative Data Structure and Analysis File Structure for Multiple Entry Cohorts
�Capturing the Consequences of Ongoing Welfare Reform: �The Case of California’s CalWORKs��NAWRS Conference�Atlanta, GA�August 25, 2015IntroductionCalWORKs Policy�SB 1041 Represents a Significant ChangeCurrent CalWORKs Policy�Comparison to Other StatesChallenges in Evaluating SB 1041RAND and AIR’s Approach to Evaluating Policy ChangeThe CalWORKs SB 1041 Evaluation �The CalWORKs SB 1041 Evaluation �Research QuestionsThe CalWORKs SB 1041 Evaluation �Research QuestionsThe CalWORKs SB 1041 Evaluation �Research QuestionsThe CalWORKs SB 1041 Evaluation �Research QuestionsEvaluation Design: �Mixed Methods with Five Study Components�Evaluation FrameworkData CollectionData AnalysisFor More Information: DiscussionSlide Number 18APPENDICESRAND and AIR Project TeamHistorical Overview of California’s �Welfare to Work PolicyCalifornia’s Welfare to Work Program �before SB 1041Key SB 1041 Policy Changes: �Time ClockKey SB 1041 Policy Changes: �Weekly Work Activity RequirementsKey SB 1041 Policy Changes: �Young Child Exemption and Cal-LearnSlide Number 26Slide Number 27Illustrative Data Structure and Analysis File Structure for Multiple Entry Cohorts