New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program. Carolyn J. Heinrich LaFollette School of Public Affairs, University of Wisconsin Peter R. Mueser University of Missouri, IMPAQ International, LLC, and IZA - PowerPoint PPT Presentation
Carolyn J. Heinrich LaFollette School of Public Affairs, University of Wisconsin Peter R. Mueser University of Missouri, IMPAQ International, LLC, and IZA Kenneth R. Troske University of Kentucky and IZA Kyung-Seong Jeon University of Missouri Daver C. Kahvecioglu IMPAQ International, LLC November 2009 New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program
Transcript
Carolyn J. HeinrichLaFollette School of Public Affairs, University of Wisconsin
Peter R. MueserUniversity of Missouri, IMPAQ International, LLC, and IZA
Kenneth R. TroskeUniversity of Kentucky and IZA
Kyung-Seong JeonUniversity of Missouri
Daver C. KahveciogluIMPAQ International, LLC
November 2009
New Estimates of Public Employment and Training Program
Net Impacts: A Nonexperimental Evaluation of
the Workforce Investment Act Program
1998 Workforce Investment Act (WIA)
Largest job-training program in the U.S.
• Implemented in 2000 in most states—replaced Job Training Partnership Act (JTPA)
Current annual federal budget: $3 billion
• Significantly lower relative to past public expenditures
Program implementation differs by state and local area
• Work-first emphasis, service sequencing, referrals, access to training
3
WIA Adult Programs
Two primary adult programs serving:• Disadvantaged workers, both
unemployed and those in low-paying and unstable jobs
• Dislocated workers who have lost jobs or are slated to be laid off
Voluntary: participants recruited by local agency staff or referred by training providers– may have specific number of slots to fill• Locally-determined eligibility standards
4
WIA Service Sequencing
Core services - outreach, job search, placement aid, and labor market information
Intensive services - comprehensive assessments, individual employment plans, counseling and career planning
Training services – mostly occupational/vocational training, some on-the-job training
• Most training provided with voucher
Similar services available to anyone in Employment Service
offices (“Wagner-Peyser” services)
Training time varies from a few months
up to 2 years
Impact Analysis
Impact estimates for • Adult program• Dislocated Worker program• Incremental impact of Training services
vs. Intensive/Core services for these programs
Using state administrative data from 12 states: Connecticut, Indiana, Kentucky, Maryland, Mississippi, Missouri, Minnesota, Montana, New Mexico, Tennessee, Utah, Wisconsin
•WIA recipients receive the most meaningful services
WIA
Wagner-Peyser Services
UI Claim
Both are plausible comparison groups because they contain individuals with employment problems & those seeking assistance
8
Measures
Outcomes: Earnings & employment 16 qtrs
Control variables: Calendar quarter of program entry Demographics: gender (exact match),
age, education, race/ethnicity Disability, veteran Local labor market Employment and earnings over the two
years prior to program entry Industry of most recent job Prior program participation (WIA, UI, ES)
9
Overview of Analysis
WIA Program Group Sample Group
Adult DW Treatment Comparison
(a) (b) (c) (d)
1. X X WIA UI Claim or Wagner-Peyser
2. X X WIA Receiving UI UI Recipients
3. X X WIA Training WIA Core/Intensive
10
Estimates
Estimates for states are combined Weighted by number of WIA
participants•WIA participants entering in PY2003 and
PY2004
Estimate of average program impact across 12 states•Effect of the treatment (WIA) on the treated•Comparison group defined by program
contact in quarter of entry (UI claim or benefit, Wagner-Peyser service receipt, WIA participants who did not receive training services for training impact estimate)
Matching Procedures
Matching is within state, within gender, and (usually) within quarter of participation
Logit specification predicts propensity score• Details of variable coding differ by state;
approx. 100 individual characteristics, labor market experience and prior program participation, geographic area within state
Matching is many-to-one, using a caliper, with replacement (“radius matching”)
Standard errors use formula recommended by Imbens (2008) based on a conditional variance estimate
Summary of Impact Estimates: Dislocated Worker Program
Pattern of results consistent with expectations that WIA services require time to produce impacts for workers facing serious difficulties in obtaining reemployment
• DW participant earnings do not reach earnings of comparable nonparticipants until more than two years after participation
Estimates also imply little incremental impact of training for DW participants
• Unlikely initial costs of training could be recouped Results for 7 High-Training states show less evidence
of bias (especially for women) but impacts on earnings even after training is complete appear minimal
• There does appear to be some impact on employment26
Conclusion
Adult Program
• We observe long-term positive impacts of the WIA program
• Training also appears to be valuable Dislocated Worker Program
• Selection on stable unobserved factors may induce positive bias in impact estimates
• Program long-run impacts difficult to gauge• Long-term impacts appear minimal• Training appears to have little long-run effect
27
Conclusion (continued)
Analysis shows both the potential benefits and the limitations of nonexperimental estimates based on administrative data
• Selection clearly affects results
• Patterns of estimates provide an indication of where impacts may be greatest
• Specification tests are important
• Comparison across control types and programs is useful
