Teaching Self-Sufficiency:30-Month Impacts of a Home Visitation and

Life Skills Education Program for Hard-to-Employ TANF Recipients

Findings from the Rural Welfare-to-Work Evaluation

Teaching Self-Sufficiency:30-Month Impacts of a Home Visitation and

Life Skills Education Program for Hard-to-Employ TANF Recipients

Findings from the Rural Welfare-to-Work Evaluation

Alicia MeckstrothAndrew Burwick

Quinn MooreAndrew McGuirk

APHSA/CBPP Teleconference on Home Visiting in TANFJanuary 9, 2008

Alicia MeckstrothAndrew Burwick

Quinn MooreAndrew McGuirk

APHSA/CBPP Teleconference on Home Visiting in TANFJanuary 9, 2008

Nebraska’s TANF Policy Context Nebraska’s TANF Policy Context

Supportive, work-oriented TANF program

Targeted education and training

Two-year time limit

Various services available in target areas

Modest unemployment and poverty

Supportive, work-oriented TANF program

Targeted education and training

Two-year time limit

Various services available in target areas

Modest unemployment and poverty

2

NOT FOR CITATION

Building Nebraska Families (BNF)Building Nebraska Families (BNF)

Nonexempt, hard-to-employ TANF recipients

University extension and state welfare agency

Masters’ level educators with very small caseloads

11 service areas throughout Nebraska

Nonexempt, hard-to-employ TANF recipients

University extension and state welfare agency

Masters’ level educators with very small caseloads

11 service areas throughout Nebraska

3

NOT FOR CITATION

Building Nebraska Families (BNF)Building Nebraska Families (BNF)

4

NOT FOR CITATION

Individualized Life Skills Education Through Home Visits

Mentoring and Informal

Counseling

Service Coordination & Advocacy

Support

• Personal Improvement: goal setting, problem-solving, character development, coping skills, relationship-building, communication skills

• Family Life: child development, parenting, family management

• Practical Life Skills: money and time management, healthy home, nutrition

Research QuestionsResearch Questions

Program implementation and costs?

Effects on employment, earnings, welfare dependence, and well-being?

Implications and lessons?

Program implementation and costs?

Effects on employment, earnings, welfare dependence, and well-being?

Implications and lessons?

5

NOT FOR CITATION

Evaluation MethodsEvaluation Methods

Experimental design (358 Ts, 242 Cs)

Follow-up telephone surveys at 18 and 30 months(87 percent and 83 percent completion rates)

Administrative records

Program service use and participation data

Site visits and focus groups

Experimental design (358 Ts, 242 Cs)

Follow-up telephone surveys at 18 and 30 months(87 percent and 83 percent completion rates)

Administrative records

Program service use and participation data

Site visits and focus groups

6

NOT FOR CITATION

Subgroup Analysis:“More Disadvantaged”

Subgroup Analysis:“More Disadvantaged”

Met 2 or more of these criteria at baseline:

- Lack of high school credential- Health condition (self or HH member)- Transportation barrier (no driver’s license or

regular access to vehicle)- No earnings in prior year- Received TANF/AFDC for 2+ years in lifetime

43 percent were more disadvantaged

Met 2 or more of these criteria at baseline:

- Lack of high school credential- Health condition (self or HH member)- Transportation barrier (no driver’s license or

regular access to vehicle)- No earnings in prior year- Received TANF/AFDC for 2+ years in lifetime

43 percent were more disadvantaged

NOT FOR CITATION

Well-Implemented ProgramWell-Implemented Program

Services delivered successfully

Intensive services- Participation over 8 months- 22 contacts- 25 hours

Program group members more likely to receive skill-building services and mentoring

Total cost per participant = $7,200 (approx)

Services delivered successfully

Intensive services- Participation over 8 months- 22 contacts- 25 hours

Program group members more likely to receive skill-building services and mentoring

Total cost per participant = $7,200 (approx)

NOT FOR CITATION

Highlights of Impact Findings for the Full Sample

Highlights of Impact Findings for the Full Sample

Limited evidence that BNF improved employment status for the full sample

No convincing evidence that BNF improved earnings for the full sample

Limited evidence that BNF improved employment status for the full sample

No convincing evidence that BNF improved earnings for the full sample 9

NOT FOR CITATION

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Control Group Program Group

SOURCE: Rural Welfare-to-Work Evaluation's 30-Month Follow-up Survey of BNF sample members.

NOTE: The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

BNF Improved Employment for the More Disadvantaged Group

BNF Improved Employment for the More Disadvantaged Group

** *

Percentage Employed

10

NOT FOR CITATION

** ** *

Program Group

Control Group

*

*

$0

$100

$200

$300

$400

$500

$600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Control Group Program Group

SOURCE: Rural Welfare-to-Work Evaluation's 30-Month Follow-up Survey of BNF sample members.

NOTE: The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

BNF Substantially Increased Earnings for the More Disadvantaged Group

BNF Substantially Increased Earnings for the More Disadvantaged Group

*** *

Average Earnings

** ** **

11

NOT FOR CITATION

** ** ** **

Program Group

Control Group

** ** * *

Program Group Members Held Better JobsProgram Group Members Held Better Jobs

12

NOT FOR CITATION

20 20 2029

0

20

40

60

80

100

Employed 35 or morehours per week

Hourly Wage > $8 Health Insurance Paid vacation

Program Group (More Disadvantaged)Control Group (More Disadvantaged)

9**

19*

6*** 8**

Percentage

SOURCE: Rural Welfare-to-Work Evaluation's 30-Month Follow-up Survey of BNF sample members.

NOTE: The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups. The sample for these variables includes working and nonworking members.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

BNF Reduced Welfare ReceiptBNF Reduced Welfare Receipt

NOT FOR CITATION

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Control Group (More Disadvantaged)

Program Group (More Disadvantaged)

SOURCE: Administrative records data from the State of Nebraska, compiled by Mathematica Policy Research, Inc.

NOTE:The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

Percentage Receiving TANF

Control Group

Program Group

** ** *** *** ** ** ***

13

*

*** *** ***

BNF Increased Household IncomeBNF Increased Household Income

14

NOT FOR CITATION

1,670

0

500

1,000

1,500

2,000

Program Group (More Disadvantaged)

Control Group (More Disadvantaged)

1,234***

0

20

40

60

80

100

72*60

Total Household Income at 30-

Month Follow-up

Living in Poverty at 30-Month Follow-up

Percentage2004 dollars

SOURCE: Rural Welfare-to-Work Evaluation's 30-Month Follow-up Survey of BNF sample members.

NOTE: The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

Fewer Health-Related Hardships, More Household Hardships

Fewer Health-Related Hardships, More Household Hardships

NOT FOR CITATION

15

146

60

44

74

37

15*23*

54**

46**

30**

61**

0

20

40

60

80

100

Program Group (More Disadvantaged)

Control Group (More Disadvantaged)

Overall health fair

or poor

Self-reported depression or mental health

issue

Spousal or partner

abuse

Had utility turned off

Ever had serious housing problem

Often or sometimes not enough money

to buy food

Percentage

SOURCE: Rural Welfare-to-Work Evaluation's 18-Month Follow-up Survey of BNF sample members.

NOTE: The estimates were adjusted using multivariate regression methods and the data were weighted to account for survey nonresponse andto equalize the size of the program and control groups.

*/**/***Significantly different from zero at the .10/.05/.01 level, two-tailed test.

What Might Influence BNF’s Impacts on Employment and Earnings?

What Might Influence BNF’s Impacts on Employment and Earnings?

Services complement existing employment-related assistance

Home visits allow for individualized support—especially valuable to more disadvantaged

Well-developed life skills curriculum

Highly-qualified, professional staff

Low caseloads

Services complement existing employment-related assistance

Home visits allow for individualized support—especially valuable to more disadvantaged

Well-developed life skills curriculum

Highly-qualified, professional staff

Low caseloads 16

NOT FOR CITATION

Questions and ImplicationsQuestions and Implications

What consideration should be given to BNF’s cost?

How might BNF operate within context of new work and participation requirements?

Could the BNF program model transfer to other environments?

What additional supports might be needed for clients who become employed?

What consideration should be given to BNF’s cost?

How might BNF operate within context of new work and participation requirements?

Could the BNF program model transfer to other environments?

What additional supports might be needed for clients who become employed?

17

NOT FOR CITATION

Rural Welfare-to-Work Strategies Demonstration Evaluation

Rural Welfare-to-Work Strategies Demonstration Evaluation

For more information:

Mathematica Policy ResearchAlicia Meckstroth, (614) 505-1401, almeckstroth@mathematica-mpr.comPrevious evaluation reports available at www.mathematica-mpr.com

U.S. DHHS, ACFMichael Dubinsky, (202) 401-3442, midubinsky@acf.hhs.govKarl Koerper, (202) 401-4535, kkoerper@acf.hhs.gov

University of Nebraska Cooperative ExtensionMarilyn Fox, (308) 385-5088, mfox@unlnotes.unl.edu(BNF curriculum)

For more information:

Mathematica Policy ResearchAlicia Meckstroth, (614) 505-1401, almeckstroth@mathematica-mpr.comPrevious evaluation reports available at www.mathematica-mpr.com

U.S. DHHS, ACFMichael Dubinsky, (202) 401-3442, midubinsky@acf.hhs.govKarl Koerper, (202) 401-4535, kkoerper@acf.hhs.gov

University of Nebraska Cooperative ExtensionMarilyn Fox, (308) 385-5088, mfox@unlnotes.unl.edu(BNF curriculum)

18