Final Evaluation of the Title IV-E Waiver Final Evaluation of the Title IV-E Waiver Child Welfare Demonstration in New Child Welfare Demonstration in New

HampshireHampshire

Ninth Annual Child Welfare Ninth Annual Child Welfare Demonstration Projects MeetingDemonstration Projects Meeting

June 2005June 2005Glenda Kaufman Kantor, University of New HampshireGlenda Kaufman Kantor, University of New Hampshire

Bernie Bluhm, NH DCYFBernie Bluhm, NH DCYF

State of New Hampshire Study Sample State of New Hampshire Study Sample SitesSites

Challenges of Challenges of NH EnvironmentNH Environment

Baseline Status of Baseline Status of Substance Abuse in Substance Abuse in

NHNH

History: Higher Incidence of History: Higher Incidence of Problem Drinking NH Women Than Problem Drinking NH Women Than

National AverageNational Average

20%

13% 14%13%

0%

5%

10%

15%

20%

25%

1991 1995

% Frequent Drinking in NH Women & Women Nationally

NH Women

National Median

Current Status of NH Current Status of NH Substance Abuse (2002)Substance Abuse (2002)

31% Past Month Illicit Drug Use – (18-25 yrs)– 2nd Highest in U.S.

50% Past Month Binge Alcohol Use – (18-25 yrs)- 3rd Highest in U.S.

21% Past Month Binge Alcohol Use – (26 yrs>) (National Average)

300 children <17 living under court order in secure facilities due to D/A charges, related crimes (2001)

Since 1994, less than 15% of NH CPS assessments completed each year have been substantiated (Founded).

Substantiation rate of NH DCYF child abuse/neglect assessments conducted from 1994-2004.

(Founded assessments/completed assessments)

14.6% 13.5% 13.1% 13.2% 12.6% 11.7% 10.5% 9.3% 10.3% 11.0% 10.5%

CY 94 CY 95 CY 96 CY 97 CY 98 CY 99 CY 00 CY 01 CY 02 CY 03 CY 04

Key Project AimsKey Project Aims

Identify & Address Parental Substance Abuse Problems co-occurring with suspected child abuse or neglect.

Better Decision-Making about Safety Reduce Substance Abuse Risk Behaviors of

Parents Fewer Subsequent Founded Referrals Prevent or Shorten Placement of Children in

Foster Care Improve Stability and Adjustment of Children Cost Neutrality/Savings of Project

Project DesignProject Design

Target PopulationTarget Population

Credible report of suspected CA/NSubstance Abuse Current Risk Factor

Related to CA/NReside in Hillsborough Co area covered

by the Manchester/Nashua D.Os.No Open Case at Intake into StudyAccessible to DCYF CPSW

LADC RoleLADC Role

Engage Client at Time of Assessment of Allegation

Immediate Screening & Assessment by LADC

Immediate individual treatment for AODA

Immediate and ongoing consultation for CPSW

For people awaiting treatmentFor people awaiting treatment

Individual counselingOn-going contact with counselor

– Treatment window extended 60 days from CPS assessment or case closure

– Treatment provider connections

For Families Receiving For Families Receiving ServicesServices

Consultant participates in case planning Keep focus on parent issues Include parenting in treatment goals Aftercare with focus on parenting Support to relative caregivers

Benefits to CPS During Benefits to CPS During AssessmentAssessment

Regular Consultation Preliminary Screening (SASSI) of Parental

Substance Abuse Impact of Parental Substance Abuse on

Safety and Risk of Harm to Children Recommendations for Services and

Treatment

Benefits for CPS Cases When Benefits for CPS Cases When Children Are In Out-Of-Home CareChildren Are In Out-Of-Home Care

Comprehensive assessment with DXAssistance with goal specific case

planningContinued consultationRecommendations for parents and

children

EvaluationEvaluation

Evaluation DesignEvaluation Design Experimental Model with True Randomized Design

to Standard/Enhanced Services at 2 District Offices Standard group received the usual services

provided by NH DCYF Parallel Data Collection for Standard and Enhanced Process & Outcomes SACWIS Data Interviews at Baseline & Follow-up Cost Benefit

Final Evaluation StatusFinal Evaluation Status

Conducted 11/15/99 through 10/15/04– 437 families eligible

212 baseline interviews (49%)156 follow-up interviews (74%)

Study Sample Demographics: Study Sample Demographics: Primary Caregiver Primary Caregiver

Enhanced Group (n=222): Mean Age: 33 Years % White: 92% Any Employment: 59% Relationship of Alleged

Perp. to Child Bio. Mother 69%

Mean Family Size Total Adults: 1.83 Total Children: 2.80

Standard Group (n=215): Mean Age: 33 Years % White: 90%• Any Employment: 63% Relationship of Alleged

Perp. to Child Bio. Mother: 72%

• Mean Family Size Total Adults: 1.85 Total Children: 2.84

Study Sample Demographics: ChildStudy Sample Demographics: Child

Enhanced Group (n=222) Female: 51% Mean Age: 8 yrs.

Median: 9 yrs. Range: 0-17 yrs.

Ethnicity: White: 87% Black: 6% Latino/Hispanic: 5% Other: 2%

Standard Group (n=215) Female: 51% Mean Age: 8 yrs.

Median: 8 yrs. Range: 0-17 yrs.

Ethnicity: White: 89% Black: 6% Latino/Hispanic: 2% Other: 3%

Maltreatment & CPS FactorsMaltreatment & CPS Factors

Enhanced Group (n=222) CPS Factors

Prior Referrals* 44%

High Risk at Entry 11% Type of Maltreatment

Physical Abuse 25% Phys. Abuse & Negl. 13%

Neglect: 52% Sexual Abuse: 3% Psychological Abuse: 1%

Standard Group (n=215) CPS Factors

Prior Referrals 51%

High Risk at Entry 14% Type of Maltreatment

Physical Abuse: 21% Phys. Abuse & Negl: 9%

Neglect: 56% Sexual Abuse: 5% Psychological Abuse: 2%

Family Risk FactorsFamily Risk Factors

Enhanced (n=222) Domestic Violence:

33%

Adult Mental Illness: 18%

Adult Phys. Disability: 4%

Homeless: 17% Incarceration (Case

Level): 32%

Standard (n=215) Domestic Violence:

33% Adult Mental Illness:

20% Adult Phys. Disability:

7% Homeless: 14% Incarceration (Case

Level): 28%

Child Risk FactorsChild Risk Factors

Enhanced Group (n=222) Mental Illness: 6% Phys. Disability: 3% Learning Disabled

11% Neonatal Addiction: 2% Severe Behavior Prob.

5% JPPO* Involvement at Case Level 26%

Standard Group (n=215) Mental Illness: 7% Phys. Disability: 5% Learning Disabled:

13% Neonatal Addiction: 1% Severe Behavior Prob.

9% JPPO* Involvement at Case Level 24%

* DHHS Division for Juvenile Justice Services Juvenile Probation & Parole Officers, involved due to child status offenses or delinquency.

Co-Morbidity in Co-Morbidity in Interview SampleInterview Sample

Victimization & Trauma Hx. Of AdultVictimization & Trauma Hx. Of Adult

0 20 40 60 80

Physical Attack Stranger

Physical Attack Know

Sexual Assault Know

Sex ssault stranger

Unwanted Sex Know

Mugged

Witness Killing

Chld. Phys. Ab.

Emotional Abuse

Ty

pe

Vic

tim

iza

tio

n

Percent of Respondents

Probability of Having a Substance Probability of Having a Substance Dependence DisorderDependence Disorder

36%

64%

0%

20%

40%

60%

80%

100%

Probability

High

Low

Co-MorbidityCo-Morbidity

45% of “high-probability” have prior diagnosis of mental illness.

45% of “high probability” have clinical levels of depression

18% prior hx of mental illness documented in initial record data.

45% Clinically Depressed using CESD measure.

Domestic Violence in Initial CPS Domestic Violence in Initial CPS Study ReferralsStudy Referrals

Over half (58%) had a prior Order of Protection at some time

Over 1/3 report DV in current year19% got a protective order on current

partner in the past

PROCESS PROCESS EVALUATIONEVALUATION

Implementation ChallengesImplementation Challenges Engagement of Client & Timing Randomization of LADC Services within Sites District Office Cultures Different Therapist Effects Attrition of LADC & CPSWs Need to build consensus between co-located

systems– Information sharing/confidentiality– Focus on “Primary Client”– Treatment recommendations from different points

of view– ASFA 12 month clock vs recovery clock

LADC: Challenges LADC: Challenges of Engagementof Engagement

Challenges of EngagementChallenges of Engagement

Correct Assignment of Subsequent Correct Assignment of Subsequent Referrals by Group & SiteReferrals by Group & Site

92.6%

99.1%

78.3%

88.6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Manchester Nashua

Enhanced

Standard

Percentage of Families Completed Percentage of Families Completed SASSI by SiteSASSI by Site

66%

54%

0%

10%20%

30%

40%

50%60%

70%

80%

90%100%

Pe

rce

nta

ge

of

Fa

mili

es

Co

mp

lete

d S

AS

SI

Site

Manchester

Nashua

Percent of Initial Referrals Founded Percent of Initial Referrals Founded by Groupby Group

13% 15%0%

50%

100%

Pe

rce

nt

of

Fa

mil

ies

Founded Dispositions

Enhanced

Standard

Final Founded Dispositions of Initial Final Founded Dispositions of Initial Referrals by Group & SiteReferrals by Group & Site

17% 13% 10% 17%0%

20%

40%

60%

80%

100%

Pe

rce

nt

of

Fa

mili

es

Manchester Nashua

Enhanced

Standard

Client Satisfaction w/ LADCClient Satisfaction w/ LADC

Manchester (n=27) Overall Satisfaction

Satisfied: 52% Overall Satisfaction by

Substantiation– No Case Ever Opened

Satisfied: 40%– Case Opened

Satisfied: 67%

Nashua (n=19) Overall Satisfaction

Satisfied: 90% Overall Satisfaction by

Substantiation– No Case Ever Opened

Satisfied: 92%– Case Opened

Satisfied: 86%

Mean Therapist Referrals for Clients Mean Therapist Referrals for Clients After Initial Eval. by Site:After Initial Eval. by Site:

0.01

0.11

0.08

0.28

0.33

0.19

0.4

0.21

0.08

0.06

0.06

0.07

0.13

0.13

0 0.1 0.2 0.3 0.4 0.5

Social Services

DomesticViolence

In-PatientServices

Out-PatientServices

Women'sHomeless Shelter

AA/NA

Mental HealthServices

Nashua

Manchester

1,2,3,4 statistical significance found between sites: ANOVA p > .05

1

2

3

4

Treatment UtilizationTreatment Utilization

Treatment Utilization by GroupTreatment Utilization by Group

12%

6%

6%

6%

12%

38%

26%

12%

10%

16%

20%

51%

0% 10% 20% 30% 40% 50% 60%

Home Based

DomesticViolence

Long-Term In-Patient

Short-Term In-Patient

Short-TermDetox

Mental Health

Enhanced

Standard

LADC: NetworkingLADC: Networking

LADC: NetworkingLADC: Networking

OutcomesOutcomes

Outcomes: Outcomes: Substance Abuse & Substance Abuse &

AssessmentAssessment

Preliminary Findings on System Status Preliminary Findings on System Status 1999: Substance Abuse Incidence & 1999: Substance Abuse Incidence &

Case OutcomeCase Outcome

Review of NH Cases for 1 month= 640 Cases 546 cases completed assessments Substance Abuse documented as a factor in

completed assessments =159/546 or 29% Proportion of substance abuse referrals

founded as cases=35/159 (22%)

Project First Step:Project First Step:Substance Abuse as a Factor in Substance Abuse as a Factor in

Completed AssessmentsCompleted Assessments 1999: Substance Abuse documented as a

factor in completed assessments – 159/546 or 29%

2004: Substance Abuse documented as a factor in completed assessments– 66% of Enhanced Group***

– 47% of Standard Group

Substantiation by High Probability of Substantiation by High Probability of Having a Substance Dependence Having a Substance Dependence

DisorderDisorder

45%

19%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

High Probability Low Probability

Case Open

* Significant statistical difference found between substantiation: Chi Sq 9.51, 1df,p=.002

Cases opened for continued services and/or placement directly supervised or paid by DCYF

Percentage of Cases Open on Percentage of Cases Open on Subsequent Referral by Group & SiteSubsequent Referral by Group & Site

20%

48% 46%40%

0%10%20%30%40%50%60%70%80%90%

100%

Manchester Nashua

Enhanced

Standard

*Significant statistical difference found between groups within Manchester: Chi Sq. 1-Sided p (.036) > .05

Enhanced

(n= 228)

Standard

(n= 219)

% CASES FOUNDED ON SUBSEQUENT REFERRALS

MEAN # SUBSEQUENT REFERRALS

46%

1.04

49%

1.20

Subsequent Referrals* by GroupSubsequent Referrals* by Group

*Referral: Report to NH DCYF based on a suspicion of child abuse or neglect. All NH citizens are mandated reporters.

Characteristics of Characteristics of Child PlacementsChild Placements

In-Home Services for Families with In-Home Services for Families with Cases by GroupCases by Group

IN-HOME SERVICES* Enhanced Standard

% FAM. ANY IN-HOME SERVICE

MEAN # FAM W/ IN-HOME SERVICE

# OF FAM. RECEIVING IN-HOME SERVICES W/ A VOLUNTARY/B-CASE

12%

0.14

2

16%

0.16

0

* “In home services” means services paid for and/or supervised by DCYF in founded cases after due process was provided or afforded. There are 8 families, in which one child received In-Home Services and was not removed, but another child within that same family was removed.

Out of Home Placement (OHP) Out of Home Placement (OHP) Types by GroupTypes by Group

Enhanced Standard

KIN CARE % FAM. ANY KIN CARE MEAN # FAM. W/ Children in KIN CARE

22%0.41

16%0.24

FOSTER CARE % FAM. ANY FOSTER CARE MEAN # Child Placements for FAM W/ FOSTER CARE Placements

63%1.78

62%2.72

OTHER OHP % FAM. ANY OHP MEAN # FAM W/ Children in OHP

26%0.82

28%0.68

Mean Number of Children in Placement per Mean Number of Children in Placement per Family by Group & SiteFamily by Group & Site

1.67

2.131.90 1.94

0

0.5

1

1.5

2

2.5

3

# o

f C

hild

ren

Manchester Nashua

Enhanced

Standard

Mean length of placement (days) per Mean length of placement (days) per child in placement by group & sitechild in placement by group & site

250 234319 281

0

50

100

150

200

250

300

350

Me

an

# o

f d

ay

s i

n p

lac

em

en

t

Manchester Nashua

Site

Enhanced

Standard

Mean Number of Placements per Mean Number of Placements per Child in Placement by Group & SiteChild in Placement by Group & Site

2.533.04

1.912.21

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

# o

f P

lac

em

en

ts

Manchester Nashua

Enhanced

Standard

Percentage of Families with TPR by GroupPercentage of Families with TPR by Group

18.0% 10.0%0%

20%

40%

60%

80%

100%

% o

f F

am

ilie

s

Enhanced

Standard

Percentage of Families with TPR by Percentage of Families with TPR by Group & SiteGroup & Site

12% 12% 8%

23%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

% o

f F

am

ilie

s

Manchester Nashua

Standard

Enhanced

Mean Length to TPR by GroupMean Length to TPR by Group

692 807

0

100

200

300

400

500

600

700

800

900

# o

f D

ay

s

Enhanced

Standard

ADULT OUTCOMESADULT OUTCOMES

W1 & W2 RX Utilization by Groups: W1 & W2 RX Utilization by Groups: Interview Sample, Self ReportsInterview Sample, Self Reports

Enhanced Standard

% Resp. attend AA

W1 (ever)

W2 (past yr.)

40% (45/112)

69% (19/28)

43% (43/100)

45% (10/22)

% Resp. help for drinking

W1 (ever)

W2 (past yr.)

25% (28/112)

48% (11/23)

32% (32/100)

43% (6/14)

% Resp. hospitalization for drinking

W1 (ever)

W2 (past yr.)16% (18/112)

27% (4/15)

18% (18/100)

11% (1/9)

W1 & W2 RX Utilization by Groups W1 & W2 RX Utilization by Groups Interview Sample: Reports on PartnerInterview Sample: Reports on Partner

Enhanced Standard

% Partner attend AA

W1 (ever)

W2 (past yr.)

31% (20/65)

62% (8/13)

44% (25/57)

46% (6/13)

% Part. help for drinking

W1 (ever)

W2 (past yr.)

25% (16/64)

25% (2/8)

22% (12/65)

43% (6/10)

% Part. hosp. for drinking

W1 (ever)

W2 (past yr.)

16% (9/64)

27% (4/15)

18% (8/54)

11% (1/9)

W1 & W2 Parent Outcomes: W1 & W2 Parent Outcomes: Interview Sample, Self ReportsInterview Sample, Self Reports

Enhanced Standard

% Clinically Depressed

W1

W2

39%

36%

40%

32%

% Heavy Drinking

W1

W2

29%

35%

53%

42%

% Hard Drug Hx

W1

W2

43%

44%

47%

48%

W1 & W2 Parent Outcomes:W1 & W2 Parent Outcomes: Interview Sample, Self ReportsInterview Sample, Self Reports

Enhanced Standard

% Employed FT

W1

W2

32%

38%

32%

24%*

%Enrolled Educ/Voc. Program

W2 28% 16%

Child OutcomesChild Outcomes

Child Outcomes Child Outcomes for Index Children ages 4-17for Index Children ages 4-17

Children in Enhanced Groups had greater declines in 7 of 8 problem categories: – Anxiety & Depression– Withdrawn/Depressed– Somatic Problems– Attention Problems– Aggressive Behavior

Child School & Health Child School & Health OutcomesOutcomes

Enhanced Group– Repeated Grade*

10%– Academic/Other

School Problems 39%

– MD concerns re: health 11%

Standard Group– Repeated Grade*

29%– Academic/Other

School Problems 43%

– MD concerns re: health 14%

Cost AnalysisCost Analysis

Project First Step: Comparison of costs by group.

$-

$200,000.00

$400,000.00

$600,000.00

$800,000.00

$1,000,000.00

$1,200,000.00

CUMULATIVE (CUM.) CTRL. - GROSS TOTAL (X 0.5 (B7c(e)))

CUM. EXPER. GRP. CNL - GROSS TOTAL

EX. Group + LADC

Shortfall EXPER. GRP. EXPEND. IN EXC. OF CNL - FFP TOTAL (B7d(e)) $280,000.00 (IV-E funds)

IV: Cost-Benefit Analysis

      - Costs to date $1,569,065.00      

- Major funding streams Title IV-E     

Project First Step: Comparison of costs by group.

$-

$200,000.00

$400,000.00

$600,000.00

$800,000.00

$1,000,000.00

$1,200,000.00

CUMULATIVE (CUM.) CTRL. - GROSS TOTAL (X 0.5 (B7c(e)))

CUM. EXPER. GRP. CNL - GROSS TOTAL

What Contributed to Higher What Contributed to Higher Costs?Costs?

Few differences between standard and enhanced

Prior referrals significantly higher among enhanced high cost group

JPPO referrals significantly greater among enhanced high cost group

More child mental health problems identified at intake for enhanced group

Cost Benefits Analysis by GroupCost Benefits Analysis by GroupIncluding LADC CostsIncluding LADC Costs

Avg. Cost/Child1

Savings – Child Not Removed2

Savings – Kin Care3

Total Avg. Cost/Child4

Total Avg. Savings5

Enhanced $23,709 $23,709 $5,952 $23,709 $29,660

Standard $20,951 $20,951 $4,920 $20,951 $25,871

ConclusionsConclusions

Key Outcomes– Significant Effects of Assessment– Strengths of Effects Diminished by Site

Differences– More Long-Term Substance Abuse Treatment of

Adults– Child Safety Outcomes: fewer subsequent

founded reports, more stability, decrease in time to TPR

– Improved Well Being for Adults & Children

LADC: ClosingLADC: Closing

Finale: LADAC clipFinale: LADAC clip