How does Cross-System Collaboration Between the Child Welfare and

Mental Health Systems Influence Children’s Mental Health Service Use and Service Effectiveness?

Findings From a Two-Phase, Mixed Methods Study

The Leadership Symposia on Evidence-Based Practice in Human Services

January 30, 2009San Diego, California

Kathy Lemon Osterling Ph.D., MSWAssistant Professor

School of Social WorkSan Jose State University

kathy.lemon-osterling@sjsu.edu

Study Contributors Research and Curriculum:

Alice M. Hines, Ph.D. Principal Investigator Peter Allen Lee, Ph.D. Co-Investigator Kathy Lemon Osterling, Ph.D. Co-Investigator Marty Tweed, MSW, Curriculum Consultant

Santa Clara County Advisory Committee: Judi Boring, formerly from Santa Clara County DFCS Judy Bushey, County consultant to CA DSS Amando Cablas, Ph.D., Santa Clara Valley Health &

Hospital System, Community Health Services Doug Klinkerman, Santa Clara Valley Health &

Hospital System, Information Systems

Purpose of Workshop Describe factors that help or hinder cross-system

collaboration between public child welfare and mental health systems

Describe the influence of children’s mental health service use on family reunification and family maintenance

Discuss the potential role of cross-system collaboration in improving mental health service use and effectiveness for children involved in both systems

Discuss how study findings can be used to promote evidence-based/evidence-informed practice

Background: Why is Cross-System Collaboration Important?

1) High need for mental health services among children involved with the child welfare system:

Experiences of trauma, maltreatment, being removed from caregivers1

50%-80% have a mental health diagnosis vs. 25% of the general child population2

Background: Why is Cross-System Collaboration Important?

2) Mental health services for the child may improve family reunification outcomes

When children are experiencing mental health problems, reunification may be seen by professionals overseeing the case as having the potential to worsen children’s mental health symptoms3

Research suggests that children’s emotional problems and externalizing behavioral problems are linked with a reduced likelihood of reunification, and an increased likelihood of reentry into out-of-home placement3,4

Background: Why is Cross-System Collaboration Important?

3) Both systems share the same client (the child) and have the same overall goal of improving child well-being

Mental health interventions are among the most commonly used service by children involved in the child welfare system5

Background: Why is Cross-System Collaboration Important?

4) Strong cross-system collaboration may improve mental health service delivery and effectiveness of services

Effective communication between systems improves assessment and service planning6

Coordinated case plans ensure that both systems are working toward the same goals

Family reunification timelines (ASFA) create a need for timely access to mental health services for children

Purpose of Study

1) Examine the influence of public mental health services on family reunification and family maintenance outcomes among children involved with the child welfare system (Quantitative)

2) Explore factors that impede or enhance cross-system collaboration between the child welfare and mental health systems (Qualitative)

Literature Review: Collaboration

General definition of collaboration: Partnerships (i.e. relationships) that

promote mutually beneficial goals, that often cannot be achieved when individuals act alone.

Literature Review: What are Critical Practice Areas in Collaboration?

The Ten Bridges7 Screening and assessment Engagement and retention Services Building community supports Values Budgets and program sustainability Information systems and outcomes Joint accountability and shared outcomes Training and staff development Links to other agencies

Literature Review: Theoretical Framework

Collaboration occurs at multiple system levels8

Mezzo Level: Differing professional orientations and overlapping, but distinctsystem structures and goals

Macro Level: Organizational support in the form of resources, funding and planning processes to guide collaborative efforts

Micro Level: Characteristics & interpersonal processes that affect establishment of trust between individuals

Literature Review: Theoretical Framework

Collaboration is a developmental process9

Literature Review: Previous Research

One study using data from NSCAW found that increased coordination (defined as “concrete linkages” such as co-location of services, joint training, formal committees, shared office space, & coordinated service provision) between child welfare and mental health services was associated with:

1) Stronger associations between service need and service use and

2) A reduction in racial/ethnic disparities in service use10

Literature Review: Previous Research

However, another study using data from 24 counties in Tennessee found that increases in service coordination were actually linked to decreases in service quality (i.e. comprehensiveness, availability, continuity and responsiveness)11

Possible interpretations: diffusion of responsibility, and a poorly structured collaboration (service coordination teams not directly working with families)

Literature Review: Previous Research

The quality and structure of the collaboration are important

Wraparound and System of Care models of service delivery are linked with positive outcomes (improved functioning of child and family and less restrictive living situations)12

However, most children involved in child welfare system receive outpatient mental health counseling

Qualitative Methods: Research Questions

Phase I: What factors impede or enhance collaboration between the child welfare and mental health systems?

Phase 2: Prior to and during early implementation of the Mental Health Services Act, what factors impede or facilitate collaborative practice between the child welfare and mental health systems, with a particular focus on transition age youth?

Qualitative Methods: Research Design, Sampling Procedures & Sample

Research Design: Case study Sampling Procedures: Combination of random

sampling and convenience sampling Sample: Over the 2 phases:

Total of 15 interview participants and 25 interviews (10 participants were interviewed twice)

Interview participants included 6 child welfare, and 6 mental health program managers or supervisors and 3 managers from community-based organizations

Total of 6 focus groups, 3 with child welfare line workers, and 3 with mental health line workers

Qualitative Methods: Data Collection, Data Analysis, Reliability & Validity

Data Collection: Semi-structured interview guide & face-to-face interviews

Data Analysis: Analysis involving multiple reviews of transcripts to identify main themes and process of creating coded categories to understand findings

Reliability & Validity: Advisory group served as peer reviewers, discussion and review of coding process by research team

Quantitative Methods: Research Questions Phases 1 and 2

1) What are the characteristics of children and youth involved in the child welfare system and the public mental health system?

2) Among children and youth involved in the child welfare system and the mental health systems, what is the influence of outpatient mental health services for children on family reunification and family maintenance outcomes?

Research Design: Correlational study with secondary data from a merged administrative dataset

Sampling Procedures: Phase 1: All children and youth whose child welfare case was

closed over a 12-month period (Jan-Dec 2004) Phase 2: All youth between the ages of 15 and 19, whose child

welfare cases were closed over an 18-month period between the ages of 15 and 19 (Nov 2005-April 2007)

Sample: Phase 1: N = 1,127 (total sample), N = 519 (received public mental health services, 46%)

Sample: Phase 2: N = 638 (total sample), N = 147 received public mental health services, 23%)

Quantitative Methods: Research Design, Sampling Procedures & Sample

Quantitative Study Methods: Measurement of Key Variables

Independent Variables: Outpatient mental health services for child or youth (may include individual,

group, or family therapy, or psychiatric medications) Mental health treatment completion

Dependent Variable: Case closure type of family stabilized: Family reunification and family

maintenance Other Variables:

Demographic variables Gender Race/ethnicity

System-Related Factors Type of maltreatment Age at entry into the CWS Number of hours of outpatient mental health services

Type of Mental Health Problem: Adult-type disorder (e.g. mood disorder) Adjustment disorder Childhood disorder

Quantitative Study Methods: Reliability and Validity, Data Analysis

Reliability & Validity: No formal tests of reliability of variables because data came

from administrative databases Operational definitions of variables were developed and

verified with input from Advisory Group

Data Analysis: Binary statistics to compare characteristics of children and

youth in both the child welfare and mental health systems compared to those only in child welfare

Logistic regression to test influence of outpatient mental health treatment completion on family stabilization, while controlling for the influence of demographic variables, system-related factors and type of mental health problem (analysis using sub-sample of children receiving outpatient services)

Qualitative Results

Challenges in Collaboration

Divisions in System Goals,

Mandates & Policies: Both Phases

Budget or Resource Problems:

Both Phases

Communication & Service

Coordination Problems:

Only Phase 1

Workload: Only

Phase 2

Qualitative Results: Divisions in System Goals, Mandates & Policies (Both Phases)

Child Welfare:

“There are inherent tensions. In mental health, they usually rely on the client to identify issues. In child welfare, we have specific issues that we feel need to be addressed because of timelines” (Program Manager, Phase 1).

“We have different mandates. We [child welfare system] are enforcers. There are different cultures because of this; they are more the ‘good guys.” (Program Manager, Phase 1).

Mental Health:

“Child welfare deals with the safety of the child and the desire to have the child be adjusted and returned to a normalized living environment. Sometimes there is an under emphasis on mental health problems and the [child welfare] system can create problems” (Program Manager, Phase 2).

“We just have different needs and different perspectives on the system—legal vs. mental health” (Line Worker, Phase 1).

Child Welfare:

“There are no opportunities for line staff to communicate with one another. There is no annual or bi-annual communication forum to talk about what each system is doing” (Program Manager).

“If a therapist is working with a family, they should meet regularly with the social worker and develop treatment goals and discuss how needs are being met. The way it is now is the therapist sends a written report to the court and doesn’t talk to the social worker; there is no mechanism to talk together” (Supervisor).

Mental Health:

“There’s a lack of structure. I’ve never been in a meeting with social workers” (Line Worker).

“Social workers may be under pressure to address certain therapeutic goals that might be set up by attorneys or judges—not by the therapist or the clinicians, and it makes it difficult, because we may not necessarily agree” (Line Worker)

Qualitative Results: Communication & Service Coordination Problems (Phase 1)

Qualitative Results: Budget or Resource Problems (Both Phases)

Child Welfare:

“For families in the voluntary [services], they often only have services for three months, and we sometimes have to spend a lot of time getting them Medi-Cal, and then there is often a waiting list to see the mental health worker, so we can eat up the time that they have just trying to get them services” (Line Worker, Phase 1)

“The parents cannot get counseling if they do not have custody of the child [because of funding restrictions], but they cannot get their children back unless they get counseling” (Line Worker, Phase 1)

Mental Health:

“The budget cuts are really impeding the collaborations, especially for mental health. [The child welfare agency] didn’t really get a cut because social services took it out of other parts of the organization. It’s really overwhelming. I’m personally spending about 40-50% of my time on dealing with the budget cut” (Program Manager, Phase 2)

“The budget reduction is huge…We can’t talk about the future when it’s all about how many people we’re laying off. I see despair at all levels” (Program Manager, Phase 2)

Qualitative Results: Workload Stressors (Phase 2)

Child Welfare:

“Collaboration is difficult because workers are under pressure to provide services. Child welfare workers have to complete court reports and there is very little time to be creative and look for resources. Trying to take time to pursue resources might be a luxury. Workers get in the habit of using the top 3, 4, or 5 resources they use all the time” (Supervisor).

“We try to structure collaboration into work time, but there is always a crisis” (Supervisor).

Mental Health:

“Time is a big impediment to collaboration. Especially with Mental Health, we’re always ready for the unexpected. We have that mindset of always being prepared, which is a distraction to thinking about the future because we’re always putting out fires” (Program Manager)

Qualitative Results

Infrastructure to Support communication

& Service Coordination: Both Phases

Service Integration:

Both Phases

Support & Planning from organizational Leaders: Only Phase 1

A High Value Placed on

Collaboration Among

Workers & in Organization: Both Phases

Factors that Facilitate Collaboration

Qualitative Results: Infrastructure to support communication & Service Coordination (Both phases)

Child Welfare:

“It would be beneficial if all the professionals involved in the youth’s life could hear all the concerns. This already happens but more is needed. It’s hard to do with two different departments. Everyone is invited but not everyone comes. Maybe making this mandatory would help” (Supervisor, Phase 2).

“We have an educational rights specialist to advocate for kids who aren’t getting what they need in the educational system. Having something similar in mental heath would help a lot” (Line Worker, Phase 1).

Mental Health:

“If the institution had some mechanism for collaboration such as meetings when there is an opportunity for lining up policy, programming and practices in a way that will allow for better collaboration at the line staff level” (Line Worker, Phase 1).

“We need consistent meetings from all parties involved, from the very beginning. Problems can be solved if we set up mechanisms to do that” (Line Worker, Phase 1).

Qualitative Results: Service Integration (Both Phases)Child Welfare:

“The location of the mental health unit at the shelter helps with collaboration. When you have someone on site that is a big factor” (Line Worker, Phase 1).

“What we should do is band together to go after funding sources. Instead of wasting our resources going after the same pot, we should get together and discuss who is best fitted to receive a certain funding, and then we should all get behind that one agency” (Program Manager, Phase 1).

“The need for cross-training is bad. A lot of mental health workers don’t understand the court process and how difficult that can be” (Supervisor, Phase 1).

Mental Health:

“I think we need to have more shared staff, more co-located staff, more co-managers, even merging departments” (Program Manager, Phase 2).

“Some people who work in mental health have very little idea of how DFCS works. Training from DFCS on what drives their system, their legal mandates, safety issues, and court issues. This would create a better appreciation” (Program Manager, Phase 1)

Qualitative Results: Support & Planning From Organizational Leaders (Phase 1)

Child Welfare:

“It flows from the Board of Supervisors and county executives to make collaboration a value in our county so that it is monitored” (Supervisor, Phase 1)

Mental Health:

“We need better coordination, the top levels need to establish policies” (Line Worker, Phase 1).

“Having a strong, committed Board of Supervisors helps collaboration—they say they want collaboration and we do it. Having the support of executive directors is very important. They have power and can make things happen” (Program Manager, Phase 1).

Qualitative Results: A High Value Placed on Collaboration Among Workers & in the Organization (Both Phases)

Child Welfare:

“We’re selling ourselves short by not seriously trying to collaborate…We could make a huge dent on our social problems by collaborating” (Program Manager, Phase 1).

“We need to see each other as going down the same path” (Program Manager, Phase 1).

Mental Health:

“We both have the same objective, so how can we make our services blend? How can we form a partnership so that we can bring resources, funding streams, and so forth to the table and then provide the services that we are good at providing. There’s a lot to gain from collaboration” (Program Manager, Phase 2).

[Collaboration can be facilitated by] “realizing that the two populations we serve are one” (Program Manager, Phase 2).

Quantitative Results: Children and youth referred to mental health system were older at entry into the child welfare system

8.97 yrs.

13.39 yrs.

8.07 yrs.

5.08 yrs.

0

2

4

6

8

10

12

14

16

Phase 1: All Ages Phase 2: 15-19 yrs.

Mental Health Services No Mental Health Services

28.0%26.6%

17.6%

12.4%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

Phase 1: All Ages Phase 2: 15-19 yrs.

Mental Health Services No Mental Health Services

Quantitative Results: Higher rate of physical abuse for children referred to mental health services

Quantitative Results: Higher rate of sexual abuse for children referred to mental health services

8.5%

11.7%

2.6% 3.0%

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

Phase 1: All Ages Phase 2: 15-19 yrs.

Mental Health Services No Mental Health Services

Quantitative Results: Adult-type disorder most common

39%

57%

27%

19%18% 19%

0%

10%

20%

30%

40%

50%

60%

Phase 1: All Ages Phase 2: 15-19 yrs.

Adult-Type Disorder Adjustment Disorder Childhood Disorder

18.80 hrs.

1.56 hrs.

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

20.00

Phase 1: All Ages Phase 2: 15-19 yrs.

Hours of Outpatient Mental Health Services

Quantitative Results: Number of hours of outpatient services low for transition age youth

Quantitative Results: Influence of outpatient mental health services for children on family

reunification and family maintenance outcomes

Among children and youth participating in outpatient mental health services, those who completed mental health services were not more likely to have a family stabilization outcome than children and youth who did not complete outpatient mental health services.

Summary & Interpretation

Study Finding:

High need for mental health

services, especially for youth entering

the CWS at an older age, those having

experienced physical or sexual abuse, and those

with Adult-type disorders

Study Finding:

Outpatient mental health services not influencing

family stabilization outcomes

Study Finding:

Number of hours of outpatient

services low for transition age

youth

Study Finding:

Challenges in cross-system collaboration between child welfare and

mental health systems

Interpretation:

Reduced access to services,

possibly influenced by

lack of resources &

system infrastructures

Interpretation:

Reduced effectiveness of

services, possibly

influenced by difficulties with joint treatment

planning

Practice & Policy Implications

Study Finding:

Numerous factors that

facilitate collaboration

were identified

Practice/Policy Implication: Identify joint

funding projects

Practice/Policy Implication:

Cross-training on system goals,

policies, & mandates

Practice/Policy Implication:

Continue to place a high value on

the importance of collaboration

Practice/Policy Implications: Co-locating mental

health and child welfare staff

Interpretation:

Improved access to

services and improved

effectiveness of services

Limitations Cross-system collaboration is not measured in the

quantitative study Small sample size for qualitative study Administrative data does not allow for measurement of

other important variables (e.g. type of therapeutic intervention, quality of therapeutic relationship, alleviation of symptoms, etc.)

Quantitative methods are correlational--although several variables are statistically controlled for in the multivariate models (demographic, system-related, type of mental health problem), this study did not use experimental methods to test the effectiveness of mental health interventions (e.g. “randomized controlled trial”)

Study conducted in one county More research is needed to address these limitations

Discussion Questions

How do child welfare and mental health systems collaborate in your county?

A number of factors that help collaboration or hinder collaboration were discussed in this presentation. Have you observed these factors operating in your own county? What other factors influence collaboration? Are there differences in collaboration challenges depending on what systems are collaborating (e.g. substance abuse, criminal justice, public assistance, etc…)? (see handout on 2-phase study on collaborative practice between child welfare and substance abuse systems, ).

How can this research be used to improve practice in your county? How can these findings be used to promote evidence-based or evidence-informed collaborative practice between child welfare and mental health systems?

References 1 Halfon, N., Zepeda, A., & Inkelas, M. (2002). Mental health services for children in foster care. UCLA Policy

Brief.

2 Halfon, N., Mendonca, A., & Berkowitz, G. (1995). Health status of children in foster care: The

experience of the Center for the Vulnerable Child. Archives of Pediatric and Adolescent

Medicine,149, 386-392.

U.S. Department of Health and Human Services (1999). Mental Health: A Report of the Surgeon

General. Rockville, MD: Author.

3 Landsverk, J., Davis, I., Ganger, W., Newton, R., & Johnson, I. (1996). Impact of child psychosocial functioning

on reunification from out-of-home placement. Children and Youth Services Review, 18(4/5), 447.462.

4 Jones, L. (1998). The social and family correlates of successful reunification of children in foster care.

Children and Youth Services Review, 20(4), 305-323.

5 Hines, A. M., Lee, P. A. Osterling, K. L., & Drabble, L. (2007). Factors predicting family reunification

for African American, Latino, Asian, and White families in the child welfare system. Journal of Child and Family

Studies, 16(2), 275-289.

6 Prince, J., & Austin, M. J. (2005). Inter-agency collaboration in the child welfare and mental health

systems. Social Work in Mental Health, 4, 1-16.

7 Young, N.K. & Gardner, S.L. (2002) TAP 27:  Navigating the Pathways:  Lessons and Promising Practices in

Linking Alcohol and Drug Services and Child Welfare.  Washington DC:  Substance Abuse and Mental

Health Services Administration

References

8 Whittington, C. (2003). A model of collaboration. In J. Weinstein & C. Whittington & T. Leiba (Eds.),

Collaboration in social work practice (pp. 39-61). London: Jessica Kingsley Publishers.

9 Gardner, S. L. & the National Center on Substance Abuse and Child Welfare (2008). Materials describing

developmetal process of collaboration: http://www.cffutures.com/presentations.shtml

10 Hurlburt, M. S., Leslie, L. K., Landsverk, J., Barth, R. P., Burns, B. J., Gibbons, R. D., Slymen, D. J., Zhang,

J. (2004). Contextual predictors of mental health service use among children open to child welfare.

Archives of General Psychiatry, 61, 1217-1224.

11 Glisson, C. & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational

coordination on the quality and outcomes of children’s service systems. Child Abuse and Neglect, 22(5), 401-421.

12 Bruns, E., Stuer, J., Force, M., & Burchard, J. (2005). Adherence to wraparound principles and

association with outcomes. Journal of Child and Family Studies, 14(4), 521-534.

Burns, B.J., Schoenwald, S.K., Burchard, J.D., Faw, L, & Santos, A. (2000). Comprehensive

community-based interventions for youth with severe emotional disorders:

multisystemic therapy and the wraparound process. Journal of Child and Family Studies, 9(3),

283-314.