Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | aron-wilkerson |
View: | 216 times |
Download: | 0 times |
Update on Family Outcomes Work
Don Bailey and Melissa Raspa ECO Advisory Board Meeting
March 7, 2012
Goals for Today
• Provide update on family outcomes APR data• Introduce ECO family outcomes and
experiences measurement system self-assessment tool
• Discuss future of family outcomes
Early Childhood Outcomes Center
Part CMeasurement Approaches
• Tools– NCSEAM survey (n = 25)– ECO Family Outcomes Survey (n = 24)
• FOS-R (n = 3)
– State-developed survey (n = 6)• Methodology
– Census (n = 44)– Sampling (n = 11)
3Early Childhood Outcomes Center
Part CPopulation Surveyed
Timing All families Families > 6 months
Total
Point in time 25 12 37
At exit 3 6 9
At IFSP meeting 3 2 5
IFSP or exit 2 1 3
Total 33 21 54
4Early Childhood Outcomes Center
Part CResponse Rates
• Overall response rate = 36%• Tool
– ECO: 41%– NCSEAM: 31%– State: 31%
• Methodology:– Census: 37%– Sample: 34%
Part CResponse Rates
• Distribution– In person: 46%– Multiple methods: 37%– Mail: 27%
• Return– In-person: 75%– Multiple methods: 36%– Mail: 27%
6Early Childhood Outcomes Center
Part C Representativeness
Criteria States (N) States (%)
Race/ethnicity 41 75
Geography (district, county) 17 31
Gender 14 25
Child’s age (at referral, at survey) 13 24
Disability/eligibility category 6 11
Length of time in services 6 11
Income 3 5
7Early Childhood Outcomes Center
Part CAPR Trend Data
4A Know rights
4BCommunicate
needs
4CDevelop and learn
0102030405060708090
100
80 8187
81 8388
84 8590
86 8690
FFY 06-07FFY 07-08FFY 08-09FFY 09-10
Early Childhood Outcomes Center
Part CTrend Data – Indicator 4A
FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100
5
10
15
20
25
30
> 50%50-59%60-69%70-79%80-89%90-100%
9Early Childhood Outcomes Center
Part CTrend Data – Indicator 4B
FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100
5
10
15
20
25
30
35
> 50%50-59%60-69%70-79%80-89%90-100%
10Early Childhood Outcomes Center
Part CTrend Data – Indicator 4C
FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100
5
10
15
20
25
30
50-59%60-69%70-79%80-89%90-100%
11Early Childhood Outcomes Center
Part CAPR by Survey Tool
4AKnow rights
4BCommunicate
needs
4C Develop and learn
0102030405060708090
10088 91 91
82 808989 89 91
ECONCSEAMState
12Early Childhood Outcomes Center
Part BMeasurement Approaches
• Tools– NCSEAM (n=43)– Adapted NCSEAM or ECO (n=8)– State (n=8)
• Methodology– Census (n=21)– Sampling (n=35)
13Early Childhood Outcomes Center
Part BResponse Rates
• Overall response rate = 24%• Distribution
– Multiple methods: 30%– Mail: 17%
• Return– In-person: 75%– Multiple methods: 36%– Mail: 27
14Early Childhood Outcomes Center
Part BAPR Trend Data
15Early Childhood Outcomes Center
Next Steps
• APR analyses for FFY 2010-2011• Examine data quality
– APR by response rates• Develop template for states to share family
outcomes data• Provide TA to states on as-needed basis
16Early Childhood Outcomes Center
FAMILY OUTCOMES AND EXPERIENCES MEASUREMENT SYSTEM
SELF ASSESSMENT TOOL
Early Childhood Outcomes Center 17
Purpose
• Provides a common language and organizing structure for discussing family outcomes and experiences
• Assist states in developing a measurement system that captures various aspects of family outcomes and experiences, such as– benefits that families receive from program participation– family satisfaction with services– perceived helpfulness of the services– family involvement with the service delivery system
18Early Childhood Outcomes Center
Framework and Self-Assessment
• FRAMEWORK– Set of components and quality indicators– Provides the structure for the self-assessment
• SELF-ASSESSMENT – Scale that provides criteria for levels of
implementation within each quality indicator– Rating assigned based on level of
implementation within each indicator 19
Early Childhood Outcomes Center
Partner States
• Connecticut• Illinois• Minnesota• Texas
20Early Childhood Outcomes Center
Framework
• Components– 7 key areas of a measurement system
• Quality indicators– 16 specific items that further describe each of
the components• Elements
– Various examples of what constitute quality at the indicator level
21Early Childhood Outcomes Center
22
Purpose
Data Collection and Transmission
Analysis
Reporting
Using Data
Evaluation
Cross-System Coordination
Components Quality Indicators
Purpose
1. State has articulated purpose(s) of COMS.
Data Collection and Transmission
2. Data collection procedures are carried out efficiently and effectively.
3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment.
4. State's method for entering, transmitting, and storing data is effective and efficient.
Analysis
5. State identifies accountability and program improvement questions related to child outcomes.
6. Local programs identify accountability and program improvement questions related to child outcomes.
7. State agency analyzes data in a timely manner.
8. Local programs analyze data in a timely manner.
9. State agency ensures completeness and accuracy of data.
Reporting
10. State agency interprets, reports, and communicates information related to child outcomes.
11. Local programs interpret, report, and communicate information related to child outcomes.
Using Data
12. State agency makes regular use of information on child outcomes to improve programs.
13. Local programs makes regular use of information on child outcomes to improve programs.
Evaluation14. State evaluates its COMS regularly.
Cross-system Coordination
15. Part C and 619 coordinate child outcomes measurement.
16. Child outcomes measurement is integrated across early childhood (EC) programs statewide.
17. Child outcomes measurement is aligned with state’s early learning guidelines/standards.
18. State has a longitudinal data system to link child outcomes data from EC program participation to K–12 data.
Elements
a. State has…b. State has…c. State agency..d. Representative..e. State agency…f. State ……g. State provides…h. State has..
Self-Assessment
• Provides guidance to states on what constitutes a high quality family experiences and outcomes measurement system.
• Assists states in setting priorities for improving their measurement system
• Provides information to assist states in advocating for resources for systems development
23
How Does it Work?
• Each quality indicator has multiple elements• Evidence for the extent of implementation for each
element is provided• Each element is rated as
• NY = Not Yet or Don’t know• IP = In Process• IF = Fully Implemented• NA = Not applicable (only where applicable)
• The quality indicator is given a rating based on the ratings of the elements
Early Childhood Outcomes Center 24
Self-assessment Scoring
25
Implementation of ElementsQuality
Indicator Score
All elements are fully implemented 7
Nearly all elements are fully implemented and the rest are in process 6
Most of the elements are fully implemented and the rest are in process. 5
At least one element is fully implemented and the rest are in process 4
All of the elements are in process 3
Some of the elements are in process 2
None of the elements are yet in process 1
Challenges with Family Framework and Self-assessment
• Going beyond the APR: Family outcomes and experiences
• Balancing expectations: Reality vs. ideal • Scoring system
– Difficulty in reaching 7– Not applicable items
Early Childhood Outcomes Center 26
Next Steps
• Finalize framework and self-assessment tool and post on ECO web site
• National conference call to introduce to broader audience
• Provide TA to states on as-needed basis
Early Childhood Outcomes Center 27
WHAT IS THE FUTURE OF FAMILY OUTCOMES?
Context
• Family-centered (FC) practices are endorsed by many professional organizations as central to high-quality practice– Institute of Medicine– American Academy of Pediatrics– American Hospital Association– American Speech-Language Hearing Association– Division for Early Childhood
We know….
• Meta-analyses of research provides strong evidence that FC practices lead to many benefits for families and children
Dunst, Trivette, & Hamby (2007)
• Meta-analysis of 47 studies of family-centered practices in a variety of settings
• Higher use of FC practices was associated with:– Satisfaction with services*– Self-efficacy beliefs*– Parenting behaviors– Personal and/or family well-being– Social support*– Child behavior and functioning
*Strongest associations
Pitrowski, Talavera, & Mayer (2009)
• Meta-analysis of 13 articles evaluating Health Steps, a pediatric care model based on– strength-based primary prevention– the assumption that child well-being and family-centered care are
necessarily linked– supporting the role of parents– positive, caring, and enduring relations with parents.
• Parents in HS were– More satisfied with services– Exhibited more desirable parenting practices– More satisfied in their role as parents– Interacted more sensitively and appropriately with child
Broom & Enriquez (2009)
• Reviewed 9 randomized trials of family-centered interventions in treating children with Type 1 diabetes
• Family-centered practices were associated with– Improved health outcomes for children– Reduced diabetes-related conflict in families– Improved family relations
Two studies using the National Survey of Children with Special Health Care Needs
• When families perceived a strong partnership between the child’s family and the child’s service provider– children’s health outcomes were improved– children received more services– families were more satisfied with services
Also
• Our NEILS analyses and our ECO papers show that family-centered practices are related to family outcomes
Epley, Summers & Turnbull (2011)
But research also shows that FC practices are not widely implemented
• Medical Expenditure Panel Survey and National Survey of CSHCN children found that only about 2/3 of families reported getting family-centered care
• A hospital survey found that only about half used FC rounds in pediatric in-patient settings
• Minority families generally report fewer FC services and less satisfaction
Dunst (2012) argues that EI has not evolved in tandem with what we know
So, if…..
• Professional organizations widely endorse FC practices, and…
• Research shows that FC practices make a difference…
• What should we do about the future of FC practices and family outcomes in early intervention and preschool programs?
Use of Family-Centered Practices
• We don’t know at a state or national level the extent to which programs are actually using family-centered principles and practices
• We don’t know what factors predict or are associated with variability in implementation of F-C practices
• NEEDED: A comprehensive description of the status of F-C practices in EI and preschool programs, and a clear understanding of why some programs are more family-centered than others
Family-centered principles, when integrated across all program activities, ought to lead to
family outcomes.
Referral and intake Determining eligibility Child assessment Family assessment Team meetings and
decision-making Service provision and
coordination Parenting practices Community support
Understand child’s abilities and special needs
Know rights and advocate effectively
Help child develop and learn
Have support systems Access the community
Family-Centered Principles
Program Activities Family Outcomes Shared philosophy Families as partners Focus on strengths Family choice of goals
and services Collaboration and
coordination of service Effective communication Flexibility Community-based
What is it about F-C practices that leads to better outcomes?
• With the exception of the NEILS study and other smaller studies, we don’t know the full range of outcomes that F-C practices can affect
• The pediatric literature has far more studies of this than EI
• NEEDED: Sophisticated large-scale studies modeling how and which F-C practices influence a variety of outcomes
The desirability & applicability of F-C practices & family outcomes in preschool
• Nothing is known about F-C practices in preschool programs for children w/disabilities
• The ECO family outcomes have never been accepted or seriously discussed for preschool
• We have some evidence that family outcomes are less likely to be achieved with older children
• NEEDED: Stakeholder opinions and consensus regarding the desirability of FC practices & appropriate family outcomes in preschool
F-C practices and child outcomes
• Is helping families a good outcome in and of itself, OR must we show that this results in benefits for children?
• We have danced around this topic for 20 years, with no resolution
• NEEDED: Either we need consensus on the argument that family benefit is a worthwhile outcome OR we need to show that F-C practices and family outcomes are important to maximize child benefit
Potential Year 5 Family Activities
• Help states use framework to improve systems for family outcomes assessment
• Targeted TA to states with low family helpfulness ratings
• Collect, compile and analyze data from states that have FOS data that can be linked with child outcomes data
• Explore with one or two states or programs use of FOS in preschool programs