Post on 15-Dec-2018
transcript
Supporting Implementation of Evidence-Based
Practices in the Early Childhood Classroom
Through Performance-Based Coaching
Patricia Snyder, University of Florida
Lise Fox, University of South Florida
Mary Louise Hemmeter, Vanderbilt University
Coaching
• Create a supportive environment for learning
• Jointly examine and reflect on current practices
• Help to apply new skills
• Within a supportive context, give specific feedback & problem-solve challenging situations
• Goal: Assist person to attain competence and confidence to engage in action, self-reflection, self correction; use new skills and strategies fluently; and use or adapt new skills and strategies in other situations, as appropriate
(Flaherty, 1999; Kinlaw, 1999; Rush & Shelden, 2006)
Performance-Based Coaching
• Practices are made specific and explicit for
teachers and coaches
• Incremental application of practices with
individualized implementation support
(structured around goal setting and action
planning)
• Structured feedback protocol
– Reflective
– Supportive
– Corrective
Coaching for Implementation
ID Focus Evidence-based practice
Practitioner needs in relation to competence
and confidence about practice
Learning about the
“practice or intervention”
Active and passive methods
Multiple exemplars
Concrete, specific
Observe Needs assessment
Set goals and develop action plan
Coach Rapport and collaboration
Active methods
Performance feedback
Observe Needs assessment
Revise goals and action plan
Measure implementation Fidelity of implementation measures
Measure outcomes Practitioner/”setting” measures
Child measures
Supporting Implementation of
Evidence-Based Practices
Evaluating the Potential Efficacy of the Teaching Pyramid Model
Helping early childhood practitioners implement promotion,
prevention, and intervention strategies that support young
children's social-emotional competence and prevent
challenging behavior
Institute of Education Sciences:
R324A07212- Examining the potential efficacy of a classroom-wide model for promoting
social-emotional development and addressing challenging behavior
Helping early childhood
practitioners implement
embedded instruction for
preschool children with
disabilities
Institute of Education Sciences:
R324A070008 - Impact of professional
development on preschool teachers’
use of embedded-instruction practices
Evaluating the Potential Efficacy of Embedded Instruction for Early Learning
Two Potential Efficacy Trials Using
Performance-Based Coaching
Teaching Pyramid intervention = IV
PD to learn about and support
implementation of TP practices
PD provided in TPI condition:
• Workshops (19.5 hr),
implementation guides,
materials, on-site coaching
PD Intervention = IV
Content focus = Embedded
Instruction
PD intervention conditions:
• Workshops (14.9 hr),
implementation guides,
materials, on-site coaching
• Workshops, implementation
guides, materials, web-based
self coaching
• BAU PD
Unpacking Coaching
To make statements about the impact of performance-based coaching, we need
• Coaching framework
• Define components of framework
• Identify phases of coaching
• Coaching protocol for each component
• Defined coaching strategies
• Adherence data
• Dosage or exposure data
• Information about “practice” content of coaching sessions
• Criteria for practice implementation fidelity
Phases of Coaching • Early Sessions
– Entry and building rapport
– Observation
– Needs assessment and initial goal setting
– Debrief and Feedback • Reflection
• Supportive feedback only*
• Latter Sessions – Continue building rapport
– Observation
– Goal setting
– Debrief and feedback • Reflection
• Supportive and corrective feedback**
• Final Session – Progress to date and remaining goals
– Strategies for sustaining
– Additional resources
– Evaluation of coach and coaching processes
Protocol for Goal Setting and
Action Planning • “Discrepancy” needs assessment
• Where am I now with respect to practice?
• Where do I want to be?
• What are logical next steps related to
implementation of practice?
• Identify initial goals and develop action
plan
• Recursive process after initial goals and
action plan are implemented
Protocol for Observation
• Observe
• Interact (if agreed to ahead)
• Make Observation Notes
• Videotape*
Protocol for Debrief and Feedback
• Open meeting
• Summarize observation and encourage reflection
• Supportive performance feedback
• Corrective performance feedback
• Targeted support
• Support for planned actions (goal setting, if needed)
• End meeting
Define Coaching Strategies
• Observing
• Problem-solving discussion
• Goal setting/planning
• Reflective conversation
• Role-play
• Performance feedback
(verbal)
• Performance feedback
(graphic)
• Side-by-side gestural
support
• Side-by-side verbal support
• Modeling
• Videotape teacher
• Video demonstration
• Review teacher’s video
• Graphing
• Provide materials or
resources
• Environmental
arrangements
• Other help in classroom
Coaching
Adherence:
Example from
Fidelity Checklist
Example from Embedded Instruction for Early Learning
Dose: On-Site
Length of coaching = 12-16 weeks
Mean number of sessions = 13
(range 7 to 17)
Mean session duration (min):
Early sessions
• Observation = 144 (R = 85-205)
• Debrief = 44 (R = 10-135)
Sessions 3 and Beyond
• Observation = 105 (R= 30 – 305)
• Debrief = 38 (R = 15- 105)
Length of coaching = 13-16 weeks
No of sessions = 16 (13 sessions
for 1 teacher, all others 16
sessions)
Mean session duration:
Early sessions
• Observation = 75 min (SD = 25)
• Debrief = 40 min (SD = 12)
Latter sessions
• Observation = 72 min (SD = 14)
• Debrief = 39 (SD = 12)
Embedded: Procedural Fidelity
Coaching Orientation
(n = 12)
Early
(n = 24)
Latter
(n = 65)
(n = 76)
Final
(n = 12)
All
Sessions
(n = 189)
Coach report
% coaching log
indicators
M (SD)
98.6
(2.1)
96.7
(3.7)
98.1
(2.7)
98.5
(3.7)
100.0 98.2
(3.2)
No. of sessions
with second
observer
4 5 15 25 4 53
Second observer
% coaching log
indicators
M (SD)
100 91.8
(9.2)
95.7
(3.4)
96.3
(4.9)
97.9
(4.2)
96.1
(5)
25
TPOT:
Procedural Fidelity: Coaching
Early Sessions
(n = 40)
Sessions 3 and
Beyond
(n = 219)
(n = 204)
Closing
Sessions
(n = 10)
All Sessions
(n = 473)
Number of
sessions with
fidelity
(percent)
14
(35)
73
(33.33)
78
(38.24)
3
(30)
168
(35.51)
Mean
percentage of
procedural
fidelity
(range)
79.2
(45.45-100)
88.84
(50-100)
88.62
(37.5-100)
89.74
(76.92-100)
86.6
(37.5-100)
26
Select Teacher and Child
Outcome Data
Evaluating the Potential Efficacy of Embedded
Instruction for Early Learning
Teacher Implementation of
What to Teach: Quality of
Learning Targets
Teacher Implementation of How to
Teach: Learning Trials
Rate of Trials
Rate of Complete Trials
Cohen’s d
effect sizes:
On-site
coach vs.
control =
1.41
Self-coach
vs. control =
1.51
Cohen’s d
effect sizes:
On-site
coach vs.
control =
1.35
On-site-
coach vs.
self = 1.23
Cohen’s d
effect sizes:
On-site
coach vs.
control =
2.20
On-site-
coach vs.
self = 1.91
Limitations and Implications
• Limitations – A priori power analyses based on alpha .20
– Small n potential efficacy trial – but promise
– Standardized and decontextualized child outcome measures
– Metrics used to evaluate “dosage” of self-coaching
– Need to explore mediators of PD-related effects
• Findings and Implications – High-quality workshops and either variant of coaching appear
sufficient for improving quality of learning targets
– On-site coaching using systematic performance-based protocol
improved rate and accuracy of embedded instruction learning
trials when compared to PD intervention with self-coaching or
BAU PD
– Different implementation supports for different components of
embedded instruction
– Social validity data strong, particularly for workshops plus on-
site coaching
Select Teacher and Child
Outcome Data
Evaluating the Potential Efficacy of the Teaching
Pyramid Model
Figure 1. Mean Teaching Pyramid Observation Tool
Total number of TPOT indicators = 108.
Wave 4 [t(40.03)=6.80, p<.001, Cohen’s d=2.6)
0
10
20
30
40
50
60
70
80
Wave 1 Wave 2 Wave 3 Wave 4
Intervention
Control
0
5
10
15
20
25
Wave 1 Wave 2 Wave 3 Wave 4
Intervention
Control
Figure 2. Mean frequency of positive social
interactions during 60 min observation session across
waves for Cohort 1 target children
Teacher Implementation Child Social Interactions
SSIS Child Outcomes
Adjusted Means Cohen’s d
Effect Size Intervention Control
Target Children
Social 88.6 84 .41
Problem Behavior 108.7 115.5 -.52*
Non Target
Children
Social 103.8 96.4 .46*
Problem Behavior 95.2 99 -.29
• Limitations
– SSIS is teacher report
– No measure of pre-academic skills/school readiness
– TPOT does not measure delivery of intensive
interventions to an individual child
– TPOT does not measure density or “dosage” of
implementation
• Implications
– Effectiveness of the professional development
approach
– Fidelity of implementation of TPOT practices is
related to improved child social and behavioral
outcomes
Limitations and Implications
Implications Related to
Performance-Based Coaching • Specify coaching framework for supporting implementation of
multi-component interventions
• Define components of coaching framework and coaching strategies
• Specify and define practice/intervention that is focus for coaching
• Ensure fidelity of implementation of coaching framework components and associated strategies
• Record and examine dosage and fidelity of implementation of coaching
• Record fidelity of implementation of practice/intervention by practitioner who is coached
Acknowledgments
Embedded Instruction for Early Learning Tools for Teachers (TfT)
Evaluating the Efficacy of the Teaching Pyramid Model: Using the TPOT
• Alana Schnitz, Vanderbilt University
• Kathleen Artman, The Ohio State University
• Kiersten Kinder, Vanderbilt University
• Denise Binder, University of South Florida
• Shelley Clarke, University of South Florida
• Kelly Feeney-Kettler, Vanderbilt University
• Jessie Morris Adams, Vanderbilt University
• Tara McLaughlin, University of Florida
• Alice Kaye Emery, University of Florida
• Crista Scott, University of Washington
• Lindsey Lawrence, University of Washington
• Sara Lubar, University of Wisconsin-Milwaukee