1

The Effects of an Intensive Model of Professional Development on the Instructional Reading Practices of

Classroom Teachers

Misty Sailors

The University of Texas at San Antonio

International Reading Association Annual Research Conference12 May 2007Toronto, Canada

2

Teacher Quality Professional Development Reading grant

Institute of Education Sciences Three year development grant Pilot (year 1)

The content of this publication does not necessarily reflect the views or policies of the Institute of Education Sciences, U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

3

This study is NOT about answering these questions…

Can children be taught to be more strategic in their thinking? (Paris, Waskik, & Turner, 1991; Pressley, Borkowski, & Schneider, 1987; Pressley, 2000 )

Are teachers teaching comprehension? (Knapp, 1995; Langer, 2000; Metsala et al., 1997; Morrow, Tracey, Woo, & Pressley, 1999; Pressley, Rankin, & Yokoi, 1996; Taylor, Pearson, Clark, & Walpole, 2000 )

Can teachers learn to teach comprehension? (Duffy, 1993)

Strategic thinking is helpful in developing metacognition in children (Paris, Waskik & Turner, 1991; Pressley, Borkowski & Schneider, 1987).

4

This study IS about…

Model of professional development with inservice classroom teachers

Describing the aspects of the model that are helpful to teachers in improving their practices…

Describing the aspects of the model that are helpful in raising the reading achievement of students…

5

Background The Study Findings Conclusions and discussions

6

Background

Teacher quality and expertise consistently and accurately predict student achievement (Darling-Hammond, 2000; Laczko-Kerr & Berliner, 2002; Snow, Burns & Griffin, 1998)

PD in the USA– 44/50 require PD

32 to maintain license

33 to maintain employment

No clear directives for content and/or context for PD activities

7

Further…

Focus of many reports (Darling-Hammond, 2000; National Commission Teaching and America’s Future, 1996; NCES, 1999)

Federal initiatives for high quality teachers (No Child Left Behind)

Emperor’s New Clothes

8

Traditional models of PD

Traditional “one shot” models of PD– Direct instruction, full day, outside expert– Decontextualized (Sandholtz, 2002)– Boring, irrelevant, forget 90% (Miller,

1998) No clear evidence that these “training

models” have significant impact on learning (Cochran-Smith & Lytle, 1999; Duffy, 2004)

9

Features of high quality PD

Structural features– Form/organization of– Duration in– Collective participation in

Substantive features– Specific content learning– Promotion of active learning– Promotion of coherence (alignment with standards)

Focused on specific instructional practices Proof!

(Butler et al., 2004; Guskey, 2002; Porter et al., 2003; Stein et al., 1999)

10

PD and Reading Teachers

Quality counts—teacher knowledge, beliefs and practices AND student achievement (Anders, Hoffman & Duffy, 2000)

Strategy instruction (Duffy, 1993a, 1993b, 2004; Duffy et al., 1997; Pressley et al., 1992)

Long-term process (Duffy, 1993a, 1993b)

11

Features of high quality reading teacher PD

Volunteer (and choice) Intensive levels of support (money, time,

contact hours) Monitoring/ Coaching/ Clinical support Reflection Deliberation and dialogue Collaboration

(Anders, Hoffman & Duffy, 2000)

Fram

ework

12

In summary,

effective PD is classroom-based, embedded in the school day, centered on specific research-based practices, and sustained over time.

13

The current study

Purpose: To explore the impact of an active, situated, and intense model of professional development on the instructional comprehension practices of classroom teachers.

Research questions: – Does an intense model of PD lead to an increased use of focus

instructional practices?– Does the increased use of focus instructional practices lead to

increased reading achievement of students from low-income and minority backgrounds?

– To what extent can improvement of instructional practices and student achievement  be attributed to various aspects of the professional development model?

14

Participants

Regular education teachers – 3 districts (6 elementary; 5 middle

schools)– N=44– Average years of teaching 9.83

(SD = 7.56)

Students – N=569– Low-income, minority families– Parental consent– Above, on, and below grade level

readers

Assigned to group at the school level to prevent experimental treatment diffusion (Cook & Campbell, 1979)

15

Design: pretest-posttest control group (Mertens, 1999)

Partial Intervention Group

Two day workshop only+

WebQuests (accessibles) (Beck & McKeown, 2004)

Full InterventionGroup

Two day workshop only+

WebQuests (accessibles) (Beck & McKeown, 2004)

+Follow-up support in

classrooms (minimum of 15 visits per year)

16

Content: Learning to teach reading strategies

Capitalize on their existing instructional strategies (think aloud and questioning)

Provide opportunities to engage in comprehension strategies

Provide children with access to their metacognition– Direct explanation model (Duffy) and Transactional model

(Pressley) Name it When and why How to do it (cognitive processes involved, subroutines)

Keep the big picture in mind

(Duffy, 1987, 1993a, 1993b, 2004; Pressley, 1987, 1989, 1990, 1992, 1995, 1996, 2000)

17

Content (continued)

Word identification/knowledge, Comprehension, and Fix-up strategies (Almasi, 2003)

Cognitive requirements of sample texts New Literacies comprehension strategies Assessment of instructional needs of students Modes of interaction (read alouds, guided reading,

and independent reading) Different grouping configurations Variety of texts (genres and formats) Across subject areas

18

Model of Intervention

Frame based on (Anders, Hoffman & Duffy, 2000)

Partial Intervention Group

Two day workshop only+

WebQuests (accessibles) (Beck & McKeown, 2004)

Full InterventionGroup

Two day workshop only+

WebQuests (accessibles) (Beck & McKeown, 2004)

+Follow-up support in

classrooms (minimum of 15 visits per year)

19

Model of Intervention: The Mentors (IRA, 2004)

Mentor 1 Ph.D. (Language and

Literacy studies) Reading specialist

(TX) certified Graduate and

undergraduate literacy education courses

Mentor 2 MEd. Practicing reading

specialist (TX) for 23 years

Undergraduate literacy education courses

Combined 15 years of classroom teaching AND 10 years of classroom based professional development experience

20

Opening the interaction

21

Specific strategies

22

Model of Intervention: Interactions

23

Resources for the teachers

Suggested explanations Co-constructed public classroom texts WebQuests

24

Closing the interaction

25

Data Collection: Student Data

Group Reading Assessment and Diagnostic Evaluation (GRADE) (AGS, 2001)– used to measure change in level of reading comprehension

Wide Range Achievement Test 3 (WRAT 3) (Wilkinson, 1993)– used to identify three “focus” students

Administered by Research Associates (graduate students and adjunct instructors)

26

Data Collection: Teacher Data

Instructional Comprehension Strategy Observation Protocol

Developed to measure implementation of content Observational note-taking and quantitative coding

process (Herbert & Attridge, 1975; Martin, 1977) Narrative account of context, materials used,

strategy content, and instructional scaffolding Units to coded based on the work of Duke (1999;

2000), Duffy (1987, 1992, 2004), and Taylor and colleagues (Taylor et al., 1999)

27

Sample narrative

Continued…

28

Coding (descriptive)

29

Coding (catagorical)

30

Data Collection: Mentor Data

Men

tor site visit rep

orts

31

Data Collection: Data Collectors

Research Associates– Seeking MEd in Literacy Studies (Master Reading

Teacher certification)– Adjunct instructors of literacy education courses

Training– One day at university– Two days in classrooms (80% agreement)– Ongoing reliability checks (interrater reliability)

32

Data Collection: Timeline

September April/May

Wo

rksho

ps

Teacher pre- observations (45 minutes)

Student pre-assessments

Teacher post- observations (45 minutes)

Student post-assessments

33

Limitations

Small sample size Volunteers– Teachers adoption of innovation

was a function of choice to be involved (Borko, Davinroy, Bliem, & Cumbo, 2000; El-Dinary et al., 1993; Garet et al., 2001; Gersten, Vaughn, Seshler, & Schiller, 1997; Linek et al., 2003; Yamagata-Lynch, 2003)

34

Data Analysis

Screened for– Missing data/extreme data– Distributional patterns of frequency counts– Examined general linear model assumptions

Multilevel modeling analytic strategies (HLM) Between group nonparametric multinomial

regression and chi-squared analysis

35

Data Analysis: Composite variables

Provided opportunities to engage in comprehension strategies (expressed as frequency counts)

– Word ID/Word knowledge;– Fix-up;– Comprehension– “comp”

Intentional instructional explanations of comprehension strategies

– Named it, told and modeled when and why, explained how to, provided practice– “intent_instruct”

Almasi, 2003

36

Findings: Question #1

Does an intensive model of professional development lead to an increased use of intentional instructional practices?

Yes. Statistically significant differences between the groups (direction of full intervention group) in the opportunities the FI teachers offered to their students to engage in “comp” (X² (1) = 3.13, p < .05)

Small effect sizes (Cramers V=.08) No statistical or practical effects for “word ID/word knowledge” or

“fix-up” strategies Yes. Statistically significant differences between the groups direction

of full intervention group) in the engagement of intentional instructional practices “intent_instruct”(X² (1) = 3.65, p<.05)

Medium practical effect (Cramers V = .40)

Cohen (1988) Small = .10-.34; med = .35 to .6; large = >.60

37

Was there a particular type of teacher who was more inclined to improve their practices?

No. Non-significance/practical effects (Years of Experience / Teaching Area / Level of Education) on dependent variable (“intent_instruct”)

Yes. Statistically significant main effect observed on dependent variable (“intent_instruct”) between groups in the engagement of intentional instructional practices “intent_instruct” by the type of certification held by teachers(F = 2.78, df = 7, 31; p<.05)

Large effect size (η² = .39)

general linear model regression/ANOVA (η² )

38

Findings: Question #2

Does the increased use of intentional instructional practices lead to increased reading achievement of students from low-income backgrounds?

Main effect of the intervention between groups across time.– Yes. Statistically significant change in reading achievement between the

treatment (mean = 419.85) and control (mean = 402.58) groups.– The fixed effects portion of the model yielded a 17.3 point difference

between groups. Main effect of the intervention overall and by learner group (above,

on, and below) – Yes. Statistically significant main effect for group (F = 4.32, df = 1, 431; p

< .05), but not for student learner level as assessed by the WRAT3.

X

39

Findings: Question #3

To what extent can improvement of instructional practices and student achievement  be attributed to various aspects of the professional development model?

No statistical relationship between change in GRADE from pre- to post- and the providing of opportunities to engage in comprehension strategies (“comp”)

Practical trend emerged– 71% of students whose teachers displayed “comp” characteristics resulted in positive GRADE score changes.

No statistical relationship between change in GRADE from pre- to post- and teachers’ use of intentional instructional practices (“intent_instruct”)

Practical trend emerged– 71% of students whose teachers displayed “intent_instruct” characteristics resulted in positive GRADE score changes.

Relationship betw

een content of P

D and student

outcome

40

Findings: Question #3

To what extent can improvement of instructional practices and student achievement  be attributed to various aspects of the professional development model?

Statistically significant differences (direction of full intervention group) regarding the number of visits made and “intent_instruct”

Regression equation – effect size of .236 (or 23.6%) There were no observable influences regarding the nature of

interactions between the teachers and mentors, including the initiation of the interaction and the content of the interaction and their engagement in intentional instruction (“comp” or “intent_instruct”)

There were large practical effects for all the predictors (contingency coefficients)

“demo” (CC) = .77 “co-teach” (CC) = .81 “feedback” (CC) = .82 “conf” (CC) = .84 “teacher_init” (CC) = .87 “mentor_init” (CC) = .82

Relatio

nsh

ip b

etween

qu

antity

and

qu

ality of m

ento

ring

and

im

plem

entatio

n o

f con

tent o

f P

D

41

Findings: Question #3

To what extent can improvement of instructional practices and student achievement  be attributed to various aspects of the professional development model?

The variable “comp” was a statistically significant predictor of “intent_instruct”

– (F = 7.99, df = 1, 25; p < .001), – Regression equation yielded an adjusted R2 of .24.

42

In summary

Teachers who are supported in classrooms can learn to

– Provide their students with opportunities to engage in comprehension strategies AND

– Explain the underlying processes of those strategies more often…

– Are associated with students who make statistically significant higher gains on measures of comprehension

Support comes in the form of – A highly qualified mentor – Interaction with the teacher in a variety of ways

43

Discussions: Common Sense versus Common Practice

Teachers who are provided with opportunities to interact with a “knowledgeable other” in the presence of their students under conditions selected by the teacher is HELPFUL in informing their instruction

Policy mandates with little regard to quality

44

Coaching as a model of PD in the USA

No Child Left Behind Virtues of coaching

– IRA, NCTE, NCTM, NSTA, NCSS (IRA, 2006 Roles and responsibilities of coaches (Dole, 2004; Roller, 2006) Characteristics of high quality coaches (Shanklin, 2006) Qualifications of reading coaches (IRA, 2004; 2006) Models of coaching (Bean, 2004; Toll, 2006; Walpole & McKenna,

2004) Guiding texts for coaches (Walpole & McKenna, 2004; Toll, 2006;

Kise, 2006; Hasbrouck & Denton, 2005; Casey, 2006; Allen, 2006) Empirical evidence????

“New is not always right.” (Wilson & Berne, 1999, p. 5)

45

In conclusion

Empirical evidence that supports qualitative features of professional development of reading teachers (Anders, Hoffman, & Duffy, 2000; Darling-Hammond, 1999; Hoffman & Pearson, 2000; National Reading Panel, 2000; Pearson 2001; Snow, Burns & Griffin, 1998)