An Experimental Study of Project GLAD ® : Results from Year 1

Theresa Deussen - Elizabeth Autio

Angela Roccograndi - Makoto Hanita

WERA * December 13, 2013

Who’s here and why?

2

3

A case in point:Growth of EL population

4

Project GLAD® (Guided Language Acquisition Design)

6

Project GLAD Input Chart(One of 35 instructional strategies)

Professional development

8

Research Questions

9

What about the impact specifically for English

learners? Does that look the same?

Do teachers implement it as

intended?

Does it impact students’ achievement in:

Reading comprehension?Vocabulary?

Writing?Science?

Study Design: Cluster Randomized Trial

30 schools21 districts101 classroomsGrade 5

Teachers

Characteristics Treatment Control

Mean years experience (SD) 11.6 (8.4) 17.2 (11.7)

Percent female 85% 82%

Percent w MA degree 95% 96%

Percent w ESL endorsement 5% 5%

Prior SIOP training 66% 70%

12

Students

Group N (percent)

Total students tested 2778

Students w pre & post 2223

Current ELLs 80 (4%)

Former ELLs 187 (8%)

Non ELLs 1956 (87%)

13

Outcomes Measures

Subject Assessment

Reading comprehension Gates-MacGinitie

Vocabulary Gates-MacGinitie

Science: rocks & minerals Scott Foresman end-of-unit

General science achievement Grade 5 Idaho state test

Writing 6+1 Traits writing, science topic

14

Year 1 Outcomes: Non ELLs

Subject Treatment Control p = Effect size

Comprehension 509.57 506.85 0.177 0.07

Vocabulary 513.21 511.18 0.187 0.06

Project science 6.55 6.05 0.101 0.24

State science 210.24 208.97 0.148 0.13

15

Year 1 Outcomes: Non ELLs

Subject Treatment Control p = Effect size

Comprehension 509.57 506.85 0.177 0.07

Vocabulary 513.21 511.18 0.187 0.06

Project science 6.55 6.05 0.101 0.24

State science 210.24 208.97 0.148 0.13

16

Year 1 Outcomes: Non ELLs

Subject p = Effect size

Comprehension 0.177 0.07

Vocabulary 0.187 0.06

Project science 0.101 0.24

State science 0.148 0.13

17

Year 1 Outcomes: Non ELLs

Subject Treatment Control p = Effect size

Ideas 4.11 3.97 0.060 0.22

Organization 4.25 3.75 0.235 0.11

Voice 4.11 4.08 0.590 0.07

Word Choice 4.11 4.05 0.181 0.16

Sent Fluency 3.90 3.84 0.239 0.11

Conventions 4.06 4.02 0.301 0.07

18

Year 1 Outcomes: Non ELLs

Writing Trait Treatment Control p = Effect size

Ideas 4.11 3.97 0.060 0.22

Organization 4.25 3.75 0.235 0.11

Voice 4.11 4.08 0.590 0.07

Word Choice 4.11 4.05 0.181 0.16

Sent Fluency 3.90 3.84 0.239 0.11

Conventions 4.06 4.02 0.301 0.07

19

Only 13% of students were current or former ELLs.

20

Year 1 Outcomes: ELLs

Subject Treatment Control p = Effect size

Comprehension 482.1 474.3 0.099 0.24

Vocabulary 479.7 474.0 0.092 0.21

Project science 5.7 5.3 0.303 0.19

State science 0.309 0.12

21

˜

˜

Year 1 Outcomes: ELLs

Writing Trait Treatment Control p = Effect size

Ideas 3.75 3.54 0.053 0.32

Organization 3.56 3.41 0.086 0.27

Voice 3.97 3.94 0.723 0.05

Word Choice 3.94 3.84 0.112 0.22

Sent Fluency 3.58 3.56 0.545 0.05

Conventions 3.77 3.76 0.882 0.02

22

˜

˜

Practical Significance?

…a work in progress…

23

What about implementation?

24

25

Walls Used Revisit R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Be

havi

or

1. Standards 2. Scouts 3. Literacy awards R1 R2 4. T-graph for social skills R1 R2 R3 R4 R5 5. Team points 6. Personal interaction

Char

ts

7. 10/2 lecture R1 R2 R3 R4 R5 R6 R7 8. Picture file cards 9. Observation chart R1 R2 R3 R4 R5 10. Inquiry chart R1 R2 R3 R4 R5 R6 R7 11. Cognitive content dictionary * R1 R2 R3 R4 R5 R6 R7 R8 12. Graphic organizer * R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 13. Pictorial input * R1 R2 R3 R4 R5 R6 R7 R8 R9 14. Comparative input * R1 R2 R3 R4 R5 R6 R7 R8 R9 15. Narrative input * R1 R2 R3 R4 R5 R6 16. Sentence patterning chart R1 R2 R3 R4 R5 R6 R7 R8 17. Chants/poetry * R1 R2 R3 R4 R5 R6 R7 R8 18. Story map R1 R2 R3 19. Mind map R1 R2 R3 R4 R5 20. Process grid R1 R2 R3 R4 R5 R6

Smal

l gro

ups 21. Team task R1 R2 R3 R4 R5 R6

22. Expert group R1 R2 R3 R4 R5 R6 R7 R8 R9 23. ELD group frame R1 R2 R3 R4 R5 R6 24. ELD review R1 R2 25. Numbered heads

Writi

ng

26. Cooperative strip paragraph * R1 R2 R3 R4 R5 R6 R7 R8 R9 27. Writers’ workshop 28. Learning logs 29. Interactive journals 30. Portfolios

Read

ing 31. Big books R1 R2 R3 R4 R5

32. Clunkers/links 33. Focused reading 34. Ear-to-ear reading

35. Home/school 36. REPEATED _________________

Project GLAD Input Chart(One of 35 instructional strategies)

27

28

Walls Score (1,2,3)

Notes Be

havi

or

1. Standards 2. Scouts 3. Literacy awards 4. T-graph for social skills 5. Team points 6. Personal interaction

Char

ts

7. 10/2 lecture 8. Picture file cards 9. Observation chart 10. Inquiry chart 11. Cognitive content dictionary 12. Graphic organizer 13. Pictorial input 14. Comparative input 15. Narrative input 16. Sentence patterning chart 17. Chants/poetry 18. Story map 19. Mind map 20. Process grid

Smal

l gro

ups 21. Team task

22. Expert group 23. ELD group frame 24. ELD review 25. Numbered heads

Writi

ng

26. Cooperative strip paragraph 27. Writers’ workshop 28. Learning logs 29. Interactive journals 30. Portfolios

Read

ing 31. Big books

32. Clunkers/links 33. Focused reading 34. Ear-to-ear reading

35. Home/school

Year 1by the numbers

Included 101 teachersConducted 167 observations(1.6 observations/teacher)

29

Treatment classrooms were different from control classrooms.

Project GLAD ® strategies were observed in:

99% of observations in treatment classrooms

5% of observations in control classrooms

While all treatment teachers used Project GLAD ® strategies, the frequency of implementation varied.

0

5

10

15

20

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

Number of teachers

Average number of strategies teachers used weekly

Mean 12.5

The quality of implementation also varied among treatment teachers.

0

5

10

15

20

10 20 30 40 50 60 70 80 90 100

Number of teachers

Quality Rating (Percent)

Mean79%

Note: Our rating scale of 1-3 means there is a floor of 33%

34

Is uneven implementation

good enough?

35

Next Steps: More Questions

36

What does impact look like in teachers’ second year of implementation?

What does implementation look like in the second year?

Do we find higher impact in classrooms with higher

levels of implementation?

Implications

What’s the take home message for you?

What would you tell a district thinking about purchasing Project GLAD ® professional development?

37

Implications

38

“Good enough” implementation

Suggestive findings for ELLs

Exploring factors that could promote strong implementation

http://projectgladstudy.educationnorthwest.org/

Theresa.Deussen@educationnorthwest.org

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A100583 to Education Northwest. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

39