Meta-findings from the Best Evidence

EncyclopaediaRobert E Slavin

University of York and Johns Hopkins University

Best Evidence encyclopaedia (BEE)

• Intended to provide easily accessible, scientifically valid summaries of the evidence base for educational programmes

• Educator’s summary – like Consumer’s Reports• Full reports written for publication in academic

journals

BEE Inclusion Standards

• Programmes compared to control group– Random or matched

• Control group within + .5 SD of experimental group at pretest

• Posttests adjusted for pretests• Measures are not inherent to treatment• Duration at least 12 weeks

Main BEE Reviews

• Primary Maths – RER, 2008• Secondary Maths – RER, 2009• Primary Reading – RER, 2009• Secondary Reading – RRQ, 2008• Struggling Readers – Educational Research Review,

in press

Meta-findings: Substantive• Textbooks: ES = +0.06 IN 77 studies

• Technology, CAI: ES = +0.11 in 130 studies

• Instructional process approaches:– Strongest effects in every review.

ES = +0.27 in 100 studies• Co-operative learning, PALS• Classroom management, motivation• Metacognitive skills

• Combined Curriculum/CAI with Instructional Process: ES= +0.26 in 39 studies-Read 180

-Success for All

Table 1: Weighted Mean Effect Sizes by Programme Category

Curricula CAI Instructional Curr/CAI +

Process IP

Maths - Primary +0.10 (13) +0.19 (38) +0.33 (36) -

Maths - Secondary +0.03 (40) +0.08 (40) +0.18 (22) -

Reading - Beginning +0.13 (8) +0.11 (10) +0.31 (18) +0.28 (22)

Reading -Upper Primary +0.07 (16) +0.06 (34) +0.23 (10) +0.29 (6)

Reading -Secondary - +0.10 (8) +0.21 (14) +0.22 (11)

Weighted Mean +0.06 (77) +0.11 (130) +0.27 (100) +0.26 (39)

Meta-findings Specific to Reading

• Programmes that emphasize structured, systematic phonics get better outcomes

• But, outcomes of phonetic approaches depend on quality of teaching– Simple adoption of phonetic books ineffective– Effective programmes use extensive training in

co-operative learning and other motivation and management methods

Meta-findings - Methodological

• Randomised and matched studies find nearly identical outcomes

• Small studies overstate outcomes (so the BEE weights by sample size)

• Measures inherent to treatments greatly overstate outcomes (so the BEE excludes them)

• Very brief studies overstate outcomes (so the BEE excludes them)

Importance of Phonics

• Almost all successful programmes emphasize structured, systematic phonics

• One-to-one tutoring by teachers – Programmes focused on phonics: ES=+0.69 (10)

– Programmes less focused on phonics: ES=+0.23 (9)

• Within-study comparisons favour phonics in 1-1– Difference: ES=+0.18 (5)

• But, adopting phonetic texts, software, or professional development is not sufficient

Teachers vs. Teaching Assistants

• Children tutored with phonetic programmes by qualified teachers gain somewhat more than those tutored by teaching assistants and volunteers.

• Teacher tutors: ES=+0.69 (10)• TA/volunteer tutors: ES=+0.24 (18)

One-to-One vs. Small Group

• Children taught phonetic programmes one-to-one gain much more

• Teacher tutors: ES=+0.69 (10)• TA/volunteer tutors: ES=+0.24 (18)• Small groups: ES=+0.31 (18)

Classroom Approaches vs. Pullout

• Classroom instructional process programmes work as well as 1-1 tutoring

• Classroom instructional process: ES=+0.56 (16)• Phonetic tutoring by teachers: ES=+0.69 (10)• Combination of initial tutoring plus classroom

process works best• Success for All: ES=+0.55 (9)

Conclusion

• Education policies should identify and help disseminate proven programmes of all kinds.

• Well- specified programmes with extensive professional development to help teachers engage and motivate children are most likely to produce positive outcomes.

• Practical, consistent, scientific reviews of research can help educators make good choices for pupils.

For more information visit:

www.bestevidence.org.uk