LEO is a Spanish version of ICON, a web-based literacy environment developed by CAST, Harvard Graduate School of Education, and Boston College through a grant from the Institute of Education Science, U.S. Department of Education.
Pontificia Universidad Católica del PerúDepartamento de Psicología y Dirección Informática
Académica
Cecilia Thorne
Teresa Nakano
Beatriz Mauchi
Lorena Landeo
Paola Ucelli
Kim Morla
Roman Huerta
Angie Vásquez
LEO
Kim Morla
Deputy Director for Academic Computing
Pontificia Universidad Católica del Perú
Country educational context
Methodology or Framework
Results to date
Key Challenges
Performance for
reading
comprehension – 2°
grade (MED, 2011)
2000+ 2009
Mexico 422 425
Argentina 418 398
Chile 410 449
Brasil 396 412
Perú 327 370
Promedio región 395 411
Promedio OECD** 496 495
**Promedio OECD considerando solo a los países que participan desde el 2000
Results : PISA 2000-
2009 (MED, 2010)
• National projects : ICT4E
– Huascarán (2000)
– OLPC– Perú (2007)
• ICT infrastracture in schools increases, but no
pedagogical use observed (Trinidad, 2009).
• Web based app integrates :
– Vocabulary exercises
– Reading strategies
• 5th graders
Web
Based
AppReading strategiesVocabulary Exercises
Methodology - 2010
Design
Quasi-
Experimental
Experimental
and control
groups
Pre and
post test
Random
classroom
assignment
Procedure - 2010
Pre-Test:Reading literacy test
Vocabulary test
Post-Test
Intervention : LEO
5th grade
Four months
Weekly sessions
Two hours p/session
Eight texts
Computer lab
Technical & Pedagogical
support
• Participants : 99 students and 3 teachers
• Normality tests: Kolmorogov-Smirnova, Shapiro-Wilk
• Instruments:
1. PIRLS Reading Literacy Test (IEA,2001) adapted by Morales (2009): narrative and informattional texts.
2. Vocabulary test (CLP)
•Students completed
between 6.5 and 7
texts.
•Vocabulary
presents better
results than reading
comprehension
activities
Medias en exámenes de comprensión y vocabulario de LEO
1211
13
1111
10
11
9
15 15
1211
14
11
13
15
0
2
4
6
8
10
12
14
16
18
20
Tex 1 Tex 2 Tex 3 Tex 4 Tex 5 Tex 6 Tex 7 Tex 8
Comprensión
Vocabulario
Activity in LEO
Acierto en las preguntas literales e
inferenciales (%)
0%
20%
40%
60%
80%
100%
Literal 81% 65% 64% 52% 78% 50% 56% 43%
Inferencial 49% 50% 69% 54% 43% 44% 54% 46%
1 2 3 4 5 6 7 8
Activity in LEO : Types of questions
Results - 2010
CLN – NARRATIVE
CLI – INFORMATIONAL
VOCABULARY
Pre Test – Experimental &
Control
Narrative texts
** p < .01
Informational texts
* p < .05: difference initial evaluation
** p < .01difference in ending evaluation
Vocabulary
* p < .05: difference initial evaluation
*** p < .001: difference ending evaluation
Effect size : groups
Before : After:
CLN : difference insignificant CLN : medium
CLI : medium CLI : medium (same)
V : small V : BIG (0.33 to 0.95)
Effect size : before and after
Control : small effect (.07,.08..09) Experimental:
CLN : medium
CLI : small
V : BIG (0.92)
Students Achievements Teacher difficultiesDifficulties with
Technology
Motivation and interest Overcome fear ICT.
Little or null
computer
knowledge
Connection failures,
misconfigured sound,,
etc
Improved in writing
exercises
Used strategies and
resources for other
classes
No real use of the
teacher’s view of the
app.
Themes transferred to
other areas / courses
Schools asked that
LEO be used in other
grades
No (poor) feedback
through app
� Big improvement in vocabulary
� Medium improvement for narrative texts
� Doesn’t help for informational texts
� Teacher training needed :
� basic computer use,
� use of LEO
� in strategies to help improve reading comprehension
� Students are too young for autonomous work
Intervention 2011
• 15 schools
• Three grades (4th, 5th, 6th)
• 91(3th), 365(4th), 242(6th)
• NO CONTROL GROUP !!!
Reading Comprehension : 2011
Reading Comprehension : 2011
Vocabulary : 2011
Vocabulary : 2011
6th Grade – an example
9 schools
6th Grade – an example –
Reading
3 3 non-sig
6 non-sig
Effect size - med
6th Grade – an example -
Reading
Effect size - BIG
6th Grade – an example -
Reading
Effect size - BIG
6th Grade – an example -
Vocabulary
Effect size - medium
6th Grade – an example -
Vocabulary
Effect size - BIG
2012
• General :
– LEO for 4th, 5th and 6th grade
• Especific :
– Narrative and Informational texts
– Vocabulary
– Evaluate exercises for informational texts (new strategies)
– Evaluate relation between activity in LEO and final results per student
Participants - 2012
• 3 Schools
• 4th, 5th and 6th grades in each school
• 1 experimental, 1 control per grade, per
school
• At least 20 students x class
• Teacher in experimental class uses LEO
Implementation - 2012
• 16 weeks
• 1 session per week / 2 hours per session
• 1 extra week – teacher training
• 1 week – pre test / 1 week – post test
• Technical and pedagogical in-site support –
first text
• Meetings w/teachers– 4 – Informational texts
Data collection - 2012
• Assess comprehension, vocabulary (before, after) – both narrative and informational texts
• New strategies – informational texts
– Evaluate
– Teacher’s feedback
– Class observations
– Answer analysis for exercises related to informational texts
Data analysis - 2012
• Difference between pre and post tests for
experimental and control groups
• Size of the effect for each test
• Correlation between tests – informational
tests and new strategies
• Systematize : observations in class and
teacher meetings : reflection about strategies,
use, exercises, difficulties, recommendations.
References
Dalton, B. & Proctor, C.P. (2007). Reading as thinking: Integrating strategy
instruction in a universally designed digital literacy environment. In D. McNamara
(Ed.), Reading Comprehension Strategies: Theory, Interventions, and Technologies
(pp. 421-440). Mahwah, NJ: Erlbaum.
Dalton, B., Proctor, C.P., Uccelli, P., Mo E., & Snow, C.E. (2011). Designing for
diversity: The role of reading strategies and interactive vocabulary in a digital
reading environment for fifth-grade monolingual English and bilingual
students. Journal of Literacy Research, 43(1), 68-100.
Proctor, C.P., Dalton, D., Uccelli, P., Biancarosa, G., Mo, E., Snow, C.E., &
Neugebauer, S. (2011). Improving Comprehension Online (ICON): Effects of deep
vocabulary instruction with bilingual and monolingual fifth graders. Reading and
Writing: An interdisciplinary journal, 24(5), 517-544.
Proctor, C.P., Uccelli, P., Dalton, B., & Snow, C.E. (2009). Understanding depth of
vocabulary online with bilingual and monolingual children. Reading and Writing
Quarterly, 25(4), 311-333.
