Dr Michael Cejnar, MBBS, FRACP, Founder FIC Technology Pty Ltd
Education Show - Melbourne, VIC
31st August 2018 (V0.1)
Improving outcomes with
ICT Learning Analytics in K-12
Effective ICT Use
http://www.oecd.org/publications/students-
computers-and-learning-9789264239555-
en.htm
OECD – PISA 2015:
Students, Computers
and Learning, Making the Connection
OECD PISA 2015 Analysis –Students, Computers and Learning
• …students who use computers very frequently at
school do a lot worse in most learning outcomes, even
after accounting for social background and student
demographics.
• The results also show …. in the countries that had
invested heavily in ICT for education… no appreciable
improvements.
Andreas Schleicher,OECD Education Director, Directorate for Education and Skills,
OECD PISA 2015 Analysis
Time spent on
Internet at
school/ day.
Source: OECD, PISA
2012 Database, Table 1.5c.
300
350
400
450
500
550
600
2005 2008 2011 2014
Year 6
Year 10
Y6 Profficiency
Y10 Profficiency
Australia
NAP–ICT 2005-2014
58min/day (our data: 90 min)
Post DER ICT Usage in STEM S.Crook
Simon J. Crook, Manjula D. Sharma. Bloom-ing Heck! The Activities of Australian Science
Teachers and Students Two Years into a 1:1 Laptop Program Across 14 High Schools.
Int. J. of Innovation in Science and Mathematics Education, 21(1), 54-69, 2013
ICT Learning
What affects Technology Enabled Learning (TEL)?
• Computer Time – amount, grades, subjects, optimum
• Usage type - Content, vs interactive vs adaptive,
consumptive vs analytical vs creative
• Role of distraction - Off-task, task-switching, copying
• ICT skills – Which important, how assess, how teach,
skills diversity, searching, dig. literacy
• Teaching – support, styles, assessments, sharing
How do we measure & analyse these?
Learning Analytics
Learning Analytics is the measurement, collection, analysis
and reporting of data about learners and their contexts, for
purposes of understanding and optimizing learning and the
environments in which it occurs.
edQuire K-12 Learning Analytics
EdQuire K-12 Live Classroom
Live Classroom
Lesson Learning Flow Display
Reflective Reports
edQuire Data
Present LA Data from edQuire use in:
•25 Schools, independent
•Years: 6-12, 2000 students
•PCs 1:1 (90%), computer labs (10%)
•110,000 lessons (57,045 using PC**) with > 25 million
activities*
•Jan 2018 – to date (Term 1-3)
*Apps and URLs, ** >1 min
Computer Usage
First questions:
• How much time is spent on the computer?
• Do all lessons use computers?
• Different subjects and year levels?
Computer Usage Time
50% of Lessons used
PC > 1 min
30% of all Teaching Time
spend on PC
(110,000 lessons)
In PC-using lessons,
85% of lesson was on
PC
(38 of av. 45 min)
Computer Usage Time
By subject:
Questions to Schools – Do you know:
•How much time is spent on computers?
•What is the optimal usage?
•What quality of the learning on computers?
Student ERIS Model of TEL
1.Engagement
Questions we asked from the data:
• How much time do they spend on lesson task?
• When students are on task what types of resources
do they access?
• How distracted are students during lessons?
• Does resource type determine student positive
engagement?
Engagement – Time on/off task
Top 5
Microsoft Word - 19%
Zoom video conferencing – 8%
Windows Explorer – 8%
Microsoft Outlook – 5%
Google Docs – 4%
Top 5
YouTube – 38%
Spotify – 5%
VLC Media player – 5%
Netflix – 3%
Broforce (Gaming App) - 2%
Mean: 8.1%
Highest Year 10: 14.2%
Lowest Year 12: 6.4%
Engagement – OFF TASK
Engagement - Distractibility
Questions we asked from the data:
• How to measure distractibility
• What does a highly distractibility student look like?
• Who gets distracted?
• How can distraction be reduced?
What is distractibility & how is it measure?
Distractibility Index
D.I is calculated from proportion of Off-task & the number of
switches. High Value indicates more distractible.
29
Student 1 – Highest Distractibility
Distractibility
Start of lesson
8.50am
1st activity – edumate at 8.53am
1st ON to OFF task switch 8.55am
Youtube.com – song
Total time off task 11 minutes.
Total switches from on to off task 29 times
End of lesson – 9.30am
Student Distractibility
Distribution of
durations of
activities
Task Duration or
Dwell TimeStudent Distractibility
by Year
Examples of LA data-driven Teacher Strategies:
1. Blocking: ‘Cold Turkey’ App was applied to highly
distracted students
….but Report data: App installed but not being used, so
reinforced
2.Co-opting Distractions: Personalize instruction to
student’s interest
3.Self-Governance: edQuire student portal was introduced
to students to give feedback on engagement behaviour.
Distractibility – Actionability
2. Co-opting Distractions
Knowing Off-task activities can be co-opted to re-engage
students:
Year 6 Lesson: 10 Subject: English
Students: 21 Total Lesson Time: 600 mins
School: Government primary school Northwest of Sydney.
•Off task ‘DC Database’= DC Comics
•Teacher personalized instruction with
DC comic characters to successfully
re-engage high risk/distractible
students.
3. Student feedback
Students given their own
On-task Engagement Feedback
Self-Regulation
N=153, 8 wk base and 8 wk Fbk
Student’s edQuire Feedback Console
Trial:
Outcomes analysis
Off-Task Relative time
Stu
de
nt E
du
ca
tion
al O
utc
om
e
Intervening for Outcomes
Off-Task Relative time
Stu
de
nt E
du
ca
tion
al O
utc
om
e
2.Resources
Engagement – ON TASK
2.ICT Skills
1.5% used advanced
internet search skills
School LMS
- Canvas
MS Word
Google Docs
PowerPoint
MS Word
Education
Perfect
Research based
activities
19% used advanced
internet search skills
Subject: Science
Year Level: 7, 8, 9,
10 School: Co-ed
Period: Feb 2018
Student N: 307
Subject: Science
Year Level: 7, 8, 9,
10 School: Co-ed
Period: Feb 2018
Student N: 300Student-centered learning
Student-self-directed
Traditional Teaching
CAD
ICT Learning Profiles
vs
Science
Bloom Taxonomy - High vs low groups
Bloom Taxonomy variability Subjects
Bloom
Blooms Digital Taxonomy
Learning Profiles by Subject
In Summary
1. Classroom computer efficacy remains
inconsistent and challenging.
2. Effective TEL needs engaged undistracted
students, the right resources and ICT skills
to use them.
3. Real-time practical classroom learning
analytics is demonstrated in K-12 and
provides timely, formative & practical data
for making TEL effective.
Thank You