20
Page 1 A tale of two universities: Organic growth of learning analytics through bespoke coevolution @dannydotliu [email protected] [email protected]
Page 1
A tale of two universities: Organic growth of learning analytics through bespoke
coevolution
@dannydotliu
Page 2
The contexts of learning analytics
Common barriers to adoption
– Policy and ethical challenges
– Culture of resistance to change
– Vendor solutions
– Data accuracy
– One-size-fits-all
Pressing institutional needs
– $Millions lost to attrition
– Larger class sizes
– More disconnected students
– Feedback very generalised
– Data are scattered
Page 3
1. Macquarie: Empowering staff with actionable LMS data
2. Sydney: Learning analytics by stealth
Learning analytics in MoodleMOTIVATIONS AND BENEFITS
• Benefits
• Staff familiarity
• Single point of access
• Learning experience data already there
• Problems
• No available learning analytics tool with actionable data
Log viewer
Statistics report
MOCLog
‘Participatory design’
5
ACADEMICS AND STUDENT SUPPORT STAFF
Staff expectations
Prototyping and development
User testing
Piloting
Feedback and further development
MEAP, enhanced
6
MOODLE ENGAGEMENT ANALYTICS PLUGIN
Stakeholder impacts
7
PERSONALISED, DATA-DRIVEN INTERVENTIONS
• 3400+ personalised emails sent, average ~46% opened
• From unit convenors and student support staff
• For census, updates, reminders
• With predominantly at-risk students
• Using logins, assessment submissions, grades, attendance
• Next: wider trials in semester 1, 2016
I was surprised someone cared/was actually monitoring, kind of a weird, I don't know totalitarian/'people are watching you' feeling? But in this situation I was happy.
Very useful. I wouldn't have been able to do such a large scale analysis and identify so many students without MEAP. I wouldn't have been able to send them such tailored, structured and consistent messages.
Page 8
1. Macquarie: Empowering staff with actionable LMS data
2. Sydney: Learning analytics by stealth
Page 9
The Student Relationship Engagement System
AttendanceInterim grades
LMS metrics
Third party tools & other
data
Page 10
Personalising connections with students
– Empowering staff
– Flexible & intuitive
– Targeted and personalised
– Multi-channel
– Benefits
– Highly customisable
– Efficient – key data in one place, operating at scale
– Connect staff and all students (not just at-risk)
Page 11
Co-evolution of the SRES
– Organic adoption by academics
– Students felt cared-for
– Reduced attrition, improved grade distribution
0
5000
10000
15000
20000
25000
0
10
20
30
40
50
60
70
2012 2013 2014 2015
Num
ber
of
stud
ent
s
Num
ber
of
units
or
scho
ols
Number of units
Number of schools
Number of students
Pilot
EWS, Track
& Connect
EWS
integrated
New
analyses
More data
types
Data
import
Page 12
Learning analytics by stealth?
– As staff data literacies grew, so have system capabilities
– Covert introduction to data-driven pedagogy
Page 13
1. Macquarie: Empowering staff with actionable LMS data
2. Sydney: Learning analytics by stealth
Page 14
Lessons learned and issues raised
– Give them what they want vs. build it and they will come– Find champions with tolerance for error
– Customisability is key
– Usefulness (eventually) trumps aesthetics (to an extent)– But people still like shiny things
– Data are not enough – connect with pedagogical, pastoral
– Surprisingly little kickback about privacy & ethics– Tension between research ethics & general ethics
– It’s tricky to measure impact
– Iterate – capabilities, implementation
– Focus on the human
Page 15
Breaking news
Page 16
Open analytics
LMS
Video
Classrooms
Mobile
Business
systems
External
coursesLearning
Record
Store (LRS)
Custom analytics engine
Page 17
Open analytics
Page 18
‘Who Are My Students’ report
Page 19
Adoption pipeline
Colvin et al. (2016) Student retention and learning analytics: A snapshot of Australian
practices and a framework for advancement. Office of Learning and Teaching, Sydney.
First, implementers require an analytic tool or combination of tools that manage data inputs
and generate outputs in the form of actionable feedback… As these increasingly meet the
real needs of learners and educators the organisational uptake is accelerated.
Page 20
1. MEAP Empowering staff with actionable LMS data
Chris Froissard, Deborah Richards, Amara Atif et al.
2. SRES Learning analytics by stealth
Charlotte Taylor, Adam Bridgeman, Kathryn Bartimote-Aufflick,
Abelardo Pardo et al.
@dannydotliu
