www.leas-box.eu [email protected] © 2016 TU Graz 1
Persuading an Open Learner Model in the Context of a University Course:
an Exploratory Study
Blandine Ginon, Clelia Boscolo, Matthew Johnson, Susan Bull
www.leas-box.eu [email protected] © 2016 TU Graz 2
Introduction
• LEA’s BOX Open Learner Model
– Provides 10 visualisations
– Provides a persuasion facility
• Evaluation
– 15 students of Italian as foreign language at University
– Use of LEA’s OLM during one semester
– Focus on the reasons to use or not use persuasion
www.leas-box.eu [email protected] © 2016 TU Graz 3
Introduction
• Open Learner Model (OLM)
– Can be accessed in a user-understandable form
• Reasons to open the learner model
– Identify strengths and weaknesses
– Support learner reflection
– Facilitate planning and self-monitoring
– Address issues of learner control over data
– Address learner model accuracy
www.leas-box.eu [email protected] © 2016 TU Graz 6
Real world limitations
• Sources of data external to the OLM
• Multiple learning applications
• Data on different levels of granularity
• All data may not be regarded by students as equally valid
• Can lead to model accuracy issues
Interactively maintained OLM may overcome these limitations
www.leas-box.eu [email protected] © 2016 TU Graz 7
Interactive Maintenance of OLMs
• Editable learner models
– Full control for learners
– Learners need to be accurate and confident in self-assessment
• Persuadable learner models
– Changes validated by teacher or system
– Evidence based approach without full learner control
– Provide additional information, request/justify changes
• Negotiated learner models
– Learners challenging their model – separate viewpoints retained for system/student
www.leas-box.eu [email protected] © 2016 TU Graz 8
LEA‘s OLM persuasion facility
• Persuasion facility
– Initiated by student
– Try to persuade the system to change the model
– Challenge evidence or provide justifications
– Aim to find an agreement between system and student
• In order to
– Make the model more accurate
– Support reflection in learning
www.leas-box.eu [email protected] © 2016 TU Graz 9
LEA‘s OLM persuasion facility
Student System
Accept/Agree Agree with the system evidence Accept a compromise
Decline Decline system
proposed compromise Decline
(e.g. last negotiation too recent)
Compromise Propose a compromise between
current level and self-assessment Propose a compromise between
current level and self-assessment
Request evidence or justifications
Request evidence for current level
Request justification for self-assessment
Provide evidence or justifications
Justification (e.g. homework)
Provide evidence for current level
Self-assess Proposition of new OLM state
Challenge evidence
Disagreement with piece of evidence
Statement Fact about the OLM
www.leas-box.eu [email protected] © 2016 TU Graz 17
LEA‘s OLM persuasion facility
• Persuasion facility configured by teachers
– Maximum threshold to increase/decrease the level
• E.g. students cannot increase their level of more than 10% per persuasion
– Minimum time between persuasions
• E.g. students cannot change their level for the same competency more than once per day
– Minimum number of activities between persuasions
• E.g. after a persuasion, students have to do at least 3 activities before they can persuade it again
www.leas-box.eu [email protected] © 2016 TU Graz 18
Study: context
• Italian as a foreign language, Department of Modern Languages, University of Birmingham
• Definition of 155 quizzes on Canvas, a learning management system
www.leas-box.eu [email protected] © 2016 TU Graz 20
Study: context
• Italian as a foreign language, Department of Modern Languages, University of Birmingham
• Definition of 155 quizzes on Canvas, a learning management system but: – Only the last score of a quiz can be visualised
– No graphic visualisation of scores
– No competency/topic based approach
www.leas-box.eu [email protected] © 2016 TU Graz 21
Study: context
• Italian as a foreign language, Department of Modern Languages, University of Birmingham
• Definition of 155 quizzes on Canvas, a learning management system but: – Only the last score of a quiz can be visualised
– No graphic visualisation of scores
– No competency/topic based approach
Use the LEA’s OLM together with Canvas
www.leas-box.eu [email protected] © 2016 TU Graz 22
Study: definition of competencies
• Definition by teachers of
– 47 grammar competencies
– 86 vocabulary competencies
• Using the Common European Framework of Reference for Languages (CEFR)
– From A1 (beginner) to C2 (fluent)
www.leas-box.eu [email protected] © 2016 TU Graz 23
Study: setup
• Canvas exports the quiz scores in CSV files
• Teacher has to configure the LEA’s Box import tool once only
– Link quizzes and competencies
• 1 quiz can be linked to several competencies
• 1 competency can be linked to several quizzes
• Possibility to import the quiz scores quickly
www.leas-box.eu [email protected] © 2016 TU Graz 24
Study: results
• 15 students from University of Birmingham
• Possibility to use Canvas and the OLM during the entire semester (optional)
• Learner models built from
– Teacher assessments
– Canvas quizzes
– Persuasion outcomes
• All interaction logged; usage questionnaires; semi-structured interviews
www.leas-box.eu [email protected] © 2016 TU Graz 25
Study: students reaons to use or not persuasion
Answers to questionnaire
www.leas-box.eu [email protected] © 2016 TU Graz 26
Study: students reaons to use or not persuasion
Interviews with 5 students
www.leas-box.eu [email protected] © 2016 TU Graz 27
Study: students reasons to use or not use persuasion
Interviews with 5 students
• Reasons to use persuasion
– Make the model more accurate
• Partially completed quiz
• Correct answer considered as wrong by Canvas
• Right answers placed in wrong boxes
• Canvas technical problem
www.leas-box.eu [email protected] © 2016 TU Graz 28
Study: students reasons to use or not use persuasion
Interviews with 5 students
• Reasons not to use persuasion
– Model already accurate
– Difficulty of self-assessment
– Not technology confident
www.leas-box.eu [email protected] © 2016 TU Graz 29
Study: students reasons to use or not use persuasion
Interviews with 5 students
• Reasons to use persuasion later
– Willing to complete more quizzes first
– Use it closer to exam period
www.leas-box.eu [email protected] © 2016 TU Graz 30
Conclusion
• Use of a persuadable OLM at university
– Understand the evidence of the system
– Willing to make the model more accurate
– Capability to use persuasion efficiently
• Explain their point of view if they disagree
• Challenge evidence
• Provide justifications
www.leas-box.eu [email protected] © 2016 TU Graz 32
www.leas-box.eu [email protected] © 2016 TU Graz 33
www.leas-box.eu [email protected] © 2016 TU Graz 34
