21
#pearsoncite
Measuring Learning Impact Using Next Gen Tools:A Learner-Centered View of Learning Affect, Behavior and Cognition
#pearsoncite
Robert Christopherson
Director of Training
iMotions Inc.
LinkedIn: robertchristopherson
Twitter: @iMotionsGlobal
Dan Shapera, PhDc
Manager, Design-based Research
Learning Experience Design
Pearson
#pearsoncite
@iMotionsGlobal
Thinking
Feeling
Doing
Learning
#pearsoncite
@iMotionsGlobal
Cognition
Motivation
Flow
Engagement
Attitude
Emotion
Facial Expressions
Verbalization
Behavior
Performance
Physical Interactions
Posture
#pearsoncite
Survey • Self-Report • Assessment • Input • Observation
@iMotionsGlobal
Measurement
#pearsoncite
@iMotionsGlobal
Continous
Planned
Interval
Spontaneous
Timing
ObservationalAnalysis
CognitiveWalkthrough
Think-aloud
Survey &Questionnaires
Interviews &Focus Groups
HeuristicEvaluation
Objective
Subjective
Qualitative Quantitative
Biometric
Sensors
Research Tools• Biometric tools
enhance traditional Human research methods
• Increase in the use of biometric research tools
• Insight into what people are thinking and feeling
Equalibrium
(Flow/Engagement)
Disequalibrium
(Confustion)Stuck
(Frustration)
Disengagement
(Boredom)
Novelty
(Surprise) Achievement
(Delight)
D’Mello, S., & Graesser, A.
(2012). Dynamics of
affective states during
complex learning. Learning
and Instruction, 22(2),
145–157.
Model of Affect in Learning(Zone of Proximal development)
#pearsoncite
Flow
#pearsoncite
@iMotionsGlobal
#pearsoncite
Human Behavior Research – Lab Setup
Stimuli with Eye Tracking Gaze
+
Live Sensor Data+
SensorsConnected+
Stimuli
+
GSR
+ EEG+
Eye Tracker+
Webcam for Facial Expressions
+
Operator live monitor Respondent being tested
Lab Setup
Interview Mobile Setup
Mobile Stand
Case Study #1: Task Exploration
Research Goal:Explore the cognitive and visual engagement of similar tasks in significantly different environments.
Research Question:What biometric markers are unique and common among cognitively engaging tasks in different digital environments?
Sensors/Variables Measured:
● EEG B-Alert x10 (Engagement, Distraction, Workload, Frontal Asymmetry)
● Eye Tracking (Fixation, AOI)
Results: Still being evaluated
● Reading tasks - high levels of
Engagement and workload depend on reading difficulty and chunking of information.
● Cognitive tasks - Varying levels of workload. Higher for task switching
Pearson Design Application
● Establishing Benchmarks for cognitive and affective engagements.
Lumosity
LA TimesBuzz Feed
Flappy Birds
Case Study #3: Instruction Materials and Engagement
Research Goal: Better understand the differences in the delivery of instructional content.
Research Question: How do text, video, and interactive versions of the same lesson differ in student engagement and frustration?
Sensors/Variables Measured:
● EEG Headset (Engagement, Frustration, Excitement Meditation)
● Eye-Tracker (Fixations, Pupil Dilation, Areas of Interest, Scan Paths, Attention Maps)
Results:
● interactive condition showed statistically significant higher levels of engagement than both PDF and video
● Interactive condition showed statistically significant lower levels of frustration than both PDF and video
○ (Check Your Understanding)
Pearson Design Application:
● desirable difficulty
● learning model design
● DBR and other formative testing
Role of Learner Affect in Design-based Research
(Stemberger & Cencic, 2014;
Sanders & Stappers, 2013)Consider Outcomes/Goals Classroom Culture
Instructor Abilities/ Training
Characteristics of the tasks
Design of Instructional Sequence
Integration of Digital tools
++
+
+
Student Motivation
+
Student Expectations
Student Prior Knowledge
+
+
#pearsoncite
Q&A
@iMotionsGlobal
#pearsoncite
Contact usRobert Christopherson
Director of Training
iMotions Inc.
LinkedIn: robertchristopherson
Twitter: @iMotionsGlobal
Dan Shapera, PhDc
Manager, Design-based Research
Learning Experience Design
Pearson
@iMotionsGlobal
● iMotions
● Facebook Usability Study Incorporating Physiological Sensors for Extracting Learning Potential
● Using Facebook to Increase Student Motivation
● Online Learning & Facebook
● Students Research Connections Between Facebook & Online Learning
● Calvo, R. a, Member, S., & Mello, S. D. (2010). Affect Detection : An Interdisciplinary Review of Models , Methods , and their Application to Learning Environments, 1(1), 1–23.
● Liu, C., Agrawal, P., Sarkar, N., & Chen, S. (2009). Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback. International Journal of Human-Computer Interaction, 25(6), 506–529. http://doi.org/10.1080/10447310902963944
● Baker, R. S. J. D., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human Computer Studies, 68(4), 223–241. http://doi.org/10.1016/j.ijhcs.2009.12.003
● D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. http://doi.org/10.1016/j.learninstruc.2011.10.001
● Sabourin, J., Mott, B., & J., L. (2011). Modeling Learner Affect with Theoreticallly Grounded Dynamic Bayesian Networks . Proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction, 286–295.
● Nacke, L. E. (2009). Affective Ludology. Technology, 2(8), 1653–2090. http://doi.org/10.1038/nchem.721
