Date post: | 11-Apr-2017 |
Category: |
Data & Analytics |
Upload: | janningr |
View: | 251 times |
Download: | 0 times |
Feature Analysis for Affect RecognitionSupporting Task Sequencing
in Adaptive Intelligent Tutoring Systems
Ruth Janning, Carlotta Schatten, Lars Schmidt-Thieme
Information Systems and Machine Learning Lab,University of Hildesheim, Germany
EC-TEL, 19 September 2014
Feature Analysis for Affect Recognition
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 1 / 18
Feature Analysis for Affect Recognition 1. Introduction
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 1 / 18
Feature Analysis for Affect Recognition 1. Introduction
(Adaptive) intelligent tutoring systems:
I Important tool for supporting education of students(e.g. in learning fractional arithmetic)
I Advantages:I Possibility for a student to practice any timeI Possibility of adaptivity and individualization for a single
student
I Possesses internal model of the student & a task sequencerI Originally, task sequencing needs expert and domain knowledgeI Former work1: new efficient task sequencer only using former
performance information→ uses performance prediction (by matrix factorization)to sequence tasks according to the theory of Vygotskys Zoneof Proximal Development
1Schatten, C. and Schmidt-Thieme, L. 2014. Adaptive Content Sequencingwithout Domain Information. In Proceedings of the Conference on computer supportededucation (CSEDU 2014).
R. Janning et al. EC-TEL, 19 September 2014 2 / 18
Feature Analysis for Affect Recognition 1. Introduction
Proposal:Support the task sequencer and performance prediction systemusing Vygotskys theory by affect recognition on features extractedfrom speech input
→ Indicates, if the last task wastoo easy,too hard orappropriatefor the student (according to the theory of Vygotskys Zone ofProximal Development)
R. Janning et al. EC-TEL, 19 September 2014 3 / 18
Feature Analysis for Affect Recognition 1. Introduction
Contributions:
I Proposal of a new approach for supporting task sequencing byaffect recognition
I Proposal and analysis of features for affect recognitionextracted from students speech input
I Verification of the possibility to use the proposed features foraffect recognition for supporting task sequencing in adaptiveintelligent tutoring systems
R. Janning et al. EC-TEL, 19 September 2014 4 / 18
Feature Analysis for Affect Recognition 2. Features, Classes, Instances, Methods
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 4 / 18
Feature Analysis for Affect Recognition 2. Features, Classes, Instances, Methods
To clarify:
I What kind of features shall be used?
I What kind of classes shall be used?
I Which instances shall be mapped to features and labelled withthe class labels?
I Which methods shall be used?
R. Janning et al. EC-TEL, 19 September 2014 5 / 18
Feature Analysis for Affect Recognition 2. Features, Classes, Instances, Methods
Features:Instead of linguistic features:→ Disfluencies features (speech pauses)
I Do not inherit error of speech recognition
I Independent from need that students use words related toaffects
Classes:
I Goal: neither bore the student with too easy tasks norfrustrate him with too hard tasks, but keep him in the Zone ofProximal Development
I → Get an answer to the question “Was this task too easy, toohard or appropriate for the student?”
I → Classes: Perceived Task-Difficulty(Labels: under-challenged, over-challenged, flow)
R. Janning et al. EC-TEL, 19 September 2014 6 / 18
Feature Analysis for Affect Recognition 2. Features, Classes, Instances, Methods
Instances:Which instances shall be mapped to features and labeled with theclass labels?
I We need at the end of a task the information, if the taskoverall was too easy, too hard or appropriate for the student.→ Instances: whole speech input of a student for one task
Methods:I Extracting speech pause information:
I Energy threshold on the decibel scale
I Affect recognition:I Support vector machine (classification method)
R. Janning et al. EC-TEL, 19 September 2014 7 / 18
Feature Analysis for Affect Recognition 3. Real Data Set
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 7 / 18
Feature Analysis for Affect Recognition 3. Real Data Set
Study:I 10 German students
I Age: 10 to 12 years
I Labelling (concurrently by tutor, retrospectively by secondreviewer) of perceived task-difficulties per task and per student
I Shown to students: paper sheet with fraction tasks
I Speech and screen recordings
R. Janning et al. EC-TEL, 19 September 2014 8 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 8 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Measurements
Symbol Explanation
m Number of students
pi Total length of pauses of student i
si Total length of speech of student i
npi Number of pause segments of student i
nsi Number of speech segments of student i
p(x)i xth pause segment of student i
s(y)i y th speech segment of student i
nti Number of tasks shown to student i
nci Number of correctly solved tasks by student i
score Overall score for student i (ncinti
)
R. Janning et al. EC-TEL, 19 September 2014 9 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Speech Features
Feature Formula
Ratio between pauses and speech pisi
Frequency of speech pause changesnpi +nsi
maxj (npj +nsj )
Percentage of pauses of input speech data pi(pi+si )
Length of maximal pause segment maxx(p(x)i )
Length of average pause segment∑
x p(x)i
npi
Length of maximal speech segment maxy (s(y)i )
Length of average speech segment∑
y s(y)i
nsi
Average number of seconds needed per task (pi+si )nti
R. Janning et al. EC-TEL, 19 September 2014 10 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Speech Features
Speech pauses carry useful information about students behaviour:
I Study showed e.g. tendency of over-challenged students tostay longer in silent thinking phases than students in flow.
Encoding of the perceived task-difficulty labels for statistical tests:
I 0 = over-challenged
I 1 = over-challenged/flow
I 2 = flow
I 3 = flow/under-challenged
I 4 = under-challenged
R. Janning et al. EC-TEL, 19 September 2014 11 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Single Speech Features
Investigating relevance of features:
1. Mapping the single feature values to the labels
2. Doing a linear regression
3. Measuring p-value: indicating the statistical significance
4. Measuring (Adjusted) R2 value: indicating how well theregression line can approximate the real data points
Best single speech features:
Feature p-value R2 Adjusted R2
Maximal pause 0.0678 0.3577 0.2774
Average length of speech 0.0873 0.3217 0.2369
Percentage of pauses 0.0923 0.3136 0.2278
R. Janning et al. EC-TEL, 19 September 2014 12 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Speech Feature Combinations
Investigating relevance of feature combinations:
1. Mapping all considered feature values to the labels
2. Doing a multivariate linear regression
3. Measuring p-value: indicating the statistical significance
4. Measuring (Adjusted) R2 value: indicating how well theregression hyper-plane can approximate the real data points
R. Janning et al. EC-TEL, 19 September 2014 13 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Speech Feature Combinations# Features p-value R2 Adjusted R2
Ratio pause speech, frequency of changes,5 seconds per task, average length of pause, 0.0284 0.9158 0.8106
average length of speechFrequency of changes, seconds per task,
5 average length of pause, maximal speech, 0.0305 0.9127 0.8035average length of speech
4 Ratio pause speech, frequency of changes,average length of pause, average length of speech 0.0154 0.8818 0.7872
4 Frequency of changes, average length of pause,maximal speech, average length of speech 0.0272 0.8501 0.7302
4 Ratio pause speech, frequency of changes,seconds per task, average length of speech 0.0288 0.8465 0.7236
4 Frequency of changes, seconds per task,maximal speech, average length of speech 0.0354 0.8324 0.6984
4 Frequency of changes, seconds per task,average length of pause, average length of speech 0.0420 0.8199 0.6759
3 Ratio pause speech,frequency of changes, average length of speech 0.0117 0.8207 0.7311
3 Frequency of changes,average length of pause, average length of speech 0.0175 0.7944 0.6916
3 Frequency of changes,maximal speech, average length of speech 0.0242 0.7699 0.6549
2 Frequency of changes,average length of speech 0.0327 0.6238 0.5163
→ There are statistically significant feature combinations.→ Features are able to describe perceived task-difficulty.
R. Janning et al. EC-TEL, 19 September 2014 14 / 18
Feature Analysis for Affect Recognition 4. Feature Analysis
Histograms as Features
Over-challengedstudent:
0 20 40 60 80 100 120
05
1015
2025
Pause length
Fre
quen
cy
0 20 40 60 80 100
02
46
810
Speech length
Fre
quen
cy
Studentin flow:
0 20 40 60 80 100 120
05
1015
2025
Pause length
Fre
quen
cy
0 20 40 60 80 100
02
46
810
Speech length
Fre
quen
cy
R. Janning et al. EC-TEL, 19 September 2014 15 / 18
Feature Analysis for Affect Recognition 5. Supporting Task Sequencing
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 15 / 18
Feature Analysis for Affect Recognition 5. Supporting Task Sequencing
(1) Use perceived task-difficultyfor rules:too easy → harderappropriate → similar difficulttoo hard → easier
(2) Feed feature values or affectinto performance predictor.
I E.g. difference between realperformance and scorecomputed by featuresindicates outliers (like if astudent felt to be in a flowbut his score is worse).
(3) Learn personalised value forthreshold based on VygotskysZone of Proximal Developmentused by the sequencer.
Task sequencer
Performanceprediction
Affectrecognition
Featureextraction
Next task
(1)
(1)
(2a)
(2b)
Formerperformances
Speechinput
R. Janning et al. EC-TEL, 19 September 2014 16 / 18
Feature Analysis for Affect Recognition 6. Conclusions
Contents
1. Introduction
2. Features, Classes, Instances, Methods
3. Real Data Set
4. Feature Analysis
5. Supporting Task Sequencing
6. Conclusions
R. Janning et al. EC-TEL, 19 September 2014 16 / 18
Feature Analysis for Affect Recognition 6. Conclusions
Proposed:
I Approach for supporting task sequencing by affect recognition
I Appropriate speech features
Shown:
I There are statistically significant feature combinations forperceived task-difficulty
I → The proposed features are applicable for affect recognitionsupporting task sequencing in intelligent tutoring systems
Next steps:
I Training of an appropriate affect recognition method
R. Janning et al. EC-TEL, 19 September 2014 17 / 18
Feature Analysis for Affect Recognition
Thank youfor your attention!
This work is co-funded by the EU project iTalk2Learnwith grant agreement no. 318051.
Project webpage: http://www.italk2learn.eu
R. Janning et al. EC-TEL, 19 September 2014 18 / 18