An adaptive hierarchical questionnaire based on the Index of Learning Styles
Alvaro Ortigosa, Pedro Paredes, Pilar Rodriguez
Universidad Autónoma de Madrid
OPAH Research grouphttp://tangow.ii.uam.es/opah
OPAH
Alvaro Ortigosa – Universidad Autonoma de Madrid
Student
The Context
AEHS traditional model
AdaptedCourse
AdaptedCourse
Course definitionCourse
definition
A(E)HS
Alvaro Ortigosa – Universidad Autonoma de Madrid
Student
The Context
AEHS traditional model
AdaptedCourse
AdaptedCourse
Course definitionCourse
definition
User Model
A(E)HS
Alvaro Ortigosa – Universidad Autonoma de Madrid
The Context
AEHS traditional model
Asking the user (or teacher or…)Deducing / inducing from user behavior
Student User Model
Alvaro Ortigosa – Universidad Autonoma de Madrid
Adapting to LS: an exampleILS VALUE ON SEQUENTIAL/GLOBAL:
Extreme and mild Sequential Well balanced
Extreme and moderate Global
Alvaro Ortigosa – Universidad Autonoma de Madrid
Adapting to LS: an exampleILS VALUE ON SEQUENTIAL/GLOBAL:
Extreme and mild Sequential Well balanced
Extreme and moderate Global
Alvaro Ortigosa – Universidad Autonoma de Madrid
Context: ILS questionnaire
For each of the four dimensions 11 questions, 2 possible answers 12 different possible values
It provides a lot of opportunities for adaptation
Alvaro Ortigosa – Universidad Autonoma de Madrid
But…
(At least in Engineering fields) Students are not motivated to fulfill questionnaires 44Q x LS + 60Q x Personality + 15’ test x IQ Surveys about teacher performance,
workload, “Bologna system”, etc. etc. “Is it part of the evaluation?”
Students tend to answer more careless as they go through the questions
As the number of questions grows, answers become less reliable
Alvaro Ortigosa – Universidad Autonoma de Madrid
However…
In our experience with teachers, most of the times they just require categorization
-11 -9 -7 -5 -3 -1 1 3 5 7 9 11
Sequential Neutral Global
-11 -9 -7 -5 -3 -1 1 3 5 7 9 11
Alvaro Ortigosa – Universidad Autonoma de Madrid
Aha!
If only three categories are needed, would it be possible to ask fewer questions?
If possible, which questions (among the 11 for a given dimension) would provide more (enough) information about the student learning style?
No, I don’t mean the AH system ;)
1) I understand something better after I a) try it out b) think it through2) I would rather be considered a) realistic b) innovative
Alvaro Ortigosa – Universidad Autonoma de Madrid
The goal
To ask each student as few questions as possible
We don’t even need to ask the same questions!
Alvaro Ortigosa – Universidad Autonoma de Madrid
The goal (II)
Not a new questionnaire, but an adaptive version of the ILS
In groups
Alone
Something Ihave done
Something Ihave thoughta lot about
…
Alvaro Ortigosa – Universidad Autonoma de Madrid
The idea
Using a database of actual answers from real students
To use machine learning techniques in order To find most relevant questions for each
dimension Depending on previous answers
Alvaro Ortigosa – Universidad Autonoma de Madrid
Using classification techniques
ModelModel
Training examples(instances)
Learning algorithm
Newinstances
Classified Instances
Alvaro Ortigosa – Universidad Autonoma de Madrid
How does a classifier work?
Each instance is represented by a set of attribute values.
Training examples are (usually) already classified.
Classifier model (usually) uses a subset of attributes (conditions, linear combinations, etc.)
Each student represented by her answers to the 11 questions
The class is the category she belongs
Which attributes (questions) does the learnt model use?
-11 -9 -7 -5 -3 -1 1 3 5 7 9 11
Sequential Neutral Global
Alvaro Ortigosa – Universidad Autonoma de Madrid
Classification trees
In classification trees, each node tests a single attribute (question).
Classification trees explicitly shows the learnt model. It points to the relevant questions.
Different branches on a classification tree can test different attributes.
Tree construction aimed to get shorter paths C4.5 algorithm chooses next attribute
(question) based on the information gain.
Alvaro Ortigosa – Universidad Autonoma de Madrid
Data collection
Three different samples: 42 secondary school level students. 88 post-secondary level students. 200 university level students
Between 15 and 30 years old 101 women and 229 men
Alvaro Ortigosa – Universidad Autonoma de Madrid
Data collection (II)
Active/reflective Sensing/intuitive
Visual/verbal Sequential/global
Alvaro Ortigosa – Universidad Autonoma de Madrid
Results I: Active/Reflective dim
Alvaro Ortigosa – Universidad Autonoma de Madrid
Results II: Sensing/Intuitive dim
Alvaro Ortigosa – Universidad Autonoma de Madrid
Results III: Visual/Verbal dim
Alvaro Ortigosa – Universidad Autonoma de Madrid
Results IV: Sequential/Global dim
Alvaro Ortigosa – Universidad Autonoma de Madrid
Results V: the four dimensions
Other results seem to indicate: a) The relevance of a question does not vary
significantly with the age of the student. b) The trees seem to converge to a common tree,
independently from the origin of the sample, or at least to a common subset of questions.
Alvaro Ortigosa – Universidad Autonoma de Madrid
Conclusions
Some questions of the ILS provide more information than others.
We were able to build dynamic (shorter) questionnaires with high precision. On the average, 4-5 questions needed for each
dimension. The size of the sample (>300) enough for providing
good information about 11 questions. Ad-hoc trees would be better only if the sample is
large enough. Gender does not seem to affect the outcome
Alvaro Ortigosa – Universidad Autonoma de Madrid
Some limitations
More categories will require more questions and larger training sets
The approach is not useful when the exact value for each dimension is needed For example, automatic grouping
Alvaro Ortigosa – Universidad Autonoma de Madrid
Thank you! Questions?