International Journal of Computational Cognition (http://www.YangSky.com/yangijcc.htm)Volume 1, Number 4, Pages 31–51, December 2003Publisher Item Identifier S 1542-5908(03)10402-2/$20.00Article electronically published on December 9, 2002 at http://www.YangSky.com/ijcc14.htm. Please citethis paper as: 〈Grammatiki Tsaganou, Maria Grigoriadou and Theodora Cavoura, “ExperimentalModel for Learners’ Cognitive Profiles of Historical Text Comprehension(Invited Paper)”, InternationalJournal of Computational Cognition (http://www.YangSky.com/yangijcc.htm), Volume 1, Number 4, Pages31–51, December 2003〉.EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVEPROFILES OF HISTORICAL TEXT COMPREHENSION

(INVITED PAPER)

GRAMMATIKI TSAGANOU, MARIA GRIGORIADOU AND THEODORACAVOURA

Abstract. This contribution presents an experimental approach tolearner modeling of Historical Text Comprehension (HTC). The LearnerModel of Historical Text Comprehension (LMHTC), infers the learner’scognitive profile and profile descriptor of HTC from his observable be-havior imitating the human expert. The experimental research, whichresulted in the design of the learner model, is described in detail. Theexploitation of quantitative and qualitative criteria and the formula-tion of the learner’s cognitive profile and profile descriptor are alsopresented. The evaluation results are analyzed and followed by a dis-cussion on the interesting research results for the dynamics of theconstructed learner model.

1. Introduction

The research interest of this work focuses on the convergence of com-puter science and cognitive science. The tendencies that prevail concern theconstruction of systems based upon the study of the evolution of knowledgeprocesses in the development of Intelligent Tutoring Systems (ITSs). Thefields of psychology and computer science represent two major viewpointsinvolved in issues of ITSs. An ITS has two major components: a humanbeing in the form of the student and a computer system in the form of anintelligent system (Fink, 1991). Cognitive psychologists look at the fieldof ITSs as a way of performing experiments to test out theories of humancognition and learning. Computer scientists want to develop computationaltechniques that appear to generate intelligent behavior, which is to knowhow to program a computer so that it can understand and interact with

Received by the editors December 08, 2002 / final version received December 10, 2002.Key words and phrases. Learner model, experimental model, cognitive profiles, his-

torical text comprehension.

c©2002 Yang’s Scientific Research Institute, LLC. All rights reserved.

31

32 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

the outside world (Self, 1993). The component of an ITS that representsthe student’s current state of knowledge and understanding of the contentis called the student model (VanLehn, 1988).

The notion diagnosis refers to the pedagogical activities of a human ex-pert aiming at collecting and inferring information about the student or hisactions (Polson & Richardson, 1988). Because this action often involves theconstruction of a student model these activities have also been called stu-dent modelling (Wenger, 1987). The diagnostic module of an ITS uncoversa hidden cognitive state of the student from his observable behavior andrelates this behavior with the students’ cognitive models.

Text comprehension is a complex interaction of basic cognitive processes(Briton, 1996). In the literature, models of text comprehension are concen-trating on the puzzle called text comprehension in an effort to give explana-tions about the cognitive process individuals make during comprehension.In recent years researchers express their interest for various components oftext comprehension by developing theories that focus on deeper levels ofcomprehension (which involve pragmatics, knowledge-based inferences andworld knowledge, problem solving) rather than on shallow levels of com-prehension (such as lexical processing, syntactic parsing and the interpreta-tion of explicit text) (Kintsch 1998) (Trabasso 1985) (Johnson-Laird, 1983)(Baudet, 1992).

In the literature student models related with comprehension are the Sim-Students, which is an integrated student model of story and equation prob-lem solving using an ACT-R based cognitive model of MacLaren (MacLaren,2002), the Empirical Assessment of Comprehension of Mathan (Mathan,2002), the Engines for Education of Schank (Schank, 1994). Moreover, themodel of literacy comprehension of Zwaan (Zwaan, 1996), which takes intoaccount the predication semantics model of text comprehension and recallof Turner (Turner, 1996) and is based on the Construction-Integration (CI)model of Kintsch (Kintsch, 1975). The model of narrative comprehensionand recall of Flecher (Fletcher, 1996) is based upon the model of Trabasso &Van den Broek (Trabasso 1985), which considers understanding of text as aprocess of finding by the reader the causal path that links text’s opening toits final. The Model of Comprehension of Historical Narration (MOCOHN)is a pencil-and-paper diagnosis model, which gives an explanation of theway students represent the world of history and of the way their cognitiveprocesses lead to comprehension of a historical text (Cavoura, 1994). MO-COHN gives a formalism of students’ cognitive models of HTC (Tsaganou,2002).

In this work our interests focus on models of historical text comprehen-sion. In Section 2, we describe the MOCOHN model. In Section 3, we

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 33

outline the design problems and the research, which resulted in the con-struction of the experimental tool. In Section 4, we present the structureof the computational learner model and the cognitive models of HTC andreport the cognitive profiles and profile descriptors. In Section 5, practicalapplication and evaluation results are presented and analyzed. In Section 6,we conclude and give some short-term perspectives.

2. Modelling HTC

2.1. Models of Text Comprehension. Mental models according to Johnson-Laird’s theory, is the basic structure of cognition: They are constructed inworking memory as a result of perception, the comprehension of discourse,or imagination. Mental models that are formulated by all individuals, helpthem see the complex world in a simplified manner, so it is easier to under-stand, explain and predict the things that happen. Knowledge structuressuch as schemata and frames are hypothesized to represent ‘backgroundknowledge’. Mental models would be the instantiation of such structureswhen they are used to plan actions, explain and predict external events.Mental models reflect the structure of the fragments of the world that theyrepresent and impose a structure of them in return.

The theory of Schank & Abelson supports schemata that are hypothet-ical mental structures for representing generic concepts (Schank, 1977).Schemata are created through experience with people, objects, and eventsin the world. When new information cannot be accommodated merely, bytuning an existing schema, it results in the creation of a new schema. Nar-rative schemata specify expected components of a story, such as the timesequence of events, and causal relations that should connect the events.

The model of Van Dijk & Kintsch assumes that readers generate a vari-ety of knowledge-based inferences when they comprehend stories (Kintsch1998). The Construction-Integration (CI) model of Kintsch is a simula-tion that models how text representations are constructed, understood andintegrated with the reader’s knowledge.

The model of Trabasso & Van den Broek assumes that text comprehen-sion is a problem solving process (Trabasso 1985). According to the model,the meaning of a narrative text is represented in long-term memory as anetwork. The nodes of this network represent the individual clauses of thetext, whereas the links represent causal and enabling relations among thoseclauses. A reader’s ability to discover the causal connections is related withtext comprehension.

The model of Baudet & Denhiere assumes that the reader’s cognitivesystem utilizes certain fundamental semantic categories for establishing andorganizing the meaning of the text (Baudet, 1992), (Le Ny, 1989). The

34 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

categories are the object, the state, the event, the action and the relations(causal, temporal, topological, ecological etc.). The model of Baudet &Denhiere, utilizing the semantic categories, considers comprehension of thetext as attribution of meaning to causal connections in the text.

2.2. Models of HTC. Comprehension of historical text is a special kindof the complex and interactive cognitive process. HTC takes into accountthe properties that differentiate an historical narration against a fiction.Historical narration is considered as an argumentative discourse, as an un-breakable system with a start and an end. Historical narration is definedas a causal and transformational system. It is characterized as transforma-tional because it describes the representation of a historical change, whichhas happened in a particular place and time. It is characterized as causalbecause it describes an historical occurrence, which is interpreted by a seriesof causal links. Historical narration is based upon causal connections.

The reader utilizes certain fundamental cognitive categories for estab-lishing and organizing the meaning of the text (Briton, 1996). During com-prehension the reader attributes meanings to causal connections betweenoccurrences in the historical text (Cavoura, 1994). In the level of com-prehension as a cognitive task, the learner composes a representation of thehistorical text. This representation is a system, which contains the cognitivecategories: event, state and action. Comprehension of the historical text isassociated with causal connections and arguments made by the reader. Thearguments are based on the three cognitive categories. For the interpre-tation of learner’s cognitive processes we analyse his discourse tracing therecognition or not of the three cognitive categories.

2.3. The Model of Comprehension of Historical Narration (MO-COHN). The MOCOHN is a model of HTC (Cavoura, 1994), (Cavoura,2000). The model depends upon the narrative approach of historical knowl-edge (Ricoeur, 1983). Based upon the mental models of Johnson-Laird andthe theory of Schank & Abelson for text comprehension, MOCOHN adaptsthe theory of Baudet & Denhiere for HTC. The model considers text com-prehension as attribution of meanings to causal connections between occur-rences in the text. For the interpretation of learners’ cognitive processesMOCOHN analyses their discourse, searching for the presence or the ab-sence of the cognitive invariables: change and agency. The model uses thecognitive invariables in order to trace the fundamental cognitive categories:state, event and action, which our cognitive system uses to represent theworld.

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 35

Table 1. Fundamental cognitive categories and cognitiveinvariables. The presence of a cognitive invariable is repre-sented by (+) and the absence by (−).

Cognitive invariablesCognitive categorieschange agency

− − state+ − event+ + action

Table 1 depicts the relation rules between the fundamental cognitive cat-egories state, event and action and the cognitive invariables change andagency in MOCOHN.

(1) Absence of change combined with absence of agency denotes thecognitive category of the state. For example, “the very hard lifepeople led for many decades before the French Revolution” denotesa state, which does not include change in the world of the historicaltext.

(2) Presence of change combined with absence of agency denotes thecognitive category of the event. For example, “the heavy winter of1788-1789 before the French Revolution” denotes an event, whichincludes accidental change without human intervention.

(3) Presence of change combined with presence of agency denotes thecognitive category of the action. For example, “after decades theking gathers the General Classes on May of 1789” denotes an action,which includes change with human intervention.

The identification and the combination of the cognitive invariables and thesignificance of the cognitive categories in the learner’s discourse characterizeand interpret the meanings of causal connections the learner makes in thehistorical text.

3. The Experimental Model of HTC

3.1. Learner Models. The learner model uncovers hidden cognitive mod-els from observable behaviour (VanLehn, 1988). The learner model is thedata structure and the diagnostic module is a process that manipulates it.Learner modelling is defined as the process of building a representation ofthe learner’s knowledge from the evidence provided by learner inputs tosolve problems (Self, 1993). The input of the diagnostic unit is garneredthrough interaction with the learner. Describing the desired output of the

36 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

program for the available input we can specify design problems in com-puter science. The information available to the diagnosis module could beanswers to questions posed by the system. The output of the diagnosticmodule forms a data structure that is the learner model, which reflects thelearner’s cognitive model.

A main issue before developing a learner model is the discussion abouthandling the uncertainty in all its dimensions. Vagueness is coming fromknowledge communication between the human expert and the developer.Uncertainty due to approximation involved during data collection or to theabstract nature of human cognition or to loss of information resulting fromits quantification can lead to errors in learner diagnosis. Successful handlingof uncertainty can improve the quality of the learner model.

3.2. The design problems. For the construction of a learner model ofHTC learner’s attitude and characteristics are recorded. MOCOHN pro-poses discourse analysis of learner’s answers to questions for tracing thecognitive invariables in their answers and the priority that the learner at-tributes to an instance of a cognitive category. The combination and theinterpretation of the cognitive invariables and the degree of significance ofthe cognitive categories are used for the assessment of the learner’s HTC.

The problems that arise are:

a) How to represent the learner’s answers in order a computer to beable to trace and identify the cognitive invariables and the degreeof significance of an instance of a cognitive category.

b) How the computer can combine the cognitive invariables and thesignificance for the assessment of the learner’s cognitive model.

c) How to overcome the uncertainty and inaccuracy in knowledge com-munication among the human expert, the developer and the com-puter.

The selection of the historical text and the construction of questions in anappropriate form are very crucial in this point. The elaboration of learner’sanswers to questions, according to the rules of MOCOHN, is also a basicpoint.

3.3. The Construction of the Experimental Tool: LMHTC. Theabove considerations guided the construction of the LMHTC, which is anexperimental tool of learners’ cognitive profiles on HTC. The constructionprocess of the experimental tool is based on both the MOCOHN model andfurther research conducted with learners. The data recorded throughoutthe experiment were used for the construction of the computational form ofthe experimental tool.

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 37

3.3.1. The First Stage of the Research. Goals. The first goal of the researchwas the construction of a computational experimental tool for the diagno-sis of learner’s HTC based up on recognition or not of the three cognitivecategories in a historical text. These categories are represented by theirinstances in the historical text. The second goal of the research was theconstruction of the learner model of HTC and the formulation of the cog-nitive profiles of HTC in detail. 20 high school students participated inthe two stages experiment. All students attended the same public school.Participation in the experiment was voluntary.

First stage. In the first stage learners are given a historical text about theFrench Revolution with five factors due to which the outbreak of the FrenchRevolution was happened. The five factors are considered as instances ofthe three cognitive categories.

Figure 1. A screenshot depicting a historical text con-cerning the outbreak of French Revolution.

In the historical text, one instance represents the cognitive category event,one instance represents the cognitive category state and three instancesrepresent the cognitive category action. For every factor a question-pair issubmitted to the learner. The first question of the question-pair is relativeto the learner’s position about the significance of this factor for the outbreak

38 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

of the French Revolution and the second question is relative to the learner’sthe justification of this position.

Learners are asked to read the text and: (1) express their position: howstrongly each factor they fell was important for the outbreak of FR and(2) write down their justification: how they justify their position. Figure1 depicts the historical text with 5 factors concerning the outbreak of theFrench Revolution.

3.3.2. Elaboration of the Results. The answers given by the learners con-cerning position were classified in-groups as they are depicted in Table 2.

Table 2. Classification of given answers during researchconcerning position.

position answersscientific learners attribute minimum importance to events and

states and maximum or medium importance to actionstowards-scientific learners attribute medium importance to events and

statesnon-scientific learners attribute maximum importance to events or

minimum importance to actions

The answers given by the learners concerning justification were classifiedin-groups as they are depicted in Table 3.

The results were taken into consideration for the construction of the com-putational experimental tool. For each question the most representative an-swers of each of the above groups of answers were selected, supplemented,in case it was necessary, and were used as alternative answers. In its finalform the experimental tool has embedded alternative answers that reflectscientific thought, towards acquiring scientific thought, and non-scientificthought. Examples of alternative answers considered, which refer to posi-tions and to justifications are depicted in Figure 2. The answers a1 to a3are alternative answers to question 3a concerning the position, whereas theanswers b1 to b5 are alternative answers to question 3b concerning the cor-responding justification. Figure 2, also indicates the characterisations of theanswers, which are not visible to the learner. Answer a1 is non-scientific,a2 is towards-scientific and a3 is scientific answer. Answer b1 is scientific,b2 is non-scientific and the other answers are towards-scientific (b3 is acontinuity answer, b4 is a quantitative answer, b5 is an attitudes answerand b6 is an experiential answer).

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 39

Table 3. Classification of given answers during researchconcerning justification.

justification answersscientific Learners grounded their answers up on the scientific

historical thought.

towards-scientific

Learners based theiranswers on the com-mon sense schemas ofexperience, quantity,continuity and

Experiential: learnersused their own expe-rience or sentiment tostrengthen their posi-tion.

attitudes, which meanslearners are towardsacquiring scientificthought.

Quantity: learnersused quantitativecriteria to strengthentheir position.Continuity: learnersperceived the world ascontinuous.Attitudes: learners ex-pressed positive or neg-ative values (for exam-ple good, bad) towardsthe historical events.

non-scientific Learners gave cyclic answers based on recycling thequestions and consequently they refer to non- scien-tific thought.

3.3.3. Definition of the Argument. For every question-pair the combinationof the learner’s position and the corresponding justification constitute thelearner’s argument. An argument is defined as complete when both positionand justification are scientific. Otherwise the argument is non-complete.The expert defines the different degrees of argument completeness. Theargument completeness, which is associated with the recognition or not ofan instance of a cognitive category, is used as a vehicle to reveal the degreeof the recognition or not the corresponding cognitive category.

Table 4 demonstrates all possible combinations of position-justificationpairs, the corresponding argument completeness and the degree of recogni-tion of a cognitive category. Possible values of the argument completenessare: complete, almost complete, intermediate, nearly incomplete and incom-plete.

40 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

Figure 2. A screenshot depicting question-pair number3 concerning the historical text, alternative answers andcharacterizations of the answers.

In the characterization of argument completeness justification weightsmore than position does. So, towards-scientific position and scientific jus-tification result in almost complete argument, whereas the opposite onethat is scientific position and towards-scientific justification result in nearlyincomplete argument.

Argument completeness describes the learning difficulties of the learnerconcerning the cognitive categories, which the learner does not recognize.This qualitative characteristic of the arguments reflects the degree of recog-nition of a cognitive category that is the degree of comprehension of thehistorical text.

3.3.4. Classification of the Cognitive Categories. Historical actions consti-tute the core of the historical discourse. According to relevant research,during the comprehension of the historical text, the recognition of the cogni-tive category action is more important than the recognition of the cognitivecategory state (Cavoura, 1994). The recognition of the cognitive category

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 41

Table

4.

Arg

umen

tco

mpl

eten

ess

valu

esco

ncer

ning

posi

tion

-jus

tific

atio

nco

mbi

nati

ons.

pos

itio

nju

stifi

cati

onar

gum

ent

com

-ple

tenes

sar

gum

ent

char

-ac

teri

zati

onst

atus

ofre

cogn

itio

nof

the

corr

espon

din

gco

gnit

ive

cate

gory

scie

ntifi

csc

ient

ific

com

plet

esc

ient

ific

reco

gnit

ion

tow

ards

scie

ntifi

csc

ient

ific

alm

ost

com

plet

e

tow

ards

-sci

enti

ficto

war

ds-r

ecog

niti

onno

n-sc

ient

ific

scie

ntifi

cin

term

edia

tesc

ient

ific

non-

scie

ntifi

cne

arly

inco

mpl

ete

scie

ntifi

cto

war

ds-s

cien

tific

tow

ards

-sci

enti

ficto

war

ds-s

cien

tific

non-

scie

ntifi

cto

war

ds-s

cien

tific

inco

mpl

ete

non-

scie

ntifi

cno

n-re

cogn

itio

nto

war

ds-s

cien

tific

non-

scie

ntifi

cno

n-sc

ient

ific

non-

scie

ntifi

c

42 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

event is less important than the recognition of the cognitive category state.The cognitive categories are classified as follows:

(1) The recognition of the cognitive category action is more importantthan the recognition of the cognitive category state, as the historicalactions constitute the core of the historical narration.

(2) The recognition of the cognitive category event is less importantthan the recognition of the cognitive category the state.

Table 5 demonstrates the quality values of the cognitive categories. Pos-sible values of the quality are superior for the action, medium for the stateand inferior for the event.

Table 5. Quality values of the cognitive categories.

cognitive category action state eventcategory quality superior medium inferior

4. The Learner Model

4.1. The Structure of the Learner Model. Figure 3 presents the struc-ture of the learner model of HTC. Model tracing is the diagnostic techniquethat is used for the construction of the learner model. The idea behindmodel tracing is to create a close correspondence between units of the inter-nal model of the learner and single steps of observable behavior (Wenger,1987). The construction resulted in mapping learner’s answers (input) to acognitive model (output).

The model identifies the significance that the learner attributes to aninstance of a cognitive category, which constitutes his position, and the cor-responding justification of this position for all instances, which representthe cognitive categories in the input information. The system traces thecognitive map of knowledge structure and combines position and justifica-tion for every instance of a category and for all categories. It composes theargument completeness by combining learners’ answers to question-pairs.Finally the system makes the mapping of learner’s argument completenessto the recognition of the cognitive categories, to the learner’s cognitive pro-file and profile descriptor.

4.2. The Cognitive Profiles of HTC. The recognition of the three gen-eral cognitive categories: event, state and action are used to formulatethe cognitive models of HTC, which reflect the learners’ levels of historical

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 43

Table

6.

Cog

niti

vem

odel

s,nu

mbe

ran

dty

pesof

reco

gniz

edco

gnit

ive

cate

gori

esan

dco

rres

pond

-in

gco

gnit

ive

profi

les,

x>

1.

Cog

niti

vem

odel

sN

umbe

ran

dty

pes

ofre

cogn

ized

byth

ele

arne

rco

gnit

ive

cate

gori

esC

ogni

tive

profi

les

NH

T-

very

low

TA

HT

nx

TA

HT

11

even

tor

1st

ate

or1

acti

onlo

wTA

HT

1xm

ore

than

one

even

tsor

mor

eth

anon

est

ates

orm

ore

than

one

acti

ons

near

lylo

w

TA

HT

2(1

even

tan

d1

stat

e)or

(1ev

ent

and

1ac

tion

)or

(1st

ate

and

1ac

tion

)be

low

inte

rmed

iate

TA

HT

2x(m

ore

than

one

even

tsan

dst

ates

)or

(mor

eth

anon

est

ates

and

acti

ons)

or(m

ore

than

one

even

tsan

dac

tion

s)

abov

ein

term

edia

te

TA

HT

31

even

t,1

stat

ean

d1

acti

onne

arly

high

TA

HT

3xm

ore

than

one

even

ts,

stat

esan

dac

-ti

ons

high

HT

allev

ents

,st

ates

and

acti

ons

very

high

44 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

Identification and recording of: - the student’s positions - the student’s justifications

Student’s answers

Student’s cognitive profile and profile descriptor

-Tracing of the cognitive map of knowledge structure -Combination of the position and the corresponding justification for an instance, which represents a cognitive category and formulation of the argument completeness -Composition of argument completeness of all the arguments and for all their instances

Mapping of the student’s argument completeness to the recognition of the cognitive categories, to the cognitive model,

the cognitive profile and the profile descriptor

Figure 3. Structure of the learner model of HTC.

thought (Tsaganou, 2002). The learner’s cognitive profile of HTC is for-mulated taking into account the number of his complete arguments. Thecognitive profile expresses the recognition or not of the cognitive categories.

Table 6 depicts the cognitive models and the cognitive profiles of HTC.The general categories of cognitive models considered are Historical Thought(HT), Towards Acquiring Historical Thought (TAHTnx) and Non-HistoricalThought (NHT). TAHTnx cognitive models are categorized in more detailaccording to the number n of recognized by the learner categories and tothe number x of their instances in the historical text. TAHT1 means thatthe learner recognizes 1 instance of a cognitive category, whereas TAHT1x

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 45

means that the learner recognizes x instances of a cognitive category, wherex > 1. The same stands for TAHT2, TAHT2x, TAHT3 and TAHT3x. Thenumber n of recognized categories and the number x of recognized instancesof every cognitive category formulate the learner’s cognitive profile.

Learners with Very Low profile seem to have serious difficulties in think-ing historically. Learners characterized by terms like low, nearly low, belowintermediate, above intermediate, nearly high and high, seem to encounterdifficulties in thinking historically. Learners with very high profile seem tohave no learning difficulties in thinking historically.

4.3. The Profile Descriptor. The profile descriptor describes the learner’scognitive profile and denotes different perspectives of the profile. The learner’sprofile descriptor is formulated taking into account all of his argumentswhich may have different degree of completeness. The profile descriptor,which models uncertainty associated with observation, depicts the learner’sproblems in the recognition of the cognitive categories and reflects his learn-ing difficulties.

The cognitive profile low means that the learner recognizes one cogni-tive category. Knowing which is that category the system can give a moredetailed description about the learner. The profile descriptor carries de-tailed description pertaining to the quality of the cognitive categories andthe completeness of the arguments, which are attached to every cognitiveprofile.

For example, if a student selects the answers a3 and b1 of figure 2, thisconstitutes a complete argument of inferior category and indicates the recog-nition of one instance of the cognitive category (in this example the categoryevent). If a student selects the answers a2 and b4 this constitutes a nearlyincomplete argument of inferior category quality and indicates no recogni-tion of the cognitive category (event).

For example, a nearly low cognitive profile of a learner, during compre-hension of a historical text with 5 factors and 5 corresponding question-pairs,can be accompanied by the following profile descriptor : “The learner givesone complete argument of inferior category, one nearly incomplete argumentof superior category, one nearly incomplete argument of superior category,one incomplete argument of superior category and one incomplete argumentof medium category ”.

5. Practical Application and Evaluation

5.1. The Second Stage of the Research. In the second stage of theresearch the same students participated and were given the historical textwith 5 factors and question-pairs with the alternative answers. For each

46 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

question-pair, the student was asked to select the position and the justifica-tion, which were the nearest to him. The answers were recorded and wereused for the assessment of the students’ argument completeness, cognitiveprofiles and profile descriptors. In this stage answers were judged by hand.

5.2. Result Analysis. Figure 4 presents the cognitive profiles of the stu-dents participated in the second stage of the research. For example, students2, 10 and 19 have recognized one cognitive category and have low profile,whereas student 7 has recognized 2 instances of a cognitive category andhas nearly low profile.

Students' cognitive profiles

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Students

Cog

niti

ve p

rofi

les

Figure 4. Students’ cognitive profiles. 1, 2, 3, 4, 5, 6, 7,8 of y axis denote very low, low, nearly low, below average,above average, nearly high, high and very high cognitiveprofile respectively. 1, 2, 3, . . . , 20 of x axis denote thestudents.

Figure 5, presents the degree of completeness of the 5 arguments, whichrepresent the comprehension of the historical text and include the corre-sponding cognitive categories. So, figure 5, depicts the profile descriptors ofthe 20 students. For example student 5, who has very low cognitive profile,has given an incomplete argument for state, a nearly incomplete argumentfor action1, an almost complete argument for event, an intermediate argu-ment for action2 and a nearly incomplete argument for action3.

Figure 6 depicts the number of students’ answers (positions and justifi-cations) and the corresponding arguments: non-scientific, towards-scientificand scientific, which were judged by the expert. This means non-recognition,towards- recognition and recognition of the corresponding cognitive cate-gories respectively. The non-scientific arguments counted for a 51% of the

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 47

Students' Profile descriptors

0

1

2

3

4

5

6

Students

Arg

umen

t com

plet

enes

s

state 1 2 2 1 1 4 3 2 1 2 2 4 4 3 2 2 1 1 1 4

action1 2 1 3 1 2 4 3 1 2 4 1 2 4 3 1 3 2 2 2 5

event 3 2 1 2 4 5 3 4 3 5 3 3 3 4 4 1 3 3 4 3

action2 4 3 3 1 3 5 5 2 3 2 4 1 2 1 3 3 4 3 3 1

action3 2 3 2 2 2 4 5 2 4 3 5 4 4 4 2 2 2 4 1 4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Figure 5. Profile Descriptors of 20 students. 1, 2, 3, 4, 5 ofy axis denote incomplete, nearly incomplete, intermediate,almost complete and complete respectively. 1, 2, 3, . . . , 20of x axis denote the students.

total. The towards-scientific arguments counted for a 38% and the scien-tific arguments counted for an 11%. Regarding the scientific arguments, thescientific positions were 30% and the scientific justifications were 21%, butonly the 11% of the arguments were scientific.

0

20

40

60

non-scientific towards-scientific

scientific

characterization of arguments

num

ber

of a

rgum

ents

(%

)

position

justification

argument

Figure 6. Argument characterizations for different per-centage of positions, justifications and arguments.

Figure 7 depicts the distribution of argument completeness for the dif-ferent factors (which reflect the cognitive categories state, action1, event,

48 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

action2, action3). Regarding the cognitive category state, as argument com-pleteness increases the number of the given by the students arguments de-greases. The same analogy does not happen for the categories event andaction, as a small but remarkable number of corresponding arguments wereintermediate, almost complete or complete. This means that students facegreater difficulties in recognising the cognitive category state than the othercognitive categories. It is worth noticing, that no student gave any completeargument for the category state.

argument completeness distibution per instance of a cognitive category

0

10

20

30

40

50

60

incomplete nearlyincomplete

intermediate almostcomplete

complete

argument completeness

num

ber

of a

rgum

ents

(%) state

action 1

event

action 2

action 3

Figure 7. Distribution of the argument completeness.

Figure 8 shows the distribution of the cognitive profiles of the studentsparticipated in the second stage of the research. Most of the students hadvery low (65%) or low (20%) cognitive profile. The students who gave atleast 1 scientific argument (low, nearly low, below average cognitive profiles)counted for a 35% of the total. The students with all their argumentsnon-scientific or towards-scientific, are those with the very low cognitiveprofile. Taking into account the argument completeness, which is what theprofile descriptor does, we can have a more detailed analysis for the verylow cognitive profile. The cognitive profile indicates the number (out of5) of complete arguments of the student. Moreover, the profile descriptorindicates all the arguments and the degree of completeness.

What we have learned from the experiment results concern two views.First, the uncertainty that exists in knowledge communication between the

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 49

0

10

20

30

40

50

60

70

very low low nearlylow

belowaverage

aboveaverage

nearlyhigh

high very high

cognitive profiles

pers

enta

ge o

f stu

dent

s

Figure 8. Distribution of the cognitive profiles of the stu-dents participated in the second stage of the research.

expert and the designer of the learner model and the inaccuracies of theinformation gathered from measurements affect significantly the design ofthe learner model. For example, the use of alternative answers to question-pairs posed to learners introduces a first level of approximation in describinglearner’s observable behaviour, moreover in representing learner’s cognitivestate and process. For the construction of a functional learner model duringthe experimental research, our effort focused on handling this uncertaintyin knowledge acquisition. Second, The conducted research confirmed ourhypotheses that most of the learners have very low or low cognitive pro-file. The research focused on a more precise study concerning learners withvery low cognitive profiles aiming at the improvement of the learner model.We introduced the argument completeness, which was used to classify thearguments and then to describe the students’ cognitive profiles.

6. Conclusions and Future Plans

The presented methodology based on MOCOHN and on further exper-imental study, resulted in the construction of the LMHTC. This learnermodel constitutes a diagnostic mechanism for inferring the learners’ char-acteristics from their observable behavior by exploiting their answers toappropriate questions. It imitates human experts’ reasoning, by exploitingquantitative and qualitative criteria, and extends computationally an exist-ing paper-and-pencil model of HTC. The learner model infers the learner’scognitive profile of HTC and his profile descriptor.

50 G. TSAGANOU, M. GRIGORIADOU AND T. CAVOURA

The LMHTC is expected to interest mainly artificial intelligence re-searchers, human experts in history didactics and the educational society.Researchers in the area of artificial intelligence may be motivated by thisdiagnosis model and experiment implementation and use of it in the con-struction of intelligent tutoring systems. A model like that can reduce hu-man experts’ work by offering a tool for quick assessment of the learners’cognitive profiles on HTC. History teachers who wish to assess the histor-ical profiles of their learners in order to design and experiment a differentpersonalized teaching strategy may find interesting the model. The outlinedlearner model supports the use of different historical texts and thus the pro-cess of designing individualized history learning. Such a learner model canbe exploited for the assessment of a learner’s cognitive model.

The significance of the research results guides our future plans accordingto computational cognition. The representation of the learner that is built,while solving the learner modelling problem is useful for the construction ofthe diagnostic module of the learner model. To provide a general frameworkfor diagnosis we plan to build this cognitive model into the computer usingfuzzy logic and case based reasoning techniques.

References

[Baudet, 1992] audet S., Denhiere G. (1992). Lecture, comprehension de texte et sciencecognitive, Presses Universitaires de France, Paris.

[Briton, 1996] riton B., Graesser Ar., (1996). Models of Understanding Text, LawrenceErlbaum Associates Inc. Publishers, Mahwah, New Jersey.

[Cavoura, 1994] avoura Th. (1994). Modalites de l’ appropriation de la connaissance his-torique. These de Doctorat, Universite de Paris VII.

[Cavoura, 2000] avoura Th. (2000). From the Epistemology of History to its Didactics,Neusis, vol. 9, pp. 169-185 (in Greek)

[Fink, 1991] ink K. P. (1991). The role of Domain Knowledge in the Design of an In-telligent Tutoring System. In Intelligent Tutoring Systems (ed.) Burns H., PartlettW. J. Redfield C-L., Lawrence Erlbaum Associates Inc. Publishers, Hillsdale, NewJersey.

[Fletcher, 1996] letcher Ch., van den Broek P., Arthur E. (1996). A Model of Narra-tive Comprehension and Recall, In Models of Understanding Text, (eds) Britton B.,Graesser C. Lawrence Erlbaum Associates Inc. Publishers, Mahwah, New Jersey.

[Johnson-Laird, 1983] ohnson-Laird P.N. (1983). Mental Models: towards a cognitive sci-ence of language, inferences and consciousness, Cambridge University Press, Cam-bridge.

[Kintsch, 1975] intsch W. & Van Dijk T. A. (1975). Comment on se rappelle et on resumedes histoires, In Langage, no 40, pp. 98-116.

[Kintsch 1998] intsch W. (1998). Comprehension: a paradigm for cognition, CambridgeUniversity Press, UK.

[Le Ny, 1989] e Ny J.-F. (1989). Science cognitive et comprehension du langage, Paris,Presses Universitaires de France.

EXPERIMENTAL MODEL FOR LEARNERS’ COGNITIVE PROFILES 51

[MacLaren, 2002] acLaren B., Koedinger K. When and Why Does Mastery LearningWork: Instructional Experiments with ACT-R “SimStudents”, Proceedings of 6th

International Conference, ITS 2002, Spain, pp. 355-366, (2002).[Mathan, 2002] athan S., Koedinger R., An Empirical Assessment of Comprehension Fos-

tering Features in an Intelligent System. Proceedings of 6th International Conference,ITS 2002, Spain, pp. 330-343, (2002).

[Polson, 1988] olson M., Richardson J. (1988). Foundations of Intelligent Tutoring Sys-tems, Lawrence Erlbaum Associates Inc. Publishers, Hillsdale, New Jersey.

[Ricoeur, 1983] icoeur P. (1983). Temps et recit, tome I, Editions du Seuil, Paris.[Self, 1993] elf J., (1993). Model-based Cognitive Diagnosis, In User Modelling and User-

Adapted Interaction, 3, 89-106.[Schank, 1977] chank R. C., et Abelson R.P. (1977). Scripts, plans, goals and under-

standing, Lawrence Erlbaum Associates Inc. Publishers, Hillsdale.[Schank, 1994] chank, R. & Cleary, C., Engines for Education,

http://www.ils.northwestern.edu/e-for-e/index.html, (1994).

[Trabasso 1985] rabasso T. & Van den Broek P. (1985). Causal Thinking and the Repre-sentation of Narrative Events, Journal of Memory and Language, 24, pp. 612-630

[Tsaganou, 2001] saganou G., Grigoriadou M., Cavoura Th.: User Model of HistoricalText Comprehension. Proceedings of the Human Computers Interaction 2001, Pan-hellenic Conference with International Participation, Patra, Greece, December 2001.

[Tsaganou, 2002] saganou G., Grigoriadou M., Cavoura Th., (2002). Historical TextComprehension: Design of Experimental Tool for Students’ Cognitive Profiles, Copy-right 2002 IEEE. Proceedings of the Hawai’i International Conference on SystemSciences, Big Island Hawaii.

[Turner, 1996] urner A., Britton B., Andraessen P., McCutchen D., (1996). PredicationSemantics Model of Text Comprehension and Recall. In Models of UnderstandingText, (eds) Britton B., Graesser C. Lawrence Erlbaum Associates Inc. Publishers,Mahwah, New Jersey.

[VanLehn, 1988] anLehn K. (1988). Student modelling, In Foundations of IntelligentTutoring Systems, (eds) Polson M., Richardson J., Lawrence Erlbaum c Inc. Pub-lishers, Hillsdale, New Jersey.

[Wenger, 1987] enger E. (1987): Artificial Intelligence and Tutoring Systems. Computa-tional and Cognitive Approaches to the Communication Knowledge. M. KaufmannPublishers, Inc., California.

[Zwaan, 1996] waan R. (1996). Toward a Model of Literary Comprehension, In Modelsof Understanding Text, (eds) Britton B., Graesser C. Lawrence Erlbaum AssociatesInc. Publishers, Mahwah, New Jersey.

Grammatiki Tsaganoua, Maria Grigoriadoua and Theodora Cavourab,aUniversity of Athens, Department of Informatics and Telecommunications,GR-15784, Athens, Greece.bUniversity of Thessaly, Dept. of Education, Argonafton & Filellinon strs,GR-38221, Volos, Greece.

E-mail address: gram@di.uoa.gr(G. Tsaganou), gregor@di.uoa.gr(M. Grigoriadou),

theokav@pre.uth.gr(T. Cavoura)