Expert Systems with Applications 40 (2013) 2092–2101

Contents lists available at SciVerse ScienceDirect

Expert Systems with Applications

journal homepage: www.elsevier .com/locate /eswa

Comparing strategies for modeling students learning styles throughreinforcement learning in adaptive and intelligent educational systems:An experimental analysis

Fabiano A. Dorça a,b,⇑, Luciano V. Lima b, Márcia A. Fernandes a, Carlos R. Lopes a

a Faculty of Computer Science (FACOM), Federal University of Uberlândia (UFU), Campus Santa Monica, Bloco 1B, Sala 1B148, Av. João Naves de Avila, 2.121, Bairro Santa Monica,CEP 38400-902, Uberlândia/MG, Brazilb Faculty of Electrical Engineering (FEELT), Federal University of Uberlândia (UFU), Campus Santa Monica, Bloco 1B, Sala 1B148, Av. João Naves de Avila, 2.121, Bairro Santa Monica,CEP 38400-902, Uberlândia/MG, Brazil

a r t i c l e i n f o a b s t r a c t

Keywords:Student modelingLearning stylesAdaptive and intelligent educationalsystemsReinforcement learningStudent evaluatione-Learning

0957-4174/$ - see front matter � 2012 Elsevier Ltd. Ahttp://dx.doi.org/10.1016/j.eswa.2012.10.014

⇑ Corresponding author at: Faculty of ComputeUniversity of Uberlândia (UFU), Campus Santa MoniJoão Naves de Avila, 2.121, Bairro Santa Monica, CEPBrazil. Tel.: +55 34 3239 4144; fax: +55 34 3239 439

E-mail addresses: fabiano@facom.ufu.br (F.A. Dorçmarcia@facom.ufu.br (M.A. Fernandes), crlopes@facom

A huge number of studies attest that learning is facilitated if teaching strategies are in accordance withstudents learning styles, making the learning process more effective and improving students perfor-mances. In this context, this paper presents an automatic, dynamic and probabilistic approach for mod-eling students learning styles based on reinforcement learning. Three different strategies for updating thestudent model are proposed and tested through experiments. The results obtained are analyzed, indicat-ing the most effective strategy. Experiments have shown that our approach is able to automatically detectand precisely adjust students’ learning styles, based on the non-deterministic and non-stationary aspectsof learning styles. Because of the probabilistic and dynamic aspects enclosed in automatic detection oflearning styles, our approach gradually and constantly adjusts the student model, taking into accountstudents’ performances, obtaining a fine-tuned student model.

� 2012 Elsevier Ltd. All rights reserved.

1. Introduction

A large number of studies attest that learning is facilitated if theteaching strategies are in accordance with the students learningstyles (LS), making the learning process more effective and consid-erably improving students performances, as pointed out by Haider,Sinha, and Chaudhary (2010), Graf, Liu, and Kinshuk (2008),Kinshuk, Liu, and Graf (2009), Graf and Liu (2008), Bajraktarevic,Hall, and Fullick (2003).

But, traditional approaches for detection of LS are inefficient(Graf & Lin, 2007, Price, 2004). Price (2004) analyzes the uncer-tainty aspect of the index of learning styles questionnaire (ILS)by identifying inconsistencies between its results and students’behavior. Roberts and Erdos (1993), as well as Price, analyzes thiskind of instrument and the problems related to it. Castillo et al. as-serts that the information about the students’ LS acquired by psy-

ll rights reserved.

r Science (FACOM), Federalca, Bloco 1B, Sala 1B148, Av.

38400-902, Uberlândia/MG,2.a), vieira@ufu.br (L.V. Lima),.ufu.br (C.R. Lopes).

chometric instruments encloses some degree of uncertainty(Castillo, Gama, & Breda, 2005).

Therefore, automatic approaches tend to be more accurate andless susceptible to errors, since they analyze data derived from aninterval of time, instead of data collected at a particular point intime (Graf, Kinshuk, & Liu, 2009a). According to (Giraffa, 1999), arealistic student model (SM) requires a dynamic updating whilethe system continuously evaluates the student’s performance.

One problem with automatic approaches is related to obtainingsufficiently reliable information, in order to build robust and reli-able SM (Graf & Lin, 2007). So, building this type of approach basedon a probabilistic model is an important research problem (Danine,Lefebvre, & Mayers, 2006).

In this context, this paper presents an automatic, dynamic andprobabilistic approach based on reinforcement learning (RL)(Sutton & Barto, 1998) for modeling students LS. The focus of thispaper is the evaluation and comparison of three different strategiesfor updating the SM during the learning process.

The LS theory that supports this approach is the LS modelproposed by Felder (1988), the Felder–Silverman’s learning stylesmodel (FSLSM). The FSLSM stands out from other theories by com-bining the main LS models, as pointed out by Kinshuk et al. (2009).Moreover, the FSLSM is the most often used in the construction ofadaptive and intelligent educational systems (AIES) (Graf &Kinshuk, 2009).

F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101 2093

The following sections of this paper are described below. Sec-tion 2 analyzes the related work. Section 3 presents in detail theproposed approach. Section 4 presents and analyzes the results ob-tained through experiments. Finally, Section 5 presents conclu-sions and discusses future work.

2. Related work

Some recent studies have presented proposals for automaticdetection of LS (Cha et al., 2006; Graf & Kinshuk, 2010; Graf &Liu, 2008; Graf et al., 2009a; Graf & Viola, 2009; Limongelli, Sciar-rone, Temperini, & Vaste, 2009). These approaches use determinis-tic inference systems based on predefined behavioral patterns toinfer students LS. One of the problems with these approaches isthe uncertainty, difficulty and complexity of developing and imple-menting rules which are able to infer LS effectively from students’actions, and to treat students’ behavior as evidences and not aspossibilities.

Furthermore, these proposals ignore important considerationsraised by Graf and Liu (2008), Marzano and Kendall (2007), Mes-sick (1976), Graf and Lin (2007), Felder and Spurlin (2005), Robertsand Erdos (1993), which are related to the non-deterministic as-pect of students behavior and to the dynamic aspect of LS. In thiscontext, the approach presented in this paper brings advances inconsidering students LS as probabilities and not as certainties.

More complex approaches can be seen in Kelly and Tangney(2005), García, Amandi, Schiaffino, and Campo (2007), Carmonaand Castillo (2008), Cabada, Estrada, and Garcia (2009), Zatarain-Cabada et al. (2009), Zatarain, Barrón-Estrada, Reyes-García, andReyes-Galaviz (2010), Carmona, Castillo, and Millán (2007). Theseapproaches use learning machine techniques, such as Bayesianand Neural Networks. Some of the problems with these approachesare both high complexity and computational cost, which arethought to be serious concerns when considering a high numberof students using the system simultaneously. Besides, in general,these approaches are highly coupled, either to the system or tothe whole teaching process, making them harder to be re-used inother systems. In some of these approaches, once acquired, the stu-dents’ LS remain the same throughout the entire learning process(Castillo et al., 2005).

Moreover, we highlight the high difficulty and high degree ofsubjectivity in the task of relating LS to students behavioral pat-terns in AIES, as pointed out by García et al. (2007). Consequently,obtaining training pairs is a complex and doubtful task, generatinguncertain, data which may contain inconsistencies, resulting inmisleading training of the network, and severely compromisingthe adaptivity process.

In this context, we strongly believe that an approach whichlearns in an unsupervised manner eliminates many difficultiesand problems encountered in traditional approaches for automaticdiagnosis of LS. Furthermore, the approach proposed in this paper isbased on RL, which has as fundamental characteristics the incre-mental learning and the avoidance of using specific knowledge ofthe application domain, making the method more general and moreeasily reusable.

Table 1Probabilistic LS.

LSp

Processing Perception

Active Reflective Sensitive Intuitive

0.28 0.72 0.09 0.91

3. Proposed approach

In this approach, students LS are stored as probability distribu-tions in the SM, indicating the probability of preference for each LSwithin each of the four dimensions of the FSLSM, here called prob-abilistic LS (LSp). Thus, we propose a probabilistic SM in which LSare processed by the system as probabilities, and not as certainties.Table 1 shows an example of LSp, representing a student who prob-ably is reflective, intuitive, verbal and sequential.

If a self-assessment questionnaire is used for initialization ofLSp, as ILS (Felder & Spurlin, 2005), the SM can be booted fromthe data obtained by the questionnaire, considering the proportionof responses scored for each LS inside a dimension. If any self-assessment questionnaire is used, LSp is initialized with 0.50(undefined preference).

Therefore, we consider students’ preferences as probabilities inthe four-dimensional FSLSM model. Due to the probabilistic natureof LS in the FSLSM model, our approach is based on probabilistic LScombinations (Franzoni & Assar, 2009). A LS combination (LSC) is a4-tuple composed by one preference from each FSLSM dimension,as stated by Definition 3.1.

Definition 3.1. Learning styles combination (LSC)

LSC ¼ ða; b; c;dja 2 D1; b 2 D2; c 2 D3;d 2 D4Þ

considering:

D1 ¼ fActiveðAÞ;ReflectiveðRÞg;D2 ¼ fSensitiveðSÞ; IntuitiveðIÞg;D3 ¼ fVisualðViÞ;VerbalðVeÞg;D4 ¼ fSequentialðSeqÞ;GlobalðGÞg:

Therefore, there are 16 possible LSCs (LSCs = {(A,S,Vi,Seq),(A,S,Vi,G), (R,S,Vi,Seq), (R,S,Vi,G), (A,S,Ve,Seq), (A,S,Ve,G), (R,S,Ve,Seq), (R,S,Ve,G), (A,I,Vi,Seq), (A,I,Vi,G), (R,I,Vi,Seq), (R,I,Vi,G),(A,I,Ve,Seq), (A,I,Ve,G), (R,I,Ve,Seq), (R,I,Ve,G)}). Specifically, wepropose that in each learning session, students must interact withlearning objects (LO) (IEEE, 2010) that satisfy a specific LSC, relat-ing LS characteristics to LO characteristics. The LSC to be consid-ered during a learning session is selected according to students’LSp. The probability of a specific LSC be selected is given by (1).Therefore, in our approach, a LSC is a specific combination of fourrandom variables (Papoulis, Pillai, & Unnikrishna, 2002). Then, inour approach, LSp describes the probability of four random vari-ables: a; b; c; d; considering Definition 3.1.

Pða; b; c;dÞ ¼ Pra � Prb � Prc � Prd ð1Þ

Thus, the probability of selecting the LSC (A,S,Vi,Seq) is given byP(A,S,Vi,Seq) = 0.28 � 0.09 � 0.45 � 0.82. The LSC defines the peda-gogical strategy to be adopted for the presentation of course con-tent during a learning session. In this context, the componentsrelated to the use of RL for LS modeling in AIES, in our approach, are:

� States (S): Possible settings of SM;� Actions (A): Pedagogical actions that the system can execute

with the intention of teaching content, maximizing the quality

Input Understanding

Visual Verbal Sequential Global

0.45 0.55 0.82 0.18

2094 F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101

of students performances in the learning process, i.e., the possi-ble LSCs given by Definition 3.1;� Perception of the environment (I:S ? S): Defines how the system

perceives the state of the student, for example, assessing thestudent’s knowledge through tests;� Reinforcement (R : S� A! R): This function defines the rein-

forcement, given by (2), to be applied to LSp, enabling the sys-tem to learn the best LSC for a student;� Action-value function (Q : S� A! R): This function estimates

the usefulness of adopting a certain action (given by a LSC) con-sidering the current state, providing a measurement parameterfor the system actions. This function is given by (1), and assignsa real number to an action, called gain, and estimates how gooda determined LSC is in a given state.

The reinforcement is calculated according to the performancevalue (PFM) obtained by the student during a learning session,according to (2). The reinforcement is inversely related to perfor-mance, since, probably, the lower the performance, the greaterthe difficulty of learning, which can probably be caused by stronginconsistency in LSp, which should be eliminated as soon as possi-ble, requiring greater reinforcement.

By the other side, it is desirable that the greater the distance be-tween LS (DLS) in a dimension, the lower the reinforcement, so thatwe can avoid abrupt increases on a considerably strong LSp, andallowing a greater increase when DLS value approximates to 0(undefined preference). The PFM value is considered in the interval[0,100] and the DLS value is considered in the interval [0,1]. A var-iable Rmax limits the value of R with the intention of preventing toolarge reinforcements when DLS or PFM tends to 0.

R ¼ 1PFM � DLS

ð2Þ

In order to decide how LSp must be updated, it is taken into accountthe LSC selected during a learning session. We experimented threedifferent strategies for updating LSp throughout the learningprocess:

� Strategy 01: the LSp that appears in the selected LSC are incre-mented when the student obtains good performance (no learn-ing problem detected);� Strategy 02: the LSp that appears in the selected LSC are incre-

mented when the student obtains good performance (no learn-ing problem detected), and are decremented when the studentobtains bad performance (learning problem detected);� Strategy 03: the LSp that appears in the selected LSC are decre-

mented when the student obtains a bad performance (learningproblem detected).

These updates are executed by the following rules, where: A andB represents LS in a FSLSM dimension; i indicates one of the fourFSLSM dimensions; R indicates the reinforcement to be appliedto the LSp; m is the minimum PFM value that characterizes goodperformance:

� Rule for updating LSp considering the Strategy 01:

ðPFM P mÞANDðLSC½i� ¼ AÞ !

LSp½i�A :¼ LSp½i�A þ a� R;

LSp½i�B :¼ LSp½i�B � a� R:

ðPFM P mÞANDðLSC½i� ¼ BÞ !

LSp½i�A :¼ LSp½i�A � a� R;

LSp½i�B :¼ LSp½i�B þ a� R:

� Rule for updating LSp considering the Strategy 02:

ðPFM P mÞANDðLSC½i� ¼ AÞ !LSp½i�A :¼ LSp½i�A þ a� R;

LSp½i�B :¼ LSp½i�B � a� R:

ðPFM P mÞANDðLSC½i� ¼ BÞ !LSp½i�A :¼ LSp½i�A � a� R;

LSp½i�B :¼ LSp½i�B þ a� R:

ðPFM < mÞANDðLSC½i� ¼ AÞ !LSp½i�A :¼ LSp½i�A � a� R;

LSp½i�B :¼ LSp½i�B þ a� R:

ðPFM < mÞANDðLSC½i� ¼ BÞ !LSp½i�A :¼ LSp½i�A þ a� R;

LSp½i�B :¼ LSp½i�B � a� R:

� Rule for updating LSp considering the Strategy 03:

ðPFM < mÞANDðLSC½i� ¼ AÞ !LSp½i�A :¼ LSp½i�A � a� R;

LSp½i�B :¼ LSp½i�B þ a� R:ðPFM < mÞANDðLSC½i� ¼ BÞ !LSp½i�A :¼ LSp½i�A þ a� R;

LSp½i�B :¼ LSp½i�B � a� R:

such that:

� LSp[i]A is the probability of preference for the LS A stored in theSM, in dimension i, with i = 1..4.� LSp[i]B is the probability of preference for the LS B stored in the

SM, in dimension i, with i = 1..4.� LSC[i] is the LS that appears in the LSC, considering the dimen-

sion i, with i = 1..4.� 0 < a < 1 is the learning rate, which indicates how fast the sys-

tem learns.

The Algorithm 3.1 implements a RL based process using Q-learning (Sutton & Barto, 1998), and takes into account the compo-nents of RL in our approach (shown above). The Algorithm 3.1sruns until the student reaches all learning objectives.

Algorithm 3.1. Q-learning applied to the automatic and dynamicmodeling of LS

Initialize LSp;while s is not the final state do

Select a concept C to be learned by the student;Select an action a, given by a LSC, to teach C to the student;Execute a adequately, presenting learning objects that teachthe concept C to the student;Evaluate the student performance on the concept C;Update the student’s cognitive level in SM over the concept C;Receive the reward r given by PFM;Calculate the reinforcement R, according to (2);Update LSp;Make s the next state, given by the new SM;

end while

At each iteration, the Q values (given by the function P) con-verge to their optimal values automatically, enabling the systemto select the best teaching strategy for each student.

Table 2Probabilistic LS.

EAp

Processing Perception Input Understanding

Active Reflective Sensitive Intuitive Visual Verbal Sequential Global

0.70 0.30 0.70 0.30 0.70 0.30 0.70 0.30

Fig. 1. Updating LSp using strategy 01.

F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101 2095

It is well-known that a variety of aspects may involvestudents performances evaluation in AIES. Thus, in order to testour approach isolating all this complexity, we developed aprobabilistic model to simulate the learning process and studentperformance, taking into consideration some aspects related tothe impact of LS in the learning process (Bajraktarevic et al.,2003; Graf & Liu, 2008; Graf et al., 2008; Haider et al., 2010;Kinshuk et al., 2009).

The main aspect of this simulated process is that when astudent’s real LS preference appears in the current LSC, learningbecomes easier and the probability of success is increased. Aspointed out by Graf et al. (2008), strong preferences produce stron-ger negative effects on students’ performances when they are notsatisfied by the teaching process, and this fact is considered byour students’ performance simulation process (SPSP). The impactof LS strengths on students’ performances is analyzed by Kinshuket al. (2009). Results show that learners with strong preferencesfor a specific learning style have more difficulties in learning thanlearners with mild LS preferences. According to (Kinshuk et al.,2009), this finding shows that learners with strong LS preferencescan especially benefit from adaptivity.

Basically, the SPSP considers an increase of difficulty when astudent’s real LS (LSr) does not appear in the LSC considered for

providing adaptivity during a learning session. In this way, theSPSP can infer the degree of difficulty to be faced by the studentduring a learning session. Increasing the probability of failure alsoincreases the level of difficulty. Therefore, considering the learningprocess as a non-deterministic process, which is influenced bymany factors besides LS, the SPSP considers that the occurrenceof inadequately adapted content may contribute to students’ fail-ure, but, cannot determine it.

Simulation is a widespread and widely used technique for test-ing educational approaches and may bring advantages, as stated byAbdullah and Cooley (2002), Vanlehn, Ohlsson, and Nason (1994),Vizcaino and du Boulay (2002), Virvou, Manos, and Katsionis(2003), Bravo and Ortigosa (2006), Mertz (1997), Meyn, Tweedie,and Glynn (1996). The next section presents some experimentsand their results.

4. Verification and validation of the proposed approach

For the experiments presented in this section, we consideredthat good performances (no learning problem detected) occurswhen PFM P 60, and bad performances (learning problem de-tected) occurs when PFM < 60, i.e., m = 60. The learning rate, a,

Table 3Results obtained using strategy 01.

Fig. 2. Student’s performances using strategy 01.

2096 F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101

which sets how fast the algorithm learns, was set to 0.8, and thelimit value for the reinforcement, Rmax, was set to 0.2.

We considered a set of 30 concepts to be learned by the student.We considered the students initial cognitive level, in all concepts,equivalent to the first level of the Bloom’s Taxonomy (Starr, Man-aris, & Stalvey, 2008). Moreover, it was considered as learningobjectives the highest level of knowledge in all concepts. The exe-cution of an experiment is completed when the student reaches alllearning objectives specified in the SM. The student’s cognitivestate in a given concept evolves through six levels, arranged in ahierarchy, from least complex to most complex.

Therefore, the simulated learning process in these experimentsmust have at least 180 learning sessions (or iterations) in order toachieve all learning goals (30 concepts � 6 cognitive levels = 180).The student’s cognitive level advances if PFM P m, and does notadvance if PFM < m. Therefore, the easier the learning process,fewer iterations are needed in order to reach all learning goals.

In order to validate the proposed approach, two variables wereobserved in the experiments:

� consistency: LSp effectively converge to LSr (student’s real learn-ing styles) during the learning process?� efficiency: LSp converges to LSr in a reasonable time, i.e., LSp

becomes consistent at the beginning of the learning process?

It is important to notice that without using simulation, it wouldbe difficult to validate this approach. This is because if testing theproposed approach on real learning process, with real students, itwould be impossible to know students LSr exactly, due to the rea-sons pointed out above, regarding uncertainty and inconsistency ofdata obtained with traditional LS modeling approaches. Thus, itwould be impossible to measure the consistency of the resultingLSp, due to the uncertainty about the LSr.

For each experiment, we set up the student’s LSr and thestrength of each preference (strong, moderate, weak or balanced).For each experiment, we present graphically the process of updat-ing LSp during the learning process. In each graph, the axis x showsthe number of iterations of the learning process, and the axis yshows the values of LSp (multiplied by 100) throughout the learn-

Fig. 3. Average performances and average learning problems using strategy 01.

Fig. 4. Updating LSp using strategy 02.

F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101 2097

ing process in each dimension of the FSLSM. The main objectivewas to observe how LSp is gradually updated throughout the learn-ing process. Thus, it was possible to clearly visualize the process ofautomatic detection of the students LS.

Moreover, for each experiment we present a graph showing thePFM obtained at steps of 5 iterations. It is also presented theaverage performance and average learning problems occurred atintervals of 20 iterations. Therefore, it is possible to see how theaverage performance increases and the amount of learning

problems decreases, as LSp is fixed and becomes consistent withLSr. Furthermore, the results about 10 repetitions of each experi-ment are presented, making it possible to compare the effective-ness of the experimented strategies.

It was verified that occurrences of learning problems wasreduced, as a result of the improvement of students performances,as LSp became consistent with LSr. The following results wereobtained at an experiment considering the initial LSp presentedin Table 2.

2098 F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101

The student’s LSr considered in the following experiments is:

LSr ¼ fReflectiveðstrongÞ; IntuitiveðstrongÞ;VerbalðmoderateÞ;GlobalðbalancedÞg

Considering this student, the initial LSp shown in Table 2 is inconsis-tent in its four dimensions. Fig. 1 graphically presents the updatingof LSp during the first execution of this experiment, using the strat-egy 01.

Table 3 shows the number of iterations, the amount of learningproblems occurred, and resulting LSp values at the end of the learn-ing process, in 10 executions of this experiment. The LSp valueshighlighted in red were not corrected during the learning process,remaining inconsistent at the end of the process. As it can be seen,the inconsistency level of resulting LSp is very high. It shows, then,that this strategy is ineffective for LS modeling.

Fig. 2 displays student’s performances at intervals of five learn-ing sessions in the first execution of this experiment. Because LSp

remains inconsistent throughout the student’s learning process,bad performances constantly occur.

Fig. 3 shows the average performance achieved by the studentand the average learning problems in steps of 20 learning sessions.

Table 4Results obtained using strategy 02.

Fig. 5. Student’s performan

For the same reasons, the average learning problems remains high,and performance remains low throughout the learning process.

Fig. 4 graphically presents the updating of LSp during the firstexecution of this experiment, using strategy 02.

Table 4 shows the number of iterations, the amount of learningproblems occurred, and resulting LSp values at the end of thelearning process in 10 executions of this experiment. We can no-tice a better level of consistency in these results, when comparedto results obtained through strategy 01. Resulting LSp values ob-tained at execution N.3 and N.5 were fully consistent. In the otherexecutions, one inconsistency remained in resulting LSp. There wasa significant reduction in the number of learning problems, conse-quently reducing the number of iterations of the learning process.Therefore, there was a significant efficiency gain in relation tostrategy 01.

Fig. 5 displays student’s performance values at intervals of fivelearning sessions, obtained at the first execution of this experi-ment. The student’s performances were improved, and learningproblems were reduced, as inconsistencies were eliminated fromLSp.

Fig. 6 shows the average performance achieved by the student,and the average learning problems in steps of 20 learning sessions.

ces using strategy 02.

Fig. 6. Average performances and average learning problems using strategy 02.

Fig. 7. Updating LSp using strategy 03.

F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101 2099

The improvement of the learning process, in relation to the resultsobtained with strategy 01, is clear.

Fig. 7 graphically presents the updating of LSp during the firstexecution of this experiment, using strategy 03.

Table 5 shows the number of iterations, the amount of learningproblems occurred, and resulting LSp values at the end of the learn-ing process in 10 executions of this experiment. We can notice abetter level of consistency in these results, when comparing themwith results obtained through strategy 02 (Table 4). It shows areduction in the number of learning problems, consequently

reducing the number of iterations in the learning process. There-fore, we can conclude that using this strategy results in an effi-ciency gain when comparing it with strategies 01 and 02.

Fig. 8 displays student’s performance values at intervals of fivelearning sessions, obtained at the first execution of this experi-ment. The student’s performance were improved, and learningproblems were reduced, as inconsistencies were eliminated fromLSp.

Fig. 9 shows the average performance achieved by the studentand the average learning problems in steps of 20 learning sessions.

Table 5Results obtained using strategy 03.

No. Iterations Learningproblems

Resulting LSp

1 208 28 {(0.2;0.8); (0.1;0.9); (0.1;0.9);(0.3;0.7)}

2 204 24 {(0.2;0.8); (0.1;0.9); (0.2;0.8);(0.3;0.7)}

3 213 33 {(0.2;0.8); (0.2;0.8); (0.2;0.8);(0.1;0.9)}

4 206 26 {(0.1;0.9); (0.2;0.8); (0.2;0.8);(0.2;0.8)}

5 201 21 {(0.2;0.8); (0.2;0.8); (0.2;0.8);(0.2;0.8)}

6 204 24 {(0.2;0.8); (0.2;0.8); (0.2;0.8);(0.1;0.9)}

7 210 30 {(0.1;0.9); (0.2;0.8); (0.2;0.8);(0.3;0.7)}

8 196 16 {(0.1;0.9); (0.2;0.8); (0.2;0.8);(0.2;0.8)}

9 200 20 {(0.2;0.8); (0.1;0.9); (0.1;0.9);(0.2;0.8)}

10 198 18 {(0.2;0.8); (0.3;0.7); (0.1;0.9);(0.3;0.7)}

Average 204 24 –

Fig. 8. Student’s performances using strategy 03.

2100 F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101

Fig. 9. Average performances and average

The improvement of the learning process, in relation to the resultsobtained with strategy 02, is clear.

It can be noticed that, in this experiment, the adaptivity wasmore adequate to the needs of the student, resulting in lower dif-ficulty in the learning process, generating fewer occurrences oflearning problems, and reducing the number of iterations of theprocess, making evident the better efficiency of the strategy 03,when compared to strategies 01 and 02. As it can be seen, the strat-egy 03 demanded less iterations of the learning process, due toreduction of learning problems. By the way, this reduction is prob-abilistic, not deterministic. Therefore, it does not always occurs (asit can be noticed when comparing Tables 4 and 5), but it tends tooccur when inconsistencies remains in the SM.

5. Conclusion

This work presents three different strategies for automaticallydetect and precisely adjust students’ LS, considering a reinforce-ment learning based approach which favors the non-deterministicand non-stationary aspects of LS (Graf & Kinshuk, 2009). Resultshave shown the efficiency and effectiveness of the proposed ap-

learning problems using strategy 03.

F.A. Dorça et al. / Expert Systems with Applications 40 (2013) 2092–2101 2101

proach, mainly by using the strategy 03 for updating the SM, allow-ing us to consider it for the implementation of promising AIES andfor the insertion of adaptivity in existing educational systems.

Because of the probabilistic and dynamic factors enclosed onstudents’ LS modeling, our approach gradually and constantlymodifies the SM using rules that detect which LS should be ad-justed at a specific point of the learning process, considering thestudent’s performance. In this way, SM converges to the student’sreal LS, as showed in Section 4.

Finally, the proposed approach solves some important problemsignored by most of the analyzed approaches, and brings advanta-ges, due to specific points, as showed in Section 2. The experimentswith this approach were done through computer simulation, whichtook into account how LS preferences exert influence on students’performances, as described by some researchers, e.g. (Alfonseca,Carro, Martín, Ortigosa, & Paredes, 2006; Graf et al., 2008; Graf,Lan, & Liu, 2009b; Haider et al., 2010; Kinshuk et al., 2009).

The evaluation of AIES is a difficult task, as pointed out in Bravoand Ortigosa (2006). Therefore, testing our approach through sim-ulation was vital, due to the time and human resources needed totest it with real students. Now that we have achieved good resultsthrough simulation, we feel confident to implement our approachin an existing LMS, like Moodle (Moodle, 2010), and test it withreal courses and real students, as a near future work. In order toachieve this goal, we are working on the development of a functionable to efficiently map LO characteristics to students’ LS.

References

Abdullah, S., & Cooley, R. (2002). Using simulated students to evaluate an adaptivetesting system. In Proceedings of international conference on computers ineducation, 2002 (pp. 614–618). IEEE.

Alfonseca, E., Carro, R., Martín, E., Ortigosa, A., & Paredes, P. (2006). The impact oflearning styles on student grouping for collaborative learning: A case study.User Modeling and User-Adapted Interaction, 16(3), 377–401.

Bajraktarevic, N., Hall, W., & Fullick, P. (2003). Incorporating learning styles inhypermedia environment: Empirical evaluation. In Proceedings of the workshopon adaptive hypermedia and adaptive web-based systems (pp. 41–52).

Bravo, J. & Ortigosa, A. (2006). Validating the evaluation of adaptive systems by userprofile simulation. In Proceedings of workshop held at the fourth internationalconference on adaptive hypermedia and adaptive web-based systems (AH2006)(pp. 479–483).

Cabada, R., Estrada, M., & Garcia, C. (2009). A fuzzy-neural network for classifyinglearning styles in a Web 2.0 and mobile learning environment. In Web Congress,2009. LE-WEB’09 (pp. 177–182). Latin American: IEEE.

Carmona, C., & Castillo, G. (2008). Designing a dynamic Bayesian network formodeling students learning styles. In Eighth IEEE International Conference onAdvanced Learning Technologies (pp. 346–350). IEEE.

Carmona, C., Castillo, G., & Millán, E. (2007). Discovering student preferences in e-learning. In Proceedings of the international workshop on applying data mining ine-learning.

Castillo, G., Gama, J., & Breda, A. (2005). An adaptive predictive model for studentmodeling. Advances in Web-based Education: Personalized Learning Environments,70–92.

Cha, H., Kim, Y., Park, S., Yoon, T., Jung, Y., & Lee, J. (2006). Learning styles diagnosisbased on user interface behaviors for the customization of learning interfaces inan intelligent tutoring system. In Intelligent Tutoring Systems (pp. 513–524).Springer.

Danine, A., Lefebvre, B., & Mayers, A. (2006). Tides-using Bayesian networks forstudent modeling. In Proceedings of the sixth international conference onadvanced learning technologies (ICALT’06). IEEE Computer Society.

Felder, R., & Silverman, L. (1988). Learning and teaching styles in engineeringeducation. Journal of Engineering Education, 78(7), 674–681.

Felder, R., & Spurlin, J. (2005). Applications, reliability and validity of the index oflearning styles. International Journal of Engineering Education, 21(1), 103–112.

Franzoni, A. L., & Assar, S. (2009). Student learning styles adaptation method basedon teaching strategies and electronic media. Educational Technology and Society,12(4), 15–29.

García, P., Amandi, A., Schiaffino, S., & Campo, M. (2007). Evaluating Bayesiannetworks’ precision for detecting students’ learning styles. Computers andEducation, 49(3), 794–808.

Giraffa, L. (1999). Uma arquitetura de tutor utilizando estados mentais. Ph.D. thesis,Universidade Federal do Rio Grande do Sul. Instituto de Informática. Programade Pós-Graduação em Computação.

Graf, S., & Kinshuk (2009). Advanced adaptivity in learning management systems byconsidering learning styles. Proceedings of the 2009 IEEE/WIC/ACM internationaljoint conference on web intelligence and intelligent agent technology (Vol. 03,pp. 235–238). IEEE Computer Society.

Graf, S., Kinshuk & Liu, T. (2009a). Supporting teachers in identifying students’learning styles in learning management systems: An automatic studentmodelling approach. Journal of Educational Technology and Society, 12(4), 3–14.

Graf, S., & Kinshuk, K. (2010). Using cognitive traits for improving the detection oflearning styles. In 2010 Workshop on database and expert systems applications(DEXA) (pp. 74–78). IEEE.

Graf, S., Lan, C., Liu, T., et al. (2009b). Investigations about the effects andeffectiveness of adaptivity for students with different learning styles. In 2009Ninth IEEE international conference on advanced learning technologies(pp. 415–419). IEEE.

Graf, S. & Lin, T. (2007). Analysing the relationship between learning styles andcognitive traits. In Seventh IEEE international conference on advanced learningtechnologies, 2007, ICALT 2007 (p. 235–239). IEEE.

Graf, S. & Liu, T. (2008). Identifying learning styles in learning management systemsby using indications from students’ behaviour. In Eighth IEEE internationalconference on advanced learning technologies, 2008, ICALT’08 (pp. 482–486). IEEE.

Graf, S., Liu, T.-C., & Kinshuk (2008). Interactions between students learning styles,achievement and behaviour in mismatched courses. In Proceedings of theinternational conference on cognition and exploratory learning in digital age(CELDA 2008) (pp. 223–230). IADIS International Conference.

Graf, S., & Viola, S. (2009). Automatic student modelling for detecting learning stylepreferences in learning management systems. Citeseer.

Haider, M., Sinha, A., & Chaudhary, B. (2010). An Investigation of relationshipbetween learning styles and performance of learners. International Journal ofEngineering Science and Technology, 2(7), 2813–2819.

IEEE. (2010). LOM (Learning Object Metadata). IEEE Learning Technology StandardsCommittee, http://ltsc.ieee.org/wg12/index.html.

Kelly, D., & Tangney, B. (2005). ’First Aid for You’: getting to know your learningstyle using machine learning. In Fifth IEEE international conference on advancedlearning technologies, 2005, ICALT 2005 (pp. 1–3). IEEE.

Kinshuk Liu, T., & Graf, S. (2009). Coping with mismatched courses: Students’behaviour and performance in courses mismatched to their learning styles.Educational Technology Research and Development, 57(6), 739–752.

Limongelli, C., Sciarrone, F., Temperini, M., & Vaste, G. (2009). Adaptive learningwith the LS-plan system: A field evaluation. IEEE Transactions on LearningTechnologies, 203–215.

Marzano, R., & Kendall, J. (2007). The new taxonomy of educational objectives. CorwinPr..

Mertz, J. (1997). Using a simulated student for instructional design. InternationalJournal of Artificial Intelligence in Education (IJAIED), 8, 116–141.

Messick, S. (1976). Personal styles and educational options. Individuality in Learning,327–368.

Meyn, S., Tweedie, R., & Glynn, P. (1996). Markov chains and stochastic stability.London: Springer.

Moodle. (2010). http://www.moodle.org/.Papoulis, A., Pillai, S., & Unnikrishna, S. (2002). Probability, random variables, and

stochastic processes (Vol. 73660116). New York: McGraw-Hill.Price, L. (2004). Individual differences in learning: Cognitive control, cognitive style,

and learning style. Educational Psychology, 24(5), 681–698.Roberts, M., & Erdos, G. (1993). Strategy selection and metacognition. Educational

Psychology, 13(3), 259–266.Starr, C., Manaris, B., & Stalvey, R. (2008). Bloom’s taxonomy revisited: Specifying

assessable learning objectives in computer science. ACM SIGCSE Bulletin, 40(1),261–265.

Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction (Vol. 116).Cambridge Univ. Press.

Vanlehn, K., Ohlsson, S., & Nason, R. (1994). Applications of simulated students: Anexploration. Journal of Artificial Intelligence in Education, 5, 135.

Virvou, M., Manos, K., & Katsionis, G. (2003). An evaluation agent that simulatesstudents’ behaviour in intelligent tutoring systems. In IEEE internationalconference on systems, man and cybernetics, 2003 (Vol. 5, pp. 4872–4877).IEEE, IEEE.

Vizcaino, A., & du Boulay, B. (2002). Using a simulated student to repair difficultiesin collaborative learning. In ICCE ’02 Proceedings of the international conferenceon computers in education. Washington, DC, USA: ACM, IEEE Computer Society.

Zatarain, R., Barrón-Estrada, L., Reyes-García, C., & Reyes-Galaviz, O. (2010).Applying intelligent systems for modeling students’ learning styles used formobile and web-based systems. Soft Computing for Intelligent Control and MobileRobotics, 3–22.

Zatarain-Cabada, R., Barrón-Estrada, M., Zepeda-Sánchez, L., Sandoval, G., Osorio-Velazquez, J., & Urias-Barrientos, J. (2009). A Kohonen network for modelingstudents’ learning styles in Web 2.0 collaborative learning systems. MICAI 2009:Advances in Artificial Intelligence (pp. 512–520).