Design a personalized e-learning system based on item response theory and artificial neural network approach

Ahmad Baylari, Gh.A. Montazer

IT Engineering Department, School of Engineering, Tarbiat Modares University, Tehran, Iran

Expert Systems with Applications, Volume 36, Issue 4, May 2009, Pages 8013-8021

Reporter： Yu Chih Lin

Outline Introduction

Item response theory

Test construction process

System architecture

System evaluation

Conclusion

Introduction In web-based educational systems the structure of learning domain and

content are presented in the static way

Without taking into

account the learners’ goals

experiences

existing knowledge

ability

interactivity

Introduction Personalization and interactivity will increase the quality of learning

This paper proposes a personalized multi-agent e-learning system

Item response theory (IRT)

• Presents adaptive tests

Artificial neural network (ANN)

• Personalized recommendations

Item response theory Item response theory (IRT) was first introduced to provide a formal

approach to adaptive testing

Three common models for ICC

One parameter logistic mode(1PL)

Two parameter logistic model(2PL)

Three parameter logistic model (3PL)

Item response theory 1PL models

i : only one parameter

bi ： item difficulty

D ： 1.7

： ability scale

Item response theory 2PL model

• : discrimination degree, added into the item characteristic function

• bi : item difficulty

• D : 1.7

• : ability scale

Item response theory 3PL model

: guess degree to the 2PL model

Potential guess behavior of examinees

Item response theory An estimation method called maximum likelihood estimator (MLE)

• Effectively estimate item parameters and examinee’s abilities

: learner can answer the i th item correctly

Q : learner cannot answer the ith item correctly(1 -

ui : 1 (correct answer) or 0 (incorrect answer)

Item response theory Item information function (IIF)  is the subject of the amount of information

Effectively distinguish between subjects potential ability to reduce the estimation error

Test information function (TIF)

• Sum of the amount of information the test results for each subject

Item response theory IIF&TIF：

(IIF) (TIF)

P’(θ) is the first derivative of Pi(θ) and Qi(θ) = 1 - Pi(θ)

I (θ) : amount of information for item,1~N

Test construction process Three types of tests

• pre-test

• post-test

• review tests( 延後測 )

All of these tests have 10 items

Use IRT-3PL model to test construction

• appropriate post-test selection for learners

Test construction process For posttest construction

System architecture Proposed a personalized multi-agent e-learning system

Middle layer contains four agents

Activity agent :

records e-learning activities

Planning agent :

agent plans the learning process

Test agent :

based on the requests of planning agent , presents appropriate test

type to the learner

Remediation agent :

analyzes the results of review tests, and diagnoses learner’s learning problems

System architecture System architecture

System design and development Collect stutdents’s responses and tests

• Diagnose their learning problems

• Recommend them appropriate learning materials

Maximum number of recommended LOs were five

Large number of responding states to a test

• states for each test

System design and development Experiment the remediation agent

Essentials of information technology management course

• Divided into several Los

• A few codes were allocated for all Los

System design and development I1 to I10 columns are item codes

R1 to R10 are corresponding responses which code

1 : correct response

0 : incorrect response

System design and development Use a back-propagation network(BPNN)

• Learning Data

Use 20 input nodes

Use 5 output neurons

System design and development Use items responses data as input data the neural network

Output neurons are recommended LOs

System design and development Normalization of data within a uniform range 0–1

Prevent larger numbers from overriding smaller ones

Prevent premature saturation of hidden nodes

No one standard procedure

Input

Output

System design and development Scale input and output variables (xi) in interval

: normalized value of , ,

Normalized the input and output data , range 0.1 ~ 0.9

System design and development ANN requires partitioning of the parent database into three subsets

Training

Test

Validation

Training used 60% of all data

Validation 10% for data

Remaining data for testing the network

System design and development Use one or two hidden layer

Trained with various neurons in each layer in MATLAB software

In hidden layer

• Use sigmoid function as activation function

For output neurons

First use linear activation function

Second use sigmoid activation function

System design and development Two different criteria used to stop training

• Training error(MSE) ：

• Maximum epoch ： 1000 epochs

Training back-propagation network

• With a random set of connection weights (weight initialization)

• Train with different architectures

System design and development For example in training a network

15 neurons in one hidden layer

• With sigmoid activation function

Output layer neurons

• With linear activation function

MSE error 7.13

•

System design and development For example in training a network

15 neurons in one hidden layer and 10 neurons in second hidden layer

• With sigmoid activation function

Output layer neurons

• With linear activation function

MSE error 0.0158

System design and development Summarizes the results of trained networks with different architectures

Network configuration 20-20-5 (network No. 11)

System evaluation Recommended LOs from the network compared with recommended LOs

from a human instructor

Output was exactly the same as the target output, 25 of 30 tests (83.3%)

Conclusion Proposed a personalized multi-agent e-learning system

Estimate learner’s ability using item response theory

Diagnose learner’s learning problems

Recommend appropriate learning materials to the learner

Neural network approach to learning material recommendation