SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning...

SimStudent:A Computational Model of Learning as a Research Toolbox for the Sciences of Learning

Noboru Matsuda

Human-Computer Interaction Institute

Carnegie Mellon University

Sim

Stu

den

t Pro

ject

PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)

Research Questions

• Building a cognitive model is hard. Can machine-learning techniques help non-expert authors build a Cognitive Tutor?

• Would like to simulate students’ learning. Can machine-learning techniques help us build a computational model of learning with a cognitive fidelity?

• I heard that students learn by teaching others. Can we use the computational model of learning to study the theory of learning by teaching?

2

Sim

Stu

den

t Pro

ject

PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU) 3

Solution: SimStudent

• Machine learning agent

– Learns procedural skills, by

– Observing model solutions & solving problems

• Fundamental technology

– Programming by Demonstration

– Inductive Logic Programming

• Knowledge representation

– Production rules (Jess)

Lau & Weld (1998). Blessing (1997).

Sim

Stu

den

t Pro

ject


SimStudent Projects

• Intelligent Authoring– Building a Cognitive Tutor as a CTAT Plug-in

• Student Modeling and Simulation– Controlled educational studies

– Error formation study

– Prerequisite conceptual knowledge study

• Teachable Peer Learner– Learning by teaching

4

Sim

Stu

den

t Pro

ject


Authoring a Cognitive Tutor

• Example-Tracing Tutor

– Little programming

– A cognitive model specific to a particular problem

• Limited generalization by editing a behavior graph

• Model-Tracing Tutor

– Powerful student model– Cognitive task analysis is hard– Writing production rules is even more challenging

• Performing a task is relatively easy…

Sim

Stu

den

t Pro

ject


Next Generation AuthoringBuild a tutor GUI

Teaching a solution

SimSt. learning

Rule simplify-LHS:

IF is-equation( Eq ),

is-lhs( Eq, Lhs ),

polynomial( Lhs ),

all-var-terms( Lhs )

Then simplify( Lhs, S-lhs ),

enter( S-lhs )

Production Rules

Rule simplify-LHS:


is-lhs( Eq, Lhs ),

polynomial( Lhs ),



enter( S-lhs )

Rule simplify-LHS:


is-lhs( Eq, Lhs ),

polynomial( Lhs ),



enter( S-lhs )

Sim

Stu

den

t Pro

ject


Demo

7

Authoring by Tutoring

Example: Learning to subtract a constant term


Learning to subtract a constant numberFirst example

subtract 1

Subtract the last term on the left-hand side…Subtract the last term on the left-hand side…

Subtract the coefficient of X…

Subtract the coefficient of X…

Subtract the difference between 4 and 3…

Subtract the difference between 4 and 3…

I see 3x, 1, x, and 4 in the equation.I wonder where the ‘1’ came from…


Example: Learning to subtract a constant termLearning to subtract a constant number

First example

subtract 1

Subtract the last term on the left-hand side…

Subtract the last term on the left-hand side…

Subtract the coefficient of X…Subtract the coefficient of X…

Subtract the difference between 4 and 3…Subtract the difference between 4 and 3…




Prior Knowledge• Feature predicates

– 18 predicates– isFractionTerm(X), isConstant(X), isPolynomial(X),…

• Operators– 42 operators– add(X,Y), coefficient(X), getFrstNumber(X),

…

10PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)

Sim

Stu

den

t Pro

ject


Example: Stoichiometry Tutor

Sim

Stu

den

t Pro

ject


SimStudent Projects






12

Sim

Stu

den

t Pro

ject


Model of Incorrect Learning

• Identify errors students commonly make

• Weaken SimStudent’s background knowledge

• Let SimStudent make an induction error

13

Sim

Stu

den

t Pro

ject


Weak Prior Knowledge Hypothesis

• Multiple ways to make sense of examples

14

3x=5 “divide by 3”3x=5 “divide by 3”

Get a coefficient and divideGet a coefficient and divide

Get a number and divideGet a number and divide

Get a denominator and multiplyGet a denominator and multiply

4/x=5

“divide by 4”

“multiply by x ”

Sim

Stu

den

t Pro

ject


Results: Learning Rate

Steps Score = 0 (if there is no rule applicable)# correct rule applications / # all rule applications

# training problems

Step Score

Student Model A set of knowledge components (KCs) Encoded in intelligent tutors to model how

students solve problems E.g. How to proceed given problems of the form Nv=N

One of the key factors that affects automated tutoring systems in making instructional decisions

Previous Approach: Require expert input Highly subjective

Proposed Approach: Use a machine-learning agent, SimStudent, to acquire

knowledge 1 Skill 1 KC Skill application for that step KC for each step

(Li et al. EDM2011) A Machine Learning Approach for Automatic Student Model Discovery

Human-generated vs SimStudent KCsHuman-generated Model

SimStudent

Comment

Total # of KCs 12 21

# of Basic Arithmetic Operation KCs

4 13 Split into finer grain sizes based on different problem forms

# of Typein KCs 4 4 Approximately the same

# of Other Transformation Operation KCs (e.g. combine like terms)

4 4 Approximately the same

4x = 20 vs.

–x = 5

Results

Human-generated Model

SimStudent

AIC 6529 6448

3-Fold Cross Validation RMSE

0.4034 0.3997

Significance Test SimStudent outperforms the human-

generated model in 4260 out of 6494 steps p < 0.001

SimStudent outperforms the human-generated model across 20 runs of cross validation

p < 0.001

Sim

Stu

den

t Pro

ject


SimStudent Projects






19

Sim

Stu

den

t Pro

ject


Learning by Teaching SimStudent

Sim

Stu

den

t Pro

ject


Learn more about SimStudents

• Project Web– www.SimStudent.org

• Contact us– Noboru Matsuda

– [email protected]

21

Date post:	03-Jan-2016
Category:	Documents
Upload:	sophie-kelley
View:	213 times
Download:	1 times

SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning...

Documents