Post on 22-Feb-2016
description
transcript
Plan Recognition in Virtual Laboratories
Ofra Amir and Ya’akov (Kobi) Gal
Ben-Gurion University of The NegevDepartment of Information Systems Engineering
2
Background
• Educational software in the sciences is including open-ended “construction” environments
• Students build and analyze models using these software
3
Virtual Labs– An interactive simulation of a chemistry lab.
4
Virtual Labs – Dilution Problem
“ Your supervisor has just asked you to prepare
500mL of 3M HNO3 for tomorrow's experiment.
In the stockroom explorer, you will find a cabinet
called Stock Solutions. Open this cabinet to find a
2.5L bottle labeled 15.4M HNO3. ”
5
The Problem
Please prepare a flask containing 500 ml of a 3M
solution…
I’m confused Boring… This is
easy!
During class:
6
The Problem
After class:
I wonder how each student performed…
7
The Goal
• Develop automatic support for teachers in their analysis of student performance
– Approach: use plan recognition to infer students’ problem solving plans
8
Related Work
• Bayesian networks to model students’ interactions with intelligent tutors [Conati et al., 2002]
• Complete algorithms for plan recognition (CSP) [Reddy et al, 2009; Quilici et al, 1998]
• Heuristic algorithm for recognizing students’ activities in pedagogical software for statistics [Gal et al, 2008]
9
Solution Building blocks
• User actions
• Recipes
• Plans
10
User Actions – Log Files
11
Recipes [Pollack, 1990]
• Basic actions are rudimentary (log actions)
• Complex actions are abstract
• A recipe for a complex action describes– A series of sub-actions for completing the action
– Constraints on these actions
12
• Representing actions:
• Recipe structure:
The Recipe Language
Action Name An action parameter
with value constraint
D[dt:H20, sid_1,did_1][ 0 ]
Parameter without value
constraint
Action Pre-conditions
D[sid_1,did_2] -> D[sid_1,did_1][ ] C[sid_2,did_2][ ]
The complex action
sub-actions did_1=sid_2
Constraints on sub-actions parameters
13
Plans in Virtual Labs
• A plan for a complex action is– A hierarchy of recipes towards completing the
action
• The plan represents students’ activity with the software
• Plan recognition– Infer students’ activities based on their actions
with the software, given a set of recipes
14
Plan Recognition Approach
Plan Recognition Algorithm
Student Plan
15
Dilution Problem Recipes
• To solve the problem, the student should:– Pour H20 to the destination flask– Pour HNO3 to the destination flask
• Students can solve this problem in many different ways
16
Dilution Problem - Example Solution 1
2 1
17
Dilution Problem - Example Solution 2
18
Challenges in The Virtual Labs Domain
• Indefinite repetition of activities
• Interleaving activities
• Trial-and-error, mistakes
• Conclusion:– Complete approach intractable for this domain
19
Build Plan Greedy Algorithm• Input:
– A set of Virtual Labs basic actions, A set of recipes• Algorithm steps:
– initialize open list with the actions from the log– for each recipe in order of increasing depth
• find match (Recipe, open list)• while match exists
– add complex action to the open list – create branches from the complex action to its sub-actions– remove sub-actions from open list– Call find match with the updated open list
20
( SM , did=ID5 , sid=ID1 )
( SM , did=ID5 , sid=ID1 )
( MO , id=ID5 )
( MO , id=ID5 )
( SM , did=ID6 , sid=ID5 )
( MO , id=ID1 )
( AS , id=ID6 )
( MO , id=ID6 )
( MO , id=ID5 )
( FC , did=ID6 ,sid=ID5 )
( SM , did=ID6 , sid=ID5 )
( C , did=ID5 , sid=ID1 )
( MO , id=ID5 )
( MO , id=ID5 )
( AS , id=ID6 )
( MO , id=ID6 )
( MO , id=ID5 )
( FC , did=ID6 ,sid=ID5 )
( C , did=ID6 , sid=ID5 )
( C , did=ID5 , sid=ID1 )
( MO , id=ID5 )
( MO , id=ID5 )
( AS , id=ID6 )
( MO , id=ID6 )
( MO , id=ID5 )
( FC , did=ID6 ,sid=ID5 )
( C , did=ID6 , sid=ID5 )
( D , did=ID6 , sid=ID1 )
P1 P2P3
Recipe: C -> SM SM Recipe: D -> C C
21
Find Match• Find match searches for actions in the open list which fulfill the recipe for the complex action
• Actions in the match can be free ordered as long as they satisfy the constraints in the recipe
• Was implemented as depth first search, but can be implemented in other ways
• Find Match is complete, given a recipe and an open list of actions
22
Dilution Problem – Partial Plan
MSC – Mixing solution componentMSI – Mixing solution through intermediate flaskSDP – Solve Dilution Problem
23
Empirical Evaluation• The algorithm was run on 20 log files taken from real
student interactions– 6 different problems– logs ranged in size from 20 actions to 187 actions– Plans ranged in depth from 3 to 14 levels
• The plans were validated by a domain expert
24
Contributions
• A new computationally efficient plan recognition algorithm that can cope with interleaving activities, mistakes, indefinite repetition.
• The algorithm can be integrated with real pedagogical software
• Shown to succeed on real-world data
25
Questions