Date post: | 22-Jan-2016 |
Category: |
Documents |
Upload: | nickolas-townsend |
View: | 222 times |
Download: | 0 times |
Planning with LoopsSome New Results
Yuxiao (Toby) Hu and Hector LevesqueUniversity of Toronto
2
Outline
• Introduction
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
3
The Planning Problem• Finitely many functional fluents, f1,...,fm,
whose domain may be finite or infinite;• Finitely many actions, a1,...,an (world-
changing or sensing);• Initial state: the possible values of each
fluent;• Goal: achieve some goal condition in all
contingencies.
Like contingent planning: solve a class of problems
Incomplete initial state with possibly infinite cases
4
Motivating Example(a variant of striped-tower in Srivastava et al. 2008)
Fluents:– stackA (a list of block colors)– stackB (a list of block colors)– stackC (a list of block colors)– hand (empty/red/blue)
World changing actions:– pickA, pickB, putB, putC;
Sensing actions– testA? (empty/nonempty)– testB? (empty/nonempty)– testH? (red/blue)
5
Motivating Example(a variant of striped-tower in Srivastava et al. 2008)
• Initially: – stackA=[blue,red,red,blue]– stackB=[ ]– stackC=[ ]– hand=empty
• Goal:– stackA=[ ]– stackB=[ ]– striped(stackC)– hand=empty
striped(X) is trueiff
X=[red,blue,…,red,blue]
6
Example 1
A linear solution1.pickA;2.putB;3.pickA;4.putC;5.pickB;6.putC;7.pickA;8.putC;9.pickA10.putC.
7
Example 2
8
Example 2 (cont.)pickA;CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; ... ... -blue: putC; ... ... - blue: putB; pickA; CASE testH? OF -red: putC; ... ... -blue: putB; ... ...
9
Example 3
Question: Is there a generalized plan solving all problems in this class?
10
Some of the Existing Approaches
• KPLANNER (Levesque 2005)generates robot programs by winding found conditional plans.
• Aranda (Srivastava et al. 2008)obtains generalized plans by winding an abstracted example plan.
• loopDISTILL (Winner and Veloso 2007)learns a “dsPlanner” by merging matching sub-plans of an example partial-order plan.
11
Outline
• Introduction
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
12
Plan Representation – Robot Programs
A robot program is defined inductively by–nil;–seq(A,P);–case(A,[R1:P1,…,Rn:Pn]);–loop(P,Q).
(Levesque 1996, 2005)
KPLANNER (Levesque 2005) uses the robot program representation.
13
Robot Program – Some ExamplespickA;CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; ... ... -blue: putC; ... ... - blue: putB; pickA; CASE testH? OF -red: putC; ... ... -blue: putB; ... ...
pickA;putB;pickA;putC;pickB;putC;pickA;putC;pickAputC.
14
Plan Representation
An FSA plan is a directed graph• Each node represents a program state
o One unique "start" state o One unique "final" stateo Non-final state associated with an
action• Each edge is associated with a
sensing resulto Sensing result of world-changing
actions can be omitted
15
Plan Execution(for a single complete initial world)
1. Use the "start state" as current program state;
2. If current state is the "final state", then stop;
3. Execute action associated to the current state;
4. Follow the edge with returned sensing result;
5. Make the node pointed to by this edge the current program state, and repeat from Step 2.
16
Robot Program vs. FSA Plan
It can be shown that all robot programs can be represented by equivalent FSA plans.
What about the reverse direction?
17
Robot Program vs. FSA Plan
feel?thirsty
drink
hungry
eat
go2bed
sleep
CASE feel? OF
- thirsty:
drink;
go2bed;
sleep;
- hungry:
eat;
go2bed;
sleep
18
Robot Program vs. FSA Plan
get?instruction suggestion
follow? think?
succeed
fail
workable
unworkableok
revise?fail
19
Outline
• Motivation
• Representing plans with loops– FSA plan: a type of finite state controller
• Constructing plans with loops– Search in the space of FSA plans
• Potential for improvements
• Conclusion
20
Generating Plans with Loops
1. Start with the smallest FSA plan with only one non-final state.
2. If the current program state is final, the goal must be satisfied.
3. Otherwise, execute the action associated to the current program state, non-deterministically pick an applicable one if none is associated.
4. For each possible sensing result of the action, follow the transition and change the current state to the transition target. If no transition is associated to the sensing result, non-deterministically pick one for it.
5. Repeat from step 2.
21
Search in the Space of FSA Plans… …
(Possible values of stackA)
22
Search in the Space of FSA Plans… …
23
Search in the Space of FSA Plans
pickA
… …
24
Search in the Space of FSA Plans
testA?
… …
empty
25
Search in the Space of FSA Plans
testA?empty
nonempty
… …
… …
26
Search in the Space of FSA Plans
testA?empty
nonempty
… …
27
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickB
28
Search in the Space of FSA Plans
testA?empty
nonempty
… …
testA?
29
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
30
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
31
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
32
Search in the Space of FSA Plans
testA?empty
nonempty
… …
pickA
33
Search in the Space of FSA Plans
testA?empty
nonempty
pickA
And finally…
35
Experimental Results
(Using the same pruning rules.)
36
Experimental Results
Statistics for Aranda are estimation from figures in (Srivastava et al. 08) without redoing their experiments.
37
Potential for Improvements
• We have formulated planning with loops as a search problem, and obtained a baseline implementation.
• It is using blind depth-first (iterative deepening) search, and does not scale well without effective pruning rules.
• However, the baseline implementation is a good starting point for adding heuristics and trying other improvement.
38
Heuristics for Action Selection
• We tried a variant of the additive heuristics (Bonet and Geffner 2001):– Assume all fluents are independent;– Count the number of action steps to change each
fluent to a value that may satisfy the goal;– Sum of steps across all fluents used as goal distance;– Try successor states with shorter distance first
39
Heuristics for Action Selection
• Appear relatively effective, but still limited:– The domain of each fluent may be infinite, so
exponential BFS was used;– There are two non-deterministic choices in the search
algorithm (choice of action and choice of transition). Greedily improving only one does not always lead to a good decision.
40
Discussion
• Planning with loops is an interesting problem
• We formally defined FSA plans, representation for loopy plans. [And a logical account for planning problems, FSA plans and correctness.]
• We formulate planning with loops as a search problem.
• FSA planner, the baseline implementation, outperforms KPLANNER, and is good starting point for heuristics and other improvements.