Planning with LoopsSome New Results

Yuxiao (Toby) Hu and Hector LevesqueUniversity of Toronto

2

Outline

• Introduction

• Representing plans with loops– FSA plan: a type of finite state controller

• Constructing plans with loops– Search in the space of FSA plans

• Potential for improvements

• Conclusion

3

The Planning Problem• Finitely many functional fluents, f1,...,fm,

whose domain may be finite or infinite;• Finitely many actions, a1,...,an (world-

changing or sensing);• Initial state: the possible values of each

fluent;• Goal: achieve some goal condition in all

contingencies.

Like contingent planning: solve a class of problems

Incomplete initial state with possibly infinite cases

4

Motivating Example(a variant of striped-tower in Srivastava et al. 2008)

Fluents:– stackA (a list of block colors)– stackB (a list of block colors)– stackC (a list of block colors)– hand (empty/red/blue)

World changing actions:– pickA, pickB, putB, putC;

Sensing actions– testA? (empty/nonempty)– testB? (empty/nonempty)– testH? (red/blue)

5

Motivating Example(a variant of striped-tower in Srivastava et al. 2008)

• Initially: – stackA=[blue,red,red,blue]– stackB=[ ]– stackC=[ ]– hand=empty

• Goal:– stackA=[ ]– stackB=[ ]– striped(stackC)– hand=empty

striped(X) is trueiff

X=[red,blue,…,red,blue]

6

Example 1

A linear solution1.pickA;2.putB;3.pickA;4.putC;5.pickB;6.putC;7.pickA;8.putC;9.pickA10.putC.

7

Example 2

8

Example 2 (cont.)pickA;CASE testH? OF    - red:        putC; pickA;        CASE testH? OF            -red:                putB; ... ...            -blue:                putC; ... ...    - blue:        putB; pickA;        CASE testH? OF            -red:                putC; ... ...            -blue:                putB; ... ...

9

Example 3

Question: Is there a generalized plan solving all problems in this class?

10

Some of the Existing Approaches

• KPLANNER (Levesque 2005)generates robot programs by winding found conditional plans.

• Aranda (Srivastava et al. 2008)obtains generalized plans by winding an abstracted example plan.

• loopDISTILL (Winner and Veloso 2007)learns a “dsPlanner” by merging matching sub-plans of an example partial-order plan.

11

Outline

• Introduction

• Representing plans with loops– FSA plan: a type of finite state controller

• Constructing plans with loops– Search in the space of FSA plans

• Potential for improvements

• Conclusion

12

Plan Representation – Robot Programs

A robot program is defined inductively by–nil;–seq(A,P);–case(A,[R1:P1,…,Rn:Pn]);–loop(P,Q).

(Levesque 1996, 2005)

KPLANNER (Levesque 2005) uses the robot program representation.

13

Robot Program – Some ExamplespickA;CASE testH? OF    - red:        putC; pickA;        CASE testH? OF            -red:                putB; ... ...            -blue:                putC; ... ...    - blue:        putB; pickA;        CASE testH? OF            -red:                putC; ... ...            -blue:                putB; ... ...

pickA;putB;pickA;putC;pickB;putC;pickA;putC;pickAputC.

14

Plan Representation

An FSA plan is a directed graph• Each node represents a program state

o One unique "start" state o One unique "final" stateo Non-final state associated with an

action• Each edge is associated with a

sensing resulto Sensing result of world-changing

actions can be omitted 

15

Plan Execution(for a single complete initial world)

1. Use the "start state" as current program state;

2. If current state is the "final state", then stop;

3. Execute action associated to the current state;

4. Follow the edge with returned sensing result;

5. Make the node pointed to by this edge the current program state, and repeat from Step 2.

16

Robot Program vs. FSA Plan

It can be shown that all robot programs can be represented by equivalent FSA plans.

What about the reverse direction?

17

Robot Program vs. FSA Plan

feel?thirsty

drink

hungry

eat

go2bed

sleep

CASE feel? OF

- thirsty:

drink;

go2bed;

sleep;

- hungry:

eat;

go2bed;

sleep

18

Robot Program vs. FSA Plan

get?instruction suggestion

follow? think?

succeed

fail

workable

unworkableok

revise?fail

19

Outline

• Motivation

• Representing plans with loops– FSA plan: a type of finite state controller

• Constructing plans with loops– Search in the space of FSA plans

• Potential for improvements

• Conclusion

20

Generating Plans with Loops

1. Start with the smallest FSA plan with only one non-final state.

2. If the current program state is final, the goal must be satisfied.

3. Otherwise, execute the action associated to the current program state, non-deterministically pick an applicable one if none is associated.

4. For each possible sensing result of the action, follow the transition and change the current state to the transition target. If no transition is associated to the sensing result, non-deterministically pick one for it.

5. Repeat from step 2.

21

Search in the Space of FSA Plans… …

(Possible values of stackA)

22

Search in the Space of FSA Plans… …

23

Search in the Space of FSA Plans

pickA

… …

24

Search in the Space of FSA Plans

testA?

… …

empty

25

Search in the Space of FSA Plans

testA?empty

nonempty

… …

… …

26

Search in the Space of FSA Plans

testA?empty

nonempty

… …

27

Search in the Space of FSA Plans

testA?empty

nonempty

… …

pickB

28

Search in the Space of FSA Plans

testA?empty

nonempty

… …

testA?

29

Search in the Space of FSA Plans

testA?empty

nonempty

… …

pickA

30

Search in the Space of FSA Plans

testA?empty

nonempty

… …

pickA

31

Search in the Space of FSA Plans

testA?empty

nonempty

… …

pickA

32

Search in the Space of FSA Plans

testA?empty

nonempty

… …

pickA

33

Search in the Space of FSA Plans

testA?empty

nonempty

pickA

And finally…

35

Experimental Results

 

(Using the same pruning rules.)

36

Experimental Results

Statistics for Aranda are estimation from figures in (Srivastava et al. 08) without redoing their experiments.

37

Potential for Improvements

• We have formulated planning with loops as a search problem, and obtained a baseline implementation.

• It is using blind depth-first (iterative deepening) search, and does not scale well without effective pruning rules.

• However, the baseline implementation is a good starting point for adding heuristics and trying other improvement.

38

Heuristics for Action Selection

• We tried a variant of the additive heuristics (Bonet and Geffner 2001):– Assume all fluents are independent;– Count the number of action steps to change each

fluent to a value that may satisfy the goal;– Sum of steps across all fluents used as goal distance;– Try successor states with shorter distance first

39

Heuristics for Action Selection

• Appear relatively effective, but still limited:– The domain of each fluent may be infinite, so

exponential BFS was used;– There are two non-deterministic choices in the search

algorithm (choice of action and choice of transition). Greedily improving only one does not always lead to a good decision.

40

Discussion

• Planning with loops is an interesting problem

• We formally defined FSA plans, representation for loopy plans. [And a logical account for planning problems, FSA plans and correctness.]

• We formulate planning with loops as a search problem.

• FSA planner, the baseline implementation, outperforms KPLANNER, and is good starting point for heuristics and other improvements.