This is a repository copy of Object search by manipulation.

White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/95177/

Version: Accepted Version

Article:

Dogar, MR, Koval, MC, Tallavajhula, A et al. (1 more author) (2014) Object search by manipulation. Autonomous Robots, 36 (1). pp. 153-167. ISSN 0929-5593

https://doi.org/10.1007/s10514-013-9372-x

eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/

Reuse

Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website.

Takedown

If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request.

Autonomous Robots manuscript No.(will be inserted by the editor)

Object Search by Manipulation

Mehmet R. Dogar · Michael C. Koval · Abhijeet Tallavajhula ·

Siddhartha S. Srinivasa

Received: date / Accepted: date

Abstract We investigate the problem of a robot search-ing for an object. This requires reasoning about both

perception and manipulation: some objects are moved

because the target may be hidden behind them, while

others are moved because they block the manipulator’s

access to other objects. We contribute a formulation ofthe object search by manipulation problem using vis-ibility and accessibility relations between objects. Wealso propose a greedy algorithm and show that it is

optimal under certain conditions. We propose a sec-

ond algorithm which takes advantage of the structure

of the visibility and accessibility relations between ob-

jects to quickly generate plans. Our empirical evalu-

ation strongly suggests that our algorithm is optimal

under all conditions. We support this claim with a par-

tial proof. Finally, we demonstrate an implementation

of both algorithms on a real robot using a real object

detection system.

1 Introduction

Imagine looking for the salt shaker in a kitchen cabinet.

Upon opening the cabinet, you are greeted with a clut-

tered view of jars, cans, and boxes—but no salt shaker.

It must be hidden near the back of the cabinet, com-

pletely obscured by the clutter. You start searching for

it by pushing some objects out of the way and moving

Mehmet R. Dogar · Michael C. Koval · Siddhartha S. Srini-vasaThe Robotics Institute, Carnegie Mellon University5000 Forbes Avenue, Pittsburgh, PA, USATel.: +1-412-973-9615E-mail: mdogar,mkoval,siddh@cs.cmu.eduAbhijeet TallavajhulaIndian Institute of Technology KharagpurE-mail: 09me1028@iitkgp.ac.in

others to the counter until, eventually, you reveal yourtarget.

Humans frequently manipulate their environment

when searching for objects. If robotic manipulators areto be successful in human environments, they require a

similar capability of searching for objects by removing

the clutter that is in the way. In this context, clutter

removal serves two purposes. First, removing clutter is

necessary to gain visibility of the target. Second, it is

necessary to gain access to objects that would be oth-

erwise inaccessible.

Prior work has addressed the issues of interacting

with objects to gain visibility and accessibility as sepa-

rate problems. Work on the sensor placement (Espinoza

et al 2011) and search by navigation (Ye and Tsotsos1995, 1999; Shubina and Tsotsos 2010; Sjo et al 2009;

Ma et al 2011; Anand et al 2013) problems focuseson moving the sensor to gain visibility. One canoni-

cal example of the sensor placement problem is the ArtGallery problem (de Berg et al 2008), which would beequivalent to instrumenting the cabinet with enough

sensors to guarantee that the salt shaker is visible. Sim-

ilarly, the search by navigation problem would involvemoving a mobile sensor through the cabinet to searchfor the target.

Conversely, the reconfiguration planning (Ben-Shaharand Rivlin 1998; Dogar and Srinivasa 2012; Ota 2009)

and manipulation planning among movable obstacles

(Stilman et al 2007; van den Berg et al 2008; Chen

and Hwang 1991; Overmars et al 2006) problems fo-cus on moving objects to grant the manipulator ac-

cess to previously-inaccessible configurations. These ap-proaches would be effective at gaining access to the saltshaker once its pose is known, but are incapable of plan-

ning before the target is visually revealed. Recent workdiscusses the object search by manipulation problem

2 Mehmet R. Dogar et al.

Camera View

Fig. 1 An example of the object search problem on a realrobot. The robot is searching for a target object (highlightedby the bounding box) on the table, but its view is occluded(drawn as gray regions) by other objects. The robot mustremove these objects to search for the target. Objects mayblock the robot’s access to other objects.

(Gupta and Sukhatme 2012; Kaelbling and Lozano-

Perez 2012) without reference to optimality. One ofthese works uses a generative model of object-object

co-occurrence and spatial constraints (Wong et al 2013)

to guide the robot’s search, which is similar to theprior distribution we introduce in §8. Other related

work includes exploring the environment with the goal

of building three-dimensional models of novel objects

using maximally informative actions (van Hoof et al

2012).

One of our key insights is that the object searchby manipulation problem requires simultaneously rea-

soning about both perception and manipulation. Someobjects are moved because they are likely to hide thetarget, while others are moved only because they pre-vent the manipulator from accessing other objects in

the scene.

Fig. 1 shows a scene in which both situations oc-

cur. In this figure, HERB (Srinivasa et al 2012)—arobotic platform designed by the Personal Robotics Lab

at Carnegie Mellon University—is searching for the whitebattery pack hidden on a cluttered table. HERB uses

its camera to detect and localize objects. As Fig. 1-Topshows, HERB is initially unable to detect the batterypack because it is occluded by the blue Pop-Tart box.

From HERB’s perspective, the battery pack could behiding in any of the occluded regions shown in Fig. 1-

Left. With no additional knowledge about the location

of the target, HERB must sequentially remove objects

from the scene subject to the physical limitations of itsmanipulator until the target is revealed. For example,

Fig. 1-Right shows that HERB is unable to grasp thelarge white box without first moving the brown juice-box out of the way.

In this paper, we formally describe the object searchby manipulation problem by defining the expected time

to find the target as a relevant optimization criterion

and the concept of accessibility and visibility relations(§2). Armed with these definitions, we are able to pro-

pose and analyze algorithms for object search by ma-nipulation. We make the following theoretical contribu-

tions:

Greedy is sometimes optimal:We prove that, underan appropriate definition of utility, the greedy approach

to removing objects is optimal under a set of conditions,and provide insight into when it is suboptimal (§3).

The connected components algorithm: We intro-

duce an alternative algorithm, called the connected com-

ponents algorithm, which takes advantage of the struc-ture of the scene to approach polynomial time com-

plexity on some scenes (§5). Our extensive experimentsshow that this algorithm produces optimal plans under

all situations, and we present a partial proof of opti-mality.

Finally, we demonstrate both algorithms on our robotHERB (§6.1 and §6.2) and provide extensive experi-

ments that confirm the algorithms’ theoretical proper-

ties (§6).

The interplay between visibility and accessibility has

revealed deep structure in the object search problem,

structure that we were able to identify and exploit to

derive the connected components algorithm. We discussseveral extensions in §7, §8, and §9. We discuss limita-

tions and future work in §10. We believe that our algo-

rithms are a step towards enabling robots to performcomplex manipulation tasks under high clutter and oc-

clusions.

2 Object Search by Manipulation

We start with a scene that is comprised of a known,

static world populated with the set of movable objects

O, each of which has known geometry and pose.

A robot perceives the scene with its sensors and has

partial knowledge of the objects that the scene contains.To the robot, the scene is comprised of the set of visibleobjects Oseen O and the volume of space V that isoccluded to its sensors. In the object search problem,

the occluded volume hides a target object target 2 O

Object Search by Manipulation 3

with known geometry, but unknown pose. For the re-

mainder of this paper, we study a specific variant of the

problem in which the target is the only hidden object,

i.e. O = Oseen [ target. We discuss the presence of

other hidden objects in §9.The robot searches for the target by removing ob-

jects from Oseen until the target is revealed to its sen-sors. As objects are removed, fewer objects remain in

the scene, which we denote by s Oseen. Oseen refers

to the initial set of visible objects and does not changeas objects are removed. We define the order in which

objects are removed as an arrangement.

Definition 1 (Arrangement) An arrangement of theset of objects o is a bijection Ao : 1, . . . , |o| ! o where

Ao(i) is the ith object removed.

Additionally, we define Ao(i, j) as the sequence of

the ith through the jth objects removed by arrangementAo.

Given an arrangement Ao that reveals the target,the expected time to find the target is

E(Ao) =

|o|X

i=1

PAo(i) · TAo(1,i) (1)

where PAo(i) is the probability that the target will berevealed after removing object Ao(i) and TAo(1,i) is the

time to move all objects up to and including Ao(i).

Our goal is to find the arrangement AOseen

that

minimizes E(AOseen

); i.e. reveals the target as quickly

as possible.

2.1 Visibility

When the robot removes a set of objects from the sceneit reveals a set of candidate poses of the target object

that were previously occluded. These revealed configu-

rations are defined in target’s configuration space C.

Definition 2 (Revealed Configurations) The setof candidate target poses Co|s C revealed by re-moving objects o s from a scene containing objects

s Oseen.

The probability of revealing the target after remov-

ing o from s is determined by the volume of Co|s. Wecall this the revealed volume of those objects.

Definition 3 (Revealed Volume) The volume Vo|s

revealed by removing objects o s from a scene con-taining objects s Oseen is

Vo|s =

Z

x2Co|s

P0(x) dx (2)

A

BC

Vjoint

VA

VBVC

(a) Initial Scene

BC

VB VC

(b) A Removed

Fig. 2 (a) An example of a scene containing a joint occlu-sion. Occlusions are drawn as dark gray and the joint occlu-sions as light gray. (b) The scene after A is removed.

where P0(x) is a prior distribution over the pose of the

target object.

We assume that P0(x) is uniform for the remainder

of this discussion to simplify our examples. We discussthe general case of using a non-uniform P0(x) to encode

semantic knowledge about the scene in §8.

Additionally, we will drop the scene s from this no-tation whenever it is obvious from the context. For ex-

ample in Fig. 2a we write VA instead of the more verbose

VA|A,B,C. Similarly, instead of using VAo(i,j)|Ao(i,|o|)

to refer to the volume revealed between the ith and jth

steps of an arrangement, we will simply use VAo(i,j).

In Fig. 2a we show the revealed volumes of objects inan example scene1. Vjoint is jointly occluded by object

A and B, and is not included in either VA or VB. This is

because Vjoint will not be revealed if only A or only B

is removed from the scene.

In Fig. 2b we show VB after A is removed from thescene in Fig. 2a. Since A is no longer in the scene, VB now

includes Vjoint. Similarly, VA would expand to include

Vjoint if B was the first object removed from the scene.Regardless of the order in which A and B are removed,

the revealed volume of A, B is VA,B = VA+VB+Vjoint.

In the most general case, an arbitrary number of objectscan jointly occlude a volume. In that case, the volumewould be revealed only after all of the occluding objectsare removed from the scene.

Given an arrangement AOseenwe compute the prob-

ability that the target will be revealed at the ith stepusing the revealed volume

PAOseen (i) =VAOseen (i)

VOseen

(3)

1 We use two-dimensional examples, e.g. Fig. 2, throughoutthe paper for clarity of illustration. Our actual formulationand implementation uses complete three-dimensional modelsof the scene, objects, and volumes.

4 Mehmet R. Dogar et al.

kk

A

B

CVB

VA

VC

A

B

CVB

VA

VC

(a) Initial Scene

k

kk

B

CVB

VC

B

CVB

VC

(b) A Removed

Fig. 3 A scene where the greedy algorithm performs sub-optimally due to an accessibility constraint.

2.2 Accessibility

The manipulator uses a motion planner to grasp an ob-

ject and remove it from the scene. To achieve this, the

object must be accessible to the manipulator. Accessi-

bility is blocked by other visible objects, and also by the

occluded volume, which the manipulator is forbidden to

enter.

Definition 4 (Accessibility Constraint) There isan accessibility constraint from an object A to object B

if A must be removed for the manipulator to access B.

Any arrangement of objects in a scene must respect

the objects’ accessibility constraints. For example, inFig. 1-Right, the access to the big box is blocked by

the smaller box in front of it.We identify the accessibility constraints using a mo-

tion planner, which returns a manipulator trajectoryfor each object in the scene. The manipulator trajec-

tory for an object sweeps a certain volume in the space(illustrated as light blue regions in Fig. 1). Objects that

penetrate the swept volume result in accessibility con-

straints. Additionally, objects for which the occludedvolume penetrates the swept volume also result in ac-

cessibility constraints.We also use the manipulator trajectory for an object

A to compute TA by estimating the time necessary toexecute the trajectory on the robot. Since there is only

a single action for each object, TA is constant for a given

scene and does not depend on the sequence in whichobjects are removed.

3 Utility and Greedy Search

In this section, we discuss a greedy approach to solving

the object search by manipulation problem.While the overall goal is to minimize the amount of

time it takes to find the target, a greedy approach re-

quires a utility function to maximize at every step. Thefaster the robot reveals large volumes, the sooner it will

AB

C

VA

VB VC

(a) Initial Scene

C

VC

B

VB

(b) A Removed

Fig. 4 A scene where the greedy algorithm performs sub-optimally due to a visibility constraint.

find the target. Using this intuition, we define the util-

ity of an object similar to the utility measures defined

for sensor placement (Ye and Tsotsos 1995; Espinoza

et al 2011).

Definition 5 (Utility) The utility of an object A isgiven by

U (A) =VATA

This measure naturally lends itself to greedy search.

A greedy algorithm for our problem ranks the accessi-ble objects in the scene based on their utility and theremoves highest utility object. This results in a new

scene, whereby the algorithm repeats until the target

is revealed. In the worst case, this continues until all

objects are removed.

Unsurprisingly, it is easy to create situations where

greedy search is suboptimal. Consider the scene in Fig. 3.In this scene, VB & VC > VA. For the sake of simplicity

we assume that the time to move each object is similar,

hence U(C) > U(A). As B is not accessible, the greedy al-gorithm compares U(A) and U(C) and chooses to move

C first, producing the final arrangement C ! A ! B.

However, moving the lower utility A first is the opti-mal choice because it reveals VB faster (Fig. 3b), and

gives the optimal arrangement A ! B ! C. It is easy tosee that greedy can be made arbitrarily suboptimal by

adding more and more objects with utility U(C) to the

scene.

We present a second example of greedy’s subopti-

mality in Fig. 4. In this scene, all objects are accessible,VC > VA, and VC > VB. The greedy algorithm inspects

the utilities and moves C first. However, there is a largevolume jointly occluded by A and B, such that when

either A or B is removed, the volume revealed by thesecond object significantly increases. We illustrate thisin Fig. 4b with A is removed. Hence, the optimal ar-

rangement is A ! B ! C because it quickly reveals thelarge volume jointly occluded by A and B.

Object Search by Manipulation 5

The examples in Fig. 3 and Fig. 4 may suggest ak-step lookahead planner for optimality. However, the

problem is fundamental: one can create scenes wherearbitrarily many objects jointly occlude large volumes,

or where arbitrarily many objects block the accessibilityto an object that hides a large volume behind it.

Surprisingly, however, it is possible to create non-trivial scenes where greedy search is optimal. We definethe requirements of such scenes in the following theo-

rem.

Theorem 1 In a scene where all objects are accessi-

ble and no volume is jointly occluded, a planner that is

greedy over utility minimizes the expected time to find

the target.

Proof Suppose that A is a minimum expected time

(i.e. optimal) arrangement. For any i, 1 i < |Oseen|,we can create a new arrangement, A, such that the ith

and (i+1)th objects are swapped; i.e. A(i) = A(i+1)

and A(i + 1) = A(i). A must be a valid arrangement

because all objects are accessible.

No volume is jointly occluded, so the revealed vol-ume of all objects will stay the same after the swap; i.e.

VA(i) = VA(i+1) and VA(i+1) = VA(i). Since the rest ofthe two arrangements are also identical, using Eq. 1 and

Eq. 3, we can compute the difference between E (A) and

E (A) to be:

E (A)−E (A) = VA(i) ·TA(i+1)−VA(i+1) ·TA(i). (4)

E (A) is optimal, therefore E (A)− E (A) ≥ 0

andVA(i)

TA(i)≥

VA(i+1)

TA(i+1),

which is simply U(A(i)) ≥ U(A(i+1)). Hence, the op-timal arrangement consists of objects sorted in weakly-

descending order by their utilities.

There can be more than one weakly-descending or-dering of the objects if multiple objects have the same

utility. To see that all weakly-descending orderings areoptimal, the same reasoning can be used to show thatswapping two objects of the same utility does not change

the expected time of an arrangement. ut

This result is rather startling. The greedy algorithmis incredibly efficient in terms of computational com-

plexity. At each step, the algorithm finds the accessi-ble object with maximum utility in linear time. In a

scene of n objects, this results in a total computationalcomplexity of O(n2). We show in §5 that the worst-case complexity of the optimal search is O(n22n). The

theorem, however, shows that there are scenes in whichgreedy is optimal. We shall show in §6 that these scenes

do occur surprisingly regularly even with randomly gen-

erated object poses. However, as we have shown above,

the greedy algorithm can also produce arbitrarily sub-optimal results.

In the next section we present an algorithm based

on A-Star search, which is always optimal but has ex-ponential computational complexity. Then, in §5 wepresent a new algorithm which approaches the polyno-

mial complexity of the greedy algorithm, yet maintainsoptimality in the general case as shown by our empiricalevaluations in §6.

4 A-Star Search Algorithm

In this section we present an optimal algorithm for solv-ing the object search by manipulation problem. We first

formulate the problem as a deterministic single-source

shortest path problem. We then find the optimal solu-tion by executing an A-Star search with an admissible

heuristic.

Formulating this problem as a deterministic single-source shortest path problem is possible only because

of special structure in the problem. The optimal policy

always removes objects from the scene in a determinis-tic order (i.e. an arrangement) until the target is found.

See §A for a derivation of this fact from a formulation

of the problem as a Markov decision process.

4.1 Single-Source Shortest Path Problem

Define a directed acyclic graph G = (N,E, c) with

nodes N 2Oseen , edges E Oseen Oseen, and costfunction c : E ! R

+. A node s 2 N is the set visi-

ble objects remaining in the scene. The directed edge(s, s \ a) 2 E removes object a from scene s.

Consider the single-source shortest path problem inG with Oseen as the start node and ; as the goal node.Let edges exist between nodes s and s0 if they differ by

a single object and if that object is accessible in s. Ev-ery path from the start to the goal removes all objects

from the scene in a different order and corresponds toa different arrangement.

Furthermore, each edge (s, s \ a) 2 E has cost

c(s, s0) =

Va|s

VOseen

TOseen\s0

where TOseen\s0 is the total time required to reach s0 =s\a from the initial scene Oseen. For every path goingthrough an edge, the cost indicates the probability that

the target will be revealed at that edge multiplied withthe time required to reach and execute the edge.

6 Mehmet R. Dogar et al.

The sum of the edge costs along any path from the

start to the goal is exactly Eq. 1. In other words, the

cost of a path in this graph is equal to the expectedtime to find the target while following corresponding

arrangement. Therefore, the minimum-cost in G corre-sponds the optimal arrangement A

Oseenthat minimizes

E(AOseen

).

4.2 Admissible Heuristic

The single-source shortest path problem can be solved

using several well-known algorithms. We use the A-Starsearch algorithm (Hart et al 1968) with an admissible

heuristic to efficiently find the optimal solution. A-Staris optimal if its heuristic is admissible, i.e. does notoverestimate the cost from a state to the goal.

Suppose we start at the arbitrary node

s = a1, a2, . . . , an and the minimum-cost path to thegoal is given by the sequence of actions [a1, a2, . . . , an].

Then, the optimal cost-to-go is

h(s) = c(s, s \ a1) + c(s \ a1, s \ a1, a2) + · · ·

+c(s \ a1, a2, ..., an−1, ;)

=

Va1|s

VOseen

(TOseen\s + Ta1)

+

Va2|s\a1

VOseen

(TOseen\s + Ta1 + Ta2)

+ · · ·

+

Van|s\a1,...,an−1

VOseen

(TOseen\s + Ta1 + · · ·+ Tan).

Now we present our heuristic,

h(s) =

Va1|s

VOseen

(TOseen\s +mina2s

Ta)

+

Va2|s\a1

VOseen

(TOseen\s +mina2s

Ta)

+ · · ·

+

Van|s\a1,...,an−1

VOseen

(TOseen\s +mina2s

Ta)

=

Vs

VOseen

TOseen\s +mina2s

Ta

]

.

Since h(s) h(s), h(s) is admissible. Intuitively, h(s)optimistically reasons that: (1) we execute the minimum-

time action and (2) it reveals all of the remaining vol-ume. Since h(s) is admissible, A-Star is optimal and

will return the minimum-cost path.

4.3 Computational Complexity

Running an A-Star search on a graph with n nodes andm edges has a worst-case complexity ofO((m+n) log n).

!BA

C

D E

F

Fig. 5 Left: An example scene. Volumes occluded by a singleobject are shown in dark gray, joint occlusions are shown inlight gray, and swept volumes are shown in light blue. Right:The corresponding graph with three connected components.

Algorithm 1: Object Search With Connected

Components

1 c1, c2, ..., cm FindConnectedComponents2 foreach connected component ci do3 A

ci AStar(ci)

4 AOseen

[ ]

5 repeat6 bag ;7 foreach component arrangement A

ci do8 for j 1 to |ci| do9 bag.Add( Aci(1, j) )

10 seq arg maxA2bag

U(A)

11 Add seq to the end of AOseen

12 Remove seq from the Aci it belongs

13 until all objects are in the plan14 return A

Oseen

Unfortunately, the graph constructed for a scene of sizen = |Oseen| has up to 2n nodes and no more than n2n

edges, resulting in a worst-case complexity of O(n22n).

Our experimental results (§6) confirm that A-Star ap-

proaches this worst-case bound in practice and it is in-tractable to run this algorithm on large scenes.

5 Connected Components Algorithm

The structure of the object search problem becomes

more clear once we represent the visibility and accessi-

bility constraints of a scene as a graph. Each node ofthis graph corresponds to an object in the scene. There

is an edge between the nodes A and B if:

– A is blocking the access to B, or vice versa; or

– A and B are jointly occluding a non-zero volume.

An example scene and the corresponding graph is inFig. 5.

We can divide the constraint graph into connected

components. A connected component of the graph is

a subgraph such that there exists a path between anytwo nodes in the subgraph (Hopcroft and Tarjan 1973).

Object Search by Manipulation 7

For example, there are three connected components in

Fig. 5: A, B, C, D, and E, F.

A key insight is that the objects in a connected

component do not affect the utility of the objects inanother connected component. Hence, we can performan optimal search, e.g. using A-Star, to solve the ar-

rangement problem for a connected component inde-pendently and then merge the solutions to produce acomplete arrangement of the scene.

It is non-trivial to merge arrangements of multipleconnected components. The complete plan may switch

from one connected component to the other and thenswitch back to a previous component. Our algorithm

provides an efficient greedy way to perform this merge.

The examples in Fig. 3 and Fig. 4 show that theutility of a single object is not informative enough to

achieve general optimality with a greedy algorithm. In-

stead, we consider the utility of removing multiple ob-

jects from the scene.

Definition 6 (Collective Utility) The collective util-

ity of a set of objects o is given by

U(o) =Vo

To

A general greedy approach which considers the col-

lective utility of all possible sequences of all sizes in thescene would quickly become infeasible as the number

of such sequences is O(|o|!). In our case, we take ad-vantage of the fact that we have optimal plans for each

connected component in which the objects are already

sorted. We then need to compute collective utilities ofonly the prefixes (i.e. the first k objects where k ranges

from 1 to the size of the connected component) of these

optimal sequences.

We present our algorithm in Algorithm 1 that usesthe collective utility of sequences from connected com-ponents to generate an arrangement of the completescene. It first identifies the connected components in

the scene (Line 1). Then it finds the optimal arrange-ment internal to a connected component using A-Starsearch (Line 3). It then merges these arrangements iter-

atively by finding the maximum utility2 prefixes of theoptimal arrangements of the connected components.

In §6 we show that Algorithm 1 generates the op-timal result in all scenes we tried it on and it uses a

fraction of the time A-Star requires on the completescene. We present a partial proof of our algorithm’s op-

timality in the appendix.

2 In the rare event that that multiple sequences share themaximum utility, the algorithm breaks the tie by choosingthe sequence with the maximum utility prefix recursively.

5.1 Complexity of the Connected ComponentsAlgorithm

The connected components algorithm divides the set

of objects into smaller sets, runs A-Star on each con-nected component, and then merges the plans for each

component. If the scene has no constraints, then there isone object per connected component and this algorithm

reduces to the greedy algorithm. Conversely, if the con-straint graph is connected, this algorithm is equivalent

to running A-Star on the full scene. Therefore, the per-

formance of this algorithm ranges from O(n2), the per-formance of the greedy algorithm, to O(n22n), the per-formance of A-Star, depending upon the size of the con-

nected components. Geometric limitations put an upperbound on the number of accessibility and joint occlu-

sion constraints that are possible in a given scene, so itis unlikely that any scene will exercise the worst case

performance. These performance gains will be most sig-

nificant on large scenes in which objects are spatiallypartitioned, e.g. on different shelves in a fridge, but will

be modest on small, densely packed scenes.

6 Experiments and Results

We investigated the performance of the different algo-rithms through extensive experiments in simulation and

on a real robot. We implemented the greedy, A-Star,

and connected components algorithms in OpenRAVE

(Diankov and Kuffner 2008). We also implemented abaseline algorithm which randomly picks an accessi-

ble object and removes it from the scene. We evalu-

ated these algorithms on randomly generated scenes.Each scene contained n objects—half juice bottles and

half large boxes—that were uniformly distributed over

a wide 1.4 0.8 m workspace. None of the generated

scenes contained hidden objects and the planner useda motion planner based on the capabilities of a sim-ple manipulator. The manipulator was only capable of

moving straight, parallel to the table and at a constantspeed of 0.1 m/s.

In our implementation, we assume that the target

rests stably on the workspace and C = SE(2). Weapproximate C with a discrete set of configurations

C C. Next, we compute a discrete approximation

of each set of revealed configurations Co|s = x 2 C :Ω(x|s\o)^¬Ω(x|s) using the visibility criterionΩ(x|s)that returns whether x 2 C is visible in the scene s.

Finally, we compute Vo|s by approximating the integral

over Co|s in Eq. 2 with a summation over Co|s. In princi-ple, our framework supports any deterministic visibility

criterion Ω : C2Oseen ! 0, 1. However partial viewsof objects are difficult to detect in practice. Therefore,

8 Mehmet R. Dogar et al.

4 6 8 10 12

Number of Objects

0

5

10

15

20

25

30

35

40

45

Exp

ect

ed

Du

rati

on

(s)

A*

Components

Greedy

Random

4 6 8 10 12

Number of Objects

10−2

10−1

100

101

102

103

Pla

nn

ing

Tim

e(s

)

4 6 8 10 12

Number of Objects

1

2

3

4

5

6

7

8

9

10

11

12

Siz

eof

Larg

est

Com

pon

en

t

0

25

50

75

100

125

150

175

200

225

Fig. 6 Performance of the random, greedy, A-Star, and connected component planners as a function of number of objects. Allresults are averaged over approximately 400 random scenes and are plotted with their 95% confidence interval. The planningtimes are presented in log-scale, where the confidence intervals are also plotted as log-scale relative errors (Baird 1995). Therelationship between scene size and the size of the largest connected component is also plotted as a two-dimensional histogram.

4 6 8 10 12

Number of Objects

0

10

20

30

40

50

60

Exp

ect

ed

Du

rati

on

(s)

A*

Components

Greedy

Random

(a) Expected Search Duration (b) Example Scenes

Fig. 7 (a) 95th percentile of expected time to find the target(b) Two example scenes where greedy performed poorly. Theblack lines denote the workspace boundary.

our implementation considers the target at a certain

pose visible if and only if it is entirely visible. This is

implemented by sampling points on the surface of the

target, raytracing from the sensor to each point, and

verifying that no rays are occluded.

We present results from scenes with 4, 6, 8, 10, and

12 objects in Fig. 6 along with the 95% confidence in-tervals. We conducted approximately 400 simulations

for each different number of objects, resulting a in to-tal of 2000 different scenes. The data in Fig. 6a shows

that the greedy algorithm becomes increasingly sub-

optimal as the number of objects increases. All threealgorithms significantly outperform the random algo-rithm, which serves as a rough upper bound for the ex-pected search duration. Unfortunately, the optimality

of A-Star comes with the cost of exponential complex-ity in the number of objects. This complexity causes theplanning time of A-Star to dominate the other planning

times shown in Fig. 6b (note the logarithmic scale).

While still optimal in all 2000 scenes, the connectedcomponents algorithm achieves much lower planning

times than A-Star. By running A-Star on smaller sub-problems, the connected components algorithm is expo-

nential in the size of the largest connected component,

k, instead of the size of the entire scene. Fig. 6c shows

that k n/2 for n 8 and increases when n = 10,causing the large increase in planning time between

n = 8 and n = 10 in Fig. 6b. With fixed computa-

tional resources, these results show that the connected

components algorithm is capable of solving most scenesof size 2n in the amount of time it would take A-Star

to solve a scene of size n. For sparse scenes, the con-

nected components algorithm achieves optimality withplanning times that are comparable those of the greedy

algorithm.

One surprising results of our experiments is that,

while greedy is not optimal in the general case, it doesremarkably well on average. We found that in 50% of

the 2000 different scenes, the greedy algorithm pro-

duced the optimal sequence. Our explanation for greedy’s

performance is that the geometry of our workspace en-

forces a tradeoff between the volume occluded by anobject and the number of objects that block its accessi-

bility: For an object to occlude a large volume it must

be near the front of the workspace, which makes it un-

likely that multiple objects can be placed in front ofit.

To see the greedy’s worst-case behavior, we plotted

the expected time to find the target for the 5% of sceneswhere greedy performed worst in Fig. 7a. Across allthe scenes, the worst performance was 2.04 times theexpected duration of the optimal sequence. We show

two example scenes where greedy performs poorly inFig. 7b. Both scenes include small bottles blocking ac-

cess to large boxes. There is very little volume hidden

behind the bottles, so the boxes are—suboptimally—removed late in the plan.

If the goal is to minimize the total time to planand also execute the search, then we must trade-off the

gains in planning time achieved by the greedy algorithm

Object Search by Manipulation 9

with the extra actions the robot needs to execute due

to the greedy algorithm’s suboptimality. In a setting

where action executions are fast and greedy is nearly

optimal, one should use the greedy algorithm. If action

executions are slow and greedy plans are increasingly

suboptimal (e.g. in environments with a large number

of objects), one should use the connected-components

algorithm.

6.1 Real Robot Implementation

We implemented the greedy and connected components

algorithms on our robot HERB. We used HERB’s cam-

era and the MOPED (Martinez et al 2010) system todetect and locate objects in the scene. We present an

example scene where HERB successfully found the tar-

get object using the greedy algorithm in Fig. 8. In thisscene the target object, a battery pack, is hidden be-

hind the large box, which also occludes the largest vol-ume. Since the large box is inaccessible, the greedy

planner compares the utilities of the other three ob-jects, and removes the largest utility object at each step.

Even though the large box is hiding a large volume, the

greedy planner removes it last, resulting in a long taskcompletion time.

In Fig. 9 the scene is the same but HERB uses the

connected components algorithm. There are three con-nected components in this scene BlueBox, Bottle,and LargeBox, SmallBox. The connected components

algorithm considers the collective utilities of multipleobjects from each connected component, including both

U(SmallBox) and U(SmallBox, LargeBox). The utility

of SmallBox is very small compared with the other im-

mediately accessible objects, but combined with theLargeBox, their utility is large enough that the algo-

rithm removes SmallBox as the first object. It then re-moves the large box and finds the target object. Wepresent the actual footage of these experiments at

http://youtu.be/i06GBj1iDOo.

6.2 Performance in Human Environments

Objects are not distributed randomly in real human en-

vironments: they display a structure specific to humanclutter. We conducted a simple evaluation of our plan-

ner by creating scenes which are similar in structure to

human clutter.

For this evaluation we identified three different placeswhere a robot might need to search for an object by

manipulation: a bookshelf, a cabinet, and a fridge. Wecaptured images of the natural clutter in these envi-

Table 1 Planning and Execution Times

Total Time Planning Execution

Shelf 132.7s 16.1s 116.6sCabinet 94.6s 26.1s 68.5sFridge 242.0s 16.7s 225.3

ronments in our lab. We display these images as the

leftmost column in Fig. 10.

Limitations of our robot’s perception system and

the difference between the sizes of a human arm/hand

and our robot’s manipulator prevented us from run-ning our planner directly on these scenes. Therefore,

we constructed new scenes that are scaled up to the

dimensions of HERB’s manipulator and consist of ob-

jects that our perception system can reliably detect.

We attempted to faithfully mimic the relative size and

configuration of objects in the original scenes as muchas possible. We hid the target object randomly in the

occluded portion of the table.

We present snapshots from our robot’s search for

the target as the rows of Fig. 10. All plans were auto-

matically generated by the connected components al-

gorithm and HERB successfully found the target in allthree scenes. Tab. 1 shows the planning time, execu-

tion time, and the total time it took the robot to find

the object. Note that execution time is the dominating

factor, emphasizing the importance of generating shortplans when searching for objects with a real robot.

7 Planning to Place Objects

The robot must place an object down before picking

up another one. If the robot is allowed to place theobject at a new pose where it creates new visibility or

accessibility relations, then the object search problembecomes a version of reconfiguration planning which isknown to be NP-Hard (Wilfong 1988). We avoid this

complexity by placing objects only at poses that donot create new accessibility or visibility relations.

Our formulation requires us to compute the timeit takes to manipulate an object before we decide the

arrangement. We use fixed placement poses on a nearbyempty surface to satisfy this constraint. We found this

to be a reasonable strategy in practice: Even when therobot is working in a densely crowded cabinet shelf,

there is usually a nearby counter or another shelf to

place objects on.

However, one can also re-use the explored space to

place objects: after an object is picked up and the robotsees the volume behind that object, the planner can

10 Mehmet R. Dogar et al.

Robot's View

!( ) = 49

!( ) = 19

!( ) = 6

!( ) = 19

!( ) = 6

!( ) = 6 !( ) = 90

Fig. 8 Greedy planner. We present the utility of all accessible objects at each step. The pose of the target (unknown to therobot) is marked with dashed lines in the illustration.

Robot's View

!( ) = 49

!( ) = 19

!( ) = 6

!( ) = 52

Fig. 9 Connected-components planner. The utilities of allprefixes from each connected component are presented at eachstep.

safely use this volume. In particular, for an arrangement

AOseen, object AOseen

(i) can be placed where:

– it avoids penetrating VAOseen (i+1,|Oseen|),– it avoids occluding VAOseen (i+1,|Oseen|),

– it avoids blocking access to the objects

AOseen(i+ 1, |Oseen|),

– it avoids colliding the placement poses of the objectsAOseen

(1, i− 1).

In Fig. 11 we illustrate each of these constraints andthe remaining feasible placement poses for an object.

Surprisingly, certain scene structures lead to very

simple and fixed placement strategies. For example, ina scene where there are no accessibility relations and

no joint occlusions (where the greedy algorithm is op-timal), an object can be placed where it was picked up:

the robot lifts an object, looks behind it, and placesit back. This strategy respects the constraints listed

above.

8 Encoding Semantic Knowledge

All of our examples of the object search by manipula-tion problem assume that the target is equally likely

to be found anywhere in the workspace. However, hu-man environments are not random: semantic knowl-

edge about the environment provides useful informationabout where a hidden object may be found. For exam-

ple, a can of soda is more likely to be in the refrigeratorthan in the dish washer.

There are several existing techniques for learning co-

occurrence probabilities objects (Kollar and Roy 2009;

Wong et al 2013) in real environments. We can natu-

rally incorporate this type of semantic knowledge into

the prior distribution P0(x). The prior distribution changes

the volume revealed by each target and can be directlyexploited by same the greedy, A-Star, and connected

components algorithms described above.

For example, suppose that the robot is searching for

a bottle of mustard in the refrigerator. Fig. 12a showsan example of this scenario where the refrigerator con-

tains a small bottle of ketchup K and two large foodcontainers A and B. The mustard M is hidden behind the

bottle of ketchup. Assuming that the robot has no prior

over the location of the mustard, VA, VB & VK. AssumingTA = TB = TK, the optimal arrangement with no prior

distribution is A ! B ! K because U(A, B) > U(K)

and U(B) > U(K).

However, this plan ignores important semantic knowl-

edge about the scene: mustard is more likely to be found

near ketchup than the containers of food. This knowl-

edge causes P0(x), shown in Fig. 12b, to be peakedaround K. This prior influences the revealed volumessuch that VK & VA, VB, the opposite relationship as

above. This prior knowledge changes the optimal ar-

rangement to K ! A ! B because U(K) > U(A, B) andreveals the mustard more quickly.

9 Replanning for Hidden Objects

All of the algorithms described above can be easily gen-eralized to handle environments that contain hidden ob-

Object Search by Manipulation 11

Fig. 10 Example executions on scenes inspired by real human environments. Scenes are inspired from a cluttered humanshelf (top), a cabinet (center), and a fridge (bottom).

Fig. 11 An example illustrating placement contraints. (a) In this scene the small box is moved to the left and then the largebox is picked up. Now the planner must place the large box. The large box cannot be placed where (b) it will penetrate thevolume which is not explored yet; (c) it will occlude the volume which is not explored yet; (d) it will block access to the objectswhich are not moved yet; (e) it will collide with the new poses of the objects which are already moved. (f) The combinedplacement constraints for the large box.

KVKA

B

VB

VA

(a) Refrigerator Scene (b) Prior Distribution P0(x)

Fig. 12 Example scene where semantic knowledge influencesthe optimal order of removing objects. The robot is search-ing for the mustard (dotted circle) and has a strong priordistribution that it is near the ketchup (K).

jects in addition to the target. However, objects must

be smaller than the target object to avoid the danger of

the arm colliding with an hidden object while search-

ing for the target. If this condition holds, then one can

simply re-execute the planner on the remaining objects

whenever an hidden object is revealed. This strategy is

Fig. 13 Example of replanning on a scene with two hiddenobjects. Each replanning stage is shown as a separate framealong with the corresponding plan. Hidden objects are shownas semi-transparent and the workspace bounds are indicatedby a black line.

optimal given the available information if there is no

a priori information about the type, number, or loca-

tion of the hidden objects and the plans are generated

using an optimal algorithm in each updated scene. If

there are k hidden objects, then this replanning strat-

egy multiplies the total planning time of an optimal

algorithm by a factor of O(k). In the case of the greedyor random algorithm, the replanning adds O(k) over-

head from reevaluating visibility after each object is

revealed.

12 Mehmet R. Dogar et al.

Fig. 13 shows an example of replanning on a scenecontaining six objects. Two objects, shown as semi-

transparent in the figure, are initially hidden and arerevealed once the occluding objects are removed. The

robot begins by executing the connected componentsplanner on a scene containing the four visible objects.After executing the first two actions in that plan, the

robot detects that a new object has been revealed andreplans for the remaining objects. In this case, the opti-mal ordering is unchanged and the newly-revealed ob-

ject is simply appended to the existing plan. After ex-ecuting another action, the second hidden object is re-vealed and the robot must replan a second time. Thistime, order of the optimal sequence is changed by the

addition of the hidden object and it would be subopti-mal to continue executing the previous plan.

10 Future Work

We are excited about exploring this problem deeper andrelaxing some of the simplifying assumptions in future

work.

Integrated Motion Planning We use a motion plan-

ner for the manipulator that is conceptually decoupled

from finding the optimal arrangement. However, there

are aspects of the object search problem that can beintegrated into the motion planning process. For exam-

ple, there may be multiple trajectories for grasping an

object that require differing numbers of objects to be

moved out of the way. In this respect, we are excitedabout studying how a more complex motion planner,

e.g. one returning a minimum-constraint violation tra-

jectory (Hauser 2012), can be integrated into our sys-tem. The object search formulation can also take into

account the motion of moving from one object to the

next one, trying to minimize the time spent in between.This can make the object search problem similar to the

traveling agent problem (Moizumi and Cybenko 2001)

where the latencies between nodes produced are pro-duced by a motion planner.Improved Perception Model. Our framework allows

for any sensor model. We will explore relaxing the con-

servative requirement of the entire target being visibleto other perceptual models that address partial visibil-ity.

Integrated Sensor Planning. Aside from reaching

to objects, the robot does not move its base in our cur-rent implementation. Through combining the ability of

search by manipulation with sensor planning, the robotcan find targets faster. Sensor planning would includeworking with multiple camera poses and planning forthe base when searching for a target in a larger envi-

ronment.

A,B,C B

A

C

B,C

A,C

A,B ...

...

A

C

...

...

found

Fig. 14 Transition graph of the of the object search MDP.States (rounded rectangles) are sets of visible objects andactions (squares) correspond to removing an object from thescene. Each action has a probability of revealing target andtransitioning into found.

A Formulation as a Markov Decision Process

In this section, we formulate the object search problem asa Markov decision process (MDP) and show that the opti-mal policy minimizes the expected time to find the target.Next, we show that the MDP is deterministic and that eachpolicy corresponds to an arrangement. This provides insightinto how the A-Star search algorithm can be used to find theoptimal policy for a stochastic problem.

A MDP is the four-tuple (S,A, Γ,R) where S is the statespace, A is the action space, Γ (s0|s, a) is the transition model,and R(s, a) is the reward function (Kaelbling et al 1996).

For the object search problem, the state s 2 2Oseen [found is the set of visible objects remaining in the sceneand an absorbing goal state found that corresponds to thetarget being visible. The scene starts with all objects presents0 = Oseen and the robot sequentially chooses an object a 2s to remove. After removing a, we transition to the successorstate s0 according to

Γ (s0|s, a) =

(

1−Va|s

Vs: s0 = s \ a

Va|s

Vs: s0 = found

and receive reward R(s, a) = −Ta. This process continuesuntil there is a transition into found and the MDP terminateswith zero reward. Fig. 14 shows a graphical depiction of thestate transition graph for a scene with three objects A, B, Cand no accessibility relations.

A policy π : S ! A specifies which action π(s) to takewhen in state s. In the case of the object search problem, πdictates which object to remove from the scene at each step.We wish to find the optimal policy π that maximizes the

sum of expected future reward E[P|Oseen|

t=1 R(st, at)].Any policy π induces the value function Λπ : S ! R,

where Λπ(s) is the sum of expected future reward from start-ing in state s and following π to termination.3 The valuefunction of the optimal policy π satisfies the Bellman equa-tion

Λπ

(s) = maxa2A

X

s02S

Γ (s0|s, a)h

R(s, a) + Λπ

(s0)i

, (5)

which recursively relates the value of state s with that of itssuccessors (Kaelbling et al 1996).

We can take advantage of the structure of the objectsearch problem to reduce the Bellman equation to a simpler

3 We use Λπ in place of V π to denote the value function toavoid confusion with revealed volume Vs.

Object Search by Manipulation 13

form. First, the object search problem has a sparse transi-tion model: Γ (s0|s, a) = 0 for all s0 62 s \ a, found. Second,Λπ(found) = 0 for the absorbing goal state. Using these ob-servations, we can simplify Eq. 5 to

Λπ

(s) = maxa2s

h

R(s, a) + Γ (s \ a|s, a)Λπ

(s \ a)

+ Γ (found|s, a)Λπ

(found)i

= maxa2s

−Ta +

1−Va|s

Vs

Λπ

(s \ a)

]

,

which, surprisingly, has the same form as the value functionof a deterministic MDP. This agrees with our earlier intuition:while the outcome the object search problem is stochastic, theoptimal order of removing objects is completely deterministic.

In fact, π is equivalent to an arrangement AπOseen

thatspecifies an open-loop order in which to remove objects. Dur-ing execution objects are removed according to Aπ

Oseenuntil

the target is revealed and the robot halts (i.e. transitions tofound). This equivalence is what enables us to formulate theobject search problem as a deterministic search in §4.

B Optimality of Connected Components

We present a partial proof of optimality for Algorithm 1.

We state a property of the collective utility as a lemma.

Lemma 1 Given an arrangement Ao,

U(Ao(1, |o|)) ≥ U(Ao(1, k))

=) U(Ao(k + 1, |o|)) ≥ U(Ao(1, |o|))

In other words, if the utility of the complete arrangement islarger than the utility of the first k objects, then the utility ofthe last |o| − k objects must be larger than the utility of thecomplete arrangement.

Proof We are given that

VAo(1,k) + VAo(k+1,|o|)

TAo(1,k) + TAo(k+1,|o|)

≥VAo(1,k)

TAo(1,k)

Rearranging yields

VAo(k+1,|o|) · TAo(1,k) ≥ VAo(1,k) · TAo(k+1,|o|)

Adding VAo(k+1,|o|) · TAo(k+1,|o|) to both sides and rear-ranging, we get

VAo(k+1,|o|)

TAo(k+1,|o|)

≥VAo(1,k) + VAo(k+1,|o|)

TAo(1,k) + TAo(k+1,|o|)

ut

Theorem 2 Given an optimal arrangement of a scene A,for any two adjacent sequence of objects in the arrangementA(i, j) and A(j + 1, k), where i j < k, if there are nei-ther accessibility constraints nor joint occlusions between theobjects in the two sequences (i.e. if the sequences are fromdifferent connected components), then the utility of the for-mer sequence is greater than or equal to the utility of thelatter sequence: U(A(i, j)) ≥ U(A(j + 1, k)).

Proof The proof proceeds similar to the proof of Theorem 1.We create a new arrangement A that is identical to A ex-cept that the two adjacent sequences are swapped:A(i, i+ k − j) = A(j + 1, k) andA(i+ k − j + 1, k) = A(i, j).Amust be a valid arrangementsince we are given that no object in A(i, j) is blocking ac-cess to A(j + 1, k). Then we can compute the differenceE (A)− E (A) to be:

jX

l=i

VA(l)

VOseen

· TA(j+1,k)

−kX

l=j+1

VA(l)

VOseen

· TA(i,j)

Since A is optimal, E(A)− E(A) ≥ 0. After cancelingout the common terms and rearranging, we are left with

jP

l=i

VA(l)

TA(i,j)

≥

kP

l=j+1

VA(l)

TA(j+1,k)

Simply, U(A(i, j)) ≥ U(A(j + 1, k)). ut

We state a lemma and leave its proof to future work.

Lemma 2 The relative ordering of objects in the optimalarrangement of a connected component will be preserved inthe optimal ordering for the complete scene. Formally, if A

c

is the optimal arrangement for a connected component c, andA

o is the optimal arrangement of o, such that c o, then

i < j =) A−1o (A

c(i)) < A−1o (A

c(j))

where 1 i, j |c|, and A−1o returns the index of an object

in the arrangement Ao.

Finally we can prove that the connected components al-gorithm is optimal.

Theorem 3 Let’s say we are given m connected componentsof a set of objects, o, and we are also given an optimal ar-rangement for each connected component Aci for i = 1, . . . ,m.Let’s say we computed the utility of all sequences of objectsin the form Aci(1, j) for all i = 1, . . . ,m and j = 1, . . . , |ci|,and found Ac(1, j) to have the maximum utility. Then anoptimal arrangement for o starts with Ac(1, j).

Proof Assume that the optimal arrangement Ao does not

start with Ac(1, j). We will prove that this is not possible.Given an arrangement of o, we can view it as a series

of partitions, where each partition consists of a contiguoussequence of objects from the same connected component. Dueto Lemma 2, each such partition in A

o can be representedas subsequences of the connected component arrangementsAci . In particular, we are interested in two partitions of theoptimal arrangement of o:

Ao = [Ac0(1, j0) . . .Ac(k, l) . . . ]

where c0 is one of the connected components, and 1 j0 |c0|.Ac(k, l) is the partition that includes the object Ac(j),hence k j l. We know that Ac(1, j) has the maximumutility of all the sequences in the form Aci(1, j) where ci isany connected component and j = 1, . . . , |ci|. Then,

U(Ac(1, j)) > U(Ac(1, k − 1)) (6)

and also

U(Ac(1, j)) > U(Ac0(1, j0)) (7)

14 Mehmet R. Dogar et al.

Using Lemma 1 and Eq. 6, we get

U(Ac(k, j)) > U(Ac(1, j))

Then from Eq. 7,

U(Ac(k, j)) > U(Ac0(1, j0)) (8)

Considering the utilities of all the partitions in Ao up to

Ac(k, l), we know that they should be ordered in descreasingorder of utility and be larger than Ac(k, j) (Theorem 2):

U(Ac0(1, j0)) > ... > U(Ac(k, j))

which contradicts Eq. 8. ut

Acknowledgements Special thanks to the members of thePersonal Robotics Lab at Carnegie Mellon University for in-sightful comments and discussions. This material is basedupon work supported by NSF-IIS-0916557 and NSF-EEC-0540865.

References

Anand A, Koppula HS, Joachims T, Saxena A (2013)Contextually guided semantic labeling and search forthree-dimensional point clouds. International Journal ofRobotics Research 32(1):19–34

Baird D (1995) Experimentation: An introduction to mea-surement theory and experiment design. Prentice-Hall(Englewood Cliffs, NJ)

Ben-Shahar O, Rivlin E (1998) Practical pushing planningfor rearrangement tasks. IEEE Transactions on Roboticsand Automation 14:549–565

van den Berg JP, Stilman M, Kuffner J, Lin MC, Manocha D(2008) Path planning among movable obstacles: A proba-bilistically complete approach. In: In International Work-shop on the Algorithmic Foundations of Robotics, pp 599–614

de Berg M, Cheong O, van Kreveld M, Overmars M (2008)Computational Geometry: Algorithms and Applications.Springer

Chen P, Hwang Y (1991) Practical path planning amongmovable obstacles. In: IEEE International Conference onRobotics and Automation

Diankov R, Kuffner J (2008) OpenRAVE: A Planning Archi-tecture for Autonomous Robotics. Tech. Rep. CMU-RI-TR-08-34, Robotics Institute

Dogar M, Srinivasa S (2012) A planning framework for non-prehensile manipulation under clutter and uncertainty.Autonomous Robots 33(3):217–236

Espinoza J, Sarmiento A, Murrieta-Cid R, Hutchinson S(2011) Motion Planning Strategy for Finding an Objectwith a Mobile Manipulator in Three-Dimensional Envi-ronments. Advanced Robotics

Gupta M, Sukhatme G (2012) Interactive perception in clut-ter. In: The RSS 2012 Workshop on Robots in Clutter:Manipulation, Perception and Navigation in Human En-vironments

Hart PE, Nilsson NJ, Raphael B (1968) A formal basisfor the heuristic determination of minimum cost paths.IEEE Transactions On Systems Science And Cybernetics4(2):100–107

Hauser K (2012) The minimum constraint removal problemwith three robotics applications. In: Workshop on the Al-gorithmic Foundations of Robotics (WAFR)

van Hoof H, Kroemer O, Ben Amor H, Peters J (2012) Max-imally informative interaction learning for scene explo-ration. In: IEEE/RSJ International Conference on Intel-ligent Robots and Systems, IEEE, pp 5152–5158

Hopcroft J, Tarjan R (1973) Algorithm 447: efficient algo-rithms for graph manipulation. Communications of theACM 16(6)

Kaelbling L, Lozano-Perez T (2012) Unifying perception, es-timation and action for mobile manipulation via beliefspace planning. In: IEEE International Conference onRobotics and Automation

Kaelbling LP, Littman ML, Moore AW (1996) Reinforcementlearning: A survey. Journal of Artificial Intelligence Re-search

Kollar T, Roy N (2009) Utilizing object-object and object-scene context when planning to find things. In: IEEE In-ternational Conference on Robotics and Automation

Ma J, Chung TH, Burdick J (2011) A probabilistic frameworkfor object search with 6-dof pose estimation. InternationalJournal of Robotics Research 30(10):1209–1228

Martinez M, Collet A, Srinivasa S (2010) MOPED: A Scal-able and Low Latency Object Recognition and Pose Es-timation System. In: IEEE International Conference onRobotics and Automation

Moizumi K, Cybenko G (2001) The traveling agent problem.Mathematics of Control, Signals, and Systems (MCSS)14(3):213–232

Ota J (2009) Rearrangement planning of multiple movableobjects by a mobile robot. Advanced Robotics pp 1–18

Overmars MH, Nieuwenhuisen D, Nieuwenhuisen D, FrankA, Overmars H (2006) An effective framework for pathplanning amidst movable obstacles. In: In InternationalWorkshop on the Algorithmic Foundations of Robotics

Shubina K, Tsotsos J (2010) Visual search for an object in a3d environment using a mobile robot. Computer Visionand Image Understanding pp 535–547

Sjo K, Lopez D, Paul C, Jensfelt P, Kragic D (2009) Objectsearch and localization for an indoor mobile robot. Jour-nal of Computing and Information Technology pp 67–80

Srinivasa S, Berenson D, Cakmak M, Collet Romea A, DogarM, Dragan A, Knepper RA, Niemueller TD, StrabalaK, Vandeweghe JM, Ziegler J (2012) Herb 2.0: Lessonslearned from developing a mobile manipulator for thehome. Proceedings of the IEEE 100(8):1–19

Stilman M, Schamburek JU, Kuffner J, Asfour T (2007) Ma-nipulation planning among movable obstacles. In: IEEEInternational Conference on Robotics and Automation

Wilfong G (1988) Motion planning in the presence of movableobstacles. In: Proceedings of the Fourth Annual Sympo-sium on Computational Geometry, pp 279–288

Wong LL, Kaelbling LP, Lozano-Perez T (2013)Manipulation-based active search for occluded ob-jects. In: IEEE International Conference on Roboticsand Automation

Ye Y, Tsotsos J (1995) Where to look next in 3d object search.In: IEEE International Symposium on Computer Vision

Ye Y, Tsotsos J (1999) Sensor planning for 3d object search.Computer Vision and Image Understanding 73(2):145–168