StarL for Programming Reliable Robotic Networks

Adam Zimmerman and Sayan MitraDepartment of Electrical and Computer Engineering

University of Illinois at Urbana-ChampaignEmail: {zimmrmn3, mitras}@illinois.edu

Abstract—This paper presents the StarL programmingparadigm, its software embodiment and applications. StarLis designed to simplify the process of writing and rea-soning about reliable distributed robotics applications. Itprovides a collection of building block functions withwell-defined interfaces and precise guarantees. Composingthese functions, it is possible to write more sophisticatedfunctions and applications which amenable to assume-guarantee style reasoning. StarL is platform independentand can be used in conjunction with any mobile roboticsystem and communication channel. Design choices madein the current Android/Java based open source implemen-tation are discussed along with three exemplar applica-tions: distributed search, geocast, and distributed painting.It is illustrated how application-level safety guarantees canbe obtained from the properties of the building-blocksand certain environmental assumptions. Experimental re-sults establish the feasibility of the StarL approach andshow that the performance of an application scales in theexpected manner with increasing number of participatingrobots.

I. INTRODUCTION

The challenge of reliably programming distributedsystems becomes aggrevated when the computers in-teract through multiple physical channels. Considerprogramming a distributed search application for aswarm. The robots should collaboratively cover a col-lection of rooms in a building in an attempt to findtargets. For this relatively simple task, robots need toexchange messages over a wireless network about therooms that have been covered and somehow decidethe assignment of uncovered rooms to robots. Theyalso need to plan their paths avoiding each other andobstacles in a shared physical space. The interactionof the subroutines handling each of these differentsubtasks can quickly overwhelm any debugging orverification effort.

Lessons from software engineering provide a simplerecipe for managing this problems: abstraction andmodularity. A complex (software) system is built byassembling simpler building-blocks or modules withwell-defined interfaces and properties. Abstractions of amodule hide its implementation details and provide asimpler description of its relevant properties. Thus, in-dividual building blocks can be unit-tested or verified

against their stated properties independently. System-level properties can be derived from the properties ofthe units using assume-guarantee style reasoning [13].For maintainability, a unit can replaced by anotherunit without perturbing the overall system, as long asthe latter conforms same interface and the abstractionprovided by the former. Finally, modular design leadsto reuse. Software development environments like Mi-crosoft’s .NET [3] provide a support modular applica-tion development by providing a common platform onwhich developers can build applications with sharedinfrastructure.

Currently there are no frameworks or tools support-ing analogous modular design, implementation, andverification of distributed robotic systems. Several re-search laboratories and companies (for example, KivaSystems [2]) around the globe focus on developingparticular distributed algorithms and applications. Forexample, there is a large body of work on formationcontrol [8], [16], [14], coverage [7], [15], searching,payload delivery, and distributed construction, amongothers (further discussion of this is in Section VI). Inimplementing these algorithms, each group uses itsown specific, home-grown and typically proprietaryhardware and software architecture to implement thealgorithms, with limited scope for reuse and modularreasoning.

We addresses this need by introducing StabilizingRobotics Language (StarL) [18] [19]. StarL is an opensource, modular programming paradigm for devel-oping distributed robotics applications. It providesspecifications and implementations of a number ofbuilding blocks including point-to-point communica-tion, broadcast, leader election, distributed path plan-ning, mutual exclusion, synchronization, and geocast.Each of these building blocks have well-defined in-terfaces and properties and they can be composedto construct more sophisticated building blocks andapplications. Distributed robotic applications can berapidly prototyped and tested by taking advantage ofthese building blocks and the StarL platform’s infras-tructure. Furthermore, since the building blocks havewell-defined assume-guarantee style properties, it ispossible to reason about the properties of high-level

applications. The implementation of StarL is organizedin a stack of four layers and can be ported to differentrobotic hardware by appropriately changing the lowestlayer. An example Java implementation for Android[1] smartphone based robots is presented. StarL alsocomes with its own discrete event simulator which cansimulate instances of StarL applications with hundredsof participating robots.

We provide an overview of the architecture of StarLin Section II. Then we illustrate application devel-opment in StarL with three examples: Geocast, dis-tributed search, and distributed painting (Section III).The modularity and reuse advantage of StarL buildingblocks become apparent in developing these applica-tions. In Section IV we briefly show how (safety) prop-erties of high-level applications can be derived fromthe properties of the building blocks and certain en-vironmental assumptions. A example multi-robot plat-form with iRobot Create robots [5], Android phones,and camera-based indoor positioning system on whichStarL has been used is discussed in Section V-A. InSection V-B experiments with this robotic platformdemonstrate the feasibility of StarL and shows that thetask completion time of a typical application scales inthe expected manner with larger groups of robots.

II. OVERVIEW OF STARL

A. Design Hierarchy

The StarL is organized into a four layer stack (seeFigure 1). Each layer groups together functionalitiesthat serve similar purposes. Interaction between layershappens through well-defined interfaces, allowing forthe implementation of any layer to be modified with-out impacting others. The lowest layer provides basicfunctions, while higher layers build on this to providemore advanced capabilities.

The platform layer interfaces directly with robothardware and communication channels. This layer’spurpose is to (a) send and receive messages overthe communication channels (Section II-B), (b) receiveor generate localization data (Section II-C), (c) issuemotion commands to the robot chassis (Section II-D),and (d) record debug traces (Section II-F). To run StarLon a robot system, the platform layer must be tailoredto interact with the system’s hardware. The platformlayer links the logic layer with the physical systemhardware.

The logic layer is built upon the platform layer. Thatis, all logic layer functionality depends only on themethods exposed by the platform layer’s interface. Itis responsible for message handling, including parsingand validating received packets. Robot motion con-trollers, communication protocols, such as the Simple

Figure 1. StarL Architecture

Acknowledgement Protocol (see Section II-B), are in-cluded in this layer.

The interface layer provides a set of methods usedto pass data in and out of the logic layer. It is an orga-nized collection of all underlying StarL functionality.Through the interface layer, applications may accesseach part of the framework. Only superficial behavioris described in the interface layer. The interface layerwill, for example, track the robots participating inan application’s execution. The interface layer alsomaintains a log file which records all steps takenby an application. This layer specifies the StarL APIand connects the underlying functionality to each APImethod.

The top layer is the application layer. This includesStarL building block functions (Section II-E) as wellas the user applications written using them. The ap-plications access the logic layer methods through theinterface layer, which then uses the platform layer toissue commands to hardware and read sensor data.

B. CommunicationMessages in StarL are directed to a particular appli-

cations using an associated type ID. When sending amessage, a message type ID is attached to the outgoingmessage. To receive messages with of a particular type,a receiver must register itself as a message listener forthat type’s ID.

StarL uses a message acknowledgement protocolcalled Simple Acknowledgement Protocol (SAP) toincrease communication reliability and detect failedtransmissions. SAP attaches a unique sequence numberto outgoing message packets. Upon receiving a packet,a robot replies with an acknowledgement for the mes-sage’s sequence number. If an acknowledgement is notreceived by the sender within a time bound, the senderretransmits with the same sequence number until an

acknowledgement is received or a retransmission limitis reached. If the retransmission limit is reached themessage is reported lost to the application layer. Allreceived packets with duplicate sequence numbers areacknowledged but not redelivered.

It is important to distinguish between packets andmessages. A message is contains data intended forother robots, and a packet is an instance of thatmessage which is transmitted. Individual packets maybe lost, but replacement duplicate packets are re-transmitted to improve the chances of message deliv-ery.

C. LocationStarL contains data structures to hold location in-

formation for participating robots and waypoints inthe environment. These waypoints may be providedby the localization component of the platform layer,or generated in the application layer and stored inthe StarL localization data. Because the localizationdata is available to all application threads throughthe interface layer, it is possible for threads to sharelocations using this structure.

D. Motion ControlThe platform independent motion controller inter-

face takes a destination location and determines theindividual chassis motions necessary to reach the desti-nation. The most basic motion controller implementingthis interface moves in a straight line to the goal, butmore advanced controllers may incorporate collisionavoidance and collaborative path planning. Becauseall robot chassis will have different atomic motioncommands (for example, the iRobot create has com-mands to turn left and right, while a quadcopterhas commands to increase or decrease blade pitch),the inputs to the atomic motion command transmit-ter in the platform layer are left undefined. For thisreason, the motion controller and motion commandtransmitter are intended to be designed together. Werequire the atomic motion controllers to satisfy thefollowing assumption: it states that a robot can movefrom its current position Xi to a given waypoint wwhile staying within bounded distance of the straightline wXi.

Assumption II.1. Consider robot i at point Xi moving topoint w with velocity v. ∃r(v) such that i is never fartherthan r(v) away from the straight line connecting Xi andXj .

E. Building BlocksThe application layer provides a wide collection of

building block functions which are useful for writingapplications for mobile robotic systems. Each function

provides some guarantees under some assumptionsabout the lower layers. In what follows, we describe aset of building blocks.

Leader Election: The leader election function selects aleader from the set of participating agents. All agentsparticipating in an election will either elect the sameleader or no leader at all if the election fails.

Assumption II.2. (a) The set of participants is known toall participants.

(b) For some constant δ > 0, all participants begin electionwithin δ-time of each other.

Proposition 1. (a) If no messages are lost, all agentswill elect the same leader.

(b) If any agent fails to receive any ballot messagesbut receives at least one leader announcementmessage, it will elect the announced leader.

(c) If insufficient ballots are received and no an-nouncement messages are received, the algorithmwill return failure in bounded time.

Currently, one of the implementations of leader elec-tion is based on randomized ballot creation and asecond implementation is based on a version of theBulley algorithm. Implementations of other electionalgorithms could as well be used.

Mutual Exclusion: The mutual exclusion functionmanages a set of permission tokens which are usedfor controlling access to shared resources in distributedapplications. Each token is held by a single robotat a given time and under additional assumptionsa requesting robot eventually obtains the requestedtokens. Under Assumption II.2, the mutual exclusionfunction guarantees the following properties.

Proposition 2. (a) No two robots hold the same tokensimultaneously.

(b) If a robot requests a token, no messages are lostand no robot holds tokens indefinitely, then therequesting robot will eventually receive the token.

(c) If no messages are lost, all robots know the identityof the owner of each token.

Our implementation of mutual exclusion works asfollows: a requesting robot sends a message to the cur-rent token holder. Upon receiving a request message,the token holder adds the requestor to a queue. Uponexiting the critical section, the token holder sends thetoken to the first robot in the queue. The names ofany remaining robots in the queue are sent alongwith the token transfer message. This allows the newtoken holder to continue passing the token to otherrobots requesting entry to the critical section. After thetoken holder sends the token to a requestor, it sends abroadcast message to all robots informing them of the

new token holder. If a non-token holding robot receivesa request message, it will forward that request on tothe proper token owner.

Barrier Synchronization: The synchronization primi-tive enables all participating robot to start the execu-tion of a function roughly at the same time. The pointin the code at which the robots synchronize is called abarrier. Once a robot reaches a barrier it waits for allthe robots to reach the barrier before it continues withthe execution.

Proposition 3. There exists a platform dependent timeconstat δ, such that if there are no message losses thenfor a given barrier point all robots continue executionfrom that point within δ time of each other.

Here δ is a parameter which depends on the roundtrip delay and the worst case execution time of thesynchronization subroutine.

In our implementation, when a robot reaches a bar-rier point it broadcasts a message containing the IDof that barrier. The robot then periodically checks forreceived synchronization broadcasts containing the IDof the current barrier. Until a synchronization broad-cast for the current barrier has been received from allrobots, the robot will not advance its execution. Thisprimitive is useful for ensuring that all robots begina procedure within bounded time of each other. Forexample, synchronizing before electing a leader willensure that n−1 robots will not time out while waitingfor ballots because 1 robot has not yet begun leaderelection.

F. StarL Simulator and Debugger

One of the tools included with the StarL frameworkis a discrete event simulator which allows applicationsto be tested without a physical robotic platform. Thesimulator features a custom implementation of theplatform layer which directs motion, message, andtrace commands into a coordinating thread referred toas the simulation engine. The simulator can execute anarbitrary number of copies of a StarL application codeto run and interact simultaneously through simulatedmessages and robotic platforms.

The StarL simulator allows a developer to run anapplication under a broad range of conditions and withany number of participating robots. Message delays,message loss rate, clock skews and offsets, and physicalenvironment size are among the tunable simulation pa-rameters. A visualizer displays the current position ofeach agent and can be extended to display additionalapplication specific information (see Figure V-A).

On startup the simulator is provided the StarLapplication to be simulated and a set of simulationparameters. A thread pool is then created with each

simulated robot running on a separate thread. Each ofthese threads may request to sleep for a certain lengthof time during its execution. All StarL applicationsshare a similar design in which main thread routinelysleeps. When this happens, the thread is halted andthe requested sleep duration is passed to a the simu-lation engine. The simulation engine is responsible fortracking each simulated robot’s current execution stateand the current simulated time. When all simulatedrobots have requested to sleep, the engine will advancesimulated time until the next thread is scheduled tobe woken up. The engine will then resume all threadsscheduled to be woken at that time.

We have also developed a tool for debugging StarLapplications. The development platform writes trace orlog files to each smartphones local file system. Thesefiles are automatically synchronized with a cloud stor-age service, providing easy access to all traces filesorganized by execution. An SMT-based tool describedin [12] analyzes these traces to automatically detectviolation of global predicates.

III. APPLICATIONS

In this section we sketch the implementation of fourStarL applications starting from a relatively simplegeocast to a sophisticated distributed search protocol.While the safety properties of the applications hold inspite of message losses, the following assumption isused for obtaining the progress guarantees.

Assumption III.1. Every message that a robot attempts tosend is eventually delivered.

A. Geocast

Function: The Geocast(m,A) application is a StarLbuilding block for a robot to send a message m toother robots in a geographical area A. For a messagem geocast at time t0 on a network of diameter Dand a platform specific time constant δ for the non-blocking Geocast(m,A) function (see Subroutine 1), thefollowing properties hold:

Proposition 4. (a) (Exclusion) Any robot located out-side A during the time interval [t0, t0+δD] will notdeliver m. No robot delivers m after t0 + δD.

(b) (Inclusion) Any robot located within A during thetime [t0, t0 + δD] will deliver m.

For a robot moving in or out of A during the geocastperiod, the message may or may not be delivered.The time constant δ is an upper-bound on the sum ofthe message round trip time (RTT) and the worst-caseexecution time of the Subroutine.

Implementation: To geocast a message m to an areaA, a robot broadcasts a special message Geo(m,A).The pseudocode implementing the delivery of geocastmessages is shown in Subroutine 1. A robot uponreceiving Geo(m,A) for the first time, rebroadcasts itand if it is located within A then delivers it.

Subroutine 1: Receive Geo(m,A)

1 if Relayed ∩m = ∅ then2 StarL.Broadcast(Geo(m,A));3 if Xi ∈ A then4 StarL.DeliverToSelf(m);5 end6 Relayed = Relayed ∪m;7 end

B. Distributed Path PlanningFunction: The distributed path planning (DPP) build-

ing block consists of a RequestPath-ComputePath func-tion pair. It enables a collection of robots to computesafe paths to a set of destinations. Consider a planargraph G = (V,E) with a subset T ⊆ E of target edges.The requirement is for the robots to collaborativelytraverse (cover) every traget edge in T , while travelingalong E and avoiding collisions.

To compactly state the properties of DPP, we firstintroduce some terms and notations. Xi is the currentposition of robot i. A waypoint sequence for robot i,Wi = {wi1, wi2, ..., wik} is a path in G. ReachTube(Wi,R)is the subset of the 2D plane such that for every pointin it, there is some point on Wi that is at most Rdistance away. Let FE(t) denote the subset of free edges,that is, the set of target edges T which have never beenassigned to any robot upto time t. Initially, FE(0) = T .A coordinator robot is elected (see Section II-E) andupon receiving a request from a participating robot itcomputes a (possibly empty) waypoint sequence for itin a manner that achieves the following properties.

Proposition 5. (a) (Safety) No two robots followingthe assigned waypoint ever collide.

(b) (Progress) At the time of a request from robot i,if there exists a free edge e ∈ FE(t) such thatthere exists a safe path between Xi and e, then thecomputed Wi will contain at least one free edge.

The computed waypoint sequence Wi is empty onlyif there are no safe paths from Xi to any of the freeedges.

Implementation: ComputePath uses an elected coor-dinator robot for target edge assignments and formaintaining safe separations. For safety, the coor-dinator must make assignments such that no two

robots are ever closer than a safety distance rs. Tothis end it maintains a set, called Unsafe , which isan overapproximation of all the points in the planewhere the robots could be. Initially, Unsafe(t0) =∪i∈IDReachTube(Xi(t0),R), that is, the union of theR-discs around each robot’s initial location. When arobot i requests a new assignment, after completingwaypoint sequence Wi, the coordinator first removesReachTube(Wi,R) and adds ReachTube(Xi,R) to Unsafe .Then, if a safe path W ′i can be found to a free edge,it adds ReachTube(W′i ,R) to Unsafe . This together withAssumption II.1 and an appropriately large choice ofR guarantees the following invariant:

Proposition 6. For any two robots i, j, Unsafe alwayscontains ReachTube(Xi,R), ReachTube(Xj,R) and ||Xi−Xj || ≥ rs.

The actual choice of the path W ′i is con-trolled by a parameter H which limits its maxi-mum length (more on this below). The subroutineComputePath(FE,E,Unsafe,Xi) computes a new (possi-bly empty ⊥) assignment Wi based on the current freeedges, available edges, unsafe region, and requestingrobot position such that(a) ReachTube(Wi,R) is disjoint with Unsafe(b) There is at least one j in the sequence such that{wij , wi(j+1)} is an edge in FE

(c) The length of Wi is at most H1

We state these properties below for future use:

Lemma 7. When each assignment Wi is made,ReachTube(Wi,R) is disjoint from Unsafe.

A requesting robot receiving an empty assign-ment remains static (within ReachTube(Xi,R)) and re-quests again after a waiting period. Upon receiving anonempty request Wi, a robot traverses the path andperiodically sends Clear(wik ,wik+1 ) messages to thecoordinator. This makes the coordinator safely removeReachTube({wik,wik+1},R) from Unsafe set, and thusfrees up more space for safe paths.

The ComputePath subroutine presented in 3 takesthe following steps to compute such an assignment:first, it is determined if a safe path Tv exists in Ebetween Xi and each safe vertex v in the vertices ofFE. If no path is found, v is assumed to be currentlyunreachable by a safe path and is removed from con-sideration. Among the feasible vertices in FE to whichsafe paths exist, one is chosen and Dv is assigned

1One case which ComputePath must account for is the following:∀e ∈ FE, |e| > H . By the above definition of ComputePath, noassignment is admissible in this case. ComputePath may resolve thisby either breaking edges longer than H into segments of maximumsize H , or temporarily violating the maximum path length constraintand assigning these long edges to robots.

Subroutine 2: Coordinator receivesRequestPath(Xi, i)

1 Unsafe =Unsafe− ReachTube(Wi,R) + ReachTube({Xi},R);

2 if (termination condition met) then3 return DONE;4 else5 Wi = ComputePath(FE,Unsafe,Xi);6 FE = FE −Wi;7 Unsafe = Unsafe+ ReachTube(Wi,R);8 StarL.Send(i,Wi);9 end

Subroutine 3: ComputePath(FE,E,Unsafe,Xi)

1 foreach v ∈ vertices(FE) do2 if PathPlanner(E,Xi, v,Unsafe) 6= ⊥ then3 Tv = PathPlanner(E,Xi, v,Unsafe);4 end5 end6 if T = ⊥ then7 return ⊥;8 else9 Dv = pick any v, concatenate Tv with the

largest (up to H − |Tv| length) nonemptycontiguous subgraph of FE starting at v;

10 return {Tv, Dv}11 end

to be the largest safe contiguous subgraph of FEreachable from v. The concatenation of Tv and Dv isreturned as the assignment Wi. Various heuristics maybe used in choosing a feasible v for optimizing differentperformance metrics. For example, the length of targetedges present in an assignment could maximized forminimizing the time spent traveling. Longer pathsalso expand Unsafe , posing more constraints on futureassignments.

We remark that DPP is not deadlock free even whenthere are free edges. Consider an edge in T that iswithin R distance of two robots. Since the edge in-tersects with Unsafe it cannot be assigned. However,deadlocks are detectable and can be resolved usingsymmetry-breaking strategies.

Subroutine 4: Coordinator receivesClear(wi(j−1), wij)

1 Unsafe = Unsafe− ReachTube({wi(j−1),wij},R);

C. Distributed Search

Function: Distributed search uses a swarm ofcamera-equipped robot to search for a target in acollection of rooms. In our robotic platform describedin Section V-A, each smartphone uses its camera tosearch for a brightly colored ball when passing througheach room. The rooms and hallways connecting themdefine the set of edges of the graph G. A room issearched when its target edge is traversed by a robot.We assume that the number of robots and the topologyof G is such that a safe path always exists between anypair of rooms. The key property of distributed searchis the following:

Proposition 8. All rooms are eventually searched.

Implementation: Distributed search is implementedusing DPP. The target edges for the rooms define theset T of target edges in the graph. Until the target isfound, DPP’s coordinator will assign safe paths to thesearching robots that lead to unsearched rooms. Oncea robot searches a room unsuccessfully, it makes a newpath request. Once the target is found, the coordinatorwill cease making new assignments and will insteadreply to requests with DONE .

Subroutine 5: Distributed search participant

1 repeat2 StarL.Send(Γ, RequestPath(i,Xi));3 wait until Receive(Assignment(Wi));4 if Wi = ⊥ then5 sleep(tr);6 else if Wi = DONE then7 return;8 else9 for j = 0 to len(Wi) do

10 StarL.GoTo(wij);11 if foundTarget then12 StarL.Broadcast(Found(wij));13 end14 StarL.Send(Γ, Clear(wi(j−1), wij));15 end16 end17 until Wi = DONE;

D. Collaborative Painting

Function: This application enables a collection ofrobots to paint a given picture. The picture is anarbitrary collection of lines in the 2D plane. The linesmay intersect and come arbitrarily close. Robots mustnot collide with each other as they travel, but once aline has been painted, it may be safely traveled over

without disrupting the image. In our robotic platform,the image is painted using light. The smartphonescreen attached to each robot is illuminated as thatrobot travels along an edge in T painting and darkenedwhen the robot is traveling. In a dark room the re-sulting light-painting is captured using long exposurephotography (see Figure V-A).

The provable properties of this application followfrom Propositions 5.

Subroutine 6: Painting participant

1 repeat2 StarL.Send(Γ, RequestPath(i,Xi));3 wait until Receive(Assignment(Wi));4 if Wi = ⊥ then5 sleep(tr);6 else if Wi = DONE then7 return;8 else9 for j = 0 to len(Wi) do

10 if line(Xi, wij) ∈ T then11 EnablePaint();12 else13 DisablePaint();14 end15 StarL.GoTo(wij);16 StarL.Send(Γ, Clear(wi(j−1), wij));17 end18 end19 until Wi = DONE;

Implementation: The DPP is used with G being adense planar graph which contains the painting as asubgraph. The set of target edges T is defined as thelines of the image to be drawn. E includes a densegraph to allow the ComputePath path planner to makeassignments bridging disjoint sections of the image.Robots paint a line by traveling along its correspondingedge in T . The termination condition used by thecoordinator for this application is FE = ⊥, indicatingthat all edges in T from the drawing have either beenpainted or assigned to a robot.

This application has been implemented on therobotic system described in V-A In this implementa-tion, the dense graph added to E is generated on-the-fly by the path planner generating random points.This technique is popularly known as probabilisticroadmaps [10]. Performance results for this applica-tion are discussed in Section V-B.

IV. PROPERTIES OF APPLICATIONS FROM BUILDINGBLOCKS

StarL enables one to formally reason about applica-tion level safety and progress properties from the prop-erties of the building blocks and certain environmentalassumptions. As an illustration, here we present aproof sketch of the safety property of the distributedpath planning subroutine stated in Proposition 6.

Proposition 6. For any two robots i, j, Unsafe alwayscontains ReachTube(Xi,R), ReachTube(Xj,R) and ||Xi −Xj || ≥ rs..

noindent Proof sketch. The proof is by induction onthe length of the execution of the system. Initially, noassignments have been made and Unsafe consists ofreach tubes surrounding each robot’s starting position.As the robots start with minimal separation of rs, theproperty is satisfied.

Consider the request from robot i to the coordinator.Suppose this request and the resulting assignmentmessages are delivered. The coordinator’s computedassignment, Wi, will be disjoint from Unsafe byLemma 7. At this time, Unsafe consists only of reachtubes of radius R surrounding stationary robots. Thus,Wi will never be closer than distance rs from any robot.

Let vmax be the upper bound on any robot’s velocity,B be the maximum robot radius, then by setting R >r(vmax)+B in Subroutine 2 and from Assumption II.1,we know that robot i always remains within distanceR of the straight line defined by Wi when completingan assignment. This places i within ReachTube(Wi,R)at all times.

Now consider the case in which robot i requestsan assignment while at least one other robot hasan assignment in progress. The computed Wi willbe disjoint from all other assignments by Lemma 7and will therefore be safe. All assignment reach tubesare disjoint from each other and Unsafe, making allassignments are safe.

Message losses do not compromise this property.Consider the case where a request message is lost.When a request is sent by i, i is stationary at Xi =wik ∈ Wi. By Lemma 7, no assignment can intersectany stationary robot’s reach tube, preventing any newassignment from intersecting Xi. In the second case, anassignment message to i is lost, and i will again remainstationary at Xi. The coordinator will update Unsafe toinclude the reach tube for the unreceived assignmentWi. Because the first point in any assignment Wi isXi, i’s current position is included in Unsafe and byLemma 7 it will not intersect any later assignments.

V. PLATFORM AND EXPERIMENTS

A. Example Platform Implementation

In this section, we briefly discuss the design of arobotic system used in our laboratory for programmingwith StarL. As mentioned earlier, StarL is platformindependent in the sense that any robotic platformcapable of being controlled by software issued com-mands can support higher-level StarL functions andapplications, that is, once the appropriate platformlayer functions are written. Our robotic platform con-sists of a collection of identical mobile robots. Eachuses an iRobot Create chassis controlled via Bluetoothby an attached Android smartphone [1]. The Androidsmartphones use Wi-Fi to communicate wirelessly andruns the StarL applications. Each chassis is outfittedwith a set of infra-red reflective markers which aretracked by a multi-camera motion capture system. Thiscamera system calculates the 3D position and orienta-tion of each robot in a local coordinate system andbroadcasts it to the robots thus providing localizationinformation.

The StarL platform layer makes extensive use of thetools included in the Android SDK. The SDK provideseasy access to integrated sensors and peripherals us-ing the Java programming language. To control theiRobot Create chassis paired to each phone, a Bluetoothsocket maintains a bidirectional link to the chassis totransmit motion commands and receive any feedback.The StarL platform layer uses a Java UDP socket totransmit and receive message packets. A separate UDPsocket receives localization broadcasts from the infra-red camera system. In total, the Android smartphoneimplementation of StarL comprises just over 7,000 linesof Java code.

B. Experimental Evaluation

The implementation of the collaborative paintingapplication of Section III-D was deployed in our lab-oratory for University of Illinois’ Engineering OpenHouse. For this demonstration 4 robots participatedin successfully creating light-paintings from picturesdrawn by visitors (see Figure V-A Right).

In this paper, we discuss the behavior of the col-laborative painting application with larger number ofrobots. The application was simulated with varyingnumbers of participating robots for two separate inputimages. The first image was constructed to preventdeadlocks from occurring, resulting in every line beingdrawn in each execution. The second image is a collec-tion of random intersecting lines in which deadlock ispossible. Both images are the same size. The executionduration (the time to completely draw an image orreach a deadlock) and the number of assignments

0

0.5

1

1.5

2

2.5

3

35

40

45

50

55

60

65

4 5 6 8 10

Ass

ign

me

nts

pe

r R

ob

ot

Seco

nd

s

Robots in Execution Execution Duration Assignments per robot

Figure 3. Collaborative painting with no deadlocks possible

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

55

60

65

70

75

80

85

90

95

4 5 6 8 10

Ass

ign

me

nts

pe

r R

ob

ot

Seco

nd

s

Robots in Execution Execution Duration Assignments per robot

Figure 4. Collaborative painting with deadlocks

made per robot were averaged over five simulationsfor each execution size. For consistency, the startingpositions of each robot were fixed in the environmentand the same robot acted as the coordinator in each ex-ecution. The value of variable H remained unchangedin each execution, causing all assignments to be ofapproximately equal length.

As seen in Figure 3, the completion time for thepainting (Execution Duration) falls with increasingnumber of robotic participants. This is expected, eachadditional robot allows more simultaneous assign-ments to be made, completing the image sooner.Because all assignments are of approximately equallength, H , the number of assignments remains roughlyconstant in each trial. This results in the number ofassignments per robot dropping in larger executions.

Figure 4 demonstrates that the results seen in Fig-ure 3 are not qualitatively impacted by image complex-ity and the presence of intersecting lines. Deadlocksdid occur in these simulations, causing the image toremain incomplete. In these experiments, the percent

Figure 2. Left: StarL simulator screenshot. Center: Example platform implementation. Right: Collaborative painting output image

of the image completed fell roughly linearly from 99%with four robots to 90% with ten. This image representsa much more realistic input to the system and theresulting data demonstrates that the DPP algorithmis capable of making significant progress under suchconditions.

VI. RELATED WORK

Distributed robots have recently begin to see indus-trial applications. Perhaps the most notable exampleof this is a commercial warehouse automation productmade by Kiva Systems [2]. This system uses a swarmof mobile robots with centralized coordination to orga-nize and transport materials throughout warehouses.

A number of robotic software frameworks similarto StarL, both open-source and commercial, are avail-able today. None of these frameworks, however, areintended for use in distributed systems. One suchframework, Robot Operating System (ROS) [4], anopen-source robot framework maintained by WillowGarage, is prominently used in research. The mainbenefit presented by these frameworks is the interop-erability each provides; a ROS application is capableof running on any robot which uses ROS.

Researchers developing multi-agent testbeds typi-cally develop customized programs for each individualapplication (demonstration) with limited focus on soft-ware engineering, programmability, and the problemof obtaining guarantees for the implemented system.

There exists a large body of literature on mathemat-ical modeling and analysis of multi-agent systems anddistributed robotic systems and even a brief survey ofthis area is beyond the scope of this paper (See, forexample, [9] [6] [17] [11]).

VII. CONCLUSIONS

Observing that there is a lack of tools supportingmodular design, development, and verification of dis-

tributed robotic systems, in this paper we introducethe StarL platform and its open source implementa-tion [19]. StarL provides specifications and implenta-tions of a number of building block functions. Thesebuilding blocks have well-defined interfaces and prop-erties and they can be composed to construct moresophisticated building blocks and applications whichare amenable to assume-guarantee style reasoning. Weillustrated application development in StarL with threeexamples: Geocast, distributed search, and distributedpainting. The modularity and reuse advantage of StarLbuilding blocks become apparent in developing theseapplications. Experiments with a real robotic platformand a simulator demonstrate the feasibility of theStarL approach and shows that the performance of atypical application (distributed painting) scales in theexpected manner. In the future, we plan on expand-ing the set of building blocks which are available inStarL. For example, we are currently implementing analgorithm for maintaining replicated state machines.Another direction of research is to develop partiallyautomated verification tools for StarL applications.

REFERENCES

[1] Android operating system. http://www.android.com.

[2] Kiva systems. http://www.kivasystems.com.

[3] .NET framework. http://www.microsoft.com/net.

[4] Robot operating system. http://www.ros.org.

[5] irobot corporation, 2009. http://www.irobot.com/.

[6] F. Bullo, J. Cortes, and S. Martinez. Distributed Controlof Robotic Networks. Applied Mathematics Series. 2009.

[7] J. Cortes, , S. Martinez, T. Karatas, and F. Bullo. Cover-age control for mobile sensing networks. IEEE Transac-tions on Robotics and Automation, 20(2):243–255, 2004.

[8] X. Defago and A. Konagaya. Circle formation foroblivious anonymous mobile robots with no commonsense of orientation. In Proc. 2nd Int’l Workshop onPrinciples of Mobile Computing (POMC’02), pages 97–104,Toulouse, France, October 2002. ACM.

[9] S. Gilbert, N. Lynch, S. Mitra, and T. Nolte. Self-stabilizing robot formations over unreliable networks.In ACM Transactions on Autonomous and Adaptive Systems(TAAS), July 2009.

[10] L. Kavraki, P. Svestka, J.-C. Latombe, and M. Over-mars. Probabilistic roadmaps for path planning inhigh-dimensional configuration spaces. Robotics andAutomation, IEEE Transactions on, 12(4):566 –580, aug1996.

[11] J. McLurkin and D. Yamins. Dynamic task assignmentin robot swarms. Proceedings of Robotics: Science andSystems, June, 8, 2005.

[12] A. Z. Parasara Sridhar Duggirala, Taylor T. Johnsonand S. Mitra. Static and dynamic analysis of timeddistributed traces. IEEE Real-Time Systems Symposium(RTSS’12), December 2012.

[13] C. S. Pasareanu, M. B. Dwyer, and M. Huth. Assume-guarantee model checking of software: A comparativecase study. In In Theoretical and Practical Aspects of SPINModel Checking, volume 1680 of Lecture Notes in ComputerScience, pages 168–183. Springer-Verlag, 1999.

[14] G. Prencipe. Corda: Distributed coordination of a set ofautonomous mobile robots. In ERSADS, pages 185–190,May 2001 2001.

[15] M. Schwager, J. McLurkin, and D. Rus. Distributedcoverage control with sensory feedback for networkedrobots. In Robotics: Science and Systems, Philadelphia,Pennsylvania, August 2006. The MIT Press.

[16] I. Suzuki and M. Yamashita. Distributed autonomousmobile robots: Formation of geometric patterns. SIAMJournal of computing, 28(4):1347–1363, 1999.

[17] F. Zhang, B. Grocholsky, V. Kumar, and M. Mintz. Co-operative Control, volume 309 of Lecture Notes in Controland Information Sciences, chapter Cooperative Controlfor Localization of Mobile Sensor Networks. SpringerVerlag, 2004.

[18] A. Zimmerman. Stabilizing robotics programminglanguage, 2011. https://bitbucket.org/hsver/starl-framework.

[19] A. Zimmerman. Starl documentation wiki,2012. https://wiki.cites.uiuc.edu/wiki/display/MitraResearch/StarL.