SUAAVE: Combining Aerial Robots and Wireless Networking

Stephen Camerono Stephen Hailesl Simon Julierl Sally McCleanu

Gerry Parru Niki Trigonio Mohamed Ahmedl Graeme McPhillipsl

Renzo De Nardil Julia Nieu Andrew Symingtono Luke Teacyu

Sonia Waharteo

oUniversity of Oxford lUniversity College London uUniversity of Ulster

Final Draft, February 2010

Abstract

The SUAAVE project is funded by EPSRC under the WINES wireless networking initiativeto consider issues of multiple aerial vehicles communicating and collaborating in performingtasks, and involves teams from University College London, University of Ulster, and the Uni-versity of Oxford. The focus of SUAAVE lies in the creation and control of swarms of UAVsthat are individually autonomous (i.e not under the direct realtime control of a human) butthat collaboratively self-organise: to sense the environment in the most efficient way possible;to respond to node failures; and to report their findings to a base station on the ground. As thefocus is on developing algorithms, rather than rugged hardware, we are using small off-the-shelfelectric quad-rotor vehicles with payloads of a few hundred grams. Although the technology isnot tied to a particular scenario, we are basing our research on a search-and-rescue scenario,whereby a swarm of vehicles are searching for a lost or injured person.

Biographies

Dr Cameron’s research area is in spatial reasoning and robotics. Prof Hailes is the PI forSUAAVE, and his research interests cover all aspects of mobile systems and security. Dr Julieris interested in the problem of developing, maintaining and conveying situation awareness, andespecially in problems which are distributed and mobile. Prof McClean’s background is instatistical modelling, with particular interests in ambient intelligence, and optimisation. ProfParr has many interests, but especially secure telecommunications. Dr Trigoni’s focus is onsensor data networks. Mr Ahmed is a research student specialising in telecommunications.Mr McPhillips and Renzo De Nardi are responsible for keeping our machines aloft, andDr Nie for its safety. Mr Symington is a research student working in the area of mobilesensor networks. Dr Teacy works on trust and distributed reasoning under uncertainty. DrWaharte’s interests include distributed algorithms in a wireless environment.

Contact Details

Dr Stephen Cameron, Oxford University Computing Laboratory, Wolfson Building, ParksRoad, Oxford OX1 3QD. (01865) 273850, stephen.cameron@comlab.ox.ac.uk.

1

1 Introduction

The SUAAVE project is funded by EPSRC un-der the WINES wireless networking initiativeto consider issues of multiple aerial vehiclescommunicating and collaborating in perform-ing tasks, and involves teams from UniversityCollege London, University of Ulster, and theUniversity of Oxford. The project runs fromlate 2008–2012, with focus on the creation andcontrol of swarms of UAVs that are individ-ually autonomous (i.e., not under the directrealtime control of a human) but that collabo-ratively self-organise: to sense the environmentin the most efficient way possible; to respondto node failures; and to report their findings toa base station on the ground.

The novelty of these mobile sensor systemsis that their movement is controlled by fullyautonomous tasking algorithms with two im-portant objectives: first, to increase sensingcoverage to rapidly identify targets; and, sec-ond, to maintain network connectivity to en-able real-time communication between UAVsand ground-based crews. The project has fourmain scientific themes: (i) wireless networkingas applied in a controllable free-space trans-mission environment with three free directionsin which UAVs can move; (ii) control theoryas applied to aerial vehicles, with the intentionof creating truly autonomous agents that canbe tasked but do not need a man-in-the-loopcontrol in real time to operate and communi-cate; (iii) artificial intelligence and optimisa-tion theory as applied to a real search prob-lem; (iv) data fusion from multiple, possiblyheterogeneous airborne sensors as applied toconstruct and present accurate information tosituation commanders. The main experimentswill be based on a simplified search-and-rescuescenario, with the idea that the UAVs will co-operate to find a ‘victim’ in an unstructuredoutdoor setting.

2 Hardware & Software

The hardware platforms we are using are off-the-shelf, electrically-powered quad-rotors; anumber of such platforms have become com-mercially available over the last few yearswith similar characteristics and we are pur-chasing a number from Ascending Technolo-gies (www.asctec.de). Such a vehicle has theadvantage of being relatively easy to prepare,control and fly and has a payload of a fewhundred grams. However, the disadvantageover petrol-powered machines being their en-durance — typically a few tens of minutes athigh load. However as an experimental plat-form long endurance is not a priority for us.

The original flying machine has the abilityto reach autonomously GPS waypoints speci-fied by the user but does not provide the com-putation and communication abilities neededfor our research. We redesigned the centralstructure of the quadrotor to provide sufficientroom for our computer board. As visible inFigure 1 the design is based on a central car-bon fibre foam core plate on top of which ismounted the original UAV electronics. Thishosts the inertial sensors1 and the two ARM7processors that run the state estimation andthe FCS (flight control system) algorithms.

Under the central plate we installed a 1.6GHz Intel Atom board2 with 1GB of ram, a16GB SSD, two 802.11a/b/g/n wireless cardsand the necessary power stabilization circuitry.A 1.3Mpix USB camera3 and the 2100mAhbattery are mounted outside the bottom en-closure. The on-board PC and the FCS areconnected through a serial link to exchangecommands and flight data. The bottom plas-

1The sensor suite include three gyros, a triaxial ac-celerometer, three magnetic sensors and a pressure sen-sor, in addition to a GPS receiver.

2MSM200X manufactured by Digital-Logic AG.3Chameleon manufactured by Point Grey Research,

Inc.

2

Figure 1: UAV platform: RF safety module (a); GPS (b); battery (c); wifi antennas (d); camera(e); base enclosure containing the PC (f); flight control system (g).

tic enclosure was realized using a 3D printingprocess and has a grid structure in order tobe light and allow sufficient airflow for the in-ternal electronics. Empirical testing revealedthat the computer board generates EMF ra-diation at levels and frequencies that heavilyinterferes with the GPS unit placed at the topof the UAV. It was therefore necessary to coatthe bottom enclosure with multiple layers ofconductive paint in order to ensure the neces-sary shielding.

After the modification the total weight of theplatform was raised to 850g; to compensatethe original propellers were substituted withlarger, stiffer and more efficient 9 inch threeblade propellers which provide more lift. Theretrofitted platform has currently a flight en-durance of about 10 minutes with PC and cam-era turned on.

Safety is a primary aspect in our project es-pecially when testing novel control and coordi-nation strategies; for this reason we equippedour UAV with a custom designed RF safetymodule completely independent from the on-

board computer. This module automaticallylands the vehicle when out of range from thebase station, but also allows to remotely landthe UAV or even to cut off the power to therotors in the eventuality of the on-board PCbecoming unresponsive. The overall system ar-chitecture is shown in Figure 2.

2.1 Software architecure

As mentioned in the previous section, thereare several microprocessors on board the UAV,some of which provide autopilot functional-ity while others ensure the safety of the plat-form. However, we strictly do not run anyproject code directly on these microproces-sors, as interfering with their normal opera-tion might cause hardware failure, which is asafety risk to both the platform and the re-searchers. In stead, the code runs on the IntelAtom board, which has significantly more com-putational and memory resources. This boardhas a single serial link that connects it to theautopilot board, permitting us to send control

3

!Figure 2: Overall System Architecture

instructions and receive low level sensor data.The software architecture that we chose was

centred around the concept of modularity. Ourgoal was to have many distinct modules run-ning as separate processes, each being respon-sible for a single operational component of thesystem. Not only does this make collaborativeproject development significantly easier, but italso makes the system robust to failure. Forexample, if the high-level navigation code ter-minates unexpectedly, the UAV is still capa-ble of performing a software landing since theremainder of the system remains operational.Our base system contains the following basemodules, which are core to the reliable opera-tion of the UAV:

1. Control Arbitrates communication withthe autopilot and safety systems over se-rial connections.

2. Safety Monitors the messages sent be-

tween all system modules. From thesemessages it infers the system state. Inthe case where the system switches to anundesirable state, it assumes control andinstructs the Control module to fly to asafe zone and land.

3. Black-box - Records all inter-modulecommunication for the purposes of debug-ging.

4. WaypointFollower Parses a list of GPSwaypoints and communicates with thecontrol module to have the UAV fly fromwaypoint to waypoint, idling for a speci-fied period.

Although the modules are operationally dis-tinct, they must be able to communicate witheach other. It is clear that, for example, theWaypointFollower module must communicateperiodically with the Control module to issue

4

actuation instructions. This implies that allmodules must subscribe to some messaging-oriented (MOM) middleware that provides amechanism for inter-process communication.A number of MOM frameworks already exist,such as RCS4, The MOOS5 and MQ4CPP6.Our goal is to keep the system extensible aspossible, so we created an implementation-agnostic interface for our software modulesthat bind it to the MOM, while also defininga strict message format for modules to adopt.This allows us to switch to another MOM whenthe need arises without changing any modulecode.

For the time being we have chosen to useThe MOOS, as it is straightforward to use andappears to have been designed with robotic ap-plications in mind. The MOOS is based on apublish/subscribe system that operates over aTCP/IP network. When a module (MOOS-App) initialises it first binds to a specific mes-saging database (MOOSDB) and requests tobe notified when certain message types are re-ceived by the MOOSDB. As such, the mes-saging service operates in a star-like topology,which is shown in Figure 3.

Figure 3: The MOOS architecture

In practice, each hardware unit in the net-work (either a UAV or a ground station) runsa single MOOSDB. It is clear that we can-

4http://www.isd.mel.nist.gov/projects/rcs/5http://www.robots.ox.ac.uk/ mobile6http://www.sixtyfourbit.org/mq4cpp.htm

not use one MOOSDB for the entire network,since this makes the fundamental assumptionof perfect network connectivity. This is sel-dom true for wireless networks and would causethe messaging in disconnected hardware unitsto break down. The MOOS also provides abridging mechanism (MOOSBridge) that al-lows multiple MOOSDBs to communicate witheach other and, hence, forward messages be-tween modules running on separate hardwareunits. However, this works on a broadcast-likemechanism, which is likely to scale poorly inmulti-hop wireless networks.

3 Communications

The UAV platform’s agility and ease of de-ployment has the potential to turn theminto mobile and robust communication plat-forms. While a number of papers havestudied the problems of network connectiv-ity maintenance and quality of service, in-cluding [BRS04, BPDKHO09, CDB04, HSL06,PBGY08, RZ07], they have often been theo-retical in their approach to the problem space,and make stringent assumptions about the op-erating environment or the state of the net-work. For example, common assumptions in-clude accurate channel models and symmetriclinks, known workloads, static locations or sim-plistic movement models and so forth. Thoughthese assumptions enable us reduce the com-plexity of the systems under investigation andprovide tractable solutions that focus on spe-cific questions, their results cannot be takenand applied directly to real operational plat-forms.

The SUAAVE project aims to help bridgethe gap between theory and practise and ex-tend the state of the art in two ways. Firstly,we will perform a series of measurements tar-geted at:

• Generating 802.11 performance data

5

based on in-air measurements and realworkloads.

• Devising new models to characterise theperformance of 802.11 on UAV, subject toscenario details.

• Comparing proposed models of 802.11characteristics and models with real dataand updating them.

And secondly, the application of Ad-Hoc andMesh networking to provide a communicationplatform for UAV swarms, as opposed to thetraditional centralised communication models.In particular, this involves:

• Ruggedising existing Ad-Hoc and Meshprotocols which often make stringent de-mands on the mobility and link failurestatistics that can be associated withnodes, to function in this space.

• Applying the derived measurement mod-els to guide the behaviour of UAVs whenperforming network centric tasks.

• Developping new algorithms to supportQoS constrained services, such as sensordata collection and video streaming.

4 Control

We require an integrated approach, where byeach UAV takes into account the limitations oftheir resources and the current state of theirpeers when pursuing their objectives. For in-stance, a UAV should not blindly follow a pathtoward its target, if that path takes it througha prohibited flight zone, or leaves it with in-sufficient battery power to return to its homebase. Other examples could be restrictionsor limits on the amount of on-board memoryto capture locally acquired sensor data whencommunications are not available. Instead,it should consider alternative actions, such asgiving up its objective or asking for assistance

from other UAVs. For this reason, it is es-sential that any such decision process shouldoperate within the bounds and constraints ofa wider safety management protocol (SMP)for ensuring none of the risks mentioned inthe previous section lead to real-life disasters.Moreover, for such an SMP to be verifiablyimplemented in a safe and reliable manner, itmust eliminate these risks in the simplest waypossible, without placing undue restrictions onthe high level decision processes that we wishto develop. We propose a basic safety archi-tecture for the SUAAVE project, to mitigatethe risks associated with operating UAVs in,or near, publicly accessible areas. In particu-lar, we wish to avoid the risks of personal in-jury (both the UAV operators and the generalpublic), damage to property through collisions,and any legal consequences due to unplannedlanding of UAVs on private property.

4.1 Safety Management Protocol

(SMP) Function

Phases of Operation To ensure safe flight,each UAV should perform a number of self-diagnostics and safety checks, not only at thestart of each mission, but also periodically dur-ing normal flight. Moreover, should any ofthese checks fail, it is essential that we haveclear procedures in place to either recover fromfailure, or to abort flight in the safest possibleway. With this in mind, we propose that, fromthe moment it is switched on until the end ofits mission, each UAV should move throughfive phases of operation:

1. Pre-flight bootstrap, during which theUAV performs initial diagnostic checksbefore taking to the air;

2. In-flight self diagnostics, where by theUAV makes further checks that must beperformed during flight;

3. Operation, in which the UAV actively

6

pursues its high-level goals;4. Recovery, in which the UAV attempts to

recover from any detected error; and5. Abort, through which the UAV attempts

to land as safely as possible, following andunrecoverable error.

The conditions that lead to transitions betweenthese phases are illustrated by the state dia-gram in Figure 4, and in more detail by theactivity diagram in Figure 5. In particular, aUAV can only reach the operation phase if allinitial checks have completed successfully, andwill only remain there so long as ongoing self-diagnostics can verify that the system is func-tioning normally. If this is not case, the UAVcan either attempt to recover from failure, orabort its mission entirely.

!Figure 4: State transition diagram for thephases of operation

The following subsections describe each ofthese phases in more detail.

Pre-Flight Bootstrap When a UAV is firstswitched on, it should ensure that all resources

!

Figure 5: Activity diagram for the phases ofoperation

necessary for safe flight are present and inworking condition. As far as possible, thisshould be done before the craft becomes air-borne. In particular, while on the ground, eachUAV should ensure that

• A communication link can be establishedwith a manned base station, possibly us-ing a routing protocol and other UAVs;

• The GPS location system is active andreporting that the UAV is located at itsdesignated home; and

• All onboard sensors are available and canrespond to control signals; Only if theabove checks have been completed suc-cessfully, should the UAV proceed to thefollowing phases of operation. Otherwise,the only advisable course of action is theabort the mission immediately and seekhuman intervention.

In-Flight Self Diagnostics Once the abovechecks have been passed, and assuming theUAV has been assigned a task that requiresflight, it should perform any remaining di-agnostics that must be performed in flight.

7

Specifically, to ensure that the GPS system andhelicopter rotators are operating normally, wepropose that the following three steps are per-formed in order7.

1. The UAV should raise to a height of 1–2 metres of the ground and hold its po-sition for 45–120 seconds. Where GPSis available this should allow GPS Lock.The location should be verified againstthe reported GPS location, and ideallysome other sensor. For example, if a setof red dots are placed on the ground, itshould be possible to verify location us-ing onboard cameras.

2. The UAV should attempt to perform afull rotation, and verify this ability usingonboard sensors.

3. The UAV should move a set distance in6 directions: up, down, north, south, eastand west. Again, success should be ver-ified using the GPS, and other on boardsensors.

In addition to these tests, many of the diag-nostics performed in the pre-flight bootstrapcould be repeated periodically during flight.Together, these checks should ensure that acomplete set of system control and status vari-ables are continually monitored to ensure safeoperation. For example, these could includeflight and other control variables, communica-tion links, sensor, CPU and other hardware re-sources (see Figure 5).

Operation and Monitoring Having per-form all initial checks successfully a UAV canthen proceed with the mission assigned to it.With reference to the software architecture de-scribed, this means that the safety layer and

7One possible problem with these proposed steps isthat the GPS may also play an active role in direct-ing these maneuvers. If this is the case, then checkingthe movements for consistency with the GPS may havelimited value.

other APIs can dynamically accept tasks fromthe application layer, while continuing to mon-itor the status of critical resources (see Fig-ure 5). In general, these task requests couldtake a variety of forms, including communica-tion with a base station or other UAVs, re-quests for resource status information or toperform specific manoeuvres. The precise formthat requests from the application layer maytake will depend on both the applications re-quirements, and the ease with which theirsafety can be verified. For example, the eas-iest way to allow the application layer to spec-ify movement is through a stream of GPS way-points. For many high-level decision processes,this level of detail may be sufficient, and itis also straightforward to verify such requestsagainst the SMP. It is also worth mention-ing that, as far as possible, the applicationlayer should use available status informationto avoid making decisions that would lead tobreaches of the SMP. In any event, it is duringthis phase of operation that the safety layershould guard against risky operation, by deny-ing requests from the application layer thatwould violate the SFP. Most importantly, thismeans that requests to move to a way pointoutside the safe flight zone should be denied,along with any course that risks collision withknown objects or other UAV flight paths. Re-quests to move beyond the range of safe land-ing zones (given current battery life) are alsoprime candidates for service denial. In additionto denial of service requests, further precau-tions could be taken by periodically repeatingsome of the checks from the previous phases.The precise timing of each check could be pre-specified by configuration parameters, or couldbe a matter of some more detailed investiga-tion. In any case, choosing the frequency ofeach check is a trade-off between its cost interms of resources used, and the benefit interms of risk minimisation.

8

Recovery and Abort Sequence In theevent that something does go wrong (some es-sential resource becomes inoperative, or theapplication layer becomes unresponsive) thena UAV may attempt to recover from the fail-ure, or execute an abort sequence to land theUAV as safely as possible. For example, recov-ery or abort procedures should be initiated inthe following scenarios:

1. The UAV looses its communication linkto a manned base station8

2. The application layer crashes or becomesunresponsive

3. The UAV’s battery level becomes danger-ously low

4. The UAV receives a manual abort signalfrom a manned base station

5. The UAV CPU develops unrecoverable er-rors

Precisely which, if any, such failures can berecovered from, without the need to abort op-eration, is up for debate. However, the mostlikely candidates involve communication. Forinstance, if a constant communication link isdeemed necessary (for example, to receive amanual abort signal) then the UAV may movein the direction of a known transmitter to at-tempt to re-establish communication. Poten-tially, each type of failure could have a numberof recovery procedures specified that are at-tempted until one succeeds or they all fail. Formany types of failures, however, the only sensi-ble procedure may be to abort operation com-pletely. As with recovery procedures, each typeof failure may result in a different set of abortprocedures being attempted in order, until one

8In theory, we may wish to explore scenarios whereUAVs can leave communication range to assume com-pletely autonomous control. However, during testflights, we shall maintain a consistent link with amanned base station for safety considerations. Loss ofcommunication for test purposes will be simulated inthe application layer.

succeeds. In particular, the following four rou-tines are applicable in all cases, and could beattempted in the stated order to minimise thecost of the failure.

Return Home In the first instance, theUAV should assess whether it has sufficient re-sources to return to its designated home loca-tion for convenient recovery, and do so if pos-sible.

Land at Alternative Safe Landing ZoneIf returning home is not possible, then the UAVshould attempt to land at an alternative des-ignated Safe Landing Zone.

Use Sensors to identify Safe LandingZone If a designated landing zone cannotbe reached (due to insufficient resources) oridentified (due to GPS failure) alternative sen-sors may be used to identify a safe landingzone. For example, this could be achieved us-ing known landmarks or other visual cues, suchas the colour of the ground.

Land Immediately by Controlled De-scent If all else fails, and assuming flight con-trols are still operational, the UAV should at-tempt to land by controlled descent. The rateof descent should be chosen so that anyone be-neath the UAV has sufficient time to identifyand avoid it.

5 Search operations with

smart UAVs

The capacity to search and explore a previ-ously unknown environment is a fundamentaltask for a number for autonomous systems,and swarms of UAV platforms provide a uniqueway to explore an large and difficult to accessterrains. However, their deployment, control

9

and coordination is not a trivial task. Con-trollers must account for numerous condition,including energy limitations and sensitivity toenvironmental hazards (in the form of natu-ral obstacles or wind). When multiple UAVsare used, the complexity of coordination meansthat they are normally flown in a fixed forma-tion relative to one another at a fixed altitudeabove the ground.

One way to mitigate these limitations is toautomate the operations of the UAVs. If theUAVs can perform in-flight collaborations andself-organise, they offer a possibility to opti-mise their strategies to sense the environmentin the more efficient manner possible.

When multiple UAVs are deployed, the sen-sory data they collect can be shared and fusedto generate a complete picture of the environ-ment — which can in turn guide the searchprocess. This task is all the more challengingas any solution that will be proposed needs toaccount for limitations in terms of processing,memory storage, energy consumption, networkavailability and so on. Several control strate-gies can be considered; biology-inspired searchtechniques such as ant algorithms are popu-lar due to their simplicity and low complexity,they however lack tight convergence bounds.Potential-based approaches, popular in pathplanning, can be extended to search and rescueoperations.

Some previous work has been carried out onstochastic optimisation and control for UAVsusing Markov Decision Processes (MDPs).These are stochastic since agents (UAVs) areuncertain of the external environment and itsfuture states. The MDP maps states to ac-tions (the policies). Optimal policies maybe learned using dynamic programming so-lutions although more recent work has fo-cussed on scalable, approximate, sub-optimalsolutions [SC02]. The agents may also beuncertain of their own actions and those ofother agents e.g. we might only exchange in-

formation between agents occasionally in or-der to reduce bandwidth consumption. Thisleads to Partially Observable Markov Deci-sion Processes (POMDP) [LK07] where againthere are challenges in providing scalable so-lutions. In such situations we can estimateunknown states from the available knowledge(from sensors etc). Where we have multipleheterogeneous agents, each agent may havetheir own POMDP and exchange knowledgefrom time to time [DC07]. Decentralised ap-proaches are well suited to coordinated con-trol, for example, token-based team coordi-nation using POMDP has been proposed by[XSSL06], where the MDP is approximated bya POMDP. A multi-thread architecture hasbeen employed for autonomous rotorcrafts op-erating under centralised control and carry-ing out search and rescue missions [TKF06].Here the threads represent different optimisa-tion goals and stochastic control was achievedvia MDPs and POMDPs.

POMDPs have been used for high level deci-sion making for UAVs tasked with search andstrike [STA+03], with a limited fuel supply.Performance prediction of an unmanned air-borne vehicle multi-agent system has been dis-cussed by [LD06] where UAV control agentsin a dynamic multi-agent system have a setof goals such as final destination and inter-mediate positions. An important aspect ofthe UAV control problem is ad-hoc commu-nications network management. In relationto this goal, multiagent POMDPs have beenused for network routing [RG03] where thestate space is: location of nodes, buffer states,and transmission success/failure status, whilethe actions are: send packet or remain idle.[BBH08] have developed a MDP-based solu-tion for UAV mission control that incorporatesfuel consumption into the objective functionfor UAV surveillance missions. In addition,the multiple goals can be conflicting and maychange over time. Such issues are discussed for

10

UAV control by [RFF09] who adapt a MDPplanner to a UAV coordination problem whereplans are generated separately over time foreach agent and then coordinated. However,many of these solutions are developed underthe assumptions that the speed and alttitude ofthe UAV are constant. The quadrotors we areusing, however, are low speed, low altitude andhighly agile. The impact of these constraints,coupled with the tradeoff off in the resolutionof the search algorithms, means that new the-oretical analysis of search algorithms must beconducted [WST10].

In the scenario we envisage in SUAAVE, theUAVs require intelligent communication, com-mand and control [FDAB05], in particular: ra-dio network intelligence, including multi-hops,mission level task control e.g. optimal searchstrategies and, intelligent flight control. Con-trol is complex in UAV environments becauseof unpredictable delays and losses in wirelesscommunication alongside a need to optimiseover multiple objectives and multiple param-eters. Much of control theory has been de-veloped for tightly coupled control systems inwhich delays are small and predictable andthere is no loss; however, investigations intomechanisms by which account can be taken ofthe distributed and stochastic nature of sys-tems such as we envisage are underway [DC07].In our situation, the only control loops that arecritically dependent on delay are those that in-volve UAV safety. For these, rather than at-tempt to solve the general problem, we willseek solutions that are approximate

However, while previous work has attemptedto use approaches such as MDPs, POMDPsand, more generally, Reinforcement Learning,it has often focussed on one aspect of the over-all optimisation problem while we attempt inSUAAVE to develop general overall search andrescue strategies. Previous work has also beenmainly tested in software, via simulation, whileour approach has the advantage of using a real

test-bed. We therefore must develop solutionswhich are lightweight, efficient and safe. In ourapproach, the UAV control mechanisms willtherefore be incorporated into a multi-agentsetting, where the agents are tasked with ad-dressing multiple heterogeneous (and possiblyconflicting) goals e.g. maximising the proba-bility of a successful search while minimisingenergy consumption. Types of agents we mustconsider include: flight (particularly separa-tion and navigation), communication, multi-hop, safety and search agents.

Overall, search and rescue operations witha swarm of UAVs present a great number ofchallenges as efficiency needs to be balancedwith practicality. One of our the main chal-lenges therefore involves implementing searchtechniques capable of leveraging the swarm tocollaborate in order to achieve a common task,while at the same time respecting the real andhard limitations of the platforms.

6 Data Fusion/Image Pro-

cessing

Given payload limitations our experimental ve-hicles’ main video sensor will be a lightweightvideo camera, possibly augmented with a crude(but low weight) infra-red camera and commongimbal mount. (So in particular we do notexpect to carry zoom lenses.) The on-boardprocessor is easily capable of taking a videostream at 10Hz and applying simple filters atframe rate; anything more intelligent has tobe applied to the filter output. In our earlierwork we have applied the Mean-Shift filter totrack ‘blobs’ [HC07]; this worked well for track-ing, but we need something more intelligent forsearching. (In our tracking work the initial tar-get was selected by a human.) We have so farexperimented with the Viola/Jones algorithmfor feature detection [VJ04], which is learningbased; one presents a number of examples of

11

camera shots to the learning algorithm withexamples of what is to to be learnt (and with-out), and the system produces a small amountof code that identifies that feature in gen-eral. The Viola/Jones algorithm works well,but only when given many (say, thousands)of training examples first which have usuallyto be tagged by hand. It also requires a lotof computer time in this phase — so much sothat only a super-computer cluster is practi-cal [FdHC09] — but the resultant classificationcode can easily be run in real-time. Quite apartfrom the human time required then, obtain-ing enough footage from aerial cameras can beproblematical; however the learning phase canbe primed with computer-generated syntheticimages to greatly reduce the amount of realfootage needed. We have begun investigatingthe utilty of SURF descriptors [SWJT10] andwe will shortly be testing related techniques,such as the use of HOG descriptors [DT05],which require a less arduous training phase.

Learning techniques are good at finding pat-terns, but less effective at generalisation. So,for example, a classifier designed to spot a per-son standing may have trouble when presnetedwith a picture of a person sheltering behind awall, even if the person is in full view. Deal-ing with such situations is likely to require theon-board programs to search within the image,which can be time-consuming. Whether ourvehicles will be able to deal with complex vi-sual situations is still an unknown; part of ourresults will be to quantify how much processingis required.

Once the ‘interesting features’ have beenidentified in an image sequence we then haveto apply some probabilistic reasoning to them,partially so we can avoid flagging up false pos-itives and partially to sensibly combine read-ings taken at different times and from differentpositions. Dealing with continuous hypotheseshas traditionally been the province of Gaussiantechniques such as the Kalman filter and its

many variants; non-Gaussian distributions canalso be tackled in a methodical setting, but thememory and processing required to estimate anarbitrary distribution can be an issue. What-ever distributions are used, one research issueis to combine sequences of hypotheses in a re-liable and effective manner. Here part of theproblem is to deal with the large number of hy-potheses that are typically generated with realimage sequences without them swamping theprocessor; but another, more insiduous, prob-lem is to combine the probability distributionswithout over-boosting the results [JU01]. Thiscan occur when readings reinforce each otherwhen they should not; most reasoning systemsassume large degrees of indepence between therandom variables, which may be true initiallybut after subsequent chains of reasoning maybe invalid. Deciding which chains are valid andwhich are not can be subtle, but important ifthe combination of readings is to be effective.We shall investigate the use of various subop-timal but robust distributed data fusion algo-rithms based on Covariance Intersection [JU01]and its generalisation [JBU06].

7 Conclusions

SUAAVE is a little unusual among projectsinvolvings collections of UAVs in its empha-sis on the networking between vehicles. Smallelectric vehicles have the potential to be low-cost and applied in many situations; but theirsmall payloads and low inertia make their con-trol a problem and their radio environmentone which is fraught with potential difficul-ties. Whilst buying vehicles off-the-shelf forthis project we were aware of how cutting-edgesuch devices would be; and indeed, we havehad to use time in this project modifying thevehicles for our purposes. (Nevertheless theinternal architecture of the vehicles has madethem relatively straightforward to deal with.)

12

So far the activities within SUAAVE havebeen focussed on getting single vehicles air-borne in a safe manner, and the basic cam-era functionality working. With the new fly-ing season upon us the emphasis moves tomulti-vehicle cooperation: first testing the ra-dio communications to and between vehicles,and then getting the vehicles to share infor-mation during the search process. We are alsocapturing images from our on-board camerasfor bench testing of algorithms, with the aimof then installing those on the vehicles them-selves.

Considerable emphasis has been placed onthe safety of the vehicles, both by use of a‘dead-mans handle’ system, but also by think-ing carefully about the flight states and thetransitions between them. The goal for theend of the project is to demonstrate a number(say, 10) of these vehicles engaging in coop-erative tasks in a reasonably realistic outdoorenvironment.

References

[BBH08] B Bethke, LF Bertuccelli, andJP How. Experimental demonstra-tion of adaptive MDP-based plan-ning with model uncertainty. InAIAA Guidance, Navigation andControl Conference and Exhibit,August 2008.

[BPDKHO09] Oleg Burdakov, Jonas KvarnstromPatrick Dohertyand Kaj Holmberg,and Per-Magnus Olsson. Position-ing unmanned aerial vehicles ascommunication relays for surveil-lance tasks. In Robotics: Scienceand Systems Conference, June 2009.

[BRS04] P. Basu, J. Redi, , and V. Shur-banov. Coordinated flocking ofUAVS for improved connectivity ofmobile ground nodes. In MILCOM2004, volume 3, pages 1628–1634,October 2004.

[CDB04] K. Chandrashekar, M.R. Dekhordi,and J.S. Baras. Providing full con-nectivity in large ad-hoc networksby dynamic placement of aerial plat-forms. In MILCOM 2004, pages1429–1436, October 2004.

[DC07] I Dusparic and V Cahill. Re-search issues in multiple policy op-timization using collaborative rein-forcement learning. In ProceedingsICSE 2007 Workshop on SoftwareEngineering for Adaptive and Self-Managing Systems (SEAMS 2007),pages 20–26, 2007.

[DT05] Navneet Dalal and Bill Triggs. His-tograms of oriented gradients forhuman detection. In CVPR, pages886–893, 2005.

[FDAB05] E. Frew, C. Dixon, B. Argrow, andT. Brown. Radio source localiza-tion by a cooperating UAV team. InAIAA Conf., 2005.

[FdHC09] Helen Flynn, Julian de Hoog, andStephen Cameron. Integrating au-tomated object detection into map-ping in usarsim. In Robots, Games,and Research: Success stories inUSARSim (IROS 2009 Workshop),St. Louis, October 2009.

[HC07] Heiko Helble and Stephen Cameron.OATS: The Oxford Aerial TrackingSystem. Robotics and AutonomousSystems, 55(9):661–666, September2007.

[HSL06] Z. Han, A.L. Swindlehurst,and K.J.R. Liu. Smart deploy-ment/movement of unmanned airvehicle to improve connectivity inmanet. In Wireless Communica-tions and Networking Conference2006, pages 252–257, April 2006.

[JBU06] Simon Justin Julier, Timothy Bai-ley, and Jeffrey Kent Uhlmann. Us-ing Exponential Mixture Models forSuboptimal Distributed Data Fu-sion. In Proceedings of the IEEENonlinear Statistical Signal Process-ing Workshop (NSSPW06), pages

13

160–163, Cambridge, UK, 13–15September 2006.

[JU01] Simon Justin Julier and Jef-frey Kent Uhlmann. GeneralDecentralized Data Fusion WithCovariance Intersection (CI). InD. Hall and J. Llinas, editor, Hand-book of Data Fusion. CRC Press,Boca Raton FL, USA, 2001.

[LD06] Z Lian and A Deshmukh. Perfor-mance prediction of an unmannedairborne vehicle multi-agent sys-tem. European Journal of Opera-tional Research, 172:680–695, 2006.

[LK07] Li and T. Kunz. Cooperative nodelocalization for tactical wireless sen-sor networks. In MILCOM 2007,2007.

[PBGY08] S. Perumal, J.S. Baras, C.J. Graff,and D.G. Yee. Aerial platformplacement algorithms to satisfy con-nectivity, capacity and survivabilityconstraints in wireless ad-hoc net-works. In KMILCOM 2008, pages1–7, November 2008.

[RFF09] E Rachelson, P Fabiani, and Gar-cia F. Adapting an MDP plan-ner to time-dependency: case studyon a UAV coordination problem.In 4th Workshop on Planning andPlan Execution for Real-World Sys-tems:Principles and Practices forPlanning in Execution, Thessa-loniki, Greece, September 2009.

[RG03] Bharaneedharan Rathnasabapathyand Piotr J. Gmytrasiewicz. For-malizing multi-agent POMDP’s inthe context of network routing. InHICSS 2003, 2003.

[RZ07] I. Rubin and R. Zhang. Place-ment of uavs as communication re-lays aiding mobile ad hoc wirelessnetworks. In MILCOM 2007, pages1–7, October 2007.

[SC02] Bruno Scherrer and FrancoisCharpillet. Cooperative co-learning: A model-based approach

for solving multi agent reinforce-ment problems. In ICTAI 2002,pages 463–468, 2002.

[STA+03] Doug Schesvold, Jingpeng Tang,Benzir Md Ahmed, Karl Altenburg,and Kendall E. Nygard. POMDPplanning for high level UAV deci-sions: Search vs. strike. In CAINE,pages 145–148, 2003.

[SWJT10] Andrew Colquhoun Symington, So-nia Waharte, Simon Justin Julier,and Niki Trigoni. Probabilistic Tar-get Detection by Camera-EquippedUAVs. In Proceedings of theIEEE International Conference onRobotics and Automation (ICRA),Anchorage, AK, USA, 3–9 May2010.

[TKF06] F. Teichteil-Koningbuch andP. Fabiani. Autonomous search andrescue rotorcraft mission stochasticplanning with generic DBNs. InArtificial Intelligence in Theoryand Practice, volume 217, pages483–492. IFIP, 2006.

[VJ04] Paul Viola and Michael Jones. Ro-bust real-time face detection. Inter-national Journal of Computer Vi-sion, 57:137–154, May 2004.

[WST10] Sonia Waharte, Andrew Syming-ton, and Niki Trigoni. ProbabilisticSearch with Agile UAVs. In Pro-ceedings of the IEEE InternationalConference on Robotics and Au-tomation (ICRA), Anchorage, AK,USA, 3–9 May 2010.

[XSSL06] Yang Xu, Paul Scerri, Katia P.Sycara, and Michael Lewis. Com-paring market and token-based co-ordination. In AAMAS 2006, pages1113–1116, 2006.

14