Collision avoidance and coalition formation of multiple
unmanned aerial vehicles in high density traffic
environments
A Thesis
Submitted for the Degree of
Doctor of Philosophy
in the Faculty of Engineering
by
Joel George Manathara
Department of Aerospace Engineering
Indian Institute of Science
Bangalore – 560 012
May 2011
Abstract
There has been an increased interest in Unmanned Aerial Vehicles (UAVs) in the past few years
because of the wide variety of possible applications of UAVs in both military and civilian sectors.
Many missions involving UAVs are executed more efficiently when multiple UAVs are deployed,
and therefore an increased use of UAVs will cause certain regions of airspace to be cluttered with
UAVs executing various autonomous missions, leading to many operational issues. An important
consideration in such high density traffic airspace with UAVs in free flight is that the UAVs should
avoid colliding with other UAVs. Another problem is to form groups (coalitions) of UAVs whose
combined capabilities are required to accomplish certain missions. This thesis studies collision
avoidance and coalition formation of UAVs in free flight of high density traffic, and proposes
algorithms that can be easily implemented in realistic scenarios.
First, this thesis addresses the problem of collision avoidance among UAVs in an airspace
with high density traffic. Collision avoidance among aircraft has been extensively addressed in
the literature from the point of view of air traffic management. However, most of these works
consider collision avoidance among only a very few aircraft. This thesis considers the problem of
collision avoidance among multiple UAVs in situations where air traffic density is very high. The
main hypothesis proposed is: whenever a UAV encounters multiple conflicts, it identifies the ‘most
threatening’ conflict and does collision avoidance maneuver to avoid it. Every UAV doing so in
a multiple UAV free flight environment results in good performance in terms of lesser number of
near misses and smaller deviations from their nominal paths. The proposed algorithm considers
that conflict which has the least time-to-go to occur as the most threatening one. The appropriate
collision avoidance maneuver, in that case, is that which increases the line of sight rate between
the concerned UAVs. The efficacy of the proposed algorithm is tested using simulations involving
random flights with high density traffic. Also, simulations are done to show that the algorithm
performs well even in the presence of noise in position and velocity measurements of neighbor-
ing UAVs. This thesis also gives algorithms to handle collision avoidance among multiple UAVs
in three dimensions. Two 3-dimensional collision avoidance algorithms are developed, one of
which approximates the three dimensional engagements to planar conflicts and uses it for conflict
resolution, while the other handles three dimensional conflicts via deconfliction maneuvers that
are derived from the original three dimensional engagement geometries. Simulations are done to
evaluate the performance of these three dimensional collision avoidance algorithms. Further, the
proposed collision avoidance algorithms are implemented using realistic six degree of freedom
UAV models for which proportional-integral controllers are designed using successive loop clo-
sure method. Specific conflict cases as well as random flights in congested airspaces are studied
using these realistic UAV models and analyzed to show how the proposed algorithm handles those
i
situations. As the collision avoidance algorithms proposed in this thesis are reactive, they perform
well with heterogeneous – different speed – UAVs, and in cases where some of the UAVs in con-
flict are non-cooperative, as demonstrated though simulations. The studies in this thesis show that
the implementation of the proposed collision avoidance algorithms leads to a safer and efficient
operational airspace occupied by multiple UAVs.
Next, a search and prosecute mission involving a large number of UAVs and targets is consid-
ered. To prosecute a target, it is at times necessary to form a sub-group of UAVs, called a coalition,
whose combined resources suffice to prosecute the target. The task is to find and prosecute all the
targets in minimum time. It is proved in the thesis that the coalition formation problem addressed
is NP-hard. This thesis gives a sub-optimal coalition formation algorithm that has polynomial
time complexity. Since the algorithm has a polynomial time complexity, it scales well making it
suitable for cases where there are a large number of UAVs and targets. Simulations are carried
out to show that this coalition formation algorithm works well. The mission completion time us-
ing the proposed coalition formation algorithm is comparable to that of the solution of the global
combinatorial optimization problem obtained using Particle Swarm Optimization algorithm. Sim-
ulations show that the algorithm performs well in the presence of a large number of UAVs and
targets. The coalition formation algorithm is then extended to handle situations where the UAVs
have limited communication ranges. UAVs with limited communication ranges lead to a time
varying UAV network over which the UAVs have to communicate to form coalitions. Toward
this, a communication protocol is proposed which also takes into account the communication and
the computational delays associated with the UAV network. Also considered in this thesis, is the
prosecution of non-stationary and maneuvering targets. Simulation studies show that the proposed
algorithm is suitable for use in large UAV groups for real-time search and prosecute tasks.
Finally, this thesis considers some multiple UAV missions that require the application of colli-
sion avoidance and coalition formation techniques. Quite often, in multiple UAV missions, UAVs
are required to come together or rendezvous to exchange information and resources. The problem
of multiple UAV rendezvous is tackled by using a consensus among the UAVs to attain rendezvous
and the collision avoidance algorithm previously developed for safety. Simulations show how the
UAVs that deviated away from the preplanned trajectory, negotiating a conflict, still attain ren-
dezvous via velocity control demanded by the consensus algorithm. Also considered is a search
and prosecute mission where the UAVs also have to avoid collisions among each other. The colli-
sion avoidance algorithm developed is used in conjunction with the proposed coalition formation
algorithm to solve this problem. These studies demonstrate that the individual collision avoidance
and coalition formation algorithms, developed in this thesis, can be seamlessly integrated to work
together.
In summary, the main contributions of this thesis include (a) novel collision avoidance algo-
ii
rithms, which are conceptually simple and easy to implement, for resolving path conflicts – both
planar and three dimensional – in a high density traffic airspace with UAVs in free flight and (b)
efficient coalition formation algorithms for search and prosecute task with large number of UAVs
and targets where UAVs have limited communication ranges and targets are maneuvering. Sim-
ulations to evaluate the performance of algorithms based on these concepts to carry out realistic
tasks by UAV swarms are also given.
iii
iv
Acknowledgements
I am grateful to my thesis advisor Prof. Debasish Ghose for the great support and directions he
always provided, and the patience he had with me. I thankfully remember Late Dr. S. Pradeep
who was my thesis advisor during the initial two years of my doctoral studies. A part of the work
presented in this thesis was done over collaboration, and I thank Dr. P. B. Sujit who coordinated
the collaborations; and my collaborators Prof. F. L. Pereira, Prof. Randal Beard, Prof. J. B. Sousa,
and Dr. Jose Pinto. I thank Dr. Ambedkar Dukkipati for all his support and guidance. I am thankful
to everyone who, directly or indirectly, helped toward making the writing of this thesis possible.
v
vi
Contents
Abstract i
Acknowledgements v
List of Tables xi
List of Figures xiii
List of Algorithms xvii
Notations xix
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Coalition Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Collision avoidance and coalition formation in multiple UAV missions . . . . . . . . . . 16
1.5 Contributions and organization of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.6 A note to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2 Collision avoidance among multiple UAVs 21
2.1 Preliminary Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.1 Some basic concepts and assumptions . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.2 Overview of the proposed solution . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2 Two dimensional collision avoidance algorithm . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.1 Conflict detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.2 Collision avoidance maneuver . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.3 Effectiveness of handling pairwise conflicts . . . . . . . . . . . . . . . . . . . . 28
2.2.4 Dubins path to destination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.5 RCA-2D: two dimensional collision avoidance algorithm . . . . . . . . . . . . . 33
2.3 Three dimensional conflict avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.1 Collision prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.2 Collision avoidance maneuver . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.3 Homing to destination in three dimension . . . . . . . . . . . . . . . . . . . . . 39
vii
2.4 Implementation of collision avoidance algorithms . . . . . . . . . . . . . . . . . . . . . . 39
2.4.1 Implementation of the RCA-2D algorithm . . . . . . . . . . . . . . . . . . . . . 39
2.4.2 Implementation of the RCA-3D-O algorithm . . . . . . . . . . . . . . . . . . . . 42
2.4.3 Implementation of the RCA-3D-I algorithm . . . . . . . . . . . . . . . . . . . . 43
2.4.4 Aircraft conflict resolution using Satisficing Game Theory (SGT) . . . . . . . . . 44
2.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.5.1 Simulation results for two dimensional case . . . . . . . . . . . . . . . . . . . . 48
2.5.2 Simulation results for 3D case . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3 Collision avoidance with realistic UAV models 67
3.1 Six DoF equations of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Trimming aircraft for a level flight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Controller design for 2D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.1 Altitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.2 Velocity hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.3 Attitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3.4 Testing the response of the UAV augmented with controller . . . . . . . . . . . . 75
3.4 Simulation results for planar collision avoidance . . . . . . . . . . . . . . . . . . . . . . 77
3.4.1 Tailored test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.4.2 Random flight test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.5 Controller design for 3D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 82
3.5.1 Six DoF model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.5.2 Pitch rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . 83
3.5.3 Turn rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . 83
3.5.4 Response of UAV augmented with controller . . . . . . . . . . . . . . . . . . . . 83
3.6 Simulations results of 3D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 86
3.6.1 Random flights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.6.2 Test case of modified random flights . . . . . . . . . . . . . . . . . . . . . . . . 88
3.6.3 Perpendicular flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.6.4 Non-cooperating UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.6.5 Heterogeneous UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
viii
3.7 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4 Coalition formation with global communication 93
4.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.2 Coalition formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.2.1 Polynomial time coalition formation algorithm (PTCFA) . . . . . . . . . . . . . 103
4.2.2 Optimal coalition formation algorithm (OCFA) . . . . . . . . . . . . . . . . . . 109
4.2.3 Complexity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.3 Simultaneous strike . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.4 Combinatorial optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.4.1 Overview of particle swarm optimization . . . . . . . . . . . . . . . . . . . . . . 115
4.4.2 Solution to the optimal coalition formation problem using PSO . . . . . . . . . . 116
4.5 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.5.1 Solutions obtained using PSO and two-stage algorithms: an example . . . . . . . 122
4.5.2 Effect of increase in number of UAVs and targets . . . . . . . . . . . . . . . . . 129
4.5.3 Coalition formation for UAVs with heterogenous speed . . . . . . . . . . . . . . 136
4.5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5 Coalition formation with limited communication 141
5.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.1.1 The mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.1.2 Target and UAV kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.2 Communication protocol for coalition formation . . . . . . . . . . . . . . . . . . . . . . 145
5.2.1 Broadcast in a limited communication network . . . . . . . . . . . . . . . . . . . 145
5.2.2 Coalition formation over a dynamic network . . . . . . . . . . . . . . . . . . . . 147
5.2.3 Structure of the broadcast messages . . . . . . . . . . . . . . . . . . . . . . . . 148
5.3 Determining ETA to target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.3.1 Stationary targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.3.2 Constant velocity targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.3.3 Maneuvering targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.4 Prosecution of targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.4.1 Stationary target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
ix
5.4.2 Constant velocity targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.4.3 Maneuvering targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5.5.1 Example scenario: stationary and constant velocity targets . . . . . . . . . . . . . 157
5.5.2 Example scenario: maneuvering targets . . . . . . . . . . . . . . . . . . . . . . 159
5.5.3 Effect of varying number of UAVs and targets . . . . . . . . . . . . . . . . . . . 162
5.5.4 Effect of varying communication range . . . . . . . . . . . . . . . . . . . . . . . 164
5.5.5 Effect of hop delay and max-hops . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6 Collision avoidance and coalition formation in multiple UAV missions 169
6.1 Rendezvous via consensus without collision avoidance . . . . . . . . . . . . . . . . . . . 170
6.1.1 The consensus protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
6.1.2 Determining ETA to target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.1.3 Handling velocity bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
6.1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.2 Rendezvous with collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.2.1 Incorporating collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.2.2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
6.3 Coalition formation in a search and prosecute mission with collision avoidance . . . . . . 181
6.3.1 Coalition formation in presence of collision avoidance . . . . . . . . . . . . . . . 182
6.3.2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.4 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
7 Conclusions 185
7.1 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
7.2 Coalition Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.3 Collision avoidance and coalition formation in multiple UAV missions . . . . . . . . . . 189
7.4 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Bibliography 191
Publications from this thesis 203
x
List of Tables
2.1 Comparison of performance of algorithms RCA-2D and SGT . . . . . . . . . . . . . 52
2.2 Comparison of computation times of RCA-2D and SGT per simulation . . . . . . . . 55
2.3 Comparison of performance of RCA-2D and SGT in the presence of noise in position
measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.4 Comparison of performance of RCA-2D and SGT in the presence of noise in heading
measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.5 Comparison of performance of algorithms RCA-2D and RCA-3D-O . . . . . . . . . 61
2.6 Comparison of computation times of algorithms RCA-2D and RCA-3D-O . . . . . . 61
2.7 Performance of the 3D algorithms for the test case of perturbed random flights . . . . 62
2.8 Performance of the 3D algorithms for the test case of modified random flights . . . . 63
3.1 The geometric and inertial parameters of the 6 DoF UAV model . . . . . . . . . . . . 71
3.2 Stability and control derivatives of the 6 DoF UAV model . . . . . . . . . . . . . . . 72
3.3 Initial positions and orientations of UAVs in Fig. 3.10 . . . . . . . . . . . . . . . . . 81
3.4 Random flight test case for 6 DoF UAV models with the inner and outer radii as 4 km
and 5 km . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.5 Random flight test case for 6 DoF UAV models with the inner and outer radii as 400 m
and 500 m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.6 Random flight test case with 3D collision avoidance for 6 DoF UAV models . . . . . 88
3.7 Performance of the 3D algorithms in 6 DoF UAV models . . . . . . . . . . . . . . . 88
3.8 Trim conditions corresponding to various steady level cruise velocities . . . . . . . . 91
3.9 Random flight test case with 3D collision avoidance for heterogeneous 6 DoF UAV
models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.1 Details of the agents which responded to the coalition formation request in the given
example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.2 Details of the agents in the given example, after sorting . . . . . . . . . . . . . . . . 106
4.3 Comparison of the mission and simulation times obtained with the PSO solution and
the two-stage algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.1 Performance of the rendezvous algorithm in congested environments . . . . . . . . . 181
xi
xii
List of Figures
1.1 An example situation requiring collision avoidance . . . . . . . . . . . . . . . . . . 5
2.1 A 2D engagement between two UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Variation of the LOS separations between two UAVs with time for various initial
engagement geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3 Collision avoidance maneuver for a 2D engagement . . . . . . . . . . . . . . . . . . 30
2.4 The minimum LOS separations attained between two UAVs for various initial engage-
ment geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5 Dubins paths to destination (the case of free terminal constraint) . . . . . . . . . . . 33
2.6 A sample situation requiring 3D collision avoidance among multiple UAVs . . . . . . 35
2.7 Engagement scenario and collision plane in a 3D conflict . . . . . . . . . . . . . . . 37
2.8 Collision avoidance in simple conflict situations . . . . . . . . . . . . . . . . . . . . 50
2.9 A schematic of the random flight test case . . . . . . . . . . . . . . . . . . . . . . . 51
2.10 Variation of near misses and efficiency with Rsen and Rdes . . . . . . . . . . . . . . . 53
2.11 A snapshot during the simulation of random flights . . . . . . . . . . . . . . . . . . 54
2.12 Scalability of number of near misses and efficiency with UAV density . . . . . . . . 57
2.13 Average decision time per simulation for different UAV densities . . . . . . . . . . . 58
2.14 Average decision time per UAV per second with increase in traffic density . . . . . . 59
2.15 A screenshot of UAVs making 3D collision avoidance maneuvers . . . . . . . . . . . 60
2.16 Collision avoidance in 3D using RCA-3D-O – the test case of perpendicular flow . . 62
2.17 The test case of modified random flights . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1 Body axis of an aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2 Altitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3 Velocity hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4 Attitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.5 Response of 6 DoF UAV model to bank angle command . . . . . . . . . . . . . . . . 76
3.6 Turn rates for 6 DoF UAV model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.7 Collision avoidance of 3 realistic UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.8 Control inputs and corresponding states for one of the UAVs in Fig. 3.7 . . . . . . . . 79
3.9 Collision avoidance of 4 realistic UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 80
xiii
3.10 Collision avoidance of 5 realistic UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.11 Pitch rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.12 Turn rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.13 UAV response to a doublet pitch rate demand . . . . . . . . . . . . . . . . . . . . . 86
3.14 UAV response to a doublet turn rate demand . . . . . . . . . . . . . . . . . . . . . . 87
3.15 Collision avoidance in three dimensional perpendicular flow . . . . . . . . . . . . . 89
3.16 The test case of array flow where none of the UAVs cooperate . . . . . . . . . . . . . 90
3.17 The test case of array flow where some of the UAVs cooperate . . . . . . . . . . . . 90
3.18 The test case of array flow where all the UAVs cooperate . . . . . . . . . . . . . . . 91
4.1 A situation where multiple agents detect multiple targets . . . . . . . . . . . . . . . 101
4.2 Sequence of events during coalition formation . . . . . . . . . . . . . . . . . . . . . 102
4.3 Dubins path and the roundabout path to a target . . . . . . . . . . . . . . . . . . . . 112
4.4 Calculation of length of roundabout path to target . . . . . . . . . . . . . . . . . . . 113
4.5 Initial positions of the UAVs and the targets for the PSO example . . . . . . . . . . . 120
4.6 The target assignment achieved using PSO in the given example . . . . . . . . . . . 121
4.7 Initial positions of the UAVs and the targets for the given example . . . . . . . . . . 123
4.8 The trajectory of U1 prosecuting T1 using PTCFA . . . . . . . . . . . . . . . . . . . 123
4.9 The trajectory of U1 prosecuting T1 using OCFA . . . . . . . . . . . . . . . . . . . . 124
4.10 Routes followed by the coalition U2,U3, and U4 formed by PTCFA to prosecute T4 . . 124
4.11 Routes followed by the coalition U3 and U4 formed by OCFA to prosecute T4 . . . . 125
4.12 UAVs U3 and U4 assigned to T3 by PTCFA . . . . . . . . . . . . . . . . . . . . . . . 125
4.13 UAV U2 assigned to T3 by OCFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.14 Trajectories of UAVs U1 and U2 prosecuting target T2 using PTCFA . . . . . . . . . . 126
4.15 Trajectories of UAVs U1,U3, and U4 prosecuting target T2 using OCFA . . . . . . . . 127
4.16 Coalition formation and prosecution of all the targets in the given example using PSO 127
4.17 Comparison of the mean mission completion times (5 targets) . . . . . . . . . . . . . 130
4.18 Mean percentage mission not accomplished (5 targets) . . . . . . . . . . . . . . . . . 131
4.19 Comparison of mean mission completion times (10 targets) . . . . . . . . . . . . . . 131
4.20 Mean percentage mission not accomplished (10 targets) . . . . . . . . . . . . . . . . 132
4.21 Comparison of mean mission completion times (15 targets) . . . . . . . . . . . . . . 132
4.22 Mean percentage mission not accomplished (15 targets) . . . . . . . . . . . . . . . . 133
4.23 Average computational time taken to form coalitions using PTCFA and OCFA . . . . 133
xiv
4.24 Mean time taken to complete a simulation using PSO . . . . . . . . . . . . . . . . . 134
4.25 Initial UAV and target locations for the given example . . . . . . . . . . . . . . . . . 137
4.26 Trajectories of UAVs U4 and U5 to target T1 . . . . . . . . . . . . . . . . . . . . . . 137
4.27 Trajectories of UAVs U1,U2, and U3 to T3 . . . . . . . . . . . . . . . . . . . . . . . 138
4.28 Trajectories of UAVs U1,U3,U4, and U5 to T2 . . . . . . . . . . . . . . . . . . . . . 138
5.1 Search and prosecute mission using UAVs with limited sensor and communication
ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.2 Illustration of why Rcomm should be greater than 2Rsen . . . . . . . . . . . . . . . . . 143
5.3 Decision process and communication protocol for coalition formation . . . . . . . . 150
5.4 Dubins and roundabout paths to a moving target . . . . . . . . . . . . . . . . . . . . 152
5.5 Illustration of the iterative method to find ETA to a moving target . . . . . . . . . . . 152
5.6 Prosection strategy for maneuvering targets . . . . . . . . . . . . . . . . . . . . . . 156
5.7 Initial positions of the UAVs and the targets for the given example . . . . . . . . . . 158
5.8 Trajectories that the coalition members take to prosecute T1 . . . . . . . . . . . . . . 159
5.9 Trajectories that the coalition members take to prosecute the moving target T2 . . . . 160
5.10 Initial position of UAVs and targets for the given example . . . . . . . . . . . . . . . 160
5.11 Prosecution of T1 by U3 and U4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
5.12 A successful coalition for T2 is not formed due to lack of resources . . . . . . . . . . 162
5.13 Prosecution of T2 by U1 and U2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.14 Variation of mission time with increase in number of UAVs . . . . . . . . . . . . . . 164
5.15 Variation of mission time with increase in communication range of UAVs . . . . . . 165
5.16 Variation of mission time with increase in maximum hops allowed . . . . . . . . . . 165
5.17 Effect of Δhop and Hmax on mission performance . . . . . . . . . . . . . . . . . . . . 166
5.18 Variation of mission time with increase in communication delay . . . . . . . . . . . 167
6.1 Calculating the Dubins distance to a target . . . . . . . . . . . . . . . . . . . . . . . 172
6.2 Rendezvous of two UAVs using velocity control . . . . . . . . . . . . . . . . . . . . 173
6.3 Rendezvous of two UAVs via velocity control and a wandering maneuver . . . . . . 174
6.4 Rendezvous of 10 UAVs with velocity bounds . . . . . . . . . . . . . . . . . . . . . 175
6.5 Rendezvous of 3 UAVs onto a slowly maneuvering target . . . . . . . . . . . . . . . 175
6.6 Rendezvous of 20 UAVs using average consensus . . . . . . . . . . . . . . . . . . . 176
6.7 Rendezvous of 20 UAVs using weighted average consensus . . . . . . . . . . . . . . 177
6.8 Rendezvous of 20 UAVs using limited communication . . . . . . . . . . . . . . . . . 178
xv
6.9 Rendezvous of 20 UAVs using global communication . . . . . . . . . . . . . . . . . 178
6.10 Rendezvous of 5 UAVs with collision avoidance . . . . . . . . . . . . . . . . . . . . 179
6.11 The variation of ETAs of UAVs while attaining rendezvous in presence of collision
avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
6.12 Congested airspace in which a rendezvous need to be achieved . . . . . . . . . . . . 181
6.13 Rendezvous with and without collision avoidance in a dense environment . . . . . . 182
6.14 Trajectories of UAVs prosecuting a target with and without collision avoidance . . . . 183
6.15 Comparison of average mission times for a search and prosecute task with and without
collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
xvi
List of Algorithms
2.1 Deconfliction algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Algorithm to implement Dubins path with free terminal constraints . . . . . . . . . . 32
2.3 Reactive Collision Avoidance 2D Algorithm . . . . . . . . . . . . . . . . . . . . . . 34
4.1 First stage of the PTCFA to form a minimum time coalition . . . . . . . . . . . . . . 104
4.2 Second stage of the PTCFA to prune members to obtain a reduced member coalition . 105
4.3 Optimal coalition formation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.4 Global solution using PSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.1 Algorithm to iteratively find the earliest interception point to a constant velocity target 153
xvii
xviii
Notations
Δhop Delay associated with each hop a message makes over network
λ Tuning parameter for algorithm RCA
ψi Heading of ith UAV
ψTj Heading of target Tj
θ LOS angle
ωmax Maximum turn rate of a UAV
ain Acceleration for deconfliction in the collision plane
aout Acceleration for deconfliction out of the collision plane
ai Acceleration of Ui
C (Ui,Tj) Set of UAVs in the coalition formed by Ui to prosecute Tj
D Destination
D(Ui,Tj) Set of distances to Tj of UAVs in C (Ui,Tj)
DTjUi
Roundabout path length from Ui to Tj
DTjUi
Dubins path length from Ui to Tj
ETAi Estimated time of arrival for Ui
Hmax Maximum number of hops a message is allowed to make
i, j Unit vectors
M Number of targets
Ni Set of neighbors of Ui
N Number of UAVs
n Vector perpendicular to collision plane
PC (Ui,Tj) UAVs that responded to coalition request of Ui for Tj
xix
pi Position of ith UAV, pi = (xi,yi,zi)
Q Dimension of each particle in PSO
RUi Resource capability vector of Ui, RUi = (RUi1 , . . . ,RUi
n )
RTi Resource capability vector of Ti, RTi = (RTi1 , . . . ,RTi
m)
R Radius of turn
Rcomm Communication range of UAV
Rdes Desired radius of turn
Rmin Minimum radius of turn
Rsen Sensor radius of UAV
r LOS separation
S Swarm size in PSO
τ Total time for mission completion
τTjc Time taken to prosecute Tj after coalition is formed
τTjUk
ETA of Uk to Tj
τTj(P,P′) Estimated time taken by target Tj to travel from point P to point P′
τUi(P,P′) Time taken by Ui to take a roundabout path from point P to point P′
τs Search time
Tj jth target
t time
tgo time to go
Ui ith UAV
V kl Velocity of lth particle at kth time step in PSO, V k
l = (vkl1,vk
l2, . . . ,vk
lQ)
VTj Velocity of target Tj
Vi Velocity of ith UAV
xx
Vmax Upper bound for UAV velocity
Vmin Lower bound for UAV velocity
vi Velocity vector of ith UAV, vi = (ui,vi,wi)
Xkl Position of lth particle at kth time step in PSO, Xk
l = (xkl1,xk
l2, . . . ,xk
lQ)
(xi,yi) Position of Ui
(xTj ,yTj) Position of Tj
Abbreviations
ATC Air Traffic Control
DoF Degree of Freedom
ETA Earliest/Estimated Time of Arrival
LOS Line of Sight
MAC Medium Access Control
OCFA Optimal Coalition Formation Algorithm
PI Proportional Integral
PN Proportional Navigation
PSO Particle Swarm Optimization
PTCFA Polynomial Time Coalition Formation Algorithm
RCA Reactive Collision Avoidance
SGT Satisficing Game Theory
TCP Transmission Control Protocol
TTL Time to Live
UAV Unmanned Aerial Vehicle
ZEM Zero Effort Miss
xxi
xxii
1 Introduction
This chapter introduces and motivates the problems that arise in missions involvingmultiple Unmanned Aerial Vehicles (UAVs). In particular, collision avoidance andcoalition formation of UAVs, which are the main topics of this thesis, are discussed.A review of existing literature is also given. The chapter concludes with an outlineof the thesis.
�n today’s world, when more and more tasks are automated and machines are
imparted intelligence to operate without human intervention, the Unmanned
Aerial Vehicles (UAVs) find a wide variety of applications like surveillance
and search. UAVs are pilotless aircraft which are partially or completely autonomous.
Research in UAVs is growing at a fast pace due to various capabilities that they offer
for military and civilian applications. Owing to its portability, low cost of acquisition,
operation, and maintenance, and absence to human risk, UAVs will continue to find
wider applications. The projected number of UAVs that will be in use in a decade from
now will be enormous certain tasks efficiently, and also to achieve robustness through
redundancy, it may at times be necessary to deploy a team of UAVs. Use of multiple
UAVs leads to several operational problems that need to be addressed. Among these,
this thesis addresses the problems of collision avoidance and coalition formation of
multiple UAVs in high density traffic environments, and their applications in multiple
UAV missions.
1.1 Motivation
Orville Wright, who flew the first recorded heavier than air powered flight in 1903,
remarked
“. . . it was nevertheless - the first time in the history of the world in which
a machine carrying a man had raised itself by its own power into the air in
full flight, had sailed forward without reduction of speed, and had finally
landed at a point as high as that from which it started.” (Wright, 1923)
The first flight, by Orville Wright, covered 120 feet (37 m) at a speed of only 6.8 miles
per hour (10.9 km/h) at about 10 feet (3.0 m) above the ground. Ever since that modest
1
first flight, the aircraft design industry has undergone an exponential rate of growth,
unparalleled by any other field of its time; except for the very recent advances in the
computer industry. Led by the slogan Faster, Farther, Higher (and Safer) (McMas-
ters & Cummings, 2002), over the years, the boundaries of speed, height, range and
endurance have been pushed farther and farther by aircraft designers. However, as in
the words of Orville Wright, aircraft always had had the tag of a “machine carrying a
man”.
Why UAVs?
Man has always been in the aircraft design loop. Design considerations included the
presence of pilot who will ultimately fly the aircraft. There is no reason to design
an aircraft of incredibly high endurance when the pilot suffers from fatigue in a much
shorter time. Similarly, there is no incentive to build a fighter aircraft that can take high
g-loads, which gives it a considerable advantage over the enemy aircraft, when the pilot
flying it ‘blacks out’ at much less tighter maneuvers. Motivated by these observations,
it is rational to conclude that the absence of a human in the aircraft cockpit throws
open the design space for new opportunities.
For example, the elimination of a pilot from a manned combat aircraft removes
many of the conventional design constraints like tolerance to g-loads, endurance limits,
safety factors, comfort, redundancy levels, and vulnerability. This at once throws open
the design parameter space; and dramatic improvements in performance measures like
increased speed, range, maneuverability, and payload can be achieved. Another exam-
ple is the coordinated turn maneuver. A coordinated turn requires the use of all the
aircraft controls – elevator, aileron, rudder, and throttle – to achieve a turn in which
the lateral velocities and forces are kept to zero. Attaintment of a coordinated turn
demands complicated control laws. However, a coordinated turn is insisted upon for
passenger comfort. In UAVs, as coordinated turn constraint is not critical, a turn can
be achieved by use of much simpler control laws.
Some missions, like oil pipeline monitoring, are monotonous jobs for humans and
they indeed do not require human presence. There are other missions, like nuclear
or chemical leakage inspection, that prohibits the presence of humans due to safety
reasons. Thus UAVs find great use in a wide variety of dull, dirty, and dangerous
missions.
2
Why autonomous UAVs?
Once the presence of a pilot is eliminated, the only ways to fly an aircraft is to do so
either remotely or autonomously. Flying an aircraft remotely has its own associated
problems like fatigue of the remote pilot, lack of adequate input and feedback for the
remote pilot, and the range limitation introduced by the necessity of communication
between a UAV and the remote pilot commands. Thus UAVs that can fly autonomously
is a more viable option. For example, a factor limiting longer endurance was the
presence of a human pilot. Introduction of autopilots for less critical long endurance
missions mitigate this problem.
As Rasmussen and Schumacher remarked in an editorial of a special issue on UAVs
(Rasmussen & Schumacher, 2007), “the lack of a human pilot,” on board or otherwise,
“enables a wealth of new operational paradigms.” However, higher levels of UAV au-
tonomy are required for the realization of these new operational capabilities. A clas-
sification of the levels of autonomy for UAV operations is given by Suresh and Ghose
(2010). Higher levels of autonomy in UAVs lead to greater flexibility and effectiveness
of UAV missions.
Why multiple autonomous UAVs?
Since UAVs are usually small owing to portability requirements, it is often necessary to
deploy a team of UAVs to accomplish certain missions. Also, there are some missions,
like search, that are more efficiently done by a group rather than a single UAV alone. In
addition, the use of multiple UAVs will add to missions, a robustness achieved through
redundancy. Thus, it is desirable for UAVs to work cooperatively in groups. The use
of sophisticated decentralized and cooperative control algorithms will enable groups
of UAVs to attain a higher level of ‘group autonomy’ (Suresh & Ghose, 2010), thus
attaining capabilities that significantly outperform that of individual UAVs.
Cooperation and coordination among multiple UAVs
The importance of the use of multiple UAVs and the necessity of undertaking a focused
research in this area was identified a decade ago (Banda, 2002). Although the use of
multiple UAVs for missions have several advantages, it will lead to many operational
issues. The UAVs will have to cooperate and coordinate with other UAVs, in the same
as well as other groups, to effectively carry out the respective missions. This is espe-
cially so in high density traffic situations. As Scerri et al. (2008) remark, “coordinating
hundreds or thousands of UAVs present a variety of new exciting challenges.”
3
The multiple UAV problems that requires cooperation and coordination include
cooperative path planning (Tsourdos et al., 2011; Shanmugavel et al., 2010; Kamal
et al., 2005), cooperative control (Tsourdos et al., 2007), and formation flying (Gu
et al., 2010; Kim et al., 2007).
One common subproblem in all multiple UAV missions is that the UAVs need to
avoid each other. The problem of collision avoidance among UAVs acquires great im-
portance in high density UAV traffic environments. This thesis addresses the problem
of collision avoidance among UAVs and gives algorithms which when implemented
by every UAV in a group results in safe missions.
Another problem of interest in multiple UAV missions, in general, in any multi-
agent systems, is that of coalition formation. Given a task, it is required to select a
subgroup, from a group of available UAVs, that has the required capabilities to com-
plete the task. This subgroup should also be the one that optimizes certain performance
measures. This thesis addresses a particular mission – search and prosecute mission –
that require coalition formation and provide algorithmic solutions.
Also addressed in the thesis are multiple UAV missions like rendezvous under
collision avoidance and coalition formation under collision avoidance requirements.
1.2 Collision Avoidance
In any multiple UAV mission, for its successful completion, the UAVs have to avoid
colliding with each other. The problem of multiple UAV collision avoidance can be
described as follows.
The problem
Consider a situation in which several UAVs are flying from different bases to their
respective destinations. These UAVs need to avoid mid-air collision with other UAVs
on their paths. For safety reasons, ‘miss distance’ or the minimum separation between
any two UAVs at any time of flight should be greater than a specified value. Any loca-
tion of two UAVs within the minimum required separation results in a near miss. The
objective is to find an algorithm that, when executed by every UAV, results in no colli-
sions and minimum number of ‘near misses’. Although it is desirable to have zero near
misses, it might be impossible to achieve this in high density air traffic scenarios like
the ones considered in this chapter. Moreover, in the case of UAVs, where there is no
4
U1
U2
U3
U4
U5
U6
safety zone
sensor range
Figure 1.1: An example situation requiring collision avoidance.
human risk, such a high level of safety requirement may not be necessary. So, the em-
phasis is on reducing the near misses while allowing some near misses, at the cost of
which a better efficiency can be obtained. Each UAV needs to achieve the above objec-
tive preferably in a decentralized manner. Situations in which communication among
UAVs may not be possible is considered. This prohibits cooperation via exchange of
information through communication between the UAVs. The collision avoidance ma-
neuvers need to be realistic and the maneuvers have to be efficient in the sense that the
deviation of a UAV from its nominal path due to collision avoidance maneuver should
be minimal to minimize late arrival at its destination. Each UAV needs to find a safe
path to its destination when limited information of positions and velocities, of only the
neighboring UAVs which are within its sensor range, is available.
Note that in the literature one may find the term collision avoidance being used for
the cases where UAVs have to avoid obstacles – both static and moving – which may,
more appropriately, be called obstacle avoidance. Although the problem of obstacle
avoidance can be handled by slight modifications of the collision avoidance algorithms
developed in this thesis, it is not of primary interest in this thesis.
Figure 1.1, which shows mid-air encounter of six UAVs during flight to their re-
spective destinations, gives an example scenario depicting the problem that we address
in this chapter. In the figure, the inner and the outer discs around UAV U1 show the
desired safety zone around the UAV and its sensor range. The same applies to all
other UAVs which have their own respective safety zones and sensor ranges. In this
particular situation, the UAVs U2, U4, and U5 are within the sensor range of UAV U1.
5
Also, UAV U1 may be on a collision course with U2 and U5. The problem of interest
is the development of a decentralized algorithm which, when employed by every UAV
with its limited sensor information, will result in safe paths for all the UAVs in a high
density traffic environment. Although it might appear that, since there is less freedom
for maneuver, in high density traffic situations it is appropriate and mandatory to use
a centralized coordination, Hoekstra et al. (2000), on the contrary, argue that “high
density situations are the ones that require the power of a distributed system.”
Collision avoidance in robotics
Collision avoidance has been an active area of research in the field of robotics. The
prominent ones are the dynamic window approach (Fox et al., 1997), the collision
cone method (Chakravarthy & Ghose, 1998), the velocity obstacle method (Fiorini
& Shiller, 1998; Shiller et al., 2001), and the inevitable collision states approach
(Fraichard & Asama, 2004; Gomez & Fraichard, 2009). These methods are devel-
oped with robotic application in mind. They perform better with more knowledge
about the obstacles (Gomez & Fraichard, 2009) (an information of the position, the
velocity, and the acceleration of an obstacle is considered as ‘more knowledge’ about
the obstacle than an information of its position and velocity alone). However, in cases
where a complete a priori knowledge of the path of an obstacle in not available, the
use of only current position and velocity values of obstacle can at some stage result
in very unrealistic changes in the magnitude and direction of demanded velocity. The
collision avoidance techniques employed widely in robotics, as mentioned above, are
typically computationally very intensive. This is acceptable for robots that can stop
to do the calculations, but not desirable for fixed wing UAVs that require a minimum
forward velocity to keep afloat in air. Since these algorithms are designed for robots,
it is assumed that the vehicles have braking forces and can stop if required and can
change their direction of motion instantaneously. Although these algorithms can be
generalized for multi-vehicle collision avoidance, they are originally designed for one
robot navigating through an environment with obstacles. Due to reasons described
above, the algorithms developed for robotic applications cannot be directly applied to
UAV problems.
Collision avoidance among aircraft/UAVs
There has been some research over the past decade on aircraft collision avoidance both
from the multiple UAV and the air traffic control points of view. All these collision
6
avoidance procedures are based on “See, Detect, and Avoid” principle (Lacher et al.,
2007). Extensive reviews of the collision avoidance algorithms are given from the air
traffic control point of view by Kuchar and Yang (2000); and by Albaker and Rahim
(2009b) from the UAV point of view. They classify the existing algorithms based on
(i) See: the type of sensing used, (ii) Detect: the conflict detection strategy used, and
(iii) Avoid: the selection of escape trajectories, maneuver realization, and the type of
multiple conflict management strategies (Kuchar & Yang, 2000; Albaker & Rahim,
2009b). They describe the possible ways in which collision avoidance algorithms can
differ, which is summarized as follows. The algorithms differ in the types of sensing
used – active or passive; and the sensing equipments used – radars, infrared sensors,
visual cameras, etc. The collision type considered can be planar or three dimensional.
Ways of collision detection include straight line projection of current velocities, con-
sideration of worst case maneuvers, probabilistic estimation of future locations, and
flight plan sharing between aircraft. The collision criterion may be time to collision,
point of closest approach, collision interval, etc. The type of maneuver used, once
a collision is predicted, can be predefined, protocol or rule based, reactive, potential
field based, optimized, or negotiation based. The execution of these maneuvers typ-
ically consist of the use of one or a combination of lateral-directional, longitudinal,
and/or velocity maneuvers. The collision avoidance algorithms also differ in the way
the multiple conflicts are handled. A brief overview of some of the collision avoidance
algorithms, which are relevant to the work in this thesis, is given below.
Collision avoidance in air traffic management
Most of the algorithms developed for air traffic management are those that guaran-
tee safe trajectories in a very low density traffic involving only two or three aircraft.
Tomlin et al. (1998) achieve conflict resolution by modeling the air traffic scenario as
a multi-agent hybrid system. The method developed in (Tomlin et al., 1998), which
is called the ‘roundabout’ method, ensures safe path for cases of two or three air-
craft only. A generalized roundabout method was developed by Frazzoli et al. (2005)
in which a decentralized hybrid control policy is used and is shown that the method
works for higher number of aircraft. In both the approaches, the efficiencies achieved
are poor due to the roundabout paths taken.
Minimum length conflict free paths are obtained by Bicchi and Pallottino (2000)
using optimization methods. As this algorithm is not decentralized, it is implementable
for only a few aircraft. Bilimoria (2000) provides a geometric optimization approach
for conflict resolution. Hwang and Tomlin (2002) propose a protocol-based conflict
7
resolution in air traffic scenarios with finite sensor radius limits. This algorithm in-
volves instantaneous velocity and heading changes, and thus might result in unrealis-
tic maneuvers. Mao et al. (2000) give a collision avoidance algorithm for intersecting
flow of aircraft. However, their algorithm also requires instantaneous changes in po-
sition and heading of aircraft which is unrealistic. Also, it is not applicable in highly
dynamic environments considered in this thesis.
Negotiation based collision avoidance
There are certain algorithms that consider aircraft as agents and deconfliction, in case
of a detected conflict, is achieved by peer to peer negotiation (Sislak et al., 2006,
2007; Albaker & Rahim, 2009a). The aircraft share their flight plans, and modify
them based on negotiation to find safe trajectories. Such an approach can be used for
air traffic management application as well as for UAV collision avoidance. In (Sislak
et al., 2006), the conflict resolution is rule-based and depends on the angle of predicted
collision between the aircraft. This algorithm is more suited for air traffic management
application rather than UAV collision avoidance. Sislak et al. (2007), for each conflict,
evaluate all possible predefined set of maneuvers for the best solution. This approach
can turn out to be computationally intensive. They consider a test case equivalent to the
random flight test case considered in this thesis to test the collision avoidance algorithm
for a high density traffic. Albaker and Rahim (2009a) use a straight projection for
conflict detection and predefined heading and speed changes for deconfliction using
a cooperative peer to peer negotiation. Negotiation requires communication between
aircraft. Therefore, these algorithms cannot be used in adversary environments where
communication may not be possible. Samek et al. (2007) use a multi-party negotiator
instead of a peer to peer negotiation. Simulations they present involve high density
traffic. In this approach, several UAVs having mutual conflicts are grouped together.
This gives the benefits of a centralized solution. The communication and computation
overhead associated with this collision avoidance algorithm is high.
Collision avoidance using artificial potential based methods
Another approach to collision avoidance is artificial potential based methods where
aircraft or UAVs are treated like ‘charged particles’ of same charge that repel each
other; the destination of an aircraft is modeled as a charge of the opposite sign so as to
attract or navigate the aircraft toward the destination. The artificial potential methods
are susceptible to local minima and require breaking forces, and therefore is not widely
8
used in UAV collision avoidance. However, several variants of potential based meth-
ods have been used for aircraft and UAV collision avoidance. Eby (1994), Eby and
Kelly (1999) give a potential field like method where once a violated separation is de-
tected, the velocity vector is changed. This change (acceleration) is proportional to the
predicted miss distance and inversely proportional to the time-to-go. These papers give
simulation results for low density flight scenarios. They study the performance of their
algorithm in the presence of degraded communication and maneuverability. Roussos
et al. (2010) use artificial potential field generated by ‘dipolar navigation functions’ to
effect a collision avoidance. They use a nonholonomic vehicle model with a constraint
on the motion along lateral axis, avoid high yaw rates, and do collision avoidance
simulations for six aircraft. Shim et al. (2003) use a potential function method along
with nonlinear model predictive control to address the problem of collision avoidance
in three dimensions. They demonstrated, in simulation, collision avoidance between
five helicopters which are modeled using six degrees of freedom and realistic con-
trol inputs. Another limitation with the potential field methods is that the computed
solutions are not guaranteed to be feasible once the aircraft dynamic limitations are
imposed. Rahmani et al. (2008), Panyakeow and Mesbahi (2010) use a navigation
function along with a swirling function to take care of constant velocity and turn rate
limits in UAVs, and the mission requirements are incorporated in the potential func-
tions. The potential methods, typically, use only the position information. Ignoring
the velocity information can lead to highly conservative solutions that are inefficient.
Potential function based methods are also computationally intensive.
Collision avoidance based on collision cone approach
Chakravarthy and Ghose (1998) developed the concept of ‘collision cone’ that can
be used to determine whether two objects, of irregular shapes and arbitrary sizes, are
on a collision course. The collision cone approach has been the basis for many colli-
sion/obstacle avoidance algorithms. Since an aircraft with a safety ball around it can be
considered as a moving obstacle of the size of the safety ball, modifications of the col-
lision cone approach can be used for aircraft deconfliction. Lalish et al. (2008), Lalish
(2009) use the collision cone approach in conjunction with a control function which
ensures collision free trajectories in a multi-vehicle scenario with no sensor range lim-
itations, provided the vehicles start in a conflict free situation. This distributed reactive
collision avoidance algorithm applies to both planar and three dimensional engagement
of aerial vehicles (Lalish, 2009). If the vehicles are in a deconflicted state initially, the
algorithm ensures the deconfliction at all later times. The algorithm is not decentral-
9
ized as a UAV implementing this algorithm requires information of all other UAVs.
This may, in general, be not possible and might lead to intensive computational re-
quirements in high density traffic environments. Also, the ‘guaranteed’ deconfliction
can at times be sacrificed for efficiency; an effort to strictly stick to assured separation
at times can lead to very inefficient trajectories. Such a situation can occur more fre-
quently in highly cluttered environments which they do not consider. Another method
based on collision cone approach is due to Smith and Harmon (2010). They make use
of the orientation rate to get a converging (or diverging) cone. A True Proportional
Navigation guidance law is used to drive the velocity vector out of the cone. The al-
gorithm was tested for different engagement geometries of two or three UAVs only.
Nonetheless, they do software-in-loop, hardware-in-loop, as well as real flight tests of
their collision avoidance algorithm.
Collision avoidance based on pairwise deconfliction and geometry
This thesis advocates the handling of multiple conflicts by ‘appropriately’ handling
pairwise conflicts. Therefore, we review some of the one-one deconfliction algorithms
in the literature. Among these are certain collision avoidance optimal algorithms for
pairwise deconfliction. Han and Bang (2004) give a Proportional Navigation (PN)
based algorithm for optimal collision avoidance among two aircraft. Merz (1991)
calculates the optimal maneuver to maximize the miss distance between two aircraft.
Using tools of optimal control, he shows that an acceleration along the predicted near
miss direction is the one that maximizes near miss. Such an acceleration is achieved
through thrust, break, climb, dive, and turns. Park et al. (2008) also use a change in
velocity vector to effect an increase in near miss in the direction of predicted near miss
for a pair of UAVs. Carbone et al. (2006) gives three dimensional geometric algo-
rithm for optimal deconfliction among two aircraft. They provide analytic solutions
for lateral-directional, longitudinal, and speed only control. The most effective control
for collision avoidance is found to be the lateral-directional maneuvers. Their studies
show that the speed only control is not very effective. This justifies the use of constant
velocity maneuvers used in this thesis to effect deconflictions. Most of the above colli-
sion avoidance algorithms use a geometric approach. These algorithms are optimal in
the sense that the velocity change required is minimal or the deviation from the nom-
inal path is minimal. However, the optimality of such algorithms creates a barrier for
them from being used for multiple UAV collision avoidance. Since these algorithms
are optimal only for a pair, they fail in the presence of multiple conflicts.
10
Dowek et al. (2005) develop a pairwise distributed resolution strategy where the
resolution maneuver is either change in vertical speed, heading, or ground speed. They
perform simulations for very high density traffic and report good results. However, no
information on the efficiency of this algorithm is available.
Shin et al. (2008), Tsourdos et al. (2011) give a multiple UAV collision avoidance
algorithm where geometric methods are used to detect pairwise conflict, and then use
a direction or velocity control to effect a deconfliction. They also study the stability of
these conflict resolution strategies.
Game theory based collision avoidance
Hill et al. (2005) and Archibald et al. (2008) use satisficing game theory to address the
problem of collision avoidance in multiple UAVs. These papers present simulations
involving test cases with high density air traffic scenarios to test the proposed algo-
rithm. This algorithm, apart from being computationally intensive, requires constant
communication between neighboring UAVs which may not always be possible. In the
sequel, this algorithm is referred to as Satisficing Game Theory (SGT) based algo-
rithm for collision avoidance and is used for comparison studies with the algorithms
developed in this thesis.
1.3 Coalition Formation
In recent times, there has been some instances of the use of UAVs for search and
prosecute missions. In a typical search and prosecute mission, multiple UAVs are
deployed that cooperate with each other as a team to detect and subsequently prosecute
the detected static and moving targets in a region of interest. Deploying multiple UAVs
for the search and prosecute mission provides robustness to the mission and reduces
the mission completion time.
The problem
Often, UAVs may have to use various types and quantities of resources to completely
prosecute a target. A single UAV may not have sufficient resources to prosecute the tar-
get, and hence a sub-team of UAVs, whose total resources are sufficient to completely
prosecute the target, needs to be formed. This sub-team of UAVs is called a coalition;
and each UAV in the coalition, a coalition member. The UAV that detected the target
11
initiates the coalition formation process, forms the coalition, and is called the coalition
leader. While the primary objective of the mission is to search and prosecute all the
targets, the other objectives are: (i) prosecute the target in minimum time, (ii) mini-
mize the coalition size, and (iii) simultaneously prosecute the target (if possible). The
objective (i) enables the UAVs to quickly accomplish the mission. The requirement
of objective (ii) can be justified as follows. The UAVs have limited sensor range and
therefore need to distribute their search effort efficiently to quickly detect the target. A
small coalition will allow the rest of the UAVs to search for targets resulting in quicker
target detection, while a larger coalition will leave only a few UAVs for distributed
search task, thus reducing the total search effort and increasing the mission time. The
objective (ii) implicitly assists in completing the mission quickly. The objective (iii) is
required to induce maximum damage to the target.
Additionally, the resources of the UAVs, such as fuel and weapons, deplete with
use and the UAVs have kinematic constraints like minimum radius of turn and veloc-
ity bounds. Since the UAVs are in motion, for the coalitions formed to be efficient,
the coalition formation algorithms must be quick with low computational overhead.
Therefore, there is a need to develop coalition formation algorithms that have low
computational complexity, satisfy the objectives (i)–(iii), and take the depletion of re-
sources and kinematic constraints into account.
Gerkey and Mataric (2004) present a taxonomy for the multi-robot task allocation
problem. The problem of coalition formation for a search and prosecute task, accord-
ing to their taxonomy, falls in the Multi Task - Multi Robot (MT-MR) instantaneous
assignment problem category because the UAVs can perform multiple tasks, namely
search and target prosecution tasks, simultaneously, and the targets may require mul-
tiple UAVs to prosecute. This thesis, in particular, addresses the coalition formation
for a search and prosecute mission. However, the abstract formalism adopted in this
thesis toward formulation and solution of the coalition formation problem makes it
applicable to a general task allocation problem.
Although we treat the UAV resources as abstract entities in this thesis, in actuality,
the UAV resource capabilities can include, but are not limited to, visual sensing power
(cameras and radars), weapon power, speed, and maneuverability. Thus, an example
of a multiple UAV mission that requires coalition is tracking and prosecuting a moving
target. Such a task requires UAVs with high visibility sensors that can track the target
and UAVs with adequate weaponry that can destroy the target.
12
Coalition formation in multi-agent systems
In general, a coalition is a group of team members that have agreed to cooperate with
each other to execute a single task (Shehory & Kraus, 1998). The coalitions formed
are temporary by nature; once the task is accomplished, the coalition members can
perform other tasks. Forming a coalition to achieve tasks is an active field of research
in the multi-agent community (Sandholm et al., 1999; Shehory et al., 1998) and the
multi-robot community (Vig & Adams, 2006a, 2005; Parker & Tang, 2006). Deter-
mining the optimal coalition from a group of agents is a computationally intensive
task and is NP-hard (Sandholm et al., 1999). Therefore, the effort has been to find
algorithms that provide approximate, near-optimal, and anytime solutions (Shehory &
Kraus, 1998; Sandholm et al., 1999; Vig & Adams, 2006b; Rahwan et al., 2009).
In multi-agent system, the agents are usually software, their resource being pieces
of codes or data, and task being computation or data mining. Campbell et al. (2002)
considers a task allocation problem in which a group of agents is presented with a se-
quence of tasks, and each agent chooses whether to join a coalition to solve a particular
task. This is typical of most of the coalition formation algorithms. Coalition formation
at large scales have been looked at by Scerri et al. (2008) and Tosic and Agha (2005).
Scerri et al. (2008) considers the problem of coordination of two hundred wide area
search munitions or UAVs to find and destroy ground based targets. However, their
work is more from the point of view of multi-agent systems as they do not consider
the dynamics or kinematics of the UAVs.
Coalition formation in robotics
The coalition formation algorithms developed in the multi-agent community cannot
be directly applied to multiple robot systems (Vig & Adams, 2006a) nor to the multi-
ple UAV systems. This is because, there is a difference in the kind of resources used
by software agents and robots. For software agents, the resource is typically a piece
of code or some data which is transferable between the agents. Whereas, for robots
or UAVs, the resources are sensors, actuators, et cetera which cannot be transferred
between them. The cooperation among UAVs or robots is achieved through informa-
tion exchange and their physical effect on the environment. In multi-agent systems, the
coalition structure depends on the member alone and not on the allocated task, whereas
in multi-robots systems, the purpose of coalition formation itself is task allocation (Vig
& Adams, 2006a).
Vig and Adams (2006a, 2005) develop a coalition formation scheme, where the
13
tasks act as agents and perform the function of an auctioneer for gathering bids and
determining the coalition. This process of forming coalitions is different from the
typical approach where a robot is an auctioneer. In (Vig & Adams, 2006a), the authors
use a negotiation process to form the coalition. It is well known that the negotiation
process requires significant amount of communication and also takes time to form
the coalition. The authors of (Vig & Adams, 2005) pose the problem as a matching
problem that is also computationally intensive. The same authors address the issue of
balance in the coalition between the coalitional size and the resource contribution in
(Vig & Adams, 2006a). The algorithms developed in (Vig & Adams, 2006a), (Vig &
Adams, 2005), or (Vig & Adams, 2006a) require significant amount of computation
time and communication which may not be possible in UAV networks as UAVs move
fast and cannot stop in mid air. Hence, these algorithms cannot be directly applied
to the UAV scenario. Parker and Tang (2006) present a coalition formation scheme
where a coalition leader robot broadcasts the existence of a task and other robots reply
by providing their availability. The leader robot evaluates all possible coalitions and
sends an accept decision to the robots that it considers suitable. The task is executed by
sharing the sensor information. However, they do not take coalition size or minimum
arrival time to the target into account.
Coalition formation in UAV groups
The problem of allocating tasks to UAVs is similar to the multi-robot task allocation
problem. However, there are two differences: first, UAVs travel with greater veloci-
ties than ground robots and cannot stop in midair; and secondly, the resources present
in the robots do not deplete with use or can be easily replenished, whereas UAV re-
sources, like fuel and weapons, deplete with use. Thus, the multi-robot task allocation
algorithms cannot be directly used for multiple UAV missions requiring coalition for-
mation.
The problem of forming a coalition has not been adequately addressed in the UAV
community. Nonetheless there are important contributions some of which we describe
below. Kingston and Schumacher (2005) assign multiple UAVs to track and prosecute
a target using an mixed integer linear programming formulation. The goal is to mini-
mize the length of task tours of the UAVs with known target positions. The resources
of the tracking agents have the same capability and the problem of depleting resources
is not addressed. Many researchers have developed task allocation algorithms for ef-
ficiently allocating UAVs to different tasks. Most of the solutions to task allocation
problems assume the UAVs (a) are homogenous, (b) can carry resources that do not
14
deplete with use, (c) can individually prosecute the target, and (d) can prosecute the
target with any resource (Nygard et al., 2001; Schumacher & Chandler, 2004; Darrah
et al., 2006; Alighanbari & How, 2006; Sujit et al., 2005). Therefore, we cannot use
these algorithms directly to the search and prosecute problem as posed above.
Some works in the literature consider optimal weapon-target or vehicle-target as-
signment problem that was shown to be NP-complete by Murphey (1999). Arslan
et al. (2007) address a generalized version of this problem using game theory, model-
ing it as a potential game, and solving it via the design of appropriate utility functions.
Shima et al. (2009) use mixed integer linear programming, tree search, and genetic
algorithms for the optimal vehicle-target assignment problem in UAV groups. They
consider depletion of resources, like fuel, but do not present a case where resource
limits are reached. The inherent assumption in the above papers is that a search of
the area of interest has already been carried out and the target locations and types are
identified. However, this assumptions limits the use of such algorithms to stationary
targets and therefore cannot be used in the presence of multiple moving/maneuvering
targets as considered in the search and prosecute task that this thesis addresses.
Coalition formation under limited communication
When UAVs with limited communication ranges are considered, the problem of coali-
tion formation assumes a new dimension. The coalition formation process, now, has
to be carried out over a time varying network formed by the UAVs moving around in
search of targets. An additional complexity arises due to the presence of moving and
randomly maneuvering targets. In the approaches described above where coalitions
are formed over groups of communicating robots, the robots do not face a situation
where they can break the communication network during the coalition formation pro-
cess since the robots can stop and form a static network until the coalition formation
process is completed. However, this is not possible in multiple fixed-wing UAV groups
that require a minimum velocity to be airborne.
Toward solving the problem of forming coalitions over a time varying network,
we, in this thesis, use TCP/IP (Transmission Control Protocol/Internet Protocol) like
protocols. Vanzin and Barber (2006) determine a mechanism of finding potential mem-
bers for coalition based on a TCP protocol. However, the technique does not address
the coalition formation issues. Although their algorithm allows for the agents to join
and leave the network at will, they do not consider networks where topology changes
due to the agents moving around. This thesis develops new techniques to determine
coalition in a dynamic network.
15
1.4 Collision avoidance and coalition formation in multiple UAV missions
Collision avoidance and coalition formation find immense applications in a wide vari-
ety of multiple UAV missions. In multiple UAV missions, it is common that the UAVs
are required to rendezvous at a point or a region to exchange confidential informa-
tion and/or share resources. This thesis considers multiple UAV rendezvous problem
where a rendezvous to a predetermined point is desired while the UAVs involved in the
operation have to do evasion maneuvers to avoid collision en route to the rendezvous
point.
Previous works address the multiple agent rendezvous problem in which multiple
agents (not necessarily UAVs) arrive simultaneously to a common location by cooper-
ating with each other (Lin et al., 2003; Tiwari et al., 2004; Lin et al., 2005; Notarste-
fano & Bullo, 2006; Das & Ghose, 2009). In these papers, rendezvous is achieved
via consensus between the agents on their positions and therefore rendezvous does not
occur at a predetermined point but is a function of the initial positions of the agents.
Sinha and Ghose (2006) use a linear cyclic pursuit of agents to attain rendezvous. They
show that, with an appropriate choice of controller gains, a rendezvous to a desired
point can be achieved. All the above papers do not consider the kinematic constraints
of UAVs like turn rate limitations and velocity bounds. Linear cyclic pursuit can be
extended to a nonlinear cyclic pursuit by adding nonholonomic constraint of minimum
radius of turn. In that case, Sinha and Ghose (2007) show that the cyclic pursuit can-
not achieve rendezvous at a point but rather will result in a ‘rendezvous about a point’.
Furukawa et al. (2005) require multiple UAVs to simultaneously arrive at a target. This
is achieved using time-optimal control of orientation and velocity. Although they ac-
count for turn rate limitations and velocity bounds present in a realistic UAV, they do
not consider any obstacle/collision avoidance. McLain and Beard (2005) solve the
optimization problem of minimizing time to rendezvous point, of a group of realis-
tic UAVs, as well as their times of exposure to known threats. However, they do not
consider collision avoidance between UAVs in free flight.
Tsourdos et al. (2005), Shanmugavel et al. (2006), Tsourdos et al. (2011) consider
a problem where multiple UAVs have to arrive at their respective destinations, simul-
taneously; the UAVs have to do so while avoiding collisions among themselves and
with obstacles. The authors of the above cited papers give solutions for this problem
based on Dubins, clothoid, and Pythagorean hodograph paths. Although these papers
consider rendezvous only in time and not in space, unlike the problem that we consider
16
in this thesis, similar techniques can be used to address the problem where UAVs have
to rendezvous not only in time but also in space.
The multiple UAV rendezvous algorithm developed in this thesis can be easily
modified for standoff tracking of stationary and moving targets. Standoff tracking finds
applications like surveillance, reconnaissance, and convey protection, among others.
The approach in the literature toward this problem is to use Lyapunov guidance vec-
tor fields (Frew et al., 2008; Summers et al., 2009). The consensus based algorithm
provided in this thesis is conceptually simpler than the methods given by above papers.
Another multiple UAV mission this thesis addresses is that of coalition formation
for a search and prosecute task in the presence of collision avoidance. This problem is
novel and has not been addressed before in the literature.
1.5 Contributions and organization of the thesis
We briefly give the outline of the thesis highlighting the major contributions.
In Chapter 2, we address the problem of collision avoidance among UAVs in
high density traffic situations, and develop simple and efficient algorithms for conflict
resolution. First, an algorithm is developed to handle planar conflicts. This thesis han-
dles multiple UAV collisions by appropriately handling pairwise conflicts, and uses
principles from missile guidance to effect conflict resolution. It is shown, through
simulations, that the proposed conflict resolution strategy handles pairwise conflicts
correctly. Simulations involving high density traffic are presented to demonstrate the
efficacy of this algorithm. Next, we develop algorithms to handle conflicts that are
inherently three dimensional. Two algorithms are developed to address three dimen-
sional conflicts, one of which is the extension of the proposed planar algorithm. Again,
through simulations, we show that these algorithms perform well in high density traffic
environments that are of interest to this thesis.
In Chapter 3, we demonstrate the ease of implementation of the conflict resolu-
tion algorithms developed in Chapter 2. Six degrees of freedom UAV models are used
as the platform to implement the collision avoidance algorithms. For planar collision
avoidance, separate altitude, attitude, and velocity hold controllers are designed, using
successive loop closure technique, to realize the guidance commands issued by the col-
lision avoidance algorithm. Flight path angle, course, and velocity hold controllers are
used to implement the three dimensional collision avoidance algorithm. These realis-
tic UAV models, augmented with controllers, using the collision avoidance algorithms
17
developed in this thesis are shown to have good collision avoidance performance in
simulations involving very high density traffic.
In Chapter 4 we consider the problem of coalition formation among multiple
UAVs for a search and prosecute mission. First, we prove that the coalition forma-
tion problem that we are tackling is indeed NP-hard. Then, a two stage polynomial
time algorithm is developed to give sub-optimal, but computationally efficient and
simple, solution to the coalition formation problem involving large number of targets
and UAVs. We solve the global optimization problem of target prosecution, when
the target locations are know a priori, using Particle Swarm Optimization algorithm.
Knowing the locations of targets a priori makes the problem that of optimal sequenc-
ing, with additional complexities introduced by resource requirements and resource
depletion. Comparison of the results shows the near optimality of the polynomial time
coalition formation algorithm that we propose.
In Chapter 5, we consider the coalition formation problem for search and prose-
cute mission when the UAVs have limited communication ranges. This situation will
occur in missions with large number of small UAVs. The UAVs being small will have
small communication ranges causing a dynamic communication network to be formed
by the UAVs exploring the environment in search of targets. In this case, we give a
communication protocol that enables the UAVs to form coalitions over time varying
networks. We study the dependence of the performance of this protocol on the mis-
sion and the network parameters, and show that our algorithm can be used in UAV
dense environments. In this chapter, we also consider the case where the targets are
moving/maneuvering.
In Chapter 6, we consider certain multiple UAV missions that require collision
avoidance and/or coalition formation among UAVs for the successful completion of
the mission. First, a multiple UAV rendezvous problem is considered under collision
avoidance. A consensus based algorithm that we develop makes rendezvous possible,
while the collision avoidance algorithm of Chapter 2 takes care of the conflicts en
route rendezvous. Next, we consider coalition formation under collision avoidance.
Simulations show that the collision avoidance and the coalition formation algorithms
that we developed can be seamlessly integrated into UAVs, for multiple UAV missions,
depending on the mission requirements without much interference between them.
Chapter 7 concludes the thesis with a summary, discussion of main results, and
directions for future research.
18
1.6 A note to the reader
All symbols and abbreviations used in the thesis are defined at the place of their first
use. For the convenience of the reader, the commonly used notations in the thesis
are given in the preamble. Some of the notations that we use are standard in the
literature. For example, the symbols used to denote the stability and control derivatives
in Chapter 3 are standard in the flight dynamics community, for instance (Stevens &
Lewis, 2003). Some notations are specific to a particular section of the thesis. At
certain places in the thesis, we use the same notation for different concepts if this does
not cause ambiguity, and in such cases, the correspondence is made clear from the
context. Since a multiple UAV system can be looked upon as a multi-agent system, at
some places in the thesis, the words agent and UAV are used interchangeably.
19
20
2 Collision avoidance among multiple UAVs
This chapter develops reactive algorithms for collision avoidance among multipleUAVs for both two and three dimensional conflicts. The main idea proposed isthat the collision avoidance among multiple UAVs can be achieved by effectivelyhandling the pairwise conflicts. The performance of the algorithms developed areanalyzed through extensive simulations.
�ultiple UAV missions lead to many emergent subproblems that need to be
addressed effectively. For any mission involving multiple UAVs, a com-
mon subproblem is that the UAVs may collide with each other. UAVs oc-
cupying the same airspace need to avoid each other to ensure a safe completion of
respective missions. The problem assumes great importance in a high density mul-
tiple UAV traffic scenario. In this chapter, we develop a simple and novel collision
avoidance algorithm to resolve conflict among UAVs that are on collision course in
an airspace occupied by a large number of UAVs. These algorithms are conceptu-
ally simple, computationally efficient and easily implementable. We show the efficacy
of the developed collision avoidance algorithms through simulations using kinematic
UAV models and high density air traffic scenarios where most of the possible conflict
geometries are expected to occur.
The organization of the chapter is as follows. Section 2.1 discusses some basic
concepts, issues, and assumptions, and gives an overview of our solution to the multi-
ple UAV collision avoidance problem. In Section 2.2 we develop a collision avoidance
algorithm to resolve planar conflicts. The algorithm is extended to handle 3D conflicts
in Section 2.3 and then a genuinely 3D collision avoidance algorithm is proposed. In
Section 2.4 we give the implementation of these algorithms. Section 2.5 presents the
simulation results. In Section 2.6 we conclude the chapter with a summary of the main
contributions.
2.1 Preliminary Discussion
First, we discuss some basic concept, issues, and assumptions, and provide an outline
of the proposed solution.
21
2.1.1 Some basic concepts and assumptions
We make the following assumptions regarding the kinematics of a UAV and its sensor
capabilities. It is assumed that a UAV has limited sensor range and knows exactly the
positions and velocities of all UAVs within its sensor range. However, through simu-
lations, we show that the collision avoidance algorithm that we develop in this chapter
also perform well in the presence of noise in position and velocity measurements. Al-
though it is assumed that the UAVs have sensors, in situations where it is desirable
to turn off the sensors to avoid getting detected, the UAVs can get information about
their neighbors through inter-UAV communication. This is possible in a cooperative
environment. Broadcast mechanisms like automatic dependent surveillance-broadcast
(ADS-B) (RTCA, 2002) can also be used in such situations, if the UAVs are ADS-B
enabled. We consider UAVs with a kinematic constraint of minimum radius of turn
which implies that a UAV cannot instantaneously change its heading.
Initially, we assume a kinematic model for UAVs as given below. However, in the
next chapter, we consider a realistic UAV model with six degree of freedom dynamics.
The kinematics of the UAV Ui with position (xi,yi), velocity Vi, and heading ψi is
xi = Vi cosψi
yi = Vi sinψi
ψi = ai/Vi (2.1)
where, ai is the commanded lateral acceleration issued by the collision avoidance guid-
ance algorithm. If Rmini is the minimum radius of turn, then |ai| ≤ V 2i /Rmini . The set
of equations in Eq. (2.1) give the two dimensional kinematics. In three dimensional
case too, we assume a similar kinematics where a lateral acceleration perpendicular to
the velocity vector provides the required turn rate. The details of two and three dimen-
sional kinematics used for simulations in this chapter are given in Section 2.4, which
explains the implementation details of the collision avoidance algorithms proposed in
this thesis.
2.1.2 Overview of the proposed solution
A broad overview of the solution that we propose in this chapter for the multiple UAV
collision avoidance, while the UAVs fly from their bases to respective destinations,
is as follows: When a UAV envisages a near miss with another UAV, it switches to
a collision avoidance mode. The collision avoidance mode of a UAV comprises the
22
following: (a) finding a UAV, from among the neighbors, which is most threatening,
and (b) performing a turn maneuver until the projected trajectories of the UAV and that
of the most threatening UAV do not conflict. At any instant of time, a UAV might have
predicted conflicts with several of its neighbors. The most threatening neighbor for a
UAV is the one with which the predicted near miss is the most immediate to occur.
The turn maneuver performed is one which ensures an increase in the Line-of-Sight
(LOS) rate between the concerned UAVs – a UAV and its most threatening neighbor.
This is inspired by and contrary to missile guidance principles where missile takes a
turn to decrease its LOS rate with the target (Zarchan, 1990) to effect a collision. The
collision avoidance maneuver would have taken the UAV out of its nominal path. It is
required that the arrival of a UAV at its destination has to be as quick as possible. The
UAV therefore takes a Dubins path (Dubins, 1957), which is a minimum time path,
from its current location to the destination.
We also extend this algorithm to three dimension (3D) which becomes readily
applicable to cases where engagements are inherently 3D. The two dimensional (2D)
collision avoidance algorithm proposed becomes a limiting case of this algorithm when
the engagement tends to be planar. Also proposed is a 3D algorithm which makes the
UAVs execute a 3D maneuver even for planar conflicts. The 3D maneuvers are ad-
vantageous in case of high density traffic, as the 3D deconfliction maneuver will take
UAVs to different altitude planes, thus reducing the possibility of subsequent conflicts.
This will become evident in the simulations presented later. The collision avoidance
mode in 3D consist of a UAV applying an acceleration in or out of its collision plane.
A collision plane for a UAV is defined as a plane that contains the UAV and the veloc-
ity vector of a UAV with which it has a predicted miss distance less than the required
separation and a minimum time-to-go. The application of acceleration in the colli-
sion plane leads to the 3D extension of the proposed 2D algorithm. The out of plane
application of acceleration pulls a UAV out of the collision plane resulting in a 3D col-
lision avoidance maneuver even for planar conflicts. In the 3D case also, a UAV heads
to its destination, after a deviation from its nominal path due to collision avoidance
maneuver, via Dubins path on a plane containing the UAV’s velocity vector and the
destination.
The central thesis of this chapter is that in a multiple UAV conflict scenario, ev-
ery UAV doing a maneuver to increase its LOS rate with a UAV with which it has the
most imminent projected near miss, results in a good collision avoidance performance.
Toward establishing this concept, we show, through simulations, that the resulting col-
lision avoidance algorithm handles pairwise conflicts correctly and every UAV using
23
this algorithm in a highly cluttered environment will result in the reduction of occur-
rence of near misses.
The collision avoidance algorithms that we develop in this chapter, implemented
by a UAV, use the position and velocity information of only those UAVs that are within
a UAV’s sensor range. The algorithms are decentralized, computationally less de-
manding, and require no communication among UAVs. They also perform well in the
presence of noise and disturbances, and are easy to implement.
2.2 Two dimensional collision avoidance algorithm
The Proportional Navigation (PN) guidance law (Guelman, 1971; Ghawghawe &
Ghose, 1996) is very robust and elegant, and its variants are therefore most widely
used for missile guidance. The PN guidance law applies a lateral acceleration to the
missile to nullify the rate of rotation of Line of Sight (LOS) or the LOS rate between
the missile and the target, and brings the missile into a collision course with the target.
For collision avoidance, a lateral acceleration which takes a vehicle away from the
collision course is what is desirable. We propose an algorithm that uses an ‘inverse’
PN for collision avoidance, in the sense that the algorithm consist of applying a lateral
acceleration to increase the LOS rate between UAVs that are on a collision course.
2.2.1 Conflict detection
The first step toward employing the proposed collision avoidance rule for a UAV is
to calculate the envisaged miss distance or the Zero Effort Miss (ZEM) with all other
UAVs which are in its sensor range. The sensor range of a UAV is assumed to be
defined by a circle of radius Rsen (sensor range radius) around the UAV. It is assumed
that every UAV has the capability to locate other UAVs in its sensor range and measure
their velocities.
In a 2D engagement scenario, consider two UAVs, U1 and U2. Let the current
positions and velocities (that is, at time t) of U1 and U2 be p1 = (x1,y1) and v1 =
(u1,v1), and p2 = (x2,y2) and v2 = (u2,v2), respectively. To keep the equations simple,
throughout this thesis unless otherwise specified, the current values of the quantities
that explicitly depends on time are represented by the variables denoting the quantities
themselves. For instance, the current position of Ui is represented as pi rather than
pi(t). At any future time t + τ, the new locations of U1 and U2 are p1 + v1τ and
p2 +v2τ , provided that the UAVs do not change their velocities. We consider collision
24
avoidance against UAVs with different but constant speeds, and use turn rates only
– no speed changes – for collision avoidance. This leaves the degree of freedom of
speed change to be used by other algorithms that co-exist with the collision avoidance
algorithm in a UAV. An example of this is later given in Chapter 6. If the separation
between the two UAVs at time t + τ is r, then
r = ‖(p1 +v1τ)− (p2 +v2τ)‖ (2.2)
where ‖ ‖ is the Euclidian norm and we get
r2 = (p1−p2)·(p1−p2)+2(p1−p2)·(v1−v2)τ +(v1−v2)·(v1−v2)τ2 (2.3)
Solving for τ from drdτ = 0 will give the time at which a minimum separation between
UAVs will occur. We call this the time-to-go (tgo) which is given as
tgo =−(p1−p2)·(v1−v2)(v1−v2)·(v1−v2)
(2.4)
If tgo > 0, then there is a time at which closest approach between two UAVs occur. The
tgo calculated as above can take negative values which means that the vehicles are on a
diverging path. That is, their paths extrapolated backward in time lead to a minimum
distance between them and thus a negative tgo. Substituting tgo for τ into the expression
for r in Eq. (2.2) will give the predicted miss distance or ZEM which is the distance of
closest approach if the UAVs proceed in their current course. Let �p = p1−p2, and
�v = v1−v2. Then
ZEM =
√‖�p‖2‖�v‖2−‖�p·�v‖2
‖�v‖2 (2.5)
In the computation of the predicted miss distance as above, it is assumed that the
heading of the UAVs do not change. Although this assumption is not strictly valid,
since the UAVs update the predicted miss distance at every instant using instantaneous
velocities, the near miss predicted using Eq. (2.5) in the course of time would approach
the ‘actual near miss’ which is obtained by taking into account the turn rates of the
UAVs. The approximate computation of predicted near miss is used in the algorithm
to keep the algorithm simple and to reduce the number of computations in view of
real time implementation. In principle, it is possible that the approximate near miss
computed as above would cause a UAV to execute unnecessary maneuvers that might
lead to reduced efficiency. The simulation results presented later show that the loss in
efficiency due to this reason, however, is less.
25
If the obtained miss distance is less than the desired separation, then a collision
avoidance maneuver has to be performed. A UAV makes a list of UAVs in its neighbor-
hood with which its miss distance is less than the minimum required separation. From
this list, the UAV chooses that UAV with which it has minimum positive tgo, which
means that they are on an immediate collision course, and does a collision avoidance
maneuver.
2.2.2 Collision avoidance maneuver
The collision avoidance maneuver that we propose involve UAVs pulling out of the
collision course. To pull out of a collision course, a UAV has to apply a lateral ac-
celeration in a way that will increase the rate of rotation of the LOS connecting the
two UAVs. This is illustrated using Fig. 2.1. Let r be the LOS separation. For the
engagement shown in Fig. 2.1, the LOS rate with respect to U1, is given as,
θ =V2 sinθ2−V1 sinθ1
r(2.6)
To increase the LOS rate in this scenario, accelerations a1 and a2 as follows should be
applied to U1 and U2, respectively, as shown in Fig. 2.1.
ai = sgn(θ)V 2
i
Ri, i = 1,2 (2.7)
where, sgn(θ) is +1 if the LOS rotates anticlockwise and −1 otherwise. Here, R1 and
R2 are the required radii of turn for U1 and U2. For a UAV with minimum radius of
turn Rmin, the required radius of turn is computed as
R = Rmin exp
(λ ×ZEM
Rdes
)(2.8)
which is an empirical relation that we propose. Here, the desired separation Rdes and
λ are tuning parameters for the algorithm. The desired separation, Rdes, takes a value
greater than the required minimum separation. Appropriate values for Rdes and λ are
obtained through Monte Carlo simulations. The choice of these tuning parameters is
a trade-off between the efficiency and the number of near misses. Section 2.5 gives a
study of the performance of the proposed collision avoidance algorithm as these tuning
parameters vary.
The direction of application of the lateral acceleration is such that the LOS rate
between the UAVs would increase. There are only two directions perpendicular to the
velocity of a UAV along which a lateral acceleration can be applied, one of which will
bring the UAV closer to collision course while the application in the other direction
26
V1
V2
a1
a2
θ1
θ2
rLOS
U1
U2
θ(x1,y1)
(x2,y2)
x
y
Figure 2.1: A 2D engagement between two UAVs.
Algorithm 2.1: Deconfliction algorithm used by UAV Ui
Find neighbors Ni that are other UAVs within the sensor range Rsen of Ui.
Calculate ZEM between Ui and each UAV Uj ∈Ni.
Find N ∗i , the neighbors with which Ui has ZEM < Rdes.
if N ∗i is not empty then
Choose Uj∗ ∈N ∗i with which Ui has least tgo.
Turn with radius of turn R to increase LOS rate between Ui and Uj∗.
end if
will cause an increase in the LOS rate thus aiding the UAV to pull out of the collision
course.
The collision avoidance algorithm proposed for a multiple UAV conflict scenario
is given in Algorithm 2.1.
The proposed collision avoidance algorithm has the effect of increasing the LOS
rate between two UAVs in a collision course. As mentioned earlier, this is exactly
opposite to the PN philosophy that applies a lateral acceleration proportional to the
LOS rate between the missile and the target, and tries to nullify the LOS rate. A
simplistic implementation of a PN based avoidance law would be to apply a lateral
acceleration inversely proportional to the LOS rate. In Eq. (2.8), the turn radius is
a strictly monotonously increasing function of ZEM. When the ZEM is zero, a UAV
executes a collision avoidance maneuver with the tightest turn. Whereas, for a higher
27
ZEM, as there is a lower risk of collision or near miss, the turn rate of the collision
avoidance maneuver of a UAV is considerably less, and thus R is higher, to minimize
deviation from the nominal path. Note that, in the algorithm, although the philosophy
of increasing the LOS rate is used, the maneuver itself does not use the LOS rate in its
expression.
The proposed multiple UAV collision avoidance algorithm operates by deconflict-
ing the most imminent pairwise collisions. However, this control action may lead a
UAV into a collision course with another UAV. Nonetheless, the central thesis of this
chapter holds that, deconflicting critical pairwise conflicts suffices to address multiple
UAV collision avoidance. Although there is no guarantee that all the conflicts will be
eventually resolved, the extensive simulations presented later reassures that it is most
likely the case. Since the proposed algorithm relies on deconflicting the most critical
conflict, it is important to show that the proposed algorithm handles pairwise con-
flicts correctly. In the following subsection, the effectiveness of the proposed collision
avoidance algorithm in handling pairwise conflict is studied.
2.2.3 Effectiveness of handling pairwise conflicts
Consider an engagement of two UAVs as in Fig. 2.1. As seen in the figure, the UAVs
U1 and U2 have an engagement geometry, determined by θ1 and θ2, and an initial
separation r. If these UAVs execute a collision avoidance maneuver by taking a turn
of constant radius R in the directions (along a1 and a2) as indicated in the figure,
then the LOS separation between them initially decreases, reaches a minimum and
then increases. The evolution of the LOS separations between U1 and U2 in time,
when UAVs execute a tightest radius of turn Rmin, is shown in Fig. 2.2 (solid curves)
for different engagement geometries (θ1 and θ2), constant velocity of V1 = V2 = V =
500 miles per hour (223.5 m/s), and an initial separation of 10 miles. These values,
that are typical of a large aircraft rather than a UAV, were chosen as we, in a later
section, compare the results obtained using our algorithm with that of an algorithm in
the literature (Archibald et al., 2008) which considers deconfliction of large aircraft in
high density traffic environments. The Air Traffic Control (ATC) regulations require
that the desired separation between two aircraft at any point of time in flight be greater
than 5 miles. As the ATC regulations are specified in miles, we, in this chapter, have
used miles as the standard unit. However, in the next chapter, where we demonstrate
the efficacy of our algorithm implemented on realistic UAV models, we use the data
corresponding to small UAVs whose dimensions are in the order of meters, as it is
28
0 5 10 15 20 256.5
7
7.5
8
8.5
9
9.5
10
time (sec)
min
imum
sep
arat
ion
(mile
s)
R = Rmin
R as in (2.8)
θ1=15°, θ
2=45°
θ1=0°, θ
2=45°
θ1=0°, θ
2=0°
Figure 2.2: Variation of the LOS separations between two UAVs with time for a turnradii R = Rmin and R as in Eq. (2.8) for various initial engagement geometries.
envisaged that small UAVs rather than large aircraft are more likely to be used in high
density traffic operations.
We make the following observations from the simulations, as described above, of
pairwise engagements of UAVs. From Fig. 2.2, we observe that the most critical en-
gagement is head-on (that is, θ1 = 0 and θ2 = 0). Also, we note that the minimum
separation and the time at which it occurs depends on the engagement geometry. Be-
low, we derive a closed form expression for the time at which minimum separation
occurs from which we will see that the minimum separation and the time at which it
occurs not only depends on θ1 and θ2 but also on initial separation r and the turn radius
R.
The maneuver as described above involves, as shown in Fig. 2.3, the UAVs U1 and
U2 taking turns along circles C1 and C2 of radius R each. Without loss of generality,
we can assume that the UAVs U1 and U2 at any time t are at (0,0) and (r,0). For this
case, the center (xc1 ,yc1) of circle C1 is given as
xc1 = −Rsinθ1
yc1 = Rcosθ1 (2.9)
The center (xc2 ,yc2) of circle C2 is
xc2 = r−Rsinθ2
yc2 = −Rcosθ2 (2.10)
29
(xc2,yc2
)
(x2,y2)
(xc1,yc1
) (x1,y1)
R
R
rDj
DjU1
U2q1 q2
C1
C2
V
V
x
y
Figure 2.3: Collision avoidance maneuver for a 2D engagement.
After a time Δt, the heading change for each UAV is Δψ = ψΔt = (V/R)Δt. Let
(x1, y1) be the position of U1 after Δt time. From Fig. 2.3, we get
x1 = xc1 +Rcos(
Δψ−(π
2−θ1
))= xc1 +Rsin(Δψ +θ1)
y1 = yc1 +Rsin(
Δψ−(π
2−θ1
))= yc1−Rcos(Δψ +θ1) (2.11)
Similarly, let (x2, y2) be the position of U2 after Δt time. Then
x2 = xc2−Rsin(Δψ−θ2)
y2 = yc2 +Rcos(Δψ−θ2) (2.12)
Thus, the separation between U1 and U2 at Δt is given as
r =√
(x1− x2)2 +(y1− y2)
2 (2.13)
We differentiate r with respect to Δψ and set the resultant to zero to find that value of
heading change Δψ at which the minimum separation occurs. This is obtained as
Δψmin = arctan
[r(cosθ1 + cosθ2)
2R{1+ cos(θ1 +θ2)}+ r(sinθ2− sinθ1)
](2.14)
Note that the heading change obtained depends on the initial engagement parameters
(θ1,θ2,r) and the radius of turn R. If V is the magnitude of velocity of UAVs, then
the heading rate during a turn of radius R is ψ = V/R. If the minimum separation
between the UAVs occurs after a time Δt from the time of initial separation, then
Δψ = (V/R)Δt. Therefore, for a given speed and radius of turn, the time Δt after
30
−100 −50 0 50 1006.5
7
7.5
8
8.5
9
9.5
10
θ2 in degrees
min
imum
sep
arat
ion
(mile
s)
R = Rmin
R as in (2.8)
θ1 = −45°
θ1 = 45°
θ1 = 0°
θ1 = 90°θ
1 = −90°
Figure 2.4: The minimum LOS separations attained between two UAVs for radii ofturn Rmin and R as in Eq. (2.8) (various initial engagement geometries).
which a minimum separation occurs depends on θ1 and θ2, since Δt, which is given
as (R/V )Δψ , depends on Δψ . This is evident from Fig. 2.2. For different initial
engagements, the evolution of the LOS separations between the UAVs with time when
the UAVs use maneuvers that depend on the predicted miss distance (that is, radius of
turn R used is as in Eq. (2.8)) are shown in Fig. 2.2 as dotted curves. Although the
effort is less in the case where R in Eq. (2.8) is used, the minimum separation achieved
is also less but is still more than the required separation.
Figure 2.4 gives a plot of how the minimum separation between two UAVs varies
with θ2 for different values of θ1. Cases where θ1 = {−90,−45,0,45,90} degrees are
plotted in the figure. The minimum separation plotted is the closest approach between
UAVs U1 and U2, when the initial engagement geometry is (θ1,θ2), and the UAVs
execute a maneuver of tightest turn with radius Rmin. It is observed that the critical
case is one where θ1 = θ2 = 0◦, that is, a head-on approach. In the algorithm of this
paper, however, instead of the tightest turn maneuver an instantaneous turn radius R, as
given in Eq. (2.8), is used which depends on the predicted miss distance between the
UAVs at each instant. The minimum separation between two UAVs for different initial
geometries for this case is also given in Fig. 2.4. The figure shows that the minimum
separation is less than that in the case where maneuvers involve the tightest turn but is
well above the required minimum separation of 5 miles even for the case of head-on
approach.
31
Algorithm 2.2: Algorithm to implement Dubins path with free terminal constraints
if UAV heading points to D then
Head straight.
else if C+ encircles D (D strictly within C+) then
Take a path along C−.
else
Take a path along C+.
end if
(refer Fig. 2.5 for definitions of D , C+, and C−)
2.2.4 Dubins path to destination
Given initial position and departure angle, and final position and arrival angle, Du-
bins curve or path gives the trajectory with minimum path length under the additional
constraint of a maximum allowed curvature of the trajectory (Dubins, 1957). In the
multiple UAV collision avoidance scenario, once a UAV has deviated from its nom-
inal path after a collision avoidance maneuver, it is desirable to reach its destination
in minimum time. Following a Dubins path to destination will help in achieving this
because, with a constant speed assumption, a minimum length path is also a minimum
time path. If every UAV chooses a Dubins path (minimum length path) from their cur-
rent locations to their respective destinations, it provides an easy method to measure
the deviation due to collision avoidance maneuvers in terms of the delay in the arrival
of a UAV at its destination. For arrival of a UAV at destination in minimum time, a
special case of Dubins curve is required where there is no arrival angle constraint. The
maximum curvature constraint takes care of the minimum radius of turn of the UAV. A
switching algorithm is proposed as follows that implements this special case of Dubins
path.
Let D be the destination, C+ a circle, whose radius is equal to the radius of min-
imum turn, tangential to the velocity vector and such that following C+ will take the
UAV toward the destination, and C− a similar circle such that following C− will take a
UAV away from the destination (refer Fig. 2.5). Then, a UAV employing Algorithm 2.2
32
Destination
C+
C- D
(a) Case were destination is outside the circle (C+)of minimum radius of turn
DestinationC+
C-
D
C+
(b) Case were destination is within the circle(C+) of minimum radius of turn
Figure 2.5: Dubins paths to destination – two possible scenarios (the case of freeterminal constraint).
will follow a Dubins path to the target. Figure 2.5 shows the possible cases of Dubins
path to a destination when there is no terminal angle constraint. Figure 2.5(a) gives
Dubins path to destination in a case where the destination is far from the UAV and
Fig. 2.5(b) illustrates the case where the destination is too close (within C+).
This algorithm is implemented with a slight modification. During simulation, a
UAV follows Algorithm 2.2 till the difference between its desired heading, in which
case the UAV’s velocity vector points directly to its destination, and actual heading is
less than a small quantity ε > 0. Beyond that, the UAV uses a proportional controller
of the form ψ = k(ψd−ψ), where ψ is the heading angle and ψd is the desired heading
of the UAV. This avoids chattering that might occur otherwise due to switching.
2.2.5 RCA-2D: two dimensional collision avoidance algorithm
Combining the algorithms for conflict resolution and heading to destination maneu-
vers, we obtain the collision avoidance algorithm, that each UAV flying to its destina-
tion implements at every time step in a multi-UAV scenario, in Algorithm 2.3. Since
a UAV reacts to the situation of a violated desired separation this 2D algorithm is
referred to as Reactive Collision Avoidance (RCA-2D) Algorithm in this thesis.
RCA-2D handles the multiple UAV scenario by considering only that collision
which is immediate to a UAV in terms of the minimum tgo. This approach results in
a good collision avoidance algorithm with good performance as we will demonstrate
through simulations. However, it is possible that by considering only the immediate
collision and doing a collision avoidance maneuver may at times lead the UAVs to
other conflicts which are too close to avoid. Also, there is a chance of chattering
33
Algorithm 2.3: Reactive Collision Avoidance 2D Algorithm (RCA-2D)
Use Algorithm 2.1 detect and avoid near misses.
if there are no predicted near misses then
Use Algorithm 2.2 for homing to destination.
end if
when a UAV faces two nearly similar conflicts – avoiding one will lead to the other
and vice versa. This usually occurs when engagement geometries have some sort of
symmetry. In such cases where engagements are close to symmetric, it is desirable to
follow a protocol which will break the symmetry and then proceed to use RCA-2D.
We remark that this algorithm can easily handle heterogeneous group of UAVs with
different velocities and radii of turn by using tuning parameters specific to individual
UAVs instead of global ones. However, to keep the situation simple, we consider only
a homogeneous group of aircraft for simulations presented in this chapter. Simulations
involving heterogeneous UAVs are presented in Chapter 3.
2.3 Three dimensional conflict avoidance
The 2D collision avoidance algorithm, developed in the last section, is simple in con-
cept and implementation. However, in high density air traffic scenarios, the number
of near misses and the deviations of UAVs from nominal paths to their respective des-
tinations due to collision avoidance maneuvers restricted to 2D may still be higher
than tolerable levels. Hence, it seems reasonable to take advantage of the unused third
dimension rather than trying to improve the 2D algorithm that may result in compli-
cated and computationally demanding solutions. Also, if the initial engagement (that
is, when UAVs appear in each other’s sensor range for the first time) is non-planar,
which probably may be the case most of the time, then a 2D deconfliction algorithm is
not helpful. A sample situation in which the conflicts among UAVs are in 3D is shown
in Fig. 2.6. The inclusion of 3D maneuvers into the set of possible collision avoidance
maneuver candidates greatly improves the freedom to maneuver. We devise a 3D col-
lision avoidance algorithm using similar ideas as in the previous section. Even in the
3D case, the key idea will be to increase the rate of rotation of the LOS, in the spirit
34
−100−50
050
100
−100
−50
0
50
−40
−20
0
20
x−range
y−range
heig
ht
Figure 2.6: A sample situation requiring 3D collision avoidance among multipleUAVs.
of inverse PN, to facilitate collision avoidance. We propose a 3D collision avoidance
algorithm as follows.
2.3.1 Collision prediction
As in the 2D case, a UAV computes the predicted miss distances between all of its
neighbors and itself, and if it is less than the required separation, it does a collision
avoidance maneuver. The predicted miss distances are calculated exactly as in the 2D
case except that the vectors have an additional component that corresponds to the third
dimension. Let U1 and U2 be two UAVs located at p1 = (x1,y1,z1) and p2 = (x2,y2,z2)
with velocity vectors v1 = (u1,v1,w1) and v2 = (u2,v2,w2) of magnitude V at time t.
At time t + τ, if r is the separation between these UAVs, then
r = ‖(p1 +v1τ)− (p2 +v2τ)‖ (2.15)
The closest approach occurs at a time tgo obtained by solving for τ from dr/dτ = 0.
This gives an expression for tgo as
tgo =−(p1−p2)·(v1−v2)(v1−v2)·(v1−v2)
(2.16)
These equations are exactly the same as Eqs. (2.2) and (2.4) in the 2D case. If tgo >
0, the separation between the UAVs will reduce over time until tgo at which time a
minimum separation occurs. The value of minimum separation can be obtained by
plugging in tgo for τ in the expression for r. An expression same as Eq. (2.5) is obtained
35
for the zero effort miss. If the calculated minimum separation is less than the desired
separation, then a collision avoidance maneuver needs to be performed.
Central to the 3D collision avoidance algorithms is the idea of a ‘collision plane’
which is defined as a two dimensional plane in 3D on which if a UAV stays will re-
sult in a collision with another UAV, provided that both the UAVs do not alter their
directions. In the 3D extension of the 2D collision avoidance algorithm, the velocity
of a UAV is projected onto its collision plane and then the 2D algorithm is applied on
the collision plane to obtain a deconfliction. This reactive 3D collision avoidance al-
gorithm in which the acceleration for collision avoidance maneuver is on the collision
plane is referred to as RCA-3D-I, where ‘I’ stands for the ‘maneuver in the collision
plane’. In the purely three dimensional collision avoidance algorithm, the idea of colli-
sion avoidance by pulling out of collision course in 2D is incorporated in 3D as pulling
out of the collision plane. This algorithm will be referred to as RCA-3D-O, where ‘O’
is for the ‘maneuver out of the collision plane’. The choice of a collision plane from
among the infinite number of possible planes in 3D and the rationale for doing so is
explained in the next subsection.
2.3.2 Collision avoidance maneuver
In 2D, there is a unique way to do the PN inspired collision avoidance. In an engage-
ment in two dimensional plane, there are only two directions in which an acceleration
perpendicular to velocity can be applied – application of acceleration in one direction
will take the vehicle toward the collision triangle while the other will pull the vehicle
out of the collision course. In 3D, there is an infinite choice of directions, all of which
lie on a plane perpendicular to the velocity vector of the UAV, along which an acceler-
ation can be applied to increase the LOS rate and thus effect collision avoidance. The
main idea behind the design of a 3D algorithm for collision avoidance is to choose a
direction, out of these infinite ones, along which an acceleration has to be applied to
achieve a collision avoidance. The direction in which acceleration is to be applied is
chosen as the one which will pull the UAV out of the collision plane. The rationale for
the specific choice of collision plane and the acceleration direction is as follows.
Analysis of guidance laws has shown that they exhibit a time scale behavior (Naidu
& Calise, 2001). It has been recently shown that, in the 2D case, better guidance laws
have larger time scale gap (Dhananjay et al., 2010) which essentially means that a good
guidance law will quickly bring the missile on a collision course on which it stays for
the rest of the engagement. Similarly, it is expected that a good 3D guidance law will
36
U2
v2
U1
v1
(a) Three dimensional engagement
(b) Collision plane
Figure 2.7: Engagement scenario and collision plane in a 3D conflict.
37
take a missile quickly to a ‘collision plane’ on which a 2D-like engagement follows.
It looks reasonable, at least in the case of a non-stationary non-maneuvering target, to
define this collision plane as the plane containing the missile, the target and the target’s
velocity vector. This is because, being on this plane is a necessary condition for the
missile to hit the target.
The development of RCA-3D-O is explained below. As in 2D where the appli-
cation of 2D collision avoidance algorithm to a UAV results in its pulling out of a
collision course, it is desirable for a 3D algorithm to drive a UAV out of the colli-
sion plane. Below, an example is given with illustration of the collision plane and the
direction in which an acceleration should be applied to effect a deconfliction in 3D.
Figure 2.7(a) shows a 3D engagement scenario. As it follows from the previous
discussion, the collision plane for a UAV U1 which is on a collision course with another
UAV U2 is a two dimensional plane containing both U1 and U2, and the velocity vector
of U2. Figure 2.7(b) illustrates this which shows the collision plane for U1. To pull
out of the collision plane, the UAV U1 ideally needs to apply an acceleration in a
direction n1 perpendicular to the collision plane as shown in Fig. 2.7(b). The vector
n1, perpendicular to the collision plane, is chosen such that n1 ·v1 > 0. However, this
may not be perpendicular to the velocity vector of the UAV. Therefore, acceleration
is applied along a1out which lies in the plane of v1 and n1 (refer Fig. 2.7(b)). Also,
a1out is perpendicular to v1 and a1out·n1 > 0. The magnitude of a1out is V 2/R where
R = Rmin exp(λ ×ZEM/Rdes). Here Rmin is minimum turn radius of U1, ZEM is the
predicted minimum separation between U1 and U2, and the desired separation Rdes and
λ are tuning parameters for the algorithm.
Having defined the collision plane, it is possible to extend the 2D algorithm in
Section 2.2 to handle three dimensional conflicts. The existence of a predicted near
miss for a UAV would imply that its velocity vector is close to the collision plane. The
3D algorithm proposed above increases the LOS rate by pulling the velocity vector of
the UAV away from the collision plane. Another way to handle a three dimensional
conflict will be to project the velocity vector of the UAV on to the collision plane and
apply the lateral acceleration to increase the LOS rate in the collision plane. This
algorithm, RCA-3D-I, is a natural extension of the proposed 2D algorithm to handle
inherently three dimensional conflicts. Let v1c be the projection of v1 on the collision
plane. Now, the 2D algorithm proposed in the previous section is used on the collision
plane assuming that U1 has a velocity v1c. The acceleration obtained a1in is in the
collision plane and is perpendicular to both v1c and v1 (see Fig. 2.7(b)). Similar to
RCA-3D-O, a1in has a magnitude equal to V 2/R.
38
RCA-3D-I and RCA-3D-O are, in some sense, at two different extremes – one
proposes a maneuver in the collision plane while the other perpendicular to it. A linear
combination of these proposed accelerations in the form a = λain +(1−λ )aout ,λ ∈[0,1], can be considered for the three dimensional collision avoidance. However, we
do not follow that approach in this thesis, but study only the ‘extreme cases’, that is,
the cases where λ is either 0 or 1.
2.3.3 Homing to destination in three dimension
Execution of a collision avoidance maneuver may take a UAV out of its nominal path
to the destination. When there are no more threats of collisions or near misses, the
UAV has to choose and follow a trajectory which will take it to its destination without
much further delay. We propose an algorithm to do this in the 3D case as follows.
Given the current location of UAV and its destination in a 3D space, a plane con-
taining the UAV, the destination, and the current velocity of the UAV can be uniquely
found. A Dubins path is followed on this plane to reach the destination. After finding
the plane containing the velocity vector and destination of a UAV, the UAV follows a
Dubins path on this plane in exactly the same manner as in 2D using Algorithm 2.2.
(Details of implementation of this is given in step 7 in Section 2.4.2 which describes
the implementation of the 3D collision avoidance algorithm.)
2.4 Implementation of collision avoidance algorithms
To evaluate the collision avoidance algorithm developed, several simulations are car-
ried out. This section gives the implementation details of algorithms RCA-2D, RCA-
3D-O, and RCA-3D-I as well as that of Satisficing Game Theory (SGT) based algo-
rithm (Archibald et al., 2008), which is used for comparative studies in section (Sec-
tion 2.5) through simulation results.
2.4.1 Implementation of the RCA-2D algorithm
Consider a general scenario where N number of UAVs fly from their origins to respec-
tive destinations. The algorithm RCA-2D consists of each UAV Ui ∈U = {U1,U2, . . . ,
UN} executing the following six steps at each time step.
Step 1: Find the neighbors within the sensor range. The set of neighbors for a UAV Ui
is
Ni = {Uj ∈U, j = i : ‖pi−p j‖ ≤ Rsen}
39
Step 2: If Ni is empty proceed to Step 5, otherwise, from among the neighbors, find
those with which Ui has a Zero Effort Miss (ZEM) less than Rdes. To do this, calculate
ZEM of Ui with a UAV Uj ∈Ni, ZEM(Uj), as in Eq. (2.2). The set of neighbors with
which Ui has a predicted miss distance less than the required separation Rdes is
N ∗i = {Uj ∈Ni : ZEM(Uj)≤ Rdes}
The UAVs Ui and Uj have velocities vi and v j. Rdes is a tuning parameter of the
algorithm.
Step 3: If N ∗i is empty (that is, no neighboring UAVs are on a collision course with
Ui) proceed to Step 5. Otherwise, find the neighbor with which it has a predicted
miss distance less than the desired separation and the least time-to-go for that violated
separation to occur as
Uj∗ = arg minUj∈N ∗
i
tgo(Uj)
where, tgo(Uj) is the time-to-go for minimum separation between Ui and Uj and can
be calculated using Eq. (2.4).
Step 4: Find the heading update for UAV Ui to avoid Uj∗ as follows. Let ψi and ψ j∗be the headings of Ui and Uj∗, respectively and V be their constant speed. Let θ be
the LOS angle and r the LOS separation as indicated in Fig. 2.1. Then the LOS rate is
given as
θ =V sin(ψ j∗ −θ)−V sin(ψi−θ)
r
Obtain the required heading as
Δψi =
⎧⎪⎪⎪⎨⎪⎪⎪⎩−Vi
RΔt , if θ ≥ 0
+Vi
RΔt , otherwise
where, R = Rmin exp(λ ×ZEM(Uj∗)/Rdes).
Here, ZEM(Uj∗) is the predicted minimum separation between Ui and Uj∗ as com-
puted using Eq. (2.2), Rmin is the minimum radius of turn, λ is a tuning parameter of
the algorithm, and Δt is the time step. After obtaining the required heading change,
proceed to Step 6.
Step 5: If a UAV is not on a collision course with any other UAVs then there is no need
for a collision avoidance maneuver. So proceed to find the heading update to follow a
40
Dubins path to destination. Let pi = (xi,yi,0) be position of UAV Ui and pt = (xt ,yt ,0)
be its destination. Also, let vi = (ui,vi,0) be the UAV velocity and ψi be its heading.
As the first step toward implementing the Dubins path to destination, find the center
(xc,yc) of the circle C+ (refer Fig. 2.5) toward the target and the turn direction Dψ as
⎛⎝ xc
yc
Dψ
⎞⎠ =
⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎩
⎛⎝ xi−Rmin sinψi
yi +Rmin cosψi
+1
⎞⎠ , if sgnz (vi×(pt−pi)) > 0
⎛⎝ xi +Rmin sinψi
yi−Rmin cosψi
−1
⎞⎠ , otherwise
Here, Dψ indicates the direction in which the UAV has to turn to follow C+. A value
of +1 for Dψ represents an anticlockwise turn and −1 a clockwise turn. We define the
function sgnz() to give the sign of z component of that vector which is the argument of
the function.
The heading change for a unit time step that takes the UAV to its destination (xt ,yt)
via the Dubins path is obtained as
Δψi =
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
+Dψ
(V
Rmin
)Δt , if ‖(xc,yc)− (xt ,yt)‖> Rmin
−Dψ
(V
Rmin
)Δt , otherwise
If the desired heading is very close to the actual heading, we use a proportional control
to avoid chattering due to frequent switching of computed values of Δψi. That is, if
|ψd −ψi| < ε then Δψi = k(ψd −ψi)Δt. For simulations k = 1 and ε = 10−6 radians
are used.
The desired heading can be found as
ψd = arctan
[(pt−pi) · j(pt−pi) · i
]
where, i and j are unit vectors along x and y coordinates, respectively.
Step 6: Update the heading and position of Ui as
ψi = ψi +Δψi , xi = xi +Vi cosψiΔt , yi = yi +Vi sinψiΔt
The RCA-2D algorithm consist of every UAV Ui ∈U implementing these steps at
every time step until they reach the respective destinations.
41
2.4.2 Implementation of the RCA-3D-O algorithm
The 3D algorithm, RCA-3D-O, is implemented by each UAV Ui,1≤ i≤N, as follows.
Step 1: Find the neighbors, Ni as in Step 1 of RCA-2D algorithm.
Step 2: If Ni is not empty, find N ∗i ⊆Ni from among the neighbors, with which there
is possibility of near miss as in Step 2 of RCA-2D. Otherwise proceed to Step 7.
Step 3: If there are no neighbors with which there is a possibility of near miss then
proceed to Step 7. Otherwise find the neighbor Uj∗ with which it has minimum tgo for
near miss to occur as in Step 3 of RCA-2D.
Step 4: Find a vector perpendicular to the collision plane of Ui as follows. Let the
positions of UAVs Ui and Uj∗ be pi and p j∗, respectively and their velocities be vi and
v j∗. Collision plane is the plane containing UAVs Ui and Uj∗, and the velocity vector
of Uj∗. Therefore, we can find a vector perpendicular to the collision plane as
ni = v j∗×(p j∗ −pi)
Step 5: Find a component of ni perpendicular to vi, which is the direction in which
acceleration has to be applied for collision avoidance, as
ai = ni− (ni · vi)vi
where ‘ˆ’ represents the unit vector along the original vector direction.
Step 6: Find the required change in velocity vector for unit time step as
Δvi =
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
+ai
(V 2
R
)Δt , if ni ·vi > 0
−ai
(V 2
R
)Δt , otherwise
Proceed to Step 8 for position and velocity update.
Step 7: Follow the Dubins path to destination if there is no need for collision avoidance
(that is, either Ni or N ∗i is empty).
The Dubins path in 3D lies on a plane containing velocity vector of UAV and its
destination point. So, it can be reduced to a 2D problem in a new coordinate system
where the x axis is along d = pt −pi, the UAV is at the origin, and the velocity vector
makes an angle ψi with x axis. The heading of UAV in this new axis, ψi, is
ψi = arccos
(vi · (pt−pi)‖vi‖‖pt −pi‖
)
42
The center of C+ circle in Fig. 2.5 in new coordinates is
xc = Rmini sin ψi , yc =−Rmini cos ψi
The acceleration direction perpendicular to the velocity vi in the original coordinates
is
d⊥ = d− (d · vi)d
Find the velocity change for a unit time step to steer the UAV toward destination along
Dubins path in 3D is
Δvi =
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
d⊥(
V 2
Rmin
)Δt , if ‖(xc,yc−‖pt−pi‖)‖> Rmin
−d⊥(
V 2
Rmin
)Δt , otherwise
Step 8: Update velocity and position of UAV Ui as
vi = vi +Δvi , pi = pi +viΔt
2.4.3 Implementation of the RCA-3D-I algorithm
The 3D algorithm RCA-3D-I implemented by each UAV Ui,1≤ i≤ N, is as follows.
Step 1: Find the neighbors, Ni as in Step 1 of RCA-2D algorithm.
Step 2: If Ni is not empty, find from among the neighbors the ones, N ∗i , with which
there is a possibility of near miss as in Step 2 of RCA-2D. Otherwise proceed to Step 9.
Step 3: Find the neighbor Uj∗ with which it has minimum tgo for near miss to occur as
in Step 3 of RCA-2D.
Step 4: Find a vector, ni, perpendicular to the collision plane as in Step 4 of RCA-3D-
O.
Step 5: Find the component of vi in the collision plane as
vic = vi− (ni ·vi)ni
where ‘ ˆ ’ represents the unit vector along the original vector direction. Now, vic and
v j∗ are on the same plane which is the collision plane.
Step 6: Find the component of LOS rate in the collision plane, θi j∗c, as follows. Let
Δp∗ = p j∗ −pi. Then,
θi j∗c =[v j∗ − (v j∗·Δp∗)Δp∗]− [vic− (vic·Δp∗)Δp∗]
‖Δp∗‖
43
Step 7: Find a component of ni perpendicular to vi, which is the direction in which
the acceleration has to be applied for collision avoidance, as
ai = ni− (ni · vi)vi
where ‘ˆ’ represents the unit vector along the original vector direction.
Step 8: Find required change in velocity vector for unit time step as
aiin =
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
+vic×ni
(V 2
i
Ri
)Δt , if θi j∗c < 0
−vic×ni
(V 2
i
Ri
)Δt , otherwise
Proceed to Step 10 for position and velocity update.
Step 9: Follow Dubins path to destination if there is no need for collision avoidance
(that is, either Ni or N ∗i is empty) as in Step 7 of RCA-3D-O.
Step 10: Update velocity and position of UAV Ui as in Step 8 of RCA-3D-O.
2.4.4 Aircraft conflict resolution using Satisficing Game Theory (SGT)
The main contribution of this chapter of the thesis is the development of a simple,
computationally efficient and easily implementable algorithm for collision avoidance
among multiple UAVs. Simulations in this chapter consider scenarios where the traffic
densities are very high. Since very few algorithms exist in the literature addressing this
problem, the satisficing game theory based algorithm proposed recently as a solution to
this problem by Hill et al. (2005) and Archibald et al. (2008) is chosen for comparison
with the results of RCA-2D and RCA-3D. As complete details of the implementation
in (Hill et al., 2005) and (Archibald et al., 2008) is not available in these papers, the
implementation of the SGT algorithm as used for simulation results presented in this
chapter is described below. The results used for comparison are obtained from this
implementation of the SGT algorithm and not directly taken from the references (Hill
et al., 2005) and (Archibald et al., 2008). However, the results obtained in the noise-
free case are similar to the ones reported in (Hill et al., 2005) and (Archibald et al.,
2008). These papers do not consider the noisy case.
Now we give a brief description of the SGT for which we mostly follow the no-
tations in (Hill et al., 2005; Archibald et al., 2008). In the satisficing game theoretic
approach, the preferences or the lack of it of a player/agent over decisions are repre-
sented using utility functions called selectability and rejectability, respectively. These
44
are maps from the space of decision variables to [0,1] and have properties that resem-
ble that of a probability distribution function. If PS and PR are the selectability and
rejectability functions and U = {u1,u2, . . . ,um} is the set of possible actions, then
PS : U −→ [0,1] , and
PR : U −→ [0,1]
with properties
PX(u)≥ 0 ∀u ∈U , and ∑u∈U
PX = 1 for X = S,R.
The construction of PS and PR is described below in Steps 3-6. The decision of a player
is
u∗ = argmaxu∈U
PS(u)−PR(u)
The key idea in applying satisficing game theory to any problem is to formulate mean-
ingful selectability and rejectability functions. In the aircraft collision avoidance prob-
lem considered, the decision of each aircraft is the choice of heading. Thus the decision
set is
U = {turn 5◦ left, turn 2.5◦ left, fly straight, turn 2.5◦ right, turn 5◦ right}
If the time step used for simulations, Δt, is 1 second, then the above decision set cor-
responds to a maximum turn rate of 5◦ per second. The choice of five discrete heading
decisions greatly reduces the complexity of computations as the SGT algorithm is
greatly dependent on the size of the decision set.
The SGT algorithm for collision avoidance consists of each UAV Ui ∈U executing
the following steps at each time step.
Step 1: Find the neighboring UAVs, Ni, as in Step 1 of RCA-2D.
Step 2: Create a priority list Pi of the UAVs in Ni as follows. All UAVs within
5 miles of respective destination belongs to one group which has priority over all the
remaining UAVs which belong to the second group. Within each group, UAVs with
higher estimated delay in arriving at the destination has higher priority. Among the
UAVs with same delay, the one with higher flight time has higher priority. All the
UAVs in Ni with higher priority than Ui forms the set Pi.
The delay of a UAV in arriving at the destination is estimated as follows. The Du-
bins path length, dDubins, to target from the current location of a UAV can be calculated
at every time. Then its delay in arriving at the destination is
delay = t +dDubins
V− (tinit + texpected)
45
where, t is the current time (the time elapsed since the starting of simulation), V is the
constant speed of UAV, tinit is the time at which UAV was introduced, and texpected is
the time taken by the UAV to take a Dubins path from origin to destination if there are
no collision avoidance maneuvers. The flight time is calculated as t− tinit .
Step 3: Compute the rejectability function for Ui as follows.
PRi(u) ∝ ∑Uj∈Pi
WR(Ui,Uj), u ∈U
for a heading choice u of Ui and current heading of Uj where, WR is given as
WR(Ui,Uj) =
⎧⎨⎩
2α , if dmin(i, j)≤ Rc
α , if Rc ≤ dmin(i, j)≤ Rnm
0, otherwise
where,
α =
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩
[1+
Rnm−dmin(i, j)Rnm
][1
d(i, j)
]β, if d(i, j)≤ 3Rnm
[1
d(i, j)
]β, otherwise
where, Rnm is the minimum separation required to avoid near miss, Rc is the separation
required to avoid physical collision, and β is a tuning parameter. Following Archibald
et al. (2008), we use a value of β = 2/3. The zero effort miss distance dmin(i, j)
between Ui and Uj, and the time-to-go, tgo, for that to occur, are calculated using
Eqs. (2.2) and (2.4). The distance from current point to point of closest approach
between Ui and Uj is d(i, j) and is calculated as V × tgo. The rejectability function is
then normalized so that ∑u∈U PRi(u) = 1.
Next step is the calculation of selectability function. For this, the simplified model
given in (Hill et al., 2005) and (Archibald et al., 2008) is used.
The selectability function of each player is influenced by preferences of other play-
ers who are higher in the priority list. The idea is that a player is ready to sacrifice his
efficiency in terms of directly heading to target if he is sure that another player higher
in the priority list will take advantage of his deviation.
Step 4: The first step in calculating the selectability map is to build the base selectabil-
ity. The base selectability of a choice of heading is more if it takes the aircraft closer
to target.
Each heading decision u, is assigned a rank r(u) depending on how close it takes
the UAV to its target. Since there are five heading decision choices, the decisions are
46
ranked from 1 to 5. Let u be the heading that takes the UAV directly to the target. The
base selectability map σSi for Ui is constructed as follows
σSi(u) ∝
⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩
3, r(u) = 12, r(u) = 22, r(u) = 3, 2.5◦ < |u−u| ≤ 5◦
1.1, r(u) = 3, 5◦ < |u−u|1.1, r(u) = 4, |u−u| ≤ 5◦ , u ∈U1, r(u) = 4, 5◦ < |u−u|1.1, r(u) = 5, |u−u| ≤ 5◦
1, r(u) = 5, 5◦ < |u−u|A justification for the selection of weights as above is not given in (Archibald et al.,
2008). The map σS is normalized to get a probability distribution like structure.
Step 5: The full selectability is computed as follows. The UAVs in priority list Pi of
Ui are partitioned into 5 sets Sik,k = 1, . . . ,5 based on the heading choice for which an
aircraft in the list has highest base selectability. For example, if Uj ∈Pi has uk as the
heading choice with highest base selectability, then Uj ∈ Sik. We have
Pi =⋃k
Sik
For each Ui a weight matrix
Wik(u,v) = ∑Uj∈Si
k
WS(Ui,Uj) , u,v ∈U
is calculated where, WS(Ui,Uj) = 1 if there is no conflict (dmin(i, j) > Rnm) when Ui
follows u and Uj follows v, and 0 otherwise. Wik is normalized over each column. Let
W ig(k) denote the cardinality of Si
k. Conditional selectability of Ui is
ρSi(u|u1,u2,u3,u4,u5) ∝5
∑k=1
W ig(k)Wik(u,uk)
where u1,u2,u3,u4,u5 ∈U . For the sake of brevity define U = U ×U ×U ×U ×U
and u = {u1,u2,u3,u4,u5}. The marginal selectability is
ρSi(u) = ∑u∈U
ρSi(u|u)σS1(u1) . . .σS5(u
5)
We use the following values
σS1 = [0.3333 0.2222 0.2222 0.1111 0.1111]
σS2 = [0.2198 0.3297 0.2198 0.1209 0.1099]
σS3 = [0.1099 0.2198 0.3297 0.2198 0.1209]
σS4 = [0.1111 0.1111 0.2222 0.3333 0.2222]
σS5 = [0.1235 0.1235 0.1358 0.2469 0.3704]
47
which is one of the mass function set constructed by the procedure given above. The
marginal selectability ρS is then normalized.
Step 6: Obtain the full selectability function as
PSi = ησSi +(1−η)ρSi
where a η = 0.001 is chosen as in reference (Archibald et al., 2008).
Step 7: Now the choice of heading direction for Ui is
u∗i = argmaxu∈U
(PSi(u)−PRi(u))
This gives the increment in heading Δψi = u∗i .
Step 8: Update the heading and position as in Step 6 of RCA-2D.
2.5 Simulation Results
The performance of the algorithms developed is tested through extensive simulations.
This section describes the test cases and presents and compares the results obtained
using the algorithms proposed in this thesis and the SGT method. All results that
are reported here are obtained from the simulations based on the implementation of
algorithms as described in the last section.
2.5.1 Simulation results for two dimensional case
We use the same performance metrics that are used to measure the performance of
collision avoidance algorithm in (Hill et al., 2005) and (Archibald et al., 2008). These
are number of near misses and efficiency, and are explained below for completeness.
Near misses: From a safety point of view, the UAVs are required to keep at least a
minimum distance between each other. Any location of two UAVs within this mini-
mum required separation results in a near miss. A good collision avoidance algorithm
will ensure fewer near misses. The value of required separation (Rnm) considered for
simulations in this section is 5 miles.
Efficiency: It is required that the UAVs reach the destination in minimum possible
time. The collision avoidance maneuver should be minimal so that UAVs arrive at
their destinations as soon as possible. If T aj is the actual flight time of the jth UAV and
T ij is its ideal flight time (that is, the time taken to reach the destination if the UAV were
48
not to make any deviation for collision avoidance), then the efficiency for N UAVs is
Efficiency =1N
N
∑j=1
T ij
T aj
(2.17)
An algorithm with higher efficiency is considered to be better than another algorithm
with similar near misses but a lower efficiency. The lower efficiency algorithm does a
more conservative collision avoidance maneuver. Thus, the efficiency metric gives an
insight into the optimality of the algorithm in terms of time taken to arrive at destina-
tion while avoiding collisions.
The papers (Hill et al., 2005) and (Archibald et al., 2008) use another performance
metric – number of collisions. A collision is deemed to have occurred if two UAVs ap-
proach within Rc = 300 feet of each other. We do not consider the number of collisions
as a performance metric as no collisions were observed in any of the simulations.
The following values are used for all the simulations in this chapter. The air-
craft are assumed to have a velocity of with magnitude equal to 500 miles per hour
(733.33 ft/s). A maximum turn rate of 5◦ per second equivalent to a turn radius of
Rmin = 5/π miles is assumed. The time step Δt is chosen as 1 second.
Figure 2.8 shows simulation results for simple cases of collision avoidance using
Algorithm 2.3 to avoid head-on collisions. We observe from the figure that RCA-2D
can handle these conflicts. However, the real test is the one in which the airspace is
congested with high density traffic.
Random flight test case
It is desirable that a test case for collision avoidance be complex and at the same time
be sufficiently scalable to test the high traffic density cases. The test case of random
flights (Hill et al., 2005; Archibald et al., 2008) is considered for simulations. In this
test case, the UAVs fly from random points on an outer circle, the destination being
random points on an inner circle. The values 120 and 100 miles are used as the radii
of outer and inner circles for simulations presented in this chapter. Figure 2.9 gives
a schematic of the random flight test. Each simulation is done for a specified aircraft
density or the number of aircraft in the concerned airspace. When the simulation be-
gins, a new aircraft is introduced at a random point on the outer circle at every 5 s
interval, which is assigned a random point on the inner circle as destination, till the
required aircraft density is achieved. Whenever an aircraft reaches the destination, it is
removed and a new aircraft is added at a randomly selected point on the outer circle to
49
(a) Head-on conflict
(b) Intersection conflict
Figure 2.8: Collision avoidance in simple conflict situations.
50
Figure 2.9: A schematic of the random flight test case.
keep the number of aircraft in the airspace constant. Once the airspace achieves a re-
quired density, information regarding efficiency and number of near misses is collected
for a simulation time of 50 minutes. Since the test case involves random introduction
of UAVs and random assignment of destination points, it is called a ‘random flights’
test case and the simulation for a particular aircraft density is conducted 20 times and
average values are computed. As there is a randomness involved in this test case, it is
expected that, if the simulations are carried out for a reasonably large number of times,
almost all the possible conflict geometries would be encountered.
It is required to choose the right values for the tuning parameters of the algorithm
RCA-2D to achieve a good performance. Toward this, extensive simulations are done
with varying values of tuning parameters Rdes, Rsen and λ , and the performance in
terms of number of near misses and efficiency are observed. Figure 2.10 gives results
of one such study for a traffic density of 20 aircraft and a constant λ = 0.5. These
plots give the average number of near misses and efficiencies for values of sensor
range radii {6,7,10,20,30,40,50,60,70,80}. As seen from Figs. 2.10(a) and 2.10(b),
the choice of Rdes and Rsen is a tradeoff between number of near misses and efficiency;
the higher the efficiency, the larger the number of near misses. After similar studies
for other aircraft densities, the values Rsen = 10 miles, Rdes = 6 miles and λ = 0.5 were
chosen for the tuning parameters. Figure 2.10(b) shows a decrease in efficiency with
an increase in sensor radius. This is contrary to the intuition that more information
should lead to better efficiency. It is to be noted that the reactive collision avoidance
algorithms developed in this chapter are not optimal algorithms that take complete
51
Table 2.1: Comparison of performance of algorithms RCA-2D and SGT: Test case ofrandom flights (averaged over 20 runs)
Number of SGT RCA-2DAircraft Near Misses Efficiency Near Misses Efficiency
20 1.95 99.45 1.35 99.0940 7.05 97.94 3.75 97.6460 17.85 94.38 12.65 96.39
advantage of all the available information, but a reactive one. For a UAV implementing
these algorithms, larger sensor radius implies more neighbors. Therefore, it may so
happen that in trying to avoid conflicts with this larger class of neighbors, a UAV
may result in taking a roundabout path avoiding the whole neighbor set as a group
while there may be a safe path through them. This behavior results in poor efficiency.
Restricting the sensor radius forces a UAV to ignore those neighbors which are far
away and take the risk of exploring a safe path through the neighbors.
Table 2.1 gives the results for low density (20 aircraft), medium density (40 air-
craft), and high density (60 aircraft) traffic. For comparison, the values obtained by
simulating the Satisficing Game Theory (SGT) based algorithm of Hill et al. (2005)
and Archibald et al. (2008) are also given in the table. They give two versions of
their algorithm – full and simplified (Hill et al., 2005; Archibald et al., 2008). For
the purpose of comparison, the simplified SGT algorithm is used as it is computation-
ally much less demanding while its performance is almost the same as that of the full
version (Archibald et al., 2008). The simplified satisficing approach to aircraft conflict
resolution SGT, and the way it is implemented for the studies in this chapter were given
in Section 2.4.4. Both the algorithms, SGT and RCA-2D, are executed and evaluated
under the same conditions for all test cases. For example, we use the same sequence
of random numbers in setting up the problem, and evaluate the performances using
the same criteria. All the values in Table 2.1 are averages over 20 runs. As seen from
Table 2.1, the number of near misses are less for RCA-2D when compared to SGT,
while the efficiencies are comparable for all the cases. Figure 2.11 gives a snapshot
during one of the simulations with a traffic density of 40 UAVs.
The performance measure ‘efficiency’ is an indicator of the overall cost (sum of
deviations from the nominal paths) of the entire group of UAVs. We do not claim
global optimality of the proposed algorithm. Nonetheless, the high values of efficien-
cies obtained in the simulation results point to a good performance of the proposed
collision avoidance algorithm.
52
0 10 20 30 40 50 60 70 800
2
4
6
8
10
12
14
16
18
20
sensor radius (miles)
near
mis
ses
R
des= 5 miles
Rdes
= 6 miles
Rdes
= 10 miles
Rdes
= 15 miles
(a) Variation of near misses
0 10 20 30 40 50 60 70 8075
80
85
90
95
100
sensor radius (miles)
effic
ienc
y
Rdes
= 5 miles
Rdes
= 6 miles
Rdes
= 10 miles
Rdes
= 15 miles
(b) Variation of efficiency
Figure 2.10: Variation of near misses and efficiency with Rsen and Rdes for a trafficdensity of 20 aircraft.
53
Figure 2.11: A snapshot during the simulation of random flights where UAVs fly fromrandom points on outer to inner circle for a traffic density of 40 aircraft.
For collision avoidance among multiple UAVs, RCA-2D has a great computational
advantage over SGT. Table 2.2 presents the average computation times taken by SGT
and RCA-2D for simulations with various air traffic densities. RCA-2D is about 14–20
times faster than SGT. Simulations were done using MATLAB on Windows platform
operating on a computer with Intel Core 2 CPU at clock speed of 2.4GHz and 2GB of
RAM. There are two parts to the total computational time: (i) the time to update tra-
jectories of the aircraft, which is the same for both the algorithms, and (ii) the decision
time, which is specific to each algorithm. What is reported in Table 2.2 is the average
decision time, that is, the average wall clock time that each program spent in the guid-
ance loop during a run of random flights of 50 minutes simulation time. This gives a
direct comparison between the computational complexities of the two algorithms. The
averages are taken over 20 simulation runs.
Simulations with noisy measurements
To study the robustness of the collision avoidance algorithm, noise is introduced in
the measurement of UAV positions and headings, and its effect on the performance of
the algorithm is studied. In previous simulations, it was assumed that the positions of
neighboring UAVs which are within the sensor range of a particular UAV are exactly
54
Table 2.2: Comparison of computation times of RCA-2D and SGT per simulation:Test case of random flights (averaged over 20 runs)
Number of Computation Time (sec)Aircraft SGT RCA-2D
20 638 3340 1611 10060 2819 206
Table 2.3: Comparison of performance of RCA-2D and SGT: Test case of randomflights with 20 aircraft in the presence of noise in measurement of position of neigh-boring aircraft (averaged over 20 runs)
Std. Dev. of Near Misses EfficiencyNoise (miles) SGT RCA-2D SGT RCA-2D
0 1.95 1.35 99.45 99.090.1 8.65 1.35 99.58 98.990.2 12.50 1.55 99.79 99.080.3 14.10 1.95 99.44 99.02
known to it. This assumption is relaxed with the assumption that the x and y coordi-
nates of other UAVs are measured with a noise which is normally distributed with a
zero mean and a standard deviation of σ (in miles). Table 2.3 gives the performance of
collision avoidance algorithms, both RCA-2D and SGT, for the case of random flights
with an aircraft density of 20 for various values of σ . From the table, it is seen that
while RCA-2D exhibits robustness to noise in position measurement, there is a sudden
degradation in the performance of the SGT in terms of the number of near misses.
The effect of noise in the measurement of heading of neighboring aircraft on the
performance of the collision avoidance algorithm RCA-2D is also studied. During
simulation, normally distributed noise of zero mean and various standard deviations is
introduced to the actual heading of the neighboring aircraft to make the measurement
noisy. The collision avoidance algorithms, RCA-2D and SGT, were implemented with
this noisy measurement. Table 2.4 presents performance results obtained where the
standard deviations are in degrees. Even in this case it is seen that the degradation in
performance of RCA-2D in terms of number of near misses when compared to SGT is
very low.
Scalability Studies
We now carry out the scalability studies for RCA-2D. The test case of random flights
55
Table 2.4: Comparison of performance of RCA-2D and SGT: Test case of randomflights with 20 aircraft in presence of noise in measurement of heading of neighboringaircraft (averaged over 20 runs)
Std. Dev. of Near Misses EfficiencyNoise (degrees) SGT RCA-2D SGT RCA-2D
0 1.95 1.35 99.45 99.095 10.40 1.80 99.92 99.0110 13.30 1.85 99.96 99.0815 14.60 2.25 99.99 99.06
for higher aircraft densities are conducted to analyze how the algorithm scales in terms
of near misses, efficiency and computational time. For the test case of random flights,
aircraft density as high as 200 UAVs per simulation is considered. Figure 2.12 gives
plots of the number of near misses and efficiency versus number of UAVs (aircraft
density) averaged over 20 runs.
Figure 2.12(a) shows how the number of near misses changes as the number of
UAVs in a simulation of the test case of random flights increases. The algorithm scales
almost linearly till around 80 aircraft for which the number of near misses is only 22.
For higher number of UAVs the near misses increases more rapidly. However, the
increase is not exponential. The degradation of efficiency with increase in number of
UAVs is graceful as seen in Fig. 2.12(b).
Figure 2.13 shows how the computational time taken per simulation scales with the
UAV density. Here again, by computational time, we mean the decision time, that is,
the wall clock time per simulation excluding the trajectory update time. From the log-
log plot in Fig. 2.13(b) it appears that the computational complexity of the algorithm is
bounded by O(n1.8) where n is the number of UAVs (aircraft density) in a random flight
test case. The computational complexity of RCA-2D as calculated above, through
simulations, is valid only for the random flight test case for which it is computed.
However, in the worst case, every UAV will have to account for and calculate predicted
near misses with every other UAVs in flight, which implies a computational complexity
of O(n2). Thus, the computational complexity obtained through simulations for the
random flight test case suggests that the random flight test case is close to a worst
case instance for RCA-2D. Figure 2.14 shows the average decision time per UAV per
second (in milliseconds) as a function of the UAV density. Even for an aircraft density
of 200, in the random flight test case, the average decision time per UAV per second is
just about 3.5 milliseconds. This vindicates the scalability of the proposed algorithm.
The linear growth of the curve in Fig. 2.14 indicates that after some high value of
56
20 40 60 80 100 120 140 160 180 2000
50
100
150
200
250
Number of UAVs
Nea
r M
isse
s
(a) Scalability of near misses
20 40 60 80 100 120 140 160 180 20082
84
86
88
90
92
94
96
98
100
Number of UAVs
Effi
cien
cy
(b) Scalability of efficiency
Figure 2.12: Scalability of number of near misses and efficiency with UAV density(averaged over 20 runs).
57
20 40 60 80 100 120 140 160 180 2000
500
1000
1500
2000
2500
Number of UAVs
Ave
rage
dec
isio
n tim
e pe
r si
mul
atio
n (s
ec)
(a) Computational complexity
101
102
101
102
103
104
Number of UAVs
Ave
rage
dec
isio
n tim
e pe
r si
mul
atio
n (s
ec)
simulation dataline with slope = 1.8
(b) Computational complexity: Log-log plot
Figure 2.13: Average decision time per simulation for different UAV densities (aver-aged over 20 runs).
58
20 40 60 80 100 120 140 160 180 2000.5
1
1.5
2
2.5
3
3.5
Number of UAVs (traffic density)
Ave
rage
dec
isio
n tim
e pe
r U
AV
per
sec
ond
(mse
c)
Figure 2.14: Average decision time per UAV per second with increase in traffic density(averaged over 20 runs).
UAV density, the computations cannot be done in real-time (computational time per
UAV per second will become more that a second). However, that aircraft density at
which the computations cease to be real-time is very high, as can be seen from the
extrapolation of the curve in Fig. 2.14, that a UAV mission at that density will anyway
not be physically feasible.
2.5.2 Simulation results for 3D case
In this subsection, we perform simulations to evaluate the performance of the proposed
3D collision avoidance algorithms. The performance indicators are number of near
misses and efficiency, as before. In the 3D case, the safety zone for each UAV is
considered as a ball of radius equal to the required separation (5 miles) centered at a
UAV. After a sensitivity study, similar to that performed for the 2D case, the values of
the tuning parameters were chosen as Rsen = 10 miles, Rdes = 6 miles and λ = 0.5. We
present the test case of random flights as well as cases where the conflicts are genuinely
3D to begin with.
Random flight test case
Simulations are done for the random flights test case involving 20 to 60 aircraft, in
59
−100
−50
0
50
100
−100−50050100
−20
0
20
x−range (miles)
heig
ht (
mile
s)
y−range (miles)
Figure 2.15: A screenshot of UAVs making 3D collision avoidance maneuvers duringrandom flight test case.
steps of 20. Since the flights are random, for each aircraft density, 20 simulations
are done and the average values of number of near misses and efficiency are cal-
culated. The tuning parameters used for all simulations in 3D are Rsen = 10 miles,
Rdes = 6 miles and λ = 0.5, which are the same as those used for the 2D algorithm. A
screenshot during one of the random flight tests is given in Fig. 2.15.
The performance of RCA-3D-O, which pulls a UAV out of the collision plane,
is compared to that of the 2D algorithm developed in Section 2.2 (RCA-2D) for the
test case of random flights. Table 2.5 presents the simulation results. It is seen from
the table that the 3D algorithm performs much better than the 2D algorithm in terms
of near misses; also, the efficiencies obtained are higher. Note that RCA-3D-I is a
3D extension of the 2D algorithm; for planar engagements, it is the same as RCA-
2D. Also, note that the computational effort required for the 3D algorithm is same as
that of 2D algorithm as seen from Table 2.6. The values reported in the table are the
average decision times, that is, the average times spent in guidance loops alone during
simulations.
Test case of perpendicular flow
There are certain test cases where the use of the third dimension naturally leads to
60
Table 2.5: Comparison of performance of algorithms RCA-2D and RCA-3D-O: Testcase of random flights (averaged over 20 runs)
Number of Near Misses EfficiencyAircraft RCA-2D RCA-3D-O RCA-2D RCA-3D-O
20 1.35 0.3 99.09 99.3540 3.75 1.0 97.64 98.0660 12.65 2.6 96.39 96.82
Table 2.6: Comparison of computation time of algorithms RCA-2D and RCA-3D-O:Test case of Random Flights (averaged over 20 runs)
Number of Computation Time (s)Aircraft RCA-2D RCA-3D-O
20 33 3240 100 10360 206 226
much more effective collision avoidance among UAVs. As an example of this, the test
case of perpendicular flow of aircraft is solved using the RCA-3D-O algorithm. This
test case, in which two streams of aircraft approach perpendicular (or at an angle) to
each other, is very challenging to solve in 2D. Mao et al. (2000) addressed this problem
in the literature by using instantaneous heading and velocity changes. Figure 2.16
shows the RCA-3D-O algorithm taking care of such a situation by letting one stream
go above the other.
Test case of perturbed 2D random flights
This test case is a minor modification of the 2D random flight case. Here, the aircraft
are introduced not on the same plane but randomly at different altitudes within a band
of 5 miles altitude. This will ensure that the conflicts that would have occurred in
the planar case would still occur (as the vertical separation is less than the required
separation (5 miles) for near miss) while the engagements are three dimensional.
Table 2.7 summarizes the performance of the 3D algorithms – RCA-3D-I and
RCA-3D-O – for the test case of perturbed 2D random flights. Both the algorithms
perform equally well. Given that RCA-3D-I is the natural extension of RCA-2D, this
result shows that the performance of the algorithm greatly improves when the en-
gagement is not strictly planar, which is a practical case. Although RCA-3D-I works
equally well as RCA-3D-O in this test case where the engagements are nearly planar,
61
−40−20
020
40
−50
0
50−20
−10
0
10
20
heig
ht (
mile
s)
x−range (miles)
y−range (miles)
Figure 2.16: Collision avoidance in 3D using RCA-3D-O – the test case of perpendic-ular flow of aircraft.
Table 2.7: Performance of the 3D algorithms for the test case of perturbed randomflights (80 UAVs, averaged over 10 runs)
Near Misses Efficiency (%)RCA-3D-I RCA-3D-O RCA-3D-I RCA-3D-O
6.3 6.5 95.98 96.31
its performance degrades when the engagements become more three dimensional in
nature. This is presented below.
Test case of modified random flights
In a real world situation, the engagement of UAVs are seldom planar. RCA-3D-I and
RCA-3D-O readily applies also to the cases where engagements are inherently 3D.
To demonstrate this, we modify the test case of random flights to create a situation
in which conflicts are inherently three dimensional. We consider a situation shown in
Fig. 2.17. The UAVs fly between two planes separated by a distance h. The UAVs
appear at random locations on the outer circle of radius rout in one of the planes and
fly toward random points on the inner circle of radius rin on the other plane. The mid-
air conflicts in such a case is purely three dimensional. As in the test case of random
62
rin
rout
h
Figure 2.17: The test case of modified random flights.
Table 2.8: Performance of the 3D algorithms for the test case of modified randomflights (80 UAVs, averaged over 10 runs)
Case Near Misses Efficiency (%)RCA-3D-I RCA-3D-O RCA-3D-I RCA-3D-O
1 2.6 2.4 98.62 98.842 8.3 4.6 97.30 96.963 11.8 6.1 96.17 96.17
flights, a new aircraft is introduced into the airspace every 5 seconds till the required
traffic density is achieved. Thereafter a UAV is replaced by a new one when it reaches
its destination.
We consider three cases for this test case. In Case 1, we consider h = 20 miles,
rin = 100 miles, and rout = 120 miles. Figure 2.6 is actually a snapshot of this test
case with 20 UAVs. In Case 2 we increase the separation between the two planes to
60 miles. To maintain similar traffic density (that is, the number of UAVs per unit
volume) for the same number of UAVs, we reduce rout to 70 miles and rin to 55 miles.
For Case 3 we have h = 100 miles, rin = 40 miles, and rout = 50 miles. Simulations
are carried out for these cases with a traffic density of 80 UAVs each. The test case
of modified random flights is solved using both the RCA-3D-in and the RCA-3D-out
algorithms. Table 2.8 gives the values of near misses and efficiencies averaged over 10
runs for each of these cases.
The average climb/descent rates of UAVs are high in a case where the separation
between the planes is large. As seen from Table 2.8, RCA-3D-O outperforms RCA-
3D-I in such cases where the engagements are highly three dimensional and the 2D
approximation by projecting the velocities on to the collision plane is not that effective.
We have demonstrated through simulations that the algorithm we developed for
63
collision avoidance among multiple UAVs works well with a kinematic model of UAV
with minimum radius of turn constraint. However, for this algorithm to be useful in a
practical situations, we need to show the efficacy of it when implemented on realistic
UAV models. This is dealt with in the next chapter.
2.6 Summary and conclusions
In this chapter, we developed conceptually simple reactive algorithms to achieve col-
lision avoidance among UAVs, flying to their respective destinations, in a multi-UAV
scenario. First, a 2D collision avoidance algorithm was developed. The algorithm
uses concepts from missile guidance theory, like predicted zero effort miss and time-
to-go, to decide the turn rate of a UAV that will lead to deconfliction of a collision
course. This algorithm tries to achieve collision avoidance by increasing the LOS rate
between the UAVs which are on a most imminent collision course. Through simula-
tions, it was shown that this 2D collision avoidance algorithm – RCA-2D – performs
well with lower near misses and higher efficiency in terms of reaching the destination
as quickly as possible while avoiding collisions. The good performance of the algo-
rithm in the presence of noisy measurements was demonstrated through simulations.
The comparison of results with an existing collision avoidance algorithm for multi-
ple UAVs showed that the proposed algorithm performs better and is about 15 times
faster. Next, a 3D collision avoidance algorithm was proposed that takes advantage of
the third dimension and readily applies to cases where the initial engagement itself is
non-planar. Toward this, we introduced the concept of the collision plane and the 3D
collision avoidance maneuver consisted of pulling out from the collision plane. The
3D collision avoidance algorithm – RCA-3D-O – was shown to perform much better
than the 2D algorithm. Another 3D algorithm – RCA-3D-I – which is a natural three
dimensional extension of the 2D algorithm was also proposed where the avoidance
maneuver takes place on the collision plane as opposed to the out of plane maneuvers
of RCA-3D-O. The algorithm RCA-3D-I was shown to perform as well as RCA-3D-O
for three dimensional engagements that are nearly planar.
The proposed algorithms, both 2D and 3D ones, are decentralized as each UAV
implements them with its limited information of its environment gathered through its
sensors. These algorithms are also simple to implement. The main thesis of this chap-
ter is that it suffices, in case of a multiple UAV conflict, for a UAV to avoid the most
imminent near miss to obtain a good collision avoidance performance. This is vindi-
cated by the simulation studies done in this chapter.
64
The collision avoidance algorithms proposed in this chapter were developed based
on the assumptions of ideal kinematics. Therefore, it remains to be shown that these
algorithms perform well when implemented on realistic UAV models. This is dealt
with in the next chapter.
65
66
3 Collision avoidance with realistic UAV models
In Chapter 2, a free-flight airspace occupied by UAVs was considered and algorithmswere proposed which, when implemented by every UAV in the airspace, results inavoiding collisions and reducing near misses among UAVs. In this chapter, these al-gorithms are tested by implementing them on realistic UAV models in a high densitytraffic simulation environment. Proportional-Integral controllers are designed to im-plement the guidance commands given by the collision avoidance algorithms on theUAV models. The results obtained illustrates the efficacy of the proposed algorithmfor collision avoidance in realistic UAVs.
�se of a kinematic UAV model allows for attaining the commanded turn rates
instantaneously. However, this may is not possible in case of a realistic
UAV leading to a possible degradation in collision avoidance performance
of the proposed algorithm implemented on realistic UAV models. In this chapter, we
show that the collision avoidance algorithms that we developed performs well when
implemented on a realistic UAV model. Toward this, we conduct, using a six Degree of
Freedom (DoF) UAV model, several simulations involving various conflict geometries.
This chapter is organized as follows. In Section 3.1 we introduce the 6 DoF UAV
model. Section 3.2 describes the trimming of the UAV model for a steady level flight.
Design of the altitude, attitude, and velocity controllers to implement the proposed 2D
collision avoidance algorithm is discussed in Section 3.3. Section 3.4 gives simulation
results for collision avoidance tests using the 2D algorithm RCA-2D implemented on
realistic 6 DoF UAV models. In Section 3.5, flight path angle, course, and velocity
hold controllers are designed to implement the 3D collision avoidance algorithm RCA-
3D-O on realistic UAV models. Simulation results of the test cases for 3D collision
avoidance algorithm is given in Section 3.6; cases involving heterogeneous and non-
cooperating UAVs are also given in this section. Section 3.7 gives a summary and
concludes the chapter.
3.1 Six DoF equations of motion
We use the standard six degree of freedom aircraft model and follow the notation of
(Stevens & Lewis, 2003). We write the equations of motion with respect to the body
67
X
Y Z
V
a
Figure 3.1: Body axis of an aircraft.
axis (refer Fig. 3.1) of the UAV. We have (u,v,w) as the velocities along the body axis
and (p,q,r) as the angular rates about the body axis. The Euler angles which fix the
orientation of the UAV relative to an inertial frame is denoted by (φ ,θ ,ψ). The angle
of attack is α and side slip is denoted as β . The elevator, aileron, and rudder defections
are (δe,δa,δr) and δT is the throttle setting.
A rigid body in three dimension has three translational and three rotational degrees
of freedom. The three force equations, corresponding to the translational degrees of
freedom, along the body axis of the UAV are
u = rv−qw+1m
Fx
v = pw− ru+1m
Fy (3.1)
w = qu− pv+1m
Fz
where, m is the mass of the UAV and the forces along the X , Y , and Z directions of the
body axis are given as,
Fx = −mgsinθ +12
ρV 2S(CX0 +CXα α +CXq
cqV
+CXδeδe)
+12
ρSprop(k2motorC
2XδT
δ 2T −V 2)
Fy = mgcosθ sinφ +12
ρV 2S(CY0 +CYβ β +CYp
bp2V
+CYr
br2V
+CYδaδa +CYδr
δr)
Fz = mgcosθ cosφ +12
ρV 2S(CZ0 +CZα α +CZq
cqV
+CZδeδe) (3.2)
68
where, the stability derivatives have the usual definitions (Stevens & Lewis, 2003),
an electric motor-propeller combination is assumed to provide thrust, and for a given
angle of attack α , the force derivatives along X and Z directions of body axis can be
found from the lift and the drag derivatives as,
CX0 = −CD0 cosα +CL0 sinα
CXα = −CDα cosα +CLα sinα
CXq = −CDq cosα +CLq sinα
CXδe= −CDδe
cosα +CLδesinα
CZ0 = −CD0 sinα−CL0 cosα (3.3)
CZα = −CDα sinα−CLα cosα
CZq = −CDq sinα−CLq cosα
CZδe= −CDδe
sinα−CLδecosα
Corresponding to the three rotational degrees of freedom of the aircraft, we have the
three moment equations as,
p = Γ1 pq−Γ2qr +Γ3L+Γ4M
q = Γ5 pr−Γ4(p2− r2)+Γ6M (3.4)
r = Γ6 pq−Γ1qr +Γ4L+Γ7M
where, the moments about X , Y , and Z are
L =12
ρV 2Sb2(Cl0 +Clβ β +Clp
bp2V
+Clrbr2V
+Clδaδa +Clδr
δr)
M =12
ρV 2Sc(Cm0 +Cmα α +Cmq
cqV
+Cmδeδe) (3.5)
N =12
ρV 2Sb2(Cn0 +Cnβ β +Cnp
bp2V
+Cnr
br2V
+Cnδaδa +Cnδr
δr)
The coefficients in the moment equations which depend on the moment of inertias of
the aircraft are give as follows.
Γ1 =Jxz(Jx− Jy + Jz)
Γ
Γ2 =Jz(Jz− Jy)+ J2
xz
Γ
Γ3 =Jz
Γ
Γ4 =Jxz
Γ
69
Γ5 =Jz− Jx
Jy
Γ6 =1Jy
Γ7 =(Jx− Jy)Jx + J2
xz
Γ(3.6)
Γ8 =Jx
Γ
where,
Γ = JxJz− J2xz (3.7)
and Jx, Jy, and Jz are the moment of inertias of the aircraft about X , Y , and Z axes and
Jxz is its only non-zero product of inertia.
We have the following three kinematic equations which relates the body rates to
the Euler angle rates.
φ = p+qsinφ tanθ + r cosφ tanθ
θ = qcosφ − r sinφ (3.8)
ψ =qsinφ + r cosφ
cosθ
The force equations (Eq. (3.1)), the moment equations (Eq. (3.4)), and the kinematic
equations (Eq. (3.8)) together form a system of nine coupled nonlinear first order or-
dinary differential equations which governs the aircraft dynamics. The position of the
aircraft can be obtained by appropriately integrating its velocity and is determined by
the navigation equations
x = cosθ cosψu+(sinφ sinθ cosψ− cosφ sinψ)v
+(cosφ sinθ cosψ + sinφ sinψ)w
y = cosθ sinψu+(sinφ sinθ cosψ + cosφ cosψ)v (3.9)
+(cosφ sinθ sinψ− sinφ cosψ)w
h = sinθu− sinφ cosθv− cosφ cosθw
For the simulations done in this chapter, we use the UAV model used in Aviones flight
simulator (MAGICC Lab, BYU, 2010). This is a UAV of span 1.4224m. Weighing
1.56kg, it is a flying wing configuration similar to Zagi – a popular remote controlled
hobby aircraft (Zagi, 2010). The geometric and inertial parameters of this UAV are
given in Table 3.1. Table 3.2 gives the stability and control derivatives of this UAV
model. The control derivatives corresponding to rudder deflection are not given in
70
Table 3.1: The geometric and inertial parameters of the UAV model used in simulations
Parameter Valuem 1.56 kg
J
⎛⎝ 0.1147 0 0.0015
0 0.0576 00.0015 0 0.1712
⎞⎠ kg m2
S 0.2589 m2
b 1.4224 mc 0.3302 m
Sprop 0.0314 m2
ρ 1.2682 kg/m3
CXδT1
kmotor 20
the table as we do not use rudder deflection to control the UAV. The control surface
deflection limits for elevator and aileron are ±40 degrees.
A small UAV, with a span of about 1.5 m, is chosen as the platform to demonstrate
the efficacy of the proposed collision avoidance algorithm as the missions involving
small UAVs in high density traffic environments are more realistic than the high density
traffic missions of large unmanned aircraft.
We apply our collision avoidance algorithms on 6 DoF UAV models. Toward
this, controllers need to be designed that will track the guidance commands issued
by the collision avoidance algorithms. This controller has to be designed to control
the aircraft about the equilibrium point, that is, about a steady level flight. Below, we
give the procedure to find the control surface deflections, throttle settings, and angle
of attack of the aircraft required (trim conditions) to maintain a steady level flight at a
given altitude and velocity.
3.2 Trimming aircraft for a level flight
The nonlinear UAV dynamics, consisting of force (Eq. (3.1)), moment (Eq. (3.4)), and
kinematic equations (Eq. (3.8)), can be compactly written as
x = f(x,δ ) (3.10)
where, state x = [u v w p q r φ θ ψ ] and control δ = [δe δa δT ]. The
variables have their usual meaning.
71
Table 3.2: Stability and control derivatives of the UAV model used in simulations
Longitudinal Value Lateral ValueCoefficients Coefficients
CL0 0.28 CY0 0CD0 0.03 Cl0 0Cm0 0 Cn0 0CLα 3.45 CYβ −0.98CDα 0.30 Clβ −0.12Cmα −0.38 Cnβ 0.25CLq 0 CYp 0CDq 0 Clp −0.26Cmq −3.6 Cnp 0.022CLδe
−0.36 CYr 0CDδe
0 Clr 0.14Cmδe
0.5 Cnr −0.35CYδa
0Clδa
0.08Cnδa
0.06
To trim a UAV, that is, to find the control surface deflections, the thrust throttle
setting, and the angle of attack for a level flight at a specified speed V , we solve the
following optimization problem
min(α,δ ) ‖ x ‖2 (3.11)
subject to x = f(x,δ )
p = q = r = 0
φ = 0, θ = α , ψ = 0
u = V cosα , β = 0, w = V sinα
The constraints in Eq. (3.11) naturally arises from the steady level flight condition. The
above minimization problem has to search in a four dimensional space (α ,δe,δa,δr)
for solutions. This multidimensional optimization problem can be reduced to a one
dimensional search problem as follows.
For an aircraft that is symmetric about the XZ plane, a level flight would mean a
zero aileron deflection, that is, δa = 0. An explicit expression for elevator deflection
in terms of α can be obtained from the q equation in Eq. (3.4) by setting q = 0 and
imposing other constrains as in Eq. (3.11).
δe =−Cm0 +Cmα αCmδe
(3.12)
72
Similarly, an explicit expression for the trim throttle setting can be obtained by setting
the u equation in Eq. (3.1) to zero.
δT =1
kmotor
√2mgsinα−ρV 2S(CX0 +CXα α +CXeδe)+ρV 2Sprop
ρSpropCXδT
(3.13)
Equations (3.12) and (3.13) along with constraints in Eq. (3.11) makes finding the trim
condition a problem of solving the w equation in Eq. (3.1) for α with w = 0. One
way to do it is to solve it as a one dimensional optimization problem by finding that αwhich minimizes w2.
For most of the simulations in this chapter, the nominal flight condition of the
UAVs, which are, in those cases, assumed to be homogenous, is a cruise flight at an
altitude of 1725m and a velocity of 12m/s. Solving the above optimization problem,
we find that in the nominal cruise flight, the UAV is trimmed at an angle of attack of
6.49 degrees (0.1133 radians) with a corresponding elevator defection of 4.93 degrees
(0.0861 radians) and a throttle setting of 0.7423.
3.3 Controller design for 2D collision avoidance
The objective of the controller, in the case of planar collision avoidance, is to hold
altitude and velocity and track the turn rate command issued by the collision avoid-
ance algorithm. We develop three decoupled controllers for holding (1) altitude, (2)
velocity, and (3) attitude (bank angle). Each of these controllers are designed by suc-
cessive loop closure technique. In all these controllers, the outer loops control the
slower variables and the inner loops control the faster ones.
The outermost loop tries to rectify, through feedback, the error between desired
and actual values of the slowest changing variable by computing through proportional-
integral blocks what the desired value of faster variable should be. The inner loops
handles error in faster variables with the innermost loop dealing with the fastest vari-
able.
3.3.1 Altitude hold controller
The altitude hold controller drives to zero any error between the actual and desired
altitudes. The altitude hold controller has the structure as shown in Fig. 3.2. The
altitude error, as seen in the figure, is fed through proportional-integral blocks which
outputs a desired pitch angle. Since a nonzero pitch angle, θref = α , is required to keep
73
KPh
KIh
s
KPq
KIq
s
KPq
KIq
s
UAVModel
href
h-
++
+
+ - -
+++ +
+
qqref
q
de
deref+
- +
Figure 3.2: Altitude hold controller.
KPV
KIV
s
UAVModel
-
+ ++
V
Vref dT
dTref+
Figure 3.3: Velocity hold controller.
a level flight, it is accounted for when computing the error in pitch. The innermost loop
in the altitude controller is the one with the fastest variable q. The trim elevator setting,
δeref is accounted for in the controller (refer Fig. 3.2).
The values of the Proportional-Integral controller gains, obtained after a manual
tuning, are the following: KPh = 5, KIh = 0.5, KPθ = 5, KIθ = 0.5, KPq = 1, and KIq =
0.1.
3.3.2 Velocity hold controller
Figure 3.3 shows the velocity hold controller. As the controller generates commands
about the equilibrium point of steady level flight, the trim throttle setting, δTref , corre-
sponding to reference velocity, Vref = V , is accounted for. The controller gain values
used for simulations are KPV = 5 and KIV = 0.5.
In reality, it is the throttle command which causes an aircraft to climb/descent
while the elevator deflection will cause a change in its pitch attitude. Therefore, ideally,
it is the throttle control that is to be used for altitude hold. However, we use the elevator
deflection as the control in the altitude hold loop and throttle as the control in the
velocity hold loop. The combined effect of the altitude and velocity hold loops will
help the aircraft to keep a steady level flight as described below.
If the aircraft is below the required altitude, then the elevator controller in the
74
KPf
KIf
s
KPp
KIp
s
UAVModel
fref
f-
++
+
-
+ ++
p
da
Figure 3.4: Attitude hold controller.
altitude hold loop pitches the aircraft up. This causes the angle of attack of aircraft,
and thus the drag, to increase. The increase in drag will cause a velocity drop, to
compensate for which the velocity hold loop will increase the throttle, causing the
aircraft to climb the required altitude.
3.3.3 Attitude hold controller
We use bank to turn the aircraft. Thus a required turn rate is obtained by rolling the
aircraft to the corresponding bank angle. Therefore, we design a controller to hold
the commanded bank angle. The controller to hold a commanded bank angle has a
structure shown in Fig. 3.4. It has two loops, of which the outer loop corrects the error
in the bank angle (slow variable) which when fed through the proportional-integral
blocks will demand the required value of the roll rate (fast variable).
The gain values for attitude hold controller that were chosen, by tuning the inner
loop first and then the outer loop by successive loop closure, are: KPφ = 5, KIφ = 0.5,
KPp = 2, and KIp = 0.05.
When a UAV suddenly banks, the lift vector tilts causing the UAV to lose altitude.
The reference altitude is regained by the combined action of altitude and velocity hold
controllers as described earlier.
3.3.4 Testing the response of the UAV augmented with controller
Both the collision avoidance and the Dubins path to destination algorithms, which
are parts of the 2D collision avoidance algorithms, demand specific turn rates. A
steady level turn rate is achieved by maintaining a bank angle while holding velocity
and altitude. Therefore, we test the response of the UAV, augmented with controllers
designed as described above, to bank angle command. We test the UAV response
75
0 5 10−20
0
20
tφ
(deg
)
0 5 101720
1725
1730
t
h (m
)
0 5 1010
15
t
V (
m/s
)
0 5 100
0.5
1
tδ T
0 5 10−40
−20
0
20
40
t
δ e (de
g)
0 5 10−40
−20
0
20
40
t
δ a (de
g)
Figure 3.5: Response (solid line) of 6 DoF UAV model to bank angle command(dashed line).
with a 3-2-1-1 bank angle command. The 3-2-1-1 is a non-periodic rectangular pulse
input which is commonly used during flight tests to excite all the modes of an aircraft.
The UAV’s response to a 3-2-1-1 bank angle command is shown in Fig. 3.5. The 3-
2-1-1 command requires the bank angle to be 15 degrees for the first three seconds,
−15 degrees for the next two seconds, again 15 degrees for a second and −15 degrees
for the subsequent second, and a zero bank angle thereafter. From Fig. 3.5 we observe
that the UAV exhibits a good response to bank angle command although the elevator
deflections get saturated at times. Also, the rate of aileron deflection demanded is
very high. It is to be noted that the velocity and altitude is maintained throughout via
appropriate elevator deflection and throttle commands.
The plot of steady state turn rate, ψ , achieved for different bank angles is given in
Fig. 3.6(a). It is because the steady state turn rate and bank angle have a linear rela-
tionship, as seen from the figure, that we give a bank angle command to obtain a turn
rate of ψ . For a desired turn rate, the corresponding bank angle required can be found
from the plot in Fig. 3.6(a). From the figure, we find the slope of the line as 0.75.
Therefore, we get the achieved turn rate for a bank angle φ as,
ψachieved = 0.75φ (3.14)
The transient of the achieved turn rate for a commanded bank angle of 10◦ is shown
76
in Fig. 3.6(b). The transient response has got a settling time of around 2 s. It was this
transient response that was used to tune the attitude hold PI controller.
3.4 Simulation results for planar collision avoidance
In this section, we test the collision avoidance algorithm developed in Section 2.2 –
RCA-2D, implemented on 6 DoF UAV models with the controller designed in Sec-
tion 3.3, in a multiple UAV scenario. The 6 DoF UAV equations along with the
controller equations are solved simultaneously using a fourth order Runge-Kutta in-
tegration scheme with a time step of 0.01 seconds.
3.4.1 Tailored test cases
Here we present some test cases to show how UAVs avoid multiple collisions in a
multiple UAV free flight scenario. In these test cases, the initial positions and orienta-
tions of the UAVs are selected so that during flight they encounter multiple conflicts.
The UAVs try to maintain a constant speed of 12 m/s and an altitude of 1725 m.
Figure 3.7 shows the trajectories of 3 UAVs in a free flight airspace which resolved
their conflicts using Algorithm 2.3. Shown in Fig. 3.8 are the important states (bank
angle, altitude and velocity) and the corresponding control inputs (aileron deflection,
elevator deflection and throttle setting) for one of the UAVs. From the figure, we
observe that the altitude and velocity were maintained throughout the simulation time.
Also, the bank angle response, as seen in the figure, indicates the collision avoidance
maneuvers that the UAV made.
Similar cases of multiple collision avoidance involving 4 and 5 UAVs are given in
Figs. 3.9 and 3.10, respectively. Table 3.3 gives the initial positions and orientations of
the UAVs involved in the tailored test case involving 5 UAVs. The destination of each
UAV is 200 meters ahead of the initial position in the direction of its initial orientation.
These cases are tailored in such a manner that a UAV avoiding one conflict gets
into another. However, as seen from the figures, all the conflicts are eventually resolved
without any occurrence of near misses. For the simulations in this section, we do not
use an actuator model. A first order actuator model is considered during the controller
design for the implementation of the 3D collision avoidance which is dealt with in
Section 3.5.
77
0 5 10 15 200
5
10
15
φ (deg)
d/dt
(ψ)
(deg
/s)
(a) Turn rate achieved for various bank angles
0 1 2 3 4 5−5
0
5
10
15
20
t (s)
d/dt
(ψ)
(deg
/s)
(b) Turn rate transient for bank angle of 10◦
Figure 3.6: Turn rates (steady state and transient) for 6 DoF UAV model.
78
meters
met
ers
−100 −80 −60 −40 −20 0 20 40
−40
−20
0
20
40
60
Figure 3.7: Collision avoidance of 3 realistic UAVs.
0 2 4 6 8 10−50
0
50
φ
0 2 4 6 8 101700
1710
1720
1730
1740
h
0 2 4 6 8 100
5
10
15
20
t
V
0 2 4 6 8 10−40
−20
0
20
40
δ a
0 2 4 6 8 10−40
−20
0
20
40
δ e
0 2 4 6 8 100
0.5
1
t
δ T
Figure 3.8: Control inputs and corresponding states for one of the UAVs in Fig. 3.7.
79
meters
met
ers
−100 −80 −60 −40 −20 0 20 40 60−80
−60
−40
−20
0
20
40
Figure 3.9: Collision avoidance of 4 realistic UAVs.
−80 −60 −40 −20 0 20 40 60 80
−60
−40
−20
0
20
40
60
meters
met
ers
U1
U2
U3
U4
U5
Figure 3.10: Collision avoidance of 5 realistic UAVs.
80
Table 3.3: Initial positions and orientations of UAVs in Fig. 3.10
UAV Initial position Initial orientation(x,y) in meters in degres
1 (60,-40) 752 (-30,-70) 903 (-90,-40) 154 (20,70) 2455 (90,10) 195
Table 3.4: Random flight test case for 6 DoF UAV models with inner and outer radiias 4 km and 5 km (values averaged over 10 runs)
No. of without collision avoidance with collision avoidanceUAVs Near misses Efficiency Near misses Efficiency
20 2.3 100 0 99.99940 8.1 100 0 99.99460 22.4 100 0 99.97980 36.9 100 0 99.981
3.4.2 Random flight test case
We consider the test case of random flights described in Section 2.5.1. Throughout
the test, all UAVs try to maintain a constant speed of 12m/s with the aid of velocity
hold loop of the controller and the maximum turn rate is limited to 10 degrees per
second. The performance measures are again near misses and efficiency as defined in
Section 2.5.1. The desired separation parameter, which is a tuning parameter, in the
proposed collision avoidance algorithm is chosen as 20m. The UAVs are assumed to
have a sensor range of 200m. The UAVs can detect the position and velocity of other
UAVs within this range. A near miss is deemed to occur if two UAVs come within
10 m of each other.
Simulations were done for 20, 40, 60, and 80 UAVs. For each case 10 simulations
were done. For the random flight test, we used radii of 4 and 5 km for the inner and
outer circles. There was no near miss in any of the simulations in this case. The results
are shown in Table 3.4. From the table, we observe that for the case of 80 UAVs, the
possible occurrences of about 37 near misses per simulation is reduced to zero, without
much loss in efficiency, when the proposed collision avoidance algorithm is used.
As we see from Table 3.4, the efficiencies achieved are high. This could be because
the area of the test field considered is large. Therefore, we consider a case where the
inner and outer radii are selected as 400 and 500 m. The results for this case is given in
81
Table 3.5: Random flight test case for 6 DoF UAV models with inner and outer radiias 400 m and 500 m (values averaged over 10 runs)
No. of without collision avoidance with collision avoidanceUAVs Near misses Efficiency Near misses Efficiency
20 218.1 100 0.1 96.1540 899.1 100 1.6 89.1760 2027.9 100 1.4 89.11
Table 3.5. As seen from the table, not applying any collision avoidance maneuver will
result in over 2000 near misses for an aircraft density of 60. The application of RCA-
2D reduces the average number of near missed to about 2. However, this reduction in
the number of near misses comes at the cost of efficiency. The efficiency reduces to
about 90% which means that some of the UAVs have to take long detours to avoid near
misses. Nonetheless, even this drop in efficiency is acceptable considering the small
size of the free flight area considered for this study and the achieved reduction in the
number of near misses.
3.5 Controller design for 3D collision avoidance
The three dimensional collision avoidance algorithm demands an acceleration in 3D,
but perpendicular to the velocity vector. This demanded acceleration can be split into
longitudinal and lateral accelerations; correspondingly, the pitch and the turn rates,
respectively. The demanded acceleration can be translated into equivalent γ and ψrequirements, where, γ and ψ are the flight path and the course angles. We propose to
achieve the demanded acceleration via controllers to hold flight path angle and course
angle. In addition, a velocity hold controller will be used to maintain a constant veloc-
ity.
3.5.1 Six DoF model
We consider the same model that is described in Section 3.1. In addition, we consider
the following first order actuator model.
δe = kδe(δec−δe)
δa = kδa(δac−δa) (3.15)
δT = kδT(δTc−δT )
82
where, δec , δac , and δTc are the commanded elevator and aileron deflections and throttle
setting, and kδe, kδa
, and kδTare the corresponding time constants. For the simulation
results presented in Section 3.6, we use kδe= kδa
= kδT= 0.1.
3.5.2 Pitch rate guidance and control loops
We design a controller to hold the flight path angle in a way similar to the altitude
hold controller in Section 3.3.1. A desired pitch rate is demanded by the collision
avoidance algorithm. The demanded γ is integrated to obtain a demanded flight path
angle which is achieved via a flight path angle hold controller. This is illustrated in
Fig. 3.11. In the design of controller, we have assumed that γ = θ −α although this
is not strictly valid in the presence of sideslip. Nonetheless, simulations show that this
does not affect much the performance of the controller. The values of the constants of
the PI-controller are chosen as KPγ = 5, KIγ = 0.5, KPq = 1, and KIq = 0.1.
3.5.3 Turn rate guidance and control loops
The attitude hold controller in Section 3.3.3 is modified by adding an extra loop to hold
the course or the heading angle. This course hold controller is used as the inner loop
of the turn rate guidance loop to attain a desired turn rate as specified by the collision
avoidance algorithm. The turn rate guidance and control loops are shown in Fig. 3.12.
The following values are chosen for the PI-controller constants. KPψ = 2.5, KIψ = 0.1,
KPφ = 2.5, and KIφ = 0.1 KPp = 2.5, and KIp = 0.1.
3.5.4 Response of UAV augmented with controller
Now, we study the response of the 6 DoF UAV model augmented with pitch and turn
rate guidance and control loops as described above. The response to pitch rate and turn
rate commands are analyzed separately. Figure 3.13 shows the response of the control
augmented UAV to a doublet pitch rate demand. The doublet command is as follows:
Starting with a level flight condition, a steady pitch rate of 10 degrees is demanded
for 5 s, a pitch rate of −10 degrees for next 5 s, and back to steady level cruise. From
Fig. 3.13 we observe that the pitch rate demands are achieved with short rise and
settling times and a little overshoot. Figure 3.13 also shows the corresponding control
surface deflections. Note the unsteady nature of the elevator and throttle deflections.
This is because, unlike a constant turn rate, the maneuver consisting of constant pitch
rate is not a steady state motion of the aircraft.
83
X
YZ
V
a
6 D
oF U
AV
Mod
el
KP q
KI q s
KP g
KI g s
1 sG
uida
nce
gde
sire
d
.
qg
-
++
+
-
+
+
+
de
UA
V
posi
tion
and
velo
city
posi
tions
and
vel
ociti
esof
nei
ghbo
ring
UA
Vs
de r
ef
+
Figure 3.11: Pitch rate guidance and control loops.
84
X
YZ
V
a
6 D
oF U
AV
Mod
el
KP p
KI p s
1 sj
KP j
KI j s
-+
+G
uida
nce
jde
sire
d
.
p-
+
+
da
KP f
KI f s
-+
++
f
UA
Vpo
sitio
n an
d ve
licity
posi
tions
and
vel
iciti
esof
nei
ghbo
ring
UA
Vs
Figure 3.12: Turn rate guidance and control loops.
85
0 5 10 15
−10
0
10γ (deg/sec)
0 5 10 15
−10
0
10δe (deg)
0 5 10 15
−10
0
10ψ(deg)
0 5 10 15
−10
0
10δa (deg)
0 5 10 158
10
12
14
V (m/s )
t (s ec)0 5 10 15
0
1
2
δT
t (s ec)
Figure 3.13: UAV response to a doublet pitch rate demand.
Figure 3.14 shows the response of the UAV to a doublet turn rate demand. The
figure also shows the corresponding control surface deflections. The turn rate response
is a bit more sluggish than the pitch rate response since it has higher number of control
loops (refer Fig. 3.12). The behavior can be further tuned by proper choice of controller
constants. However, we do not further emphasize on the control design as, for the
purpose of this thesis, it suffices to show that the guidance algorithms developed are
easily implementable on realistic UAV models.
3.6 Simulations results of 3D collision avoidance
In this section, we present various simulation scenarios using the three dimensional
collision avoidance algorithm implemented on realistic 6 DoF UAV models.
For collision avoidance, the guidance algorithm requires velocity of UAVs in the
inertial frame of reference. The 6 DoF model provides the velocity in the body frame
of reference. The velocity components in the inertial frame of reference is obtained
from that of body frame of reference via the following transformation.⎡⎣ u
vw
⎤⎦
inertial
= Tb2i
⎡⎣ u
vw
⎤⎦
body
(3.16)
86
0 5 10 15
−10
0
10ψ (deg/sec)
0 5 10 15
−10
0
10δe (deg)
0 5 10 15
−10
0
10γ (deg)
0 5 10 15
−10
0
10δa (deg)
0 5 10 158
10
12
14
V (m/s )
t (s ec)0 5 10 15
0
1
2
δT
t (s ec)
Figure 3.14: UAV response to a doublet turn rate demand.
where,
Tb2i =
⎡⎣ cosθ cosψ sinφ sinθ cosψ− cosθ sinψ cosφ sinθ cosψ + sinφ sinψ
cosθ sinψ sinφ sinθ sinψ + cosφ cosψ cosφ sinθ sinψ− sinφ cosψ−sinθ sinφ cosθ cosφ cosθ
⎤⎦
(3.17)
3.6.1 Random flights
We consider the case of random flights as described in Section 3.4.2. UAVs fly from a
random point on the outer circle of radius 500 m to a point on the inner circle of radius
400 m. A specified aircraft density is maintained by replacing the UAVs that reach
the destination by new ones. The near misses and efficiency achieved for various
aircraft density – number of aircraft in the concerned airspace – is given in Table 3.6.
Note that the collision avoidance performance of the 3D collision avoidance algorithm
implemented on realistic UAV models in terms of near misses is same as that of the
2D algorithm (refer Table 3.5). However, the efficiency of the 3D algorithm is much
higher – close to 100%. As the test case of random flights is a planar test case, use of
a 3D algorithm that takes advantage of the otherwise unused third dimension allows
to negotiate conflicts, in a high density traffic, very efficiently. As the UAVs, trying
87
Table 3.6: Random flight test case with 3D collision avoidance for 6 DoF UAV models,and inner and outer radii of 400 and 500 m (values averaged over 10 runs)
No. of UAVs Near misses Efficiency20 0.4 99.9240 1.2 99.8660 2.7 99.79
Table 3.7: Performance of the 3D algorithms in 6 DoF UAV models for the test caseof modified random flights (40 UAVs, averaged over 10 runs)
Case Near Misses Efficiency (%)1 0.7 99.752 0.7 99.64
to avoid each other, using the 3D collision avoidance algorithm, climb to different
altitudes will reduce further conflicts.
3.6.2 Test case of modified random flights
As stated in Chapter 2, the real power of 3D collision avoidance algorithm is in han-
dling inherently three dimensional conflicts, and such a test case – modified random
flights – was presented in Section 2.5.2. We consider two cases. Case 1: h = 10 m,
rin = 300 m and rout = 400 m. Case 2: h = 20 m, rin = 200 m and rout = 300 m. Re-
fer Fig. 2.17 and Section 2.5.2 for clarifications. The average results of Monte Carlo
simulations of the modified random flights with an aircraft density of 40 is given in
Table 3.7.
3.6.3 Perpendicular flow
The test case of perpendicular flow as considered in Section 2.5.2 is considered now.
As expected, one stream of aircraft goes under and the other climbs over the other.
This is shown in Fig. 3.15. This maneuver is more efficient than the aircraft sticking
to the plane and doing an interleaving motion as given in (Mao et al., 2000).
3.6.4 Non-cooperating UAVs
As the collision avoidance algorithms that we proposed in Chapter 2 are reactive, it
works as effectively even if some of the aircraft in the conflict are non-cooperating.
88
Figure 3.15: Collision avoidance in three dimensional perpendicular flow.
By ‘non-cooperating’ UAV, we mean that the corresponding UAV does not alter its
path even if there is any conflict. To illustrate this, we consider the following test
case – the test case of array flow – as illustrated in Fig. 3.16. A collision avoidance is
achieved even if only one set of UAV performs the collision avoidance maneuver. This
is shown in Fig. 3.17. The case where all the aircraft cooperate is shown in Fig. 3.18.
3.6.5 Heterogeneous UAVs
The collision avoidance algorithms work well in the presence of heterogeneous – dif-
ferent speed – UAVs. We consider the test case of random flights with heterogeneous
UAVs. When a UAV is introduced into the airspace, it is randomly assigned one of the
following four – 10, 12, 15, or 18 meters per second – speeds which it will maintain
through the rest of its flight.
The trim conditions – trim angle of attack, trim elevator deflection (δere f ), and trim
throttle setting (δTre f ) – corresponding to different velocities that we consider are given
in Table 3.8. Note that the motor that drives the propeller is so modeled that the throttle
control δT can take values between 0.5 and 2.0. Results of the Monte Carlo runs of
this test case is presented in Table 3.9.
89
Figure 3.16: The test case of array flow where none of the UAVs cooperate resultingin collisions.
Figure 3.17: The test case of array flow where some of the UAVs cooperate.
90
Figure 3.18: The test case of array flow where all the UAVs cooperate.
Table 3.8: Trim conditions corresponding to various steady level cruise velocities
V αtrim δere f δTre f
m/s degrees degrees10 11.43 8.69 0.6612 6.49 4.93 0.7415 2.39 1.82 0.8718 0.14 0.10 1.01
Table 3.9: Random flight test case with 3D collision avoidance, and inner and outerradii of 400 and 500 m, for heterogeneous 6 DoF UAV models (values averaged over10 runs)
No. of UAVs Near misses Efficiency20 0.6 99.9040 1.5 99.8260 2.3 99.75
91
3.7 Summary and conclusions
In this chapter, we implemented the collision avoidance algorithms developed in
Chapter 2 on realistic six degree of freedom UAV models augmented with controllers
designed using successive loop closure. The studies in this chapter showed that the
proposed collision avoidance algorithms are easily implementable in realistic UAVs,
and thus hold promise for applications in multiple UAV missions. Although the col-
lision avoidance algorithms were developed assuming kinematic UAV models, their
performances did not degrade upon use in realistic 6 DoF UAV models. We used tai-
lored test cases to validate the implementation of these algorithms, this is, to ensure
that the conflicts are handled correctly. Through simulations, in a random free-flight
high density traffic environment, we showed that the proposed collision avoidance al-
gorithms, when implemented in realistic UAV models, performs well. Also, we carried
out simulations to demonstrate that the collision avoidance algorithms perform well in
the presence of heterogeneous and non-cooperating UAVs.
Having addressed the collision avoidance of multiple UAVs, we now turn our at-
tention to another important problem in multiple UAV missions – coalition formation.
In the next chapter, we consider the coalition formation of UAVs, that can communi-
cate with each other, for target search and prosecute missions.
92
4 Coalition formation with global communication
Many missions involving multiple UAVs require the UAVs to form coalitions in or-der to successfully complete the mission. This chapter looks at the multiple UAVmission to search and prosecute multiple targets and give computationally efficientcoalition formation algorithms. Once the targets are detected by UAVs, coalitionsare formed by UAVs communicating with each other where it is assumed that thecommunication is global or all-to-all. The coalition formation problem is shown tobe NP-hard. This chapter develops a sub-optimal but polynomial time algorithm,and the solution thus obtained is compared with that obtained using Particle SwarmOptimization. Simulations are carried out to study the effects of variation in numberof UAVs and targets on the coalition formation process. The effect of using hetero-geneous UAVs for the search and prosecute mission is also studied.
�he UAVs are usually small, to facilitate portability. Therefore, it may not be
possible for a single UAV to carry all the resources required to successfully
carry out a mission. This necessitates the formation of coalitions of UAVs
to complete certain missions. In this chapter, we in particular consider a multiple UAV
search and prosecute mission. The UAVs perform a distributed search to find targets.
Once a target is detected, the UAVs communicate with each other to form coalitions
such that the combined resources of the coalition are sufficient to prosecute the target.
The chapter is organized as follows. Section 4.1 gives the problem description and
formulation. The coalition formation algorithms for the search and prosecute mission
and their analysis are given in Section 4.2. Section 4.3 gives the algorithms that the
UAVs in a coalition should employ to rendezvous at the target to inflict maximum
damage. Optimal solution to the problem using Particle Swarm Optimization is given
in Section 4.4. Section 4.5 gives the simulation results. Section 4.6 concludes the
chapter with a summary.
4.1 Problem formulation
We consider a scenario in which a search and prosecute mission is carried out on a
battlefield using N heterogeneous UAVs (also called agents). The UAVs are hetero-
geneous as they can carry different types of resources. The quantity of the resources
that a UAV can carry is limited. Some of these resources are consumable, that is, the
93
resources deplete with use, like fuel and weapons, while others, such as sensors, do
not deplete. Each UAV is identified by its unique token number Ui, i = 1, . . . ,N that is
assigned prior to the start of the mission. We assume that there are M targets whose
positions are unknown.
We assume that a UAV, Ui, can carry n types of resources and its resource capability
vector is denoted by RUi of the form
RUi =(
RUi1 , . . . ,RUi
n
), i = 1, . . . ,N (4.1)
where, RUip , p = 1, . . . ,n represents the number of type-p resources held by UAV Ui.
For example, RUi = (4,2,0,6) implies that the UAV Ui has four type-1 resources
(RUi1 = 4), two type-2 resources (RUi
2 = 2), no type-3 resources (RUi3 = 0), and six type-4
resources (RUi4 = 6).
We assume that the UAVs have limited sensor range and do not have a priori knowl-
edge of the target locations and their resources. To detect the targets, the UAVs have
to carry out a search task. Although there are several search strategies for efficiently
finding targets in an area of interest (Sujit & Ghose, 2009, 2010, 2011), and in partic-
ular, search strategies for a search and prosecute mission (George et al., 2011), in this
thesis, we assume that the UAVs perform a random search. Once a UAV Ui detects a
target Tj, we assume that the UAV that detected it can also determine the type of re-
sources required to prosecute the target. If m different types of resources are required
to prosecute target Tj, then the resource requirement vector is represented as
RTj =(
RTj
1 , . . . ,RTjm
), j = 1, . . . ,M (4.2)
where, RTjq , q = 1, . . . ,m represents the quantity of type-q resources required to prose-
cute the target Tj and m≤ n by assumption. For example, RTj = (3,0,5) indicates that
to prosecute target Tj, the UAVs need three type-1 resources (RTj
1 = 3), zero type-2
resources (RTj
2 = 0), and five type-3 resources (RTj
3 = 5).
We assume that each UAV has the capability to communicate with other UAVs.
Initially, we consider global communication where every UAV can communicate with
every other UAV in the operating zone. This assumption is later relaxed in Chapter 5
where we assume that a UAV can communicate with only those UAVs that are within
its finite communication range.
Once a target Tj is located and identified by a UAV Ui, it assumes the role of a
coalition leader and broadcasts the target resource requirement vector (RTj ). The UAVs
in the search region, that are within the communication range of Ui, possessing at least
94
one of the required resources to prosecute the target Tj will respond to Ui with their cost
and resource capabilities. The coalition leader attempts to form a coalition from among
the UAVs that responded such that the combined resource capabilities of the coalition
is greater than or equal to the target resource requirement. After a feasible coalition is
formed, the UAVs in the coalition rendezvous at the target to inflict maximum damage.
Thus, the cost of a UAV to prosecute a target is based on the earliest time of arrival
(ETA) of the UAV at the target location using Dubins curves.
The objective function that we consider to minimize is the time for prosecution
and, subsequently, the time for mission completion. Other possible objectives can be
to minimize fuel consumption, effort, et cetera (George et al., 2011), which we will
not consider in this thesis.
Assume that the UAV team takes τ time units to accomplish a mission of prose-
cuting all the targets. The mission time τ depends on the time taken by the UAVs to
detect the targets which in turn depends on the performance of the search strategy and
the time taken to form coalitions and to prosecute each detected target. The objective
is to accomplish the mission in minimum time. That is, the global objective is
minτ (4.3)
where the minimization is over all possible coalitions and prosecution sequences. Here
τ is bounded above by the sum of τs and ∑Mj=1 τTj
c , where τs is the search time and τTjc
represents the time taken by the coalition to prosecute the target Tj once it is detected.
The mission time τ is not equal to but bounded by the above sum because some of the
target prosecutions can happen simultaneously. It is to be noted that the search time τs
implicitly depends on the coalition formation strategies. This is because the choice of
coalition will decide the UAVs that are free to carry out the search task. For example,
for a given target, out of two coalitions with same prosecution time, the coalition with
lower size will make more UAVs available for search which may lead to a reduction
in the total search time. We attempt to achieve the global objective in Eq. (4.3) by (a)
minimizing the time taken to prosecute each target, that is,
minτTjc , j = 1, . . . ,M (4.4)
where the minimization is over all possible coalitions, and (b) by minimizing the size
of the coalition formed to prosecute it. Also, a simultaneous strike is desirable to
successfully prosecute the target. Thus, the task of a coalition leader is to form a coali-
tion such that (i) the target is prosecuted in minimum time, (ii) the coalition contains
minimum number of UAVs, and (iii) the coalition members simultaneously prosecute
95
the target. The objective (i) (minimizing the time-to-attack the target) is a desirable
quantity in achieving the objective given in Eq. (4.4), the objective (ii) (which is to
minimize coalition size) ensures that the UAVs distribute their search effort so that the
targets are detected quickly and the mission is accomplished faster, and the objective
(iii) induces maximum damage to the target.
The coalition leader who forms a coalition to prosecute a target is that UAV which
detected the target. The role of the coalition leader Ui that detected target Tj is to form
a coalition C (Ui,Tj), with combined resources RC (Ui,Tj) ≥ RTj , to prosecute Tj taking
into account (i)–(iii). Let the set of all UAVs that responded to the coalition formation
request of Ui to prosecute target Tj, the potential coalition members, be denoted by
PC (Ui,Tj). Let D(Ui,Tj) = {DTjU : U ∈PC (Tj,Ui)} represent the set of earliest time
of arrivals for the UAVs in the set of potential coalition members, and DTjU denotes the
earliest time of arrival (ETA) of the UAV U at the location of target Tj using Dubins
curves (Dubins, 1957). The ETA is used as the cost metric. To determine a coalition,
the coalition leader solves the following optimization problem.
argminC ′(Ui,Tj)⊆PC (Ui,Tj)
(max
U∈C ′(Ui,Tj)D
TjU
)(4.5)
subject to
(∑
U∈C ′(Ui,Tj)RU
p
)≥ R
Tjp , for all p = 1, . . . ,m (4.6)
The solution set C (Ui,Tj), to this optimization problem is the smallest coalition which
satisfies Eq. (4.6), that is, if C ′′(Ui,Tj) ⊆ PC (Ui,Tj) is another set which is a so-
lution to Eqs. (4.5) and (4.6), then |C (Ui,Tj)| ≤ |C ′′(Ui,Tj)|. The set obtained as a
solution to the above optimization problem satisfies the objectives (i) and (ii). It is to
be noted that the optimum value of objective function in Eq. (4.5) is the earliest time
at which all the members of the coalition, C (Ui,Tj), can arrive at the target, Tj, which
is equal to τTjc , the time taken by coalition to prosecute Tj. Therefore, solving the op-
timization problem in Eqs. (4.5) and (4.6) for every target will minimize the objective
function given in Eq. (4.4). Also, the coalition members have to prosecute the target
simultaneously (objective (iii)). In order to prosecute simultaneously, the UAVs have
to arrive at the target location at the same time. The minimum time (objective (i)) for
all the UAV in a coalition to arrive at the target is determined by the UAV that has
the highest ETA to target. As we need to determine both the smallest coalition and
the coalition that can prosecute that target in minimum time, the objective function is
coupled. This coupling makes the optimization problem nontrivial to be solved using
standard techniques. We call this optimization problem the minimum time minimum
member coalition formation problem and show that it is NP-hard.
96
Theorem 4.1 The problem of finding the minimum-time minimum-member coalition
that can prosecute a target with a given resource requirement is NP-hard.
Proof: We show this by reducing, in polynomial time, the NP-hard bottleneck travel-
ing salesman problem to an instance of the minimum time minimum member coalition
formation problem; the decision version of the bottleneck traveling salesman problem
is NP-complete. The bottleneck traveling salesman problem can be stated as follows.
Find the Hamiltonian cycle in a weighted graph which minimizes the
weight of the most weighty edge of the cycle (Garey & Johnson, 1979).
The bottleneck traveling salesman problem is NP-hard (Garey & Johnson, 1979). For
a given graph (V ,E ) with |V | = m and |E | = N, we can assume, without loss of
generality, that V = {1,2, . . . ,m} and E = {1,2, . . . ,N}. Let each vertex denote a type
of resource, and let each edge correspond to a UAV. For an edge k joining vertices i and
j, the corresponding UAV Uk will have one i-type resource and one j-type resource.
Thus the resource vector of Uk is RUk = (0,0, . . . ,0,1,0, . . . ,0,1,0, . . . ,0); with 1’s at
ith and jth positions and 0’s elsewhere. Let the value of the edge k correspond to the
earliest time of arrival of the corresponding UAV Uk. Define a target with resource
requirement of two of each type of resources, that is, the resource requirement is the
m-tuple (2,2, . . . ,2). Given this correspondence, it follows that the minimum time
minimum member coalition that can prosecute the target corresponds to a Hamiltonian
cycle with least value for the most weighted edge.
The target resource requirement of 2 for each resource type along with the fact
that every UAV has at most one resource of any given type will ensure that all the
vertices are visited, that is, every vertex will have at least two associated edges. The
minimum member requirement will ensure that no more than two UAVs with same
type of resource appear in the coalition, that is, no more that two edges are associated
with every vertex. The above two together gives a Hamilton path. The minimum time
requirement of the coalition will ensure that the weight of the most weighty edge of
the corresponding Hamiltonian is the least of all possible Hamiltonian cycles.
Thus the optimization problem that we are tackling is NP-hard and therefore tough
to solve; any exact solution to the problem will not scale well. The solution to the
optimization problem (Eqs. (4.5) and (4.6)) can be found by searching the complete
solution space which is computationally expensive. The computational effort, in that
case, increases exponentially with the number of UAVs and targets, since the problem
97
is NP-hard as shown above, unless P=NP. Since the coalition formation algorithm that
we develop is indented to be used in high density traffic environments, it necessarily
has to be scalable. Therefore, we give a sub-optimal, but polynomial time, algorithm
for the minimum-time minimum-member coalition formation problem. Our solution
handles the problem by decomposing it into two-stages. In the first stage, we will
determine a coalition set that can prosecute the target in minimum time, and using this
set we determine the smallest size coalition that satisfies the constraints. We develop
both a suboptimal polynomial time algorithm as well as an optimal algorithm. In the
next section, we describe the two-stage coalition formation algorithms.
The UAV that detects a target becomes the coalition leader and forms the coalition.
Hence, the coalition formation algorithms are decentralized by nature which has sig-
nificant advantages over centralized solution concepts. However, if the target locations
and their resources, the UAV positions and their resources are known a priori, then we
can solve the global optimization problem off-line. The global optimization problem
has to determine the appropriate coalitions for each target taking the reduction of the
UAV resources into account. The global solution should also determine the order in
which targets are prosecuted. In order to find a solution to this highly complex problem
and to take the dynamics of the resource depletion into account, we propose a Particle
Swarm Optimization (PSO) based solution which we describe in Section 4.4.
The UAVs are subjected to kinematic constraints preventing instantaneous course
changes. We assume that the autopilots of the UAVs hold a constant altitude and
ground speed. The kinematics of the ith UAV is therefore modeled using first order
kinematics as
xi = Vi cosψi,
yi = Vi sinψi,
ψi = k(ψdi −ψi), i = 1, . . . ,N (4.7)
where, xi and yi gives the UAV location, ψi is its heading, ψdi is the desired heading,
Vi is the ground speed, and k is an autopilot gain. We assume the heading rate to be
constrained within bounds. That is
−ωmax ≤ ψi ≤ ωmax, i = 1, . . . ,N (4.8)
The UAVs perform a search task to detect targets. When a target is found, a coalition
with other UAVs is formed to prosecute the target. In the next section, the coalition
formation process is discussed.
98
4.2 Coalition formation
The heterogeneous UAV swarm has to prosecute the targets by forming coalitions.
A coalition is formed based on the target resource requirements for prosecution. A
coalition is a temporal team and the agent/UAV that initiates a coalition formation
request is the coalition leader while the agents which form the rest of the coalition
are called coalition members. The coalition leader may not necessarily be a part of
the coalition because (a) it does not possess the required resources, or (b) the rest of
the coalition can perform the task without its presence, or (c) the coalition leader is
committed to be a part of another coalition. The role of the coalition leader is to form
a coalition and provide the simultaneous strike information to the coalition members.
Once the coalition is formed and the leader broadcasts the coalition information, its
role ceases.
In the search region, many UAVs can detect targets and assume the role of a coali-
tion leader to determine coalitions for the detected targets. When a coalition leader
announces its target information, all the UAVs that have not been assigned to any tar-
get and which have any of the required resources will respond. Other coalition leaders
may also respond to the coalition formation requests. When an agent detects multiple
targets simultaneously, it randomly chooses one of the targets and becomes coalition
leader to form a coalition to prosecute that target.
If a coalition leader is not able to form a feasible coalition (probably because the
UAVs having required resources are already part of another coalition), then the coali-
tion leader re-broadcasts the coalition request as long as the target is within its sensor
range. Once the target is out of its sensor range, it ‘forgets’ the target and that target
will be prosecuted only when it is detected again and a successful coalition is formed
by some UAV.
Since the number of UAVs available to form coalitions are limited, there can be sit-
uations where none of the coalition leaders are able to form coalitions due to deadlocks.
Deadlock is a situation where multiple coalition leaders attempt to form coalitions but
none of them are able to form a coalition. This situation can occur since the total avail-
able UAV resources (sums of all the UAV resources) are not appropriately distributed
between the coalition leaders. A deadlock usually is eventually solved in time as the
UAVs are in motion, however, this situation causes delay in forming the coalitions and
hence the objective of minimizing the coalition prosecution time may not be achieved
efficiently. Note that the deadlock is different from a situation where the total available
99
agent resources are not sufficient to prosecute a target, in which case, the target will
never be prosecuted. Below, we give a mechanism to break deadlocks and show its
efficacy through an example.
To break deadlocks, as described above, that might occur in the course of a search
and prosecute mission, the UAVs use unique token numbers which are assigned to them
prior to the commencement of the search mission. The token mechanism determines
the preference with which agents respond to simultaneous multiple coalition forma-
tion requests. Thus, if a UAV receives simultaneous requests from multiple coalition
leaders, all of which it can contribute to, then it responds to the coalition leader with
the highest token number.
Other deadlock avoiding mechanisms can be used. For example, consider the ETA
of a UAV to target location broadcast by a coalition leader as the deadlock avoidance
mechanism. Assume that a UAV Uk receives requests from various potential coalition
leaders. Then the UAV Uk will determine ETA for each of the targets and select a
coalition leader which had broadcast the target to which Uk has minimum ETA.
Although using ETA is a simple metric, it may not avoid deadlocks consistently.
For example, consider a situation shown in Fig. 4.1, where UAVs U1 and U2 detect
targets T1 and T2, respectively and need to form coalitions. Let the target resources
be RT1 = (2,3,4), and RT2 = (3,1,2), and UAV resources be RU1 = (1,2,1) ,RU2 =
(2,1,0) ,and RU3 = (1,1,2) ,RU4 = (1,1,1). When the coalition formation request is
broadcast by U1 and U2 for T1 and T2, respectively, then the UAVs U1 and U3 will
respond to proposal of U1 as their ETA to T1 is smaller than ETA to T2. The UAVs
U4 and U2 send their bid to U2, because their ETA to T2 is smaller than to T1. In this
case, the coalition resources of target T1 are RC (U1,T1) = (2,3,3), while that of T2 are
RC (U2,T2) = (3,2,1). We can see that RC (U1,T1) and RC (U2,T2) do not meet the RT1 and RT2
constraints and therefore no coalitions are formed causing a deadlock. This situation
may prevail for some more time until the geometry of the UAV locations changes and
one of the coalition leaders finds sufficient coalition resources to form a coalition.
In the above example if we use the token mechanism then the UAVs will respond
to U2 first as U2 has higher token number than U1. The agents U1,U2,U3, and U4 will
respond to the request and a coalition constituting U2 and U3 is formed. After U2
determines its coalition, U1 will broadcast again and U4 will send its information as a
response to the broadcast of U1. However, the cumulative resources of U1 and U4 do
not satisfy the target requirement for T1, hence a coalition is not formed. Using this
scheme, we can see that at least one coalition has been formed to execute the task as
compared to no coalitions. As long as the sum of resources of the members intending
100
U1
U3 U4
U2
T2
T1
Figure 4.1: A situation where multiple agents detect multiple targets.
to be a part of the coalition is greater than or equal to the resources of the target, the
token mechanism guarantees that at least one coalition is formed. For simulations in
this chapter, we use the token mechanism.
Once a UAV detects a target, it has to form a coalition that depends on the current
state of the UAV. Figure 4.2 shows two different states (S1 and S2) and how the UAV
makes a coalition. The sequence of actions carried out during these states are described
below.
Coalition leader has all the required capabilities (S1)
In this case, UAV Ui detects target Tj that requires RTj resources. If RUip ≥ R
Tjp for all
p = 1, . . . ,m, and if Ui is not already a part of another coalition, then Ui would attack
target Tj without requesting a coalition with other UAVs. This is a special case of
coalition formation where only one UAV comprises the coalition. The UAV determines
the route to travel using Dubins curves (Dubins, 1957), then proceeds toward the target
as shown by the state S1 in Fig. 4.2.
Coalition leader has partial resources or no resources (S2)
This state occurs when a UAV Ui detects target Tj that requires RTj resources, but
Ui has insufficient resources. In this case, Ui assumes the role of coalition leader and
broadcasts the information about the target, that is, its location and required resources,
to the other UAVs. The UAVs that have at least one of the types of required resources
will send their cost to arrive at the target (ETA) and their available resource vector. The
coalition leader considers all the responses and determines whether a coalition can be
101
In c
oalit
ion?
Det
ect T
j and
dete
rmin
e RT j
Ui h
as a
ll th
ere
sour
ces
Ui h
as in
suff
icie
ntre
sour
ces
Det
erm
ine
path
to ta
rget
and
pros
ecut
e it
Bro
adca
st T
j and
RT j
Rec
eive
pro
posa
ls
for c
oalit
ions
Coa
litio
n fe
asib
le?
Bro
adca
st
'dis
card
coa
litio
n'
Det
erm
ine
and
broa
dcas
t t cT j
and
C (U
i ,T j
)
Che
ck if
Uk
has a
ny o
f the
re
quire
d re
sour
ces
for T
j
No
coal
ition
inte
rest
Cal
cula
te D
T j a
nd
broa
dcas
t alo
ng w
ith R
Uk
Uk
No
coal
ition
inte
rest
Det
erm
ine
path
to ta
rget
and
pros
ecut
e it
Dec
isio
n pr
oses
s for
coa
litio
n le
ader
Ui
Dec
isio
n pr
oses
s for
pot
entia
l coa
litio
n m
embe
r Uk
To o
ther
UA
Vs
From
oth
er U
AV
s
To o
ther
UA
Vs
S1
S2
Yes
No
Yes
No
Yes
No
Figure 4.2: Sequence of events during coalition formation.
102
formed. If a coalition cannot be formed, then it sends a ‘discard coalition’ broadcast
as shown in Fig. 4.2, otherwise, it will form a coalition and broadcast the coalition
information along with the information about the time at which they are expected to
rendezvous at the target. The selected coalition members will replan their paths using
simultaneous strike mechanism that we develop in Section 4.3) to prosecute the target.
The rejected agents will discard their potential interest in the coalition and continue to
perform their search task. During this process, the potential members who announced
their availability for coalition will be in a wait state to receive information from the
coalition leader. The complete sequence of actions that happens during the coalition
formation for a target by the coalition leader as well as by other UAVs are shown in
Fig. 4.2.
The coalition leader has to solve the optimization problem described by Eqs. (4.5)
and (4.6) to determine the coalition members. Since solving the optimization prob-
lem is computationally intensive, we develop two coalition formation algorithms: (i)
polynomial time coalition formation algorithm (which is sub-optimal) and (ii) opti-
mal coalition formation algorithm. Both the algorithms use a two-stage mechanism to
produce solutions that have low computational complexity.
4.2.1 Polynomial time coalition formation algorithm (PTCFA)
Determining the smallest coalition that would successfully prosecute the target in
minimum time can be accomplished in two stages. In the first stage, we determine the
set of UAVs that can achieve the minimum time requirement and then prune this set
to achieve the minimum member coalition in the second stage. The process to achieve
this task is given in Algorithms 4.1 and 4.2. We assume that UAV Ui is the coalition
leader and it has detected target Tj that requires RTj resources.
Algorithm 4.1 begins by initializing the coalition set to empty and the coalition
resources to zero (line 1). First, the responses of the potential coalition members are
sorted in the ascending order of their ETA to target (line 2). The algorithm takes one
UAV (Uk) at a time (line 4) from the ordered list, include Uk in the coalition C (Ui,Tj)
(line 5), update the coalition resource vector RC (Ui,Tj) (line 6) and the coalition time
τTjc (line 7). Then it checks whether the target resource constraint is met (line 8). If the
constraint is not met, then the process of including the next agent and its resources and
verifying the resource constraint continues until the target resource constraint is met
or the UAVs in the sorted potential coalition member set is exhausted in which case
a feasible coalition is not formed (line 17). Once the resource constraint is met, the
algorithm returns the feasible coalition C (Ui,Tj) and the coalition time τTjc (line 15).
103
Algorithm 4.1: First stage of the PTCFA to form a minimum time coalition
Input: Potential coalition members PC (Ui,Tj) and their ETAs D(Ui,Tj)
Output: Coalition C (Ui,Tj) and coalition time τTjc
1: C (Ui,Tj) = /0 and RC (Ui,Tj) = 0
2: [Dsorted, PCsorted] = sortAscending(D(Ui,Tj),PC (Ui,Tj))
3: for l = 1 to |PC (Ui,Tj)| do
4: Uk←PCsorted(l)
5: C (Ui,Tj)← append Uk
6: RC (Ui,Tj)← RC (Ui,Tj) +RUk
7: τTjc ← Dsorted(l)
8: if RC (Ui,Tj)p ≥ R
Tjp , for all p then
9: break
10: else
11: continue
12: end if
13: end for
14: if RC (Ui,Tj)p ≥ R
Tjp , for all p then
15: return C (Ui,Tj) and τTjc
16: else
17: return No feasible coalitions
18: end if
104
Algorithm 4.2: Second stage of the PTCFA to prune members to obtain a reducedmember coalition
Input: Minimum time coalition C (Ui,Tj) from Algorithm 4.1
Output: Pruned coalition C (Ui,Tj)
1: for l = 1 : |C (Ui,Tj)| do
2: Uk← C (Ui,Tj)(l)
3: RC (Ui,Tj)← RC (Ui,Tj)−RUk
4: if RC (Ui,Tj) ≥ R
Tjp , for all p then
5: C (Ui,Tj)← C (Ui,Tj)\Uk
6: RC (Ui,Tj)← RC (Ui,Tj)
7: end if
8: end for
9: return C (Ui,Tj)
To illustrate Algorithm 4.1, we give a hypothetical example where UAV U1 detects
a target T1 that requires only two types of resources. The target resource requirement
is RT1 = (5,3), with RT11 = 5 and RT2
2 = 3. Suppose that the UAVs that responded
to the coalition leader with their resources and cost are given in Table 4.1. As per
Algorithm 4.1, the possible coalition members are sorted in ascending order of their
cost. The new list is given in Table 4.2.
The ordered set of coalition members is PC sorted = {U2,U3,U6,U1,U4,U5}. The
coalition is formed by recruiting members from this ordered set starting with the first
member. The progress of the coalition formation process is as follows. Initially U2,
the first member in the ordered set is selected into the coalition making the coalition
C (U1,T1) = {U2} with RC (U1,T1) = (1,3). Since RC (U1,T1) is not sufficient to prosecute
the target, the next UAV, U3, is added to C (U1,T1) forming C (U1,T1) = {U2,U3} with
combined resources RC (U1,T1) = (2,4). As RC (U1,T1) is still not sufficient, the next UAV
U6 is added to C (U1,T1) which becomes {U2,U3,U6} and the resource vector of coali-
tion becomes RC (U1,T1) = (2,6). The coalition resources are still not sufficient, hence
105
Table 4.1: List of the agents which responded to the coalition formation request in thegiven example, their available resources and costs to the target
UAV Resources cost in secondsU1 (2, 1) 123U2 (1, 3) 47U3 (1, 1) 63U4 (2, 0) 172U5 (3, 2) 207U6 (0, 2) 96
Table 4.2: List of the agents in the given example, after sorting
UAV Resources cost in secondsU2 (1, 3) 47U3 (1, 1) 63U6 (0, 2) 96U1 (2, 1) 123U4 (2, 0) 172U5 (3, 2) 207
the next UAV in the list, U1, is included in C (U1,T1) resulting in RC (U1,T1) = (4,7).
The resource RC (U1,T1) is still short of the requirement and U4 is recruited to the coali-
tion resulting in C (U1,T1) = {U2,U3,U6,U1,U4} and RC (U1,T1) = (6,7), which satisfies
the target resource requirement. The Algorithm 4.1 terminates once it determines the
coalition that satisfies the target resource constraint. The prosecution time for simul-
taneous rendezvous is determined by the coalition member whose cost (ETA) is the
highest. In the present case, the ETA of U4 is the highest with 172 s. Algorithm 4.1
will ensure a coalition with minimum time to target. This is proved in the following
theorem.
Theorem 4.2 Algorithm 4.1 generates an optimal minimal time coalition set.
Proof: Let N be the number of agents that responded to a coalition formation request
by Ui to prosecute target Tj that requires resources RTj . Let the set of potential coalition
members be given by PC (Ui,Tj) = {Ur1 , . . . ,UrN}, where r1, . . . ,rN ∈{1, . . . ,N}, with
corresponding costs (ETA to target) as D(Ui,Tj) ={
DTjUr1
, . . . ,DTjUrN
}. Assume that
∑U∈PC (Ui,Tj) RU ≥ RTj . Algorithm 4.1 orders the set PC (Ui,Tj) to form PCsorted =
{Us1 , . . . ,UsN} where s1, . . . ,sN is a permutation of r1, . . . ,rN such that the correspond-
106
ing ordered cost Dsorted ={
DTjUs1
, . . . ,DTjUsN
}has the property that D
TjUsk≤ D
TjUsk+1
,∀k ∈{1, . . . ,N− 1}. For a given l ∈ {1, . . . ,N}, let PC l
sorted = {Us1 , . . . ,Usl} be the first l
elements of the ordered set PCsorted . The target prosecution time for PC lsorted is D
TjUsl
.
The algorithm then finds the least l such that ∑U∈PC lsorted
RU ≥ RTj . Let lmin be the least
integer in {1, . . . ,N} with the above property. Then PC lminsorted is the coalition formed
by Algorithm 4.1 and the claim is that this is a coalition that prosecutes the target Tj
in minimum possible time. Clearly, PC lminsorted is a feasible coalition as the combined
resources of the coalition are enough to prosecute the target. Since the elements of
Dsorted are ordered in ascending order, the addition of an agent Usk with k > lmin will
not decrease the coalition time. Similarly, removal of any agent from PC lminsorted other
than Uslminwill not decrease the coalition time which is D
TjUslmin
. Whereas, removal of
Uslminwill result in an infeasible coalition as lmin is the least integer such that PC lmin
sorted
satisfies the resource requirement. Thus PC lminsorted is a feasible coalition with minimum
time to prosecute target Tj.
Once the minimum time coalition is formed by Algorithm 4.1, we need to prune those
members whose resources are not required to form a coalition with least size. However,
finding a minimum member coalition is NP-hard as shown below.
Theorem 4.3 The problem of finding minimum member coalition is NP-hard.
Proof: We prove this by reducing the set covering problem to an instance of the
minimum member coalition problem. The set covering problem can be defined as
follows.
Find a sub-collection of least cardinality that covers the given collection
of finite sets.
Set covering problem is NP-hard (Cormen et al., 2001). Let S1,S2, . . . ,SN be the given
sets. Let S =⋃N
i=1 Si. The set coveing problem requires us to find a Γ ⊂ {1,2, . . . ,N}of minimum cardinality such that
⋃γ∈Γ Sγ = S.
Let the cardinality of S be m. Without loss of generality, the elements of the set S
can be represented as {1,2, . . . ,m}. Let each element correspond to a type of resource.
Let each set Si, i ∈ {1,2, . . . ,N} correspond to a UAV Ui whose resource capability is
defined as follows. A UAV Ui has one p-type resource, for p = 1,2, . . . ,m, if and only
if p∈ Si. Thus, if RUi = (RUi1 ,RUi
2 , . . . ,RUim ) is the resource capability of Ui, then RUi
p = 1
if p ∈ Si, and 0 otherwise for p = 1,2, . . . ,m. Define a target that requires one resource
of each type, that is, the target resource vector is (1,1, . . . ,1).
107
With this correspondence, the minimum member coalition that can prosecute the
target corresponds to the minimum set cover of S. Since one each of every type of
resource is required, the resultant coalition that can prosecute the target will not only
be of minimum size but also covers the set S.
Since finding a minimum member coalition is NP-hard, it may not be possible to de-
termine this coalition without exhaustively searching all possible coalitions. However,
this will be computationally expensive, especially for the dense traffic situations that
we consider. Therefore, we give a sub-optimal, but polynomial time, algorithm to
obtain a coalition which has lesser number of members than the coalition formed by
Algorithm 4.1. This is given in Algorithm 4.2. The algorithm checks for each UAV Uk
in C (Ui,Tj), the coalition formed by Algorithm 4.1, whether the resources of Uk are
required for the coalition or not by removing its resources from RC (Ui,Tj) (line 3). If
the resources of Uk are not required, then the UAV Uk is removed from C (Ui,Tj) (line
5) and its resources are deducted from the RC (Ui,Tj) (line 6). This process is carried
out for all the UAVs Uk ∈ C (Ui,Tj). The Algorithms 4.1 and 4.2 together forms the
two-stage Polynomial Time Coalition Formation Algorithm (PTCFA).
We continue the example given above to illustrate the second stage of the coali-
tion formation given by Algorithm 4.2. The input to Algorithm 4.2 is the mini-
mum time coalition found by Algorithm 4.1. For the example we consider, this is
C (Ui,Tj) = {U2,U3,U6, U1,U4} with combined resources RC (Ui,Tj) = (6,7). The first
UAV in C (Ui,Tj) is U2. Therefore, its resources are removed from RC (Ui,Tj) to get
RC (Ui,Tj) = (5,4). As the target requirement is RT1 = (5,3), the condition R
C (Ui,Tj) ≥RT1 is met, and hence the UAV U2 is removed for the coalition and its resources
from the coalition resource vector. This results in C (Ui,Tj) = {U3,U6,U1,U4} and
RC (Ui,Tj) = (5,4). The next UAV in C (Ui,Tj) is U3, and with its resources removed
from RC (Ui,Tj), we get RC (Ui,Tj) = (4,3). Since, removing UAV U3 from C (Ui,Tj) re-
sulted in not meeting the target resource constraint, U3 and its resources are restored
in the coalition. Removal of any of the remaining UAVs in C (Ui,Tj) will cause a vio-
lation of the target resource requirement. Hence, we get the coalition after the second
stage as C (Ui,Tj) = {U3,U6,U1,U4} with resources RC (Ui,Tj) = (5,4). Determining a
feasible coalition set with a small size as above involves less computational complexity
but can produce sub-optimal solutions as all possible sub-coalitions and their resource
contributions are not considered. Therefore, below we give another algorithm that will
find an optimal minimum member coalition.
108
Algorithm 4.3: Optimal coalition formation algorithm
Stage 1: Use Algorithm 4.1 to determine C (Ui,Tj)
Stage 2: Solve
argminC ′(Ui,Tj)⊆C (Ui,Tj)
|C ′(Ui,Tj)| (4.9)
subject to
(∑
U∈C ′(Ui,Tj)RU
p
)≥ R
Tjp , for all p = 1, . . . ,m (4.10)
4.2.2 Optimal coalition formation algorithm (OCFA)
To determine the coalition which has an optimal size also, we begin with the minimum
time coalition generated by Algorithm 4.1. Next, we prune the UAVs from this set to
obtain the optimal member coalition. To achieve this, in the second stage, we formulate
an integer programming problem to determine the least number of agents that have
sufficient resources to prosecute the target. The formulation of the problem is given in
Algorithm 4.3.
Algorithm 4.1 provides optimal minimum time and the Stage 2 of Algorithm 4.3
generates an optimal minimum size coalition. Hence the two-stage algorithm (Al-
gorithm 4.3) generates an optimal minimum time and minimum size coalition. We
continue the example given above to describe the functioning of the second stage of
the optimal coalition formation algorithm. The optimal coalition obtained after solving
Eqs. (4.9) and (4.10) as an integer programming problem is C (Ui,Tj) = {U2,U1,U4}with RC (Ui,Tj) = (5,4).
From the coalitions obtained using PTCFA and OCFA, we observe that the coali-
tion formed using OCFA has the smaller size with 3 members and it is the optimal
solution for a single target. The coalition formed using PTCFA is sub-optimal as it
may not always find a minimum member coalition although its solution is always time
optimal.
4.2.3 Complexity analysis
Now we analyze the complexity of the PTCFA and OCFA algorithms. The PTCFA
produces a sub-optimal solution but it has polynomial time complexity. Hence, it
109
can be used for real-time applications. Below, we give a result on the complexity of
PTCFA.
Theorem 4.4 The computational complexity of the PTCFA is O(N(logN +m)).
Proof: Assume that N UAVs respond to the coalition request and the target Tj re-
quires m types of resources to prosecute. The first stage (using Algorithm 4.1), which
includes the sorting of proposals requires O(N logN) computations. The steps 7 and
9 (addition and comparison of m dimensional vector) takes Nm computations each.
Hence the computational complexity of the first stage is O(N logN)+O(Nm). The sec-
ond stage (using Algorithm 4.2) requires Nm computations each for steps 4 and 5 (sub-
straction and comparison of m dimensional vector), that is, a computational complexity
of O(Nm). Therefore, the algorithm complexity of PTCFA is O(N logN)+O(Nm).
The computational complexity for the optimal coalition formation algorithm (OCFA)
has a polynomial time complexity for the first stage as the first stage of OCFA is same
as that of PTCFA. But we use an integer programming technique to solve the optimiza-
tion problem in the second stage of OCFA. Although solving an integer programming
problem is NP-hard, there are pseudo-polynomial algorithms to generate optimal so-
lutions (Papadimitriou & Steiglitz, 1982). To solve the integer programming problem
in the second stage, we used the bintprog command in MATLAB that in turn uses
a branch and bound technique to compute the solution.
When N is very large, the computational time for optimal size coalition depends
on two factors: the number of UAVs and the quantity of their resources. During the
initial phase of the mission, the UAVs have resources in good quantity that the coalition
leader may receive a higher number of proposals. However, since every UAV is full
of resources, the coalition selected after stage 1 will have lower number of UAVs. As
the minimum member coalition is to be found from a subset of this set which is small,
the computational time will be small even if OFCA is used. The case is reversed
during the final stages where the UAVs may have fewer resources and the coalition
size is necessarily larger to meet the target resource requirements. As the coalition size
becomes large the computational time increases. The above analysis indicates that the
computational time for the optimal member coalition formation mainly depends on the
distribution of the UAV resources.
110
4.3 Simultaneous strike
The coalition leader Ui determines the latest arrival time τTjc for the coalition and
provides this information to the accepted coalition members. The coalition members
need to adjust their paths such that their time to arrive at the target is equal to τTjc . There
are several methods that can be used to achieve simultaneous strike by solving the
rendezvous problem (Lin et al., 2003; Tiwari et al., 2004; Lin et al., 2005; Notarstefano
& Bullo, 2006). We develop a different approach where there is no communication
between the coalition members to achieve simultaneous strike. The coalition members
need only the information of τTjc from the coalition leader. The UAVs then find the
radius R, bounded below by the minimum radius of turn, of the Dubins curve to be
followed by the UAV such that the ETA of the UAV is equal to τTjc .
Assume that Ui is the coalition leader for target Tj forming a coalition C (Ui,Tj)
with latest arrival time of τTjc . Each member Uk ∈C (Ui,Tj) has to adjust its path length
by finding a turning radius Rk such that DTjUk
= τTjc . Since, Rk cannot be calculated
using a closed form solution, we calculate Rk iteratively until the condition DTjUk
= τTjc
is satisfied.
The selection of the coalition members in the first stage of PTCFA and OCFA is
based on the ETA of the UAV from the target. The choice of ETA and its computation
are explained below.
Given the UAV position with its heading and the target location, we consider two
possible paths to the target: (i) Dubins shortest path and (ii) roundabout path, as shown
in Fig. 4.3(a). Both paths use a minimum turn radius. Although Dubins path is the
shortest distance to target, we choose the roundabout path length as ETA to target for
the following reason. Consider a situation as shown in Fig. 4.3(b) where UAVs U1
and U2 are part of the coalition to prosecute T1 and need to rendezvous at T1. If both
UAVs take Dubins path to target, since DT1U2
> DT1U1
(refer Fig. 4.3(b)), the UAV U1
increases its turn radius to increase its path length to target. However, if DT1U2
is much
larger that DT1U1
, then it can so happen that the turn radius chosen by U1 is big enough
to encircle the target as shown by the dotted circle in Fig. 4.3(b). In that case, the UAV
U1 may have to choose a Dubins path, similar to that in Fig. 2.5(b), which may have a
path length larger than DT1U2
. Thus there can be a discontinuity in the achievable ETA
when one uses the shortest length Dubins path to target. To circumvent this problem,
we propose that the UAVs use the roundabout path to target and its length as ETA to
target. This path length can be increased continuously, by increasing the turn radius,
111
T1
U1
DT1U1
DT1U1
__
(a) Dubins shortest path and the round-about path to a target
T1
U1
DT1U1
DT1U1
__
U2
DT1U2
DT1U2
__
(b) Justification for the choice of roundabout path length asETA
Figure 4.3: Dubins path and the roundabout path to a target.
to match the coalition time τTjc . In this case, any value of τTj
c ≥ DU1Tj
is guaranteed to
be achieved, as opposed to the case where Dubins path is used.
To calculate the ETA of a UAV using the roundabout path from its current position
to a target, we need to consider two cases as shown in Fig. 4.4. Case (a): The target is
on the right hand side of the UAV and the UAV has to turn counter-clockwise to follow
the roundabout path to the target. Case (b): The target is on the left hand side of the
UAV and the UAV has to make a clockwise turn to reach the target using a roundabout
path.
Let (xi,yi) be the location of the UAV and ψi be its heading. Let (xT ,yT ) be the
target location. Let (xc,yc) be the center of the circle tangent to the UAV’s velocity
and drawn away from the target (in the direction of turn). For case (a), this is given as
xc = xi−Rsinψi
yc = yi +Rcosψi (4.11)
and for case (b), this is
xc = xi +Rsinψi
yc = yi−Rcosψi (4.12)
where R is the radius of the circle along which the UAV makes a turn.
112
Ui
(xi,yi)
(xc,yc)
q1q3
q2
j
(xT,yT)
i
(a)
Ui
(xi,yi)
(xc,yc)
q1
-q3q2
j
(xT,yT)i
(b)
Figure 4.4: Calculation of length of roundabout path to target.
In both the cases, we find angles θ1,θ2 and θ3, shown in Fig. 4.4, as follows.
θ1 = arccos
(√(xT − xc)2 +(yT − yc)2
R
)
θ2 =π2−ψi (4.13)
θ3 = arctan
(yT − yc
xT − xc
)
where, θ3 ∈ (−π,π]. For case (a), the ETA is found as
DTUi
= (R{2π− [θ1− (π− (θ2 +θ3))]}+R tanθ1)/Vi (4.14)
and for case (b), it is
DTUi
= (R{2π− [θ1− (θ2 +θ3)]}+R tanθ1)/Vi (4.15)
where Vi is the ground speed of the UAV.
Note that the roundabout path for each UAV in a coalition, for the simultaneous
prosecution of target, can be found by adjusting the corresponding radius. Given the
coalition time τTc for a target T , a UAV Ui adjusts its path length to target, DT
Ui, to
match the coalition time by choosing an appropriate radius of turn. Since the radius
of turn, R, cannot be explicitly solved from Eqs. (4.14) or (4.15), we find it iteratively.
We start with two guess values of radii of turns – one giving a path length greater than
τTc , and other giving a path length less than τT
c . Then, we use a bisection method to
iteratively find a radius of turn which gives the length of the roundabout path to target
close to the coalition time τTc within the desired accuracy.
113
It is to be noted that although the path length of the roundabout path is greater
than that of Dubins shortest path, the additional time required is bounded above by
2πRmin/V where V is UAV’s velocity and Rmin is the minimum radius of turn.
Once the coalition is formed, the UAVs travel along the computed roundabout path
to target. During this process, the coalition members may detect other targets. In that
case, the detecting member will become a coalition leader for that target and carry out
the coalition generation process. However, this UAV will not participate as a member
of the coalition, since it is already a part of another coalition. From the computational
analysis described in the previous section, the two-stage algorithms have low compu-
tational complexity and require less communication – only three broadcast – to make
a coalition. Hence, the UAVs in a coalition can afford to be the coalition leaders for
the detected targets.
The algorithms developed in this chapter are used to form coalitions and prosecute
targets as and when they are detected in a multiple target scenario where the UAVs
have to search for the targets. Algorithm 4.3 provides an optimal solution to prosecute
a single target. However, it is necessary to know the deviation of the proposed solu-
tion in a multiple target scenario from that of the global optimal solution where the
information about all targets are known a priori. For this purpose, we formulate the
problem as a global static optimization problem and solve it. In the next section, we
will discuss the solution of this combinatorial optimization problem.
4.4 Combinatorial optimization problem
The performance of the mission using the above algorithms – PTCFA and OCFA
– is sub-optimal for an entire search and prosecution mission. In order to know the
deviation of the mission performance of the PTCFA and OCFA algorithms from that of
the global solution obtained using a priori information of the target locations, we need
to solve the global optimization problem of minimizing the sum of prosecution times
for all the targets (refer Eq.( 4.4)). The solution to this combinatorial optimization
problem is NP-hard, in part because it is difficult to capture the dynamics of depletion
the UAV resources after each target prosecution. We will solve the global optimization
problem using the Particle Swarm Optimization (PSO) technique. The PSO allows us
to model the optimization problem taking the resources dynamics into account. Also,
PSO is faster compared to genetic algorithms (Parsopoulos & Vrahatis, 2002). Due
to these features we adopt PSO to solve the optimization problem of minimizing total
114
time to prosecute all the targets, given target locations and resource requirements, and
initial positions and resources of UAVs.
4.4.1 Overview of particle swarm optimization
PSO is a population based stochastic optimization technique developed by Eberhart
and Kennedy (1995), Kennedy and Eberhart (1995). The key idea behind the algorithm
is to simulate the social behavior of bird flocks, fish schools, etc. Each particle in a
swarm is a potential solution in the search space. The particle adjusts its velocity ac-
cording to its own flying experiences and its flock’s experiences. The PSO technique is
similar to the evolutionary computation techniques in (Parsopoulos & Vrahatis, 2002),
however, in PSO each particle can adapt its velocity to move in the search space and
has memory of its best position.
If we assume the optimization problem to be of Q-dimension, then each particle
in the swarm S, at the kth time step, can be represented as Xkl = (xk
l1,xk
l2, . . . ,xk
lQ), l =
1, . . . ,S. The best previous position attained by the particle is represented as Pkl =
(pkl1, pk
l2, . . . , pk
lQ), the velocity of the particle is V k
l = (vkl1,vk
l2, . . . ,vk
lQ), and the best
global position achieved by the swarm is represented as Pkg = (pk
g1, pkg2, . . . , pk
gQ), where
we denote the time step (iteration number) as the superscript k. The particles in the
swarm are updated according to the following equations
vk+1ld
= χ(wvkld + c1r1(pk
ld − xkld )+ c2r2(pk
gd− xkld )), (4.16)
xk+1ld
= xkld + vk+1
ld(4.17)
for d = 1, . . . ,Q, where, r1 and r2 are uniformly distributed random numbers in [0, 1],
c1 and c2 are positive constants representing cognitive and social parameters, w is
the inertia weight, and χ is the constriction factor. The role of inertia weight w is
to create a balance between global and local explorations. Initially, it is necessary to
explore the search space and then reduce w as the solution reaches the optimum value
(Shi & Eberhart, 1998). The constants c1 and c2 aid in convergence of the solution.
The random parameters r1 and r2 are used to maintain the diversity of the swarm
population, while the constriction factor controls the effect of velocity on the particles.
The update equations given in (4.16) and (4.17) are modified from the original
equations developed by Eberhart and Kennedy (1995), which did not have the inertia
and constriction factors. Next, we will determine coalitions for all the targets and
the sequence in which the targets are prosecuted by solving the global optimization
problem using PSO.
115
4.4.2 Solution to the optimal coalition formation problem using PSO
The value of the particle in each dimension can, in general, be a decimal value, but
for the target assignment problem we need it to be an integer. The usual method of
using PSO for integer programming optimization methods is to round the value of the
particle (Laskari et al., 2002; Nemhauser & Wolsey, 1999). we will also use the same
scheme to solve our coalition formation integer programming problem using PSO.
We slightly modify the notation of the particle Xkl to suit our problem. We assume
that N UAVs and M targets are present in the search space. The lth particle at the kth
iteration is
Xkl = (xk
l1,1,xk
l1,2. . . ,xk
l1,M,xk
l2,1,xk
l2,2, . . . ,xk
l2,M, . . . ,xk
lN,1,xk
lN,2, . . . ,xk
lN,M) (4.18)
where, xkli, j∈ {0, . . . ,M} and indicates that UAV Ui will prosecute target xk
li, jin the jth
sequence. A value of 0 for xkli, j
represents a search. The dimension of the particle is
NM. The first M dimensions of the particle present the order in which the first agent
would prosecute different targets. The dimensions M + 1 to 2M present the order in
which the second agent will prosecute targets, and so on.
The PSO algorithm for optimal coalition formation is discussed below. The main
algorithm is given in Algorithm 4.4. The input to the algorithm are the particle swarm,
swarm parameters, termination conditions including the maximum number of itera-
tions (nIterations), and the UAV and target positions and resources. The algorithm will
output the state of best particle in the swarm – particle with least cost. For each iter-
ation, all the particles are evaluated using evaluateSwarm function (line 3) described
in Algorithms 4.4 to 4.4. Before evaluating a particle in this function, it is checked
whether the state of the particle is a feasible one or not (line 19). If not, then the
value of the particle is set to infinity (line 48). A valid particle is evaluated for the
cost of the coalition formation solution suggested by its state (lines 22 to 46). If the
coalition solution does not prosecute all the targets, then the particle is assigned a high
cost (line 45). For each sequence of prosecution, the targets to be prosecuted during
that sequence is identified (line 24). Then, for each of these targets, Tj, we find all
the UAVs (uavsPreferringTarget) that are willing to prosecute Tj during that sequence
(line 28). If a particular target is already assigned in a previous sequence, then the
cost of that particle is set to infinity (line 39). If the combined resources of UAVs in
uavsPreferringTarget is greater than or equal to the target resource requirement, then
a coalition is formed and the coalition cost is added to the cost of the particle (line
31). Else, a coalition is not feasible and the particle is assigned a very high cost (line
36). If the coalition is feasible, then the target Tj is added to the targetsAssigned list,
116
Algorithm 4.4: Global solution using PSO
Input: swarm S, swarm parameters, termination conditions, UAV and target positionsand resources
Output: particle in S with least cost
1: bestParticleValue = ∞, bestParticle = [ ]
2: for kIteration = 1 to nIterations do
3: swarmParticleValues← evaluateSwarm(S)
4: [minParticleValue, minParticleIndex]← min(swarmParticleValues)
5: if minParticleValue ≤ bestParticleValue then
6: bestParticleValue← minParticleValue;
7: bestParticle← S(minParticleIndex);
8: end if
9: if termination conditions are met then
10: return bestParticle
11: break
12: else
13: update S using Eqs. (4.16) and (4.17)
14: roundoff xkli, j
to nearest integers for every particle in S
15: end if
16: end for
117
Algorithm 4.4: (continued) Global solution using PSO
17: Function cost = evaluateSwarm(S)
18: for lParticle = 1 to S do
19: validParticle← if each entry xkli, j
in the sate of the particle belongs to {0,1, . . . ,M}
20: targetsAssigned← [ ]
21: if validParticle then
22: cost(lParticle)← 0
23: for jSequence = 1 to M do
24: targetsInCurrentSequence← S(lParticle,jSequence:M:NM)
25: while targetsInCurrentSequence = /0 do
26: Tj ← targetsInCurrentSequence(1)
27: if Tj /∈ targetsAssigned then
28: uavsPreferringTarget← UAVs with entry as Tj in jSequenceth sequence
29: coalitionResources← sum of resources of uavsPreferringTarget
30: if coalitionResources ≥ RTj then
31: cost(lParticle)← cost(lParticle) + max(ETA(uavsPreferringTarget))
32: targetsAssigned← append Tj
33: reduce the resources of the UAVs in uavsPreferringTarget
34: record UAV-target assignment, coalition time, and prosecution sequence
35: else
36: cost(lParticle)← ∞
37: end if
118
Algorithm 4.4: (continued) Global solution using PSO
38: else
39: cost(lParticle)← ∞
40: end if
41: targetsInCurrentSequence = targetsInCurrentSequence \ Tj
42: end while
43: end for
44: if targetsAssigned = {1, . . . ,M} then
45: cost(lParticle)← ∞
46: end if
47: else
48: cost(lParticle)← ∞
49: end if
50: end for
resources of UAVs in the coalition are depleted, and the UAV-target assignment, coali-
tion time and the sequence of prosecution are recorded (lines 32 to 34). The coalition
time, which is added to the particle cost (line 31) is found as the maximum of ETAs
of all the UAVs in the coalition (uavsPreferringTarget). This ETA to target computed
for a UAV includes the coalition time for the targets that the UAV is assigned to in
any of the previous sequences. Once all the particles are evaluated, we determine that
particle (minParticleIndex) which has the minimum cost (minParticleValue) (line 4).
If minParticleValue is less than the previous best value then the bestParticleValue and
bestParticle variables are updated (lines 5 to 8). If any of the termination conditions
is met, then the algorithm outputs the state of the best particle (line 10). Else, the
swarm is updated using the PSO Eqs. (4.16) and (4.17) (line 13). The algorithm termi-
nates when one of the termination conditions is met or when the maximum number of
specified iterations (nIterations) is reached.
119
Figure 4.5: Initial positions of the UAVs and the targets for the example given toillustrate PSO algorithm.
The PSO algorithm produces the optimal coalition assignment for all the targets. It
minimizes the objective function in Eq. (4.4). However, the dimension of the particle
depends on the number of UAVs and targets. With an increase in the number of UAVs
and targets, the computational time increases as PSO has to search on NM dimensional
solution space.
A hypothetical scenario is considered to demonstrate the execution of the PSO
algorithm. Consider a scenario with 3 targets and 4 UAVs whose initial positions are
given in Fig. 4.5. The resources of the UAVs are RU1 = (1,2,3), RU2 = (2,0,1), RU3 =
(1,3,1), and RU4 = (1,2,1), and the target resource requirements for this example are
RT1 = (2,1,0), RT2 = (1,0,1), and RT3 = (0,1,3).
The solution, Xkl , obtained using the PSO algorithm, after 532 iterations, is (300
020100100). From the solution, according to line 24 of Algorithm 4.4, targetsInCur-
rentSequence = [3011]. This makes T3 the current target Tj (line 26). Since only one
UAV has target T3, and it satisfies the resource requirement, U1 is assigned to T3 (line
34). The distance of the UAV to target using the roundabout path is added to the cost in
line 31. The target is removed from the targetsInCurrentSequence vector which is now
[011] (line 41). The next entry (target) in the solution vector is 0, which is a search,
hence it is not taken into account. Therefore the targetsInCurrentSequence is [11].
The next target is T1, and carrying out procedure similar to before, we can see that the
120
Figure 4.6: The target assignment achieved using PSO in the example given to illus-trate PSO algorithm.
resource constraint is satisfied when U3 and U4 are assigned to T1. The coalition cost is
added to the particle cost and the target is removed from the targetsInCurrentSequence
list. Since the targetsInCurrentSequence list is empty for the current sequence, the next
sequence is considered. For the second sequence, we have targetsInCurrentSequence
= [0200]. Since, target T2 is the only target present in this sequence, it is assigned
to U2 as resource constraints are met. The particle cost is updated by adding the cost
of U2 moving to T2. The remaining sequence has only search task and thus does not
contribute to the cost. Figure 4.6 shows the assignment of the UAVs to targets. In the
figure we can see that U1 is assigned to T3, U3 and U4 are assigned to T1, and U2 is
assigned to T2. The cost given by the PSO solution is the sum of the coalition times
for all the targets. Note that the formulation of the PSO problem would allow multiple
solution vectors giving the same solution with identical cost. For example, a parti-
cle with state (300200100100) also would have given exactly the same UAV-target
assignment and cost as the one given in the example.
In the next section, we conduct extensive simulation studies to evaluate the perfor-
mances of the coalition formation algorithms which we developed in this chapter for
search and prosecute missions.
121
4.5 Simulation results
Monte-Carlo simulations are done to evaluate the performances of the single target
coalition formation algorithms (PTCFA and OCFA) as well as the global optimal coali-
tions for all the targets. The complexity of the optimization problem increases with in-
crease in number of UAVs and targets. We will analyze the performance of algorithms
in terms of the time required to complete the mission (mission completion time), time
required to form a coalition (coalition formation time), and the time required to simu-
late one complete mission with a given target and UAV distribution (simulation time)
for both the two-stage algorithms and the PSO solution.
4.5.1 Solutions obtained using PSO and two-stage algorithms: an example
We consider a sample mission scenario with 4 UAVs and 4 targets and analyze the
time taken by each algorithm to form a coalition, the time taken to accomplish the
mission, and the simulation time (that is, the time the computer takes to simulate a
given mission scenario). The region of interest is an area 1000 m × 1000 m in size,
and the initial position of the UAVs and targets are shown in Figure 4.7. Every UAV is
assumed to have a constant speed of 10 m/s and a sensor range of 300 m. A UAV that
goes out of the 1000 m×1000 m region of interest during its search takes a minimum
radius turn to come back into the region. The available resources of the UAVs are
RU1 = (2,3,4), RU2 = (2,1,3), RU3 = (3,2,4), and RU4 = (2,2,0), and the required
resources of the targets are RT1 = (1,1,2), RT2 = (3,2,4), RT3 = (2,1,2), and RT4 =
(3,4,1).
PTCFA solution
The UAVs have to search for targets and when detected need to form coalitions. As
shown in Fig. 4.8 at time t = 0.6 s, U1 detects target T1 and its resources satisfy the
condition RU1 ≥ RT1 . Hence, the UAV is in state S1 and it prosecutes the target without
sending a coalition request as described in Section 4.2. The trajectory of U1 traveling
toward T1 following a roundabout path is shown in Fig. 4.8. At time t = 1 s, U3 detects
T4 and forms a coalition with UAVs U2 and U4. These UAVs adjust their path and travel
toward T4 which is shown in Fig. 4.10. After prosecuting the assigned targets, the
UAVs continue to search the region. At t = 147.8 s, U3 detects T3 and a coalition with
U4 is formed. These UAVs adjust their path length to meet the rendezvous constraint
122
Figure 4.7: Initial positions of the UAVs and the targets for the given example.
U2
U4
U1
U3
T3
T4
T1
T2
Figure 4.8: The trajectory of U1 prosecuting T1 using PTCFA.
123
U2
U1
U4
U3
T3
T1
T4
T2
Figure 4.9: The trajectory of U1 prosecuting T1 using OCFA.
T3
T4
T2
T1
U2 U3
U4
U1
Figure 4.10: Routes followed by the coalition U2,U3, and U4 formed by PTCFA toprosecute T4.
124
U2
U4
U3
U1
T3
T2
T4
T1
Figure 4.11: Routes followed by the coalition U3 and U4 formed by OCFA to prosecuteT4.
T2
T3
U3
U4 U2
U1
Figure 4.12: UAVs U3 and U4 assigned to T3 by PTCFA.
125
T3
T2
T1
T4
U4
U2
U1
U3
Figure 4.13: UAV U2 assigned to T3 by OCFA.
T2
U3U4
U1
U2
Figure 4.14: Trajectories of UAVs U1 and U2 prosecuting target T2 using PTCFA.
126
T2
U2
U1
U4 U3
Figure 4.15: Trajectories of UAVs U1,U3, and U4 prosecuting target T2 using OCFA.
T1
T3
T2
T4
U2
U3
U1
U4
Figure 4.16: Coalition formation and prosecution of all the targets in the given exampleusing PSO.
127
and their trajectory is shown in Fig. 4.12. During this time, T2 is detected by U1 at
t = 164 s and a coalition with U2 is formed. The routes taken by U1 and U2 are shown
in Fig. 4.14. The total mission was accomplished in 263 seconds. Next, we execute
the same mission using OCFA, which will demonstrate that the coalition formed using
PTCFA is sub-optimal.
OCFA solution
In the mission using OCFA, U1 detects T1 at t = 0.6 s and it assigns itself to T1 as the
resource requirement is met. The route followed by U1 to T1 is shown in Fig. 4.9. At
t = 1 s, U3 detects T4 and, unlike PTCFA, generates a coalition along with U4 only (the
coalition U3 generated using PTCFA produced a coalition with 3 UAVs). Figure 4.11
shows the trajectory of UAVs U3 and U4 traveling to T4. UAV U2 detects target T3 at
t = 1.4 s and since it satisfies the target resource constraint and is not assigned to any
target, the state of U2 is S1 and thus produces a single member coalition. The route
taken by U2 to T3 is shown in Fig. 4.13. The final target T2 is detected by U1 at time
t = 155 s and a coalition with UAVs U2,U3, and U4 is formed. The trajectories of these
UAVs to target is shown in Fig. 4.15. The mission is accomplished in 213.8 s using
OCFA.
PSO solution
When the target positions and their resources are known a priori, we can solve the
combinatorial optimization problem of coalition formation using PSO. Figure 4.16
shows the assignment and the routes that the UAVs take to accomplish the mission.
UAV U1 is initially assigned to T1 and then to T4 in the next sequence, while U4 is
assigned to T4 only. So, U4 takes a path, the length of which is equal to the sum of path
length of U1 prosecuting target T1 and the length of the roundabout path of U1 from
T1 to T4. The entire mission is accomplished in 114.3 seconds using the assignment
determined by the PSO. The optimal solution is obtained within 600 iterations.
Comparison of performance of coalition formation algorithms
Table 4.3 summarizes the mission time and simulation time for a mission using PSO
solution and the two-stage coalition formation algorithms. The simulations were car-
ried out using MATLAB on a 1.83 GHz, 1 GB RAM machine. The performance of the
mission using PTCFA is poorer than that achieved using OCFA in terms of the mission
128
Table 4.3: Comparison of the mission and simulation times obtained with the PSOsolution and the two-stage algorithms
Algorithm Mission time (s) Simulation time (s)PSO 114.3 17.3PTCFA 263.0 3.31OCFA 213.8 3.83
completion time, because the number of UAVs performing search task is reduced in
PTCFA as the coalitions are not of minimal size and hence the search to detect targets
is less efficient. Although the performance of the mission using OCFA is better, the
computational time taken is higher than that for the PTCFA algorithm.
From the table we can see that the mission can be accomplished in much lesser
time using the PSO solution than using any of the two-stage algorithms as the target
positions are known a priori in this case. However, the simulation time taken to com-
plete the mission using the PSO is much higher than any of the two-stage algorithms.
In case of PSO solution, the time taken to produce a solution increases exponentially
as we increase the number of UAVs and targets. This effect is studied in the next
sub-section.
Another advantage of using two-stage algorithms that we developed is that it is
decentralized and can take uncertainties in the environment into account, which may
include incomplete knowledge of resources and positions of the targets and the UAVs.
4.5.2 Effect of increase in number of UAVs and targets
A set of 100 simulations are carried out using PTCFA, OCFA and PSO solutions on
a 1000 m × 1000 m area. For the simulations using PTCFA and OCFA, the parame-
ters for the UAVs are a constant speed of 10 m/s, minimum turning radius of 50 m, a
sensor range of 100 m, and a simulation time of 1000 s. The simulation time of 1000 s
signifies that the mission is abandoned after 1000 s even if all the targets could not be
prosecuted within this time. The simulation parameters for the PSO coalition forma-
tion algorithm are c1 = 0.5,c2 = 0.5,χ = 1,w ∈ [0.4 0.7], and the maximum number
of iterations = 5000. For each simulation, the target positions and resources, and the
UAV positions and resources are randomly generated. For each target, the quantity
of each type of resource required for its prosecution is chosen as a random integer
between 0 and 3. We allocated the quantity of resources for the UAVs based on the
number of available targets. Thus, each type of UAV resource is a random integer be-
129
5 10 15 200
100
200
300
400
500
600
700
Number of UAVs
Ave
rage
mis
sion
com
plet
ion
time
PTCFAOCFAPSO solution
Figure 4.17: Comparison of the mean mission completion times in seconds for a mis-sion with 5 targets plotted against the number of UAVs in the mission.
tween 0 and (number of targets)/2. A UAV can carry a maximum of 3 different types
of resources. For a given number of targets, we study the mission performance using
various coalition formation algorithms for varying the number of UAVs – 5, 10, 15 and
20. We consider cases where the number of targets are 5, 10, and 15. We also study
the effect of percentage mission accomplished, time taken to form the coalition using
PTCFA and OCFA (coalition formation time), and the CPU time required to carry out
each simulation (simulation time).
The resources for the UAVs and the targets are generated randomly as described
above. The simulations are carried out in different ways for the two-stage and the
PSO solutions. For the simulations using PTCFA and OCFA, the total time required
to prosecute all the targets is recorded as the mission time. If some of the targets
are not prosecuted within the simulation time, either due to lack of UAV resources
or because they were not found during the search, then the percentage of the mission
accomplished is calculated as
% mission completed =Number of target prosecuted
Total number of targets×100.
The UAV team does not have a priori knowledge of whether the cumulative resources
of the team are sufficient to prosecute all the targets or not. If the resources are not
sufficient to prosecute all the targets, then the UAVs continue the search task till the
end of simulation time, that is 1000 s, else, the time taken to prosecute all the targets is
130
5 10 15 2050
55
60
65
70
75
80
85
90
95
100
Number of UAVs
Ave
rage
per
cent
age
mis
sion
com
plet
ed
PTCFAOCFAPSO solution
Figure 4.18: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 5 targets plotted against the number of UAVs in the mis-sion.
5 10 15 200
100
200
300
400
500
600
700
800
900
1000
Number of UAVs
Ave
rage
mis
sion
com
plet
ion
time
PTCFAOCFAPSO solution
Figure 4.19: Comparison of mean mission completion times in seconds for a missionwith 10 targets plotted against the number of UAVs in the mission.
131
5 10 15 200
10
20
30
40
50
60
70
80
90
100
Number of UAVs
Ave
rage
per
cent
age
mis
sion
com
plet
ed
PTCFAOCFAPSO solution
Figure 4.20: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 10 targets plotted against the number of UAVs in themission.
5 10 15 200
100
200
300
400
500
600
700
800
900
1000
Number of UAVs
Ave
rage
mis
sion
com
plet
ion
time
PTCFAOCFAPSO solution
Figure 4.21: Comparison of mean mission completion times in seconds for a missionwith 15 targets plotted against the number of UAVs used for the mission.
132
5 10 15 200
10
20
30
40
50
60
70
80
90
100
Number of UAVs
Ave
rage
per
cent
age
mis
sion
com
plet
ed
PTCFAOCFAPSO solution
Figure 4.22: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 15 targets plotted against the number of UAVs used forthe mission.
5 10 15 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Number of UAVs
Ave
rage
coa
litio
n tim
e in
mill
i sec
OCFA
PTCFA
Figure 4.23: Average computational time taken to form coalitions using PTCFA andOCFA.
133
5 10 15 200
500
1000
1500
2000
2500
3000
3500
4000
Number of UAVs
Ave
rage
sim
ulat
ion
time
in s
ec
5 target10 targets15 targets
Figure 4.24: Mean time taken to complete a simulation using PSO.
recorded as mission time. We also record the time to form the coalition and the CPU
time needed to perform the simulation.
For the PSO based optimal solution, each simulation was carried out by running
the PSO algorithm for a given number of iterations (5000) and the mission time ob-
tained from the solution and the CPU time taken for the solution are stored. When the
cumulative UAV resources are less than the cumulative target resources then a solution
using PSO cannot be obtained and the percentage mission accomplished is 0%, since
a feasible solution could not be found, and the mission time in that case is treated as
1000 s.
The performance curves for the Monte-Carlo simulations are shown in Figs. 4.17–
4.22. Let us first consider a mission with 5 targets and varying number of UAVs as
shown in Fig. 4.17. From the figure, we can see that the optimal solution obtained
from PSO is better than the solutions obtained using two-stage algorithms. However,
the deviation of the performance by PTCFA and OCFA algorithms from that of the
optimal is marginal. This shows that the algorithms that we developed are not far from
being optimal. The time to accomplish a mission has two components (i) the search
time to detect the target and (ii) the time to prosecute the target. If the UAVs have small
sensor range, then they will take more time to detect all the targets. Also, the target
detection time depends on the kind of search strategy used by the UAVs. Because of
134
the lower sensor range and randomized search strategy, the mission completion time is
higher for the two-stage algorithms. The mission time using these algorithms can be
reduced by increasing the sensor range and also using a better search scheme that can
detect targets early.
Figure 4.18 shows the percentage of the missions that were completed. From the
figure we observe that the percentage mission completed when we use PSO solution
is low for 5 UAVs compared to that obtained using the PTCFA and OCFA algorithms.
This is because for some of the Monte Carlo runs, the UAVs did not have sufficient
cumulative resources to prosecute all the targets. In those cases the problem does
not have an optimal solution leading to zero percent mission completion, and hence
decrease in the percentage mission completed. But, for PTCFA or OCFA algorithms,
as the targets are prosecuted when detected, the percentage mission accomplished is
higher than that of the PSO solution. When the number of UAVs are increased to 10,
15 and 20, all the 5 targets are prosecuted by the UAVs, therefore the mean percentage
mission completed is 100% for all the algorithms.
The mission performance for the case of 10 targets is shown in Fig. 4.19. For the
mission with 5 UAVs, 99% of the simulations did not have sufficient UAV resources to
prosecute all the targets. Hence, the average time for the mission is high. However, for
the simulations with higher number of UAVs, the mission time decreases. The PSO
solutions are better than the PTCFA and OCFA solutions. Although the PSO solution
is better in terms of mission completion time, the percentage mission accomplished is
lower than that achieved using PTCFA and OCFA algorithms due to the reason that we
gave earlier. Similar behaviors in the performances of algorithms can also be seen for
mission with 15 targets and is given in Figs. 4.21 and 4.22.
From these simulations we can conclude that even though the PTCFA produces
sub-optimal coalitions, its performance is comparable to that achieved using OCFA
and PSO solution. Thus, owing to its polynomial time complexity and hence scalabil-
ity, PTCFA stands as a good candidate for a real-time coalition formation algorithm to
be implemented in a large scale search and prosecution mission.
When a coalition leader receives proposals from potential coalition members, it
uses PTCFA or OCFA to determine the coalition for the task. Figure 4.23 shows the
mean computational time taken to form coalitions for various target and UAV distri-
butions using these algorithms. From the figure we can see that the time taken for
OCFA is higher than that using PTCFA for any given UAV and target distribution.
However, the time taken to solve these algorithms is less than one millisecond for both
the algorithms which is encouraging for real-time applications.
135
With increase in the number of UAVs or targets, the simulation time increases
when the PSO algorithm is used. Figure 4.24 shows the average time taken (with
standard deviation bars), to complete one simulation using PSO. For a given number
of targets, the total number of computations increases with increasing number of UAVs
and hence the time to determine the optimal solution also increases.
4.5.3 Coalition formation for UAVs with heterogenous speed
In all the previous simulations, we assumed that the UAVs have same constant speed.
However, we can consider UAVs with different speeds without changing any part of
the algorithm. This is because the algorithm uses ETA of the UAVs which implicitly
captures their velocity information. We present an example with five UAVs and three
targets. The initial locations of the UAVs and the targets are shown in Fig. 4.25. The
UAVs use the PTCFA algorithm to determine coalitions. The UAV speeds are v1 =
8 m/s, v2 = 10 m/s, v3 = 13 m/s, v4 = 15 m/s, and v5 = 18 m/s. The UAV resources
are RU1 = (1,1,1), RU2 = (2,1,1), RU3 = (2,2,1), RU4 = (2,2,1), and RU5 = (1,1,1),
and the target resources are RT1 = (1,3,1), RT2 = (3,3,1), and RT3 = (3,1,3). The
simulation environment is the same as described in Section 4.5.2.
At time t = 0.4 s, the UAV U3 detects target T1 and forms a coalition of UAVs U4
and U5. The trajectories followed by U4 and U5 to prosecute T1 is shown in Fig. 4.26.
As the mission progress, UAV U2 detects target T3 and forms a coalition consisting of
U1,U2 and U3 at time t = 23.1 s. The trajectories of these UAVs to T3 are shown in
Fig. 4.27. The UAVs U4 and U5 prosecute T1 at t = 27.4 s and subsequently they pros-
ecute T3 at time t = 98.6 s. The final target T2 is detected by U3 and forms a coalition
of U1,U3,U4, and U5 at time t = 123 s. The target is prosecuted at time t = 198.2 s and
the trajectories of the UAVs are shown in Fig. 4.28. From the trajectories, we can see
that even though the initial ETAs of the UAVs are different, the simultaneous strike
mechanism ensures that all the UAVs reach the target at the specified time which is the
maximum of ETAs of all UAVs in the coalition.
4.5.4 Discussion
The simulation results show that the combinatorial optimization solution using PSO
provides the lowest mission completion time. However, if there is an uncertainty in
the number of resources allocated to UAV or in the information about the targets, then
the solution obtained using combinatorial optimization is not valid any more. The
advantages of forming coalitions in a decentralized manner is that the UAVs need not
136
T1
T2
T3
U1
U2U3
U4
U5
Figure 4.25: Initial UAV and target locations for the given example.
T1
T2
T3
U1
U2U3
U4
U5
Figure 4.26: Trajectories of UAVs U4 and U5 to target T1.
137
T1
T2
T3
U1
U2
U3
U4U5
Figure 4.27: Trajectories of UAVs U1,U2, and U3 to T3.
T2
U1
U2U3
U4
U5
Figure 4.28: Trajectories of UAVs U1,U3,U4, and U5 to T2.
138
have any a priori information about other members in the team or the targets. The
information is required only while forming coalitions. Thus, even if the resources of
some of the UAVs are exhausted or cannot be used for some reason, the decentralized
coalition formation algorithms can handle these uncertainties and still form feasible
coalitions.
Minimum member coalition
The coalition formation algorithm, OCFA, can be modified by considering only a
single objective of minimizing the number of UAVs in a coalition. The time taken to
prosecute the target is not taken into account, in this case. This can be achieved by
removing the first stage of the two-stage algorithms and solving only the second stage
of OFCA.
Improving the performance of PTCFA
The second stage of the polynomial time algorithm attempts to determine the mini-
mum number of agents required for the coalition through a systematic pruning of the
coalition formed during stage 1 (refer Algorithm 4.2). However, this algorithm is naive
and we can incorporate different mechanisms that can provide better solutions. A pos-
sible mechanism is to sort the UAVs by the resources and then screen them, in which
case, the UAVs with low resources will be pruned from the coalition that might result
in a coalition with lesser members. However, there is no guarantee that even such a
mechanism would produce an optimal member coalition.
Other applications
The coalition formation algorithm can be used for many other applications where a
sub-team of UAVs are required to perform a task. These include cooperative tracking
of a suspect on the freeways and in urban environments, patrolling sea shores, and
cooperatively tracking and prosecuting moving targets in a battle-field. In these appli-
cations, the coalitions are formed to share sensor information and the resources may
not deplete. The proposed coalition formation algorithms can be easily modified de-
pending on the mission. For example, consider a cooperative target tracking mission
where the coalition leader detects a target and assigns a coalition of UAVs to track the
target with varying sensors on board and prosecute it (Kingston & Schumacher, 2005).
In this case, the UAV with the prosecute capabilities can have depleting resources.
139
4.6 Summary and conclusions
In this chapter, we addressed the minimum time minimum member coalition forma-
tion problem for a search and prosecute task, and showed that the problem is NP-hard.
Then we presented two decentralized two-stage coalition formation algorithms. The
first one has polynomial time complexity and the second uses binary integer program-
ming technique with low computational overhead. The algorithms determine a time
optimal set of agents that have sufficient resources to simultaneously prosecute a target.
In order to determine the deviation of the solution provided by the proposed algorithms
from an optimal solution, we developed a PSO formulation of the target prosecution
problem to solve the centralized combinatorial optimization problem. Simulation re-
sults showed that the mission performances of the two-stage algorithms are close to
the PSO solution. The mission times taken by the proposed algorithms, as well as the
time to form coalitions, are less and this holds promise for real-time applications.
This chapter assumed global communication, that is, every UAV can communicate
with every other UAV, which is seldom the case in a realistic UAV mission. Coalition
formation of UAVs with limited communication ranges adds an extra layer of com-
plexity to the problem. This is because the coalition, in this case, has to be formed
over a time varying network produced by UAVs with limited communication ranges in
motion. In the next chapter, we consider this problem and propose a communication
protocol for effective coalition formations over dynamic networks.
140
5 Coalition formation with limited communication
This chapter relaxes the assumption of global communication that was central tocoalition formation algorithm developed in Chapter 4. Also, the prosection of con-stant velocity and maneuvering targets, in addition to that of stationary targets, isconsidered. UAVs with limited communication ranges moving around in search oftargets form a time varying network. Forming coalitions over a dynamic networkis challenging. This chapter develops a mechanism to find potential coalition mem-bers over a network with time varying topology using principles from internet pro-tocol. Simulation results of specific cases are presented that show how coalitions areformed over a dynamic network with moving targets and then Monte-Carlo simu-lations are carried out to study the effects of number of UAVs and targets, and thecommunication parameters on the coalition formation process.
�n this chapter, we make the search and prosecute task considered in Chap-
ter 4 more realistic by imposing the following realistic conditions (i) UAVs
have limited communication ranges and (ii) the targets are not necessarily
stationary. The addition of the condition of limited communication range makes the
coalition formation problem complex. This is because, the limited communication
paradigm, along with the fact that UAVs are in a state of constant motion, gives rise to
a time varying communication network. It is over this changing network that the UAVs
have to communicate to form feasible coalitions to prosecute the detected targets. We
use ideas from internet protocol to solve this problem.
To form a coalition, the request from the coalition leader, seeking participation of
other UAVs in the coalition, should first reach the rest of the UAVs. The coalition
request has to propagate over the network formed by the UAVs with limited communi-
cation ranges. Some of the UAVs receive the coalition proposal, not directly from the
coalition leader but through some intermediate UAVs which propagate (re-broadcast)
the message of the coalition leader. These intermediate UAVs are referred to as relay
nodes. Each re-broadcast will cause a delay in the information propagation which may
result in different UAVs getting the same information at different times.
Since the network is dynamic, finding potential coalition members is difficult as
the network relay nodes (other UAVs) may go out of each others’ communication
ranges disrupting the communication network which is essential to form coalitions.
Apart from this, the earliest time to arrive (ETA) at the target by a UAV depends on
its position which may change considerably by the time the coalition leader makes a
141
TargetsUAVs
T
Ui
j
2
3
4
5
6
7
UU
U
U
U
U
Figure 5.1: Search and prosecute mission using UAVs with limited sensor and com-munication ranges – a sample scenario.
decision about formation of a suitable coalition (coalition formation process takes time
due to communication delays involved). Also, the ETA varies with time for a moving
target which is an important consideration if a rendezvous at target is to be achieved for
prosecution. The coalition formation algorithm should take into account these facts.
In this chapter, we develop a solution that addresses the above issues. This chap-
ter is organized as follows. After formulating the problem in Section 5.1, we propose
a communication protocol, using concepts from internet protocol, that will enable a
coalition leader to find potential coalition members, and form coalitions over a time
varying dynamic communication network in Section 5.2. We discuss, in Section 5.3,
the computation of ETA, and in Section 5.4, the prosecution procedure, for various tar-
gets – stationary, constant velocity, and maneuvering. Section 5.5 gives the simulation
results to show the efficacy of the proposed solution. Section 5.6 gives a summary and
concludes the chapter.
5.1 Problem Formulation
The search and prosecute mission using multiple UAVs with limited communication
and sensor ranges, and limited resources is described below.
5.1.1 The mission
We consider a search and prosecute mission, as in Section 4.1, on a bounded region
142
Figure 5.2: Illustration of why Rcomm should be greater than 2Rsen.
that contains M targets whose initial positions are unknown, using N UAVs with lim-
ited communication and sensor ranges, and limited resources (refer Fig. 5.1). The
objective is to prosecute all the targets in minimum time. As in Chapter 4, the UAVs
are heterogeneous because they carry different types of resources in varying quantities.
Also, the UAVs can have different but constant speeds. The UAVs carry out a search
task to locate targets using their sensors that have a limited sensor range of Rsen. When
Ui detects a target Tj, as shown in Fig. 5.1, we assume that it can ascertain the type of
resources required to prosecute the target.
Unlike the case considered in Chapter 4, each UAV has a limited communication
range (Rcomm) and can communicate with only those UAVs that are within this range.
We also assume that the communication range of a UAV is at least twice as large as
its sensor range, that is, Rcomm > 2Rsen. This assumption is introduced because if the
communication range is smaller than 2Rsen, then multiple UAVs that are not within
the communication ranges of each other can detect the same target resulting in the
formation of multiple coalitions for the same target. This is illustrated in Fig 5.2. For
sake of clarity, communication ranges are shown in the figure for only one UAV.
When a UAV Ui detects and identifies a target Tj, it initiates a coalition formation
process as described in the flowchart in Fig. 4.2 of Chapter 4. However, with the
assumption of limited communication ranges for UAVs, this process has to be carried
out over a time varying network. The issue of determining the potential coalition
members for a coalition through the dynamic network is addressed in Section 5.2.
Recall from Section 4.2 that the coalition leader forms a coalition to prosecute the
target (i) in minimum time, (ii) using minimum number of UAVs, (iii) simultaneously.
143
The objective (i) is desirable as it minimizes the fuel usage during a specific prosecu-
tion, the objective (ii) ensures that the UAVs distribute their search effort so that the
targets are detected quickly and the mission of finding and prosecuting all the targets
is accomplished quickly, and the objective (iii) is essential to inflict maximum damage
to target.
In this chapter, we consider stationary, constant velocity non-maneuvering, and
maneuvering targets. The objective of prosecuting the detected target in minimum
time depends on the correct computation of Earliest Time of Arrival (ETA) to the
target. This is dealt with in Section 5.3. In the case of maneuvering targets, it is
difficult to achieve the objective of simultaneous strike with constant velocity UAVs,
and therefore, is relaxed. The prosecution sequence that we follow for various targets
is given in Section 5.4.
5.1.2 Target and UAV kinematics
We consider the following target kinematics. For a non-maneuvering constant velocity
target Tj with a constant heading ψTj , we have
xTj = VTj cosψTj
yTj = VTj sinψTj
where,(xTj ,yTj
)is the position of Tj and VTj is its velocity.
The kinematics of a maneuvering target, considered in this chapter, is as follows.
A maneuvering target Tj moves straight at its initial speed VTj for a short random time
interval, the length of which is bounded. At the end of this time period, the target
takes a turn randomly either to the left or to the right with a specified turn radius for
a random time interval. After the maneuver, the target goes straight again for another
short period of time and the process repeats.
The UAV kinematics is same as that considered in Chapter 4 (Eqs (4.7) and (4.8)).
We assume that the velocities of all the targets are less than the velocities of any of the
UAVs. Since the mission is assumed to be over a bounded region, when a target reaches
the boundary it makes a random maneuver to enter back into the boundary. Since the
random maneuver that a target might employ at the boundary is not predictable, the
coalition leader forms a coalition only if the target can be prosecuted when it is well
within the boundary. A boundary is defined to keep the simulations tractable and not
due to any limitation of the coalition formation algorithm itself. Also,in a real world
search and prosecute mission, there will always be a region of interest, the targets
beyond which is not of value for the mission.
144
5.2 Communication protocol for coalition formation
The UAVs need to form coalitions over a time varying network and the coalition
formation process depends heavily on the communication protocol employed. The
coalition formation process involves the following communications (refer Fig. 4.2) (i)
request of the coalition leader seeking the participation of other UAVs in the coalition
to be formed to prosecute a target that it detected, (ii) response from potential coalition
members about their willingness to participate in that coalition, and (iii) message from
coalition leader about the success/failure of the coalition formation process. When
the UAVs have limited communication ranges, these communications have to be done
over a time varying network. We address this problem by trying to find a sub-network,
that is invariant for the time period of the coalition formation process, over which the
communications happen in an uninterrupted manner.
Although the coalition formation process we developed in Section 4.2 is simple,
when implemented over a dynamic network, like the one we consider in this chapter,
highly non-trivial situations arise which need to be addressed effectively. For example,
an intermediate UAV can fail to communicate a decision to a required UAV or by the
time the coalition decision is declared, a potential coalition member can be out of
communication range, thus not getting a confirmation about the success/failure status
of the coalition formation process which leads to uncertainty about its membership in
the coalition. To address such situations, we need to design behaviors that will either
enable the agent to make its decision consistently whenever such kind of events occur,
or avoid these situations altogether. We adopt the second strategy, and is described in
detail in this section.
5.2.1 Broadcast in a limited communication network
Like in Chapter 4, we use the broadcast of messages as the mechanism of commu-
nication between the UAVs. In the network formed by UAVs with limited commu-
nication ranges, a UAV will receive information from another UAV (say, a coalition
leader) through intermediate UAVs called relay nodes. For example, in Fig. 5.1, the
UAV U7 receives information about target Tj from Ui through the information path
Ui→U2→U3→U7, where the UAVs U2 and U3 act as the relay nodes re-broadcasting
the message from Ui. This method of communication between wireless nodes is re-
ferred to in the literature as ‘flooding’ (Obraczka et al., 2001; Williams & Camp,
145
2002). To mimic a real-world wireless network, we associate a delay with each mes-
sage hop – transmission of message from one UAV to another – called the hop delay
denoted by Δhop. The hop delay, as we consider it, includes the queuing delay, the mes-
sage processing time at a relay node, and the propagation delay, which are the usual
delays associated with a standard wireless network. The wireless communication stan-
dard – the IEEE 802.11 (Medium Access Control) MAC protocol – handles broadcast
packets by sending it to all the neighbors as soon as the carrier is sensed free (IEEE-SA
Standards Board, 2007). However, it does not use any collision detection to ensure the
successful delivery of the broadcasted packets. There are some schemes (Alagar et al.,
1995; Tang & Gerla, 2001; Chen & Huang, 2005) that ensure reliable MAC designs
for broadcast transmission but add overhead to the MAC layer. We assume that the
UAVs use one such protocol and the hop delay will include the associated overheads
also. The value of Δhop can be considered a constant taking the value of the estimated
maximum possible delay in the network (Lamport, 1984). For the example in Fig. 5.1,
the UAV U7 would receive the message from Ui within 3Δhop seconds as the message
from Ui has to make 3 hops to reach U7.
A message broadcast by a UAV flows through the network as it is re-broadcast by
relay nodes. To prevent the message from floating around in the network indefinitely,
thus reducing the bandwidth of the network, we use the notion of time-to-live (TTL) for
a message – a notion used in packet switching (Clark, 2003) in the Internet Protocol.
We consider the TTL of a message to be the maximum number of hops that it is
allowed to make before it is abandoned (not re-broadcast) by the nodes in the network
and specify it in terms of maximum number of hops (Hmax) a message can make in a
network. The TTL concept is implemented as follows. Each message has in its body a
hop counter, H, which is initially set to Hmax. Whenever a relay UAV re-broadcast the
message, H is reset to H−1. If H = 0, then the message is not re-broadcast by a relay
UAV and thus gets abandoned.
A UAV that acts as a relay node keeps a log of every message that it re-broadcasts.
If it receives a message that is already in the log, it will not be re-broadcast. This is to
prevent clogging the network with messages that are already broadcast. For example,
in Fig. 5.1, U2 gets a message from Ui which it broadcasts. But, it may receive the
same message from U6 later. However, this will be ignored by U2 as it has already
broadcast that message. The log of a broadcast message is kept until the TTL of the
message.
146
5.2.2 Coalition formation over a dynamic network
To determine a coalition, the coalition leader requires the following information from
the potential members: (a) earliest time of arrival (ETA) at the target, and (b) resources
available for target prosecution. However, in the present case, since the target is non-
stationary and the network has delays, there is an ambiguity about the procedure for
calculation of ETA. In order to determine ETA in a consistent manner, we draw in-
spiration from the way contracts are carried out for tasks in real life. An excellent
example of this can be the task of ‘call for papers’ for a conference. The conference
committee will broadcast the task with the submission deadline, the deadline for ac-
ceptance/rejection notification, and the final submission deadline. Thus, there exists
deadlines for every part of the process to accomplish the task. Similarly, the coalition
leader uses intermediate deadlines to receive the information and to form coalitions.
The coalition leader will broadcast the availability of the target along with the deadline
for proposal submission, the deadline to receive decisions, and the deadline to pros-
ecute the target. The coalition leader will also broadcast the exact time at which the
coalition members are required to start the execution maneuver. The time at which a
coalition member is expected to start its prosecution maneuver is called the go-ahead
time, denoted by τTj
G . A potential coalition member uses this information to estimate
its ETA to the target; ETA being the earliest time to arrive at the target since τTj
G . This
is described in detail in the next section.
We circumvent the problem of forming a coalition over a time varying network
by finding a sub-network that is invariant during the coalition formation process. This
is achieved as follows. A UAV agrees to be a relay node for a message it received
only if it is going to be in the communication range of the UAV from which it re-
ceived the message for the entire coalition formation period. Every message contains
the information on the latest time by which the coalition process should end. For a
coalition formation process that was initiated to prosecute target Tj, this is the same as
the go-ahead time τTj
G , determination of which is described below.
The go-ahead time is determined by the coalition leader as a time that is greater
than the time required for the entire coalition formation process. If Hmax is the maxi-
mum number of allowed hops for a message with a hop delay of Δhop associated with
each hop, then the coalition request, the response to this request, and the message on
coalition membership together takes at most 3HmaxΔhop seconds to propagate over the
network. Each potential coalition member is given a time window of Δw seconds be-
fore they respond to a coalition participation request from a coalition leader. This will
147
give the potential coalition members time to consolidate the requests and respond to the
appropriate coalition leader (coalition leader with highest token number as discussed
in Section 4.2). The coalition leader takes a time of Δc seconds to form a feasible coali-
tion. Thus the total coalition process is upper bounded by 3HmaxΔhop + Δw + Δc. The
coalition leader, therefore, can choose the go-ahead time, τTj
G as 3HmaxΔhop +Δw +Δc.
From the above discussion, it is clear that the time for proposal submission by
potential coalition members is 2HmaxΔhop +Δw seconds after the dispatch of the coali-
tion participation request. A potential coalition member shows interest in a coalition
only if it is in the communication range of the UAV from which it received the mes-
sage until τTj
G . This behavior ensures a sub-network that is invariant over the period
of coalition formation. A potential coalition member Uk, if has any resource required
to prosecute the target, estimates its position at time τTj
G – called its go-ahead loca-
tion, denoted by GTjUk
. It then broadcasts its ETA to the target from GTjUk
along with
the information on its resource capabilities. The coalition leader determines a feasible
coalition using PTCFA (cf. Section4.2.1), OCFA (cf. Section4.2.2), or the PSO solu-
tion (cf. Section 4.4) and the message is broadcast. In case where a simultaneous strike
is desirable, the message is broadcast along with the information on time to intercept
the target. If a coalition leader cannot form a feasible coalition for a target, it waits
for a specified time (a value of 5 s is used in simulations) before a coalition formation
request is broadcast again. The UAVs in coalition reach their respective go-ahead loca-
tions (GTjUk
) and proceed to prosecute the target. On the other hand, the rejected UAVs
continue their previous task.
The above mechanism assumes that all the UAVs have synchronized clocks. If
the time varying communication graph of the UAVs are periodically jointly connected,
which will be the case as the UAVs operate in a bounded region, then the synchroniza-
tion of internal clocks can be achieved.
5.2.3 Structure of the broadcast messages
We give the structure of various messages – coalition participation request, response
to the coalition participation request, and confirmation of coalition membership – that
are broadcast in the network. Coalition leader Ui broadcasts the request in the form
〈Ui,Tj,τTj
G ,H,Hmax〉
where, τTj
G is the go-ahead time, Tj represents the relevant information about the target,
and H is the hop counter which is set to Hmax by the coalition leader. The presence
148
of Hmax in the message is to aid the potential coalition member to initialize the hop
counter H for the proposal message that it is sending back to the coalition leader.
The information on τTj
G , apart from being useful for a potential coalition member to
calculate its ETA to target in a consistent way, enables the intermediate UAVs to decide
whether to be relay nodes and potential coalition members. The go-ahead time also
helps the relay nodes in deciding when to delete the message from its log of sent-
messages in view of saving space.
A potential coalition member, if it has at least one of the required resources to
prosecute a target, will respond to the coalition participation request by a coalition
leader. A typical response of a potential coalition member, Uk, is of the form
〈Uk,Ui,RUk ,τTj
Uk,τTj
G ,H〉
where, RUk is the resource vector of Uk, τTjUk
is its ETA to target, and H is the hop
counter for the message. Similar to the case of broadcast from the coalition leader, the
τTj
G information will help an intermediate UAV to decide whether to be a relay node.
The message send by the coalition leader Ui after a coalition is formed is of the
form
〈Ui,C (Ui,Tj),τTj
G ,H〉
Here, the coalition set C (Ui,Tj) is empty if a successful coalition cannot be formed.
The decision process and the communication protocol during a coalition forma-
tion process, within a group of UAVs with limited communication ranges executing
a search and prosecute task, is given in Fig. 5.3. Although, the coalition leader, the
relay nodes, and the potential coalition members are shown separately in the figure for
clarity, every UAV, including the coalition leader, acts as both a relay node as well as
a potential coalition member.
5.3 Determining ETA to target
A coalition leader needs the information of ETAs of the potential coalition members
to determine a feasible coalition. How a potential coalition member calculates its ETA
to the target, when the target is stationary, non-stationary and non-maneuvering, is
described below.
149
Uk w
ithin
com
m. r
ange
of
rela
y U
AV
?
Uk ha
s any
re
sour
ces
for T
j?
Cal
cula
te t
T j
and
broa
dcas
t alo
ngw
ith R
Uk
Ui
In c
oalit
ion?
Det
arm
ine
path
to ta
rget
and
pros
ecut
e it
No
coal
ition
inte
rest
Mes
sage
in
log
orH
=0?
Ul 1 w
ithin
com
m. r
ange
of
broa
dcas
ting
UA
V?
Re-
broa
dcas
t mes
sage
with
H
H-1
Log
mes
sage
No
broa
dcas
t
Bro
adca
st <
Ui,T
j,t T j
,H,H
max
>G
Rec
eive
pro
posa
lsfo
r coa
litio
ns
Coa
litio
nfe
asib
le?
Det
erm
ine
and
broa
dcas
tt
T j a
nd C
(Ui,T
j)c
Bro
adca
st'd
isca
rd c
oalit
ion'
Mes
sage
from
coa
litio
n le
ader
Mes
sage
to c
oalit
ion
lead
er
Coa
litio
n le
ader
Ui
Rel
ay n
ode
Ul 1
Rel
ay n
ode
Ul n
Pote
ntia
l coa
litio
n m
embe
r Uk
No
Yes
Yes
No
No
No
NoNo
Yes
Yes
Yes
Yes
Figure 5.3: Decision process and communication protocol for coalition formation ofUAVs with limited communication ranges.
150
5.3.1 Stationary targets
The determination of ETA to a stationary target is same as described in Section 4.3;
we compute the ‘roundabout Dubins path length’ to the target.
5.3.2 Constant velocity targets
Similar to the case of stationary targets, the length of the actual Dubins path to target
is not a consistent measure to calculate ETA for the case of moving target too. This is
illustrated in Fig. 5.4. In Fig. 5.4, UAV U1 has a lower Dubins path length to target as
it can intercept the target at point 1 while the earliest interception point for U2 is point
2. Now, if U1 and U2 form a coalition, then U1 has to adjust its trajectory to intercept
the target at point 2 so as to achieve a simultaneous strike, which, as clearly seen in
Fig. 5.4(a), is not possible if it is to follow a Dubins path. Hence we use the roundabout
path as shown in Fig. 5.4(b) to the target. As shown in the figure, U1, in this case, can
take a path with a higher turn radius and hence a higher path length to intercept the
target at point 2 – the dotted path in the figure. Since the target is moving, we employ
an iterative technique to calculate a UAV’s ETA to target using its roundabout path
length to target. This is similar to an algorithm, referred to as MADDOG (Reid et al.,
1981), found in the missile guidance literature and is explained below.
The potential coalition member Uk, from the message of the coalition leader, knows
the position (ITj
0 ) and velocity of the target at time τTj
G . Also, the UAV Uk can predict
its own location at τTj
G denoted by GTjUk
. Let ITj
B be the point where the target crosses the
boundary (refer Fig. 5.5). Note that a boundary is defined to demarcate the region of
interest; a target out of the boundary is considered not of interest for the multiple UAV
search and prosecute mission. We denote the time taken by Uk to travel from a point
P to another point P′ using the roundabout path with minimum radius as τUk (P,P′).
This can be calculated using Eq. (4.14) or (4.15). Let τTj (P,P′) denote the time taken
by target Tj to travel from P to P′. We first check whether it is possible for Uk to
intercept Tj before it crosses the boundary. If τUk(GTjUk
, ITj
B) > τTj(ITj
0 , ITj
B), then Uk
cannot intercept the target before it crosses the boundary and hence will not respond to
the coalition participation request; else, choose a point (say, midpoint) between ITj
0 and
ITj
B . Denote this point by ITjm . Now, use the bisection method given in Algorithm 5.1
to find the point at which Uk can intercept Tj at the earliest. Given this point, the path
length to that point can be found using Eq. (4.14) or (4.15) from which the ETA to
target, τTjUk
, can be calculated. Figure 5.4(b) illustrates two iterations of the bisection
method described in Algorithm 5.1.
151
12U2
U1
(a) Dubins paths to moving target
12U2
U1
(b) Roundabout paths to moving target
Figure 5.4: Dubins and roundabout paths to a moving target.
UAV
Target
ITjm
(1)ITj
(2)
B,
ITj0
(1)
ITj0
(2),
ITjB
(1)
GTjUk
1
2 ITjm
(2)
Boundary
Figure 5.5: Illustration of the iterative method to find ETA to a moving target usingroundabout path.
152
Algorithm 5.1: Algorithm to iteratively find the earliest interception point to a constantvelocity target
Input: Current position (ITj
0 ) of target Tj, the point at which it crosses the boundary
of the region of interest (ITj
B ), and the error tolerance for the iteration scheme
Output: Point of earliest interception of the target
Initialize: ITj
0
(0)← ITj
0 , ITj
B
(0)← ITj
B
ITjm
(0)←midPoint([ITj
0 ITj
B ])
for i = 1,2, . . . do
if τUk(GTjUk
, ITjm
(i−1)) < τTj(I
Tj
0
(i−1), I
Tjm
(i−1)) then
ITj
0
(i)← ITj
0
(i−1), I
Tj
B
(i)← ITjm
(i−1), I
Tjm
(i)←midPoint([ITj
0
(i−1)ITjm
(i−1)])
else
ITj
0
(i)← ITjm
(i−1), I
Tj
B
(i)← ITj
B
(i−1), I
Tjm
(i)←midPoint([ITjm
(i−1)ITj
B
(i−1)])
end if
if length([ITj
0
(i)ITj
B
(i)]) < tolerance then
return midPoint([ITj
0
(i)ITj
B
(i)])
break
end if
end for
153
5.3.3 Maneuvering targets
An accurate estimation of ETA is required for synchronization among UAVs to achieve
a simultaneous strike at the target. However, in case of maneuvering target, as achiev-
ing a simultaneous strike can be difficult, we follow a different strategy for prosecution
as explained in next section. Therefore, the role of ETA, in the case of maneuvering
targets, is only to decide the order in which potential coalition members will be re-
cruited into the coalition (stage 1 of PTCFA, cf. Algorithm 4.1), and hence is not that
critical. Hence, we compute ETA as the time a UAV takes to travel from its go-ahead
location (the location of a UAV at the end of coalition formation process) to the point
where the target was initially detected; this information is available in the coalition
participation request message sent by the coalition leader.
5.4 Prosecution of targets
In this section, we describe the prosecution procedure for stationary, constant velocity,
and maneuvering targets.
5.4.1 Stationary target
The prosecution strategy for stationary targets is the same as that in Chapter 4 – simul-
taneous strike to inflict maximum damage. A minor difference between the coalition
formation algorithm in Chapter 4 and that proposed in this chapter is that the coalition
formation algorithm, as modified in this chapter, takes into account network delays.
In Chapter 4, the coalition formation was assumed to be instantaneous and therefore
the prosecution maneuvers also start at the time of detection of the target. However,
with network delays accounted for, the prosecution maneuver starts at the go-ahead
time decided by the coalition formation leader at the time of initiating the coalition
formation process.
5.4.2 Constant velocity targets
To inflict maximum damage during prosecution, the UAVs in a coalition are required
to simultaneously arrive at the target. Let τTjUk
be the ETA to target Tj of UAV Uk.
Since the UAV velocity is assumed to be greater than the target velocity, it is clear that
the UAV Uk can intercept a target at any time greater than or equal to τTjUk
. Thus, the
coalition leader selects that ETA of that UAV in the coalition which has the highest
154
ETA to target as the coalition time τTjc . Every UAV in the coalition adjusts its path to
intercept the target at τTjc as follows.
Given τTjc , the location of the target at that time can be estimated from the target
information broadcast by the coalition leader. A UAV Uk calculates its roundabout
path length from its go-ahead location GTjUk
to the target’s estimated location at τTjc
using Eq. (4.14) or (4.15). Then, it iteratively determines the radius of turn that will
cause the UAV to intercept the target exactly at τTjc . Every UAV in the coalition doing
so will ensure the simultaneous arrival at the target and its subsequent prosecution.
5.4.3 Maneuvering targets
A simultaneous strike on a randomly maneuver target using UAVs moving at constant
speeds1 can turn out to be extremely difficult. Therefore, for the case of maneuvering
target, we relax the requirement of simultaneous strike. The UAVs in the coalition,
therefore, prosecute the target sequentially; each UAV follows its quickest path to the
target.
Since the target is maneuvering randomly, it is required to continuously track the
target. The UAV that tracks a target for other coalition members updates them on the
position of the maneuvering target periodically.
During prosecution, after each UAV in the coalition prosecutes it, the target ve-
locity reduces by a random amount and the target makes a turn either to the left or to
the right (randomly chosen) with a specified turn radius for a random duration of time.
This is done to mimic the behavior of a target after being partially hit.
To illustrate the prosection procedure for a maneuvering target, we give an example
of 2 UAVs and a target with initial conditions as shown in Fig. 5.6(a). In this example
U1 detects the target and becomes the coalition leader. We consider a few different
cases.
Case 1: U1 has sufficient resources to prosecute the target alone. In this case, U1
proceeds to prosecute the target by itself. This is shown in Fig. 5.6(b).
Case 2: U2 is the only member in the coalition formed to prosecute target. In this case,
although U1 is not part of the coalition, since the target is not within the sensor range
of the coalition member U2, the UAV U1 will track the target and periodically updates
U2 on the position of the target until the target is within the sensor range of U2. This is
depicted in Fig. 5.6(c).
1Rendezvous of multiple UAV onto a slowly maneuvering target, by each UAV varying its speedwithin allowable velocity bounds, is dealt with in Chapter 6
155
−400 0 400−300
0
400
U1
U2
T1
meters
met
ers
(a)
−400 0 400−300
0
400
U1
U2
T1
meters
met
ers
Interception point
(b)
−400 0 400−300
0
400
U1
U2
T1
meters
met
ers
Interception point
(c)
−400 0 400−300
0
400
U1
U2
T1
meters
met
ers
Interception points
(d)
Figure 5.6: Prosection strategy for maneuvering targets: various cases
Case 3: Both U1 and U2 are in the coalition formed by U1 to prosecute the target. In
this case, U1 proceed to prosecute the target. After the interception with target and
prosecuting it, U1 continues to track the partially hit target, periodically communicat-
ing with U2 to give updated location of target, until the target comes within the sensor
range of U2, after which, U1 continues its search mission and U2 proceeds to intercept
the target. This scenario is illustrated in Fig. 5.6(d).
5.5 Simulation Results
The performance of the coalition formation algorithm for UAVs with limited com-
munication ranges in a search and prosecute mission is studied via simulations. Ini-
tially, we present examples to show how the coalition formation is carried out and then
present Monte Carlo simulation results. Through Monte Carlo simulations, we study
the effects of the maximum number of allowed hops (Hmax) and the hop delay Δhop on
the performance of a search and prosecute mission.
For all the simulations presented in this section, the UAVs are assumed to have a
maximum turn rate of 20 deg/s. We consider the following behavior for a maneuvering
156
target in simulations: it moves ahead with the initial velocity for a random period
of time bounded by 5 s, and then takes a turn randomly either to left or right with a
turn radius of 100 m for a random time interval in the range of 0 to 3 s. This process
is repeated. After being hit by a UAV, the velocity of a target reduces by a random
amount and it does a turn maneuver either to left or right for a random time duration
which is less than 3 s. A UAV that is tracking a maneuvering target communicates
with the other UAVs, at every 1 s interval, to provide the information of the updated
position of the target.
5.5.1 Example scenario: stationary and constant velocity targets
We consider a scenario with four UAVs and two targets in a bounded region of 1000 m
× 1000 m as shown in Fig. 5.7. Of the two targets, one is static and the other is
moving. The target resource requirements are RT1 = (3,2,2) and RT2 = (2,1,2). The
moving target T2 has a velocity of 8 m/s with an initial heading of 0 degrees. The
resource capabilities of the UAVs are RU1 = (1,0,1), RU2 = (2,1,3), RU3 = (2,2,0),
and RU4 = (1,2,1). The UAVs have different but constant speeds given as V1 = 15 m/s,
V2 = 18 m/s, V3 = 20 m/s, and V4 = 24 m/s. Figure 5.7 shows the initial placement
of the UAVs and the targets. The dotted circles represent the sensor range of the
vehicles, and the dotted lines connecting the vehicles represents the communication
links between them. The sensor range is 150 m and the communication range is 2.5
times the sensor range, that is, 375 m. The maximum number of allowed hops for a
message is 2 and Δhop = 0.1 s, Δw = 2Δhop, and Δc = 0.3 s.
The sequence of events during the simulation are as follows. At time t = 4 s, U3
detects T1 and since it does not have enough resources, it broadcasts the proposal.
The UAV U3 sets the go-ahead time as τT1G = 5.5 s. The coalition participation request
message from U3 reaches the UAVs U2, U4 and U1 within Δhop seconds (at t = 4.1 s) as
these UAVs are within the communication range of U3. The information reaches the
neighbors after Δhop seconds (t = 4.1). The UAVs accept the request as they are going
to be within the communication range of U3 till t = 5.5 s, but wait for Δw seconds to see
if any other coalition request reaches them. Since, the network is small and there are
no more proposals, the neighboring agents reply to U1 at t = 4.3 s with their resources
and ETA to target. The coalition leader receives the replies by t = 4.4 s from U1, U2,
and U4 with their ETA as 49.82 s, 30.22 s, and 22.58 s. The coalition leader U3 includes
itself in the set of potential coalition members and determines a feasible coalition as
follows:
157
0 100 200 300 400 500 600 700 800 900 1000 11000
100
200
300
400
500
600
700
800
900
1000
meters
met
ers
T1
T2
12
3
4
Figure 5.7: Initial positions of the UAVs and the targets for the given example.
The coalition leader uses the optimal coalition formation algorithm (OCFA) in
Section 4.2.2. The algorithm, as described earlier, works in two stages. In the first stage
a minimum time coalition is formed and then a minimum size coalition is obtained.
The minimum time coalition formed, after the first stage, consists of UAVs U4, U3,
and U2 with coalition time as 30.22 s. The minimum size coalition obtained after the
second stage comprises U3, and U2. The coalition leader broadcasts the decision to the
UAVs which reaches them by t = 4.8 s. The accepted UAVs U3 and U2 adjust their
trajectories, as shown in Fig. 5.8, to prosecute the target simultaneously. Note that
although the confirmation message reaches all the UAVs in the coalition by t = 4.8 s,
they start the coalition maneuver, as per the protocol, only at the pre-determined go-
ahead time which is τT1G = 5.5 s.
As the mission progresses, at time t = 104 s, target T2 is detected by UAV U4. By
this time, the target T2 which initially set out with a heading of 0 degrees has reached
the boundary and taken an arbitrary turn back into the boundary by virtue of which it
has a different current heading. The UAV U4 sends a coalition formation request to
its immediate neighbors U1 and U2. Then, U1 re-broadcasts the message to U3 that
is within its communication range. The coalition leader U4 receives the responses
from U1 and U2 by t = 104.4 s. The coalition leader waits till t = 104.6 s for the
158
0 100 200 300 400 500 600 700 800 900 1000 11000
100
200
300
400
500
600
700
800
900
1000
meters
met
ers
T1
T2
1
2
3
4
Trajecory
Figure 5.8: Trajectories that the coalition members take to prosecute T1.
proposal from U3. However, U3 does not send a proposal due to lack of resources
which have been consumed while prosecuting target T1. Including itself also into the
set of potential coalition members, the coalition leader U4 executes Algorithm 4.3 to
form a coalition consisting of U2 and U4. These UAVs simultaneously prosecute the
target at the interception point, shown as a square in the Fig. 5.9, determined by the
coalition leader. The trajectories taken by UAVs U2 and U4 are shown in the same
figure. The complete mission is accomplished in 126.9 s.
5.5.2 Example scenario: maneuvering targets
We now give an example involving randomly maneuvering target. In this example, we
consider 4 UAVs and 2 targets. The initial positions of the UAVs and the targets are
shown in Fig. 5.10. The UAVs have initial speeds as V1 = 20, V2 = 18, V3 = 12, and
V4 = 15 meters per second, respectively. The target T1 has an initial speed of 5 m/s and
T2 has an initial speed of 10 m/s. The resource capabilities of UAVs are RU1 = (1,0,1),
RU2 =(3,0,0), RU3 =(1,2,1), and RU4 =(2,1,2). The resource requirements of targets
are as follows: RT1 = (2,3,2) and RT2 = (3,0,1).
After the start of the mission, at t = 4.6 s, the target T1 is detected by U1 and
initiates a coalition formation process over the UAV network which, at that time, is
159
0 100 200 300 400 500 600 700 800 900 1000 11000
100
200
300
400
500
600
700
800
900
1000
meters
met
ers
T2
1
2
3
4
Interception point
Trajectory
Figure 5.9: Trajectories that the coalition members take to prosecute the moving targetT2.
−500 0 500−500
0
500
U1
U2
U3
U4
T1
T2
meters
met
ers
Figure 5.10: Initial position of UAVs and targets for the given example.
160
−500 0 500−500
0
500
U1
U2
U3
U4
T1
T2
meters
met
ers
I3
I4
Figure 5.11: Prosecution of T1 by U3 and U4.
completely connected. The network parameters for this example are Hmax = 3, Δhop =
0.1 s, Δw = 0.2 s, Δc = 0.3 s, and a communication range of 500 m for all UAVs. The
coalition leader U1 sets the go-ahead time as 3HmaxΔhop +Δw +Δc = 1.4 seconds after
the target detection time, and therefore, a coalition {U3,U4} is formed at t = 6.0 s. For
coalition formation, U1 used the PTCFA described in Section 4.2.1. Since the target is
not in the sensor range of either U3 or U4, the coalition leader U1, although not in the
coalition, tracks the target and broadcast its updated position information periodically
until the target is within the sensor range of one of the UAVs in the coalition (U3 in
this case). The UAV U3 prosecutes the target at t = 23.0 s and U4 at t = 26.9 s. The
trajectories of U1, U3, U4, and T1 from the target detection time till its elimination is
shown in Fig. 5.11. The locations at which U3 and U4 intercepts T1 is marked in the
figure as I3 and I4, respectively.
At t = 27.1 s, immediately after the prosecution of T1, the target T2 is detected by
U1. This is shown in Fig. 5.12. After the prosecution of T1, the resource status of the
UAVs are as follows: RU1 = (1,0,1), RU2 = (3,0,0), RU3 = (0,0,0), RU4 = (1,0,1).
As seen in Fig. 5.12, the only UAVs that are in the communication range of U1 are
U3 and U4. However, the combined resources of all the three UAVs are not enough to
prosecute the target. Therefore, the search is continued.
Further in the mission, at t = 131.6 s, the target T2 is again detected, this time, by
161
−500 0 500−500
0
500
U1
U2
U3
U4
T2
meters
met
ers
Figure 5.12: A successful coalition for target T2 is not formed due to lack of resources.
U2 which initiates a coalition formation process. A successful coalition of {U1,U2}is formed at the end of go-ahead time of this coalition formation process, t = 133 s.
The UAVs U2 and U1 proceed to intercept the target T2 at t = 148.3 s and t = 150.5 s,
respectively. The trajectories of U1, U2, and T2, from the time of detection of T2 until
its complete prosecution is shown in Fig. 5.13.
5.5.3 Effect of varying number of UAVs and targets
We study the performance of the coalition formation algorithm with the increase in
number of UAVs used for the search and prosecute mission. The performance is mea-
sured in terms of the time required for mission completion.
The parameters used for the simulation are as follows. A bounded region of 1000 m
× 1000 m is considered for simulations. The sensor range is 150 m and the communi-
cation range is 300 m. The maximum number of allowed hops for a message is 3 and
Δhop = 0.1 s, Δw = 0.2 s, and Δc = 0.3 s. UAVs have fixed but different velocities in the
range 10 to 20 m/s randomly chosen at the start of each simulation. To start with, every
target has a fixed velocity between 0 and 10 m/s. We consider the target kinematics as
in Section 5.1.2. The prosecution of targets is as per Section 5.4. Also, if the target is
near the boundary, the UAV that detected it will track the target until it comes within
the region of interest and then a coalition request is made. If a UAV, which is part
162
−500 0 500−500
0
500
U1
U2
U3
U4
T2
meters
met
ers
Figure 5.13: Prosecution of T2 by U1 and U2.
of a coalition, detects a target on its way to prosecution of some other target, then it
broadcasts this information to nearby UAVs. Those UAVs that receive this message
moves to the region where the target was detected. This leads to a directed search that
will aid a quicker prosecution of targets which otherwise would have to be detected
again.
The simulation is done for a maximum simulation time of 1500 s. If the mission is
not complete (that is, all the targets are not prosecuted) within that time, then the mis-
sion time is deemed to be 1500 s. The targets are assumed to have 3 type of resources.
The quantity of each type is a random integer between 0 and 3. The UAVs also have
similar resource distribution. However, if the total UAV resources are not enough to
prosecute the target, then resources are added to UAVs so that the mission is feasible.
The above setting is used for all the remaining simulations in this chapter.
Figure 5.14 gives the total mission time, that is, the time required to find and
prosecute all the targets, as a function of the number of UAVs used in the mission.
Cases where the number of target are 5, 10, and 15 are considered. The results reported
are averages over 100 Monte Carlo runs with different initial positions and orientations
of UAVs and targets. As seen in the figure, for a fixed number of targets, the time to
prosecute all the targets decreases as the number of UAVs deployed increases.
163
0 5 10 15 20 25 30 35 4050
100
150
200
250
300
350
400
450
500
550
Number of UAVs
Mis
sion
Tim
e (s
)
15 Targets10 Targets 5 Targets
Figure 5.14: Variation of mission time with increase in number of UAVs for variousnumber of targets.
5.5.4 Effect of varying communication range
We study the effect of communication ranges of UAVs on the completion time of the
search and prosecute mission. Simulation settings are the same as described above
except for the communication range which is varied. The results obtained are shown
in Fig. 5.15. As expected, the mission performance increases with increase in com-
munication ranges of UAVs. This is because with more communication range it is
possible for the coalition leader to interact with more number of UAVs leading to a
higher chance of a successful coalition being formed. At very high values of commu-
nication range, a UAV will be able to communicate with all other UAVs without the
need for a relay agent. In which case, the problem becomes equivalent to that with
global communication as described in Chapter 4.
5.5.5 Effect of hop delay and max-hops
The goal of this experiment is to study the effects of communication delays and num-
ber of maximum allowed hops on the mission performance. The simulation setting is
the same as described before (Section 5.5.3).
Figure 5.16 shows the variation of average mission time (averaged over 100 runs)
taken by 10 and 20 UAVs to prosecute 10 targets with respect to Hmax. We study the
164
Figure 5.15: Variation of mission time with increase in communication range of UAVsfor a mission with 10 UAVs and 10 targets.
Figure 5.16: Variation of mission time with increase in maximum hops allowed for 10and 20 UAVs prosecuting 10 targets.
165
1 2 3 4 5
200
400
600
800
1000
1200
1400
Maximum hops allowed (Hmax
)
Mis
sion
Tim
e (s
)
Δhop
= 0.1
Δhop
= 0.5
Δhop
= 1
Figure 5.17: Effect of hop delay Δhop and maximum number of allowed hops Hmax onmission performance (different curves correspond to Δhop = 0.1, 0.5, and 1 s).
effect of increase in Hmax for a given Δhop = 0.1 s. Consider the change in mission time
for Hmax varying from 1 to 5 as shown in the Fig. 5.16. With the increase in Hmax, the
performance improves, that is, the mission time decreases. This is because, as the net-
work depth is more, the coalition leader can get more potential coalition members than
being restricted to only the immediate neighbors as in the case of a low value of Hmax,
say, Hmax = 1, and hence will be able to make feasible coalitions which otherwise
may not have been possible. However, further increasing Hmax will result in a slight
degradation of performance which occurs because the communication protocol that
we employ requires a sub-network that is invariant throughout the coalition formation
period. As it is difficult to find a wider network (higher Hmax) that stays invariant for a
specified duration of time, the mission time increases with Hmax, and even more so in
the case where Δhop is high (For the cases presented in Fig. 5.16, Δhop = 0.1 s). Thus,
the degradation in mission performance with increase in Hmax will be more prominent
for high Δhop. This effect is clearly seen in Fig. 5.17 where the combined effect of
Hmax and Δhop is captured. The figure shows the change in mission time for a search
and prosecute task with 10 UAVs and 10 targets with respect to maximum number
of allowed hops for different values of communication delay. Note the degradation
of mission performance for the cases where Δhop = 0.5 and 1 s when the number of
maximum allowed hops is more than 3.
166
Figure 5.18: Variation of mission time with increase in communication delay for 10and 20 UAVs prosecuting 10 targets.
Figure 5.18 shows the effect of communication delay on the mission completion
time for search and prosecute missions with 10 and 20 UAVs and 10 targets. As seen
from the figure, the mission time increases with increase in communication delay. This
is because an increase in communication delay will cause an increase in individual
coalition formation period. Since our communication protocol depends on finding a
network that has an invariant topology throughout the coalition formation process, it
becomes harder to find such a network. This results in the inability of a coalition
leader to find potential members to form feasible coalitions leading to higher mission
completion times.
From these studies, we conclude that the hop delay in network and the selection of
Hmax are important for the performance of the mission in terms of mission time. Also,
note that all the simulations had UAVs with different speed which shows the efficacy
of the algorithm for use with heterogeneous UAVs.
5.6 Summary and conclusions
In this chapter, we addressed the problem of search and prosecution when the UAVs
have limited communication ranges. Toward this, we proposed a communication pro-
tocol, based on concepts from internet protocol, to form coalitions over a network
with time varying topology. Apart from stationary targets, we considered the cases of
search and prosecution of constant velocity and maneuvering targets. We presented a
167
method to compute ETA to a moving target in an unambiguous way; this was important
for the coalition formation algorithm. Through simulations, we showed that the pro-
posed methods suffices to form successful coalitions to prosecute targets – stationary,
constant velocity, and maneuvering – even when UAVs have limited communication
ranges. We showed that the mission performance, in this case, is dependent on the
network parameters like communication range, hop delay, and the maximum number
of hops a message is allowed to make in the network.
Until now, we have dealt with the problems of collision avoidance and coalition
formation of multiple UAVs separately. However, multiple UAV missions require these
algorithms to exist together or co-exist with other algorithms that are required for
successful mission completion. In the next chapter, we consider a few multiple UAV
missions that require collision avoidance and/or coalition formation and demonstrate
that the algorithms that we developed in this thesis can be seamlessly integrated to
work together with other algorithms for the overall success of the mission.
168
6 Collision avoidance and coalition formation inmultiple UAV missions
This chapter considers multiple UAV missions that require the applications of colli-sion avoidance and coalition formation algorithms developed in this thesis. First, theproblem of rendezvous of multiple UAVs doing collision avoidance maneuvers onthe way to the point of rendezvous is considered. To achieve rendezvous, the UAVsuse a consensus algorithm on their respective expected times of arrival at rendezvouspoint to control the velocity within some given bounds or a ‘wandering maneuver’.The efficacy of the algorithm is shown through simulations. Also addressed is themultiple UAV search and prosecute mission with collision avoidance. This missionrequires both the collision avoidance as well as the coalition formation algorithms tobe used. Simulations show that these algorithms can be used as independent guid-ance modules with only a small interference between them.
�n this chapter, we address certain UAV missions that require the use of col-
lision avoidance and coalition formation algorithms developed in the pre-
vious chapters. First, we consider the problem of rendezvous of multiple
UAVs. Such a rendezvous problem was encountered in Chapter 4 where the UAVs
in a coalition are required to rendezvous at the target to inflict maximum damage.
However, in that case, the issue of collision avoidance was neglected. On the way
to the rendezvous point, the UAVs in the coalition may encounter conflicts in trajec-
tories with other UAVs. These conflicts need to be avoided via appropriate collision
avoidance maneuvers. To achieve a deconfliction, the UAVs can employ the collision
avoidance algorithm developed in Chapter 2 (Algorithm 2.3). In this chapter, we pro-
pose a consensus based algorithm for a group of UAVs to rendezvous in the presence
of collision avoidance maneuvers.
Also addressed in this chapter is the multiple UAV search and prosecute mission
in the presence of collision avoidance. Chapters 4 and 5 made the assumption that
the UAVs are flying at altitudes separated by a safe distance which made collision
avoidance unnecessary. In this chapter, we relax this assumption and consider coali-
tion formation under collision avoidance. Since the coalition formation and collision
avoidance algorithms developed are higher level algorithms (that is, they are outer loop
guidance algorithms rather than inner loop control algorithms), they can be plugged in
as separate guidance modules with their priorities specified appropriately. In our work,
169
we assume that collision avoidance has higher priority than coalition formation.
This chapter is organized as follows. In Section 6.1 an algorithm based on consen-
sus is proposed for rendezvous under collision avoidance. We present the simulation
results of a group of UAVs rendezvousing in the presence of collision avoidance ma-
neuvers in Section 6.2. Section 6.3 addresses the problem of coalition formation under
collision avoidance. Section 6.4 concludes the chapter with a summary.
6.1 Rendezvous via consensus without collision avoidance
In Chapter 4, the rendezvous was taken care of by assuming that the UAVs will adjust
their trajectories to reach a predetermined location in space at a predetermined time.
However, because of collision avoidance maneuvers, it can so happen that a UAV
deviated too much from the nominal trajectory during deconfliction that it cannot reach
the predetermined location at the specified time even if it travels at the maximum
speed. In such cases, the UAVs in the coalition need to communicate with each other
and arrive at a new rendezvous time. In this section, we propose a way to handle this.
We assume that the UAVs rendezvousing to the same point (that is, UAVs in the same
coalition) do not try to avoid each other. Since rendezvousing to a point will cause
a collision, it is assumed that the UAVs will make a split maneuver when they reach
within a specified radius of the rendezvous point.
6.1.1 The consensus protocol
We propose that rendezvous be achieved by UAVs through a consensus on their Esti-
mated Time of Arrival (ETA) at the rendezvous point. The estimated arrival time for
Ui is denoted as ETAi. For a UAV Ui, we have the following kinematics and control.
xi = Vi cosψi (6.1)
yi = Vi sinψi (6.2)
ψi = ai (6.3)
Vi = fi(ETANi) (6.4)
where Vi is its velocity, ai is the angular acceleration and belongs to {−Vi/Rmin, 0,
+Vi/Rmin} (that is, UAV either turn left, goes straight, or turn right with minimum
turn radius Rmin), and ETANi = {ETA j : j ∈Ni} (set of ETAs of all the neighbors of
170
i). The set of functions fi, i = 1, . . . ,N, is the consensus protocol. In particular, we
consider the average consensus protocol given as
fi(ETANi) = ETAi− 1|Ni| ∑
j∈Ni
ETA j (6.5)
Also, we impose bounds on UAV velocity, that is,
Vmin ≤Vi ≤Vmax i = 1, . . . ,N (6.6)
If the communication is global, then Ni = {1, . . . ,N}. The next subsection discusses
the determination of ETA to target.
6.1.2 Determining ETA to target
It is assumed that all UAVs turn with a minimum radius denoted by Rmin. To find the
ETA to target (rendezvous point) using Dubins path, we need to consider two cases as
shown in Fig. 6.1. We consider only those cases where the UAVs start from locations
sufficiently far away from the target. In case (a), the target is to the left of the UAV
and in case (b), the target is on the right hand side of the UAV. In both cases, we
draw a circle tangent to the velocity vector of the UAV toward the target. Due to the
assumption that the target is far enough away from the UAV, this circle will not encircle
the target. For case (a), the center of this circle is given as
xc = xi−Rmin sinψi
yc = yi +Rmin cosψi (6.7)
and for case (b), it is
xc = xi +Rmin sinψi
yc = yi−Rmin cosψi (6.8)
We find different angles (θ1,θ2, and θ4), as shown in Fig. 6.1 for case (a) as (refer
Fig. 6.1(a))
θ1 = arctan
(yT − yc
xT − xc
)
θ2 = arccos
(Rmin√
(xT − xc)2 +(yT − yc)2
)(6.9)
θ3 =π2−ψi
171
q1
q2
q3
(xT,yT)
(xi,yi)
(xc,yc)
ji
(a)
q1
q2
q3(xT,yT)
(xi,yi)
(xc,yc)
ji
(b)
Figure 6.1: Calculating the Dubins distance to a target.
where, (xT ,yT ) is the target location and for case (b) as (refer Fig. 6.1(b))
θ1 = arctan
(yT − yc
xT − xc
)
θ2 = arccos
(Rmin√
(xT − xc)2 +(yT − yc)2
)(6.10)
θ3 =π2
+ψi
Now, the ETA to target can be obtained, for case (a), as
ETAi =√
(xT − xc)2 +(yT − yc)2−R2min +Rmin (θ3 +θ1−θ2)mod2π (6.11)
and for case (b) as
ETAi =√
(xT − xc)2 +(yT − yc)2−R2min +Rmin (θ3− (θ1 +θ2))mod2π (6.12)
where, θ mod2π is defined as the addition of θ with an integral multiple of 2π such
that θ ∈ [0,2π).
6.1.3 Handling velocity bounds
A UAV Ui chooses an angular acceleration ai which takes it toward the target (this can
be found using Algorithm 2.2). At the same time, it updates its velocity using Eq. (6.4)
according to the consensus protocol in Eq. (6.5). To take into account the upper and
lower velocity bounds as in Eq. (6.6), we have the following behavioral rules.
Rule 1 If a UAV’s velocity hits the upper bound, it stays with a velocity of Vmax until Vi
takes a value less than Vmax as updated by Eq. (6.4).
Rule 2 If a UAV’s velocity hits the lower bound, it stays with Vmin until Vi takes a value
greater than Vmin as updated by Eq. (6.4) and at the same time, maneuver away
from the target in an attempt to increase its ETA.
172
−200 −100 0 100 200
−300
−200
−100
0
100
200
300
400
500
600
UAV Trajectories
0 10 20 300
10
20
30
40
time (s)E
TA
(s)
0 10 20 3012
14
16
18
U1 U2
time (s)
velo
city
(m
/s)
Figure 6.2: Rendezvous of two UAVs using velocity control.
The wandering maneuver for a UAV, required by Rule 2, is obtained by applying an
acceleration negative to that which takes a UAV to the target.
It is shown through simulations that the velocity update using a function as in
Eq. (6.5), along with the above simple rules suffices for multiple UAV rendezvous to
occur.
6.1.4 Examples
For the simulations in this chapter, all UAVs use a radius of turn of 100 m. Figure 6.2
shows a case of two UAVs where consensus is reached on their ETAs by adjusting
the velocities of the individual UAVs. The figure shows the UAV trajectories and the
variation of their velocities and ETAs with time. In this case, as seen in Fig. 6.2, none
of the UAVs hit the velocity bounds.
Figure 6.3 shows a case of rendezvous of two UAVs starting at arbitrary locations
with arbitrary orientations, where UAVs U1 hits the lower velocity bound and does a
‘wandering maneuver’ to increase its ETA while U2 hits the upper velocity bound. A
wandering maneuver, as described earlier, consists of a turn in the direction opposite
to what would have taken it to target. Figure 6.3 clearly shows the increase in ETA of
U1 with the wandering maneuver.
173
−200 0 200 400 600−600
−400
−200
0
200
400
600
800
1000
UAV Trajectories
0 20 40 600
20
40
60
80
time (s)
ET
A (
s)
0 20 40 605
10
15
20
25U1
U2
time (s)
velo
city
(m
/s)
Figure 6.3: Rendezvous of two UAVs, U1 and U2, using velocity control and a wan-dering maneuver by U1.
To demonstrate the efficacy of the proposed rendezvous algorithm, we show the
rendezvous of 10 UAVs starting at arbitrary locations with arbitrary orientations in
Fig. 6.4. It is also possible for a group of UAVs to rendezvous on to a slowly mov-
ing/maneuvering target by considering the rendezvous location as the current location
of the target. Example of such a rendezvous is shown in Fig. 6.5.
6.1.5 Discussion
Unlike the traditional consensus, where all the agents converge to a constant consensus
value, in the present case, the consensus value – the ETA to target – changes with time
as UAVs are in motion.
In the simulations till now, we used the average consensus protocol. In this case,
the rate of change of velocity is determined by Eq.(6.5). For cases where number
of UAVs that need to rendezvous are high, such a consensus protocol may result in
slow convergence. This will be more prominent in cases where most of the UAVs in
the group have their ETA values close to each other (as a result, close to the group
average), while the ETAs of a few UAVs are significantly different from the group
average. Such a situation may also arise when the UAVs reach closer to the rendezvous
point: a few UAVs ‘lagging’ behind has already reached their upper speed limit, while
174
−1000 −500 0 500 1000−1000
−800
−600
−400
−200
0
200
400
600
800
1000
0 10 20 30 40 500
20
40
60
80
0 10 20 30 40 505
10
15
20
25
UAV Trajectories
time (s)
velo
city
(m
/s)
time (s)
ET
A (
s)
Figure 6.4: Rendezvous of 10 UAVs using velocity control within bounds and wander-ing maneuvers.
−200 0 200
−500
−400
−300
−200
−100
0
100
200
300
400
UAV trajectories
meters
met
ers0 10 20 30
0
10
20
30
40
time (s)
ET
A (
s)
0 10 20 305
10
15
20
25
time (s)
spee
d (m
/s)
target trajectory
Figure 6.5: Rendezvous of 3 UAVs onto a slowly maneuvering target.
175
−500 0 500
−1500
−1000
−500
0
500
1000
1500
UAV trajectories
0 20 40 600
20
40
60
80
100
time
ET
A (
s)
0 20 40 605
10
15
20
25
time (s)
velo
city
(m
/s)
slow convergence
Figure 6.6: Rendezvous of 20 UAVs using average consensus.
most of the other UAVs have ETAs that are too close to the group average to make
necessary maneuvers to ‘wait’ for the few lagging behind. This slow convergence
behavior (note that as time progresses, the ETAs do not quite merge into a single line)
is seen in Fig. 6.6 which shows the rendezvous of 20 UAVs using the average consensus
protocol.
The convergence can be made faster by use of a different consensus protocol. For
example, consider the following function for velocity update
fi(ETANi) = ETAi− min j∈Ni(ETA j)+max j∈Ni(ETA j)2
(6.13)
In this protocol, a ‘weighted average’ of ETAs, instead of the mean of ETAs, is used. In
particular, the protocol uses the mean of the maximum and minimum of ETA values of
all the UAVs. This helps to mitigate the problem of slow convergence described above
to an extent. For the same initial conditions as in the case depicted in Fig 6.7, using
Eq.(6.13) instead of Eq.(6.5) for velocity update results in a faster convergence of ETA
as seen from Figure 6.7. Note that the convergence of ETA values of different UAVs
into a single line is better in this case when compared to the convergence in Fig. 6.6.
The lesser ‘spread’ or a better convergence implies a more accurate rendezvous.
A rendezvous is possible even when only local communication is possible. That
is, the UAVs have limited communication ranges and therefore can communicate with
176
−500 0 500
−1500
−1000
−500
0
500
1000
1500
UAV trajectories
0 20 40 600
20
40
60
80
100
timeE
TA
(s)
0 20 40 605
10
15
20
25
time (s)
velo
city
(m
/s)
Figure 6.7: Rendezvous of 20 UAVs using weighted average consensus.
only those UAVs that are within its communication range. The requirement for a ren-
dezvous to occur in that case is that the underlying communication graph, formed by
the limited range communication UAVs, should eventually be connected. We con-
sider a case in which every UAV has a communication range equal to 300 m. The
rendezvous of 20 UAVs under this communication constraint is shown in Fig. 6.8.
Figure 6.9 shows the rendezvous, for same initial conditions, under global communi-
cation. For rendezvous using local communication, we note from Fig. 6.8 that initially
sub-groups that are formed by UAVs close to each other attain individual local consen-
sus, which, as time progresses coalesce into a single global consensus. From Fig. 6.8,
we can observe that at around 10 s, at least two distinct local consensus which later
merges into a global consensus.
6.2 Rendezvous with collision avoidance
Now we show that the rendezvous algorithm that we developed can be used to achieve
a rendezvous under collision avoidance.
6.2.1 Incorporating collision avoidance
The UAVs trying to converge to a point avoid other UAVs on the way. Algorithm 2.3
177
−500 0 500 1000
−1500
−1000
−500
0
500
1000
1500
UAV trajectories
meters
0 20 40 600
20
40
60
80
100
ET
A (
s)
0 20 40 605
10
15
20
25
time (s)
velo
city
(m
/s)
Figure 6.8: Rendezvous of 20 UAVs using limited communication.
−500 0 500 1000
−1500
−1000
−500
0
500
1000
1500
UAV trajectories
meters
0 20 40 600
20
40
60
80
100
ET
A (
s)
0 20 40 605
10
15
20
25
time (s)
velo
city
(m
/s)
Figure 6.9: Rendezvous of 20 UAVs using global communication.
178
−500 0 500
−1000
−500
0
500
1000
1500
U1
U2
U3 U
4
U5
UAV Trajectories
meters
met
ers0 20 40 60 80
0
20
40
60
80
100
time (s)E
TA
(s)
0 20 40 60 805
10
15
20
25
time (s)
velo
city
(m
/s)
Figure 6.10: Rendezvous of 5 UAVs with collision avoidance.
is used to effect a deconfliction. It is assumed that the other UAVs are non-cooperating
and do not do a collision avoidance maneuver which is a more stringent case. Collision
avoidance maneuvers by a UAV will cause an increase in its ETA to target. However,
the adaptation of velocity through consensus in conjunction with the wandering ma-
neuver will ensure that the UAVs still achieve rendezvous. We do not consider collision
avoidance among the UAVs that are rendezvousing (we assume that they split off at
the end of rendezvous).
6.2.2 Simulation results
Figure 6.10 shows rendezvous of 5 UAVs where three of them avoid collisions with
other UAVs on the way to the target. Our rendezvous algorithm is seen to aid a ren-
dezvous even when the involved UAVs do collision avoidance maneuvers to avoid
trajectory conflicts with other UAVs. Figure 6.11 shows that the collision avoidance
maneuvers causes changes in ETAs of the corresponding UAVs but are promptly nul-
lified by the consensus protocol to guarantee a rendezvous.
We now turn our attention to rendezvous of a group of UAVs avoiding collisions in
a high density traffic environment. Figure 6.12 gives a picture of the congested airspace
where UAVs U1, U2, U3, and U4 have to attain a rendezvous. The figure shows a sce-
nario in which 40 other UAVs always occupy the concerned airspace; these UAVs are
179
0 10 20 30 40 50 60 70 800
10
20
30
40
50
60
70
80
time
ET
A
26 28 30
34
36
38
time
ET
A
Figure 6.11: The variation of ETAs of UAVs while attaining rendezvous in presenceof collision avoidance.
assumed to be non-cooperative, that is, they do not perform collision avoidance ma-
neuvers. A particular case of rendezvous of UAVs U1, U2, U3, and U4 with and without
collision avoidance with other UAVs is shown in Figure 6.13. The solid lines are the
paths that these UAVs take to the rendezvous point while not performing any collision
avoidance maneuvers; the dashed lines are the rendezvous trajectories with maneuvers
to avoid collisions. The performance of the collision avoidance algorithm and the con-
sensus algorithm for rendezvous in such a mission for various aircraft densities are
shown in Table 6.1. In this simulation set up, 4 UAVs, starting at a random point along
each edge of the airspace as shown in Figure 6.12, have to rendezvous at the center.
As seen in the same figure, there are other UAVs originating at random points along
one of the four edges (boundaries) of the concerned airspace and flying to the opposite
edge. During a particular simulation, the number of these UAVs in the airspace is kept
constant by introducing new UAVs to replace the ones which reached their respective
destinations. Every UAV is assumed to have a speed of 15 m/s. The UAVs U1, U2,
U3, and U4 have a minimum radius of turn of 100 m. Since the environment is ran-
dom, the values reported in Table 6.1 are averages over 30 runs. We note from the
table that use of the collision avoidance algorithm RCA-2D, developed in Chapter 2,
keeps the number of near misses to very low values for the kind of congested environ-
ment considered. In Table 6.1, the efficiency represents the ratio of the time taken for
180
−1000 0 1000−1000
0
1000
meters
met
ers
U1
U2
U3
U4
Figure 6.12: Congested airspace in which a rendezvous need to be achieved.
Table 6.1: Performance of the rendezvous algorithm in congested environments (aver-aged over 30 runs)
Aircraft Density Near Misses Efficiency (%)20 0.83 83.5040 1.10 81.5360 2.63 66.84
rendezvous with collision avoidance to the time taken without collision avoidance ma-
neuvers. The efficiencies obtained are less because the collision avoidance maneuver
of one of the UAVs will cause all the other UAVs also to alter their paths so as to attain
a rendezvous leading to a large reduction in efficiency. This is in contrast to the cases
considered before in this thesis where the deviations of a few UAVs did not affect the
overall efficiency much.
6.3 Coalition formation in a search and prosecute mission with collision avoidance
In this section, we consider a multiple UAV mission that requires both the collision
avoidance and the coalition formation algorithms to co-exist for the successful com-
pletion of the mission.
181
−1000 −500 0 500 1000
−500
0
500
meters
met
ers
U1
U2
U3
U4
Figure 6.13: Rendezvous with and without collision avoidance in a dense environment.
6.3.1 Coalition formation in presence of collision avoidance
We consider multiple UAV search and prosecute missions, as described in Chapter 5,
with the requirement that the UAVs have to avoid collisions among themselves. To-
ward this, we use the collision avoidance and coalition formation algorithms developed
in previous chapters. These algorithms are higher level algorithms. That is, they form
the part of outer guidance loop independent of the inner control loop. Therefore, it is
possible to ‘plug in’ the collision avoidance and coalition formation algorithms as sep-
arate guidance modules with care taken to specify the priorities of each according to
the mission in hand. For our search and prosecute mission, we require that the collision
avoidance is given priority to coalition formation and subsequent prosecution. For ex-
ample, if a UAV on the way to target prosecution encounters a trajectory conflict with
some other UAV, then the conflict is resolved before proceeding to prosecution. This
behavior will lead to a slight degradation of the performance of the coalition formation
algorithm in terms of the mission completion times. However, as will be demonstrated
through simulations, these deviations are not large.
Figure 6.14 shows the trajectories of two UAVs and a target during a search and
prosecute mission. The solid lines show the UAV trajectories prosecuting targets when
no collision avoidance algorithm is employed; the dashed lines show the UAV trajecto-
ries when they use RCA-2D for collision avoidance among themselves. As seen from
182
−300 −200 −100 0 100 200 300−300
−200
−100
0
100
200
300
meters
met
ers
conflict point
U2
U1
T1
solid line: without collision avoidancedashed line: with collision avoidance
Figure 6.14: Trajectories of UAVs prosecuting a target (solid lines – without collisionavoidance; dashed lines – with collision avoidance).
the figure, it takes longer to prosecute the target when the UAVs have to also avoid
colliding with each other.
6.3.2 Simulation results
Simulations are carried out to study the performance of the coalition formation algo-
rithm in conjunction with the collision avoidance algorithm for multiple UAV search
and prosecute mission. The simulation setting is the same as that described in Sec-
tion 5.5.3.
We consider only the case of prosecution of 10 targets by varying the number of
UAVs. The comparison of the mission completion times with and without collision
avoidance is shown in Fig. 6.15. The values reported and averaged over 100 Monte
Carlo runs. As seen in the figure, there is a slight degradation in the mission perfor-
mance in terms of the mission completion time. However, the deviations are not very
large. Nonetheless, as seen in Fig. 6.15, even in the case where collision avoidance al-
gorithm is used, a similar scale-up of the mission completion time is obtained with the
increase in number of UAVs as in the case without collision avoidance. The average
number of near misses observed for the cases of 10, 20, and 30 UAVs are 0.1, 0.29,
and 0.98 respectively.
183
0 5 10 15 20 25 30 35 4050
100
150
200
250
300
350
400
450
500
Number of UAVs
Mis
sion
Tim
e (s
)
without collision avoidancewith collision avoidance
Figure 6.15: Comparison of average mission times as a function of number of UAVsfor a search and prosecute task for 10 targets with and without collision avoidance.
The above result point to the fact that the collision avoidance and the coalition
formation algorithms can be integrated easily into UAV missions with very less inter-
ference between the two.
6.4 Summary and conclusions
In this chapter, we developed an algorithm for rendezvous of multiple UAVs in the
presence of collision avoidance maneuvers. Toward this, we used the principles of
consensus and showed that achieving consensus on ETA (which is time varying) will
enable a rendezvous. Simulations show that using consensus protocol on ETA will
guarantee rendezvous. Next, we considered the multiple UAV search and prosecute
mission with the additional requirement that the UAVs should avoid collisions among
themselves. Although the missions took slightly longer with the collision avoidance
algorithm activated, the reduction of mission times with increase in number of UAVs
used is achieved in this case also. This points to the fact that the collision avoidance
and the coalition formation algorithms can be used in conjunction in multiple UAV
missions that require both the algorithms.
184
7 Conclusions
This chapter serves as a summary of the thesis and discusses the important resultsand the conclusions arrived at in this thesis. Some directions for future work, arisingfrom the research presented in this thesis, are also provided.
�he inevitable use of UAVs for dull, dirty, and dangerous missions lead to
the use of multiple UAVs, and subsequently to the multiple UAV opera-
tional problems like coordination and cooperation among UAVs. This the-
sis addressed the problems of collision avoidance and coalition formation of UAVs in
high density traffic environments which has applications in many multiple UAV mis-
sions. This chapter concludes the thesis with a summary, highlighting novelties and
directions for further research opened up by the work presented in this thesis.
7.1 Collision Avoidance
One of the main problems that this thesis addressed was that of collision avoidance
among UAVs in high density traffic environments. We advocated the following: When
a UAV faces multiple conflicts, it chooses the most threatening conflict and executes an
appropriate conflict resolution maneuver. We defined the most threatful conflict as one
that has the least time to go for a near miss to occur, and the appropriate conflict reso-
lution maneuver as the one that increases the LOS rate between the concerned UAVs.
Through simulations, we showed that this local strategy gives good performance in
terms of low number of near misses and higher efficiency. The algorithm is simple
and easy to implement. This algorithm was shown to be robust to noise in position
and velocity measurements. The algorithm, as it is reactive, will work even if one of
the UAVs in each pairwise conflict does not perform avoidance maneuver. This al-
gorithm was then implemented in six degree of freedom UAV model augmented with
controllers designed to execute the guidance commands. Simulations using realistic
UAV models showed good performance which is a certificate of both the effectiveness
and ease of implementation of the proposed algorithm. We also developed algorithms
to handle three dimensional conflicts. Toward this, we introduced the concept of a
185
collision plane and proposed a conflict resolution maneuver that consists of either (a)
pulling out of the collision plane, or (b) doing a turn in the collision plane; the second
being a generalization of the two dimensional collision avoidance algorithm developed
earlier to handle three dimensional conflicts.
The two and three dimensional collision avoidance algorithms developed in this
thesis were implemented, in simulation, on 6 DoF realistic UAV models. Toward this,
we designed PI-controllers using successive loop closure technique which when used
in conjunction with the collision avoidance guidance algorithms provided the 6 DoF
UAV models with the required acceleration so as to perform well in high density traffic
simulation scenarios. Nonetheless, the response of the 6 DoF UAV model augmented
with PI control loops was a bit sluggish. Also, the PI-controllers required the infor-
mation of some of the aircraft states as feedback. In this thesis, we assumed that the
required states are available, free of noise. However, in the case of real-world UAVs,
owing to the use of low cost sensors, some states are not measured and hence need to
be estimated, and the measured states are noisy. An estimator should, in principle, take
this into account. However, the main focus of this thesis was to develop conceptually
simple and effective guidance algorithms and demonstrate their ease of implementa-
tion. We, therefore, identify the development of sophisticated controllers/estimators –
to be used in high density traffic scenarios – as an area of future work. Also, neither
the controllers nor the guidance algorithms developed in this thesis take into account
situations where considerable wind is present. Although the presence of wind will not
affect the collision avoidance performance, it is critical for tracking shortest paths to
destinations.
We demonstrated the efficacy of the developed collision avoidance algorithms
through extensive simulations. Two important concepts for any collision avoidance
algorithm are stability and robustness. Broadly, stability of a collision avoidance algo-
rithm refers to its breakdown via ‘domino effect’ and/or unbounded deviations from
nominal trajectories (Mao et al., 2000), when the traffic density increases beyond cer-
tain value; robustness can be looked upon as the smooth degradation of the perfor-
mance of the algorithm with increase in uncertainty of the position, velocity, et cetera
and other disturbances. It is desirable to have theoretical results guaranteeing the sta-
bility and robustness of conflict resolution algorithms. However, as Hoekstra et al.
(2000) observes, this can be extremely difficult. They note that even simple rules, like
cellular automata, leads to disasters and any conflict resolution algorithm is more com-
plicated than cellular automata. “Consequently,” they argue, it may be difficult or even
“impossible to guarantee stability” and robustness of collision avoidance algorithms
186
and they are at best “only be studied using simulations.”
There is no effective measure to quantify the stability and robustness of decen-
tralized collision avoidance algorithms. It is possible to envisage a situation where a
decentralized deconflcition algorithm shows good performance, in terms of number
of near misses, till a specific density and then the performance degrades drastically
leading to a phase transition like behavior. However, there exist no measures either
to quantify this phenomenon or to predict the critical density at which degradation
starts. The nonexistence of such measures is an indication of the complexity of the
problem of quantifying the stability of decentralized collision avoidance algorithms.
The measures of stability defined in the literature favors centralized algorithms thus
leading to wrong conclusions about the performance of decentralized algorithms. For
example, Krozel et al. (2001) defines a stability parameter in terms of the number of
detected conflicts. They argue that if a deconfliction algorithm leads to more number
of detected conflicts, it is unstable. However, it is straightforward to see that the use
of any decentralized algorithm, that operates with only local information, will invari-
ably lead to more detected conflicts than a centralized algorithm, especially in high
density traffic situations; leave aside the fact that a centralized algorithm will be com-
putationally too expensive to handle such a dense environment. It is not the number of
conflict detections, rather the eventual resolution of them, without much compromise
in achieved efficiency, that is the true indicator of stability of an algorithm. Thus, the
measures quantifying phase transition like behaviors, as described before, will better
represent the stability of decentralized algorithms. However, such measures are yet
to emerge. Development of meaningful and useful performance measures to analyze
collision avoidance algorithms designed to handle high density traffic situations will
constitute good future work in this direction.
7.2 Coalition Formation
Another problem that this thesis addressed is the problem of coalition formation among
multiple UAVs. Although, in particular, we considered the coalition formation for a
search and prosecute task, our abstract approach to the problem makes it applicable
to any general task allocation problem. First, we proved that the minimum time mini-
mum member coalition formation problem that we address is NP-hard. Then we gave
a two stage polynomial time algorithm to solve the problem, giving a sub-optimal so-
lution. This algorithm scales well for use in high density traffic situations. This was
187
shown through simulations. Our local strategy for coalition formation gave perfor-
mance comparable to that of the global optimization problem, where the target loca-
tions are known a priori, obtained using Particle Swarm Optimization technique; while
the computational advantage of the proposed two stage algorithm is huge.
The coalition formation over a time varying network was addressed next. Such
a situation arises when the UAVs have limited communication ranges and motion of
UAVs, searching for targets, will cause the formation of a communication network that
is dynamic. Our strategy of tackling the problem of coalition formation over a time
varying network was to find a sub-network that is invariant over the coalition forma-
tion period. We presented communication protocols and showed through simulations
that our approach worked well. Dependence of the performance of the proposed algo-
rithm on mission parameters and network parameters were studied extensively. Studies
showed that the mission performance, in terms of mission completion time, depends
heavily on network parameters like delay. Our approach – finding a sub-network in-
variant over the coalition formation period – was successful as our coalition formation
algorithm is fast and thus the coalition formation time is much lesser compared to the
timescale at which the network topology changes because of the UAVs moving around
in search of targets. Such an approach will definitely fail if the network changes at
a much faster rate. Algorithms for efficiently forming coalitions over such networks
can, possibly, be designed by drawing ideas from communication protocols used in
wireless mobile ad hoc networks.
Managing the available bandwidth effectively becomes important when one is
dealing with groups of hundreds and thousands UAVs. The coalition formation algo-
rithms we proposed, although requires very low communication between the coalition
leader and the potential coalition members for the coalition formation process, the use
of broadcast and flooding mechanisms may cause a bandwidth crunch in very large
groups of UAVs caused by the presence of a large number of communicating relay
nodes in between the coalition leader and the potential coalition members. The band-
width crunch can, to an extent, be mitigated, at the cost of the mission completion
time, by reducing communication ranges and the number of hops a message is allowed
to make. There is scope for design of better communication protocols in these cases.
In this thesis, we considered a sequential prosection of a maneuvering target. An
interesting extension of this can be the simultaneous prosecution of maneuvering tar-
gets, for which, the technique of rendezvous via consensus – presented in Chapter 6
of the thesis – can be used. Also, the search and prosecution of ground targets in the
presence of wind will constitute a good extension of the present work.
188
7.3 Collision avoidance and coalition formation in multiple UAV missions
There are several multiple UAV missions that requires collision avoidance and coali-
tion formation among UAVs. Collision avoidance is a mandatory requirement in all
multiple UAV missions. We considered one such mission – multiple UAV rendezvous
under collision avoidance. The UAVs used a consensus on their expected time of
arrival at the rendezvous point to make simultaneous arrival possible, even in the pres-
ence of collision avoidance maneuvers performed by some of the UAVs en route.
The proposed rendezvous algorithm was then modified for tracking a target – sta-
tionary or maneuvering – from a standoff distance. The algorithm in its current form
does not allow controlling of the spacing between the UAVs tracking the target. In-
corporation of this feature into the consensus based standoff tracking is a potential
extension of the current work. Rendezvous and target tracking using consensus is sim-
ple and effective, as consensus based algorithms are naturally suited for multi-UAV
coordination problems (Ren & Beard, 2008), and there remains a lot to be explored in
this direction.
The rendezvous using consensus depended on the velocity control to achieve ren-
dezvous. This was possible because the collision avoidance algorithm used only turn
maneuvers for deconflcition leaving the freedom to vary velocities within bounds to
other algorithms that may have to work in conjunction, and may require this flexibility,
to make the mission successful. If the collision avoidance algorithm were to be com-
plex, using all available degrees of freedom, not much authority will be left for other
applications. This feature of leaving enough room for other applications is the charac-
teristic of a simple algorithm and is present in the algorithms developed in this thesis,
which is more explicitly seen in the mission that we considered – coalition formation
for search and prosecution in presence of collision avoidance.
As the collision avoidance and coalition formation algorithms developed in this
thesis are high level algorithms, they can be plugged in independently to the set of
functions of a UAV with priorities defined appropriately. For example, in a search and
prosecute mission with collision avoidance, the task of collision avoidance has higher
priority than the task of coalition formation, barring pathological situations. Since
these algorithms are simple, as mentioned earlier, the interaction and interference be-
tween these modules are minimal as depicted by the simulations presented.
For the search and prosecute mission for maneuvering targets, we adopted a se-
quential prosecution procedure. Simultaneous prosecution of maneuvering targets,
189
when the UAVs in the coalition also have to perform collision avoidance maneuvers,
is a good problem that finds practical application, and can be addressed using a slight
modification of our rendezvous via consensus algorithm.
7.4 Concluding remarks
Processes where locally rational or optimal interactions leading to global equilibrium,
optimality, and globally desirable behaviors are ubiquitous in nature. Flocking in birds
(Cucker & Smale, 2007) is an example. This characteristic emerges even out of some
theories that do not explicitly take into account such considerations. An example is
the best response mechanism in game theory where “each player (is) performing only
simple ‘locally rational’ actions and yet, mysteriously, the system would reach a global
equilibrium” (Nisan et al., 2011). The algorithms proposed in this thesis – collision
avoidance and coalition formation algorithms – have the characteristic of being ‘local’,
both in space and time; multiple conflicts were handled by resolving the most immi-
nent conflict, and coalitions were formed as and when the targets were detected and
communicating with only the neighbors. The good ‘global’ performance, in terms of
less number of near misses with high efficiency and small mission completion times, of
these algorithms implicitly brings out this characteristic or property – ‘locally rational
interactions leading to globally acceptable behavior’ – present in the coordination and
cooperation among UAVs in large groups. The knowledge of this characteristic can be
exploited to design useful multiple UAV mission algorithms which are conceptually
simple, yet effective and scalable.
190
Bibliography
Alagar, S., Venkatesan, S., & Cleveland, J. (1995). Reliable broadcast in mobile wireless networks.
In Proceedings of IEEE Military Communications Conference, San Diego, USA, pp. 236–
240.
Albaker, B. M., & Rahim, N. A. (2009a). Straight projection conflict detection and coopera-
tive avoidance for autonomous unmanned aircraft systems. In Proceedings of the 4th IEEE
Conference on Industrial Electronics and Applications, Kuala Lumpur, Malaysia, pp. 1965–
1969.
Albaker, B. M., & Rahim, N. A. (2009b). A survey of collision avoidance approaches for un-
manned aerial vehicles. In Proceedings of the International Conference for Technical Post-
graduates, Kuala Lumpur, Malaysia.
Alighanbari, M., & How, J. (2006). Robust decentralized task assignment for cooperative UAVs.
In AIAA Guidance, Navigation, and Control Conference and Exhibit, Keystone, USA. Paper
No. AIAA-2006-6454.
Archibald, J. K., Hill, J. C., Jepsen, N. A., Strirling, W. C., & Frost, R. L. (2008). A satisficing ap-
proach to aircraft conflict resolution. IEEE Transactions on System, Man, and Cybernetics,
Part C: Applications and Reviews, 38, pp. 510–521.
Arslan, G., Marden, J. R., & Shamma, J. S. (2007). Autonomous vehicle-target assignment :
a game-theoretical formulation. Journal of Dynamic Systems, Measurement, and Control,
129, pp. 584–596.
Banda, S. S. (2002). Future directions in control of unmanned aerial vehicles. In Proceedings of
AFOSR Workshop on Future Directions in Control, Arlington, USA.
Bicchi, A., & Pallottino, L. (2000). An optimal cooperative conflict resolution for air traffic
management systems. IEEE Transactions on Intelligent Transportation Systems, 1, pp. 221–
232.
Bilimoria, K. D. (2000). A geometric optimization approach to aircraft conflict resolution. In
AIAA Guidance, Navigation, and Control Conference and Exhibit, Denver, USA. Paper
No. AIAA-2000-4265.
191
Campbell, A., Wu, A. S., & Shumaker, R. (2002). Multi-agent task allocation: Learning when
to say no. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary
Computation, New York, USA, pp. 201–208.
Carbone, C., Ciniglio, U., Corraro, F., & Luongo, S. (2006). A novel 3D geometric algorithm for
aircraft autonomous collision avoidance. In Proceedings of the 45th IEEE Conference on
Decision and Control, San Diego, USA.
Chakravarthy, A., & Ghose, D. (1998). Obstacle avoidance in a dynamic environment: a collision
cone approach. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and
Humans, 28, pp. 562–574.
Chen, J., & Huang, M. (2005). A broadcast engagement ack mechanism for reliable broadcast
transmission in mobile ad hoc networks. IEICE Transactions on Communications, 88, pp.
3570–3578.
Clark, M. P. (2003). Data Networks, IP and the Internet: Protocols, Design and Operation. John
Wiley & Sons Ltd., Chichester.
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to Algorithms.
MIT Press, Cambridge.
Cucker, F., & Smale, S. (2007). Emergent behavior in flocks. IEEE Transactions on Automatic
Control, 52, pp. 852–862.
Darrah, M., Nil, W., & Stolarik, B. (2006). UAV cooperative task assignments for a SEAD mis-
sion using genetic algorithms. In AIAA Guidance, Navigation, and Control Conference and
Exhibit, Keystone, Colorado. Paper No. AIAA-2006-6456.
Das, K., & Ghose, D. (2009). Positional consensus in multi-agent systems using broadcast control
mechanisms. In Proceedings of the American Control Conference, St. Louis, USA, pp.
5731–5736.
Dhananjay, N., Ghose, D., & Bhat, M. S. (2010). Performance analysis of guidance laws based
on timescale gap. IEEE Transactions on Control Systems Technology, 18, pp. 574–590.
Dowek, G., Munoz, C. A., & Carreno, V. A. (2005). Provably safe coordinated strategy for
distributed conflict resolution. In AIAA Guidance, Navigation, and Control Conference and
Exhibit, San Francisco, USA. Paper No. AIAA-2005-6047.
Dubins, L. E. (1957). On curves of minimal length with a constraint on average curvature, and with
prescribed initial and terminal positions and tangents. American Journal of Mathematics, 79,
pp. 497–516.
192
Eberhart, R. C., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Pro-
ceedings of the Symposium on Micro Machine and Human Science, Piscataway, USA, pp.
39–43.
Eby, M., & Kelly, W. E. (1999). Free flight separation assurance using distributed algorithms. In
Proceedings of the IEEE Aerospace Conference, San Francisco, USA, pp. 429–441.
Eby, M. S. (1994). A self-organizational approach for resolving air traffic conflicts. The Lincoln
Laboratory Journal, 7, pp. 239–254.
Fiorini, P., & Shiller, Z. (1998). Motion planning in dynamic environments using velocity obsta-
cles. The International Journal of Robotics Research, 17, p. 760772.
Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic window approach to collision avoidance.
IEEE Robotics and Automation Magazine, 4, pp. 23–33.
Fraichard, T., & Asama, H. (2004). Inevitable collision states – a step towards safer robots?.
Advanced Robotics, 18, pp. 1001–1024.
Frazzoli, E., Pallottino, L., Scordio, V., & Bicchi, A. (2005). Decentralized coopperative conflict
resolution for mutiple nonholonomic vehicles. In AIAA Guidance, Navigation and Control
Conference and Exhibit, San Francisco, USA. Paper No. AIAA-2005-6048.
Frew, E. W., Lawrence, D. A., & Morris, S. (2008). Coordinated standoff tracking of moving
targets using Lyapunov guidance vector fields. Journal of Guidance, Control, and Dynamics,
31, pp. 290–306.
Furukawa, T., Bourgault, F., Whyte, H. F. D., & Dissanayake, G. (2005). Dynamic allocation
and control of coordinated uavs to engage multiple targets in a time-optimal manner. In
Proceedings of the IEEE Conference on Robotics and Automation, Barcelona, Spain, pp.
2353–2358.
Garey, M. R., & Johnson, D. S. (1979). Computers and Intractability: A Guide to the Theory of
NP-Completeness. W.H. Freeman & Co., New York.
George, J., Sujit, P. B., & Sousa, J. B. (2011). Search strategies on multiple uav search and destroy
missions. Journal of Intelligent and Robotic Systems, 61, pp. 355–367.
Gerkey, B., & Mataric, M. J. (2004). Formal framework for the study of task allocation in multi-
robot systems. International Journal of Robotics Research, 23, pp. 939–954.
Ghawghawe, S. N., & Ghose, D. (1996). Pure proportional navigation against time-varying target
maneuvers. IEEE Transactions on Aerospace and Electronic Systems, 32, pp. 1336–1347.
193
Gomez, M. L., & Fraichard, T. (2009). Benchmarking collision avoidance schemes for dynamic
environments. In Proceedings of the ICRA Workshop on Safe Navigation in Open and Dy-
namic Environments, Kobe, Japan.
Gu, D.-W., Kim, Y., & Postlethwaite, I. (2010). How tight is sphere-packed formation flying?.
Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-
neering, 224, pp. 427–435.
Guelman, M. (1971). A qualitative study of proportional navigation. IEEE Transactions on
Aerospace and Electronic Systems, 7, pp. 637–643.
Han, S.-C., & Bang, H. (2004). Proportional navigation-based optimal collision avoidance for
UAVs. In Proceedings of the 2nd International Conference on Autonomous Robots and
Agents, Palmerston North, New Zealand.
Hill, J. C., Archibald, J. K., Stirling, W. C., & Frost, R. L. (2005). A multi-agent system ar-
chitecture for distributed air traffic control. In AIAA Guidance, Navigation, and Control
Conference and Exhibit, San Francisco, USA. Paper No. AIAA-2005-6049.
Hoekstra, J., Ruigrok, R. C. J., & van Gent, R. N. H. W. (2000). Free flight in a crowded airspace?.
In 3rd USA/Europe Air Traffic Management R&D Seminar, Napoli, Italy.
Hwang, I., & Tomlin, C. (2002). Protocol-based conflict resolution for finite information horizon.
In Proccedings of the American Control Conference, Anchorage, USA, pp. 748–753.
IEEE-SA Standards Board (2007). IEEE wireless LAN medium access control (MAC) and physi-
cal layer (PHY) specifications. Industry standard, IEEE Computer Society, New York, USA.
Kamal, Y. A., Gu, D.-W., & Postlethwaite, I. (2005). Real time trajectory planning for UAVs using
MILP. In Proceedings of the 44th IEEE Conference on Decision and Control and European
Control Conference, Seville, Spain, pp. 3381–3386.
Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. In Proceedings of the IEEE
International Conference on Neural Networks, Piscataway, USA.
Kim, Y., Gu, D.-W., & Postlethwaite, I. (2007). Tight formation flying and sphere packing. In
Proceedings of the American Control Conference, New York, USA, pp. 1085–1090.
Kingston, D. B., & Schumacher, C. J. (2005). Time-dependent cooperative assignment. In Pro-
ceedings of the American Control Conference, Portland, USA, pp. 4084–4089.
194
Krozel, J., Peters, M., Bilimoria, K. D., Lee, C., & Mitchell, J. S. B. (2001). System perfor-
mance characteristics of centralized and decentralized air traffic separation strategies. In
Proceedings of the 4th USA/Europe Air Traffic Management R&D Seminar, Santa Fe, USA.
Kuchar, J. K., & Yang, L. C. (2000). A review of conflict detection and resolution modeling
methods. IEEE Transactions on Intelligent Transportation Systems, 1, pp. 179–189.
Lacher, A. R., Maroney, D. R., & Zeitlin, A. D. (2007). Unmanned aircraft collision avoidance:
Technology assessment and evaluation methods. In Proceedings of the 7th USA/Europe Air
Traffic Management Research and Development Seminar, Barcelona, Spain.
Lalish, E. (2009). Distributed reactive collision avoidance. Ph.D. thesis, University of Washing-
ton, Seattle, USA.
Lalish, E., Morgansen, A., & Tsukamaki, T. (2008). Decentralized reactive collision avoidance
for multiple unicycle-type vehicles. In Proccedings of the American Control Conference,
Seattle, USA, pp. 5055–5061.
Lamport, L. (1984). Using time instead of timeout for fault-tolerant distributed systems. ACM
Transactions on Programming Languages and Systems, 6, pp. 254–280.
Laskari, E. C., Parsopoulos, K. E., & Vrahatis, M. (2002). Particle swatn optimization for in-
teger programming. In Proceedings of the IEEE Congress on Evolutionary Computation,
Honolulu, USA, pp. 1582–1587.
Lin, J., Morse, A. S., & Anderson, B. D. O. (2003). The multi-agent rendezvous problem. In
Proceedings of the IEEE Conference on Decision and Control, Maui, USA, pp. 1508–1513.
Lin, Z., Francis, B., & Maggiore, M. (2005). Necessary and sufficient graphical conditions for
formation control of unicycles. IEEE Transactions on Automatic Control, 50, pp. 121–127.
MAGICC Lab, BYU (2010). Aviones: UAV Flight Simulator. http://aviones.
sourceforge.net/. (Accessed: 2010, May 11).
Mao, Z.-H., Feron, E., & Bilimoria, K. (2000). Stability of intersecting aircraft flows under decen-
tralized conflict avoidance rules. In AIAA Guidance, Navigation, and Control Conference,
Denver, USA. Paper No. AIAA-2000-4271.
McLain, T. W., & Beard, W. R. (2005). Coordination variables, coordination functions, and co-
operative timing missions. AIAA Journal of Guidance, Control, and Dynamics, 28, pp.
150–161.
195
McMasters, J. H., & Cummings, R. M. (2002). Airplane design - past, present, and future. Journal
of Aircraft, 39, pp. 10–17.
Merz, A. W. (1991). Maximum-miss aircraft collision avoidance. Dynamics and Control, 1, pp.
25–34.
Murphey, R. A. (1999). Target-based weapon target assignment problems. In Pardalos, P. M., &
Pitsoulis, L. S. (Eds.), Nonlinear Assignment Problems: Algorithms and Applications, pp.
39–53. Kluwer Academic Publishers, Dordrecht.
Naidu, D. S., & Calise, A. J. (2001). Singular perturbations and time scales in guidance and control
of aerospace systems. Journal of Guidance, Control, and Dynamics, 24, pp. 1057–1078.
Nemhauser, G. L., & Wolsey, L. A. (1999). Integer programming. In Nemhauser, G. L., Rinnoy
Kan, A. H. G., & Todd, M. J. (Eds.), Optimization, Vol. 1 of Handbooks in Operations
Research and Management Science. Elsevier, Amsterdam.
Nisan, N., Schapira, M., Valiant, G., & Zohar, A. (2011). Best response mechanisms. In Proceed-
ings of the 2nd Symposium on Innovations in Computer Science, Beijing, China.
Notarstefano, G., & Bullo, F. (2006). Distributed consensus on enclosing shapes and minimum
time rendezvous. In Proceedings of the IEEE Conference on Decision and Control, San
Deigo, USA, pp. 4295–4300.
Nygard, K. E., Chandler, P. R., & Pachter, M. (2001). Dynamic network flow optimization models
for air vehicle resource allocation. In Proceedings of the the American Control Conference,
Arlington, USA, pp. 1853–1858.
Obraczka, K., Viswanath, K., & Tsudik, G. (2001). Flooding for reliable multicast in multi-hop
ad hoc networks. Wireless Networks, 7, pp. 627–634.
Panyakeow, P., & Mesbahi, M. (2010). Decentralized deconfliction algorithms for unicycle UAVs.
In Proceedings of the American Control Conference, Baltimore, USA, pp. 794–799.
Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial Optimization: Algorithms and Com-
plexity. Prentice-Hall, New Jersey.
Park, J.-W., Oh, H.-D., & Tahk, M.-J. (2008). UAV collision avoidance based on geometric
approach. In Proceedings of the SICE Annual Conference, Tokyo, Japan.
Parker, L. E., & Tang, F. (2006). Building multi-robot coalitions through automated task solution
synthesis. Proceedings of the IEEE, 94, pp. 1289–1305.
196
Parsopoulos, K. E., & Vrahatis, M. N. (2002). Recent approaches to global optimization problems
through particle swarm optimization. Natural Computing, 1, pp. 235–306.
Rahmani, A., Kosuge, K., Tsukamaki, T., & Mesbahi, M. (2008). Multiple UAV deconfliction via
navigation functions. In AIAA Guidance, Navigation, and Control Conference and Exhibit,
Honolulu, USA. Paper No. AIAA-2008-6626.
Rahwan, T., Ramchurn, S., Jennings, N., & Giovannucci, A. (2009). An anytime algorithm for
optimal coalition structure generation. Journal of Artificial Intelligence Research, 34, pp.
521–567.
Rasmussen, S., & Schumacher, C. (2007). Cooperative control of unmanned aerial vehicles. In-
ternational Journal of Robust and Nonliner Control, 18, pp. 115–117.
Reid, J. G., Furlough, T. L., & Young, J. T. (1981). Maximum acceleration design, digital optimal
guidance. In Proceedings of the AIAA Guidance and Control Conference, Albuquerque,
USA, pp. 416–422. Paper No. AIAA-1981-1782.
Ren, W., & Beard, R. W. (2008). Distributed Consensus in Multi-Vehicle Cooperative Control:
Theory and Applications. Springer-Verlag, London.
Roussos, G., Dimarogonas, D. V., & Kyriakopoulos, K. J. (2010). 3D navigation and collision
avoidance for nonholonomic aircraft-like vehicles. International Journal of Adaptive Con-
trol and Signal Processing, 24, pp. 900–920.
RTCA (2002). Minimum aviation system performance standards for automatic dependent surveil-
lance broadcast (ads-b). Tech. rep., Do-242A, RTCA.
Samek, J., Sislak, D., Volf, P., & Pechoucek, M. (2007). Multi-party collision avoidance among
unmanned aerial vehicles. In Proceedings of the International Conference on Intelligent
Agent Technology, Fremont, USA, pp. 403–406.
Sandholm, T., Larson, K., Andersson, M., Shehory, O., & Tohme, F. (1999). Coalition structure
generation with worst case guarantees. Artificial Intelligence, 111, pp. 209–238.
Scerri, P., Liao, E., Xu, Y., Lewis, M., Lai, G., & Sycara, K. (2008). Coordinating very large
groups of wide area search munitions. In Grundel, D., Murphey, R., & Pardalos, P. M.
(Eds.), Theory and Algorithms for Cooperative Systems, Vol. 4 of Series on Computers and
Operations Research, pp. 451–480. World Scientific Publishing Co. Pte. Ltd., Singapore.
Schumacher, C., & Chandler, P. (2004). UAV task assignment with timing constraints via mixed-
integer linear programming. In AIAA Unmanned Unlimited Technical Conference, Workshop
and Exhibit, Chicago, USA. Paper No. AIAA-2004-6410.
197
Shanmugavel, M., Tsourdos, A., White, B., & Zbikowski, R. (2010). Co-operative path planning
of multiple UAVs using Dubins paths with clothoid arcs. Control Engineering Practice, 18,
pp. 1084–1092.
Shanmugavel, M., Tsourdos, A., Zbikowski, R., White, B. A., Rabbath, C. A., & Lechevin, N.
(2006). A solution to simultaneous arrival of multiple uavs using pythagorean hodograph
curves. In Proceedings of the American Control Conference, Minneapolis, USA.
Shehory, O. M., & Kraus, S. (1998). Methods for task allocation via agent coalition formation.
Artificial Intelligence, 101, pp. 165–200.
Shehory, O. M., Sycara, K., & Jha, S. (1998). Multi-agent coordination through coalition for-
mation. In Singh, M. P., Rao, A., & Wooldridge, M. J. (Eds.), Intelligent Agents IV: Agent
Theories, Architectures and Languages, Vol. 1365 of Lecture Notes in Artificial Intelligence,
pp. 143–154. Springer-Verlag, Berlin.
Shi, Y., & Eberhart, R. C. (1998). A modified particle swarm optimizer. In Proceedings of the
IEEE Conference on Evolutionary Computation, Anchorage, USA, pp. 69–73.
Shiller, Z., Large, F., & Sekhavat, S. (2001). Motion planning in dynamic environments: obstacles
moving along arbitrary trajectories. In Proceedings of IEEE International Conference on
Robotics and Automation, Seoul, Korea, pp. 3716–3721.
Shim, D. H., Kim, H. J., & Sastry, S. (2003). Decentralized nonlinear model predictive control of
multiple ying robots in dynamic enviornments. In Proccedings of the IEEE Conference on
Decision and Control, Maui, USA, pp. 3621–3626.
Shima, T., Schumacher, C., & Rasmussen, S. (2009). Multiple assignments. In Shima, T., &
Rasmussen, S. (Eds.), UAV Cooperative Decision and Control: Challenges and Practical
Approaches, No. 18 in Advances in Design and Control, pp. 53–72. SIAM, Philadelphia.
Shin, H., Tsourdos, A., White, B., Shanmugavel, M., & Tahk, M. (2008). UAV conflict detection
and resolution for static and dynamic obstacles. In AIAA Guidance, Navigation, and Control
Conference and Exhibit, Honolulu, USA. Paper No. AIAA-2008-6521.
Sinha, A., & Ghose, D. (2006). Generalization of linear cyclic pursuit with application to ren-
dezvous of multiple autonomous agents. IEEE Transactions on Automatic Control, 51, pp.
1819–1824.
Sinha, A., & Ghose, D. (2007). Generalization of non linear cyclic pursuit. Automatica, 43, pp.
1954–1960.
198
Sislak, D., Rehak, M., Pechoucek, M., Pavlicek, D., & Uller, M. (2006). Negotiation-based
approach to unmanned aerial vehicles. In Proceedings of the IEEE Workshop on Distributed
Intelligent Systems: Collective Intelligence and Its Applications, Prague, Czech Republic,
pp. 279–284.
Sislak, D., Volf, P., Komenda, A., Samek, J., & Pechoucek, M. (2007). Agent-based multi-layer
collision avoidance to unmanned aerial vehicles. In Proceedings of the International Con-
ference on Integration of Knowledge Intensive Multi-Agent Systems, Waltham, USA, pp.
365–370.
Smith, A. L., & Harmon, F. G. (2010). UAS collision avoidance algorithm based on an aggregate
collision cone approach. Journal of Aerospace Engineering. DOI: 10.1061/(ASCE)AS.
1943-5525.0000081.
Stevens, B. L., & Lewis, F. L. (2003). Aircraft Control and Simulation. John Wiley and Sons,
New Jersey.
Sujit, P. B., & Ghose, D. (2009). Negotiation schemes for multi-agent cooperative search. Pro-
ceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-
neering, 223, pp. 791–813.
Sujit, P. B., & Ghose, D. (2010). Two agent cooperative search using game theoretical models
with endurance time constraints. Engineering Optimization, 42, pp. 617–639.
Sujit, P. B., & Ghose, D. (2011). Self assessment based decision making for multi-agent coopera-
tive search. IEEE Transactions on Automation Science and Engineering. (to appear).
Sujit, P. B., Sinha, A., & Ghose, D. (2005). Multi-UAV task allocation using team theory. In
Proceedings of the IEEE Conference on Decision and Control, and European Control Con-
ference, Seville, Spain, pp. 1497–1502.
Summers, T. H., Akella, M. R., & Mears, M. J. (2009). Coordinated standoff tracking of moving
targets: Control laws and information architectures. Journal of Guidance, Control, and
Dynamics, 32, pp. 56–69.
Suresh, M., & Ghose, D. (2010). Role of information and communication in redefining unmanned
aerial vehicle autonomous control levels. Proceedings of the Institution of Mechanical En-
gineers, Part G: Journal of Aerospace Engineering, 224, pp. 171–197.
Tang, K., & Gerla, M. (2001). MAC reliable broadcast in ad hoc networks. In Proceedings of the
IEEE Military Communications Conference, pp. 1008–1013.
199
Tiwari, A., Fung, J., Carson, J. M., Bhattacharya, R., & Murray, R. M. (2004). A framework for
lyapunov certificates for multi-vehicle rendezvous problems. In Proceedings of the Ameri-
can Control Conference, Boston, USA, pp. 5582–5587.
Tomlin, C., Pappas, G. J., & Sastry, S. (1998). Conflict resolution for air traffic management: a
case study in multi-agent hybrid systems. IEEE Transactions on Automatic Control, 43, pp.
509–521.
Tosic, P. T., & Agha, G. A. (2005). Maximal clique based distributed coalition formation for task
allocation in large-scale multi-agent systems. In Ishida, T., Gasser, L., & Nakashima, H.
(Eds.), Massively Multi-Agent Systems I, Vol. 3446 of Lecture Notes in Artificial Intelligence,
pp. 104–120. Springer-Verlag, Berlin.
Tsourdos, A., Jeyaraman, S., Shanmugavel, M., White, B. A., & Zbikowski, R. (2005). A formal
model approach for the analysis and validation of the cooperative path planning of a UAV
team. In Proceedings of the IEE Seminar on Autonomous Agents in Control, Bicester, UK,
pp. 67–73.
Tsourdos, A., White, B., & Shanmugavel, M. (2011). Cooperative Path Planning of Unmanned
Aerial Vehicles. John Wiley and Sons, West Sussex.
Tsourdos, A., Zbikowski, R., & White, B. A. (2007). Cooperative control strategies for swarm
of unmanned aerial vehicles under motion uncertainty. In Proceedings of the Institution of
Engineering and Technology Conference on Autonomous Systems, London, UK.
Vanzin, M. M., & Barber, K. S. (2006). Decentralized partner finding in multi-agent systems.
In Scerri, P., Vincent, R., & Mailler, R. (Eds.), Coordination of Large-Scale Multiagent
Systems, pp. 75–98. Springer, New York.
Vig, L., & Adams, A. J. (2005). A framework for multi-robot coalition formation. In Proceedings
of the 2nd Indian International Conference on Artificial Intelligence, Pune, India, pp. 347–
363.
Vig, L., & Adams, A. J. (2006a). Market-based multi-robot coalition formation. In Gini, M., &
Voyles, R. (Eds.), Distributed Autonomous Robotic Systems 7, pp. 227–236. Springer-Verlag,
Tokyo.
Vig, L., & Adams, A. J. (2006b). Multi-robot coalition formation. IEEE Transactions on Robotics,
22, pp. 637–649.
200
Williams, B., & Camp, T. (2002). Comparison of broadcasting techniques for mobile ad hoc
networks. In Proceedings of the 3rd ACM Interational Symposium on Mobile Ad Hoc Net-
working and Computing, Lausanne, Switzerland, pp. 194–205.
Wright, O. (1923). How we made the first flight. Aviation, 15, pp. 737–741. Reprinted from
Flying, December, 1913.
Zagi (2010) http://zagi.com/. (Accessed: 2010, May 11).
Zarchan, P. (1990). Tactical and Strategic Missile Guidance, Vol. 124 of Progress in Astronautics
and Aeronautics. AIAA, Washington, DC.
201
202
Publications from this thesis
1. Manathara, J. G., & Ghose, D. (2011). A reactive algorithm for collision avoidance among multi-
ple unmanned aerial vehicles. (manuscript under preparation).
2. Manathara, J. G., & Ghose, D. (2011). Multiple UAV collision avoidance using realistic UAV
models. Journal of Aircraft Engineering and Aerospace Technology. (under revision).
3. Manathara, J. G., & Ghose, D. (2011). Rendezvous of multiple UAVs under collision avoidance
using consensus. Journal of Aerospace Engineering. (under revision).
4. Manathara, J. G., Sujit, P. B., & Beard, R. W. (2011). Multiple UAV coalitions for a search and
prosecute mission. Journal of Intelligent and Robotic Systems, 62, pp. 125–158. DOI: 10.1007/
s10846-010-9439-2
5. George, J., & Ghose, D. (2010). Multiple UAV collision avoidance using realistic UAV models. In
Proceedings of the International Conference on Intelligent Unmanned Systems, Bali, Indonesia.
6. George, J., & Ghose, D. (2010). Multiple UAV rendezvous under collision avoidance. In Pro-
ceedings of the International Conference on Intelligent Unmanned Systems, Bali, Indonesia.
7. George, J. M., Sujit, P. B., & Sousa, J. B. (2010). Coalition formation with communication delays
and maneuvering targets. In AIAA Guidance, Navigation and Control Conference and Exhibit,
Toronto, Canada. Paper No. AIAA-2010-8422.
8. George, J. M., Sujit, P. B., Sousa, J. B., & Pereira, F. L. (2010). Coalition formation with com-
munication ranges and moving targets. In Proceedings of the American Control Conference, Bal-
timore, USA, pp. 1605–1610.
9. George, J. M., Pinto, J., Sujit, P. B., & Sousa, J. B. (2010). Multiple UAV coalition formation
strategies. In Proceedings of 9th International Conference on Autonomous Agents and Multiagent
Systems (AAMAS 2010), Toronto, Canada, pp. 1503–1504.
10. George, J., & Ghose, D. (2009). A reactive inverse PN algorithm for collision avoidance among
multiple unmanned aerial vehicles. In Proceedings of the American Control Conference, Missouri,
USA, pp. 3890–3895.
11. George, J., & Ghose, D. (2009). A 3-D reactive algorithm for collision avoidance among multiple
unmanned aerial vehicles in dense traffic situations. In Proceedings of the International Confer-
ence and Exhibition on Aerospace Engineering, Bangalore, India, pp. 1215–1222.
203