Multi-robot path planning using co-evolutionary genetic programming

Expert Systems with Applications 39 (2012) 3817–3831

Contents lists available at SciVerse ScienceDirect

Expert Systems with Applications

journal homepage: www.elsevier .com/locate /eswa

Multi-robot path planning using co-evolutionary genetic programming

Rahul Kala ⇑School of Cybernetics, School of Systems Engineering, University of Reading, Whiteknights, Reading, Berkshire, UK

a r t i c l e i n f o a b s t r a c t

Keywords:Path planningMotion planningMobile roboticsGenetic programmingGrammatical evolutionCo-operative evolutionMulti-robot systems

0957-4174/$ - see front matter � 2011 Elsevier Ltd. Adoi:10.1016/j.eswa.2011.09.090

⇑ Tel.: +44 (0) 7424752843.E-mail address: [email protected]: http://rkala.99k.org/

Motion planning for multiple mobile robots must ensure the optimality of the path of each and everyrobot, as well as overall path optimality, which requires cooperation amongst robots. The paper proposesa solution to the problem, considering different source and goal of each robot. Each robot uses a grammarbased genetic programming for figuring the optimal path in a maze-like map, while a master evolution-ary algorithm caters to the needs of overall path optimality. Co-operation amongst the individual robots’evolutionary algorithms ensures generation of overall optimal paths. The other feature of the algorithmincludes local optimization using memory based lookup where optimal paths between various crosses inmap are stored and regularly updated. Feature called wait for robot is used in place of conventionallyused priority based techniques. Experiments are carried out with a number of maps, scenarios, and dif-ferent robotic speeds. Experimental results confirm the usefulness of the algorithm in a variety ofscenarios.

� 2011 Elsevier Ltd. All rights reserved.

1. Introduction

The problem of multi-robot motion planning deals with compu-tation of paths of various robots such that each robot has an opti-mal or near optimal path, but the overall path of all the robotscombined is optimal. This is a more complex task as compared toa single-robot motion planning, where the factor of coordinationamong the various robots is not applicable, and the single robotcan use its own means to compute the path (Parker, Schneider, &Schultz, 2005). The problem of motion planning may be centralizedor decentralized. In centralized planning all the robots are centrallyplanned by a planner, usually taking into account all the complexinteractions that they may have. This results in the generation ofa very complex configuration space, over which the search is tobe performed. The decentralized planning, on the other hand, hasan independent planner for every robot. Each robot is planned sep-arately in its own configuration space, which makes the planningmuch simpler. Then efforts may be made to avoid the possibilityof collision between the various robots. The centralized planningis more time consuming, but optimal as compared to decentralizedplanning (Arai & Ota, 1992; Sánchez-Ante & Latombe, 2002).

In this paper we assume a simple problem modeling scenario.We have a maze like map of M � N grids. Each grid may be blackor white, denoting the presence or absence of obstacle, which areall assumed to be stationary. The grid is however not the unit posi-tion for robots which can occupy partial grid positions on the map.

ll rights reserved.

The complete map consists of horizontal and vertical lanes, whichcris-cross each other at grids known as crosses. The task is to moven robots R1,R2, . . .,Rn. Each robot Ri has a start grid Si, which is itsinitial location; a goal grid Gi, which is the destination where it in-tends to arrive at the end of its journey; and a constant speed ofmotion Vi (0 < Vi 6 1) grids/unit time step. All robots are assumedto be of the same size of 1 � 1 grids. The planner needs to planthe path of all the robots Ri. Let the path of robot Ri be given byPi(t), which denotes its position at time t. We know that Pi(0) = Si.Let the path generated be such that the robot Ri reaches the goalGi at time Ti, i.e. Pi(Ti) = Gi. We assume that after reaching the goal,the robot disappears from the goal, and does not obstruct other ro-bots to pass by, i.e. Pi(Ti + 1) = NIL. In this assumption we differfrom the works over robot priorities by Oliver, Saptharishi, Dolan,Trebi-Ollennu, and Khosla (1999), Bennewitz, Burgard, and Thrun(2001, 2002), and Carpin and Pagello (2009). The planner needsto work such that the average time of travel for all the robots isas small as possible, i.e. min

PTið Þ=n. At the same time the planner

needs to ensure that no collision occurs. Hence Pi (t) – Pj (t),1 6 i, j 6 n, i – j.

Genetic programming is extensively used evolutionary tech-nique for problem solving. Here we represent the individual as aprogram and hence try to evolve it using the methodology of theevolutionary algorithms. The solution is obtained by executing theprogram so formed. Tree based representation of a program is a verycommon representation of individuals of genetic programming(Koza, 1992; Shukla, Tiwari, & Kala, 2010). The linear representationof the program of genetic programming gives rise to grammaticalevolution. Here the individual is a sequence of integers. Every prob-lem has its grammar, which represents the program syntax. The

http://dx.doi.org/10.1016/j.eswa.2011.09.090

mailto:[email protected]

http://rkala.99k.org/


http://www.sciencedirect.com/science/journal/09574174

http://www.elsevier.com/locate/eswa

3818 R. Kala / Expert Systems with Applications 39 (2012) 3817–3831

grammar is specified by Backus-Naur or BNF form. The generation ofthe phenotype solution from its genotype solution or sequence ofintegers takes place by selecting and firing rules specified in thegrammar (O’Neill & Ryan, 2001, 2003). Our implementation of thegenetic programming in this paper is similar to the general algo-rithm of grammatical evolution.

The evolutionary algorithms are used for solving various kindsof problems. These algorithms however face problems when thedimensionality of the problem becomes very large. In such a casethe optimization provided by these algorithms becomes very slow,with fairly large chance of the algorithm getting struck at some lo-cal optima (Kala, Shukla, & Tiwari, 2010a). Co-operative evolution-ary algorithms or co-evolutionary algorithms break up the probleminto a number of smaller problems that together solve the mainproblems. The smaller problems have smaller dimensionality,and are hence very easy to be optimized. The different sub-prob-lems or sub-modules evolve in parallel, which ultimately resultsin generation of very good modules that unite with each other tosolve the problem well. Cooperation is encouraged between mod-ules, and a module is given high fitness not only because it resultsin generation of higher fitness solutions, but also because it enablesother modules to achieve high fitness value (Potter & De Jong,1994, 2000; García-Pedrajas, Hervás-Martínez, & Muñoz Pérez,2003).

In this paper we first present a co-evolutionary genetic program-ming based planning of multiple robots. Each of the individuals hasits own optimization process that is based on the principles ofgrammatical evolution. These all try to generate and optimize thepaths of the individual robots. However these need to consider thepossibility of collision between the robots, for which the factor ofco-operation comes into play and has been induced in the algo-rithm. A master genetic algorithm runs to select the best paths ofthe robots, out of all the paths available in the individualoptimizations.

The major problem with the assumed scenario is the variablespeed. This puts forward a number of scenarios, where sub-opti-mality may be possible without careful planning. Consider a bigcorridor with two robots A and B marching towards it. Considermaximum velocity of A being higher, i.e. VA� VB. Consider that ro-bot B is closer to the corridor entry, and would enter it before A.But if robot B enters the corridor first, robot A would have to followit, and both robots would be able to walk with a maximum velocityof VB. It may be clearly seen that optimal strategy would be in caseB waits for robot A to enter the corridor, and follows it inside thecorridor. In such a case both robots would get a chance to travelby their maximum velocities. Hence we implement the conceptof wait for robot, where a robot may be asked to wait so that someother robot may cross.

Optimization by evolutionary algorithms may get very complexin problems having too many dimensions and multiple modalities.In such a case it is often useful to use a local search strategy basedon predefined heuristics that can aid the evolutionary algorithms.This saves the algorithm from likely convergence into local optima.The local search algorithm carries forward the task to converge thealgorithm into some local or global optima, while the evolutionaryalgorithm does the task of locating the best possible regions, wherethe global optima may lie. In such a case the combined efforts ofheuristics and evolutionary algorithms results in better optimiza-tion (Kala, Shukla, & Tiwari, 2009). The same problem of complexoptimization is prevalent in the present algorithm. Here the evolu-tionary algorithm tries to optimize the individual robotic path. Insuch a case also we use a local search strategy which tries to modifythe path to make it smaller in length. Longer paths are replaced bysmaller paths. For carrying forward this replacement, an externalmemory is reserved that stores the smallest paths between all pairsof crosses. This table is regularly updated as more paths are found.

The novelty of the paper is threefold. We first propose a co-evo-lutionary genetic programming model for solving the problem.This model uses principles of cooperative evolution to generatepaths that are optimal in time, by mixing centralized as well asdecentralized planning. Secondly, we propose the model and issueswith multi-speed, map based (single-lane) multi-robot motionplanning. This includes the use of the wait for robot feature incor-porated into the evolutionary approach. Thirdly we propose use ofexternal memory for local optimization of the paths. This memoryis initialized, updated, and queried for reduction in paths of indi-vidual robots.

The paper is organized as follows. In Section 2 we present someof the related works into the field. In Section 3 we present the co-evolutionary genetic programming model of the algorithm. Section4 presents the concept of wait for robot. Section 5 presents thememory based local search used in the algorithm. The experimen-tal results are given in Section 6. Section 7 gives the conclusionremarks.

2. Related works

The entire paradigm of path planning of multi-robots involve alarge set of issues that range from dynamic changes in the roboticplan, to validity of non-holonomic constants, and execution andmemory time complexity. Oliver et al. (1999) give a general archi-tecture of path planning in these scalable scenarios. They followeda decentralized approach of planning where every robot does itsown independent path planning. The map is built centrally at startand all changes are continuously monitored and any needed cor-rection is made. Possible collisions are resolved by the coordina-tion module, that uses a priority based coordination with ahigher priority robot allowed to make the collision prone move,while the other robot compromises or waits. Possible collisionsare predicted be extrapolation of the motion of robots, and anypossibility is marked by a check-point. One of the robots is allowedto move, while the path of other may be re-planned.

Motion planning may be regarded as a part of the entire roboticarchitecture, for which again numerous models have been built.Asama, Matsumoto, and Ishida (1989) present a complete systemfor multi-robot planning, coordination, and control in their de-signed system ACTRESS (ACTor-based Robots and EquipmentsSynthetic System). In this system the authors demonstrate themechanisms in which the different modules including sensors,processing, planning, manipulators, communication etc. come intoplay for intelligent and autonomous task accomplishment. Theextension of their work can be found in Asama et al. (1991) wherethe authors developed a multi-level path planning. The planningwas different for static and dynamic cases. Special preventionswere taken for deadlock avoidance that may take place many timesin planning of multiple robots. The complete hierarchy of thesystem includes static path planning, use of local algorithm, useof mobile rules, task prioritization, deadlock avoidance by low-level deadlock solver, deadlock avoidance by high-level deadlocksolver, and human operator expertise. Vendrell, Mellado, and Cre-spo (2001) presented a general framework for motion planningwith multiple robots. They highlighted the need to plan, sense,and accordingly re-plan by every robot. The authors further pres-ent a hierarchical model of the various robot activities that rangefrom high abstraction to low abstraction. The hierarchy includesmission, task, action, motions, trajectories, and robot order.

A number of models have been made into the earlier works ofthe authors on single robots that revolve around evolutionary algo-rithms. In Kala, Shukla, and Tiwari (2010b) a hierarchical imple-mentation of a customized evolutionary algorithm was proposed.A number of evolutionary operators were defined to result in

R. Kala / Expert Systems with Applications 39 (2012) 3817–3831 3819

better convergence. The hierarchical nature enabled the robot towork in dynamic environment. The coarser evolutionary algorithmmainly looked at the static obstacles, and their optimality, the finerevolutionary algorithm tried to escape from all dynamic obstaclesusing a space-time graph map. In another work Kala, Shukla, andTiwari (2011) used evolutionary algorithm for generation of pathin an incremental manner. Here the complexity of path or thenumber of turns that the path could have increased with time.The concept of momentum was floated which controlled the reso-lution with which the robot checked for the path feasibility andother metrics.

The general approach used in our algorithm is of evolutionaryalgorithm, which is a member of the class of probabilistic algo-rithms, for which a bulk of research has been done. Kavraki andLatombe (1997) and Kavraki, Svestka, Latombe, and Overmars(2002) presented a method for multi-robot path planning calledas the Probabilistic Road Map (PRM), which has since then beenextensively modified and used for research. The algorithm consistsof an offline phase and an online phase. In the offline phase thealgorithm learns the map, and stores the summary about the pathsbetween various points in a special data structure called as theroad map. This step involves identification of a number of nodesthat might represent difficult points in the configuration space,and are chosen by heuristics. The number of points is further in-creased by selection of points around these points. This selectionmay go on till required points are obtained, with the selectionbeing weighted by a node’s difficulty or other selected heuristics.A local planning technique is used to swiftly compute the connec-tivity of these points to the other points. The collection of thesevertices and edges makes the road map to use used for online plan-ning. The online planner is query based which gets a query for apath and reacts to it by returning the shortest path based on thepresent map as well as the summarizations stored in the offlinephase. This path may be further optimized by some local optimiza-tion technique, to increase the path optimality.

Bohlin and Kavralki (2000) further extend the basic PRM tobuild a lazy PRM. The PRM does a large number of collision checksthat usually results in a large wastage of time. In lazy PRM the con-cept is to generate random nodes in the offline mode and try toconnect each node to its neighbors for connectivity assessment.The feasibility of the paths is checked lazily in a coarser to finermanner. An initial lazy checking of the path feasibility reducesnumber of checks, at the same time giving an idea of the feasibility.If path is found to be feasible, it may be checked at a finer level aswell. The in-feasible paths are enhanced by placement of thenodes. The PRM methods can be adapted to present a strong vari-ance between central and de-central path planning with multiplerobots. A comparison of centralized planning, decoupled planningwith global coordination, and decoupled planning with pair wisecoordination was done by Sánchez-Ante and Latombe (2002). Alarge number of experiments of different number of robots withlarge degrees of freedom and scenarios were done. Experimentalresults confirmed that the requirement of robot dependency wasa major factor behind the better performance of the coordinationlevel.

Clark (2005) used a probabilistic roadmap technique for solvingthe problem of motion planning of multiple robots. They used asingle query based probabilistic roadmap planner. Here the authormade a complete architecture comprising of software, communica-tion, planning, and control where a robot could get the positionalupdates from all the other robots to get the complete roadmapand make decision. Here the author considered possibility ofbreaking of connectivity between two robots, resulting in genera-tion of two unconnected sub-networks. The algorithm could tracksuch changes and still make decisions of motion. Another probabi-listic implementation can be seen in the work of Clark, Rock, and

Latombe (2003) who solved the problem of path planning for mul-tiple-robot space system. Their planning algorithm generated ran-dom points or milestones and attempted to reach one milestonefrom the other, till the location was near to the goal. Heuristicswere used for the selection of the milestone, whose neighborhoodbecame the possible location of next milestone. Re-planning wasan integral version of the algorithm, as the map could change inreal time as per availability of information. Combined part resultsof a single decentralized planning system are used to serve as mile-stones of the centralized planner that checks for feasibility in com-bined motion.

Svestka and Overmars (1998) present a probabilistic solutionfor solving the problem of multi-robot path planning. In thisapproach they define the paradigm of coordinated graphs, whichallows only a collision free motion of the mobile robot. The robotis not allowed to collide with any of the moving robots. Theirapproach is partially central in nature, where the major task ofplanning lies with a central authority, on whose decisions the indi-vidual robots are made to move. The entire configuration space ismade discrete with respect to the movement of the multiple ro-bots. This the authors do by the concept of flat graphs, which takesinto account the prospective motion of multiple robots. Observingthe highly time consuming nature of the algorithm and the lowscalability to the number of robots (large attributed to the centralplanning mechanism), the authors further extend their work byusing multi-level graph clustering. Here the original graph isdecomposed into a number of independent sub-graphs. This lay-ered solution to the problem limits the number of nodes of thegraph, which ultimately affects the time complexity of the algo-rithm in an exponential manner. The path hence generated bythe algorithm is given some characteristics of non-central natureby introduction of mechanisms of smoothness of the path, forgreater optimality. This is done by the application of a local plan-ning algorithm for every robot in motion.

Dongyong, Jinyin, Matsumoto, and Yamane (2006) used an evo-lutionary approach to solve the problem. Here the paths keptimproving with time. The authors used a chaos based evolution,where the individuals were disturbed by some factors dependingupon user set parameters. If the disturbed individual resulted ina better fitness, it was retained else the parent was retained. Thecrossover and mutation rates were adapted with time as per thefitness value. The authors optimized the navigation time, pathlength, and path smoothness. Kuffner and LaValle (2000) presentan interesting application of another probabilistic algorithm calledRRT. Here the RRT Trees start exploding both from the source aswell as from the goal. The intersection of the two trees hence pro-vides the path early. Another deviation from the conventional RRTTrees that the authors present is that the exploration of any nodecontinues to take place in the same direction till the goal, an obsta-cle, or map wall is met. This results in heavy exploration at everystep of the algorithm, and the general ideal goes a long way in en-abling the RRT escape from a local minima.

Wang and Wu (2005) make use of cooperative co-evolution insolving the problem of multi-robot path planning. This is primarilyan evolutionary approach, where every robot has its own evolu-tion. The evolution of all robots are shared, coordinated and com-municated by a central mechanism. Every robot, in its evolution,enjoys a fitness that is a function of the length of path travelledand the constraints violated. The constraints refer to the individualrobot constraints as well as the constraints to avoid collision withother robot. Slusny, Neruda, and Vidnerová (2010) used evolution-ary radial basis function networks and reinforcement learning forthe task of localization and motion planning of a robot. The modelswere later analyzed by rule extraction technique, which convertedthe model to a rule based model for easy analysis and understand-ing. Kala, Shukla, and Tiwari (2010c) also used genetic algorithms

Crosses

Fig. 1. Various types of crosses. The figure depicts the numerous possible crossesthat may be possible in a map where vertical and horizontal roads criss cross. Thefigure shows crosses with 4, 3, and 2 paths.


for the optimization of a fuzzy inference system that was used formotion planning. This approach was performed at two levels. Atthe first level a probabilistic A⁄ algorithm was used for making arough sketch of the path, which was later on made detailed bythe fuzzy based planner. In another work Kala, Shukla, and Tiwari(2010d) used evolutionary algorithms to make the rough sketch ofthe path, which was later traversed by the fuzzy planner. In thiswork again the concepts of incremental evolution were used.

Another novel part of the algorithm was of variable speeds,which again has its inspiration from literature, with numerousrelated models. Kolushev and Bogdanov (2000) take into accountthe speeds of the moving robots while deciding their path in agraphical representation of the map or the configuration space.The robots may speed down or speed up below the maximumallowable velocity to avoid collisions. At any edge, the speed is alsotaken account, and correspondingly both the lowest path lengthplan and the shortest duration plan may be found out. A simplegraph search algorithm may be used for planning. Rules are madeto alter the velocity such that no collision occurs at the nodes aswell as the edges of the graph.

Priority based schemes have been extensively used for motionplanning. Bennewitz, Burgard, and Thrun (2002) proposed a prior-itized A⁄ algorithm to coordinate the motion of multiple robots.Here the planning was done using a simple A⁄ approach. Every ro-bot had a priority, based on which the motion of the robots werecarried out in case of possibilities of collision. The authors used asimple hill climbing approach to compute the best combinationof priorities for the overall movement of robots. Priority is a majormethod for solving collisions in a multi-robot motion planning sys-tem. Bennewitz, Burgard, and Thrun (2001) consider the problemof finding the optimal priority scheme as the given problem. Theirapproach uses a combination of hill climbing and random search toset the correct priority scheme of the robots. Every priority schemeis judged by the optimality of paths generated in terms of length.At any step random exchange of priorities takes place. If theseswaps result in better optimality, the change is retained, else re-jected. The entire process is started multiple times from differentconfigurations to escape from being trapped at local minima. Wediffer from these approaches by the fact that in our case the robotsvanish after reaching their goals, and hence do not restrict otherrobot movements. The concept of priority here is implementedby the concept of wait for robot.

3. Cooperative genetic programming

The basic working of the algorithm is a cooperative genetic pro-gramming planner. This planner operates in two levels: master andslave, similar to the architecture in Moriarty (1997), Moriarty andMiikkulainen (1997) and García-Pedrajas et al. (2003). The slavegenetic programming is basically a decentralized path planningfor all the various robots in the system. Using this level the systemtries to generate better and better paths for the individual motionof the various robots. The second level is the master level. Each ro-bot in the slave has numerous genetic programming individualover which it tries to carry forward the optimization. The masteris simply a genetic algorithm that tries to select the best combina-tion of paths for the various robots, such that the overall path of allrobots combined is optimal. In simple terms it may be regarded asthe slaves optimize the individual robot paths, and the master opti-mizes the net work plan of all robots. However it needs to be notedthat the slaves are conscious of cooperation amongst each other togenerate collision free paths, and to help each other to optimize.Section 3.1 discusses the slave genetic programming. Section 3.2presents the master genetic algorithm. The complete algorithmalong with the interaction between the master and slave levels isdone in Section 3.3.

3.1. Slave genetic programming

The slave genetic programming operates at the level of individ-ual robots for the generation of their paths, which when combinedwith the other paths of the other robots result in an overall optimalmovement of the entire pool of robots. Each robot has its own in-stance of genetic programming which comprises of multiple indi-viduals representing the potential path of movement of the robotfrom source and possibly finishing at the goal.

Based on the principles of grammatical evolution (O’Neill &Ryan, 2001, 2003), we assume that the individual representationis in form of sequence of integers. In this manner we adopt a linearrepresentation of the individual representing the path of the robot.Each integer denotes the movement of the robot. We know thatonce the robot is traveling on a straight path, it would continueto follow the same path, until it has got some point where it ispossible to make a turn. The turns can only me made at somecrossing, where multiple lanes meet each other. Whenever the ro-bot meets a crossing, it needs to decide which way it has to follow.The number of ways possible may vary depending upon the type ofcrossing. The decision to which of the path is to be followed isdecided by the individual, which uniquely determines the path ofthe robot. Before we discuss this conversion of the genotype orthe sequence of integers to the phenotype or the robotic path, letus formally define some terms.

The direction of motion of any robot Ri may be given by d = {lef-t, right,up,down} representing the four possible directions of mo-tion. We assume the various paths to criss-cross each other onlyin multiples of right angles. A robot Ri traveling in direction d ata point (x,y) is said to be at a crossing if any path criss-crossesthe path of traversal of the robot in direction d at point (x,y). Itmay be verified that the definition of cross does not depend upondirection d. Hence

Crossðx; yÞ ¼true ðx; yÞ is a crossing; i:e: a vertical and

horizontal line intersectsfalse otherwise

8><>: ð1Þ

Each of the marked positions is a crossing as shown in Fig. 1. Therobot is said to be struck at the location (x,y) while moving in direc-tion d if it is not possible to move in the same direction. This mayhappen because of reaching of map limits, end of the road, or havingsome static obstacle ahead. Hence

Individual Genotype

Initialize the pointer ci

and get corresponding initial direction of

motion

Initially place the robot at source

While robot is not at goal

Make robot move in next direction

Robot at cross or struck?

Update the pointer ci and get corresponding next

direction of motion

Pointer ci at last gene

Reset ci

Pointer ci

never reset

No Yes

No

Yes

Return computed path

End

Fig. 2. Conversion of a genetic programming genotype to robotic path phenotype.The algorithm takes an individual which is a collection or integers. The robot ismoved in its computed direction. Whenever any cross is encountered, the pointedinteger is consulted to get the next direction of motion, and the pointer is updated.The algorithm terminates if goal is found or when the individual terminates twice.


Struckðx;y;dÞ ¼true Robot can no longer move in direction

d while at ðx;yÞfalse otherwise

8><>: ð2Þ

We are further interested in the number of possible moves (alongwith the actual moves) at every crossing, which would be used by ro-bot for decision making. A robot would normally not like to traverseback into the same direction in which it was traveling since the trav-eled path would be wasteful. This however may be needed if the ro-bot was struck. Let paths possible at any crossing (Cross(x,y) = true)at location (x,y) and direction d may hence be given by D(x,y,d),where D(x,y,d) � {left, right,up,down}. Let size(D) denote the totalnumber of elements in the set or the total number of moves possible.Hence

Dðx; y;dÞ ¼ptðx; yÞ� � d struckðx; y;dÞ ¼ false

ptðx; yÞ struckðx; y;dÞ ¼ true

�ð3Þ

Here pt(x,y) is a function that looks into the possibility of all themoves {left, right,up,down} and returns the set of possible moves.�d is the direction of motion opposite to d.

Let the genetic individual for robot Ri (that is the ith instance of

genetic programming) be given by Ii Ii1; I

i2; I

i3; . . . Ii

o

D E. We need to

convert the genotype Ii into the phenotype or the robotic path.Let ci be the pointer which points towards the integers in genotypeand returns them whenever queried for decision making. We fur-ther define moveA(x,y,ci) as a function that returns the initial direc-tion of motion of the robot while robot is at position (x,y) withpointer ci. Here the robot is allowed to take any possible way. Fur-ther let moveB(x,y,d,ci) be a function that returns the next direc-tion of motion of the robot while at position (x,y) moving withdirection d with pointer ci. Here the robot is not allowed to moveback in direction it is coming from, unless it is struck. We needto first compute the initial direction of motion of the robot. We

take a pointer ci initially set to the first integer (ci ¼ Ii1). The robot

is initially located at its source P(0) = Si. For the source we selectthe possible directions of movement of the robot. This is givenby D = pt(x,y). Here (x,y) = Si. The number of possible moves are gi-ven by size(D). We select the kth move (d = Dk) in D where k is gi-ven by ci modsize(D) as the first move of the robot. Hence

moveAðx; y; ciÞ ¼ Dk;

D ¼ ptðx; yÞ;k ¼ ci mod sizeðDÞ

ð4Þ

The pointer is updated to point to the next location (ci ¼ Ii2). Now

the robot is made to move in the same direction (d) till it reachesa crossing (Cross(x,y) = true) or it gets struck (Struck(x, y,d) = true).If either of these conditions is met, the robot has to make a decisionregarding the next move. For this it first queries the total number ofmoves possible (D(x,y,d)) and then uses the integer pointed out bythe pointer ci to decide which one of these moves is to be executed.Let the move be moveB(x,y,d,ci) which is given by kth move in D(d = Dk) where k is given by ci modsize(D). Hence

moveBðx; y; ciÞ ¼ Dk;

D ¼ Dðx; y; dÞ;k ¼ ci mod sizeðDÞ

ð5Þ

The pointer is further incremented to point to next integer. In thismechanism the algorithm keeps iterating with decision being madeat every crossing and when the robot is struck.

The motion of the robot may stop in either of two possible ways,either the robot reaches its goal, or the genotype is short of integersthat is crossing after ci ¼ Ii

n is to be processed. In case the stop isdue to the termination of integer sequence, we reset the pointer(ci ¼ Ii

1) and allow the algorithm to proceed. This reset may

however take place only once in the entire motion of the robot.In this manner the genotype may be finally converted into the phe-notype. The complete algorithm is summarized in Fig. 2.

We study the conversion by an example. Let the map be asgiven in Fig. 3. The start position and goal of the robot is markedas shown in figure. Let the genotype be given by h2,5,3,5,2,1,3,3,3,4,5,2,2i. Initially the pointer is at 2. The possible movesare {down}. The algorithm selects the 2 mod 1, i.e. the 0th movewhich is down (note that ordering starts from 0 and not 1). The ro-bot marches down, until it reaches cross A. Here possible moves are{right,down}. The pointer is at 5. The robot selects 5 mod 2, i.e. 1stmove, which comes out to be down. The robot marches down tillcross B. Here the possible move is {right}. Pointer is at 3. The moveselected is 3mod1, i.e. right (0th move). The robot comes to pointC. Now possible moves are {right,down}. Pointer is at 2. The moveselected is 2mod2, i.e. right (0th move). Robot moves till cross D.Here possible moves are {up}, which is selected by pointer at1.The robot travels up and reached cross E for which the possiblemove is only right. This is selected by pointer at 3. This makesthe robot reach cross F with possible moves {right,down}. Pointeris at 2. The selected move is 3mod2, i.e. the first move or down. Ro-bot moves in the same direction until it reaches cross G. The rest ofthe motion may be verified accordingly. At the end the robot willterminate its journey at goal, with some genes to spare.

The next important task to be carried out in the implementationof the slave genetic programming is the genetic operators. We usetwo genetic operators for the algorithm, crossover and mutation.Crossover is applied to the top elite individuals. The number of

A

B C D

E F Source

G

Goal

H I

Fig. 3. Sample map for conversion of genotype to phenotype. Figure is a samplemap that shows the path used by a randomly generated robot for motion fromsource. The path traversed comes out to be Source ? A ? B ? C ? D ? E ? F ?G ? H ? I ? Goal.

Genetic Programming for Robot 1



Genetic Programming for Robot n

Slave Genetic Programming Individuals

Master Genetic Algorithm

Fig. 4. Individual representation of master genetic algorithm. Every individual ofthe master genetic algorithm points to an individual of the slave geneticprogramming. Figure represents one individual of the master genetic algorithm.


individuals that undergo crossover is given by cross � popSize,where cross is the crossover rate and popSize is the total numberof individuals in the population pool. These individuals are pairedin groups of 2 and crossover is applied to each paid. Scatteredcrossover technique is used for the generation of two children fromevery pair of parents. The generated children replace the two poor-est fitness individuals in the population pool. The mutation opera-tor is applied to rest of the individuals in the population pool. Allthese left individuals undergo change in their genes, where thechange is given by a uniform mutation. Here the gene value ischanged to a random value with a probability given by the muta-tion rate.

The last part left in the implementation of the slave genetic pro-gramming is to have a fitness evaluation technique. We have al-ready seen the mechanism of conversion of a genotype or aninteger sequence into its phenotype or the robotic path. The fitnessfor any individual is measured by two factors denoting the individ-ual and cooperative aspects of the overall path planning. The firstfactor for robot Ri (Fi

1) measures the fitness of the individual path.Using this measure, an individual tries to generate shorter andshorter paths which may ultimately contribute towards the overalloptimality of all the robots. This fitness is measured by the totallength of the path. In case the robot did not reach the goal, a pen-alty is added proportional to the distance between the last pointtouched by the robot and the actual goal location. This may be gi-ven by

Fi1 ¼ li þ akPi � Gik ð6Þ

Here li is the total length of path, Gi is the goal and Pi is the last pointtouched by the robot. a is the penalty constant canned as the pen-alty constant for non-reach ability of goal.

By optimizing based on this objective it is highly likely thatindividual robots disregard the cooperation factor which is a majorfactor in co-evolution and enables different modules to developcharacteristics to contribute well to the overall problem. This fac-tor is served by the other factor of the fitness function (Fi

2). In thisfactor we look into the possible collisions between the robot withother robot, or the factors by which this path results in sub-optimalpaths of other robots. The path of this robot is a part of multiplepaths of the other robots. These paths are generated by the masteras we shall see later. We select the best e individuals of the master.Each of these individuals specifies the paths of each of the robots.We simply replace the path corresponding to robot Ri in all these e

individuals by the current individual. We note the difference in thefitness value, which is a measure of Fi

2. This may be hence given by

Fi2 ¼

Xe

j¼1

ðFitðXj;Ri!Ii Þ � FitðXjÞÞ ð7Þ

Here Xj is the jth best individual of the master. Xj;Ri!Ii denotes bestjth individual of master with the path of robot Ri replaced by thecurrent genotype Ii. The net fitness may be given by

Fi ¼ Fi1 þ Fi

2 ð8Þ

3.2. Master genetic algorithm

While the slave genetic programming does the task of genera-tion of good paths for the individual robot, which have already ta-ken the factor of cooperation into them, the task left is theevolution of the overall working strategy of the entire algorithm.This task is done by the master genetic algorithm, which decideshow each and every robot is to move.

The first task is the individual representation of the master ge-netic algorithm. The individual is a set of n pointers, each pointingto some individual of the slave genetic programming. Let thesepointers or the genetic algorithm individual be given by hJ1, J2, J3,. . ., Jni. Here any pointer Ji points to one of the individuals of theith slave genetic programming (Ii) that represents the path of theith robot. This may be easily understood from Fig. 4.

The next task is the application of the genetic operators for theindividual. Similar to the slave genetic programming, we use thegenetic operators of crossover and mutation in this technique.The crossover mixes two parent individuals and generates twochildren. These two children replace the two weakest individualsof the genetic algorithm. A scattered crossover technique is fol-lowed in which every child randomly takes half of the pointers

For all instances of genetic programming, initialize their individuals

Initialize master genetic algorithm individuals

While stopping criterion is not met

For all instances of slave genetic programming

Selection

Crossover

Mutation

Selection in Genetic Algorithm

Crossover in Genetic Algorithm

Mutation in Genetic Algorithm

Fitness Evaluation in Genetic Algorithm

Fitness Evaluation

End

Return Best individual

End

Fig. 5. The basic evolutionary algorithm.

VB

RA

RB

Goal

Initial Locations

VA

Long Corridor

A

(x, y)

(x’, y’) (x’’, y’’)

Fig. 6. The problems with planning with multiple speeds. The figure shows tworobots RA and RB moving with velocities VA and VB (VA� VB). If RB enters the corridorfirst, RA will have to follow for the rest part of the journey with reduced speeds.Optimal path has RA entering before RB.


from first parent and the other half from the other parent. Themutation operator simply changes the genes of the individual.The gene is simply a pointer to some individual of the slave geneticprogramming. The mutation deletes an existing pointer and makesit point to some other individual in the genetic programming. Pref-erence may be given to the newly created individuals of the geneticprogramming.

The last task to be carried out is the fitness evaluation. The fit-ness evaluation of the master genetic algorithm simply returns theaverage traveling time of each of the robots. The traveling time ofthe robot is measured by the time that the robot takes to reach itsgoal. However the master genetic algorithm needs to ensure thatall the paths are traversed without any kind of collision betweenthe robots. For this a simultaneous simulation of all robots is done,knowing that the various robots have different speeds. It is furtherpossible that some of the robots do not reach their goal. For boththese aspects a penalty is added to the fitness function. The resul-tant fitness function of the master genetic algorithm hencebecomes

F ¼Xn

i¼1

Ti þ akPi � Gik !

þ b � C ð9Þ

Here Ti is the time the robot Ri reaches its goal Gi, Pi is the last pointtouched by the robot in its journey, a and b are penalty constants(b > a), C is the time first collision occurs (0 is no collision occurs).a is called as the penalty constant for non-reach ability of goal. bis called as the penalty constant for collision.

The penalty constant for collision (b) is usually kept higher thanthe penalty constant for non-reach ability of goal (a). This is toencourage collision free movements. It would be better not to havecollisions that not to have a robot reach its goal. The path may befeasible only when both these factors are zero, but characteristicpositions of robots in the map may result in scenarios where it isnot possible to have all the robots have a collision free journey totheir goals.

3.3. Algorithm outline

We have already discussed the individual master and slave evo-lutionary algorithms. In this section we briefly give a combinedpicture of these two algorithms. The complete algorithm is pre-sented in Fig. 5. It may be easily seen that the algorithm is iterativein nature, where both the evolutionary algorithms run in parallel.They hence share a common stopping criterion. A unit iterationof all the evolutionary algorithms is executed in parallel. This givesall of these a chance to evolve their population as per the changingscenarios and findings. It may be noted that all the various evolu-tionary algorithms do not run in isolation, but are rather depen-dent on each other. A boost in the fitness value of the mastergenetic algorithm by selection of some individual from some slavegenetic programming may mean the slave genetic programmingbeing re-directed to produce different kinds of individuals. Simi-larly if some new individual produced by one of the slave geneticprogramming results in improvement of the overall path beingcontrolled by the master genetic programming, it may redirectthe master to produce more individuals of the same kind. We havealready discussed about the factor of cooperation between the var-ious slave genetic algorithms.

4. Waiting for robot

The algorithm framework presented in Section 3 is self-suffi-cient to carry planning of multiple robots, but it fails on a numberof scenarios for which we introduce the concept of waiting for ro-bot. Consider the case where a slower robot is following a higher

speed robot. This scenario was discussed earlier and was presentedin Fig. 6 to ease the rest of the discussion. In this scenario the pathof the two robots are shown. It would be better if the slower robotwould have waited at point A for the faster robot to move. In such acase the slower robot would have followed the faster robot whichnaturally means a better moving strategy minimizing the collec-tive motion time of all the robots. In reality with multiple criss-crosses and multiple robots there can be complex scenarios whereone robot waiting for another robot may be fruitful.

We informally state that wait may only be performed at thecrossing (Cross(x,y) = true), more precisely at a unit step fromcrossing, since we are to leave the crossing free for the moving ro-bot. Waiting at any other location other than crossing may beequivalent to waiting at some crossing. Further the wait must al-ways end with some robot crossing the point of crossing for which


the robot is waiting. It is natural that there is no use waiting morethan that, unless the robot is actually waiting for another robotafter which it plans to move. This would be equivalent to the robotwaiting for the second crossing robot with no necessary relationwith the first crossing robot. It may be hence noted that the con-cept of waiting for robot at crossing is important and may furtherboost the performance of the model presented in Section 3. Wehence formally define the concept and modify the algorithm pre-sented in Section 3 as per its requirements.

We defined paths possible at any crossing (Cross(x,y) = true) atlocation (x,y) and direction d be given by D(x,y,d), whereDðx; y; dÞ � fleft; right;up;downg. Now the robot may be addition-ally asked to wait at some crossing and hence the option is addedto D(x,y,d). The modified value is given by Dmod(x,y,d) =D(x,y,d) [ {wait}. Here wait signifies that the robot is being askedto wait for a robot. size(Dmod) denotes the total number of ele-ments in the set or the total number of moves possible. Considerthe genotype pointer is pointing towards any location ci at any in-stance of time. Now the move made by the robot moveB(x,y,d,ci) isgiven by

moveB ðx; y; ciÞ ¼ Dk;

D ¼ Dmodðx; y;dÞ;k ¼ ci mod sizeðDmodÞ

ð10Þ

Similarly the first move of the robot is modified to work over Dmod

in place of D. The restriction however here is that the first movecannot be a wait. This is because it is not necessary that the robotis initially located at a crossing, which is a requirement for the waitmove. The first move moveA(x,y,cj) is hence given by

moveA ðx; y; ciÞ ¼ Dk;

D ¼ ptðx; yÞ;k ¼ ci mod size ðDmodÞmoveA ðx; y; cjÞ ¼ fwaitg 8j < i:

ð11Þ

More precisely the robot has to wait just before the crossing and noton the crossing. In case it waits on the crossing, it would prohibitany robot to pass by including the robot for which it is waiting topass by. Hence the decision towards whether the robot has to waitor not needs to be made before the robot enters into crossing. Thismay not be the immediately preceding step of crossing (as robotshave fractional velocities) but the step after which the robot entersanywhere within the crossing region. We define the penultimateposition of the robot (x,y) such that any step further from this indirection d would result in the robot entering into the crossing area.Let this be given by pen(x,y,d). Now as soon as any robot enters intopen(x,y,d), it queries the next crossing Cross(x0,y0,d) and gets themove which it would be expected to make moveB(x0,y0,d). Here(x0,y0) is the location of the next crossing (in this query the pointerci is not altered). In case this move comes to be wait (or move-B(x0,y0,d) = {wait}), the robot does not move and waits at (x,y) andthe pointer ci is incremented (or the wait move is already executed).Note that after the robot is allowed to move, it would reach thecrossing and the next move would be made as per the new integerpointed by ci. In case the move does not come to be wait, the robotcontinues its motion.

Further when a robot is waiting, it must always wait for somerobot that is guaranteed to cross from the crossing of wait. Thisensures that we know exactly about when the robot may be al-lowed to resume its journey. Let (x,y) be the location of a robotRi penultimate to crossing. Let (x0,y0) be the location of the crossing.Further let (x00,y00) be the position next to crossing (outside thecrossing area) where Ri would be moving after wait is over. Sincewe know the complete genotype of motion of Ri, we can easilycompute its entire path, excluding the waits which are dependent

on the movement of the other robot and cannot be computed inisolation. We search for robots Rj (j – i) which are presently movingand would be found at location (x00,y00) at some instance of timeahead but not after Ri reaches its goal if allowed to move withoutwaits. Again it be noted that complete path of Rj is already knownexcluding the waits. If we get some Rj such that Rj is moving and Rj

crosses (x00,y00) at some time ahead but not after Ri reaches its goal ifallowed to move without waits, we state that the robot Ri will waitfor Rj (Ri ? Rj). The current state of Ri is labeled as waiting, and all itis not allowed to move further. Only after Rj crosses location (x00,y00),the robot Ri is allowed to move. In case no Rj (j – i) satisfies the cri-terion, the wait of Ri is cancelled and its normal motion continues.

It is important to put the check that Rj should be moving to es-cape from deadlock. If this condition is not checked it would bepossible to have a situation Ri ? Rj and Rj ? Ri, in which case bothRi and Rj are not moving and waiting for each other. More generallyit would be possible to have a scenario Ra ? Rb ? Rc, ? Rd, . . . ? Ra,which is a deadlock.

In this manner we may be able to model multiple deadlock freescenarios with multiple robots waiting in any complex or simplemanner. Hence by this introduction of wait in the problem model-ing, we are able to generate more flexible movement strategy ofthe multiple robots.

5. Memory based local search

The model of the algorithm discussed in Section 4 may take areasonably long time to carry out the task of optimization. It is acommon technique to assist the evolutionary process by some heu-ristic means. In this algorithm we do this by storing a lookup tableat a dedicated memory and use the same for optimization. The pur-pose of the lookup table is to store information about shortestpaths between crosses. In case the path of any of the robots hap-pens to be larger than the path stored in this lookup table, wemay easily replace it by the smallest path already known. In sucha case the robot would reach the goal in a much smaller pathand if the modified path has no collisions, the overall optimalityof the solution may be enhanced. The lookup table enables the ro-bot to quickly attain the optimal path, while it may be reasonablyaway from the same.

Another use of the lookup table is to enable cooperationamongst the robots. If one robot finds an optimal path betweenany pair of crosses, it must be able to share the same with the otherrobots, to enable them benefit from the finding. By making a look-up table if a robot finds the smallest path between any pair ofcrosses (as per the current exploration), it stores it into the lookuptable which may be accessed by all the other robots if they need togo between the same pair of crosses in their journey.

The lookup table or memory is supposed to store smallest pathsbetween crosses. Suppose that there are v crosses (Cross(x,y) = -true) in the system. We select a total of g (g 6 v) crosses randomlyfrom the available v crosses that participate in the lookup table. gdecides the size of the lookup table. The lookup table is a datastructure that stores the smallest path between any of thesecrosses, where each cross is defined by the robot position (x,y)and direction d. Let the selected crosses be (V1,V2, . . .,Vg). The look-up table stores the minimal path length Val(Vi,Vj) between fromcross Vi to cross Vj. It further stores the actual set of genotype inte-gers that result in the generation of smallest path P(Vi,Vj) fromcross Vi to cross Vj.

One of the tasks associated is the initialization and update of thelookup table. Initially we set

ValðVi;VjÞ ¼infinity Vi – Vj

0 Vi ¼ Vj

�ð12Þ

Table 1Summary of situation and results for simulation with 2 robots.

S. No. Factor Robot 1 Robot 2

1 Source (0,2) (24,23)2 Goal (24,23) (0,2)3 Speed 0.5 0.34 Reached Goal Yes Yes5 Collision No No6 Time 142 3837 Path Length 66 115

Fig. 7a. Path traced by robot 1 for simulation with 2 robots.


PðVi;VjÞ ¼ NIL ð13Þ

As we proceed with the algorithm, we get multiple paths on con-verting the genetic individual genotypes to corresponding pheno-type paths. For the converted paths we check all pairs of crossesfor possibility of shorter paths. Hence

ValðVi;VjÞ ¼minflength P0Vi!Vj

� �; ValoldðVi;VjÞ8Vi

and Vj in generated path P; Vj comes after Vi ð14Þ

Here Valold(Vi,Vj) is the previous value and Val(Vi,Vj) is the updated

value of the data structure. length P0Vi!Vj

� �is the length of path be-

tween Vi and Vj in P0. Similarly the path needs to be modified. Hencewe get

PðVi;VjÞ ¼P0Vi!Vj

length P0Vi!Vj

� �< ValoldðVi;VjÞ

no change otherwise

(ð15Þ

In this manner the lookup table is ready and keeps getting updatedas newer paths are found. The last part left is to use the table as thelocal search of the individuals. For any path of the slave genetic pro-gramming we apply an additional operator reduce that attempts toshorten the path of the robot with a probability red. If the probabil-ity red is very low, there is almost no replacement and the factor oflocal search is almost lost. In case the factor is very high, it is likelythat the individual slave genetic programming algorithms keepdeveloping characteristics as per the characteristics in the lookuptable, and the overall algorithm might get converged at some localminima.

For any path in its genotype form which needs to be reduced,we randomly try to select pairs (Vi,Vj) in the path P0 of the robot,such that P0Vi!Vj > ValðVi;VjÞ. For this path we replace the genotypeintegers from Vi to Vj by P(Vi,Vj). Suitable genes may be added atthe two ends of the path. The previous path P0 (S ? Vi�1 ? Vi ?Vj ? Vj+1 ? G) hence becomes (S ? Vi�1 ? P(Vi,Vj) ? Vj+1 ? G)where S is the source and G is the last pointed touched by therobot. If no Vi, Vj satisfy the criterion, that is the path is optimizedas per the lookup table, then no actions take place by this operator.The complete operation may be repeated over a number of times,as it is likely that the generated path may further be reduced bythe lookup table.

In this manner the application of this additional operator to thegenetic programming individual may result in an enhanced opti-mization to the problem, considering the reasonably complex nat-ure of the problem. This plays a major role in giving good resultsearly.

6. Results

The algorithm was tested on a simulation tool built in JAVA. Thecomplete algorithm was coded in multiple modules which in-cluded the master genetic algorithm, slave genetic programming,and the operations over the individuals of these two evolutionarytechniques. A simulation tool used the genotype of the evolution-ary technique to simulate the complete system and generate thenecessary phenotype paths. This was done separately for boththe master and the slave evolutionary techniques. A JAVA Appletbased GUI client was built that used multi-threading to show theoptimal motion strategy of all the robots, as calculated by the algo-rithm. The maze-like map was given to the algorithm in a form of aBMP image. The paint utility was used for the generation of themap.

Maze-like maps were used for all the testing experiments. Therobots could only move in four directions. Hence in these maps itis not necessary that the optimal path lies close to the straight linejoining the source and the goal, which would have been the case

for robots which can move in multiple directions. If the motion isto be made from one corner of a square to its diagonally oppositecorner, in these maps it makes no difference whether the journeyis made close to the diagonal joining the corners, or using the ver-tical and horizontal edges. The distance of travel (which wouldcome out to be equivalent to the Manhattan distance) would bethe same. For the same reasons the map used for testing purposeshad a number of turns such that the final optimal path betweenany source and goal is large and complex enough.

The algorithm was first simulated for 2 robots. The intentionwas to check the path optimality of both these robots, as well astheir cooperation to figure out a collision-free path. In order to en-sure a complex overall path, the two robots were given diagonallyopposite source and goals. Further to make collisions move likely,the source of the first robot was made the goal of the other, andvice versa. The first robot was initially located at (0,2) and had toreach a goal (24,23). The other robot was initially placed at(24,23) and had to reach a goal of (0,2). The speed of the two ro-bots was 0.5 and 0.3. The simulation was carried out for a totalof 20 iterations. The master genetic algorithm had a populationsize of 400. The mutation rate was fixed to 0.1. The top 40% individ-uals were used for the crossover operations. The rest individualswere generated by mutation operation. The genetic programminghad a population size of 250 individuals. The mutation rate was0.1. Here the top 20% individuals were used for crossover, the restbeing generated by mutation. The individual had a size of 25 realnumbers or genes. The penalty constant for not reaching goalwas 100, and the penalty constant for collision was 1000. The algo-rithm, upon simulation generated the paths, which were displayed.The parameters and results are summarized in Table 1. The motionof the robots along with time may be seen in video 1. The generalmotion of the robots is also given in Fig. 7 for ease of discussion.

Fig. 7b. Path traced by robot 2 for simulation with 2 robots. Fig. 8a. Path traced by robot 1 for simulation with 3 robots.

Fig. 8b. Path traced by robot 2 for simulation with 3 robots.


It can be easily seen that as per the set locations, the collisionsbetween the robots was natural. In this case the robots made use ofthe wait feature and hence first robot waited for the second robotat point A (shown in Fig. 7). This enabled the two robots to move totheir own goals. It may again be easily seen that wait can only beapplied when the robots do not go in opposite directions. Supposea robot is waiting at a cross, on the way where the other robotwants to go to. It is natural that there would be a collision, sincethe moving robot would collide with the waiting robot. Furtherwe observe that the speeds of the robots play a big role in decidingthe path. The paths for same set of parameters would turn out to bedifferent if the velocities are changed. The velocities decide thepossible points of collision, to which the colliding robots cooperateto find a collision free strategy.

The other execution was carried over the same map. To test thealgorithm performance in higher number of robots, we added an-other robot to the simulation. So there were a total of 3 robots thattried to find their goal from the starting initial positions. The evo-lutionary parameters were kept same as the previous run. Thesource of the first robot was (0,2), while its goal was (24,23) andits speed was 0.4. The second robot had the source as (24,23), goalas (0,2), and speed 0.7. The third robot had source (23,0), goal(1,24) and a speed 0.8. The simulation was carried out and the re-sults were displayed using the GUI tool. The parameters and resultsare summarized in Table 2. The motion of all the robots is given invideo 2. The results are further drawn in summarized form inFig. 8.

Here as well we see that every robot tried to simultaneously ca-ter to two needs, to find an optimal path, and to find a collision freepath. It is evident that there are limited options or paths, and hencean optimal path may not be collision free and vice versa. An overallsolution is hence difficult to obtain. It may further be seen thatspeed is a major factor. Consider the point A as shown in Fig. 9. If

Fig. 8c. Path traced by robot 3 for simulation with 3 robots.


S. No. Factor Robot 1 Robot 2 Robot 3

1 Source (0,2) (24,23) (23,0)2 Goal (24,23) (0,2) (1,24)3 Speed 0.4 0.7 0.84 Reached goal Yes Yes Yes5 Collision No No No6 Time 166 167 717 Path length 66 117 57




Fig. 9d. Path traced by robot 4 for simulation with 4 robots.


robot 1 traveling from top left corner was traveling faster, it wouldhave resulted in some collision with robot 2 traveling from bottomright corner. This would have resulted in either of the robot wait-ing, or both of them force to recalculate their paths.

We further test the scalability of the algorithm with more ro-bots. The next simulation involved 4 robots. The initial locationsof the four robots were (0,2), (24,21), (23,0), and (1,24). The goalswere specified as (10,24), (23,0), (1,24), and (24,21). The speedswere set to be 0.6, 0.8, 0.7, and 0.8. The evolutionary parameterswere kept same as the previous run. The result in this case is givenin video 3, and the same is summarized in Fig. 9. The parametersand result statistics is given in Table 3.

It may be seen that though the modeling scenario was reason-ably complex with multiple robots that were supposed to find theirway out to the goal, the algorithm was able to generate these pathsfor all robots. All the robots could reach their goals in a reasonablyoptimal path. It may be possible for better paths to exist or theindividual robots, but this may lead to collisions and hence needsto be avoided. The factor of optimization is the average runningtime of the robots, which is optimal in this case. It may be empha-sized that every addition in robot means an immense increase inpossibilities at the coordination level. Two robots may only needto ensure that they somehow do not collide, but multiple robotscan interact in plentiful ways. We may avoid collision betweentwo robots by some change in their paths, but the changed pathsmight result in more collisions with any of the other robots. Hencea very complex interaction amongst robots makes the problemdifficult.

The last experiment was done using 5 robots. The locations andthe speeds of the robots were changed. The source of first robotwas (2,0) and goal was (10,24). The speed of this robot was 0.6.The second robot had a source of (24,21) and goal of (23,0). Thespeed of the robot was 0.8. The third robot was at (24,3) and


S. No. Factor Robot 1 Robot 2 Robot 3 Robot 4

1 Source (0,2) (24,21) (23,0) (1,24)2 Goal (10,24) (23,0) (1,24) (24,21)3 Speed 0.6 0.8 0.7 0.84 Reached goal Yes Yes Yes Yes5 Collision No No No No6 Time 112 57 81 887 Path length 67 46 57 71



was supposed to move to (0,19). The speed was kept as 0.7. Thefourth robot had a source (0,19) and goal (24,21). The speed was0.8. The fifth and the last robot was had a source (11,0), goal(0,14), and speed 0.2. The simulation was carried out and the finalresults were recorded. The movement of all the robots is shown invideo 4. Fig. 10 is the equivalent figure for the same simulation.The parameters are results are given in Table 4.

It may again be verified that despite heavy complexity, the algo-rithm could figure out a collision-free motion strategy for the mo-tion of the robots. It needs to be again noted that motion of a robotby a path completely blocks the path. For the entire duration thatthe robot occupies that path, no robot would be able to move inany direction without collision. Hence the number of alternatepaths plays a major role in the system. Too many robots with lessnumber of alternative path might mean no path may be possiblefor some or the other robot that reaches the goal without collision.Either the robot may not reach goal at all, or it may reach the goalwith collision. In some cases the robot might have to take too manyunnecessary (and in some cases redundant) paths before movingon a path that takes it to goal. Hence there is a limitation of themap to the maximum number of robots that can move in. Forthe same reasons we do not experiment with larger number of



Fig. 10d. Path traced by robot 4 for simulation with 5 robots.

Fig. 10e. Path traced by robot 4 for simulation with 5 robots.

Fig. 12a. Path traced by robot 1 for simulation with 4 robots without memorybased optimization.

Fig. 12b. Path traced by robot 2 for simulation with 4 robots without memorybased optimization.


S. No. Factor Robot 1 Robot 2 Robot 3 Robot 4 Robot 5

1. Source (2,0) (24,21) (24,3) (0,19) (11,0)2. Goal (10,24) (23,0) (0,19) (24,21) (0,14)3. Speed 0.6 0.8 0.7 0.8 0.24. Reached goal Yes Yes Yes Yes Yes5. Collision No No No No No6. Time 115 69 81 83 1497. Path length 69 56 57 67 30


robots. Hence we see that multiple robots may be optimally movedinto the map.

The next scenario we generate is a very simple one. Here wemake two robots move in opposite directions, so as to make themreach a common goal. In this scenario we aim to test the working ofthe wait feature of the algorithm. It is evident that it is not possibleto move the robots in a manner that collision can be avoided with-out any robot waiting. The map has one crossing and one of the ro-bots needs to wait so that the motion may terminate withoutcollision. The simulation for this case is given in video 5 andFig. 11. It may be seen that one of the robot waits for the other. Thisagain proves the usefulness of the wait feature of the algorithm.

Apart from the designed algorithm and the wait feature, theother module we designed was the local optimizations using alookup table. We stated that this lookup table was important fora robot to use the findings of the previous generations, as well asthe other robots. This enables swift motion towards the optima.We executed the same scenario with 4 robots without the lookuptable. We observed that the results were highly sub-optimal. Manytimes robots were struck at some loops, where they returned backat a point they had visited earlier. The results with the execution ofthe program without lookup table are given in Table 4. The paths ofthe various robots may be seen in Fig. 12. Video 6 shows the mo-tion of the robots. This shows that the module of lookup tablewas effective and played a vital role in making the algorithm.

We further study the execution time of the algorithm for the in-crease in number of robots. Based on our understanding of thealgorithm it may be easily seen that the execution time should in-crease with increasing number of robots. This is because of the factthat for every increase in robot, a genetic programming instance iscreated and executed. Also the simulation plays a keen role indeciding the execution time. If the path to goal for all the robotsis very simple, they would all attain their goals and the complete

Fig. 11. Path traced by two robots using wait for robot feature.

Fig. 12c. Path traced by robot 3 for simulation with 4 robots without memorybased optimization.

Fig. 12d. Path traced by robot 4 for simulation with 4 robots without memorybased optimization.

Fig. 14. Execution time vs generations.


simulation would stop. On the other hand if there is a collision inthe path of the robot, or it does not reach the goal till the end, thereis a lot of computation that is performed. This is because for a verylong time (until the genomic length is completely iterated) some orthe other robot keeps wandering at the map. The increase in num-ber of robots increases the possibility of collisions to a great extentand hence multiple paths need to be generated and tried. All thisconsumes a lot of time and execution time further increases. Forthe same reasons simulations having slower moving robots wouldtake more computation time. We take the same map as discussedin the above discussions. We plot the execution time for the vari-ous numbers of robots. This is shown in Fig. 13. It can be easilyseen that there is more than linear increase in computation timefor increase in robots. This is because of the cooperation factor asdiscussed.

We further extend the analysis to the stud y of the optimizationof a single scenario. We study how the optimization time changesalong with generations. Every generation has a different executiontime. We may infer that the execution time depends upon the pathof the robots. Every individual of both the genetic programmingand genetic algorithm represents paths which need to be simu-lated multiple times. Longer paths would mean longer simulationtime and hence the total time of execution of that generationwould increase. As we move along with generations, paths startgetting close to optimal solutions. The lengths of the paths reduce.

Fig. 13. Execution time vs number of robots.

More robots start to reach their goal, which means less wanderingin the maps. Hence in general the execution time for higher gener-ations is smaller than the execution time for the smaller genera-tions. However the evolutionary operators may sometimes resultin too fit or too unfit individuals. In case of the former, there is asharp decrease in execution time, while the latter results in a sharpincrease of the same. The sharp increase or decrease may be aver-aged out to some degree by the other individuals in the populationpool. Fig. 14 shows the graph for the execution time for variousgenerations. The general trend has been plotted as a dashed trendline. The simulation of 4 robots was used.

7. Conclusions

In this paper we attempted to solve the problem of multi-robotmotion planning. The modeling scenario had a maze-like mapwhere the different robots were initially located at distinct placesand were given their own goals that they were supposed to reach.We further assumed that each robot moves with its own speed.The algorithm framework made use of co-evolutionary geneticprogramming. The task of planning was performed at two levels.At the first level a linear representation of genetic programmingwas used. The individual in this case consisted of instructions formovement whenever a cross is encountered. There was a differentinstance for every robot. The other level consisted of a genetic algo-rithm instance. This algorithm selected the individuals from thegenetic programming and tried to generate a combination suchthat the overall path of all the robots combined is optimal. An indi-vidual of this level had pointers pointing out to genetic program-ming instances. The resultant algorithm could solve the proposedproblem, but there were a number of scenarios that it could notsolve. For this we included the feature of wait for robot, where arobot may be asked to wait before some crossing till a robot passedthe same. In this manner robots were capable of escaping from rea-sonably complex scenarios in a cooperative manner so as to reachtheir own goals. Many collision prone scenarios could hence becontrolled, making collision free planning possible. We furthersaw that this feature resulted in generation of optimal paths. Thisalgorithm also solved the purpose, but the resulting problem be-came highly complex. This necessitated the need of some localsearch strategy, which we included into the algorithm in form ofa lookup table. Here, as the algorithm proceeds, optimal paths be-tween any two crossings are monitored and stored. The path of anyindividual may hence be modified, and it may be made to follow bythe pre-computed optimal path, rather than the path as per itsgenes. This feature hence made it possible to generate optimalpaths reasonably early in highly complex scenarios.

The algorithm was developed as a JAVA software, which couldread BMP images as maps. The various parameters of the evolu-tionary algorithm as well as the problem specifications could be


defined as well. A number of executions of the algorithm were car-ried out with different number of robots. In all scenarios we sawthat the algorithm could figure out a collision free path for all ro-bots from the source to the goal. The paths seemed optimal inlength and travel time. The different runs were done with differentpositions of the robots. This ensures that the algorithm generatesoptimal results to different modeling scenarios. We further didexperimentation of the algorithm with the same scenarios withoutthe memory based optimization. It was seen that the algorithmcould not perform well. The resultant paths were not optimal. Fur-ther the effectiveness of the wait for robot feature was experi-mented by a simple scenario, and the resultant path had a robotto wait for the other robot. The experimental results prove thatthe algorithm could effectively solve the proposed problem asper the mentioned modeling scenarios in a mix of easy to complexscenarios.

In all the cases presented, it was a common observation that thespeed of the various robots played a big role in deciding theirpaths. The different speeds meant different points of likely colli-sion, for which some alternative strategy had to be built. Furtherit was observed that the later generations of the algorithm werea lot quicker. This was due to the optimized nature of the individ-uals at those times. The increase in the number of robots impliedan increase in the complexity which increased the execution time.

The presented approach however has some limitations as well.The biggest limitation is the optimization time of the algorithm.This algorithm may hence not be usable for many real time scenar-ios. It may also be possible that the real world scenario may havedynamic maps, which the present approach does not consider.The other limitation of the algorithm is that the modeling scenariogets very complicated for the addition of large number of robots. Inlust to have an optimal overall path with a central planning tech-nique, we put a restriction on the maximum possible number ofrobots that may be computed in real time. In real world it maybe possible to move any number of robots in some complex mech-anism. This approach may not be able to evolve the same. The addi-tion of some heuristics to the planning algorithm may help tofurther increase its scalability to such scenarios. These heuristicsmay further help to get the execution time of the algorithm down.All these problems may be worked over into the future. Further anexhaustive testing of the algorithm with different models, maps,and robot locations and velocities may be done. This would clearlyspeak out the positive and negative aspects of the algorithm.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, inthe online version, at doi:10.1016/j.eswa.2011.09.090.

References

Arai, T., & Ota, J. (1992). Motion planning of multiple mobile robots. In Proceedings ofthe 1992 IEEE/RSJ international conference on intelligent robots and systems (pp.1761–1768).

Asama, H., Matsumoto, A., & Ishida, Y. (1989). Design of an autonomous anddistributed robot system. In ACTRESS, IEEE/RSJ international workshop onintelligent robots and systems, Tsukuba, Japan (pp. 283–290).

Asama, H., Ozaki, K., Itakura, H., Matsumoto, A., Ishida, Y., & Endo, I. (1991). Collisionavoidance among multiple mobile robots based on rules and communication. InIEEE/RSJ international workshop on intelligent robots and systems, Osaka, Japan(pp. 1215–1220).

Bennewitz, M., Burgard, W., & Thrun, S. (2002). Finding and optimizing solvablepriority schemes for decoupled path planning techniques for teams of mobilerobots. Robotics and Autonomous Systems, 41, 89–99.

Bennewitz, M., Burgard, W., & Thrun, S. (2001). Optimizing schedules for prioritizedpath planning of multi-robot systems. In Proceedings 2001 IEEE internationalconference on robotics and automation (pp. 271–276).

Bohlin, R., & Kavralki, L. E. (2000). Path planning using lazy PRM. In Proceedings ofthe IEEE international conference on robotics and automation, San Francisco, CA(pp. 521–528).

Carpin, S., & Pagello, E. (2009). An experimental study of distributed robotcoordination. Robotics and Autonomous Systems, 57, 129–133.

Clark, C. M. (2005). Probabilistic road map sampling strategies for multi-robotmotion planning. Robotics and Autonomous Systems, 53, 244–264.

Clark, C. M., Rock, S. M., & Latombe, J. C. (2003). Dynamic networks for motionplanning in multi-robot space systems. In International symposium of artificialintelligence, robotics and automation in space.

Dongyong, Y., Jinyin, C., Matsumoto, N., & Yamane, Y. (2006). Multi-robot pathplanning based on cooperative co-evolution and adaptive CGA. In Proceedings ofthe IEEE/WIC/ACM international conference on intelligent agent technology, HongKong (pp. 547–550).

García-Pedrajas, N., Hervás-Martínez, C., & Muñoz Pérez, J. (2003). COVNET: acooperative coevolutionary model for evolving artificial neural networks. IEEETransactions on Neural Networks, 14(3), 575–596.

Kala, R., Shukla, A., & Tiwari, R. (2009) Comparative analysis of intelligent hybridsystems for detection of PIMA Indian diabetes. In Proceedings of the IEEE 2009world congress on nature and biologically inspired computing, Coimbatore, India(pp. 947–952).

Kala, R., Shukla, A., & Tiwari, R. (2010a). Clustering based hierarchical geneticalgorithm for complex fitness landscapes. International Journal of IntelligentSystems Technologies and Applications, 9(2), 185–205.

Kala, R., Shukla, A., & Tiwari, R. (2010b). Dynamic environment robot path planningusing hierarchical evolutionary algorithms. Cybernetics and Systems, 41(6),435–454.

Kala, R., Shukla, A., & Tiwari, R. (2010c). Fusion of probabilistic A⁄ algorithm andfuzzy inference system for robotic path planning. Artificial Intelligence Review,33(4), 275–306.

Kala, R., Shukla, A., & Tiwari, R. (2010d). Evolving robotic path with geneticallyoptimized fuzzy planner. International Journal of Computational Vision andRobotics, 1(4), 415–429.

Kala, R., Shukla, A., & Tiwari, R. (2011). Robotic path planning using evolutionarymomentum based exploration. Journal of Experimental and Theoretical ArtificialIntelligence, 23(4), 469–495.

Kavraki, L. E., & Latombe, J. C. (1997). Probabilistic roadmaps for robot pathplanning. In Practical motion planning in robotics: Current approaches and futuredirections (pp. 33–53). John Wiley.

Kavraki, L. E., Svestka, P., Latombe, J. C., & Overmars, M. E. (2002). Probabilisticroadmaps for path planning in high dimensional spaces. IEEE Transactions onRobotics and Automation, 12(4), 566–580.

Kolushev, F. A., & Bogdanov, A. A. (2000). Multi-agent optimal path planning formobile robots in environment with obstacles. Proceedings of the PSI’99. Lecturenotes in computer science (Vol. 1755, pp. 503–510). Berlin, Heidelberg: Springer-Verlag.

Koza, J. R. (1992). Genetic programming: on the programming of computers by meansof natural selection. Cambridge, Mass.: MIT Press.

Kuffner, J. J., & LaValle, S. M. (2000). RRT-connect: An efficient approach to single-query path planning. In Proceedings of the 2000 IEEE international conference onrobotics and automation, San Francisco, CA , USA (Vol. 2, pp. 995–1001).

Moriarty, D. E. (1997). Symbiotic evolution of neural networks in sequential decisiontasks. Ph.D. thesis, Department of Computer Science, University of Texas, Austin,Texas.

Moriarty, D. E., & Miikkulainen, R. (1997). Forming neural networks throughefficient and adaptive coevolution. Evolutionary Computation, 5(4), 373–399.

Oliver, C. S., Saptharishi, M., Dolan, J. M., Trebi-Ollennu, A., & Khosla, P. K. (1999).Multi-robot path planning by predicting structure in a dynamic environment. InProceedings of the first IFAC conference on mechatronic systems (Vol. II, pp. 593–598).

O’Neill, M., & Ryan, C. (2001). Grammatical evolution. IEEE Transactions onEvolutionary Computation, 5(4), 349–358.

O’Neill, M., & Ryan, C. (2003). Grammatical evolution: Evolutionary automaticprogramming. Genetic programming (Vol. 4). Berlin: Kluwer AcademicPublishers.

Parker, L. P., Schneider, F. E., & Schultz, A. C. (2005). Multi-robot systems: Fromswarms to intelligent automata (Vol. 3). New York: Springer-Verlag.

Potter, M. A., & De Jong, K. A. (1994). A cooperative coevolutionary approach tofunction optimization. In Y. Davidor & H.-P. Schwefel (Eds.), Proceedings of thethird conference on parallel problem solving from nature (pp. 249–257). Berlin,Germany: Springer-Verlag.

Potter, M. A., & De Jong, K. A. (2000). Cooperative coevolution: An architecture forevolving coadapted subcomponents. Evolutionary Computation, 8(1), 1–29.

Sánchez-Ante, G., & Latombe, J. C. (2002). Using a PRM planner to comparecentralized and decoupled planning for multi-robot systems. In Proceedings ofthe IEEE international conference on robotics and automation (pp. 2112–2119).

Shukla, A., Tiwari, R., & Kala, R. (2010). Towards hybrid and adaptive computing.Berlin, Heidelberg: Springer-Verlag.

Slusny, S., Neruda, R., & Vidnerová, P. (2010). Comparison of behavior-based andplanning techniques on the small robot maze exploration problem. NeuralNetworks, 23, 560–567.

Svestka, P., & Overmars, M. H. (1998). Coordinated path planning for multiplerobots. Robotics and Autonomous Systems, 23, 125–152.

Vendrell, E., Mellado, M., & Crespo, A. (2001). Robot planning and re-planning usingdecomposition, abstraction, deduction, and prediction. Engineering Applicationsof Artificial Intelligence, 14, 505–518.

Wang, M., & Wu, T. (2005). Cooperative co-evolution based distributed pathplanning of multiple mobile robots. Journal of Zhejiang University Science, 6A(7),697–706.


Date post:	05-Sep-2016
Category:	Documents
Upload:	rahul-kala
View:	217 times
Download:	1 times

Multi-robot path planning using co-evolutionary genetic programming

Documents