Solving the Clustered Traveling Salesman

Problem via TSP methods

Yongliang Lu a, Jin-Kao Hao b,c, Qinghua Wu a,∗aSchool of Management, Huazhong University of Science and Technology, No.

1037, Luoyu Road, Wuhan, ChinabLERIA, Universite d’Angers, 2 Boulevard Lavoisier, 49045 Angers, France

cInstitut Universitaire de France, 1 Rue Descartes, 75231 Paris, France

Abstract

The Clustered Traveling Salesman Problem (CTSP) is a variant of the popularTraveling Salesman Problem (TSP) arising from a number of real-life applications.In this work, we explore an uncharted solution approach that solves the CTSPby transforming it to the well-studied TSP. For this purpose, we first investigatea technique to convert a CTSP instance to a TSP and then apply popular TSPsolvers (including exact and heuristic solvers) to solve the resulting TSP instance.We want to answer the following questions: How do state-of-the-art TSP solversperform on clustered instances converted from the CTSP? Do state-of-the-art TSPsolvers compete well with the best performing methods specifically designed for theCTSP? For this purpose, we present intensive computational experiments on variousCTSP benchmark instances to draw conclusions.

Keywords: Traveling salesman; Heuristics; Clustered traveling salesman; Combi-natorial optimization

1 Introduction

The Clustered Traveling Salesman Problem (CTSP), originally proposed byChisman [7], is an extension of the classic Traveling Salesman Problem (TSP)where the cities are grouped into clusters and each cluster of cities must bevisited contiguously. Formally, the problem is defined on a symmetric complete

∗ Corresponding author.Email addresses: luyongliang@hust.edu.cn (Yongliang Lu),

jin-kao.hao@univ-angers.fr (Jin-Kao Hao), qinghuawu1005@gmail.com(Qinghua Wu).

Preprint submitted to Elsevier 13 July 2020

arX

iv:2

007.

0525

4v1

[cs

.AI]

10

Jul 2

020

weighted graph G = (V,E) with a set of vertices V = {1, 2, ..., n} and a setof edges E = {(i, j) : i, j ∈ V, i 6= j}. The vertex set V is partitioned intodisjoint clusters V1, V2, ..., Vm (V1 ∪ V2 ∪ ... ∪ Vm = V ). Let C be an n × nsymmetric distance matrix such that cij (i, j = 1, 2..., n, i 6= j) representsthe travel cost between two corresponding vertices i and j, and satisfies thetriangle inequality rule. The objective of the CTSP is to find a minimum costHamiltonian circuit over all the vertices, where the vertices of each clustermust be visited consecutively.

V1

V3

V4

V2

Fig. 1. A feasible solution for an instance of the CTSP

Fig. 1 shows a feasible solution for a CTSP instance, where the solution corre-sponds to a Hamiltonian cycle such that the vertices of each cluster are visitedcontiguously.

The CTSP can be formally modelled as the following integer programmingmodel [7], where without loss of generality, the salesman is assumed to leaveorigin city 1 and return to 1.

min f =n∑

i=1

n∑j=1

cijxij (1)

subject ton∑

j=1

xij = 1 ∀i ∈ V (2)

n∑i=1

xij = 1 ∀j ∈ V (3)

ui − uj + (n− 1)xij ≤ n− 2 2 ≤ i 6= j ≤ n (4)∑i∈Vk

∑j∈Vk

xij = |Vk| − 1 ∀Vk ⊂ V, |Vk| ≥ 1, k = 1, 2, ...,m (5)

xij ∈ {0, 1} ∀i, j ∈ V (6)

2

ui ≥ 0 2 ≤ i ≤ n (7)

where xij = 1 if city j is visited immediately after city i; xij = 0 otherwise.

Objective function (1) seeks to minimize the total distance traveled by thesalesman. Constraints (2) and (3) ensure that each city is visited exactly once.Constraints (4) eliminate subtours, while constraints (5) guarantee that thecities of each cluster are visited contiguously. The remaining constraints arerelated to the decision variables.

One notices that the CTSP is equivalent to the TSP when there is a singlecluster or when each cluster contains exactly one vertex. Therefore, the CTSPis NP-hard, and thus computationally challenging in the general case. From apractical perspective, the CTSP is a versatile modeling tool for several opera-tional research applications arising in a wide variety of areas, including auto-mated warehouse routing [7], emergency vehicle dispatching [33], productionplanning [22], disk defragmentation [19], and commercial transactions withsupermarkets, shops and grocery suppliers [11]. As a result, effective solutionmethods for the CTSP can help to solve these practical problems. In fact, thecomputational challenge and wide range of applications of the problem havemotivated a variety of approaches that are reviewed in Section 2. However,unlike the classic TSP problem for which many powerful methods have beenintroduced in the past decades, studies on the CTSP are still quite limited.

In this work, we investigate a problem transformation approach mentionedin [7] (1975), which converts the CTSP to the TSP and assess the interestof popular modern TSP solvers for solving the converted instances. To ourknowledge, this is the first large computational study testing modern TSPsolvers on solving the CTSP. The work is motivated by the following consid-erations. First, this transformation was tested in [18] (1985) and [22] (1979).Many powerful modern TSP solvers have not been tested for solving the CTSP.Second, intensive researches on the TSP have led to the development of verypowerful solvers. Thus, it is interesting to know whether we can take advan-tage of these solvers to effectively solve the CTSP. Third, the TSP instancesconverted from the CTSP are characterized by their cluster structures. Theseinstances constitute interesting test cases for existing TSP solvers. This workaims thus to answer the following questions.

• How do state-of-the-art exact TSP solvers perform on clustered instancesconverted from the CTSP?• How do state-of-the-art inexact (heuristic) TSP solvers perform on clustered

instances converted from the CTSP?• Do state-of-the-art TSP solvers compete well with the best performing

methods specifically designed for the CTSP?

3

Answering these questions helps to enrich the state-of-the-art of solving theCTSP and gain novel knowledge on using powerful TSP methods to solve newproblems.

The remainder of this paper is organized as follows. Section 2 reviews existingsolution methods for the CTSP. Section 3 presents the transformation of theCTSP to the TSP and three popular TSP methods (solvers). Section 4 showscomputational studies of the TSP solvers applied to the clustered instancesand comparisons with existing algorithms dedicated to the CTSP. Finally,concluding remarks are provided in Section 5.

2 Literature review on existing solution methods

There are several dedicated solution algorithms for solving the CTSP that arebased on exact, approximation, and metaheuristic approaches.

Along with the introduction of the CTSP, Chisman [7] proposed a branch-and-bound algorithm to solve the integer programming model presented in Section1. Jongens and Volgenant [18] developed an algorithm based on the 1-treerelaxation to provide lower bounds as well as a heuristic to find satisfactoryupper bounds. Mestria et al. [23] used the mathematical formulation of [7] andIBM Parallel CPLEX solver (version 11.2) to obtain lower bounds for mediumCTSP instances (|V | ≤ 1000).

Various a-approximation algorithms [5,10,12] have been developed for theCTSP. These approximation algorithms require either the starting and end-ing vertices in each cluster or a prespecified order of visiting the clusters inthe tour as inputs, and solve the inter-cluster and intra-cluster problems in-dependently. Recently, Bao and Liu [6] presented a new 2.17-approximationalgorithm where no starting and ending vertices were specified.

Given that the CTSP is a NP-hard problem, a number of heuristic and meta-heuristic algorithms have also been investigated, which aim to provide high-quality solutions in acceptable computation time, but without provable op-timal guarantee of the attained solutions. For example, Laporte et al. [20]presented a tabu search algorithm to solve a particular case of the CTSP,where the clusters are visited in a prespecified order. Potvin and Guertin [30]developed a genetic algorithm for the CTSP that finds inter-cluster paths andthen intra-cluster paths. Later, Ding et al. [8] proposed a two-level geneticalgorithm for the CTSP. In the first level, a genetic algorithm is used to findthe shortest Hamiltonian cycle for each cluster. In the second level, a modifiedgenetic algorithm is applied to merge the Hamiltonian cycles of all the clustersinto a complete tour.

4

In addition to these early heuristic algorithms, Mestria et al. [23] investi-gated GRASP (Greedy Randomized Adaptive Search Procedure) with path-relinking. Among the six proposed heuristics, one heuristic corresponds to thetraditional GRASP procedure whereas the other heuristics include differentpath relinking procedures. In [24], Mestria studied a hybrid heuristic, which isbased on a combination of GRASP, Iterated Local Search (ILS) and VariableNeighborhood Descent (VND). Recently, Mestria [25] presented another com-plex hybrid algorithm (VNRDGILS) which mixes GRASP, ILS, and VariableNeighborhood Random Descent to explore several neighborhoods. Accordingto the computational results reported in [23,24,25], these GRASP-based al-gorithms are among the best performing heuristics specially designed for theCTSP currently available in the literature.

In this work, we explore the uncharted problem transformation approach thatconverts the CTSP to the conventional TSP and employs popular (exact andinexact) TSP solvers to solve the TSP instances converted from the CTSPbenchmark instances.

3 Solving the CTSP via TSP methods

3.1 Transformation of the CTSP to the TSP

The basic idea of this transformation of the CTSP to the TSP is to add alarge artificial cost M to all inter-cluster edges in order to force the salesmanto visit all the cities within each cluster before leaving it.

Given a CTSP instance G = (V,E) with distance matrix C, we define a TSPinstance G′ = (V ′, E ′) with distance matrix C

′as follow.

• Define V = V ′ and E = E ′.• Define the travel distance c

′ij in G′ by

c′

ij =

cij +M if i and j belong to different clusters

cij otherwise

Obviously, if the value of M is sufficiently large, then the best Hamiltoniancycle in G′ is a feasible CTSP solution in G, in which the vertices of eachcluster are visited contiguously.

Property. An optimal solution to the TSP instance corresponds to an optimalsolution to the original CTSP instance.

Proof. Let S ′ and S be the optimal solutions of the TSP instance G′ and

5

the original CTSP instance G, respectively. Let m be the number of clustersof G. To minimize the total travel cost, there are only m inter-cluster edges inS ′. Therefore, S ′ is a feasible CTSP solution for G and satisfies the followingrelation:

f(S ′) = f(S) +m×M

Obviously, S ′ corresponds to S by subtracting the constant m×M .

3.2 Solution methods for the TSP

There are numerous solution methods for the TSP [3]. In this work, we adoptthree very popular TSP solvers whose codes are publicly available, includingone exact solver (Concorde [1]) and two inexact (heuristic) solvers (LHK-2[14] and GA-EAX [29]).

Notice that the TSP instance converted from a CTSP instance has a par-ticular feature that the vertices are grouped into clusters and the distancebetween each pair of vertices within a same cluster is in general small, whilethis distance is large for two vertices from different clusters. Along with thepresentation of the TSP solvers, we discuss their suitability for solving suchclustered instances each time this is appropriate.

3.2.1 Exact Concorde solver

Concorde is an advanced exact TSP solver for the symmetric TSP based onBranch-and-Bound and problem specific cutting plane methods [1]. It makesuse of a specifically designed QSopt linear programming solver. According to[16], Concorde is the best performing exact algorithm for the TSP. As shownin [3], Concorde can solve benchmark instances from TSPLIB with up to 1000vertices to optimality within a reasonable computation time and it also solveslarge TSP instances at the cost of a long computation time.

The run time behavior of Concorde has been investigated essentially on ran-dom uniform instances. For instance, in [3], Applegate et al. investigated therun time required by Concorde for solving random uniform instances and in-dicated that the run time increases as an exponential function of instancesize |V |. In [16], Hoos and Stutzle further demonstrated that the median run

time required by Concorde scales with instance size |V | of the form ab√|V |

(a ≈ 0.21, b ≈ 1.24) on the widely studied class of uniform random TSPinstances. To our knowledge, no study has been reported concerning the be-havior of Concorde on sharply clustered instances. As a result, the currentstudy will provide useful information on this issue.

6

3.2.2 Lin-Kernighan based heuristic solver

According to the TSP literature, a majority of the best performing TSP heuris-tic algorithms is based on the Lin-Kernighan (LK) heuristic [21] and its exten-sions. The LK heuristic is a variable-depth k-opt local search procedure, wherethe k-opt neighborhood is partially searched with a smart pruning strategy.LK explores the most promising neighbors within the k-opt neighborhood,that is, the set of feasible tours obtained by removing k edges and addingother k edges such that the resulting tour is feasible. Several improved ver-sions of the basic LK heuristic have been introduced within the iterated localsearch framework (e.g., [4,13,14,26]).

Among these iterated LK algorithms, Helsgaun’s LKH [13,14] is the uncon-tested state-of-the-art heuristic TSP solver. In [13], Helsgaun developed aniterated version of LK together with an efficient implementation of the LKalgorithm, known as the Lin-Kernighan-Helsgaun (LKH-1) heuristic, where a5-opt move is used as the basic move to broaden the search and an α-measuremethod based on sensitivity analysis of minimum spanning trees is used torestrict the search to relative few of the α-nearest neighbors of a vertex tospeed up the search process. Later, in [14], Helsgaun further extended LKH-1by developing a highly effective implementation of the k-opt procedure (calledLKH-2), which eliminated many of the limitations and shortcomings of LKH-1. Furthermore, LKH-2 specially extended the data structures of LKH-1 tosolve very large TSP instances. The main features of LKH-2 include (1) usingsequential and non-sequential k-opt moves, (2) using several partitioning pro-cedures to partition a large TSP instance into smaller subproblems, (3) usinga tour merging procedure to generate a better solution from two or more localoptimum solutions, and (4) applying a backbone-guided search to guide thelocal search to make biased local perturbations. LKH-2 is considered to beone of most effective heuristic methods for finding very high-quality solutionsfor various large TSP instances.

However, the LK algorithm and any LK-based algorithms are unsuitable forclustered instances of the TSP because they require much longer runningtimes on such instances than on uniformly distributed instances [28]. The mainreason why the LK heuristic stumbles on clustered instances is that relativelylarge inter-cluster edges serve as bait edges. When removing such a bait edge,the LK heuristic is tricked into long and often fruitless searches. More precisely,each time an edge bridging two clusters is removed, the cumulative gain risesenormously, and the procedure is encouraged to perform very deep searches.To alleviate the problem, a cluster compensation technique was proposed in[28] for the Lin-Kernighan heuristic to limit its performance degradation. In[14], Helsgaun showed that the LKH-2 algorithm performs significantly worseon sharply clustered instances than on uniform random instances. However,no effective method was proposed in [14] to remedy this difficulty.

7

3.2.3 Edge assembly crossover based genetic algorithm

Population-based evolutionary algorithms are another well-known approachfor the TSP. A popular example is the powerful genetic algorithm introducedby Nagata and Kobayashi in [29]. This algorithm (called GA-EAX, see Algo-rithm 1) is characterized by its powerful edge assembly crossover (EAX) op-erator introduced in [27] with an efficient implementation and a cost-effectiveselection strategy for maintaining population diversity.

Algorithm 1 GA-EAX for the CTSPRequire: TSP instance G, population size p; number of offspring solutions r gen-

erated from each parent pairEnsure: best solution S∗

1: POP = {P1, P2, ..., Pp} ← Initial Population(G)2: while stopping condition is not met do3: Randomly shuffle the solutions in POP4: for i = 1, 2, ..., p do5: S1 ← Pi, S2 ← Pi+1 /* Note: Pp+1 = P1 */6: (o1, ..., or)← EAX(S1, S2)7: Pi ← Select Best(o1, ..., or, S1)8: end for9: end while

10: S∗ ← Best(POP )11: Return S∗

The key EAX operator generates, from two high-quality tours (parents), oneoffspring tour by first inheriting the edges from the parents to construct dis-joint subtours and then connecting the subtours with new edges in a greedyfashion (similar to building a minimal spanning tree). Let SA and SB be theparents, EAX operates as follows (see Fig. 2 for an example):

SA

SB

GAB AB-cycles

E-set Intermediate Offspring

Fig. 2. Illustrative example of the EAX crossover operator

8

(1) Generate an undirected multigraph defined as GAB = (V,EA∪EB), whereEA and EB are the sets of edges of parents SA and SB, respectively.

(2) Extract all AB-cycles from GAB. An AB-cycle is defined as a cycle inGAB, such that edges of EA and edges of EB are alternately linked.

(3) Construct an E-set by selecting AB-cycles according to a given selectionstrategy (e.g., single, k-multiple, block and block2 [29]), where an E-setis a set of AB-cycles.

(4) Copy parent SA to an intermediate solution o. Then, remove the edges ofEA in the E-set from o and add those of EB in the E-set to o. This leadsto an intermediate solution o with one or more subtours.

(5) Connect all the subtours in o with new short edges to generate a completetour (a feasible offspring solution) by using a greedy heuristic.

Note that different versions of EAX can be developed by using different selec-tion strategies of AB-cycles for constructing E-sets. The GA-EAX algorithmemploys the single and block2 strategies to generates offspring solutions fromparent solutions. To maintain a healthy population diversity, GA-EAX alsouses an edge entropy measure to select the solution to be used to replace aparent in the population.

Other studies (e.g., [15]) also indicated the usefulness of edge-assembly-likecrossovers for solving clustered instances of the TSP. As shown in the nextsection, the EAX-based genetic algorithm performs remarkably well on all theclustered instances transformed from the CTSP.

4 Computational experiments

In this section, we evaluate the capacity of the TSP solvers presented in Section3.2 to solve the CTSP via its transformation to the TSP. For this purpose,we examine their qualitative performances and run time efficiencies on variousCTSP benchmark instances and make comparisons with the best dedicatedCTSP algorithms in the literature.

4.1 Benchmark instances

Our computational assessments are based on three sets of 45 CTSP benchmarkinstances with 101 to 5000 vertices. Sets 1 and 2 include 20 medium instances(101 ≤ |V | ≤ 1000) and 15 large instances (1173 ≤ |V | ≤ 2000), which areclassical and widely used in the CTSP literature (e.g., [23,24,25]). Set 3 is anew set of 10 very large instances with 3000 and 5000 vertices.

9

Sets 1 and 2 (35 instances): These instances belong to the following sixtypes: (1) instances taken from the TSPLIB [32] where the clusters are gen-erated by using a k-means clustering algorithm; (2) instances created froma selection of classic TSP instances [17], where the clusters are created bygrouping the vertices in geometric centers; (3) instances generated by usingthe Concorde interface [2]; (4) instances generated using the layout proposedin [19]; (5) instances similar to type 2, but generated with different parame-ters; (6) instances adapted from the TSPLIB [32], where the rectangular floorplan is divided into several quadrilaterals and each quadrilateral correspondsto a cluster.

Set 3 (10 instances): These instances were created from 10 very large TSPinstances [17] with 3000 and 5000 vertices. Following [24], for these instances,m geometric centers are selected and the clusters are created by grouping thevertices in the geometric centers, where the coordinates of geometric centersare selected uniformly in the interval [0,1000) and m is the number of clusters.

All these instances are available at https://github.com/lyldft/ctsp.

4.2 TSP solvers and experimental protocol

For our study, we employed three popular TSP solvers presented in Section3.2, which are among the most powerful methods for the TSP in the literature.

• Exact Concorde TSP solver 1 : We used version Concorde-03.12.19 and ranthe solver with its default parameter setting with a cutoff time of 24 CPUhours per instance.• Inexact LKH-2 TSP solver 2 : LKH-2 is an iterated local search procedure

and typically terminates after a fixed number of iterations (default is |V |).We observed that LKH-2 with this default stopping condition becomes tootime consuming on our clustered instances (see discussion in Section 3.2.2).In our experiment, we used a shorter number of iterations of 0.1*|V | and0.2*|V | while using the default values for the other parameters of LKH-2.• Inexact GA-EAX TSP solver 3 : We used GA-EAX with its default param-

eter setting given in [29]: p = 300, r = 30 and GA-EAX terminates if thedifference between the average tour length and the shortest tour length inthe population is less than 0.001.

The experiments were carried out on a computer running Windows 7 withan Intel Core i7-4790 processor (3.60 GHz and 8 GB of RAM). Given the

1 http://www.math.uwaterloo.ca/tsp/concorde/index.html2 http://akira.ruc.dk/~keld/research/LKH/3 https://github.com/sugia/GA-for-TSP

10

stochastic nature of LKH-2 and GA-EAX, we ran each algorithm 10 times foreach instances while the deterministic Concorde TSP solver was run one timeto solve each instance.

4.3 Computational results and comparison of popular TSP solvers

Our computational studies aim to answer the following questions: How dostate-of-the-art exact TSP solvers perform on clustered instances convertedfrom the CTSP? How do state-of-the-art inexact (heuristic) TSP solvers per-form on clustered instances converted from the CTSP?

The results of the three TSP solvers (Concorde, LKH-2, GA-EAX) on the 20medium and 15 large CTSP benchmark instances are summarized in Tables1 and 2. Columns 1 to 3 show the basic information of each instance: theinstance name (Instance), the number of vertices (|V |) and the number ofclusters (m). Column 4 gives the optimal objective value reported by theexact Concorde TSP solver, followed by the required run time in seconds. Forboth the LKH-2 and GA-EAX solvers, we show the best (B-Err) and average(A-Err) results over 10 independent runs in the form of the percentage gapto the optimal solution, as well as the average run time in seconds. If thebest solution over 10 independent runs equals the optimal solution obtainedwith the exact Concorde TSP solver, the corresponding cell in column B-Errshows ‘=’ along with the number of runs that succeeded in finding the optimalsolution. Finally, row ‘Avg.’ provides the average run time in seconds for eachapproach, and the average gap between the average objective values obtainedwith LKH-2/GA-EAX and the optimal values obtained with the ConcordeTSP solver.

From Tables 1-2, we can make the following observations.

First, the exact Concorde TSP solver performs very well for these 35 instancesand is able to solve all of them exactly. Specifically, the 20 medium instancescan be solved easily in a short run time (an average of about 30 seconds). The15 large instances are more difficult and the run time needed to solve theseinstances increases considerably (an average of 1553 seconds, reaching 9663seconds for the most difficult instance).

Second, the inexact LKH-2 TSP solver does not performs as well as Concorde.With the stopping condition of 0.1*|V | iterations, LKH-2 misses respectively2 and 8 optimal solutions for the medium and large instances with an averagerun time of 49.9 and 371.6 seconds. LKH-2 obtains improved results (optimalsolution for one more medium instance and 3 large instances) with the relaxedcondition of 0.2*|V | iterations. However, in this case, LKH-2 requires roughlydoubled its run time.

11

Table 1Computational results of the TSP solvers Concorde, LKH-2 and GA-EAX onmedium CTSP instances.

Exact TSP solver LKH-2(10 runs) GA-EAX(10 runs)

Concorde No. of iterations=0.1*|V | No. of iterations=0.2*|V | Default configuration

Instance |V | m Opt. Time B-Err A-Err Time B-Err A-Err Time B-Err A-Err Time

i-50-gil262 262 50 135431 2.4 =(10) 0.0000 1.1 =(10) 0.0000 1.8 =(10) 0.0000 1.8

10-lin318 318 10 529584 2.6 =(1) 0.0020 13.1 =(1) 0.0020 22.4 =(10) 0.0000 1.9

10-pcb442 442 10 537419 28.9 =(8) 0.0032 65.2 =(8) 0.0014 112.0 =(10) 0.0000 6.4

C1k.0 1000 10 132521027 34.0 =(3) 0.0063 78.7 =(1) 0.0056 168.5 =(10) 0.0000 15.2

C1k.1 1000 10 129128125 27.5 0.0002 0.0013 82.6 =(3) 0.0007 148.9 =(10) 0.0000 13.7

C1k.2 1000 10 142784000 122.7 0.0009 0.0092 221.4 0.0009 0.0092 371.2 =(10) 0.0000 16.7

300-6 300 6 8934 4.4 =(10) 0.0000 25.9 =(10) 0.0000 52.6 =(10) 0.0000 1.9

400-6 400 6 9045 6.8 =(10) 0.0000 47.6 =(10) 0.0000 102.9 =(10) 0.0000 4.5

700-20 700 20 41425 19.5 =(4) 0.0092 289.4 =(9) 0.0017 590.8 =(10) 0.0000 10.6

200-4-h 200 4 62777 0.7 =(10) 0.0000 4.0 =(10) 0.0000 6.5 =(10) 0.0000 1.0

200-4-x1 200 4 60574 1.7 =(10) 0.0000 4.2 =(10) 0.0000 5.9 =(10) 0.0000 1.0

600-8-z 600 8 128891 5.8 =(8) 0.0063 48.0 =(6) 0.0049 84.9 =(10) 0.0000 5.6

600-8-x2 600 8 128891 4.7 =(8) 0.0063 48.1 =(6) 0.0049 84.8 =(10) 0.0000 5.6

300-5-108 300 5 67760 1.7 =(10) 0.0000 15.2 =(10) 0.0000 23.5 =(10) 0.0000 2.3

300-20-111 300 20 309739 2.4 =(9) 0.0002 12.4 =(10) 0.0000 19.9 =(10) 0.0000 2.0

500-15-306 500 15 194818 3.6 =(10) 0.0000 18.9 =(10) 0.0000 32.8 =(10) 0.0000 4.6

500-25-308 500 25 365447 9.1 =(2) 0.0112 11.7 =(2) 0.0098 19.8 =(9) 0.0001 5.4

25-eil101 101 25 23671 0.3 =(10) 0.0000 0.4 =(10) 0.0000 0.6 =(10) 0.0000 0.8

42-a280 280 42 129645 2.2 =(4) 0.0079 1.9 =(10) 0.0000 3.7 =(10) 0.0000 1.9

144-rat783 783 144 914228 287.4 =(3) 0.0011 8.3 =(4) 0.0003 13.1 =(10) 0.0000 9.5

Avg. 28.4 0.0032 49.9 0.0020 93.3 0.0000 5.6

Table 2Computational results of the TSP solvers Concorde, LKH-2 and GA-EAX on largeCTSP instances.

Exact TSP solver LKH-2 (10 runs) GA-EAX (10 runs)

Concorde No. of iterations=0.1*|V | No. of iterations=0.2*|V | Default configuration

Instance |V | m Opt. Time B-Err A-Err Time B-Err A-Err Time B-Err A-Err Time

49-pcb1173 1173 49 61600 9663.2 =(1) 0.1432 140.1 =(1) 0.0994 206.8 =(6) 0.0031 28.7

100-pcb1173 1173 100 63382 835.9 0.0063 0.1426 50.5 0.0063 0.1186 77.8 =(10) 0.0000 27.4

144-pcb1173 1173 144 62142 67.0 0.0016 0.2092 19.2 0.0016 0.2079 38.5 =(10) 0.0000 16.4

10-nrw1379 1379 10 58783 958.4 =(3) 0.0129 336.0 =(7) 0.0043 420.9 =(4) 0.0061 23.2

12-nrw1379 1379 12 59129 112.3 =(2) 0.0027 33.5 =(5) 0.0008 90.7 =(10) 0.0000 22.8

1500-10-503 1500 10 11116 80.4 0.0540 0.1367 447.5 =(1) 0.0387 897.6 =(10) 0.0000 21.6

1500-20-504 1500 20 15698 75.5 =(2) 0.0803 319.7 =(4) 0.0032 569.4 =(4) 0.0344 29.3

1500-50-505 1500 50 22900 66.9 =(1) 0.2000 150.5 =(1) 0.1865 299.1 =(7) 0.0013 30.9

1500-100-506 1500 100 29799 127.8 =(2) 0.0718 49.5 =(3) 0.0302 87.8 =(9) 0.0010 36.6

1500-150-507 1500 150 34068 137.6 =(1) 0.0361 42.5 =(1) 0.0302 78.2 =(10) 0.0000 28.9

2000-10-a 2000 10 105368 7957.5 0.0294 0.0793 2139.4 0.0019 0.0789 4071.9 =(1) 0.0877 45.5

2000-10-h 2000 10 33708 1283.1 0.0059 0.0311 700.7 0.0059 0.0311 1314.9 =(9) 0.0012 33.9

2000-10-z 2000 10 33509 296.4 0.0030 0.0406 279.6 =(1) 0.0164 430.0 =(9) 0.0003 36.0

2000-10-x1 2000 10 33792 1412.7 0.0355 0.0462 581.8 0.0296 0.0394 1387.4 =(8) 0.0059 36.0

2000-10-x2 2000 10 33509 226.0 0.0030 0.0406 283.5 =(1) 0.0164 430.0 =(9) 0.0003 36.7

Avg. 1553.4 0.0849 371.6 0.0601 693.4 0.0094 30.3

Third, the GA-EAX solver performs remarkably well by attaining the optimalvalues for all 35 instances. For the 20 medium instances, GA-EAX consistentlyhits the optimal solutions for each of its 10 run (except for one instance forwhich it has a hit of 9 out of 10). For the 15 large instances, except 3 cases,GA-EAX hits the optimum of each instance at least 6 times out of 10 runs.The average run time is only 5.6 seconds for the medium instances and 30.3seconds for the large instances. Compared to the Concorde TSP solver and the

12

LKH-2 TSP solver, the GA-EAX algorithm is thus extremely time efficient.Moreover, contrary to the Concorde and LKH-2 solvers, the computation timerequired by GA-EAX remains very stable across the instances of the same set,indicating a high robustness and scalability of this solver.

Table 3Computational results of the TSP solvers Concorde, LKH-2 and GA-EAX on onnew very large CTSP instances.

Exact TSP solver LKH-2 (10 runs) GA-EAX (10 runs)

Concorde No. of iterations=0.1*|V | No. of iterations=0.2*|V | Default configuration

Instance |V | m Opt. Time B-Err A-Err Time B-Err A-Err Time B-Err A-Err Time

i-1 3000 50 23571 1110.2 =(1) 0.2719 1738.4 =(1) 0.2232 2652.0 =(2) 0.1786 94.6

i-2 3000 100 32750 9899.2 0.0641 0.1658 640.5 0.0641 0.1463 1212.1 =(3) 0.0134 100.2

i-3 3000 150 36898 1367.0 =(3) 0.0740 269.5 =(4) 0.0279 468.6 =(9) 0.0003 107.5

i-4 3000 200 41825 1557.4 =(1) 0.0904 256.7 =(3) 0.0550 439.5 =(3) 0.0033 105.2

i-5 5000 150 (50587) >24h -0.6306 -0.5905 1421.6 -0.6306 -0.6134 2617.1 -0.6306 -0.6292 216.9

i-6 5000 200 57185 5943.4 =(1) 0.0371 965.1 =(1) 0.0344 1858.3 =(3) 0.0044 242.8

i-7 5000 250 62464 2912.2 =(3) 0.0155 575.8 =(5) 0.0120 1261.9 =(8) 0.0005 278.2

i-8 5000 300 (65990) >24h -0.0788 -0.0339 751.3 -0.0833 -0.0511 1305.7 -0.0955 -0.0899 294.0

i-9 5000 350 (70794) >24h -1.0425 -0.9841 613.4 -1.0594 -1.0033 1419.7 -1.0580 -1.0519 308.3

i-10 5000 400 74459 25639.4 0.0013 0.0283 420.6 =(1) 0.0200 691.6 =(1) 0.0056 259.7

Avg. 761.3 1392.7 200.7

Table 3 presents the results of the three TSP solvers on the 10 new verylarge CTSP instances of Set 3. Notice that if an instance cannot be solvedexactly by the Concorde TSP solver, the percentage gaps (B-Err and A-Err)are calculated using the Concorde’s best upper bound. In this case, column‘Opt.’ corresponds to the best upper bound from Concorde, and a negative(positive) gap indicates a better (worse) result compared to this bound.

From Table 3, we can make the following observations. First, Concorde man-ages to optimally solve 7 out of these 10 very large instances with a run timeranging from 1100 seconds to more than 25000 seconds. For these 7 instances,LKH-2 attains the optimal solutions for 6 instances while GA-EAX reaches alloptimal solutions. Second, for the three instances that cannot be solved exactlyby Concorde, both LKH-2 and GA-EAX report better results than the bestupper bounds of Concorde. However, LKH-2 has a worse performance bothin terms of solution quality and computation time compared with GA-EAX.Third, GA-EAX has an excellent time efficiency across the instances of thisset and scales very well with the increase of instance sizes. These observationsare consistent with those from Tables 1-2.

To sum, the exact Concorde TSP solver is very efficient for the CTSP instanceswith up to 1000 vertices and becomes time consuming for larger instances.The inexact LKH-2 TSP solver has troubles to solve these clustered instances,which is consistent with previous studies such as [14,28]. The EAX-basedgenetic algorithm performs remarkably well both in terms of solution qualityand computational efficiency and scales well with the instance sizes.

To deepen our computational study, we call upon to the performance profile, a

13

100 100 100 100.01 100.01 100.01 100.01 100.01 100.02 100.02 100.020

0.2

0.4

0.6

0.8

1

Performance ratio

Per

cent

age

ofpr

oble

ms

solv

edPerformance Profile

GA-EAXLKH-2(0.1*|V |)LKH-2(0.2*|V |)

(a) Average results

100 100.2 100.4 100.6 100.8 101 101.2 101.4 101.6 101.80

0.2

0.4

0.6

0.8

1

Performance ratio

Per

cent

age

ofpr

oble

ms

solv

ed

Performance Profile

GA-EAXLKH-2(0.1*|V |)LKH-2(0.2*|V |)

(b) Average run time

Fig. 3. Performance profiles comparing solution quality and computing time.

analytic tool for evaluating the performances of multiple compared optimiza-tion algorithms [9]. The performance profile uses a cumulative distributionfunction for a performance metric, such as run time, objective function val-ues, number of iterations, and so on. For a given metric, the performanceprofile associated to an algorithm s indicates the probability ρs(τ) that thealgorithm attains results which are within a factor τ of the best result attainedby all compared algorithms over a set of problem instances. A higher proba-bility indicates a better algorithmic performance under the given metric. Thevalue of ρs(1) is the probability that the algorithm will win over the rest ofthe compared algorithms.

To make a fair and meaningful comparison with this tool, we focus on the twoinexact solvers LKH-2 and GA-EAX and run each solver 10 times on eachof the 45 instances. We use the software ‘perprof-py’ [31] to draw the perfor-mance profiles (see Figure 3) where the quality of the solution is measured bythe average objective value and average run time. These performance profilesshows a clear dominance of GA-EAX over LKH-2 both in terms of solutionquality and run time efficiency.

4.4 TSP solvers v.s. state-of-the-art CTSP heuristics

In Section 4.3, we identified GA-EAX as the most suitable method for solvingclustered instances converted from the CTSP. We now answer the followingquestion: Does GA-EAX compete well with state-of-the-art CTSP heuristicsspecially designed for the problem?

14

Table 4List of the reference algorithms for the CTSP

Algorithm name Reference Search strategy

VNRDGILS [25](2018) A hybrid heuristic based on GRASP, ILS and VNRD

HHGILS [24](2016) A hybrid heuristic based on GRASP, ILS and VND

GPR1R2 [23](2013) A GRASP with Path Relinking PR1 and PR2

GPR1 [23](2013) A GRASP with Path Relinking PR1

GPR2 [23](2013) A GRASP with Path Relinking PR2

GPR3 [23](2013) A GRASP with Path Relinking PR3

GPR4 [23](2013) A GRASP with Path Relinking PR4

GRASP [23](2013) A traditional GRASP heuristic

TLGA [8](2007) A two-level genetic algorithm

For this purpose, we adopt three best performing CTSP heuristics: VNRDG-ILS [25], HHGILS [24], and GPR1R2 [23]. Indeed, according to the experi-mental studies reported in [23,24,25], these three heuristics perform the bestamong the recent CTSP heuristics available in the literature (see Table 4).This study is based on the 35 medium and large instances of Sets 1 and 2(no results for the three CTSP heuristics are available on the 10 very largeinstances of Set 3).

Table 5 provides a summary of the results of the GA-EAX TSP solver alongwith the results reported by the three CTSP algorithms on the medium andlarge instances. For each instance and algorithm, columns ‘fbest’, ‘favg’ and‘t(s)’ show respectively the best objective value over 10 independent runs, theaverage objective value and the average run time in seconds. To determinewhether there exists a statistically significant difference in performance be-tween the GA-EAX TSP solver and each CTSP algorithm in terms of bestand average results, the p-values from the Wilcoxon signed-rank tests are givenin the last row of the tables. Entries with “-” mean that the corresponding re-sults are not available in the literature. The best objective values obtained bythe compared algorithms are indicated in bold if they attain the optimal solu-tion. Notice that the results of the CTSP algorithms (VNRDGILS, HHGILSand GPR1R2) correspond to 10 executions per instance on a computer with2.83 GHz Intel Core 2 CPU and 8 GB RAM and the time limit per run wasset to 720 seconds for medium instances and 1080 seconds for large instances.

From Table 5, we observe that compared to the three CTSP algorithms, theGA-EAX TSP solver attains consistently the optimal solutions for all 35medium and large CTSP instances. However, among the three CTSP algo-rithms, the optimal result is obtained only for 1 instance by HHGILS. Al-though the experimental platforms are different, we further observe that theGA-EAX TSP solver is more than an order of magnitude faster than the CTSPalgorithms while reporting much better results.

Figure 4 provides boxplot graphs to compare the distribution and range of theaverage results for each compared algorithm, except GPR1R2 for the mediuminstances since its results on several medium instances are not available. In

15

Tab

le5.

Com

para

tive

resu

lts

bet

wee

nth

eG

A-E

AX

TSP

solv

eran

dth

ree

CT

SP

algo

rith

ms

onm

ediu

man

dla

rge

CT

SP

inst

ance

s.G

A-E

AX

VN

RD

GIL

SH

HG

ILS

GP

R1R

2

Inst

ance

|V|

mfbest

favg

t(s)

fbest

favg

t(s)

fbest

favg

t(s)

fbest

favg

t(s)

i-50-g

il262

262

50

135431

135431.0

1.8

135483

135510.2

720.0

135510

135578

720.0

--

-

10-l

in318

318

10

529584

529584.0

1.9

530604

530871.4

720.0

530283

530817.9

720.0

530443

532697.9

720.0

10-p

cb442

442

10

537419

537419.0

6.4

538309

538903.4

720.0

538958

539988.3

720.0

540043

543104.2

720.0

C1k.0

1000

10

132521027

132521027.0

15.2

133260549

133490775.9

720.0

133287594

133776274.1

720.0

133490776

133708187.6

720.0

C1k.1

1000

10

129128125

129128125.0

13.7

129877874

130035540.2

720.0

129825403

130206778.3

720.0

130193590

130391693.5

720.0

C1k.2

1000

10

142784000

142784000.0

16.7

143321630

143481489.6

720.0

143278093

143525149.6

720.0

--

-

300-6

300

68934

8934.0

1.9

8935

8941.1

720.0

8934

8942.9

720.0

8959

8985.3

720.0

400-6

400

69045

9045.0

4.5

9053

9062.3

720.0

9051

9063.2

720.0

--

-

700-2

0700

20

41425

41425.0

10.6

41456

41489.7

720.0

41452

41485.6

720.0

41540

41573.3

720.0

200-4

-h200

462777

62777.0

1.0

62867

63058.3

720.0

62804

63058.3

720.0

62994

63710.2

720.0

200-4

-x1

200

460574

60574.0

1.0

60637

60796.2

720.0

60931

61378.5

720.0

--

-

600-8

-z600

8128891

128891.0

5.6

129468

129862.7

720.0

129416

129928.6

720.0

130459

131235.1

720.0

600-8

-x2

600

8128891

128891.0

5.6

129246

129533.9

720.0

129246

129691.5

720.0

--

-

300-5

-108

300

567760

67760.0

2.3

67766

67868.7

720.0

67814

67930.5

720.0

--

-

300-2

0-1

11

300

20

309739

309739.0

2.0

310146

310270.9

720.0

310209

310427

720.0

309928

310551.9

720.0

500-1

5-3

06

500

15

194818

194818.0

4.6

194946

195201.5

720.0

195202

195438.1

720.0

--

-

500-2

5-3

08

500

25

365447

365447.2

5.4

365717

365937.8

720.0

365828

366085

720.0

366232

366785.7

720.0

25-e

il101

101

25

23671

23671.0

0.8

23673

23685.2

720.0

23678

23690

720.0

23676

23711.3

720.0

42-a

280

280

42

129645

129645.0

1.9

129729

129755.2

720.0

129716

129833.2

720.0

--

-

144-r

at7

83

783

144

914228

914228.0

9.5

915088

915179.8

720.0

915180

915363.2

720.0

915547

915913.7

720.0

49-p

cb1173

1173

49

61600

61601.9

28.7

65750

66487.7

1080.0

67043

68260.7

1080.0

70651

73311.9

1080.0

100-p

cb1173

1173

100

63382

63382

27.4

68708

69383.2

1080.0

68786

70640.8

1080.0

72512

74871.7

1080.0

144-p

cb1173

1173

144

62142

62142

16.4

68414

68941.4

1080.0

66830

69084.3

1080.0

72889

74621.6

1080.0

10-n

rw1379

1379

10

58783

58786.6

23.2

63951

64895.9

1080.0

63620

64643.9

1080.0

66747

68955.8

1080.0

12-n

rw1379

1379

12

59129

59129

22.8

62893

63532.3

1080.0

63558

64741.6

1080.0

66444

69141.2

1080.0

1500-1

0-5

03

1500

10

11116

11116

21.6

11969

12103.0

1080.0

11986

12109.5

1080.0

12278

12531.4

1080.0

1500-2

0-5

04

1500

20

15698

15703.4

29.3

16678

16867.4

1080.0

17107

17315.7

1080.0

17252

17589.1

1080.0

1500-5

0-5

05

1500

50

22900

22900.3

30.9

24631

24803.6

1080.0

25264

25558.9

1080.0

25124

25761.5

1080.0

1500-1

00-5

06

1500

100

29799

29799.3

36.6

32474

32616.8

1080.0

32260

33760.6

1080.0

33110

33692.7

1080.0

1500-1

50-5

07

1500

150

34068

34068

28.9

37357

38251.1

1080.0

37658

38433.1

1080.0

38767

39478.0

1080.0

2000-1

0-a

2000

10

105368

105460.4

45.5

115779

116897.3

1080.0

116254

116881.4

1080.0

116473

118297.5

1080.0

2000-1

0-h

2000

10

33708

33708.4

33.9

36806

38351.8

1080.0

36447

37305.1

1080.0

37529

38861.8

1080.0

2000-1

0-z

2000

10

33509

33509.1

36.0

36815

38035.7

1080.0

37059

37443.7

1080.0

37440

38765.9

1080.0

2000-1

0-x

12000

10

33792

33794

36.0

36783

37488.6

1080.0

36752

37704.0

1080.0

37262

39253.1

1080.0

2000-1

0-x

22000

10

33509

33509.1

36.7

37132

38240.6

1080.0

36660

37117.1

1080.0

37704

38699.5

1080.0

p-v

alu

e2.4

77e-7

2.4

77e-7

3.6

51e-7

2.4

77e-7

16

●

●

GA−EAX VNRDGILS HHGILS

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0

0.190.24

●

●

●

●

●

●

% g

ap to

the

optim

al s

olut

ion

(a) Medium instance set

●●

●

GA−EAX VNRDGILS HHGILS GPR1R2

05

1015

20

0

10.410.93

15.69

●

●

●

●

●

●

●

●

% g

ap to

the

optim

al s

olut

ion

(b) Large instance set

Fig. 4. Boxplots of the normalized average objective values for the medium instanceset and large instance sets.

this figure, the average objective value favg of a given algorithm is normalizedaccording to the relation y = 100 ∗ (favg − fopt)/fopt, where fopt is the optimalvalue. The plots in Figure 4 show clear differences in the distributions of theaverage results between GA-EAX and each compared CTSP heuristic, whichfurther confirms the efficiency of the GA-EAX TSP solver with respect tothese dedicated CTSP heuristics.Table 6Statistical results for the GA-EAX TSP solver and three state-of-the-art CTSPalgorithms on the medium instance set and large instance set. Dominating valuesare indicated in bold.

GA-EAX VNRDGILS HHGILS GPR1R2

Medium instance set Optimal solutions 20/20 0/20 1/20 0/20

Average Gapbest/Gapavg(%) 0.00/0.00 0.18/0.30 0.21/0.40 0.39/0.73

Average time (s) 5.6 720.0 720.0 720.0

Large instance set Optimal solutions 15/15 0/15 0/15 0/15

Average Gapbest/Gapavg(%) 0.00/0.01 8.61/10.39 11.04/15.51 12.24/15.49

Average time (s) 30.3 1080.0 1080.0 1080.0

Table 6 summarizes the statistical results for each compared algorithm on thetwo sets of medium and large instances. The first row indicates the numberof optimal solutions found by each approach. The average percentage gap ofthe best/average result from the optimal result is provided in row ‘AverageGapbest/Gapavg’. Finally, row ‘Average time (s)’ provides the average run timein seconds for each algorithm. From Table 6, we observe that the GA-EAXsolver significantly outperforms the three CTSP algorithms on the mediumand large instances in terms of both the best and the average results. For thelarge instance set, the improvement gaps between the results of GA-EAX and

17

those of the CTSP methods are very high, ranging from 10.39% to 15.49%.Furthermore, in terms of the average run time, GA-EAX is about 30 to 130times faster than the CTSP algorithms. The above results thus indicate thatthe GA-EAX TSP solver has a strong dominance over current best performingCTSP approaches in the literature. Finally, the results of the Concorde TSPsolver and the LKH-2 solver reported in Section 4.3 indicate that these TSPsolvers also dominate the current best CTSP algorithms in the literature.

5 Conclusions

This paper presents the first large computational study on testing modernTSP solvers for solving the CTSP. According to the computational resultsfrom the exact Concorde TSP solver and the inexact LKH-2 and GA-EAXTSP solvers on two sets of medium and large CTSP benchmark instancesavailable in the literature (with up to 2000 vertices) and a new set of verylarge CTSP instances (with up to 5000 vertices), we can make the followingconclusions.

• The exact Concorde TSP solver can optimally solve all medium and largeCTSP instances, but fails to solve three very large instances with 5000 ver-tices in 24 hours. Its solution time increases considerably with the instancesizes.• Due to the clustering nature of the transformed instances, the powerful

inexact LKH-2 TSP solver does not perform well. LKH-2 reports a worseperformance both in terms of solution quality and computation time, com-pared with GA-EAX.• The GA-EAX solver performs remarkably well both in terms of solution

quality and computational efficiency, with a very high scalability. It canstably attain the optimal solutions for all medium and large CTSP instancesavailable in the literature with a short time.• The TSP solvers all dominate the current best performing CTSP heuristics

specially designed for the problem. This is particular true for the GA-EAXsolver, which is 30 to 130 times faster than the state-of-the-art CTSP heuris-tics to find much better results.

Finally, this study also indicates that the existing CTSP benchmark instancesin the literature are not challenging for modern TSP solvers even if they remaindifficult for the current CTSP algorithms.

18

Acknowledgment

This work is partially supported by the National Natural Science FoundationProgram of China [Grant No. 71401059, 71771099] and the China PostdoctoralScience Foundation [Grant No. 2019M662649].

References

[1] Applegate, D., Bixby, R., Chvatal, V., & Cook, W. (2001). Concorde TSP solverhttp://www.math.uwaterloo.ca/tsp/concorde/index.html.

[2] Applegate, D., Bixby, R., & Chvatal, V. (2007). Concorde TSP Solver. WilliamCook, School of Industrial and Systems Engineering, Georgia Tech.

[3] Applegate, D., Bixby, R., Chvatal, V., & Cook, W. (2006). The travelingsalesman problem: a computational study. Princeton University Press.

[4] Applegate, D., Cook, W., & Rohe, A. (2003). Chained Lin-Kernighan for largetraveling salesman problems. INFORMS Journal on Computing, 15(1), 82-92.

[5] Anily, S., Bramel, J., & Hertz, A. (1999). A 53-approximation algorithm forthe clustered traveling salesman tour and path problems. Operations ResearchLetters, 24(1-2), 29-35.

[6] Bao, X., & Liu, Z. (2012). An improved approximation algorithm for theclustered traveling salesman problem. Information Processing Letters, 112(23),908-910.

[7] Chisman, J. A. (1975). The clustered traveling salesman problem. Computers& Operations Research, 2(2), 115-119.

[8] Ding, C., Cheng, Y., & He, M. (2007). Two-level genetic algorithm for clusteredtraveling salesman problem with application in large-scale TSPs. TsinghuaScience and Technology, 12(4), 459-465.

[9] Dolan, E. D., & More, J. J. (2002). Benchmarking optimization software withperformance profiles. Mathematical Programming, 91(2), 201-213.

[10] Gendreau, M., Laporte, G., & Hertz, A. (1997). An approximation algorithmfor the traveling salesman problem with backhauls. Operations Research, 45(4),639-641.

[11] Ghaziri, H., & Osman, I. H. (2003). A neural network algorithm for the travelingsalesman problem with backhauls. Computers & Industrial Engineering, 44(2),267-281.

[12] Guttmann-Beck, N., Hassin, R., Khuller, S., & Raghavachari, B. (2000).Approximation algorithms with bounded performance guarantees for theclustered traveling salesman problem. Algorithmica, 28(4), 422-437.

19

[13] Helsgaun, K. (2000). An effective implementation of the Lin-Kernighantraveling salesman heuristic. European Journal of Operational Research, 126(1),106-130.

[14] Helsgaun, K. (2009). General k-opt submoves for the Lin-Kernighan TSPheuristic. Mathematical Programming Computation, 1(2-3), 119-163.

[15] Hains, D., Whitley, D., & Howe, A. (2012). Improving Lin-Kernighan-Helsgaunwith crossover on clustered instances of the TSP. In International Conferenceon Parallel Problem Solving from Nature (pp. 388-397). Springer, Berlin,Heidelberg.

[16] Hoos, H. H., & Stutzle, T. (2014). On the empirical scaling of run-time forfinding optimal solutions to the travelling salesman problem. European Journalof Operational Research, 238(1), 87-94.

[17] Johnson, D. S., & McGeoch, L. A. (2007). Experimental analysis of heuristicsfor the STSP. In The Traveling Salesman Problem and its Variations (pp. 369-443). Springer, Boston, HEA.

[18] Jongens, K., & Volgenant, T. (1985). The symmetric clustered travelingsalesman problem. European Journal of Operational Research, 19(1), 68-75.

[19] Laporte, G., & Palekar, U. (2002). Some applications of the clustered travellingsalesman problem. Journal of the Operational Research Society, 53(9), 972-976.

[20] Laporte, G., Potvin, J. Y., & Quilleret, F. (1997). A tabu search heuristic usinggenetic diversification for the clustered traveling salesman problem. Journal ofHeuristics, 2(3), 187-200.

[21] Lin, S., & Kernighan, B. W. (1973). An effective heuristic algorithm for thetraveling-salesman problem. Operations Research, 21(2), 498-516.

[22] Lokin, F. C. J. (1979). Procedures for travelling salesman problems withadditional constraints. European Journal of Operational Research, 3(2), 135-141.

[23] Mestria, M., Ochi, L. S., & de Lima Martins, S. (2013). GRASP with pathrelinking for the symmetric Euclidean clustered traveling salesman problem.Computers & Operations Research, 40(12), 3218-3229.

[24] Mestria, M. (2016). A hybrid heuristic algorithm for the clustered travelingsalesman problem. Pesquisa Operacional, 36(1), 113-132.

[25] Mestria, M. (2018). New hybrid heuristic algorithm for the clustered travelingsalesman problem. Computers & Industrial Engineering, 116, 1-12.

[26] Martin, O., Otto, S. W., & Felten, E. W. (1991). Large-step Markov chainsfor the traveling salesman problem. Oregon Graduate Institute of Science andTechnology, Department of Computer Science and Engineering.

[27] Nagata, Y.,& Kobayashi, S. (1997) Edge assembly crossover: A high-powergenetic algorithm for the traveling salesman problem. Proc. 7th Internat. Conf.Genetic Algorithms (Morgan Kaufmann,San Francisco), 450-457.

20

[28] Neto, D. (1999). Efficient Cluster Compensation For Lin-Kernighan Heuristics.PhD thesis, University of Toronto.

[29] Nagata, Y., & Kobayashi, S. (2013). A powerful genetic algorithm using edgeassembly crossover for the traveling salesman problem. INFORMS Journal onComputing, 25(2), 346-363.

[30] Potvin, J. Y., & Guertin, F. (1996). The clustered traveling salesman problem:A genetic approach. In Meta-Heuristics (pp. 619-631). Springer, Boston, HEA.

[31] Siqueira, A. S., da Silva, R. C., & Santos, L. R. (2016). Perprof-py: A pythonpackage for performance profile of mathematical optimization software. Journalof Open Research Software, 4(1).

[32] Reinelt, G. (1991). TSPLIB-A traveling salesman problem library. ORSAJournal on Computing, 3(4), 376-384.

[33] Weintraub, A., Aboud, J., Fernandez, C., Laporte, G., & Ramirez, E. (1999).An emergency vehicle dispatching system for an electric utility in Chile. Journalof the Operational Research Society, 50(7), 690-696.

[34] Whitley, D., Hains, D., & Howe, A. (2010). A hybrid genetic algorithmfor the traveling salesman problem using generalized partition crossover. InInternational Conference on Parallel Problem Solving from Nature (pp. 566-575). Springer, Berlin, Heidelberg.

[35] Watson, J., Ross, C., Eisele, V., Denton, J., Bins, J., Guerra, C. D. W. A. H.,& Howe, A. (1998). The traveling salesman problem, edge assembly crossover,and 2-opt. In International Conference on Parallel Problem Solving from Nature(pp. 823-832). Springer, Berlin, Heidelberg.

21