Module on Large-ScaleInteger Programming & Combinatorial
Optimization
Module on Large-ScaleInteger Programming & Combinatorial
Optimization
Traveling salesman problemFacility locationNetwork design
Traveling salesman problemFacility locationNetwork design
Three LecturesThree Lectures
Games/ChallengesApplications, Models, and
Solution Methods
Games/ChallengesApplications, Models, and
Solution Methods
Traveling Salesman Problem
Traveling Traveling Salesman Salesman ProblemProblem
1Thomas L. MagnantiThomas L. Magnanti
AgendaAgenda
OriginsElectronic Component PlacementTSP ModelTurbine Vane PlacementOther Applications of TSPSolution Methods
Heuristic MethodsLagrangian relaxation (bounding methods)
Some large scale instances (computations)
OriginsElectronic Component PlacementTSP ModelTurbine Vane PlacementOther Applications of TSPSolution Methods
Heuristic MethodsLagrangian relaxation (bounding methods)
Some large scale instances (computations)
2
Icosian game Icosian Icosian game game
3
William Rowan HamiltonWilliam Rowan Hamilton
Icosian game Icosian Icosian game game
Hamiltonian Path and the TSPHamiltonian Path and the TSPHamiltonian Path and the TSP4
1
20
19
2
3 456 7
89
10
131415
1617
18
11 12
Interest in Traveling Salesman Problem (TSP)
Interest in Traveling Salesman Problem (TSP)
Arises in Many ApplicationsAlluring (Captures Imagination)Notoriously Difficult to SolveHas Attracted Best Minds in Math/CS/OR for 40 Years
Arises in Many ApplicationsAlluring (Captures Imagination)Notoriously Difficult to SolveHas Attracted Best Minds in Math/CS/OR for 40 Years
5
The Traveling Salesman Problem and Electronics Assembly
The Traveling The Traveling Salesman Salesman Problem and Problem and Electronics Electronics AssemblyAssembly
6
FeedersFeedersFeeders
Placement LocationsPlacement LocationsPlacement Locations
7
Placement ProblemPlacement Problem
8
Placement SequencePlacement Sequence
Home Position
Home Home PositionPosition
9
Traveling Salesman InterpretationTraveling Salesman Interpretation
xjk =1 if placement k follows placement j
0 otherwise
xxjkjk ==11 if placement k follows if placement k follows placement jplacement j
0 otherwise0 otherwise
cjk cost of placing module k after placing module j
ccjkjk cost of placing module cost of placing module k after placing module jk after placing module j
10
Model IngredientsModel Ingredients
Minimize Σj Σk cjkxjkMinimize Minimize ΣΣj j ΣΣkk ccjkjkxxjkjk
Σkxjk = 1 for each jΣj xjk = 1 for each k
xjk > 0 for all j & k
ΣΣkkxxjkjk == 1 for each j1 for each jΣΣjj xxjkjk == 1 for each k1 for each k
xxjkjk >> 0 for all j & k0 for all j & k
subject tosubject tosubject to
Proper TSP Model?Proper TSP Model?Proper TSP Model?11
Assignment ProblemAssignment Problem
12
Subtour SolutionSubtour Solution
Assignment Model
+Subtour Breaking Constraints
Assignment ModelAssignment Model
++Subtour Breaking ConstraintsSubtour Breaking Constraints
Σj∈S Σk∈S xjk < |S| - 1 for all subsets S of nodes {2,3,…,n}
Σ Σjj∈∈SS Σ Σkk∈∈SS xxjkjk << |S| |S| -- 1 for all subsets 1 for all subsets S of nodes {2,3,…,n}S of nodes {2,3,…,n}
Proper TSP Model?Proper TSP Model?Proper TSP Model?1313
TSP ModelTSP Model
Implications for IC InsertionsImplications for IC Insertions
Manual Designs ⇒ Long Time10 hours for 70 to 100 components
Better Solutions10-25% improvements by optimization
Manual Designs ⇒ Long Time10 hours for 70 to 100 components
Better Solutions10-25% improvements by optimization
Other Applications SimilarOther Applications SimilarOther Applications Similar23
Feeder PlacementFeeder Placement
Modeling?Solution Methods?
HeuristicOptimization
Modeling?Solution Methods?
HeuristicOptimization
24
Other Applications of TSP?
Other Other Applications Applications of TSP?of TSP?
??
??
14
Machine SchedulingMachine SchedulingMachine Scheduling
Machine “Visits” JobsTravel Time = Set up TimeMachine “Visits” JobsMachine “Visits” JobsTravel Time = Set up TimeTravel Time = Set up Time
1515
Other Applications of TSPOther Applications of TSP
Other ApplicationsOther Applications
Analyzing the structure of crystalsMaterial handling in a warehouseClustering of data arraysCutting stock problemsGenome sequencingStarlight interferometer satellite positioningDNA universal stringsCollecting coins from payphones
Analyzing the structure of crystalsMaterial handling in a warehouseClustering of data arraysCutting stock problemsGenome sequencingStarlight interferometer satellite positioningDNA universal stringsCollecting coins from payphones
Solving the TSPSolving the TSP
2 5
3 0
3 5
4 0
4 5
5 0
7 08 09 01 0 01 1 01 2 0
Lo ng itude
L a t i t u d e
S E
P O
S F
LA
D E
TU
DA
HU N O
ME
A T
WA
P HNY
B O
P I
C L
D T
CNINS L
K S
O M
MN
CH
ML
2 5
3 0
3 5
4 0
4 5
5 0
7 08 09 01 0 01 1 01 2 0
Lo ng itude
L a t i t u d e
S E
P O
S F
LA
D E
TU
DA
HU N O
ME
A T
WA
P HNY
B O
P I
C L
D T
CNINS L
K S
O M
MN
CH
ML
2020
26 City Traveling Salesman Problem26 City Traveling Salesman Problem
LatitudeLatitude
Finding a Good SolutionFinding a Good Solution
How?How good is good?
LP boundsCombinatorial bounds
How?How?How good is good?How good is good?
LP boundsLP boundsCombinatorial boundsCombinatorial bounds
21
Solution MethodsSolution Methods
HeuristicsGrowing solutions: nearest neighbor, farthest neighbor, nearest insertionImprovement procedures: 2-opt, 3-opt
Optimization MethodsBounding: LP relaxation, Lagrangian dual Polyhedral methods (cutting planes)
HeuristicsGrowing solutions: nearest neighbor, farthest neighbor, nearest insertionImprovement procedures: 2-opt, 3-opt
Optimization MethodsBounding: LP relaxation, Lagrangian dual Polyhedral methods (cutting planes)
HeuristicsHeuristics
Build TourNearest NeighborNearest/Farthest Insertion
Improve TourSwapping Edges
Build TourNearest NeighborNearest/Farthest Insertion
Improve TourSwapping Edges
22
Insertion HeuristicsInsertion Heuristics
1
5 2
3
4 6
1
5 2
3
4 6
x
x
NearestInsertionNearestInsertion
FarthestInsertionFarthestInsertion
Tour ImprovementsTour Improvements
1
5 2
3
4 6
1
5 2
3
4 6
2-optEliminate 2 arcs and reconnect
Choose bestalternative at each step
2-optEliminate 2 arcs and reconnect
Choose bestalternative at each step
Exploiting Embedded StructureExploiting Embedded Structure
{ }
1 1
1
1
2 2
minimize
subject to 1 for all 1, 2, ,
1 for all 1, 2, ,
2
1 for all 2,3, ,
n nij iji j
nijj
niji
n niji j
iji S j S
ij
c x
x i n
x j n
x n
x S S n
x
= =
=
=
= =
∈ ∈
= =
= =
= −
≤ − ⊆
≥
∑ ∑∑∑
∑ ∑∑ ∑
K
K
K
0 and integerRedundant ConstraintRedundant Constraint
Minimum spanning tree on nodes 2 to nMinimum spanning tree on nodes 2 to n
Underlying StructureUnderlying Structure
1
5 2
3
4 6
Decomposed TourDecomposed Tour
Arc in and outof node 1
Arc in and outof node 1
Path (hence tree) on nodes 2 to nPath (hence tree) on nodes 2 to n
Lagrangian RelaxationLagrangian Relaxation
1 1 2 1
2 1
11
11
2 2
( , ) minimize 1
1
subject to 1
1
2
n n n nij ij i iji j i j
n nj ijj i
njj
nii
n niji j
ij
L u v c x u x
v x
x
x
x n
x
= = = =
= =
=
=
= =
= + − + −
=
=
= −
∑ ∑ ∑ ∑∑ ∑
∑∑
∑ ∑{ }1 for all 2,3, ,
0 and integeri S j S
ij
S S n
x∈ ∈
≤ − ⊆
≥
∑ ∑ K
Improving Lagrangian Lower BoundImproving Lagrangian Lower Bound
1
5 2
3
4 6
Solution x* to Lagrangian RelaxationSolution x* to Lagrangian Relaxation
Too many arcs outIncrease u5c5j+ u5 + vj more expensive
Too few arcs outDecrease u6c6j+ u6 + vj less expensive
Solution Approach (Dual Ascent)Solution Approach (Dual Ascent)
Solve L(u,v)Use costs cij+ vi + ujSelect least cost arc out of and into node 1Find minimal spanning tree on nodes 2 to n (easy)
Let x* be optimal solution to L(u,v)Using x* alter u and v to increase lower bound L(u,v)Iterate to solve Lagrangian dual
d = max u,v L(u,v)
Solve L(u,v)Use costs cij+ vi + ujSelect least cost arc out of and into node 1Find minimal spanning tree on nodes 2 to n (easy)
Let x* be optimal solution to L(u,v)Using x* alter u and v to increase lower bound L(u,v)Iterate to solve Lagrangian dual
d = max u,v L(u,v)
Dual Ascent in GeneralDual Ascent in General
* = minimize subject to
v cxAx bx X
=∈
ComplicatingConstraintsComplicatingConstraints
[ ]( ) minimize subject to L u cx u Ax b
x X= + −
∈
{ }: , 0X x Dx d x= = ≥e.g.,
Lagrangian Dual
Dual Ascent in GeneralDual Ascent in General
If ( ) ( ) ( )k k k kL u cx u u Ax u b = + − ( ) ( )k kL u Ax u b∇ ≈ −
1 ( )k k kku u Ax u bθ+ = − −
2
( ) *
( )
kk
k k
cx u v
Ax u b
λθ
− =−
Dual Ascent ConvergenceDual Ascent Convergence
Theorem
If
and are bounded,
then c(xk) converges to v*.
1 20 2kε λ ε< ≤ ≤ −
( )kAx u b−
In practice, continue with fixed λk except half λk after some number (50?) of iterations if c(xk) doesn’t decrease
Solving Minimum Spanning TreeSolving Minimum Spanning Tree
Order arcs from smallest to highest costsChoose arcs in order
If arc does not form an undirected circuit with arcs already chosen, then choose arc;Otherwise eliminate arc from consideration
Order arcs from smallest to highest costsChoose arcs in order
If arc does not form an undirected circuit with arcs already chosen, then choose arc;Otherwise eliminate arc from consideration
Greedy (Kruskal Algorithm)Greedy (Kruskal Algorithm)
Amazing Facts!Amazing Facts!
Greedy algorithm (and several variants) solves the minimum spanning problem
The linear programming relaxation of the formulation we have given for the minimum spanning problem always has an integer solution (the underlying polyhedron has integer extreme points)
Greedy algorithm (and several variants) solves the minimum spanning problem
The linear programming relaxation of the formulation we have given for the minimum spanning problem always has an integer solution (the underlying polyhedron has integer extreme points)
Solving Network Flow ProblemsSolving Network Flow Problems
Giden (Graphical Environment for Network Optimization)
Demonstration of • Minimum spanning trees• Maximum flows• Minimum cost flows
(Based on Ahuja, Magnanti, Orlin’s book Network Flows)(Based on Ahuja, Magnanti, Orlin’s book Network Flows)
2103 hole printed circuitboard example
2103 hole printed circuitboard example
11,849 hole printed circuitboard example
11,849 hole printed circuitboard example
11,849 hole printed circuitboard solution
13,094,345 seconds total on 55 CPUs
11,849 hole printed circuitboard solution
13,094,345 seconds total on 55 CPUs
Traveling salesman problem through 15,112 cities in Germany
22.6 years ofcomputation on network of 110 Processors
See http://www.math.princeton.edu/tsp/
Traveling salesman problem through 15,112 cities in Germany
22.6 years ofcomputation on network of 110 Processors
See http://www.math.princeton.edu/tsp/
Today’s LessonsToday’s Lessons
Traveling salesman problem arises in numerous applicationsProblem is a large-scale integer programMany heuristic methods: often find good solutionsLagrangian dual (bounding) exploits special problem structure (embedded minimal spanning tree) MST is easy to solve
Traveling salesman problem arises in numerous applicationsProblem is a large-scale integer programMany heuristic methods: often find good solutionsLagrangian dual (bounding) exploits special problem structure (embedded minimal spanning tree) MST is easy to solve
We did NOT examine polyhedral (cutting plane) methodsWe did NOT examine polyhedral (cutting plane) methods
