Integer and Combinatorial Optimization: Algorithm...

Integer and Combinatorial Optimization:Algorithm Complexity

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

January 2019

Mitchell Algorithm Complexity 1 / 23

How run-time grows as problem size grows

Outline

1 How run-time grows as problem size grows

2 Polynomial time algorithms and the class PExamples of problems in P



Big-O notation

DefinitionLet f and g be functions from the set of integers or the set of reals tothe set of real numbers. The function f (k) is O(g(k)) whenever thereexist positive constants C and k ′ such that

|f (k)| ≤ C|g(k)| ∀ k ≥ k ′.



Size of an instance

The size of an instance is the number of binary characters required tostore the problem.For example, given a positive integer x , we set p = blog2(x)c. Then

2p ≤ x < 2p+1 and x =

p∑i=0

δi2i for some binary δi .

For example, 42 = 0× 20 + 1× 21 + 0× 22 + 1× 23 + 0× 24 + 1× 25.So we need O(log(x)) bits to store x .

We don’t need to specify the base for the logarithm here, because

if x > 0 and y = loga(x), z = logb(x) then y = z loga(b).



Data representation

The major classification of algorithm efficiency is insensitive to thechoice of data representation, for the most part.

There are two restrictions:1 The alphabet used to represent the data must contain at least two

symbols. So the space to store a positive integer x is O(log(x)),instead of x with a unary alphabet.

2 The “data” can’t contain information that requires a lot ofcalculation. For example, can’t store the length of all tours as partof the representation of a traveling salesman problem.



Data representation

The major classification of algorithm efficiency is insensitive to thechoice of data representation, for the most part.

There are two restrictions:1 The alphabet used to represent the data must contain at least two

symbols. So the space to store a positive integer x is O(log(x)),instead of x with a unary alphabet.

2 The “data” can’t contain information that requires a lot ofcalculation. For example, can’t store the length of all tours as partof the representation of a traveling salesman problem.



Computation time

The time required for an elementary operation (addition, subtraction,multiplication, division, comparison) is counted as unit time.

(Need to be a little careful if multiply very large numbers together.)



Instances

Perfect matching, linear programming, and node packing are allexamples of optimization problems.

An optimization problem X consists of an infinite number of instancesd1,d2, . . . where the data for instance di is given by a binary string oflength l(di).

For example, an instance of a perfect matching problem is given bylists of the vertices and edges of the graph G = (V ,E).



Worst case running time

Let A be an algorithm the solves every instance of X in finite time.

Let gA(di) be the running time of A on instance di .

Measure performance of A on X using the worst-case running time, foreach problem size k :

fA(k) := max{gA(di) : l(di) = k}.

Advantages of using worst-case performance measure:Gives a guarantee of performance of the algorithmIndependent of the probability distribution of the instances.Appears to be the easiest to analyze.



















Klee-MintyFor example, the simplex algorithm solves any linear optimizationproblem in finite time (provided we use an anti-cycling rule).

In the worst case, it may require O(2n) iterations to solve a problem inn nonnegative variables with n inequality constraints (namely, theKlee-Minty cube).

x1

x2

x3

(0,0,0)(0,100,0)

(0,0,10000)

(1,0,0)

(1,80,0)

(0,100,8000)

(1,80,8200)(1,0,9800)


Polynomial time algorithms and the class P

Outline





Feasibility problems

For example, an instance of the perfect matching feasibility problemconsists of a graph G = (V ,E) and asks the question “Does there exista perfect matching on this graph”? The answer is either Yes or No.

DefinitionA feasibility problem X consists of a set of instances D and a set offeasible instances F ⊆ D. Given an instance d ∈ D, determinewhether d ∈ F. The answer is either Yes or No.



Binary searchMany instances of optimization problems can be solved using binarysearch on a sequence of instances of feasibility problems.

For example, can solve the 0− 1 linear optimization problem

minx{cT x : Ax ≥ b, x ∈ Bn}

by repeatedly solving the 0− 1 linear feasibility problem

Does there exist x ∈ Bn with Ax ≥ b, cT x ≤ z?

At each iteration, we have upper and lower bounds on the optimalvalue z∗. The initial bounds on z∗ can be taken to be

zUB =∑

j:cj>0

cj , zLB =∑

j:cj<0

cj

since x is binary. We solve the feasibility problem withz = 1

2(zLB + zUB). We assume all the data is integral, so the optimal

value is integral, so we can maintain integral values for zLB, zUB, and z.Mitchell Algorithm Complexity 12 / 23


Binary search algorithm

1 Initialize zUB, zLB.2 If zLB = zUB, STOP and return the optimal value

z∗ = zLB = zUB.3 Let z =

⌊12(z

LB + zUB)⌋.

4 Solve the 0− 1 linear feasibility problem


5 If answer is YES: update zUB ← z, return to Step 2.6 If answer is NO: update zLB ← z + 1, return to Step 2.

Let z0 be the initial value of zUB − zLB. The number of iterations of thealgorithm is O(log2(z

0)), since the difference between the bounds ishalved at each iteration.






⌊12(z

LB + zUB)⌋.











⌊12(z

LB + zUB)⌋.











⌊12(z

LB + zUB)⌋.











⌊12(z

LB + zUB)⌋.











⌊12(z

LB + zUB)⌋.








The Class P

DefinitionAlgorithm A is a polynomial time algorithm for the feasibility problemX if fA(k) is O(kp) for some fixed p.

P denotes the class of feasibility problems that can be solved inpolynomial time.

Problem X ∈ P if and only if there is a polynomial time algorithm for X.


Polynomial time algorithms and the class P Examples of problems in P

Outline





Minimum spanning tree with upper boundGiven a graph G = (V ,E), edge weights we for e ∈ E , and an integerz, does there exist a spanning tree with total weight no greater than z?

Use a greedy algorithm.

1

2

3

4

5

6

7

8

2

351 5

4

67

4

6

7

64

8

3 5

Weight ≤ 25?

Solution:

1

2

3

4

5

6

7

8

2

31

4

6

4

3

YES (weight 23)



Minimum spanning tree with upper boundGiven a graph G = (V ,E), edge weights we for e ∈ E , and an integerz, does there exist a spanning tree with total weight no greater than z?

Use a greedy algorithm.

1

2

3

4

5

6

7

8

2

351 5

4

67

4

6

7

64

8

3 5

Weight ≤ 25?

Solution:

1

2

3

4

5

6

7

8

2

31

4

6

4

3

YES (weight 23)



Shortest path with upper bound

Given a graph G = (V ,E), edge weights we for e ∈ E , source node sand sink node t , and an integer z, does there exist a path from s to twith total length no greater than z?Use Dijkstra’s algorithm if edge weights are nonnegative.

s

2

3

4

tcs2 = 2, xs2 = 1

cs3 = 6

c24 = 1, x24 = 1

c32 =5

c3t = 3, x3t = 1

c43 = 2, x43 = 1

c4t =6

Path ≤ 6? NO



Perfect matching feasibility problemGiven a graph G = (V ,E), does there exist a perfect matching?Use Edmonds blossom (alternating path) algorithm.

1

2

3

4

5

6

No alternating path between unmatched vertices 2 and 6



Linear programming with bound

Given m × n matrix A, vectors b ∈ IRm and c ∈ IRn, and a scalar z,does there exist a nonnegative x ∈ IRn satisfying Ax = b and cT x ≤ z?

Use the ellipsoid algorithm or some interior point algorithms.



Exponential functions

DefinitionThe function f (k) is exponential if there exist positive constantsc1, c2 > 0, constants d1,d2 > 1, and a positive constant k ′ such that

c1dk1 ≤ f (k) ≤ c2dk

2 ∀ k ≥ k ′.

For example, let b be the largest integer for an instance of anoptimization problem X , so this requires storage O(log2(b)).

Assume the total storage is also O(log2(b)).

If algorithm A runs in time O(b) then it requires exponential time forthis instance, since

b = 2log2(b).



The knapsack problem

The knapsack problem max{cT x : aT x ≤ b, x binary} can be solvedin O(b) iterations using dynamic programming.

However, the storage requirement is only O(log(b)), assuming b is thelargest integer in the problem definition.

So the runtime is exponential in the storage requirement.



Max Flow

Consider a max flow problem on the following graph. We seek tomaximize flow from node 1 to node 4. The arc capacities are indicated.

1

2

3

4M

M

1M

M



Solve with augmenting path method

1

2

3

4t t

t

1

2

3

4

(M, 1)

(M, 0)

(1, 1)

(M, 0)

(M, 1)

take t = 1

(ue, xe)

Next iteration:

1

2

3

4t

−t

t

1

2

3

4

(M, 1)

(M, 1)

(1, 0)

(M, 1)

(M, 1)

take t = 1

(ue, xe)

Need 2M iterations to get to optimal solution, which is exponential inthe storage requirement. To get polynomial algorithm, need to selectaugmenting path with fewest edges.




1

2

3

4t t

t

1

2

3

4

(M, 1)

(M, 0)

(1, 1)

(M, 0)

(M, 1)

take t = 1

(ue, xe)

Next iteration:

1

2

3

4t

−t

t

1

2

3

4

(M, 1)

(M, 1)

(1, 0)

(M, 1)

(M, 1)

take t = 1

(ue, xe)





1

2

3

4t t

t

1

2

3

4

(M, 1)

(M, 0)

(1, 1)

(M, 0)

(M, 1)

take t = 1

(ue, xe)

Next iteration:

1

2

3

4t

−t

t

1

2

3

4

(M, 1)

(M, 1)

(1, 0)

(M, 1)

(M, 1)

take t = 1

(ue, xe)





1

2

3

4t t

t

1

2

3

4

(M, 1)

(M, 0)

(1, 1)

(M, 0)

(M, 1)

take t = 1

(ue, xe)

Next iteration:

1

2

3

4t

−t

t

1

2

3

4

(M, 1)

(M, 1)

(1, 0)

(M, 1)

(M, 1)

take t = 1

(ue, xe)



Date post:	30-Jun-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Integer and Combinatorial Optimization: Algorithm...

Documents