Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

1

Linear Programming and its Usage in Linear Programming and its Usage in Approximation Algorithms for NP Hard Approximation Algorithms for NP Hard

Optimization ProblemsOptimization Problems

M. Reza Rahimi, M. Reza Rahimi, November 2005.November 2005.

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

2

Outline

• Introduction

• Linear Programming Overview

• NP Complexity Class

• Approximation Algorithms

• Case Study1: Minimum Weight Vertex Cover

• Case Study2: MAXSAT Problem

• Conclusion


3

Introduction

• One of the most challenging problem in complexity theory is problem P vs. NPP vs. NP.

• Research on this open problem leads researchers to think of new methods and different approaches.

• For example PCP, IP, approximation algorithms,...

• For some problems it is proved that there does not exist any suitable approximation unless P=NP.


4

• So it seems that research on approximation algorithms may lead us to good results about P vs. NP.

• General Method for Approximation Algorithms of NP Hard Optimization is Greedy Method.Greedy Method.

• But we must search for Suitable FrameworkSuitable Framework for studying NP problems.


5

• Linear ProgrammingLinear Programming was this brake through method.

• In this talk I focus on LP and its usage in NP Hard Optimization Problems.


6

Linear Programming Overview

• General formulation of linear programming is presented as follow:

Maximize C1X1 + C2X2 + … + CdXd

Subject to A1,1X1 + … + A1,dXd ≤ b1

A2,1X1 + … + A2,dXd ≤ b2

…

An,1X1 + … + An,dXd ≤ bn

All calculation and Numbers are on Real Numbers.All calculation and Numbers are on Real Numbers.


7

• There are several Polynomial Algorithms for this problem such as EllipsoidEllipsoid and InteriorInterior pointpoint methodmethod.

• I neglect talking about them.

• For these algorithms please refer to optimization references.


8

NP Complexity Class

• NP Complexity Class is only related to Decision Problem.

• For example:

• So for any optimization problem we consider its related decision problem.

• This will give us intuition about its hardness.

}.assignment satisfying a is :{ SAT


9

• We know the following definition about NP:

• We can look at this process like this:

rejects. y)V(x, and )xP(y y, L x2.

accepts. y)V(x, and )xP(y y, L x1.

, x P(.), P, V(.,.) NPL

x x

yFind Certificate Y V(X,Y)Y

X X


10

• Example:

• There is one another important concept in complexity which is Reduction.Reduction.

.assignment trueno is Then there 2)if

time.polynomialin it check could and ,

assignment Truegiven are We if 1)

:Because NP

}.assignment satisfying a is :{

0

0

SAT

SAT

SAT

SAT


11

.)( Function) Time l(Polynomia *

:

*LxfLxfLL

Definition

P

*L

L

L

L *L

f

f

Fig1: Graphical Diagram of Reduction Concept


12

)xx(x)xxx()xx(x

:examplefor form in this is 3(

.3SAT

: that show we

e}.satisfiabl and cnf-3 is |{-3

}.assignment esatisfiabl is |{

:

321321321

P

CNF

CNF

CNF

SAT

Example


13

)()( 2321 xxxx

-x1

->

x2 x3 x4

^

<->y1 y2

le. truth tabusing

form OR-3 into converted becan sparanthesieach Now

)())(())(( 21243211 yyyxxxxy


14

10 ,

1)1(1

)1(

:Proof

. 3

: thatshow willWe

. Obviously

}. ,

matrices,integer are and |,{

:

321321

321321

111

11

ii

P

mnnmn

n

mnmmnm

XX

XXXXXX

XXXXXX

GPROGRAMMININTEGERCNF

NPGPROGRAMMININTEGER

bXAX

bAbAGPROGRAMMININTEGER

Example


15

• We conclude this section by the following theorem:

Every language in NP is polynomial time reducible to SAT language.

• Language like SAT, INTEGER-PROGRAMMING, Hamiltonian Cycle, and 3-CNF are also the same and they belong to NP-Complete set.


16

Approximation Algorithms

• The following describes the connection of NP optimization Complete problem and its related optimization problem.

minimun. iscost itswhich

Cyclen Hamiltonia thefind ,Ggraph weightednegativenon complete have we

:

:

b}. equal

or than less iscost itssuch that Cyclen Hamiltonia exists

thereandgraph weightednegativenon complete is G |,{

:

nn

nn

TSPOP

VersiononOptimizati

bGTSP

VersionDecision

nn


17

• Bound of Approximation:Bound of Approximation:• Suppose that problem P has an optimum solution with cost C-

op. Algorithm X solves problem P

with bound ρ(n) if it finds feasible solution with cost C such as:

max {c/c-op, c-op/c} ≤ ρ(n).

• Note that with this definition we always have ρ(n) ≥1.

• In the following I try to show that sometimes, finding good approximation for optimization problems is very hard.


18

• Theorem: Theorem:

It does not exist any polynomial timepolynomial time approximation algorithm with Polynomial BoundPolynomial Bound for OPT-TSPOPT-TSP unless P=NP.P=NP.

• Proof:

we prove that if such an algorithm exists then we can Solve Hamiltonian Cycle in Polynomial Time.

For each graph G with n vertex convertIt into weighted complete graph G1 as the following procedure: 1) Assign 1 to each of its edge.2) Assign nρ(n) to the other edges.

G has Hamiltonian Cycle Iff

OPT-TSP(G1)=n


19

• I think that it’s now time to study Randomized Rounding technique for solving optimization problems.

• I explore it with two examples.


20

Case Study1: Minimum Weight Vertex Cover

• Now we study randomized roundingrandomized rounding method in approximation algorithms with an example.

• Definition:• Vertex Cover:Vertex Cover: In undirected graph G=(V,E),G=(V,E), Set AA of

vertices is said vertex cover if

for every (u,v) (u,v) εε EE then u u εε A or v A or v εε AA . .

• Minimum Weight Vertex Cover:Minimum Weight Vertex Cover: In undirected graph G=(V,E)G=(V,E) which each vertex has w(v)w(v) as positive weight Set AA of vertices is said minimum weight vertex cover if for every (u,v) (u,v) εε EE then u u εε A or v A or v εε A and w(A) is the least.A and w(A) is the least.


21

Fig2: Vertex Cover


22

• It is proved that decision version of this optimization problem is NP Hard.

• At the first step we model it as integer programming.

1)(0

1)()( Vv,Vvw(v)x(v)min

:Method gProgramminInteger

0. x(v)else

1.x(v)SetCover Vertex Weight Minimum vif

{0,1}V:x

vx

uxvxu


23

• As it was known it is believed that there is no polynomial time algorithm for integer programming.

• We must think about another method.

• We investigate its related linear programming.

1)(0

1)()( Vv,Vvw(v)x(v)min

:Method gProgramminLinear

[0,1]V:x

vx

uxvxu


24

• The Last method is called RelaxationRelaxation to linearlinear programmingprogramming.

• It’s now time to state the approximation

algorithm.

C.Return 6)

{v}CC then 5)

/*Method Rounding * /0.5(v)x if do 4)

Veach vfor 3)

g.programminlinear osolution t optimalan x Compute 2)

C 1)

w)Cover(G,Vertex Min Weightion Approximat


25

2)C(Z)C(5.0

).(5.05.0)(5.0)(

)()()()(Z

.cover vertex is C Obviously

).C(Z have weObviously

g.programminlinear theof valueoptimal The ::Z

g.programmininteger ofset optimal real The::C

algorithm. ofsoulution The ::C

:Proof

**

5.0)(,

5.0)(,

*

**

*

*

ww

Cwvwvw

vxvwvxvw

w

CvvxVv

Vv vxVv


26

• HastadHastad proved that there is no Polynomial time algorithm for vertex cover that achieves an approximation better that 1.161.16 unless P=NP (1997).

• Now we consider another example.


27

Case Study2: MAXSAT ProblemCase Study2: MAXSAT Problem

trueis Cj

343213

243212

143211

432143214321

trueis Cj

jj

j

j

wMax

w)xxx(xC

w)xxxx(C

w)xxx(xC

).xxx)(xxxxx)(xxx(x

:Example

w

:maximizes that variables toassignment Find

.0 weight Whas C Clauseeach with Formula CNF-KGiven

:MAXSAT


28

• We state the following randomized algorithm for this problem. (Johnson 1974).

bound.ion approximat )]2

1-[(1 has algorithm preceding The

:Claim

return 2)

0.5.y probabilit with 1or 0 to

assign randomly ,...x xlesfor variab 1)

1k

C

n1

J

istrue

jw


29

1k

kC

jjC

jj trueis C

j

kj

kj

Cjj

trueis Cj

jj

)]2

1-1[(

.)2

1-1(}E{Iw}Iw{}w{

).2

1() 0p(I

).2

1-1() 1p(I

Iww

.0not if esatisfiablC clause if 1I

: variblesrandom following theDefine

:Proof

jjj

jj

jCjwEE


30

• We can extend Johnson algorithm as follow:

same. just the is proof The

))1(1(}w{

:Claim

return 2)

p-1 and py probabilit with 0or 1 to

assign tly independen andrandomly ,...x xlesfor variab 1)

trueis Cj

C

ii

n1

j

J

iij xif

ixif

iC

j

istruej

ppwE

w


31

• Now we model it with IP and relax it.

z,y 1z0

1y0

z)y-(1y

zwMax

:MAXSATfor Model gProgramminInteger

jij

i

j x if

i xif

i

Cjjj

ii

jC

z,y 1z0

1y0

z)y-(1y

zwMax

:MAXSATfor Relaxation

jij

i

j x if

i xif

i

Cjjj

ii

jC

RelaxationRelaxation


32

ratio.ion approximat 1/e)-(1 has algorithm above The

:Claim

))1

1(1()(y)y-(1-1

have we

Literals,k with C clauseany for and LP to)z,(ysolution feasibleany for

:Lemma

return )3

.yp with algorithmJohnson extended use 2)

).z,(y vector find and LP theSolve 1)

Algorithm

1-

x ifi

xifi

j**

i*

i

**

ii

jk z

k


33

1**

***

)/11(

))1

1(1())1

1(1(

))1

1(1(})()1(1{

})()1(1{

:Proof

e

Zk

Zk

zwk

yyw

ppw

IPk

LPk

Cjj

k

x xii

Cj

x xii

Cj

ji ij

i ij


34


35

• Algorithm

1. Compute the value from Johnson algorithm (w1).

2. Compute the value from LP Based algorithm (w2).

3. Return MAX(w1,w2).


36

Conclusion

• In this talk the randomized rounding method is explored with two example.

• This new method opens new insight into approximation algorithms and complexity theory.

The EndThe End

Date post:	23-Jan-2015
Category:	Education
Upload:	reza-rahimi
View:	2,373 times
Download:	9 times

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

Education