Date post: | 01-Apr-2015 |
Category: |
Documents |
Upload: | cory-marske |
View: | 237 times |
Download: | 4 times |
COSC 6114
Prof. Andy Mirzaian
General Overview Introduction Fundamentals Duality Major Algorithms Open Problems
2D Linear Programming O(n log n) time by computation of feasible region O(n) time Randomized Incremental Method O(n) time Deterministic Prune-&-Search Method Smallest Enclosing Disk
TOPICS
References:• [M. de Berge et al] chapter 4• [CLRS] chapter 29• [Edelsbrunner ‘87] chapter 10• Lecture Notes 4, 7-13• Reference books on “Optimization” listed on the course web-site
General Overview
The LP Problem
subject to:
dd xcxcxc 2211
ndndnn
dd
dd
bxaxaxa
bxaxaxa
bxaxaxa
2211
22222121
11212111
maximize
Applications
• The most widely used Mathematical Optimization Model.
• Management science (Operations Research).
• Engineering, technology, industry, commerce, economics.
• Efficient resource allocation:
– Airline transportation,
– Communication network – opt. transmission routing,
– Factory inventory/production control,
– Fund management, stock portfolio optimization.
• Approximation of hard optimization problems.
• . . .
FeasibleRegion
Example in 2D
optimum basic
constraints
max x1 + 8x2
subject to:
(1) x1 3 (2) x2 2(3) –3x1 + 4x2 14(4) 4x1 – 3x2 25(5) x1 + x2 15
x2
x1
(1)
(2)
(4)
(5)(3)
(5)(3)
optimum x1 + 8x
2
x1 = 46/7x2 = 59/7
= 74
Example in 3D
subject to: zmaximize zOptimum
(x,y,z)=(0,0,3)
x
y
z
0
0
0
2
3
z
y
x
y
zyx
History of LP 3000-200 BC: Egypt, Babylon, India, China, Greece: [geometry & algebra]
Egypt: polyhedra & pyramids. India: Sulabha suutrah (Easy Solution Procedures) [2 equations, 2 unknowns] China: Jiuzhang suanshu (9 Chapters on the Mathematical Art) [Precursor of Gauss-Jordan elimination method on linear equations] Greece: Pythagoras, Euclid, Archimedes, …
825 AD: Persia: Muhammad ibn-Musa Alkhawrazmi (author of 2 influential books):“Al-Maqhaleh fi Hisab al-jabr w’almoqhabeleh” (An essay on Algebra and equations) “Kitab al-Jam’a wal-Tafreeq bil Hisab al-Hindi” (Book on Hindu Arithmetic).
originated the words algebra & algorithm for solution procedures of algebraic systems.
Fourier [1826], Motzkin [1933] [Fourier-Motzkin elimination method on linear inequalities]
Minkowski [1896], Farkas [1902], De la Vallée Poussin [1910], von Neumann [1930’s], Kantorovich [1939], Gale [1960] [LP duality theory & precursor of Simplex]
George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case, but effective in practice.
Leonid Khachiyan [1979]: Ellipsoid algorithm.
The first weakly polynomial-time LP algorithm: poly(n,d,L).
Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time. IPM variations are very well studied.
Megiddo-Dyer [1984]: Prune-&-Search method. O(n) time if the dimension is a fixed constant. Super-exponential on dimension.
LP in matrix form:
subject to: Ax b
maximize cT x ),,,( 21 dT cccc
nidjijaA ..1
..1)(
),,,( 21 nT bbbb
Canonical Form: ),,,( 21 dT xxxx
We assume all vectors are column vectors. Transpose (T) if you need a row vector.
LP feasible regiond
idid22i11i in plane-hyper bxaxaxa
F = { xd | Ax b }
didid22i11i in space-half bxaxaxa
Primal Dual LPs:Primal:maximize cT xsubject to: Ax b
(canonical form)
Dual: minimize yT bsubject to: yT A = cT
y 0 (standard form)
A
xT
y b
cT
n
d
Primal-Dual LP Correspondence
PRIMAL DUAL
max cTx min yTb
Constraint: aiT x bi
ajT x bj
akT x = bk
(aiT = i-th row of constraint matrix A)
Variable: yi 0 yj 0 yk unrestricted
Variable: xi 0 xj 0 xk unrestricted
Constraint: yTAi ci yTAj cj yTAk = ck
(Ai = i-th column of constraint matrix A)
FACT: Dual of the Dual is the Primal.
Example 1: Primal-Dual
PRIMAL:max 16 x1 - 23 x2 + 43 x3 + 82 x4
subject to: 3 x1 + 6 x2 - 9 x3 + 4 x4 239 -9 x1 + 8 x2 + 17 x3 - 14 x4 = 582 5 x1 + 12 x2 + 21 x3 + 26 x4 -364
x10, x20, x40
DUAL:min 239 y1 + 582 y2 - 364 y3
subject to: 3 y1 - 9 y2 + 5 y3 16 6 y1 + 8 y2 + 12 y3 -23 -9 y1 + 17 y2 + 21 y3 = 43 4 y1 - 14 y2 + 26 y3 82
y10, y30
Example 2: The Diet Problem
This is one of the earliest Primal-Dual LP formulations. Dantzig attributes it to:[G. J. Stigler, “The cost of subsistence,” J. Farm Econ. 27, no. 2, May 1945, 303-314].
Primal: A homemaker has n choices of food to buy, and each food has some of each of m nutrients:
Given: aij = amount of the ith nutrient in a unit of the jth food, i=1..m, j=1..n,
ri = yearly minimum requirement of the ith nutrient, i=1..m,
cj = cost per unit of the jth food, j=1..n.
Determine: xj = yearly consumption of the jth food, j=1..n.
Objective: Minimize total food purchasing cost cTx for the homemaker.
Constraints: Meet yearly nutritional requirement (Ax r) with nonnegative amount of each food (x 0).
Primal: min cTx subject to: Ax r , x 0.
Example 2: The Diet ProblemPrimal: min cTx subject to: Ax r , x 0.
Dual: max rTy subject to: ATy c , y 0.
Dual interpretation:
A pill-maker wishes to market pills containing each of the m nutrients, yi = the price per unit of nutrient i, i=1..m.He wishes to be competitive with the price of real food, andmaximize his profit from the sale of pills for an adequate diet.
The dual constraints
m
1ijiij 1..njcya
express the fact that the cost in pill form of all the nutrients in the j th food
is no greater than the cost of the jth food itself.
The objective function rTy is simply the pill maker’s profit for an adequate diet.
Example 3: Network Max Flow
Flow Network: N=(V,E,s,t,c) V = vertex setE = directed edge sets = source vertext = terminal vertexc: E + edge capacities
find max flow f: E + satisfying flow conservation & capacity constraints
see alsoAAW
maximize v Maximize Net flow out of sourcesubject to:
v for x=s
Sy fxy - Sy fyx = -v for x=t Flow Conservation Law
0 for xV - {s,t}
fe ce for e E Flow Capacity Constraintsfe 0 for e E Flow Non-negativity Constraints
Assume flow is 0 on non-edges.
Example 3: Network Max Flow
e
i
j
i
j
e
+1
-1
0
0
0
fe
ce
A =
A: vertex-edge incidence matrix Rows indexed by vertices V, columns by edges E. For edge e=(i,j)E : Aie = +1, Aje = -1, Ake = 0 kV-{i,j}.
Example 3: Network Max Flow
e
i
j
fe
ce
s)constraint(capacity
n)oconcervati (flow
0
0
:s.t.
maximize
:Flow)t -s(Max :Primal
cf
other rows
row t v
row s v
Af
v
e
i
j
e
ce0σ
1ππ
0σππ :s.t.
σc minimize
:Cut)t -s(Min : Dual
ts
ijji
Eeee
E)j,i(
LP: Fundamental Facts
LP optimal solution set Feasible region F = { xd | Ax b } is the intersection of n half-spaces. F is a convex polyhedron (possibly empty or unbounded). Convexity: local optimality global optimality.
A supporting hyper-plane is one that does not intersect the interior of F. A face of F is the intersection of F with some supporting hyper-plane.
We consider F itself as a (full dimensional) face of F. is also a face of F. Vertex 0 dimensional face. Edge (bounded) 1 dimensional face.
optimum objective value z* = max { cTx : xF} (may not exist or be unbounded)
supporting hyper-plane cTx = z* optimum face F* = { x : cTx = z* , xF} F* interior(F) =
F is pointed if it doesn’t contain any line. Hence, F= or has a vertex (i.e., rank(A)=d).
F is pointed every face of F is pointed. F* & pointed F* contains a vertex of F (an optimal vertex).
[In that case, we can focus on searching for an optimal vertex.]
For further details see the upcoming slides.
Separation LemmaThe Hahn-Banach Separation Theorem:
Consider any non-empty closed convex set P and a point b in n .bP there is a hyper-plane that separates b from P, i.e.,
hyper-plane yTx = for some yn and , such that yTb < and yTz zP.
yT x =
P
b
Proof: Consider the closest point pP to point b. Take the hyper-plane through p and orthogonal to segment bp.
p
Extreme PointsFACT: A non-empty closed convex set P has an extreme point
if and only if P does not contain any line.
Proof (): Suppose P contains a line L.We must prove that no point xP is an extreme point of P.
Let L’ be the line parallel to L that passes through x.
CLAIM: P contains L’. This claim implies x is not an extreme point of P. (Why?)
Refutation of the Claim leads to a contradiction as follows:Suppose there is a point b L’ – P. By the Separation Lemma, there is a hyperplane H that separates b from P.H intersects L’ (at some point between x and b).H must also intersect line L (which is parallel to L’).That would contradict the fact that H separates b from P L.This contradiction proves the Claim.
L
x L’b
H
Extreme PointsFACT: A non-empty closed convex set P has an extreme point
if and only if P does not contain any line.
Proof (): Suppose P does not contain any line.We apply induction on the dimension.If P is a single point, then it is an extreme point.
Now suppose dim(P) > 0.Since P does not contain any line,
P is a proper subset of aff(P) (affine hull of P). So, there is a point b aff(P) – P. By the Separation Lemma,
there is a hyperplane H separating b from P in aff(P).
Move H parallelwise toward P until H is tangent to P.Let f = HP. dim(f) dim(H) < dim(P).
f is a non-empty closed convex set that contains no line.So, by the induction hypothesis, f must have an extreme point x.x is an extreme point of P also. (Why?)
H
P
f
Extreme PointsThe Krein-Milman Theorem:A closed bounded convex set is equal to the convex hull of its extreme points.
Proof: By convexity & closedness, the given set P contains the convex hull of its extreme points.
The converse, i.e., every point xP can be represented as a convex combination of extreme points of P, is proved by induction on the dimension.
If x is an extreme point of P, we are done. Otherwise, draw a line L through x.Let yz be the maximal segment of L in P.x is a convex combination of the boundary points y and z (as shown).By induction, y & z can be represented as convex combinations of extreme points of the lower dimensional closed bounded convex sets PH1 and PH2, respectively.Hence, x is a convex combination of extreme points of P (details for exercise).
H1
H2
yx z
P
L
Farkas LemmaFarkas Lemma:Let And, bn. Then exactly one of the following must hold:
(1) xd : Ax b.(2) yn : yTA = 0, yTb < 0, y0.
Proof: [(1) & (2) cannot both hold] Else: 0 = 0x = (yTA)x = yT(Ax) yTb < 0.
[ (1) (2)]: P = { zn | z=Ax+s, xd , sn , s0 } is convex & non-empty(0P).
(1) Þ bP y,: yTb < yTz zP by Separation LemmaÞ y,: yTb < yT(Ax+s) s 0,xÞ y,: yTb < (yTA)x+ yTs s 0,xÞ y,: yTb < yTs s 0, & yTA = 0 if yTA0, take x=(-1- yTs) ATy ||yTA||-2
Þ y,: yTb < yTs s 0, & yTA = 0, & y 0 if yi <0, take si= - (1+||)/yi & sj=0 for jiÞ y,: yTb < 0 yTs s 0, & yTA = 0, & y 0 if >0, take s = (/2)y ||y||-2
Þ y: yTb < 0, yTA = 0, y 0Þ (2)
Farkas LemmaFarkas Lemma (alternative versions): Let And, bn.
1) xd : Ax b yn : yTA = 0, yTb < 0, y0.
2) xd : Ax b, x 0 yn : yTA 0, yTb < 0, y0.
3) xd : Ax = b yn : yTA = 0, yTb < 0.
4) xd : Ax = b, x 0 yn : yTA 0, yTb < 0.
Convex Cones, Sets & Polyhedra
cone(R) = {m1r1+m2r2+ … +mkrk | mi0 } cone(R)={0} if R=. Extreme rays of the cone are shown bold.
0
Minkowski sum (of vector sets): P+Q = {x+y | xP, yQ}
CH(V) = {1v1+2v2+ … +mvm | Si i=1,i0 }
Recession Coneof the polyhedron
Convex Cone: C ( d), 0 C, C + C C, and 0: C C.
Finitely Generated Convex Cones and Convex Sets: Let V = {v1, … ,vm} and R = {r1, … ,rk}
+ =
CH(V) + cone(R):
polyhedron
Caratheodory’s Theorem
Let X be a non-empty set in d.
(a) xcone(X) & x0 if and only if x= m1x1+m2x2+ … +mkxk for some m1, … ,mk > 0
and for some linearly independent x1, … ,xk X.(Hence, k d.)
(b) xCH(X) if and only if x= m1x1+m2x2+ … +mkxk for some m1, … ,mk > 0, Si mi
=1. and for some affinely independent x1, … ,xk X.(I.e., x2-x1, … ,xk-x1 are linearly independent. Hence, k d+1.)
0
x
cone(X)
(a)
CH(X)x
(b)
X
x1
x2
x1
x2
x3
Caratheodory’s Theorem - Proof
(a) Consider a minimal k such thatx = m1x1+m2x2+ … +mkxk , mi >0, xiX, i=1..k.
(xi) are linearly dependent r1, … ,rk : at least one ri >0 & 0 = r1x1+r2x2+ … +rkxk.
So, x = (m1 – a r1) x1+ (m2 – a r2) x2+ … + (mk – a rk) xk .
Take a = min { mi / ri | ri >0}.
All coefficients remain non-negative and at least one becomes 0.
This contradicts minimality of k.
CH(X)
cone(Y)
Y
x
(b) Apply proof of (a) in d+1 to cone(Y) where Y = CH(X){1} = { (x,1) d+1 | xCH(X) }.
Polyhedra - Theorem
Fundamental Theorem of Polyhedra [Minkowski-Weyl]:
A subset P d is the intersection of a finite set of half-spaces:
P = { xd | Ax b } for some A nd, b n
if and only if
P is the sum of convex hull of a finite set of points plus conical combination of a finite set of vectors:
P = CH(V) + cone(R) for some V dm , R dm’ = { x d | x = Vl + Rm, m0, l0, Si li = 1 }.
P is a polytope P is bounded R=, V = vertex set of P.
Corollary: A polytope can be viewed in 2 equivalent ways: 1) as intersection of half spaces (defined by its facets), 2) as convex hull of its vertices (cf. Krein-Milman Theorem).
If P is pointed, columns of V are vertices of P, columns of R are extreme rays of P.
Polyhedra – Example
2x3x,1x,0xx|xP 211212
V = (v1 , v2)
R = ( r1 , r2)
1λ,0λ,0μ,
μμ
1103
λλ
1111
xx
xP iii2
1
2
1
2
12
v2
v1
r2
r1
P
x1
x2
RV
Polyhedra – Proof
Proof of (): [Assume polyhedron P is pointed.] Let Q = { x d | x = Vl + Rm, m0, l0, Si li = 1 } where columns vi of V are
extreme points of P, and columns ri of R are extreme rays of P. We claim Q = P:Q P: xQ implies x =y + z for some y=V , l l0, Si li = 1, and z= Rm, m0.
So yP, since y is a convex combination of extreme points of P. Also, z is a conic combination of extreme rays of P, so y+z P. So xP.
Q P: WLOG assume P. If xQ, then ( ,l m) mm’ such that v1 l1 + v2 l2 + …+ vm lm + r1 m1 + r2 m2 +… + rm’ mm’ = x l1 + l2 + …+ lm = 1 l 0, m 0.
So, by Farkas Lemma (p,po)d
such that i: pTvi + po 0, pTri 0, pTx + po > 0.Now consider the LP max {pTy | yP}.For all rays ri of P, pTri 0. Hence, the LP has a bounded optimum.So, the optimum must be achieved at an extreme point of P.But for every extreme point vi of P we have pTx > -po pTvi. So, pTx > -po max {pTy | yP}. (The hyperplane pTy = -po separates x from P.)Therefore, xP. [What happens if P is not pointed? See exercise 7.]
Polyhedra – Proof
Proof of ():
Q is the projection onto x of the polyhedron
{ (x, ,l m) dmm’ | x - Vl – R = 0m , m0, l0, Si li = 1 }.
Hence, Q is also a polyhedron (i.e, intersection of a finite number of half-spaces).
Bases & Basic Solutions
Bases & Basic Solutions:
Basis B: Any set of d linearly independent rows of A.
B dd and det(B) 0. N = remaining non-basic rows of A.
Basic Solution: BxB = bB xB = B-1 bB .
Basic Feasible Solution (BFS): If xB is feasible, i.e., NxB bN .
Each BFS corresponds to a vertex of the feasible polyhedron.
xTB
bB
bN
B
N
A b
Fundamental Theorem of LPFor any instance of LP exactly one of the following three possibilities holds:
(a) Infeasible.(b) Feasible but no bounded optimum.(c) Bounded optimum.
[Note: Feasible polyhedron could be unbounded even if optimum is bounded. It depends on the direction of the objective vector.]
Moreover, if A has full rank (i.e., basis), then every nonempty face of the feasible polyhedron contains a BFS, and this implies:
(1) feasible solution BFS.(2) optimum solution optimum solution that is a BFS.
c
Proof: If there is a basis, the basic cone contains the feasible region but does not contain any line. So the feasible region doesnot contain any line, hence it is pointed. So everynon-empty face of it (including the optimal face, if non-empty) is pointed, and thus contains a vertex.(For details see exercise 4.)
LP Duality TheoremPrimal: max { cTx | Ax b }Dual: min { yTb | yTA = cT, y 0 }
Weak Duality: x Primal feasible & y Dual feasible cTx yTb.
Proof: cTx = (yTA)x = yT(Ax) yTb.
Corollary:(1) Primal unbounded optimum Dual infeasible(2) Dual unbounded optimum Primal infeasible(3) Primal & Dual both feasible both have bounded optima.
[Note: Primal & Dual may both be infeasible. See Exercise 5.]
Strong Duality: If x & y are Primal/Dual feasible solutions, then
x & y are Primal/Dual optima cTx = yTb.
Proof: By Farkas Lemma. See Exercise 8.
Example 1
P: max -18 x1 + 27 x2 subject to: -4 x1 + 5 x2 8 x1 + x2 7
x10, x2 0
D: min 8 y1 + 7 y2
subject to: -4 y1 + y2 -18 5 y1 + y2 27
y10, y2 0
Optimum Solution for P: x1 = 3, x2 = 4
opt objective value =
-18*3 + 27*4 = 54
Optimum Solution for D: y1 = 5, y2 = 2
opt objective value =
8*5 + 7*2 = 54
Example 2: The Diet Problem
Primal: min cTx subject to: Ax r , x 0. [Homemaker]
Dual: max rTy subject to: ATy c , y 0. [Pill Maker]
Week Duality:
For every primal feasible x and dual feasible y: cTx rTy.
Interpretation: the pill maker cannot beat the real food marketfor an adequate diet.
Strong Duality:
min {cTx | Ax r , x 0} = max { rTy | ATy c , y 0}.
Interpretation: homemaker’s minimum cost food budget is equal to pill maker’s maximum profit for an adequate diet.
LP Optimization Feasibility
Primal:maximize cT xsubject to: Ax b
Dual: minimize yT bsubject to: yT A = cT
y 0
Is there (x,y) that satisfies:
bT y - cT x 0
Ax bAT y = c y 0
Complementary Slackness
Primal:maximize cT xsubject to: aT
i x bi , i=1..n
xj 0 , j=1..d
Dual: minimize yT b
subject to: ATj y cj , j=1..d
yi 0 , i=1..n
Complementary Slackness Condition (CSC): yi (bi - aT
i x ) = 0 , i=1..n
xj (ATj y - cj ) = 0 , j=1..d
FACT: Suppose x & y are Primal-Dual feasible. Then the following statements are equivalent:
(a) x and y are Primal-Dual optimal,(b) cTx = yTb,(c) x & y satisfy the CSC.
Example 3: Max-Flow Min-Cut
See AAW for animation:
cf
v
v
Af
v
0
rowsother 0
t row
s row
:s.t.
maximize
:Flow)t -s(Max :Primal
0σ
1ππ
0σππ :s.t.
σc minimize
:Cut)t -s(Min : Dual
ts
ijji
Eeee
E)j,i(
S
(v net out-flow)
t
f = c , =1
f = 0 , = 0
= 0
= 0 = 0CC
= 1
Min-Max Saddle-Point Property
FACT: For all functions f : XY :
max min f (x,y) min max f (x,y) . xX yY yY xX
Proof:
Suppose LHS = f(x1,y1) and RHS = f(x2,y2).
Then, f(x1,y1) ≤ f(x1,y2) ≤ f(x2,y2) .
x1
x2
y1 y2
Min-Max Saddle-Point Property
FACT: f has the SPP if and only if
max min f (x,y) = min max f (x,y) . xX yY yY xX
Function f has the Saddle-Point Property (SPP) if it has a saddle-point, i.e., a point (x0, y0) XY such that
xX f (x , y0) ≤ f (x0 , y0) ≤ f(x0 , y) yY .
Duality via LagrangianP* = max {cT x | Ax b } x
D* = min { yT b | ATy = c } y0
Lagrangian: L(x,y) = cT x + yT(b – Ax) = yT b + xT(c – ATy)
Fact: P* = max min L(x,y) , D* = min max L(x,y) . x y0 y0 x
Proof: See Exercise 9.
Weak Duality : max min L(x,y) min max L(x,y) x y0 y0 x
P* D* .
Strong Duality : P or D feasible Lagrangian has SPP.
max min L(x,y) = min max L(x,y) x y0 y0 x
P* = D* .
George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case,
but effective in practice.
Leonid Khachiyan [1979]: Ellipsoid algorithm.
The first weakly polynomial-time LP algorithm: poly(n,d,L).
Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time.
IPM variations are very well studied.
Dantzig – Khachiyan – Karmarkar
Dantzig: Simplex Algorithm
Pivots along the edges of the feasible polyhedron,monotonically improving the objective value.
c
Pivot Rule: Decides which row leaves & which row enters the basis (i.e., which edge of thefeasible polyhedron to follow next).
Each BFS corresponds to a vertex of the feasible polyhedron.
Simplex Pivot:A basic row leaves the basis anda suitable row enters the basis. A non-degenerate pivot moves the BFS along an edge of the feasible polyhedronfrom one vertex to an adjacent one.
Center of GravityA set of points K d is called a convex body
if it is a full dimensional, closed, bounded, convex set.
FACT: [Grünbaum 1960]Let point c be the center of gravity of a convex body K and H be an arbitrary hyperplane passing through point c. Suppose H divides K into two bodies K+ and K– . Then the volumes vol(K+) and vol(K–) satisfy:
.)(vol11})(vol,)(volmax{ 11 KKK
d
d
cK+
K– H
d
d 1111
e
d
dd
1111lim 11
Iterative Central Bisection
FACT: Suppose the feasible set is a convex body F which is contained in a ball C0 of radius R and contains a ball C1 of radius r.Suppose we start with (the center of) the larger ball and iteratively perform central bisection until center of gravity of the remaining convex body falls within F.The number of such iterations is O(d log R/r).
Proof: vol(C0) = vdRd vol(C1) = vdrd
vd = volume of the unit ball in d .
With each iteration, the volume drops by a constant factor.
# iterations = O( log vol(C0)/vol(C1) ).
r
R
F
C0
C1
Iterative Central Bisection
The # central bisection iterations O(d log R/r)is polynomial in the bit-size of the LP instance.
However, each iteration involves computing the volume or center of gravity of the resulting convex body.
This task seems to be hard even for LP instances. We don’t know how to compute them in polynomial time.
So, we resort to approximation1) approximate the center of gravity2) approximate the convex body
What simple approximating “shape” to use? Bounding Box, Ball, ellipsoid, …?
Ellipsoids
1
1
1
1
2
1
2
oTT
od
od
odd
zxAAzx|x
zxA|x
y,Ayzx:y|xE
zo = center of ellipsoid E. A = a dd non-singular matrix.
An ellipsoid is an affine transform of the Euclidean unit ball.
zo
E
Ayzxy o
xy
0
Ellipsoids
Example:
1
00
00
22
22
21
212
2
1
2
1
a
x
a
xxx
E
z,a
aA o
1a
2a
1x
2x
E
Ellipsoids
)Adet(v)E( dvol
( vd = volume of the unit ball in d .)
zo
E
cxopt
}|{maxarg Exxcx Txopt
cAcAAz
T
T
o
12 y,Ayzx:y|xE o
dd
Löwner-John Ellipsoid
FACT 1: [K. Löwner 1934, Fritz John, 1948]Let K be a convex body in d. Then1. There is a unique maximum volume ellipsoid Ein(K) contained in
K.2. K is contained in the ellipsoid that results from Ein(K) by
centrally expanding Ein(K) by a factor of d (independent of K).
3. If K is centrally symmetric, then the factor d in part (2) can be improved to d .
Löwner-John Ellipsoid
FACT 2: [K. Löwner 1934, Fritz John, 1948]Let K be a convex body in d. Then1. There is a unique minimum volume ellipsoid Eout(K) that
contains K.2. K contains the ellipsoid that results from Eout(K) by centrally
shrinking Eout(K) by a factor of d (independent of K).
3. If K is centrally symmetric, then the factor d in part (2) can be improved to d .
Löwner-John metric approximation
Corollary: Any metric in d can be approximated by a Löwner-Johnmetric with distortion factor at most d .
Proof: Let || . ||B denote an arbitrary metric norm with (centrally symmetric) unit ball B.
|| x ||B = min { l | x lB } for any x d .
Let ELJ = { xd | || A–1 x ||2 1 } be the min volume Löwner-John ellipsoidthat contains B.
The metric norm with ELJ as the unit ball is || x ||LJ = || A–1 x ||2 for any x d .
.1 have We LJLJ Ed
BE
.|||||||||||| Therefore, LJBLJ xdxx
ELJ
B
LJEd
1
c
xk-1
xopt
zo
Iterative Löwner-John Ellipsoidal Cut
)()( ooutT
ooptT zxczxc
)()1()( 01 zxcxxc opt
Tdkopt
T
dedxxc
xxc
koptT
koptT
/111)(
)(
1
)()1( 11
koptT
d xxc
)( okT zxcd )()( kopt
Toopt
T xxcdzxcd
xk
xout
dkexxc
xxc
optT
koptT
/
)(
)(
0
Iterative Löwner-John Ellipsoidal Cut
iterations
)(log
after
)(
0
xxc d Ok
xxc
optT
koptT
dkexxc
xxc
optT
koptT
/
)(
)(
0
# iterations is polynomial in the bit-size of the LP instance.
However, each iteration is expensive and involves computing
Löwner-John Ellipsoid of a convex polytope.
What next? Approximate the convex body by an ellipsoid.
Khachiyan: Ellipsoid MethodCircumscribed ellipsoid volume reduction by cuttings.
Ek
)1(21
)(vol
)(vol 1
de
k
k
E
E
zk
Ek+1
zk+1
# iterations = O(d2 log R/r)
Time per iteration = O(d n)
The New York Times, November 7, 1979
Continued on next slide.
Karmarkar: Interior Point Method
maximize cTx + i log si
subject to Ax+s=b, s > 0
= =1
=0.01
=
=1=0.01
Central path
Logarithmic Barrier and Central Path
IPM: Primal-Dual Central Path(P) max cTx (D) min bTy
s.t. Ax+s=b, s 0 s.t. ATy=c, y 0
Optimality Conditions on (x,s,y):
Ax+s=b, s 0 (Primal feasibility)
ATy=c, y 0 (Dual feasibility)
si yi = 0 i (Complementary Slackness)
Primal-Dual Central Path parameterized by > 0:
Ax+s=b, s > 0 (Primal strict feasibility)
ATy=c, y > 0 (Dual strict feasibility)
si yi = i (Centrality)
IPM: Primal-Dual Central Path
Algorithmic Framework:
Assume both (P) and (D) have strictly feasible solutions. Then, for any > 0, the
above non-linear system of equations has a unique solution (x(), s(), y()).These points form the Primal-Dual Central Path { (x(), s(), y()) | (0, ) }.
lim 0 (x(), s(), y()) = (x*, s*, y*) is optimal solution of the Primal-Dual LP.
Update approximate solution to the system while iteratively reducing towards 0.
Central Path:Ax+s=b, s > 0 ATy=c, y > 0 si yi = i
Open Problem
Can LP be solved in strongly polynomial time?
Strongly Polynomial:(i) Time = poly(n,d) [i.e., independent of input coefficient sizes]
(ii) Bit-length of each program variable = poly(n,d,L).
Weakly Polynomial:(i) Time = poly(n,d,L), [L = bit-length of input coefficients]
(ii) Bit-length of each program variable = poly(n,d,L).
[ Steve Smale, “Mathematical Problems for the Next Century’’, Mathematical Intelligencer, 20:7–15, 1998] lists 18 famous open problems and invites the scientific community to solve them in the 21st century. Problem 9 on that list is the above open problem on LP.
Best known upper-bound [Kalai, Kleitman 1992]:
D(d,n) nlog d + 2
For a recent survey on this topic click here.
Hirsch’s Conjecture [1957]Hirsch’s Conjecture:
Diameter of any d-polyhedron with n facets satisfiesD(d,n) n – d.
That is, any pair of its vertices are connected by a path of at most n d edges.
This conjecture was disproved by Francisco Santos in June 2010.Click here to see Santos’s paper.
LP in dimension 2
2D Linear Programming
Fy
x
Objective function = c1 x + c2 y, cT =(c1 , c2)Feasible region F
c
optimummax = c 1 x
+ c 2 y
2D Linear Programming
Feasible region F is the intersection of n half-planes.
F is (empty, bounded or unbounded) convex polygon with n vertices.
F can be computed in O(n log n) time by divide-&-conquer
(See Lecture-Slide 3).
If F is empty, then LP is infeasible. Otherwise, we can check its vertices, and its possibly up to
2 unbounded edges, to determine the optimum. The latter step can be done by binary search in O(log n) time. If objective changes but constraints do not, we can update the optimum
in only O(log n) time. (We don’t need to start from scratch).
Improvement Next:
Feasible region need not be computed to find the optimum vertex.
Optimum can be found in O(n) time both randomized & deterministic.
2D LP Example: Manufacturing with Molds
2D LP Example: Manufacturing with Molds
The Geometry of Casting: Is there a mold for an n-faceted 3D polytope P such that P can be removed from the mold by translation?
Lemma: P can be removed from its mold with a single translation in direction d
d makes an angle 90 with the outward normal of all non-top facets of P.
mold
P
f
f ’
(f ‘) = - (f)
d
Corollary: Many small translations possible Single translation possible.
2D LP Example: Manufacturing with Molds
The Geometry of Casting: Is there a mold for an n-faceted 3D polytope P such that P can be removed from the mold by translation?
mold
P
f
f ’
(f ‘) = - (f)
d
(f) = ( x(f) , y(f) , z(f) ) outward normal to facet f of P.
dT.(f) ≤ 0 non-top facet f of P
x(f) . x + y(f) . y + z(f) ≤ 0 f n-1 constraints
x
y
z
z=1d=(x,y,1)
THEOREM: The mold casting problem can be solved in O(n log n) time. (This will be improved to O(n) time on the next slides.)
Randomized IncrementalAlgorithm
Randomization
Random(k): Returns an integer i 1..k, each with equal probability 1/k. [Use a random number generator.]
Algorithm RandomPermute (A) O(n) time
Input: Array A[1..n]
Output: A random permutation of A[1..n] with each of n! possible permutations equally likely.
for k n downto 2 do Swap A[k] with A[Random(k)]
end.
This is a basic “initial” part of many randomized incremental algorithms.
2D LP: Incremental Algorithm
Method: Add constraints one-by-one, while maintaining the current optimum vertex.
Input: (H, c), H = { H(1), H(2), … , H(n)} n half-planes, c = objective vectorOutput: Infeasible: (i,j,k), or
Unbounded: , or Optimum: v = argmaxx { cTx | x H(1)H(2)…H(n) }.
Define: C(i) = H(1)H(2)…H(i) , for i = 1..n v(i) = optimum vertex of C(i) , for i=2..n. cTv(i) = max { cTx | x C(i) }
Note: C(1) C(2) … C(n).
e
v
Infeasible Unbounded Non-unique optimum Unique optimum
cPossible Outcomes:
H(i) H(j)
H(k)
2D LP: Incremental Algorithm
LEMMA: (1) v(i-1) H(i) v(i-1) C(i) v(i) v(i-1). (2) v(i-1) H(i) (2a) C(i) = , or (2b) v(i) L(i) C(i-1), L(i) =bounding-line of H(i).
H(i) v(i)=v(i-1)
(1)
C(i-1)
H(i)
(2a)
C(i-1)
v(i-1)
H(j)
H(k)
H(i)
(2b)
C(i-1)
v(i-1)
v(i)
L(i)L(i)
c
2D LP: Incremental Algorithm
Algorithm PreProcess (H,c) O(n) time 1. min { angle between c and outward normal of H(i) | i=1..n }
H(i) = most restrictive constraint with angle Swap H(i) with H(1) (L(1) is bounding-line of H(1)).
2. If (parallel) H(j) with angle and H(1)H(j)= then return “infeasible”.3. If L(1) H(j) is unbounded for all H(j) H, then
most restrictive L(1) H(j) over all H(j) H
return (“unbounded”, )4. If L(1) H(j) is bounded for some H(j)H, then
Swap H(j) with H(2) and return “bounded”.
c L(1)
H(1)
L(1)H(j)
H(2)H(1)
(1) (3) (4)
L(1)
H(1)
(2)
H(j)
Disjoint p
arallel.
v(2)
Input: (H, c), H = { H(1), H(2), … , H(n)} n half-planes, c = objective vector
Output: Solution to max { cTx | x H(1)H(2)…H(n) }
1. if PreProcess(H,c) returns (“unbounded”, ) or “infeasible”
then return the same answer
(* else bounded or infeasible *)
2. v(2) vertex of H(1) H(2)
3. RandomPermute (H[3..n])
4. for i 3..n do
5. if v(i-1) H(i) then v(i) v(i-1)
6. else v(i) optimum vertex p of L(i)(H(1)…H(i-1)) (* 1D LP *)
7. if p does not exist then return “infeasible”
8. end-for
9. return (“optimum”, v(n))
end.
Randomized Incremental 2D LP Algorithm
THEOREM: 2D LP Randomized Incremental algorithm has the following complexity: Space complexity = O(n) Time Complexity: (a) O(n2) worst-case
(b) O(n) expected-case.
Randomized Incremental 2D LP Algorithm
.)()(3
2
n
i
nOiO
Proof of (a): Line 6 is a 1D LP with i-1constraints and takes O(i) time.
Total time over for-loop of lines 4-8:
Proof of (b): Define 0/1 random variables
Lines 5-7 take O( i*X(i) +1) time. Total time is
Randomized Incremental 2D LP Algorithm
.n..3ifor otherwise0
H(i)1)v(iif1)i(X
n
3i
)i(X*)i(O)n(OT
])i(X*)i(O[E)n(O]T[En
3i
v(i) is defined by 2 H(j)’s. The probability that one of them is H(i) is 2/(i-2).This does not depend on C(i). Hence, remove the “Fix” assumption.
n
i
nOi
iOnOTE3
).(2
2*)()(][ :Therefore
Expected time:
)2/(i-2 ]H(i))1v(i-[Pr ]X(i)[E
])i(X[E*)i(O)n(On
3i
BackwardsAnalysis
linearity ofexpectation
“Fix” C(i) = { H(1), H(2)} {H(3), …, H(i)}
Random : C(i-1) = C(i) – {H(i)} random
Randomized Incremental LP in d dimension
Randomized Incremental algorithm can be used (with minor modifications)in d dimensions, recursively.
For example, for 3D LP line 6 becomes a 2D LP with i-1 constraints.In general, it will be a (d-1) dimensional LP.
Define: T(n,d) = running time of algorithm for n constraints & d variables.
i
]di)1d,1i(T[)dn(O)d,n(T
This would be exponential in both n and d.
otherwise0
H(i)1)v(iif1)i(X
Expected time (by backwards analysis):
slide]next on [proof . dn for ))(!()],([
*)]1,1([)()],([
.]1[Pr][
1
dndOdnTE
di
ddiTEdnOdnTE
d/(i-d) H(i))v(i- X(i)En
di
Randomized Incremental LP in d dimension
(1)i!
1F whered))(nO(d!dFd!3d)c(nd)T(n,
1d
0idd:Solution
.dFd!3d)c(n
1)d(ncddFd!3d)c(n
d)1)(ncd(ddd)c(n3Fd!d)c(n3cdn
d)1)(ncd(d1)!(d
1Fd!d)c(n3cdn
d)1)(ncd(dd)(nFcd!3cdn
1)(dF1)!3(dd)c(idi
dcdnd)T(n,
:1)d(n
d
2d
d
d
1d
n
1di1d
StepInduction
.cdFcd!3i!
2icd! 1)d-T(d,d 1)cd(d d)1,T(d
:1)d(n
:non induction By
d
1d
0iexpansioniterativeby
)1d(G)d(G
Basis:Proof
[For nd, T(n,d) = O(dn2), by Gaussian elimination.]
n
1di
dfor n )1,d1T(icdnT(n,d) did
Randomized Incremental Algorithm for
Smallest Enclosing Disk
Smallest Enclosing Disk
pi+1
Lemma: (1) pi D(i-1) D(i) = D(i-1) (2) pi D(i-1) pi lies on the boundary of D(i).
pi
D(i+1)
D(i)=D(i-1)
Input: A set P={p1, p2, … , pn } of n points in the plane.
Output: Smallest enclosing disk D of P.
Lemma: Output is unique.
Incremental Construction:P[1..i] = {p1, p2, … , pi }
D(i)= smallest enclosing disk of P[1..i] .
Smallest Enclosing Disk
LEMMA: Let P and R be disjoint point sets in the plane. pP, R possibly empty.Define MD(P, R) = minimum disk D such that P D & R D ( D = boundary of D). (1) If MD(P, R) exists, then it’s unique,(2) p MD(P-{p}, R) MD(P,R) = MD(P-{p}, R),(3) p MD(P-{p}, R) MD(P, R) = MD(P-{p}, R {p}).
D(0)
D(1)R
p
D()
(3) D(0) MD(P-{p}, R) D(1) MD(P, R) D() (1-) D(0) + D(1) 0 1
As goes from 0 to 1, D() continuously deforms from D(0) to D(1) s.t.
D(0) D(1) D().
p D(1) – D(0) by continuity smallest *, 0 < * 1 s.t.
p D(*) p D(*).
P D(*) & R D(*) *=1 by uniqueness.
Therefore, p is on the boundary of D(1).
Proof: (1) If non-unique smaller such disk:
(2) is obvious.
Smallest Enclosing Disk
Algorithm MinDisk (P[1..n])1. RandomPermute(P[1..n])2. D(2) smallest enclosing disk of P[1..2]3. for i 3..n do4. if pi D(i-1) then D(i) D(i-1)5. else D(i) MinDiskWithPoint (P[1..i-1] , pi) 6. return D(n)
Procedure MinDiskWithPoint (P[1..j],q)1. RandomPermute(P[1..j])
2. D(1) smallest enclosing disk of p1 and q3. for i 2..j do
4. if pi D(i-1) then D(i) D(i-1)
5. else D(i) MinDiskWith2Points (P[1..i-1] , q, pi) 6. return D(j)
Procedure MinDiskWith2Points (P[1..j],q1,q2)
1. D(0) smallest enclosing disk of q1 and q2
2. for i 1..j do
3. if pi D(i-1) then D(i) D(i-1)
4. else D(i) Disk (q1, q2, pi) 5. return D(j)
Sho
w r
ando
m p
erm
utat
ion
only
onc
e is
eno
ugh
Smallest Enclosing Disk
Proof: Space O(n) is obvious.MinDiskWith2Points (P,q1,q2) takes O(n) time.MinDiskWithPoint (P,q) takes time:
n
2i
)i(X*)i(O)n(OT where otherwise0
)1i(D(i)Dpif1)i(X i
n
2i
).n(Oi
2*)i(O)n(O]T[E
analysis) backwards(by i/2)]i(X[E
Apply this idea once more: expected running time of MinDisk is also O(n).
P[1..i]
q
“Fix” P[1..i] = {p1, … , pi} backwards P[1..i-1] = {p1, … , pi} - {pi}
one of these is pi with prob. 2/i
THEOREM: The smallest enclosing disk of n points in the plane can be computed in randomized O(n) expected time and O(n) space.
2D LPPrune-&-Search
2D LP: Megiddo(83), Dyer(84): Prune-&-Search
STEP 1: Transform so that the objective hyper-plane becomes horizontal. (a,b) (0,0), so WLOG b 0. Coordinate transform:
Y = ax + by i = ai – (a/b) bi
X = x i = bi /b
min { Y | i X + i Y + ci 0 , i=1..n }
X
Y
Ffeasibleregion
optimum
Original LP: min { ax + by | ai x + bi y + ci 0 , i=1..n }
2D LP: Prune-&-Search
STEP 2: Partition the constraints (the index set [1..n])I0 = { i | i = 0 } vertical line constraints determine feasible x-interval
I- = { i | i < 0 } I+ = { i | i > 0 }
STEP 3:
0,Ii|c
minU
0,Ii|c
maxU
UXU
i0i
i2
i0i
i1
21
X
Y
F
U1 U2
STEP 4:
IiXY : become Iin sconstraint
IiXY : become Iin sconstraint So
c, Define
ii
ii
i
ii
i
ii
min { Y | i X + i Y + ci 0 , i=1..n }
2D LP: Prune-&-Search
STEP 5: )X(max)X(F,)X(min)X(F Define iiIi
iiIi
F+
F-
So, the transformed constraints are: F-(X) Y F+ (X).Since we want to minimize Y, our objective function is F-(X).
STEP 6: Our new (transformed) problem is:
X
Y
F
U1U2
21 UXU
)X(F)X(F s.t.
)X(F minimize
optimum
F+
F-
2D LP: Prune-&-Search
7(a): Given X [U1 , U2 ] , we can determine in O(n) time:
F-(X) = max { i X + i | i I- }
F+(X) = min { i X + i | i I+ }
f- (L)
(X) = left slope of F-(X) at X
f- (R)
(X) = right slope of F-(X) at X
f+ (L)
(X) = left slope of F+(X) at X
f+ (R)
(X) = right slope of F+(X) at X
Note: if only one i I- achieves the “max” in F-(X), then f-
(L) (X) = f-
(R) (X) = i . Similarly with F+(X).
X’ X
F-
STEP 7: An Evaluation Stage:
2D LP: Prune-&-Search
7(b): In O(n) time we can determine for X [U1 , U2 ], whether: X achieves the minimum of F-(X) and is feasible (hence optimum). X is infeasible & there is no solution to the LP X is infeasible and we know which side (left/right) of X the feasible region F lies. X is feasible and we know which side of X the optimum solution lies.
In the latter two cases, shrink the interval [U1 , U2 ] accordingly.
F-(X)
F+(X)
X X
XX
f- (L)
(X) > f+ (L)
(X)
infeasibleF to
the left
f- (R)
(X) < f+ (R)
(X)
F to the right
f- (L)
(X) f+ (L)
(X) & f- (R)
(X) f+ (R)
(X) F = ; LP is infeasible
OR
STEP 7: An Evaluation Stage:
2D LP: Prune-&-Search
METHOD:
A series of evaluation stages . Prune at least a fixed fraction (1-) of the constraints in each stage.
These constraints won’t play a role in determining the optimum vertex. After the k-th stage, # constraints remaining k n. At most log n stages. Total time:
).n(OnOnαO α11
nlog
0k
kα
2D LP: Prune-&-Search
METHOD: (continued)
= 3/4 is achievable. Which X to select to evaluate then prune? Pair up constraints in I+ (do the same for I- ).
Suppose i,j I+ are paired up.
para
llel
iiXY
jjXY eliminate
U1 U2
eliminateXij
U1 U2
eliminate Xij
U1 U2
Xij eliminateneither one
METHOD: (continued)
Next evaluation point:
X* = median of the Xij’s of the non-eliminated pairs. O(n) time.
Half of the Xij’s will be on the wrong side of X*.
We will eliminate one line from at least half of the pairs. So, at least ¼ of the lines will be eliminated in this stage. At most = ¾ of the lines remain.
THEOREM: A 2-variable LP with n constraints can be solved in O(n) timein the worst-case by Megiddo-Dyer’s Prune-&-Search method.
GENERALIZATION: Megiddo[84] showed d dim. LP can also be solved by this technique in O(n) time assuming d is a fixed constant.[However, the hidden constant in O(n) is super-exponential in d.]
2D LP: Prune-&-Search
Extensions & Applications
The Smallest Enclosing Disk:2
i2
i
ni1y,x
)yy()xx(maxmin
THEOREM [Megiddo]: The smallest enclosing disk can be solved in O(n) timein the worst-case by the Prune-&-Search method.
See Lecture Notes 7-13 for more on this topic.
n..1ifor )yy()xx(z s.t.
z minimize
2i
2i A Quadratic
Program.See Exercise 16.
Exercises
1. A Non-Linear Primal Dual Example: [Rochat, Vecten, Fauguier, Pilatte, 1811] We are given 3 points A,B,C in general position in the plane.
Primal: Consider any equilateral triangle PQR whose 3 sides pass through A,B,C, respectively. (See Figure below.) Let H denote the altitude of the equilateral triangle PQR. Find PQR to maximize H.
Dual: Consider any point S, with L as the sum of the lengths of the 3 segments L = SA + SB + SC. Find S to minimize L. [The optimum S is called the Steiner point of ABC.]
Prove the following claims:a) For any primal-dual feasible solutions H L.b) At optimality max H = min L.c) At optimality SA QR, SB RP, SC PQ,
where means perpendicular.
A
B
C P
Q
R
S
2. LP Primal Dual Example: Give the dual of the following LP instance:
max 5 x1 + 12 x2 - 6 x3 + 9 x4 + 4 x5 subject to: -2 x1 + 5 x2 + 13 x3 + 6 x4 + 7 x5 = 53 11 x1 - 4 x2 + 5 x3 + 8 x4 - 2 x5 31 -4 x1 + 6 x2 + 14 x3 + 2 x4 + 7 x5 43 8 x1 - 3 x2 - 7 x3 + 9 x4 - 14 x5 61 x10, x3 0, x40, x50
3. Fractional Knapsack Problem (FKP):
FKP is the LP max { vTx | wTx W, 0xi1, i=1..n }, where vT=(v1 ,…, vn ) and wT =(w1 ,…, wn ) are vectors of item values and weights,and W is the knapsack capacity. Assume these are all positive reals.The real variable xi , 0xi1, is the selected fraction of the i-th item, for i=1..n.
a) Give the dual of FKP.
b) Use the Duality Theorem to show that a simple greedy algorithm obtains optimal primal-dual solutions. [Hint: sort vi / wi , for i=1..n.]
4. Polyhedral Vertices versus Basic Solutions: Consider the (possibly empty) polyhedron F = { xd | Ax b } for some And, bn, d n. Assume rank(A)=d (i.e., A contains d linearly independent rows). Show the following facts:
a) F has a basic solution (not necessarily feasible).
b) The cone defined by the constraints corresponding to a basis is pointed (i.e., does not contain any line).
c) F is pointed.
d) Every non-empty face of F has a vertex.
e) Every vertex of F corresponds to at least one basic feasible solution.
f) A vertex could correspond to more than one basic feasible solution.
g) Every vertex of F is the unique optimum LP solution for somelinear objective vector and with F as the feasible region.
5. Give LP Primal-Dual examples where:
a) Both the Primal and the Dual are infeasible.
b) The Primal is unbounded and the Dual is infeasible.
c) Both the Primal and the Dual have bounded optima.
6. Farkas Lemma (Alternative versions):Prove the remaining 3 versions of Farkas Lemma.
7. Fundamental Theorem of Polyhedra [Minkowski-Weyl] when P is not pointed:a) Example 1: Consider the non-pointed polyhedron P={(x1, x2)T2 | x11, -x11}. Show that P={ x 2 | x = Vl + Rm, m0, l0, Si li = 1 }, where V consists of the 2 columns v1=(1,0)T, v2=(-1,0)T, and R consists of the 2 columns r1=(0,1)T, r2=(0,-1)T. [Note that v1 and v2 are not extreme points of P.]b) Example 2: Show the alternative representation (according to the Theorem) for the non-pointed polyhedron P = {(x1, x2, x3)T3 | x1+x2 -x31, -x1+x2 -x31}. c) Prove the () part of the Theorem when P is not pointed. [Our referenced book [Ziegler, “Lectures on Polytopes”] contains a proof using the Fourier-Motzkin elimination method.]
8. Strong Duality Theorem: Prove the Strong Duality Theorem using Farkas Lemma. [Hint: a = min{yTb : yTA=c, y0} implies the system {yTb – a < 0, yTA - c = 0, y0} has no solution in y. Reformulate this as F.L.(2) using an extra variable y0. What is F.L.(1)?]
9. LP Duality via Lagrangian: Consider the primal-dual pair of linear programs (Primal) P* = max { cTx | Ax < b } (Dual) D* = min { bTy | ATy = c and y > 0 }
and their Lagrangian L(x,y) = cTx + yT (b – Ax) = bTy + xT (c – ATy) .
Define the functions p(x) = miny>0 L(x,y) andd(y) = maxx L(x,y).
That is, for any x, p(x) is obtained by a y > 0 that minimizes the Lagrangian (for the given x). Similarly, for any y > 0, d(y) is obtained by an x that maximizes the Lagrangian (for the given y).
a) Show that
b) Show that P* = maxx p(x) = maxx miny>0 L(x,y) and D* = miny>0 d(y) = miny>0 maxx L(x,y).
c) Show that if at least one of the Primal or the Dual LP has a feasible point, thenthe Lagrangian has the Saddle Point Property, i.e., maxx miny>0 L(x,y) = miny>0 maxx L(x,y).
[Can you prove this without assuming the LP Strong Duality Theorem?]
. otherwise
if
, otherwise
if
cyAybyd
bAxxcxp
TT
T
)(
)(
10. In the restricted version of the mold casting problem we insist that the object is removed from its mold using a vertical translation (perpendicular to the top facet).(a) Prove that in this case only a constant number of top facets are possible.(b) Give a linear time algorithm that determines all such possible top facets.
11. Let H be a set of at least 3 half-planes. A half-plane hH is called redundant if itsremoval does not change H (the intersection of half-planes in H). Prove that h is redundant if and only if h’h’’ h, for some h’,h’’H – {h}.
12. Prove that RandomPermute(A[1..n]) is correct, i.e., each of the n! permutationsof A can be the output with equal probability 1/n!.
13. Show how to implement MinDisk using a single routine MinDiskWithPoints(P,R)that computes MD(P,R) as defined before. Your algorithm should compute only asingle random permutation during the entire computation.
14. Show the “bridge finding” problem in Kirkpatrick-Seidel’s O(n log h) time 2D CHalgorithm can be solved in O(n) time by Megiddo-Dyer’s technique.
15. Minimum Area Annulus: An annulus is the portion of the plane contained between two concentric circles. Let a set P of n points in the plane be given. Give an efficient algorithm to find the minimum area annulus that contains P.[Hint: This can be solved by an LP. We want to find a center (x,y) and two radii R and r to minimize R2 – r2 subject to the 2n constraints r2 (x-xi)2 + (y – yi)2 R2 . Now instead of the variables x,y,r,R, use the variables x, y, w x2 +y2 - R2 and z x2 +y2 - r2. Also ensure that the LP solution satisfies x2 +y2 = w + R2 = z + r2 max{w,z}. How?]
16. Minimum Enclosing Disk in 2D: Show this problem can be solved in O(n) time by prune-&-search.[Hint: First show the restricted problem where the disk center lies on a given line can be solvedin O(n) time. Then show it can be decided in O(n) time on which side of this line the center of the (unrestricted) minimum enclosing disk lies. Then apply prune-&-search to this scheme.]
17. Maximum Inscribed Sphere: Let P d be a convex polytope, given as intersection of n half-spaces. Give a formulation of the problem to determine the largest sphere inscribed in P.You don’t need to solve the problem; just give a good formulation of it.[The center of this sphere is called the Chebyshev center of the polytope.]
END