Claudio Arbib

Università di L’Aquila

Operations Research

Duality Theory

Content

• Compatible systems of linear inequalities

• Theorems of the alternative

• Gale’s Theorem

• Fàrkas’ Lemma

• Duality theory in Linear Programming

• Strong Duality Theorem

• The dual of a linear program

• Weak Duality Theorem and other corollaries

• Rules to construct the dual problem

Compatible systems of linear

inequalities

• By Fourier’s Theorem a system of linear inequalities Ax < b with

x ∈ IRn, A ∈ IRm×n , b ∈ IRm

is compatible if and only if another linear system A’x < b’, obtained by

conic combination of the given inequalities, with

A’ = [0, A°] ∈ IRp×n, b’ ∈ IRp,

is in turn compatible.

Theorems of the alternative

• Iterating n times Fourier’s Theorem, one has that Ax < b is compatible if and

only if there exist specific conic combinations of its inequalities that produce

a compatible system A(n)x < b(n), where

A(n) = [0, …, 0] ∈ IRq×n , b(n) ∈ IRq

• But [0, …, 0]x < b(n) is compatible if and only if b(n) > 0.

• So in order to have Ax < b incompatible it must be possible to find a vector

of multipliers y > 0 that combines

– the rows of A so as to obtain the null row 0

– the components of b so as to obtain a real bi(n) < 0

Gale’s Theorem

The previous discussion is summarized by

Theorem (Gale): The linear system Ax < b is compatible if and only if the system y > 0, yA = 0, yb < 0 is incompatible.

• Gale’s Theorem is called the first theorem of the alternative, because it expresses the compatibility of one system in terms of the incompatibility of another.

• We call Ax < b the primal system, and y > 0, yA = 0, yb < 0 the dual system.

• With a primal system of the form Ax > b, the dual writes y > 0, yA = 0, yb > 0.

P

ExamplePrimal system P) 2x1 + 4x2 < 5

x1 – 3x2 < 6

Dual system D) 2y1 + y2 = 0

4y1 – 3y2 = 0

5y1 + 6y2 < 0

y1, y2 > 0 x2

x1

(0, 5/4)

(5/2, 0)

(0, –2)

(6, 0)(0, 0)

y1

y2

D is clearly

incompatible

(1, –2)

(3, 4)

Fàrkas’ Lemma

Gale’s Theorem is not the only theorem of the alternative:

Theorem (Fàrkas): The linear system (standard primal) Ax = b, x > 0 is

compatible if and only if the system

yA > 0, yb < 0

(or, equivalently, the system yA < 0, yb > 0) is incompatible.

Proof: Ax = b, x > 0 ⇔ Ax < b, –Ax < –b, –x < 0 compatible iff (Gale):

z –A = 0, z > 0, z –b < 0.A

–I

b

0

Set z = [u, v, w]. To write uA – vA – w = 0 with w > 0 means (u – v)A > 0.

Calling y = (u – v) the thesis follows.

(Observe that y can have negative components).

x3

x1

x2

Example

Primal system P) x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

Dual system D) y < 0, y > 0, 3y > 0

6y < 0

y

(0, 0, –3)

(6, 0, 0)(0, 2, 0)

P

intersection = {0}

D is clearly

incompatible

Comment

• The theorems of the alternative provide us with an important mean to tackle

the problem of deciding whether a polyhedron is empty or not

• They allow us to transform a problem with a universal quantifier (∀) in one

with an existence quantifier (∃).

In fact a polyhedron Ax < b is empty if for all x ∈ IRn there exists a row i such that aix > bi.

The theorems of the alternative make it unnecessary to check that for all x by

looking for just one y such that yb < 0 which belongs to another polyhedron

(the dual of Ax < b).

• As a matter of fact, the difference between an “easy” and a “difficult” problem

is often marked by the possibility or impossibility of such a practice. For

instance, the very definition of the class NP is based on this distinction.

Duality Theory in LP

• Consider an LP problem in standard form:

P) min cx

Ax = bx > 0

Theorem (strong duality): A feasible solution x* of problem P is optimal

if and only if thare exists a y* belonging to

D = {y ∈ IRm: yA < c}

such that y*b > cx*

x3

x1

x2

Example

Problem P) min x1 – 2x2 + 4x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

D = {y∈IR: y < 1, 3y < –2 , –2y < 4}

i.e., D = {y∈IR: y < 1, y < –2/3 , y > –2}

y

P

x* = (6, 0, 0) cx* = +6

y* = –2/3 y*b = –4

x* non optimal

>

x3

x1

x2

Example

Problem P) min x1 – 2x2 + 4x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

D = {y∈IR: y < 1, 3y < –2 , –2y < 4}

i.e., D = {y∈IR: y < 1, y < –2/3 , y > –2}

y

P

x* = (0, 2, 0) cx* = –4

y* = –2/3 y*b = –4

x* optimal

<

Strong duality

Proof:

Let x* be feasible for problem P and assume y*b > cx* for some y* ∈D.

Then the system

yA < c

–yb < –cx* namely y[A, –b] < [c, –cx*]

turns out to be compatible.

If we apply Gale’s Theorem to such a system we see that the system

[A, –b][ ] = 0, [ ] > 0, [c, –cx*][ ] < 0

is necessarily incompatible.

x

λ

x

λ

x

λ

Strong duality

Proof (contd.):

In other words no x, λ > 0 fulfils

Ax = λb, cx < λcx*

and this is true, in particular, for λ = 1, which implies that no feasible x for

P exists which fulfils

cx < cx*

and so x* is optimal for P.

Strong duality

Proof (contd.):

Conversely, if the dual system y[A, –b] < [c, –cx*] is incompatible, then the

primal Ax = λb, cx < λcx* has a solution x°, λ° > 0.

– If λ° > 0, x°/ λ° is P-feasible and better than x*.

– If λ° = 0, one hasAx° = 0, x° > 0 and cx° < 0, hence x* + x° is feasible and better than x*.

Therefore, x* is not optimal.

End proof

The dual problem

• The theorem just proved justifies the introduction of a new problem

D) max yb

yA < c

• This is called the dual of problem P.

In turn, P is called the primal problem.

• The dual of a linear program (in standard form) is still a linear program

(in general form).

• The dual problem has

– a variable for each constraint of the primal,

– a constraint for each variable of the primal.

Proprieties of the dual

Theorem (reciprocity): Problem P is the dual of problem D.

Theorem (weak duality or dominance): For any pair x ∈ P, y ∈ D of solutions one has yb < cx.

Proof: Reciprocity is readily seen by rewriting D in standard form adding non-

negative slack variables, and then writing the dual of the problem so obtained.

To see dominance it suffices to combine the columns of yA < c (constraints of

D) using the components of x as multipliers. Since the combination is conic,

the inequality is preserved:

yAx < cx

The thesis is obtained by the associative property (y(Ax) < cx) and by

observing that Ax = b.

A few corollaries

Corollary: x* ∈ P and y* ∈ D are optimal if and only if

y*b = cx*

Proof: by combining weak and strong duality.

Corollary (complementary slackness): x* ∈ P and y* ∈ D are optimal if and only if

(c – y*A)·x* = y*·(Ax* – b) = 0

Proof: this corollary says that optimal dual (primal) slacks are orthogonal to

any optimal primal (duale) solution.

The first condition rewrites cx* = y*Ax*, and since Ax* = b it is equivalent

to the previous corollary.

The second one is true ∀y*, because Ax* = b.

Example

Primal problem P) min 4x1 + 3x2 + x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

Dual problem D) max 6y

y < 4, 3y < 3, –2y < 1x3

x1

x2 y

0

41−1/2

dual optimum 6

Pprimal optimum 6

(0, 2, 0)

A few corollaries

Corollary: if problem P (problem D) is unbounded from below (from above)

then problem D (problem P) has no solution.

Proof: it directly derives from weak duality.

For example, suppose by contradiction that P is unbounded from below (i.e.,

for any x ∈ P there exists an x° ∈ P such that cx° < cx) and that, however, D

is non-empty (i.e., there exists one y° ∈ D).

This clearly contradicts weak duality, according to which one has y°b < cx,

∀x ∈ P, and therefore it can’t be cx → –∞.

(A similar argument can be used for the case D unbounded).

Example

Primal problem P) min – 4x1 – 3x2 – x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

Dual problem D) max 6y

y < –4, 3y < –3, –2y < –1

y

0

–1 1/2 empty dual

x3

x1

x2

P

–4

unbounded primal

Summarizing

××××impossibleimpossibleD has a finite

optimum

impossible?××××D = Ø

impossible××××impossibleD unbounded

P has a finite

optimumP = ØP unbounded

Rules to construct the dual

Rule 1: Write the primal as a minimization problem with > and/or =

constraints. The dual will then be a maximization problem with =

and/or < constraints.

Rule 2: Add a dual variable yi for any primal constraint: yi will be

• > 0 if the primal constraint is > (loose constraint)

• free if the primal constraints is = (strict constraint)

Rule 3: The dual objective function is a linear combination of the yi’s with

the primal left-hand side b. The dual hand side is instead the primal

cost vector c.

Rule 4: Add a dual constraint for any primal variable xj: this constraint will

have the form

• of < (loose constraint) if xj is > 0

• of = (strict constraint) if xj is free

Example 1

Primal problem P) max 5x1 – x2 + 2x3

x1 + 4x2 – 6x3 < 62x1 – x3 = 42x1 + 3x2 > 5

x2, x3 > 0

Rewrite (Rule 1) P) min – 5x1 + x2 – 2x3

– x1 – 4x2 + 6x3 > –62x1 – x3 = 42x1 + 3x2 > 5

x2, x3 > 0

Dual problem D) max – 6y1 + 4y2 + 5y3

y1, y3 > 0

– y1 + 2y2 + 2y3 = –5

– 4y1 + 3y3 < 1

6y1 – y2 < –2

x3

x1

x2

P

Example 2

Primal problem P) min 4x1 + 3x2 + x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

primal optimum

39/5 = 7,8

What is the dual problem?

x1 + x2 + x3 > 3

(0, 12/5, 3/5)(3/2, 3/2, 0)

(6, 0, 0)

x3

x1

x2

P

Example 3

Primal problem P) min 4x1 + 3x2 + x3

x1 + 3x2 – 2x3 = 6

x1, x2 , x3 > 0

x1 + x2 + x3 > 3

max

What is the dual optimum value?

What is the dual problem?

(0, 12/5, 3/5)(3/2, 3/2, 0)

(6, 0, 0)