Week 2: Greedy Algorithms

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 1

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 2

Greedy Algorithms:

A greedy algorithm always makes the choice that looks best at the moment.

It makes a local optimal choice in the hope that this choice will lead to a globally optimal solution.

Greedy algorithms yield optimal solutions for many (but not all) problems.

Knapsack problem:

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 3

Knapsack problem: Input: n items: (w1, v1), …, (wn, vn) and a number W

the i-th item’s is worth vi dollars with weight wi. at most W pounds to be carried.

Output: Some items which are most valuable and with total weight at most W. Two versions:

0-1 Knapsack: Items cannot be divided into pieces fractional knapsack: Items can be divided into pieces

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 4

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 5

The 0-1 Knapsack problem:

The 0-1 knapsack problem: N items, where the i-th item is worth vi dollars and

weight wi pounds. 11 p 3 p 4p 58 p 8p 88p vi and wi are integers.

3$ 6 $ 35$ 8$ 28$ 66$

We can carry at most W (integer) pounds. How to take as valuable a load as possible.

An item cannot be divided into pieces. The fractional knapsack problem: The same setting, but the thief can take fractions of items. W may not be integer.

W

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 6

Solve the fractional Knapsack

problem: Greedy on the value per pound vi/wi.

Each time, take the item with maximum vi/wi . If exceeds W, take fractions of the item.

Example: (1, 5$), (2, 9$), (3, 12$), (3, 11$) and w=4.

vi/wi : 5 4.5 4.0 3.667 First: (1, 5$), Second: (2, 9$), Third: 1/3 (3,

12$) Total W: 1, 3, 4. Can only take part of

item

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 7

Proof of correctness: (The hard part)

Let X = i1, i2, …ik be the optimal items taken. Consider the item j : (vj, wj) with the highest v /w. if j is not used in X (the optimal solution), get rid of some

items with total weight wj (possibly fractional items) and add item j.

(since fractional items are allowed, we can do it.) Total value is increased. Why?

One more item selected by greedy is added to X Repeat the process, X is changed to contain all items

selected by greedy WITHOUT decreasing the total value taken by the thief.

X

i1 w1

i2 w2 wj

.

.

.

ip wp

.

.

.

ikwk

jj

jj

j

jpp

j

jp

j

jp

j

j

j

j

p

pp

p

pp

pp

vw

vw

w

vwqwww

w

vwq

w

vw

w

vw

w

vw

w

vwq

w

vw

w

vw

w

vw

vqvvv

)()()%...(

)(%)(...)()(

)(%)(...)()(

%...

121

121

1

11

2

22

1

11

121

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 8

The 0-1 knapsack problem cannot be solved optimally by

greedy Counter example: (moderate part) W=10 2 1.8 Items found (6pounds, 12dollars), (5pounds, 9

dollar), 1.8 1. 1 (5pounds, 9 dollars), (3pounds, 3 dollars), (3 pounds, 3

dollars) If we first take (6, 12) according to greedy

algorithm, then solution is (6, 12), (3, 3) (total value is 12+3=15).

However, a better solution is (5, 9), (5, 9) with total value 18.

To show that a statement does not hold, we only have to give an example.

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 9

A subset of mutually

compatibles jobs: {c, f}

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 10

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 11

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 12

Sorting the n jobs based on fi needs O(nlog n) time

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 13

Example: Jobs (s, f): (0, 10), (3, 4), (2, 8), (1, 5), (4, 5), (4, 8), (5, 6) (7,9).

Sorting based on fi:

(3, 4) (1, 5), (4, 5) (5, 6) (4,8) (2,8) (7, 9)(0,10).Selecting jobs:(3,4) (4, 5) (5,6) (7, 9)

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 14

How to Prove Optimality How can we prove the schedule returned is optimal?

Let A be the schedule returned by this algorithm. Let OPT be some optimal solution.

It is impossible to show that A = OPT, instead we need only to show that |A| = |OPT|.

Note the distinction: instead of proving directly that a choice of intervals A is the same as an optimal choice, we prove that it has the same number of intervals as an optimal. Therefore, it is optimal.

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 15

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 16

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 17

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 18

Sort jobs based on finish time: b, c, a, e, d, f, g, h.

Greedy algorithm Selects: b, e, h.

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 19

Depth: The maximum No. of jobs required at any time.

Depth:3

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 20

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 21

Depth: The maximum No. of jobs required at any time.

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 22

Greedy on start time

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 23

Depth: The maximum No. of jobs required at any time.

Depth:3

Greedy Algorithm:

a

c

b

d

e

g

f

j

i

h

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 24

ddepth

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 25

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 26

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 27

0 10 11

l1=0, l2=1l2=0, l1=0

110 1

l1=0, l2=1l1=9, l2=0

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 28

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 29

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 30

n1n2 … nk

Example: 1, 2, 3, 4, 5, 6, 7, 8, 9

1, 2, 6, 7, 3, 4, 5, 8, 9

di<dj

We check every pair of consecutive

numbers, if there is not inversion, then the whole sequenc has no

inversion.

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 31

di<dj

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 32

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 33

Example: Job ti di

1 2 22 3 43 4 64 4 85 6 10

2 5 9 13 19j1 j2 j3 j4 j5

112/04/19 CS4335 Design and Analysis of Algorithms/WANG Lusheng

Page 34