greedy algorithm.ppt

7/27/2019 greedy algorithm.ppt

1/30

CS3381 Des & Anal of Alg (2001-2002 SemA)

City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 1http://www.cs.cityu.edu.hk/~helena

Greedy Algorithms


2/30



Greedy Algorithms

Coming upCasual Introduction: Two Knapsack Problems

An Activity-Selection Problem

Greedy Algorithm Design

Huffman Codes

(Chap 16.1-16.3)


3/30



2 Knapsack Problems

A thief robbing a store finds n items.

ith item: worth vi dollars

wi pounds

W, wi, vi are integers.

He can carry at most W pounds.

1. 0-1 Knapsack Problem:

Which itemsshould I

take?


4/30



2 Knapsack Problems

A thief robbing a store finds n items.

ith item: worth vi dollars

wi pounds

W, wi, vi are integers.

He can carry at most W pounds.

He can take fractions of items.

2. Fractional Knapsack Problem:

?


5/30



2 Knapsack ProblemsDynamic Programming Solution

If jth item isremovedfrom his load,

Both problems exhibit the optimal-substructure property:

Consider the mostvaluable load thatweighs at most Wpounds.

the remaining load must bethe most valuable load

weighting at most W-wj thathe can take from the n-1original items excluding j.

=> Can be solved by dynamic programming


6/30




Example: 0-1 Knapsack Problem

Suppose there are n=100 ingots:30 Gold ingots: each $10000, 8 pounds (most expensive)20 Silver ingots: each $2000, 3 pound per piece50 Copper ingots: each $500, 5 pound per piece

Then, the most valuable load for to fill W pounds

= The most valuable way among the followings:

(1) take 1 gold ingot + the most valuable way to fill W-8 poundsfrom 29 gold ingots, 20 silver ingots and 50 copper ingots

(2) take 1 silveringot + the most valuable way to fill W-3 poundsfrom 30 gold ingots, 19 silver ingots and 50 copper ingots

(3) take 1 copperingot + the most valuable way to fill W-5 pounds

from 30 gold ingots, 20 silver ingots and 49copper ingots


7/30




Example: Fractional Knapsack Problem

Suppose there are totally n = 100 pounds of metal dust:30 pounds Gold dust: each pound $10000 (most expensive)20 pounds Silver dust: each pound $200050 pounds Copper dust: each pound $500

Then, the most valuable way to fill a capacity of W pounds

= The most valuable way among the followings:

(1) take 1 pound ofgold + the most valuable way to fill W-1 poundsfrom 29 pounds of gold, 20 pounds of silver, 50 pounds of copper

(2) take 1 pound ofsilver+ the most valuable way to fill W-1 poundsfrom 30 pounds of gold, 19 pounds of silver, 50 pounds of copper

(3) take 1 pound copper+ the most valuable way to fill W-1 pounds

from 30 pounds of gold, 20 pounds of silver, 49 pounds of copper


8/30



2 Knapsack ProblemsBy Greedy Strategy

Both problems are similar. But Fractional Knapsack Problem

can be solved in a greedy strategy.

Step 1. Compute the value per pound for each item

Eg. gold dust: $10000 per pound (most expensive)

Silver dust: $2000 per poundCopper dust: $500 per pound

Step 2. Take as much as possible of the most expensive(ie. Gold dust)

Step 3. If the supply of that item is exhausted (ie. no moregold) and he can still carry more, he takes asmuch as possible of the item that is next most

expensive and so forth until he cant carry anymore.


9/30



Knapsack ProblemsBy Greedy Strategy

We can solve the Fractional KnapsackProblem by a greedy algorithm:

Always makes the choice that looks best

at the moment.

ie. A locally optimal Choice

To see why we cant

solve 0-1 Knapsack

Problem by greedy

strategy, read Chp 16.2.


10/30



Greedy Algorithms

2 techniques for solving optimization problems:1. Dynamic Programming

2. Greedy Algorithms (Greedy Strategy)

Greedy Approach can solve

these problems:

For the optimization problems:

Dynamic Programming can

solve these problems:

For some optimization problems,Dynamic Programming is overkillGreedy Strategy is simpler and more efficient.


11/30



Activity-Selection Problem

For a set of proposed activities that wish touse a lecture hall, select a maximum-sizesubset of compatible activities.

Set of activities: S={a1,a2,an}

Duration of activity ai: [start_timei, finish_timei)

Activities sorted in increasing o rder of f in ish t im e:

i 1 2 3 4 5 6 7 8 9 10 11

start_timei 1 3 0 5 3 5 6 8 8 2 12

finish_timei 4 5 6 7 8 9 10 11 12 13 14


12/30



Activity-Selection Problem

i 1 2 3 4 5 6 7 8 9 10 11start_timei 1 3 0 5 3 5 6 8 8 2 12

finish_timei 4 5 6 7 8 9 10 11 12 13 14

Compatible activities:{a3, a9, a11},{a

1

,a4

,a8

,a11

},{a2,a4,a9,a11}

0123456789

1011121314

time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11


13/30



Activity-Selection ProblemDynamic Programming Solution (Step 1)

Step 1. Characterize the structure of an optimal solution.

S: i 1 2 3 4 5 6 7 8 9 10 11(=n)start_timei 1 3 0 5 3 5 6 8 8 2 12finish_timei 4 5 6 7 8 9 10 11 12 13 14

eg

Definition:

Sij={akS: finish_timeistart_timek


14/30




S: i 1 2 3 4 5 6 7 8 9 10 11(=n)start_timei 1 3 0 5 3 5 6 8 8 2 12finish_timei 4 5 6 7 8 9 10 11 12 13 14

Add fictitious activities: a0

and an+1

:

S: i 0 1 2 3 4 5 6 7 8 9 10 11 12start_timei 1 3 0 5 3 5 6 8 8 2 12 finish_timei 0 4 5 6 7 8 9 10 11 12 13 14

ie. S0,n+1={a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11}= S

Note: If i>=j then Si,j=

01234567891011121314

time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11a0 a12


15/30



Substructure:


Suppose a solution to Si,jincludes activity ak,

then,2 subproblems aregenerated: Si,k, Sk,j

The problem:For a set of proposed activities that wishto use a lecture hall, select a maximum-size subset of compatible activities

Select a maximum-sizesubset of compatibleactivities from S0,n+1.

=

0123456789

1011121314

time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11

0123456789

1011121314

time a1 a2 a3 a4 a5 a6a7a8 a9 a10a11

The maximum-size subset Ai,jof compatible activities is:

Ai,j=Ai,k U {ak} U Ak,jSuppose a solution to S0,n+1 contains a7, then,

2 subproblems are generated: S0,7 and S7,n+1


16/30




Step 2. Recursively define an optimal solution

Let c[i,j] = number of activities in a maximum-size subset ofcompatible activities in Si,j.

If i>=j, then Si,j

=, ie. c[i,j]=0.

0 if Si,j=

Maxi


17/30



Activity-Selection ProblemGreedy Strategy Solution

Consider any nonempty subproblemS

i,j, and let a

mbe the activity in S

i,j

with the earliest finish time.

01234567891011121314

time a1 a2 a3

okok

a4 a5

okokokok

a6

okokokok

a7

okokok

a8

okokokok

a9 a10a11

eg. S2,11={a4,a6,a7,a8,a9}

Among {a4,a6,a7,a8,a9}, a4 willfinish earliest

1. A4 is used in the solution

2. After choosing A4, there are 2subproblems: S2,4 and S4,11.But S2,4 is empty. Only S4,11

remains as a subproblem.

Then,

1. Am is used in some maximum-size subset of compatible

activities of Si,j.2. The subproblem Si,m is empty,

so that choosing am leaves thesubproblem Sm,j as the onlyone that may be nonempty.

0 if Si,j=

Maxi


18/30




That is,

To solve S0,12, we select a1 that will finish earliest, and solve for S1,12.To solve S1,12, we select a4 that will finish earliest, and solve for S4,12.

To solve S4,12, we select a8 that will finish earliest, and solve for S8,12.

Greedy Choices (Locally optimal choice)

To leave as much opportunity as possible for the

remaining activities to be scheduled.

Solve the problem in a

top-down fashion

Hence, to solve the Si,j:

1. Choose the activity am with the earliestfinish time.

2. Solution of Si,j = {am} U Solution ofsubproblem Sm,j

0

123456789

1011121314

time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11a0 a12


19/30



Recursive-Activity-Selector(i,j)

1 m = i+1// Find first activity in Si,j

2 while m < j and start_timem < finish_timei

3 do m = m + 1

4 if m < j5 then return {am} U Recursive-Activity-Selector(m,j)

6 else return


Order of calls:

Recursive-Activity-Selector(0,12)Recursive-Activity-Selector(1,12)

Recursive-Activity-Selector(4,12)



0123456789

1011121314


m=2

Okay

m=3

Okay

m=4

break

the loop

{11}

{8,11}

{4,8,11}

{1,4,8,11}


20/30



Iterative-Activity-Selector()

1 Answer = {a1}2 last_selected=1

3 for m = 2 to n

4 if start_timem>=finish_timelast_selected

5 then Answer = Answer U {am}6 last_selected = m

7 return Answer


0123456789

1011121314



21/30




For both Recursive-Activity-Selector andIterative-Activity-Selector,

Running times are (n)Reason: each am are examined once.


22/30




Steps of Greedy Algorithm Design:

1. Formulate the optimization problem in theform: we make a choice and we are leftwith one subproblem to solve.

2. Show that the greedy choice can lead toan optimal solution, so that the greedychoice is always safe.

3. Demonstrate thatan optimal solution to original problem =greedy choice + an optimal solution to thesubproblem

Optimal

Substructure

Property

Greedy-

Choice

Property

A good clue that

that a greedy

strategy will solve

the problem.


23/30




Comparison:

Dynamic Programming Greedy Algorithms

At each step, the choice isdetermined based on

solutions of subproblems.

At each step, we quickly make achoice that currently looks best.

--A local optimal (greedy) choice.

Bottom-up approach Top-down approach

Sub-problems are solved first. Greedy choice can be made firstbefore solving further sub-problems.

Can be slower, more complex Usually faster, simpler


24/30



Huffman Codes

Huffman Codes For compressing data (sequence of characters)

Widely used

Very efficient (saving 20-90%)

Use a table to keep frequencies of occurrence

of characters.

Output binary string.

Todays

weather is nice

001 0110 0 0 100

1000 1110


25/30



Huffman Codes

Frequency Fixed-length Variable-length

codeword codeword

a 45000 000 0

b 13000 001 101

c 12000 010 100

d 16000 011 111e 9000 100 1101

f 5000 101 1100

Example:

A file of 100,000 characters.

Containing only a to e

300,000 bits

1*45000 + 3*13000 + 3*12000 +3*16000 + 4*9000 + 4*5000

= 224,000 bits

1*45000 + 3*13000 + 3*12000 +3*16000 + 4*9000 + 4*5000

= 224,000 bits

eg. abc = 000001010 eg. abc = 0101100


26/30



01

a:45 b:13 c:12 d:16 e:9 f:5

0

0

0

0

0

1

1

1 1

Huffman CodesThe coding schemes can be represented by trees:

Frequency Fixed-length(in thousands) codeword

a 45 000

b 13 001

c 12 010

d 16 011e 9 100

f 5 101

100

86 14

1401

58 28

a:45 b:13 c:12 d:16 e:9 f:5

0

0

0

0

0

1

1

1 1

Frequency Variable-length(in thousands) codeword

a 45 0

b 13 101

c 12 100

d 16 111e 9 1101

f 5 1100

100

55

0125 30

0

0

0

1

1

1

a:45

14

e:9 f:5

0 1d:16b:13 c:12

01

a:45 b:13 c:12 d:16 e:9 f:5

0

0

0

0

0

1

1

1 114

0158 28

a:45 b:13 c:12 d:16 e:9 f:5

0

0

0

0

0

1

1

1 1

86 14

1401

58 28

a:45 b:13 c:12 d:16 e:9 f:5

0

0

0

0

0

1

1

1 1

Not a fullbinary tree

A full binary treeevery nonleaf node

has 2 children

A file of 100,000

characters.


27/30



Huffman Codes

Frequency Codeword

a 45000 0b 13000 101

c 12000 100

d 16000 111

e 9000 1101

f 5000 1100

100

55

0125 30

0

0

0

1

1

1

a:45

14

e:9 f:5

0 1d:16b:13 c:12

To find an optimal code for a file:

1. The coding must be unambiguous.Consider codes in which no codeword is also a

prefix of other codeword. => Prefix Codes

Prefix Codes are unambiguous.

Once the codewords are decided, it is easy tocompress (encode) and decompress (decode).

2. File size must be smallest.=> Can be represented by a full binary tree.=> Usually less frequent characters are at bottom

Let C be the alphabet (eg. C={a,b,c,d,e,f})For each character c, no. of bits to encode all cs

occurrences = freqc*depthcFile size B(T) = cCfreqc*depthcEg. abc is coded as 0101100


28/30



Huffman Codes

Huffman code (1952) was invented to solve it.

A Greedy Approach.

Q: A min-priority queue f:5 e:9 c:12 b:13 d:16 a:45

100

55

25 30

a:45

14

e:9 f:5

d:16b:13 c:12

c:12 b:13 d:16 a:4514

f:5 e:9

d:16

a:45

14

25

c:12 b:13

30

f:5 e:9

a:45

d:1614

25

c:12 b:13

30

55

f:5 e:9

d:16 a:4514 25

c:12 b:13f:5 e:9

How do we find the

optimal prefix code?


29/30



Huffman Codes

HUFFMAN(C)

1 Build Q from C

2 For i = 1 to |C|-1

3 Allocate a new node z

4 z.left = x = EXTRACT_MIN(Q)5 z.right = y = EXTRACT_MIN(Q)

6 z.freq = x.freq + y.freq

7 Insert z into Q in correct position.

8 Return EXTRACT_MIN(Q)

Q: A min-priority queue f:5 e:9 c:12 b:13 d:16 a:45

c:12 b:13 d:16 a:4514

f:5 e:9

d:16 a:4514 25

c:12 b:13f:5 e:9

.

If Q is implemented as a binary

min-heap,

Build Q from C is O(n)

EXTRACT_MIN(Q) is O(lg n)

Insert z into Q is O(lg n)

Huffman(C) is O(n lg n)

How is it greedy?


30/30


Greedy Algorithms

SummaryCasual Introduction: Two Knapsack Problems

An Activity-Selection Problem


Steps of Greedy Algorithm Design

Optimal Substructure Property

Greedy-Choice PropertyComparison with Dynamic Programming

Huffman Codes

Date post:	14-Apr-2018
Category:	Documents
Upload:	rahul-rahul
View:	246 times
Download:	0 times

greedy algorithm.ppt

Documents