Date post: | 12-Nov-2014 |
Category: |
Documents |
Upload: | giri-saranu |
View: | 6,242 times |
Download: | 9 times |
unit5 1
Greedy Algorithm
Lecturer notes is purely design and developed by Er. Mohd. Arif Siddique
Lecturer Dept. of Computer Science & ITRGEC, Meerut
Unit 3 Greedy Algorithms 2
General form of Greedy Algorithms
Algorithm greedy (C) //C is an input set having n element
{
S // S is a solution set
while not solution (S) and C do
x extract best element from C.
if (x is feasible)
Then C C - {x} // extract X from input set C
S S{x} // add X into solution
if solution (S) = then return “no solutions”
Time complexity excluding sorting: O(n)
Unit 3 Greedy Algorithms 3
A Greedy Algorithm
A greedy algorithm always makes the choice that looks best at the moment. In dynamic programming, the optimal solution is described in a recursive
manner, and then is computed ``bottom up''. Dynamic programming is a powerful technique, but it often leads to algorithms with higher than desired running times.
An alternative design technique, called greedy algorithms. This method typically leads to simpler and faster algorithms, but it is not as powerful or as widely applicable as dynamic programming.
The greedy concept make the choice that looks best at the moment in this hope that local optimal choices lead to global optimal solution
Even when greedy algorithms do not produce the optimal solution, they often provide fast heuristics (non-optimal solution strategies), and are often used in finding good approximations.
Unit 3 Greedy Algorithms 4
Problem 1:Problem 1: Activity-Selection ProblemActivity-Selection Problem
Problem:Given a set S = {1, 2, …, n} of n proposed activities, with a
start time si and a finish time fi for each activity i, select a maximum-size
set of mutually compatible activities.
Compatible Activities:
Activities i and j are compatible if the internal [si, fi) and [sj, fj) do not
overlap, i.e, i and j are compatible if si≥ fj and sj ≥ fi
Goal: Select a maximum-size set of mutually compatible activities
Unit 3 Greedy Algorithms 5
Greedy Activity-Selection AlgorithmSort the input activities by increasing finishing time. f1 ≤ f2 ≤ . . . ≤ fn
Algo GREEDY-ACTIVITY-SELECTOR (s[ ], f[ ]){n = length [s] A={i} j = 1 For(i = 2 to n ){
if (si ≥ fj ){
A= AU{i} j = i
}}
return set A }
Time complexity excluding sorting: O(n)
Unit 3 Greedy Algorithms 6
ExampleGiven 11 activities A p q r s t u v w x y z
s 1 3 0 5 3 5 6 8 8 2 12
f 4 5 6 7 8 9 10 11 12 13 14
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Here all input activities are already sorted by increasing finishing time.
Unit 3 Greedy Algorithms 7
Example cont…
0p
q
r
s
t
u
v
w
x
y
z
1413121110987654321
Initialization A={p}
Activity ={p}
Unit 3 Greedy Algorithms 8
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p}
Activity ‘p’ and ‘q’ are not compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 9
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p}
Activity ‘p’ and ‘r’ are not compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 10
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, }
Activity ‘p’ and ‘s’ are compatible, So
add ‘s’ in to solution
Example cont…
Unit 3 Greedy Algorithms 11
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, }
Activity ‘s’ and ‘t’ arenot compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 12
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, }
Activity ‘s’ and ‘u’ arenot compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 13
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, }
Activity ‘s’ and ‘v’ arenot compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 14
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, w, }
Activity ‘s’ and ‘w’ are
compatible, So add
‘w’ in to solution
Example cont…
Unit 3 Greedy Algorithms 15
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, w, }
Activity ‘s’ and ‘x’ arenot compatible (overlapped) So
don't add it in to solution
Example cont…
Unit 3 Greedy Algorithms 16
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ={p, s, w, }
Activity ‘w’ and ‘y’ arenot compatible (overlapped), So
do not add ‘y’ in to solution
Example cont…
Unit 3 Greedy Algorithms 17
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ‘w’ and ‘z’ arecompatible, So add ‘z’
in to solution
Example cont…
Activity ={p, s, w, z }
Unit 3 Greedy Algorithms 18
p
q
r
s
t
u
v
w
x
y
z
0 1413121110987654321
Activity ‘w’ and ‘z’ arecompatible, So add ‘z’
in to solution
Example cont
Activity ={p, s, w, z }
Unit 3 Greedy Algorithms 19
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
Unit 3 Greedy Algorithms 20
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
Unit 3 Greedy Algorithms 21
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
C
Unit 3 Greedy Algorithms 22
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
A
Unit 3 Greedy Algorithms 23
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
E
Unit 3 Greedy Algorithms 24
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
ED
Unit 3 Greedy Algorithms 25
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
E F
Unit 3 Greedy Algorithms 26
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
E G
Unit 3 Greedy Algorithms 27
Activity Selection (Interval Scheduling)
Time0
A (3)
C (2)
F (6)
B (1)
D (5)
G (7)
E (4)
1 2 3 4 5 6 7 8 9 10 11
H (8)
0 1 2 3 4 5 6 7 8 9 10 11
B
C (2)
E H
Unit 3 Greedy Algorithms 28
Elements of the Greedy Strategy
․ When to apply greedy algorithms? Greedy-choice property: A global optimal solution can be
arrived at by making a locally optimal (greedy) choice. Dynamic programming needs to check the solutions to
subproblems. Optimal substructure: An optimal solution to the problem
contains within its optimal solutions to subproblems. E.g., if A is an optimal solution to S, then A' = A - {1} is an optimal
solution to S' = {i S: si f1}.
․ Greedy algorithms (heuristics) do not always produce optimal solutions.
․ Greedy algorithms v.s. dynamic programming (DP) Common: optimal substructure Difference: greedy-choice property DP can be used if greedy solutions are not optimal.
Unit 3 Greedy Algorithms 29
Problem 2:Problem 2: Fractional Knapsack Problem
The 0-1 knapsack problem
(Each item is either taken or not taken)
The fractional knapsack problem
(Allow to take fraction of items.)
Knapsack Problem
Classification of Knapsack Problem:
There are n items in a store. For i =1,2, . . . , n, item i has weight
wi > 0 and benefit bi > 0. Thief can carry a maximum weight of W
pounds in a knapsack. In this version of a problem the items can
be broken into smaller piece, so the thief may decide to carry only
a fraction of object i.
Unit 3 Greedy Algorithms 30
These line takes θ(n)
These line takes θ(n)
Fractional Knapsack Problem cont..Algo Greedy-fractional-knapsack (Item[n], w[ ], b[ ], W){Knap=0Weight = 0Benefit=0for each item i v[i] = b[i] / w[i] // value per pound
while (weight <=W) { i= Extract item of maximum value from list if(weight + w[i] ≤ W) { Knap=Knap U Item[i] weight = weight + w[i] benefit= benefit +v[i] } else{ Knap=Knap U Item[i] weight = W benefit = (W - weight) * v[i] / w[i] // Factoring item, W – weight is needed weight
}} return x}
Unit 3 Greedy Algorithms 31
Fractional Knapsack Problem cont..Algo Greedy-fractional-knapsack (Item[n], w[ ], b[ ], W){Knap=0Weight = 0Benefit=0for each item i v[i] = b[i] / w[i] // value per pound
while (weight <=W) { i= Extract item of maximum value from list if(weight + w[i] ≤ W) { Knap=Knap U Item[i] weight = weight + w[i] benefit= benefit +v[i] } else{ Knap=Knap U Item[i] weight = W benefit = (W - weight) * v[i] / w[i] // Factoring item, W – weight is needed weight
}} return Knap}
So time co
mplexity e
xcluding so
rting: O
(n)
Unit 3 Greedy Algorithms 32
Example
Item A B C D E F G
Benefits (Rs.) 12 10 8 11 14 7 9
Weight (Kg.) 4 6 5 7 3 1 6
Let S ={a, b, c, d, e, f, g} denote a set of objects with weights and benefits
as given in the table below. What is an optimal solution to the fractional
knapsack problem for S assuming that we have a sack that can hold objects
with total weight 18?
Carrying capacity W = 18 Kg.
Unit 3 Greedy Algorithms 33
Example cont…
First we must calculate the “value" for the each items,which is defined as value = benefits/weight. I do this andgive them in the table below:
Item A B C D E F G
benefits 12 10 8 11 14 7 9
Weight 4 6 5 7 3 1 6
Value 3 1.67 1.6 1.57 4.67 7 1.5
value = benefit / weight
Unit 3 Greedy Algorithms 34
Example cont…
Item A B C D E F G
benefits 12 10 8 11 14 7 9
Weight 4 6 5 7 3 1 6
Value 3 1.67 1.6 1.57 4.67 7 1.5
Sort this table according to decreasing value
Item F E A B C D G
benefits 7 14 12 10 8 11 9
Weight 1 3 4 6 5 7 6
Value 7 4.67 3 1.67 1.6 1.57 1.5
Unit 3 Greedy Algorithms 35
Example cont…
Item F E A B C D G
benefits 7 14 12 10 8 11 9
Weight 1 3 4 6 5 7 6
Value 7 4.67 3 1.67 1.6 1.57 1.5
W=18
Initially
Knapsack
weight = 0
benefits =0
W=18
Unit 3 Greedy Algorithms 36
1.5
6
9
G
1.57
7
11
D
1.6
5
8
C
1.67
6
10
B
3
4
12
A
4.67
3
14
E
7
1
7
F
Weight
Value
benefits
Item
W=18
FKnapsack
weight = 1
benefits =7
W=18
Select maximum valued item ‘F’, Here (weight+w[F])<W
Put whole item ‘F’ into knapsack. Add weight[F] with weight
and benefit[F] with benefit.
Example cont…
Unit 3 Greedy Algorithms 37
1.5
6
9
G
1.57
7
11
D
1.6
5
8
C
1.67
6
10
B
3
4
12
A
4.67
3
14
E
7
1
7
F
Weight
Value
benefits
Item
W=18
F, EKnapsack
weight = 1+3 =4
benefits =7+14 =21
W=18
Now select next maximum valued item ‘E’, Here (weight+w[E])<W Put whole item ‘E’ into knapsack. Add weight[E] with weight and benefit[E] with benefit.
Example cont…
Unit 3 Greedy Algorithms 38
1.5
6
9
G
1.57
7
11
D
1.6
5
8
C
1.67
6
10
B
3
4
12
A
4.67
3
14
E
7
1
7
F
Weight
Value
benefits
Item
W=18
F, E, AKnapsack W=18
Now select next maximum valued item ‘A’, Here (weight+w[A])<=W Put whole item ‘A’ into knapsack. Add weight[A] with weight and benefit[A] with benefit.
Example cont…
weight = 1+3 =4+4=8
benefits =7+14 =21+12=33
Unit 3 Greedy Algorithms 39
1.5
6
9
G
1.57
7
11
D
1.6
5
8
C
1.67
6
10
B
3
4
12
A
4.67
3
14
E
7
1
7
F
Weight
Value
benefits
Item
W=18
F, E, A, BKnapsack W=18
Now select next maximum valued item ‘B’, Here (weight+w[B])<=W Put whole item ‘B’ into knapsack. Add weight[B] with weight and benefit[B] with benefit.
Example cont…
weight = 1+3 =4+4=8+6=14
benefits =7+14 =21+12=33+10=43
Unit 3 Greedy Algorithms 40
F, E, A, B, CKnapsack W=18
Now select next maximum valued item ‘C’, Here (weight+w[C])
⊀W Put whole item ‘B’ into knapsack. And calculate weight and benefit as follows:
1.5
6
9
G
1.57
7
11
D
1.6
5
8
C
1.67
6
10
B
3
4
12
A
4.67
3
14
E
7
1
7
F
Weight
Value
benefits
Item
W=18
Example cont…
benefits =7+14 =21+12=33+10=43+(needed weight)* value[C]
=43+(4*1.6) = 43+6.4 = 49.4
needed weight = W – weight = 18 – 14 = 4Put whole item ‘B’ into knapsack.
weight = W = 18
Unit 3 Greedy Algorithms 41
Example cont…
Item F E A B C D G
benefits 7 14 12 10 8 11 9
Weight 1 3 4 6 5 7 6
Value 7 4.67 3 1.67 1.6 1.57 1.5
W=18
Remaining items D, G could not put into Knapsack (bag) because bag is full
i.e weight = W
F, E, A, B, CKnapsack =
Weight in bag = W = 18
Benefits = Rs. 49.4
Unit 3 Greedy Algorithms 42
Complexity Analysis
If the items are already sorted into decreasing order of vi / wi, then the while-loop takes a time in O(n); Therefore, the total time including the sort is in O(n log n).
If we keep the items in heap with largest vi/wi at the root. Then
creating the heap takes O(n) time
while-loop now takes O(log n) time (since heap property must be restored after the removal of root)
Unit 3 Greedy Algorithms 43
Huffman coding is an algorithm used for lossless data compression developed by David A. Huffman as a PhD student at MIT in 1952, and published in A Method for the Construction of Minimum-Redundancy Codes.
Professor David A. Huffman (August 9, 1925 - October 7, 1999)
"Huffman Codes" are widely used applications that involve the compression and transmission of digital data, such as: fax machines, modems, computer networks, and high-definition television (HDTV), etc.
Introduction
Unit 3 Greedy Algorithms 44
Reducing the space required to store files on disk or tape
Reducing the time to transmit large files.
Huffman savings are between 20% - 90%
Motivation
Unit 3 Greedy Algorithms 45
Basic Idea
Huffman Codes:
Suppose we wish to save a text (ASCII) file on the disk or to transmit it though a network using an encoding scheme that minimizes the number of bits required. In fixed-size-encoding -scheme, without compression, characters are typically encoded by their ASCII codes with 8 bits per character. We can do better if we variable-size-encoding. In the variable-size-encoding-scheme we assign different code of different length according to their frequencies of occurrences.
Unit 3 Greedy Algorithms 46
Example:
Suppose you have a file with 100K characters.
For simplicity assume that there are only 6 distinct characters in the file from a through f, with frequencies as indicated below.
We represent the file using a unique binary string for each character.
a b c d e f
Frequency(in 1000s)
45 13 12 16 9 5
Fixed-length codeword
000 001 010 011 100 101
Space = (45*3 + 13*3 + 12*3 + 16*3 + 9*3 + 5*3) * 1000
= 300K bits
Unit 3 Greedy Algorithms 47
Can we do better ??
By using variable-length codes instead of fixed-length codes.
Idea : Giving frequent characters short codewords, and infrequent characters long codewords.
i.e. The length of the encoded character is inversely proportional to that character's frequency.
YES !!
Space = (45*1 + 13*3 + 12*3 + 16*3 + 9*4 + 5*4) * 1000=224K bits
a b c d e f Bit Required
Frequency(in 1000s)
45 13 12 16 9 5
Fixed-length codeword
000 001 010 011 100 101 300K bits
Variable-length codeword
0 101 100 111 1101 1100 224K bits
Savings = 25%
Unit 3 Greedy Algorithms 48
Codes in which no codeword is also a prefix of some other codeword.
("prefix-free codes" would have been a more appropriate name)
Variable-length codeword
0 101 100 111 1101 1100
It is very easy to encode and decode using prefix codes.
No Ambiguity !!
It is possible to show (although we won't do so here) that the optimal data compression achievable by a character code can always be achieved with a prefix code, so there is no loss of generality in restricting attention to prefix codes.
PREFIX CODES
Unit 3 Greedy Algorithms 49
Huffman invented a greedy algorithm that constructs an optimal prefix code called a Huffman code. The algorithm builds the tree T corresponding to the optimal code in a bottom-up manner.
It begins with a set of |C| leaves and performs a sequence of |C| - 1 "merging" operations to create the final tree.
Greedy Choice?
The two smallest nodes are chosen at each step, and this local decision results in a globally optimal encoding tree.
In general, greedy algorithms use small-grained, or local minimal/maximal choices to result in a global minimum/maximum
Constructing a Huffman code
Unit 3 Greedy Algorithms 50
Greedy Algorithm for Huffman Tree
Algo Greedy-Huffman(C,f){n length[C]Q C:ffor (i 1 to n-1) { z Allocate-Node x left[z] Extract-Min(Q) y right[z] Extract-Min(Q) f[z] f[x]+f[y] Insert(Q, z) }return Extract-Min(Q)}
Unit 3 Greedy Algorithms 51
Constructing a Huffman Coding Tree
a:45b:13e:9f:5 c:12 d:16
14
a:45b:13c:12 d:16
e:9f:5
14
From following Lines
z Allocate-Node
x left[z] Extract-Min(Q)
y right[z] Extract-Min(Q)
f[z] f[x]+f[y]
From following LinesInsert(Q, z)
Unit 3 Greedy Algorithms 52
Constructing a Huffman Coding Tree
25
a:45b:13c:12 d:16
e:9f:5
14
a:4525
b:13c:12
d:16
e:9f:5
14
From following Lines
z Allocate-Node
x left[z] Extract-Min(Q)
y right[z] Extract-Min(Q)
f[z] f[x]+f[y]
From following LinesInsert(Q, z)
Unit 3 Greedy Algorithms 53
30
Constructing a Huffman Coding Tree
a:4525
b:13c:12
d:16
e:9f:5
14
30 a:45
d:16
e:9f:5
14
25
b:13c:12
From following Lines
z Allocate-Node
x left[z] Extract-Min(Q)
y right[z] Extract-Min(Q)
f[z] f[x]+f[y]
From following LinesInsert(Q, z)
Unit 3 Greedy Algorithms 54
55
3025
14
a:45
b:13
e:9f:5
c:12 d:16
Constructing a Huffman Coding Tree
From following Lines
z Allocate-Node
x left[z] Extract-Min(Q)
y right[z] Extract-Min(Q)
f[z] f[x]+f[y]
Insert(Q, z)
Unit 3 Greedy Algorithms 55
100
55
3025
14
a:45
b:13
e:9f:5
c:12 d:16
Constructing a Huffman Coding Tree
From following Lines
z Allocate-Node
x left[z] Extract-Min(Q)
y right[z] Extract-Min(Q)
f[z] f[x]+f[y]
Insert(Q, z)
Unit 3 Greedy Algorithms 56
Adding binary code in this tree
100
55
3025
14
a:45
b:13
e:9f:5
c:12 d:16
00
0
0
11
1
1
10
Add ‘0’ with left child, and ‘1’with right child of each node
Unit 3 Greedy Algorithms 57
100
55
3025
14
a:45
b:13
e:9f:5
c:12 d:16
00
0
0
11
1
1
10
Finding binary code from this tree
Char Freq Codeword
a 45 0
b 13 101
c 12 100
d 16 111
e 9 1101
f 5 1100
Unit 3 Greedy Algorithms 58
The analysis of the running time of Huffman's algorithm assumes that Q is implemented as a binary min-heap.
• For a set C of n characters, the initialization of Q in line 2 can be performed in O(n) time using the BUILD-MIN-HEAP procedure.
• The for loop in lines 3-8 is executed exactly |n| - 1 times. Each heap operation requires time O(log n). The loop contributes = (|n| - 1) * O(log n) = O(nlog n)
Thus, the total running time of HUFFMAN on a set of n characters = O(n) + O(nlog n)
= O(n log n)
Running Time analys
Unit 3 Greedy Algorithms 59
Thank You