S

Greedy Algorithms

Greedy Algorithm

Greedy Algorithm - Makes locally optimal choice at each stage. - For optimization problems. If the local optimum is a part of the global

optimum, we get the global optimum.

Greedy Algorithm vs Dynamic Programming

DynamicProgramming

GreedyAlgorithm

Knapsack Problem

n items a thief’s knapsack of

size W

Knapsack Problem

0-1 knapsack problem - Each item must be either taken or left behind. Fractional knapsack problem - The thief can take fractions of items.

Knapsack Problem

453

0201

0$60 $10

0$120

$135 knapsac

k($6/unit)

($5/unit)

($4/unit)

($3/unit)

n = 4,W = 50

50

Fractional Knapsack Problem

5045 3

0 20 1

0$60$10

0

$120$13

5 knapsack

($6/unit)

($5/unit)

($4/unit)($3/

unit)

10

20

2030

Greedy algorithm:greatest value per unit

50

$240

0-1 Knapsack Problem

45

30

20

10

$60

$100

$120

$135 optima

l

($6/unit)

($5/unit)

($4/unit)

($3/unit)

50 50

value

50

valueper unit

50

10

20

45

30

20

$160

$135

$220

0-1 Knapsack Problem

optimal

50 50

value

50

valueper unit

50

10

20

45

30

20

$160

$135

$220Difficult to get the

optimal solution with a greedy strategy.Dynamic Programming :

Optimal Substructure vsSubproblem Solution

48

5 7

4

1

3

2

T1,5 T1,6

T2,5 T2,6

Ti,j : the solution of a subproblem

A subproblem solution?

A local optimum?

Greedy Algorithm vs Dynamic Programming

Dynamic Programming

Greedy Algorithm

Computes all subproblems

Find a local optimum

Always finds the optimal solution

May not be able to find the optimal solution

Less efficient More efficient

Optimal Substructure vsSubproblem Solution

Subproblem solution - From all subproblem solutions Optimal substructure - With only constant number of parameters - Without subproblems or future choices

Usuallytop-down

UsuallyBottom-up

Huffman Codes

A lossless data compression algorithm. It uses variable-length code

Variable-Length Code

Six characters : a, b, c, d, e, f

How can we represent them with binary strings?

a b c d e fFixed-length

000 001 010 011 100 101

Variable-Length Code

Six characters : a, b, c, d, e, f

How can we represent them with binary strings?

a b c d e fVariable-length

0 1 00 01 10 11

What 0010 menas?

0 0 1 0 = a a b a0 01 0 = a d a00 10 = c e…

Prefix Code

Six characters : a, b, c, d, e, f

How can we represent them with binary strings?

a b c d e fVariable-length

0 101 100 111 1101 1100

No codeword is a prefix of another codeword.

Prefix Code

Six characters : a, b, c, d, e, f

How can we represent them with binary strings?

a b c d e fVariable-length

0 101 100 111 1101 1100

No codeword is a prefix of another codeword.

Variable-Length Code

a b c d e f Total

Frequency 45 13 12 16 9 5 100Fixed-length 000 001 010 011 100 101 300

Variable-length

0 101 100 111 1101

1100

224

Is it the optimal way?

Variable-Length Code

a b c d e f Total

Frequency 45 13 12 16 9 5 100Fixed-length 000 001 010 011 100 101 300

Variable-length

0 101 100 111 1101

1100

224

Alternative? 0 1100

1101

10 1110

1111

231

Huffman Tree

100cha

rcode

freq

a 000 45b 001 13c 010 12d 011 16e 100 9f 101 5

Fixed-length code

86 14

58 1428

a: 45b: 13

c: 12d: 16

e: 9f: 5

0 1

0 1 0

0 1 0 1 0 1

Huffman Tree

char

code

freq

a 0 45b 101 13c 100 12d 111 16e 110

19

f 1100

5

Variable-length code

100

55

25

14

30

a: 45

b: 13c: 12 d: 16

f: 5 e: 9

0 1

0 1

0 1

10

0 1

Huffman’s Algorithm

A

B0

0

Every non-leaf node has two children.

B

0

Observation 1The longest code : at least 2 characters

Huffman’s Algorithm

C H

I J

G10

A10

F10

Observation 1Observation 2

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two characters

D E

B10

Huffman’s Algorithm

Observation 1Observation 2

Observation 3

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two charactersA non-leaf node : handled like a leaf node

d: 1614

30 x: 30T T ’

Huffman’s Algorithm

d: 1614

30 x: 30

A non-leaf node : handled like a leaf node

Observation 3

e: 9 f: 510

T T ’10

Total length in T = Total length in T ‘ + 16 * 1 + (9 + 5) * 2

Huffman’s Algorithm

Observation 1Observation 2

Observation 3

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two charactersA non-leaf node : handled like a leaf node

d: 1614

30 x: 30T T ’

Huffman’s Algorithm

a: 45 b: 13 c: 12 d: 16 e: 9 f: 5

Merging two least frequent nodes.

Huffman’s Algorithm

a: 45 b: 13 c: 12 d: 16

e: 9 f: 5

Merging two least frequent nodes.

1410

Huffman’s Algorithm

a: 45

c: 12 b: 13

d: 16

e: 9 f: 5

Merging two least frequent nodes.

1410

2510

Huffman’s Algorithm

a: 45

c: 12 b: 13 d: 16

e: 9 f: 5

Merging two least frequent nodes.

1410

2510

3010

Huffman’s Algorithm

a: 45

c: 12 b: 13 d: 16

e: 9 f: 5

Merging two least frequent nodes.

1410

2510

3010

550 1

Huffman’s Algorithm

a: 45

c: 12 b: 13 d: 16

e: 9 f: 5

Merging two least frequent nodes.

1410

2510

3010

550 1

1000 1 cha

rcode

a 0b 101c 100d 111e 110

1f 110

0

Greedy Algorithm

Optimization Algorithms : finds a proper local optimum.

Not as powerful as Dynamic Programming, but simpler.

Greedy algorithms - Knapsack Problem - Huffman Code

S

Graphs

What is a graph?

1 2

3 4

5

V = { 1, 2, 3, 4, 5 }E = { {1,2}, {1,3}, {2,3}, {2,4}, {2,5}, {3,4} }

G = ( V, E )

Directed and Undirected

1 2

3 4

5

Directed graph

1 2

3 4

5

Undirected graph

Representations of Graphs

1 2

3 4

5

Directed graph

12345

2 /3122 /

4 /3 /3 /

Adjacency-list

Representations of Graphs

1 2

3 4

5

Directed graph Adjacency-Matrix1 2 3 4

5

5

34

21

Representations of Graphs

Adjacency-Matrix

Adjacency-list

Adjacency List Adjacency Matrix

space |V| + |E| |V|2

Finding all edges

|V| + |E| |V|2

Finding one edge

num of edges 1

Adjacency Matrix of Graphs

1 2

3 4

5

Directed graph Adjacency-Matrix1 2 3 4

5

5

34

21

Adjacency Matrix of Graphs

A = AT =

Weighted Graphs

1 2

3 4

5

1.7

0.4 2.0

-0.33.1

-0.23.6-

2.1