08.10.09 IT 60101: Lecture #20 1

Foundation of Computing Systems

Lecture 20

Classic Optimization Problems

08.10.09 IT 60101: Lecture #20 2

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 3

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 4

Coin Changing Problem

=

08.10.09 IT 60101: Lecture #20 5

Coin Changing Problem

• Problem Definition

Given a set of denomination of the coins, say c1, c2, c3, …, cn, such that c1 > c2 > c3 > …> cn = 1.

Given an amount A, the problem is to determine the minimum number of coins needed to make A in change.

Note that• The problem may assume cn=1, so that it is always possible to

make change for any amount A.

• Any number of coins for each denomination is available.

08.10.09 IT 60101: Lecture #20 6

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 7

Optimal Retrieval Problem

Block 1Block 2

Block 3Block 4

08.10.09 IT 60101: Lecture #20 8

Optimal Retrieval Problem

Problem Definition Suppose, that n programs having length l1, l2, …, ln are stored on a tape. If

the programs are stored in the order li1, li2, …, lin, the time to retrieve any program ik is

(We assume that retrieval time starts at the beginning of the tape. The above retrieval time for a program ik results from the fact that in order to retrieve program ik, we must pass all of the programs stored before program ik).

The average retrieval time is defined as

The problem is to find an ordering of the program to minimize the average retrieval time.

k

jik jlT

1

n

kkTn

T1

1

08.10.09 IT 60101: Lecture #20 9

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 10

Traveling Salesman Problem

08.10.09 IT 60101: Lecture #20 11

Traveling Salesman Problem

Problem definition A salesman has a list of cities, each of which he must visit exactly

once. There are direct roads between each pair of cities on the list. The problem is to find route the salesman should follow so that he travels the shortest possible distance on a round trip , starting at any one of the cities and then returning there. He should not travel a city twice except the starting city.

08.10.09 IT 60101: Lecture #20 12

Different Versions of Traveling Salesman Problems

Hamiltonian Cycle Problem This problem is to find Hamiltonian cycle with minimum weight graph in

graph theory and stated as below.

Given an connected weighted graph and it is required to find a tour (a cycle through all the vertices exactly once except the first ) of minimum weight.

Euclidian Traveling Salesman Problem This problem is to determine the shortest path closed tour that connects a

given set of n-points on a plane.

Bitonic Tour Problem A tour starts at the left most point, go strictly left to right, and then go strictly

right to left and finally back to the starting point. The objective function and constraint is same as in the Traveling Salesman Problem.

08.10.09 IT 60101: Lecture #20 13

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 14

Knapsack Problem

08.10.09 IT 60101: Lecture #20 15

Knapsack Problem

The knapsack problem can be posed as follows.

– A thief robbing a store finds n items: the i-th item is worth vi and weight wi. The thief would take as valuable as possible but he can carry at most W weights in his knapsack. Which items should the thief take?

08.10.09 IT 60101: Lecture #20 16

Knapsack Problems

The knapsack problem can be generalized and formally stated as follows.

Given n items and a knapsack. The knapsack has a capacity W. Any item say i, has a weight wi and a value vi per unit. If a quantity xi of the item i is placed into the knapsack, then a profit vi.xi is earned.

The objective is to obtain a filling of the knapsack that maximizes the total profit earned.

Since the knapsack capacity is W, we require the total weight of all chosen object to be at most W.

Formally the problem may be stated as follows.

n

iii xv

1

n

iii Wwx

1Maximize Subject to

08.10.09 IT 60101: Lecture #20 17

Different Versions Knapsack Problems

Discrete Knapsack ProblemHere, all vi’s, wi’s and W are integers and each item must be taken or left behind.This problem is also called “integer” knapsack, and “0-1” knapsack problem

Continuous Knapsack ProblemIn this version of the knapsack problem, we can take any quantity (i.e. a fraction) of each item (rather that having to make a binary (0 or 1) choice for each item.)

Note:You can think 0-1 knapsack problem as being like a gold ingot while an item in continuous problem is more like gold dust.

08.10.09 IT 60101: Lecture #20 18

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 19

Some Well Known Problems

• Some problems are well known to learn designing algorithms

• Sorting • Searching• Coin Changing• Optimal Retrieval Problem• Traveling Salesman Problem• Knapsack Problem• Bin Packing Problem• Minimum Spanning Tree Problem• Shortest Path Problem• Huffman Tree Problem• Optimal Binary Search Tree Problem

08.10.09 IT 60101: Lecture #20 20

Bin Packing Problem

08.10.09 IT 60101: Lecture #20 21

Bin Packing Problem

Problem definition

Given n items of sizes s1, s2, …, sn. All sizes satisfy 0<si ≤1.

The problem is to pack these items in the fewest number of bins, given that each bin has unit capacity.