VLSI CAD: Logic to Layout Algorithms -...

transcript

SI545VLSI CAD: Logic to Layout

Algorithms

Announcements

TA for this course

LeileiWang, wangll@shanghaitech.edu.cn

Office hour: 1:30-2:30pm, Thurs., 301@H2.

Course page on Piazza

References and Copyright

Slides used (Modified when necessary)

Majid Sarrafzadeh, 2001

Department of Computer Science, UCLA

Kurt Keutzer, Dept. of EECS, UC-Berekeley

http://www-cad.eecs.berkeley.edu/~niraj/ee244/index.htm

Rajesh Gupta, UC-Irvinehttp://www.ics.uci.edu/~rgupta/ics280.html

Kia Bazargan, University of Minnesota

Naveed A. Sherwani, 1999

Computational Complexity

Combinatorial Optimization Problems

Problems with discrete variables

Examples:

SORTING

Given N integer numbers, write them in increasing order.

KNAPSACK

Given a bag of size S, and items {(s1, v1), (s2, v2), ..., (sn, vn)},

where si and vi are the size and value of item i respectively, find a

subset of items with maximum overall value that fit in the bag.

Graph Definition

Graph: set of “objects” and their “connections”

Formal definition:

G = (V, E), V={v1, v2, ..., vn}, E={e1, e2, ..., em}

V: set of vertices (nodes), E: set of edges (links, arcs)

Directed graph: ek = (vi, vj)

Undirected graph: ek={vi, vj}

Weighted graph: w: E R, w(ek) is the “weight” of ek.

Decision Problem vs. Optimization Problem

Decision problem – ask for a “yes” or “no” answer

Example: Hamiltonian cycle - simple cycle (no repeated vertices)

that contains all nodes in the graph; TRAVELING SALESMAN.

Optimization problem – find optimal value of a target variable

Example: Traveling salesman problem (TSP) - min-cost Hamiltonian

Optimization problem is harder than its decision problem version11

Algorithm

An algorithm defines a procedure for solving a computational

problem

Examples:

Quick sort, bubble sort, insert sort, heap sort

Dynamic programming method for the knapsack problem

http://en.wikipedia.org/wiki/Knapsack_problem

http://en.wikipedia.org/wiki/Bubble_sorthttp://en.wikipedia.org/wiki/Quicksort

http://en.wikipedia.org/wiki/Heapsorthttp://en.wikipedia.org/wiki/Insert_sort

Bubble Sort

for (j=1; j < N; j++) { for (i=1; i < N-j+1; i++) {

if (a[i] > a[i+1]) {hold = a[i];a[i] = a[i+1];a[i+1] = hold;

e.g., initial ordering: 10, 5, 3, 8, 6, 1

http://en.wikipedia.org/wiki/Bubble_sort

Bubble Sort Animation

Index [htt

Image:B

Complexity

Measure the efficiency of a algorithm

Time and memory

Computational complexity

Time complexity, space complexity

Average case, worst case

Scalability (with respect to input size n) is important

How does the running time of an algorithm change when the input size doubles?

Function of input size (n).Examples: n2+3n, 2n, nlog n, ...

Generally, large input sizes are of interest (n > 1,000 or even n > 1,000,000)

Function Growth Examples

Function Growth

1 6 11 16 21 26 n

f(n) 5n 0.1n^2+2n+40 2 n log n 0.00001 2^n

Function Growth

10 210 410 610 810 1010 n

f(n)5n 0.1n^2+2n+40 2 n log n 0.00001 2^n

Asymptotic Notions

A notion that ignores the “constants” and describes the “trend” of a function for large values of the input.

Definition

Big-Oh notation f(n) = O(g(n)).

if constants K and n0 can be found such that: n n0, f(n) K g(n) g is called an “upper bound” for f (f is “of order” g: f will not grow larger than g by

more than a constant factor).

Examples: 1/3 n2 = O (n2) 0.02 n2 + 127 n + 1923 = O (n2)

Tractability: solvable in polynomial time

Polynomial time = O(nk), otherwise exponential

Asymptotic Notions (cont.)

Definition (cont.) Big-Omega notation f(n) = ( g(n) )

if constants K and n0 can be found such that: n n0, f(n) K g(n) g is called a “lower bound” for f

Question: If f(n)=O(g(n)), then is g(n)= (f(n))?

Big-Theta notation f(n) = ( g(n) )

if g is both an upper and lower bound for f- Describes the growth of a function more accurately than O or

Example:n3 + 4 n (n2)4 n2 + 1024 = (n2)

Asymptotic Notions (cont.)

How to find the order of a function?

Not always easy, esp. if you start from an algorithm.

Focus on the “dominant” term 4 n3 + 100 n2 + log n

n + n log(n)

n! > Kn > nK > log n > log log n > K

What do asymptotic notations mean in practice?

If algorithm A has “time complexity” O(n2) and algorithm B has time complexity O(n log n), then which algorithm is better?

If problem P has a lower bound of (n log n), then there is NO WAY you can find an algorithm that solves the problem in O(n) time.

The Complexity Classes P vs. NP

A problem’s complexity = the time complexity of the most efficient possible algorithm.

Problems are classified into “easier” and “harder” categories Class P: a polynomial time algorithm is known for the problem (hence, it

is a tractable problem)Examples: search problem, minimum spanning tree, etc.

Class NP (non-deterministic polynomial time) A decision problem Can be solved in polynomial time on a nondeterministic computer

Nondeterministic: multiple possibilities available for next step in computation A solution is verifiable in polynomial time on a deterministic computer

Example: TRAVELING SALESMAN (=Hamilton cycle with length ≤ k).

P NP Is P = NP? (Find out and become famous!)

NP-Complete

Practically, for a problem in NP but not in P: polynomial solution not found yet (probably does not exist) exact (optimal) solution can be found using an algorithm with exponential time complexity

Class NP-complete (NPC) The most difficult problems in the NP class Till now, the best algorithm for worst-case NPC problems has

exponential time complexity

http://zh.wikipedia.org/wiki/File:Complexity_classes.svg

Examples of NPC Problems

Does a graph have a Hamiltonian cycle? Hamiltonian cycle/TRAVELING SALESMAN

3SAT: Given a Boolean expression expressed as POS with 3 literals in each sum, is it satisfiable? Example: (𝑥 + 𝑦 + 𝑧)( 𝑥 + 𝑦)( 𝑦 + 𝑧)( 𝑥 + 𝑦 + 𝑧) 2SAT can be solved in polynomial time!

Find a maximal clique in a graph Clique = set of vertices so that every pair of vertices in the set is connected by an

edge (complete subgraph) Find a maximal independent set in a graph

A max set of vertices so that no pair of vertices is connected

VLSI fixed-outline floorplanning

“Bible” of NP-completeness M. R. Garey and D. S. Johnson, Computers and Intractability, W. H. Freeman and

Company, New York, NY, 1979.

NP-Hard

Class NP-hard A special definition - solution not verifiable in polynomial time on a

deterministic computer At least as difficult to solve as NPC Example: TSP problem (solution check O(𝑛 ∙ 𝑛!))

Most CAD problems are NPC, NP-hard, or worse Be happy with a “reasonably good” solution

Reading material on time complexity Textbook [Ch. 4.2, Wang], plus Princeton slides.

http://en.wikipedia.org/wiki/NP_(complexity)Stephen Cook, 1982’s Turing award

Algorithm Types

Deterministic vs. probabilistic (or randomized)

Graph algorithms

Heuristics: apply heuristics to find a good, but not necessarily

optimal solution

Greedy algorithm

Dynamic programming

Branch-and-bound

Simulated annealing

Genetic

Mathematical programming

Read [Ch. 4.4, Wang] for the detailed discussion on these algorithms.

Graph Algorithms

Graph Representation: Adjacency List

a · b · d · c ·

a · d ·

e · b ·

a · b · d ·

Graph Representation: Adjacency Matrix

b c d ea1 1 1 00

0 0 1 01

0 0 0 01

1 0 0 11

0 0 1 00

b c d ea1 0 1 00

0 0 1 00

0 0 0 01

0 0 0 11

0 0 0 00

Edge/Vertex Weights in Graphs

Edge weights Usually represent the “cost” of an edge

Examples: Distance between two cities

Width of a data bus

Representation Adjacency matrix: instead of 0/1, keep weight

Adjacency list: keep the weight in the linked list item

Node weight Usually used to enforce some “capacity” constraint

Examples: The size of gates in a circuit

The delay of operations in a “data dependency graph”

Hypergraphs

Hypergraph definition

Similar to graphs, but edges not between pairs of vertices, but

between a set of vertices

Directed/undirected versions possible

Just like graphs, a node can be the source (or be connected to) of

multiple hyperedges

Data structure?

https://www8.cs.umu.se/kurser/TDBA77/VT06/algorithms/BOOK/BOOK3/NODE132.HTM

Graph Search Algorithms

Purpose: to visit all the nodes

Algorithms

Depth-first search

Breadth-first search

Topological (on directed acyclic graph)

Examples

Depth-First Search Algorithm

struct vertex {...int marked;

dfs ( v ) v.marked 1print v

for each (v, u) Eif (u.marked != 1) // not visited yet?

dfs (u)

Algorithm DEPTH_FIRST_SEARCH ( V, E )

for each v Vv.marked 0 // not visited yet

for each v Vif (v.marked == 0)

dfs (v)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”)....

dfs ( a ) a.marked 1print a

for each (a, u) Eif (u.marked != 1)

dfs (u)

for each (a, u) Eif (b.marked != 1)

dfs (b)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”)....

dfs(“b”)....u=b

dfs ( b ) b.marked 1print b

for each (b, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”).... dfs(“b”)

for each (b, u) Eif (a.marked != 1)

dfs (a)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”).... dfs(“b”)

....u=a

for each (b, u) Eif (d.marked != 1)

dfs (d)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”).... dfs(“b”)

.... dfs(“d”)....

dfs ( d ) d.marked 1print d

for each (d, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”).... dfs(“b”)

.... dfs(“d”)....

for each (d, u) Eif (e.marked != 1)

dfs (e)

a · b · d · c ·

a · d ·

e · b ·

dfs(“a”).... dfs(“b”)

.... dfs(“d”).... dfs(“e”)

dfs ( e ) e.marked 1print e

for each (e, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

.... dfs(“d”).... dfs(“e”)

for each (e, u) Eif (d.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

.... dfs(“d”).... dfs(“e”)

for each (e, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

.... dfs(“d”).... dfs(“e”)

for each (d, u) Eif (b.marked != 1)

dfs (b)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

.... dfs(“d”)....

for each (d, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

.... dfs(“d”)....

for each (b, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”).... dfs(“b”)

for each (a, u) Eif (d.marked != 1)

dfs (d)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”)....

for each (a, u) Eif (c.marked != 1)

dfs (c)

a · b · d · c ·

a · d ·

e · b ·

a b d e

dfs(“a”)....

dfs(“c”)....

dfs ( c ) c.marked 1print c

for each (c, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e c

dfs(“a”).... dfs(“c”)

for each (c, u) Eif (a.marked != 1)

dfs (a)

a · b · d · c ·

a · d ·

e · b ·

a b d e c

dfs(“a”).... dfs(“c”)

....u=a

for each (c, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e c

dfs(“a”).... dfs(“c”)

for each (a, u) Eif (u.marked != 1)

dfs (u)

a · b · d · c ·

a · d ·

e · b ·

a b d e c

dfs(“a”)....

Breadth-First Search Algorithm

bfs ( v , Q) v.marked 1

for each (v, u) Eif (u.marked != 1) // not visited yet?

Q Q + u

Algorithm BREADTH_FIRST_SEARCH ( V, E )Q {v0} // an arbitrary nodewhile Q != {}

v Q.pop()if (v.marked != 1)

print vbfs (v) // explore successors

There is something wrong with this code. What is it?

Distance in (non-weighted) Graphs

Distance dG(u,v):

Length of a shortest u--v path in G.

d(u,v)=2

d(x,y) =

Moore’s Breadth-First Search Algorithm

Objective:

Find d(u,v) for a given pair (u,v) and find a shortest path u--v

How does it work?

Do BFS, and assign (w) the first time you visit a node.

(w)=depth in BFS.

Data structure

Q a queue containing vertices to be visited

(w) length of the shortest path u--w (initially )

(w) parent node of w on u--w

Algorithm:1. Initialize

(w) for wu

2. If Q , x pop(Q) else stop: “no path u--v”

3. (x,y)E,if (y)=

(y) (x)+1

4. If (v)= return to step 25. Follow (w) pointers from v to u.

E. F. Moore (1959), The shortest path through a maze. In Proceedings of the International

Symposium on the Theory of Switching, Harvard University Press, pp. 285–292.

v7v8 v9

v10Q = u

Node u v1 v2 v3 v4 v5 v6 v7 v8 v9 v10

- - - - - - - - - - -

Algorithm:

1. Initialize

(w) for wu

2. If Q , x pop(Q)

else stop: “no path u--v”

3. (x,y)E,

if (y)=

(y) (x)+1

4. If (v)= return to step 2

5. Follow (w) pointers from v to u.

v7v8 v9

Q = v1, v2, v5

0 1 1 1

- u u - - u - - - - -

Algorithm:

1. Initialize

(w) for wu

2. If Q , x pop(Q)

3. (x,y)E,

if (y)=

(y) (x)+1

v7v8 v9

Q = v4, v3, v6, v10

0 1 1 2 2 1 2 2

- u u v2 v1 u v5 - - - v5

Algorithm:

1. Initialize

(w) for wu

2. If Q , x pop(Q)

3. (x,y)E,

if (y)=

(y) (x)+1

v7v8 v9

Q = v7

0 1 1 2 2 1 2 3 2

- u u v2 v1 u v5 v4 - - v5

Algorithm:

1. Initialize

(w) for wu

2. If Q , x pop(Q)

3. (x,y)E,

if (y)=

(y) (x)+1

Notes on Moore’s Algorithm

Time complexity?

Space complexity?

Maze Routing

Problem: find a connection from A to B

Quite useful for two pin nets, rip-up-and-reroute, etc.

Obstacle

Lee’s Algorithm

1010 10

C. Y. Lee (1961), An algorithm for path connection and its applications. IRE Transactions on

Electronic Computers, EC-10(3), pp. 346–365.

A more space-efficient version

Other improvements

Line search algorithms

Reduced memory requirements

Sends out a line and uses it to guide the search

No guarantees on being able to find a route

Distance in Weighted Graphs

Why does a simple BFS not work any more?

Your locally best solution is not your globally

best one

First, v1 and v2 will be visited

v2 should be revisited

Dijkstra’s Algorithm

Objective:

Find d(u,v) for all pairs (u,v) (fixed u) and the corresponding shortest paths u--v

How does it work?

Start from the source, augment the set of nodes whose shortest path is found.

decrease (w) from to d(u,w) as you find shorter distances to w. (w) changed accordingly.

Data structure:

S the set of nodes whose d(u,v) is found

(w) current length of the shortest path u--w

(w) current parent node of w on u—w

Dijkstra’s Algorithm

Algorithm:

1. Initialize

(v) for vu

S {u}, vj = u

2. For each vS’ s.t. (vj,v)E

If (v) > (vj) + w(vj,v)

(v) (ui) + w(vj,v)

(v) vj

3. Find m=min{(v)|vS’ } and (vk)==m

S S {vk}

4. If |S|<|V|, goto step 2

Dijkstra’s Algorithm - an example

Edges examined

updated, min picked

S Augmented

Edges examined

updated, min picked

S Augmented

Edges examined

updated, min picked

S augmented

Edges examined

updated, min picked

S augmented

Edges examined

updated, min picked

S augmented

Dijkstra’s Algorithm - why does it work?

Proof by contradiction

Suppose v is the first node being added to S such that (v) > d(u0,v)

(d(u0,v) is the “real” shortest u0--v path)

The assumption that (v) and d(u0,v) are different, means there are

different paths with lengths (v) and d(u0,v)

Consider the path that has length d(u0,v). Let x be the first node in

S’ on this path

(v) > d(u0,v) by assumption and d(u0,v) ≥ (x)=>

(v) > (x) => contradiction

(since the min value is the one added to S in each iteration)

Static Timing Analysis (STA)

Finding the longest path in a general graph is NP-complete, even

when edges are not weighted

Polynomial for DAG (directed acyclic graphs)

Topological search on a DAG

In circuit graphs, “static timing analysis (STA)”…

…refers to the problem of finding the max delay from the input pins

of the circuit (esp nodes) to each gate

Max delay of the output pins determines clock period

In sequential circuits, FF input acts as output pin, FF output acts as

input pin

Critical path is a path with max delay among all paths

In addition to the “arrival time” of each node, we are interested in

knowing the “slack” of each node/edge

STA Example: Arrival Times

Assumptions: All inputs arrive at time 0 All gate delays = 1 All wire delays = 0

Question: Arrival time of each gate? Circuit delay?

ti = max {tj} + di

STA Example: Required Times

Assumptions:

All inputs arrive at time 0

All gate delays = 1, wire delay = 0

Clock period = 7

Question: maximum required time (RT) of each gate? (i.e., if the gate output is generated later than RT, clk period is violated)

ri = min {rj-dj }

STA Example: Slack

Assumptions:

All inputs arrive at time 0

All gate delays = 1, wire delay = 0

Clock period = 7

Question: What is the maximum amount of delay each gate can be slower not to violate timing?

si = ri-ti

6-4=2 7-4=3

STA: Issues

If the delay of one gate changes, what is the time complexity of

updating the slack of all nodes?

How can slack be used?

Maintain a list of K-most critical paths?

Refer to Chapter 5 of

Sachin S. Sapatnekar, Timing, Kluwer Academic Publishers, Boston,

MA, 2004.

Another Application of Longest Paths

Layout compaction

Given a set of blocks, place them in the most compact way possible

The 2D compaction problem is often solved as a sequence of 1D

problems, in the x and y directions

𝑥𝑏 − 𝑥𝑎 ≥ 𝑊𝑎

A set of such “≥” constraints

represents a longest path problem

Example

2.4 1.2

1.21.2

Horizontal constraint graph Gh

Vertical constraint graph Gv

Example: negative-weight edges

Suppose you need a and b to be “stuck” together

Then, xb – xa = wa

In other words,

xb – xa ≤ wa

xb – xa ≥ wa

To represent these in a longest path problem (use “≥”), rewrite as

xa – xb ≥ -wa

xb – xa ≥ wa

a b-wa

Bellman-Ford Algorithm

Dijkstra’s doesn’t work when a graph has negative edges

Intuition – we cannot be greedy any more on the assumption that

the lengths of paths will only increase in the future

Bellman-Ford algorithm detects negative cycles (returns false) or

returns the shortest path-tree

Bellman-Ford Algorithm (shortest path)

Bellman-Ford(G,w,s)

for each vertex u V

d(u) = parent(u) = NIL

d(s) = 0

for i 1 to |V|-1

for each edge (u,v) E

d(v) = min[d(v),d(u)+w(u,v)]

for each edge (u,v) E do

if d(u) + w(u,v) < d(v) then // negative cycle

return false

return true

http://en.wikipedia.org/wiki/Bellman-Ford_algorithm

Bellman-Ford Example

Bellman-Ford running time:

(|V|-1)|E| + |E| = (VE)

Minimum Spanning Tree (MST)

Tree (usually undirected)

Connected graph with no cycles

|E| = |V| - 1

Spanning tree

Connected subgraph that covers all vertices

If the original graph not tree, graph has

several spanning trees

Minimum spanning tree

Spanning tree with minimum sum of edge weights(among all spanning trees)

Example: build a railway system to connect N cities, with the smallest total length of the railroad

http://en.wikipedia.org/wiki/File:Minimum_spanning_tree.svg

Difference Between MST and Shortest Path

Why can’t Dijkstra solve the MST problem?

Shortest path: min sum of edge weight to individual nodes Each node appear at most once in the shortest path

MST: min sum of TOTAL edge weights that connect all vertices

Minimum Spanning Tree Algorithms

Basic idea:

Start from a vertex (or edge), and expand the tree,

avoiding loops (i.e., add a “safe” edge)

Pick the minimum weight edge at each step

Known algorithms

Prim: start from a vertex, expand the connected tree

Kruskal: start with the min weight edge, add min weight edges

while avoiding cycles (build a forest of small trees, merge them)

Prim’s Algorithm for MST

Data structure:

S set of nodes added to the tree so far

S’ set of nodes not added to the tree yet

T the edges of the MST built so far

(w) current length of the shortest edge (v, w)that connects w to the current tree

(w) potential parent node of w in the final MST(current parent that connects w to the currenttree)

Prim’s Algorithm

Initialize S, S’ and T S {u0}, S’ V \ {u0} // u0 is any vertex T { } vS’ , (v)

Initialize and for the vertices adjacent to u0 For each vS’ s.t. (u0,v)E,

(v) w((u0,v))

(v) u0

While (S’ != ) Find uS’, s.t. vS’ , (u) (v) S S {u}, S’ S’ \ {u}, T T { ((u), u) } For each v s.t. (u, v) E,

If (v) > w((u,v)) then(v) w((u,v))(v) u

Prim’s Algorithm Example

S = {v1}

Node v1 v2 v3 v4 v5

- 2 4 3

- v1 v1 v1 -

S = {v1}

Node v1 v2 v3 v4 v5

- 2 4 3

- v1 v1 v1 -

S = {v1, v2}

Node v1 v2 v3 v4 v5

- 2 4 3

- v1 v1 v1 -

S = {v1, v2}

Node v1 v2 v3 v4 v5

- 2 1 2 5

- v1 v2 v2 v2

S = {v1, v2}

Node v1 v2 v3 v4 v5

- 2 1 2 5

- v1 v2 v2 v2

S = {v1, v2, v3}

Node v1 v2 v3 v4 v5

- 2 1 2 5

- v1 v2 v2 v2

S = {v1, v2, v3}

Node v1 v2 v3 v4 v5

- 2 1 1 2

- v1 v2 v3 v3

S = {v1, v2, v3}

Node v1 v2 v3 v4 v5

- 2 1 1 2

- v1 v2 v3 v3

S = {v1, v2, v3, v4}

Node v1 v2 v3 v4 v5

- 2 1 1 2

- v1 v2 v3 v3

S = {v1, v2, v3, v4}

Node v1 v2 v3 v4 v5

- 2 1 1 2

- v1 v2 v3 v3

S = {v1, v2, v3, v4, v5}

Node v1 v2 v3 v4 v5

- 2 1 1 2

- v1 v2 v3 v3

Kruskal’s algorithm

S = {}, S’ = E

Sort S’ in nondecreasing order of weights

While S’ {}

For each edge (u,v) S’ in sorted order

S’ = S’ \ (u,v)

If (u,v) does not create a cycle with the edges in S

S = S (u,v)

Kruskal’s Algorithm Example

S = {}

S = {(v2,v3)}

S = {(v2,v3), (v3,v4)}

S = {(v2,v3), (v3,v4), (v3,v5)}

S = {(v2,v3), (v3,v4), (v3,v5),(v2,v1)}

Other Graph Algorithms of Interest...

Min-cut partitioning

Graph coloring

Maximum clique, independent set

Steiner tree [http://en.wikipedia.org/wiki/Steiner_tree]

Dynamic Programming

Read the first two examples in the document written by Michael

A. Trick – see the “Useful Links” slide

http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html

0-1 Knapsack problem

Given N discrete items of size si and value vi, how to fill a knapsack

of size M to get the maximum value? There is only one of each item

that can be either taken in whole or left out.

Dynamic Programming: Knapsack

Partial solution constructed for items 1..(i-1) for knapsack sizes from 0..M (call this array A)

c0 c1 c2 cM-1 cM…i-1

0 1 2 M-1 M

• For each knapsack size, how to extend the solution from 1..(i-1) to include i?

Option 1: do not take item i

cw-si+vi

Option 2: take item i

Example of Knapsack

𝒊 𝒔𝒊 𝒗𝒊

1 10 40

2 3 20

3 5 30

Mathematical Programming

General Form

ILP Example

Branch and Bound

Convex Optimization

Useful Links

Books on STL and templates (thanks to Arvind Karandikar @ U of M for suggesting thebooks) : Nicolai M. Josuttis, “The C++ Standard Library: A Tutorial and Reference”, Addison-Wesley,

1999, ISBN: 0-201-37926-0. David Vanevoorde and Nicolai M. Josuttis, “C++ Templates, the Complete Guide”, Addison-

Wesley, 2003, ISBN: 0-201-73484-2.

Time Complexity and Asymptotic Notations

www.cs.sunysb.edu/~skiena/548/lectures/lecture2.ps

Asymptotic bounds: definitions and theoremshttp://userpages.umbc.edu/~anastasi/Courses/341/Spr00/Lectures/Asymptotic/asymptotic/asymptotic.html

www.cs.yorku.ca/~ruppert/6115W-01/bigO.ps

CMSC 341 (at CSEE/UMBC) Lecture 2http://www.csee.umbc.edu/courses/undergraduate/CMSC341/fall02/Lectures/AA/AsymptoticAnalysis.ppt

Michael A. Trick, “A Tutorial on Dynamic Programming”, http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html

To Read Further (Chapter 4)

Heuristic algorithms

Greedy

Branch-and-Bound

Simulated Annealing

Genetic Algorithms

Mathematical programming

Linear programming (LP)

Integer LP

Convex optimization

Good References on Algorithms

VLSI CAD: Logic to Layout Algorithms -...

Documents