Analysis of AlgorithmsCS 477/677

Instructor: Monica Nicolescu

Lecture 11

Interval Trees

• Useful for representing a set of intervals– E.g.: time intervals of various events

• Each interval i has a low[i] and a high[i]

CS 477/677 - Lecture 11 2

Interval Trees

Def.: Interval tree = a red-black tree that

maintains a dynamic set of elements, each

element x having associated an interval int[x].

• Operations on interval trees:

– INTERVAL-INSERT(T, x)

– INTERVAL-DELETE(T, x)

– INTERVAL-SEARCH(T, i)

CS 477/677 - Lecture 11 3

Interval Properties

• Intervals i and j overlap iff:

low[i] ≤ high[j] and low[j] ≤ high[i]

• Intervals i and j do not overlap iff:

high[i] < low[j] or high[j] < low[i]

i

ji

j

i

j

i

j

i j j i

CS 477/677 - Lecture 11 4

Interval Trichotomy

• Any two intervals i and j satisfy the interval

trichotomy: exactly one of the following three

properties holds:

a) i and j overlap,

b) i is to the left of j (high[i] < low[j])

c) i is to the right of j (high[j] < low[i])

CS 477/677 - Lecture 11 5

Designing Interval Trees1. Underlying data structure

– Red-black trees– Each node x contains: an interval int[x],

and the key: low[int[x]]– An inorder tree walk will list intervals

sorted by their low endpoint

2. Additional information– max[x] = maximum endpoint value in

subtree rooted at x

3. Maintaining the information

max[x] =

Constant work at each node, so still O(lgn) time

high[int[x]]max max[left[x]]

max[right[x]]

CS 477/677 - Lecture 11 6

Designing Interval Trees

4. Develop new operations• INTERVAL-SEARCH(T, i):

– Returns a pointer to an element x in the interval tree T, such that int[x] overlaps with i, or NIL otherwise

• Idea:• Check if int[x]

overlaps with i• Max[left[x]] ≥ low[i]

– Go left

• Otherwise, go right

[16, 21]

30

[25, 30]

30

[26, 26]

26

[17, 19]

20

[19, 20]

20

[8, 9]

23

[15, 23]

23

[5, 8]

10

[6, 10]

10

[0, 3]

3

[22, 25]

highlow

CS 477/677 - Lecture 11 7

Example

[16, 21]

30

[25, 30]

30

[26, 26]

26

[17, 19]

20

[19, 20]

20

[8, 9]

23

[15, 23]

23

[5, 8]

10

[6, 10]

10

[0, 3]

3

i = [11, 14] x

xx

x = NIL

i = [22, 25]

CS 477/677 - Lecture 11 8

INTERVAL-SEARCH(T, i)

1. x ← root[T]

2. while x nil[T] and i does not overlap int[x]

3. do if left[x] nil[T] and

max[left[x]] ≥ low[i]

4. then x ← left[x]

5. else x ← right[x]

6. return x

CS 477/677 - Lecture 11 9

Theorem

At the execution of interval search: if the search goes right, then either:– There is an overlap in right subtree, or– There is no overlap in either subtree

• Similar when the search goes left• It is safe to always proceed in only

one direction

CS 477/677 - Lecture 11 10

Theorem

• Proof: If search goes right:– If there is an overlap in right subtree, done– If there is no overlap in right show there is no

overlap in left– Went right because:

left[x] = nil[T] no overlap in left, or

max[left[x]] < low[i] no overlap in left

i

max[left[x]]

low[x]

CS 477/677 - Lecture 11 11

Theorem - Proof

If search goes left:• If there is an overlap in left subtree, done• If there is no overlap in left, show there is no overlap in right• Went left because:

low[i] ≤ max[left[x]] = high[j] for some j in left subtree

int[root]

max

[low[j], high[j]]max[j]

[low[k], high[k]]max[k]

high[j] < low[i]high[i] < low[j]

i ij

low[j] < low[k]

k

No overlap!

high[i] <

low[k]

max[left]

CS 477/677 - Lecture 11 12

Dynamic Programming

• An algorithm design technique for optimization

problems (similar to divide and conquer)

• Divide and conquer

– Partition the problem into independent subproblems

– Solve the subproblems recursively

– Combine the solutions to solve the original problem

CS 477/677 - Lecture 11 13

Dynamic Programming

• Used for optimization problems

– Find a solution with the optimal value (minimum or

maximum)

– A set of choices must be made to get an optimal

solution

– There may be many solutions that return the optimal

value: we want to find one of them

CS 477/677 - Lecture 11 14

Dynamic Programming

• Applicable when subproblems are not independent

– Subproblems share subsubproblems

E.g.: Fibonacci numbers: • Recurrence: F(n) = F(n-1) + F(n-2)• Boundary conditions: F(1) = 0, F(2) = 1• Compute: F(5) = 3, F(3) = 1, F(4) = 2

– A divide and conquer approach would repeatedly solve the

common subproblems

– Dynamic programming solves every subproblem just once and

stores the answer in a table

CS 477/677 - Lecture 11 15

Dynamic Programming Algorithm

1. Characterize the structure of an optimal

solution

2. Recursively define the value of an optimal

solution

3. Compute the value of an optimal solution in a

bottom-up fashion

4. Construct an optimal solution from computed

information

CS 477/677 - Lecture 11 16

Elements of Dynamic Programming

• Optimal Substructure– An optimal solution to a problem contains within it an

optimal solution to subproblems

– Optimal solution to the entire problem is built in a

bottom-up manner from optimal solutions to

subproblems

• Overlapping Subproblems– If a recursive algorithm revisits the same subproblems

again and again the problem has overlapping

subproblems

CS 477/677 - Lecture 11 17

Assembly Line Scheduling• Automobile factory with two assembly lines

– Each line has n stations: S1,1, . . . , S1,n and S2,1, . . . , S2,n

– Corresponding stations S1, j and S2, j perform the same function but can take different amounts of time a1, j and a2, j

– Times to enter are e1 and e2 and times to exit are x1 and x2

CS 477/677 - Lecture 11 18

Assembly Line• After going through a station, the car can either:

– stay on same line at no cost, or – transfer to other line: cost after Si,j is ti,j , i = 1, 2, j = 1, . . . ,

n-1

CS 477/677 - Lecture 11 19

Assembly Line Scheduling• Problem:

What stations should be chosen from line 1 and what from line 2 in order to minimize the total time through the factory for one car?

CS 477/677 - Lecture 11 20

One Solution

• Brute force– Enumerate all possibilities of selecting stations– Compute how long it takes in each case and choose

the best one

– There are 2n possible ways to choose stations– Infeasible when n is large

1 0 0 1 1

1 if choosing line 1 at step j (= n)

1 2 3 4 n

0 if choosing line 2 at step j (= 3)

CS 477/677 - Lecture 11 21

1. Structure of the Optimal Solution

• How do we compute the minimum time of going through the station?

CS 477/677 - Lecture 11 22

1. Structure of the Optimal Solution

• Let’s consider all possible ways to get from the starting point through station S1,j

– We have two choices of how to get to S1, j:• Through S1, j - 1, then directly to S1, j

• Through S2, j - 1, then transfer over to S1, j

a1,ja1,j-1

a2,j-1

t2,j-1

S1,jS1,j-1

S2,j-1

CS 477/677 - Lecture 11 23

1. Structure of the Optimal Solution

• Suppose that the fastest way through S1, j is through S1, j – 1

– We must have taken the fastest way from entry through S1, j – 1

– If there were a faster way through S1, j - 1, we would use it instead

• Similarly for S2, j – 1

a1,ja1,j-1

a2,j-1

t2,j-1

S1,jS1,j-1

S2,j-1

CS 477/677 - Lecture 11 24

Optimal Substructure

• Generalization: an optimal solution to the problem find the fastest way through S1, j contains within it an optimal solution to subproblems: find the fastest way through S1, j - 1 or S2, j - 1.

• This is referred to as the optimal substructure property

• We use this property to construct an optimal solution to a problem from optimal solutions to subproblems CS 477/677 - Lecture 11 25

2. A Recursive Solution

• Define the value of an optimal solution in terms of the optimal solution to subproblems

• Assembly line subproblems– Finding the fastest way through each station j on each line i

(i = 1,2, j = 1, 2, …, n)

CS 477/677 - Lecture 11 26

2. A Recursive Solution

• f* = the fastest time to get through the entire factory• fi[j] = the fastest time to get from the starting point through

station Si,j

f* = min (f1[n] + x1, f2[n] + x2)

CS 477/677 - Lecture 11 27

2. A Recursive Solution

• fi[j] = the fastest time to get from the starting point through station Si,j

• j = 1 (getting through station 1)

f1[1] = e1 + a1,1

f2[1] = e2 + a2,1

CS 477/677 - Lecture 11 28

2. A Recursive Solution• Compute fi[j] for j = 2, 3, …,n, and i = 1, 2

• Fastest way through S1, j is either:– the way through S1, j - 1 then directly through S1, j, or

f1[j - 1] + a1,j

– the way through S2, j - 1, transfer from line 2 to line 1, then through S1, j

f2[j -1] + t2,j-1 + a1,j

f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)

a1,ja1,j-1

a2,j-1

t2,j-1

S1,jS1,j-1

S2,j-1

CS 477/677 - Lecture 11 29

2. A Recursive Solution

e1 + a1,1 if j = 1

f1[j] =

min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j) if j ≥ 2

e2 + a2,1 if j = 1

f2[j] =

min(f2[j - 1] + a2,j ,f1[j -1] + t1,j-1 + a2,j) if j ≥ 2 CS 477/677 - Lecture 11 30

3. Computing the Optimal Value

f* = min (f1[n] + x1, f2[n] + x2)

f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)

• Solving top-down would result in exponential running time

f1[j]

f2[j]

1 2 3 4 5

f1(5)

f2(5)

f1(4)

f2(4)

f1(3)

f2(3)

2 times4 times

f1(2)

f2(2)

f1(1)

f2(1)

CS 477/677 - Lecture 11 31

3. Computing the Optimal Value

• For j ≥ 2, each value fi[j] depends only on the values of f1[j – 1] and f2[j - 1]

• Compute the values of fi[j]– in increasing order of j

• Bottom-up approach– First find optimal solutions to subproblems– Find an optimal solution to the problem from the subproblems

f1[j]

f2[j]

1 2 3 4 5

increasing j

CS 477/677 - Lecture 11 32

4. Construct the Optimal Solution

• We need the information about which line has been used

at each station:

– li[j] – the line number (1, 2) whose station (j - 1) has been used

to get in fastest time through Si,j, j = 2, 3, …, n

– l* – the line number (1, 2) whose station n has been used to get

in fastest time through the exit point

l1[j]

l2[j]

2 3 4 5

increasing j

CS 477/677 - Lecture 11 33

CS 477/677 - Lecture 11 34

FASTEST-WAY(a, t, e, x, n)1. f1[1] ← e1 + a1,1

2. f2[1] ← e2 + a2,1

3. for j ← 2 to n

4. do if f1[j - 1] + a1,j ≤ f2[j - 1] + t2, j-1 + a1, j

5. then f1[j] ← f1[j - 1] + a1, j

6. l1[j] ← 1

7. else f1[j] ← f2[j - 1] + t2, j-1 + a1, j

8. l1[j] ← 2

9. if f2[j - 1] + a2, j ≤ f1[j - 1] + t1, j-1 + a2, j

10. then f2[j] ← f2[j - 1] + a2, j

11. l2[j] ← 2

12. else f2[j] ← f1[j - 1] + t1, j-1 + a2, j

13. l2[j] ← 1

Compute initial values of f1 and f2

Compute the values of f1[j] and l1[j]

Compute the values of f2[j] and l2[j]

CS 477/677 - Lecture 11 35

FASTEST-WAY(a, t, e, x, n) (cont.)

14. if f1[n] + x1 ≤ f2[n] + x2

15. then f* = f1[n] + x1

16. l* = 1

17. else f* = f2[n] + x2

18. l* = 2

Compute the values of the fastest time through theentire factory

Example

e1 + a1,1, if j = 1

f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j) if j ≥ 2

f* = 35[1]f1[j]

f2[j]

1 2 3 4 5

9

12 16[1]

18[1] 20[2]

22[2]

24[1]

25[1]

32[1]

30[2]

CS 477/677 - Lecture 11 36

CS 477/677 - Lecture 11 37

4. Construct an Optimal Solution

Alg.: PRINT-STATIONS(l, n)

i ← l* print “line ” i “, station ” n for j ← n downto 2

do i ←li[j]

print “line ” i “, station ” j - 1

f1[j] l1[j]

f2[j] l2[j]

1 2 3 4 5

9

12 16[1]

18[1] 20[2]

22[2]

24[1]

25[1]

32[1]

30[2]l* = 1

line 1, station 5

line 1, station 4

line 1, station 3

line 2, station 2

line 1, station 1

CS 477/677 - Lecture 11 38

Dynamic Programming Algorithm

1. Characterize the structure of an optimal solution– Fastest time through a station depends on the fastest time on

previous stations

2. Recursively define the value of an optimal solution– f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)

3. Compute the value of an optimal solution in a bottom-up

fashion– Fill in the fastest time table in increasing order of j (station #)

4. Construct an optimal solution from computed information– Use an additional table to help reconstruct the optimal solution

Readings

• Chapters 14, 15

CS 477/677 - Lecture 11 39