COSC 3100Decrease and
ConquerInstructor: Tanvir
Decrease and Conquer• Exploit the relationship between a
solution to a given instance of a problem and a solution to its smaller instance
• Top-down: recursive• Bottom-up: iterative• 3 major types:
– Decrease by a constant– Decrease by a constant factor– Variable size decrease
Decrease and Conquer (contd.)
• Decrease by a constant– Compute an where a ≠ 0 and n is a
nonnegative– an = an-1 × a– Top down: recursive
• f(n) =
– Bottom up: iterative• Multiply 1 by a, n times
f(n-1) × a if n > 0
1 if n = 0
We shall see moreinteresting ones inthis chapter!
Can you improve an computation ?
Decrease and Conquer (contd.)
• Decrease by a constant factor (usually 2)
• an = (𝒂𝒏 /𝟐)𝟐
(𝒂 (𝒏−𝟏 )/𝟐 )𝟐 ∙𝒂𝟏
If n is even and positive
If n is odd
If n = 0
529 =?
Decrease and Conquer (contd.), Decr. By const.
factorALGORITHM Exponentiate(a, n)if n = 0
return 1tmp <- Exponentiate(a, n >> 1)if (n & 1) = 0 // n is even
return tmp*tmpelse
return tmp*tmp*a
What’s the time complexity ?Θ(lgn)
(𝒂𝒏 /𝟐)𝟐
(𝒂 (𝒏−𝟏 )/𝟐 )𝟐 ∙𝒂𝟏
If n even
If n is odd
If n = 0
Decrease and Conquer (contd.)
• Variable-size decreaseWe already have seen one example.
Which one ?gcd(m, n) = gcd(n, m mod n)
gcd( 31415, 14142 ) = gcd( 14142, 3131 )= gcd( 3131, 1618 )= gcd( 1618, 1513 )= gcd( 1513, 105 )= gcd( 105, 43 )= gcd( 43, 19 )= gcd( 19, 5 )= gcd( 5, 4 )= gcd( 4, 1 )= gcd( 1, 0 )
Decr. By 11011Decr. By 1513Decr. By 105Decr. By 1408Decr. By 62Decr. By 24Decr. By 14
Decr. By 1Decr. By 3
Decr. By 1
Decrease and Conquer (contd.)
• Apply decrease by 1 technique to sorting an array A[0..n-1]
• We assume that the smaller instance A[0..n-2] has already been sorted.
• How can we solve for A[0..n-1] now?
Find the appropriate position for A[n-1]among the sorted elements and insert it
there.
Scan the sorted subarray A[0..n-2] from right to left until an element smaller thanor equal to A[n-1] is found, then insert A[n-1]right after that element.
(Straight)INSERTION
SORT
Insertion SortALGORITHM InsertionSort(A[0..n-1])for i <- 1 to n-1 do
v <- A[i]j <- i-1while j ≥ 0 and A[j] > v do
A[j+1] <- A[j]j <- j-1
A[j+1] <- v
89 | 45 68 90 29 34 17
45 89 | 68 90 29 34 17
45 68 89 | 90 29 34 17
45 68 89 90 | 29 34 17
29 45 68 89 90 | 34 17
29 34 45 68 89 90 | 17
17 29 34 45 68 89 90
Input size: nBasic op: A[j] > v
Why not j ≥ 0 ?C(n) depends on input type ?
Insertion Sort (contd.)ALGORITHM InsertionSort(A[0..n-1])for i <- 1 to n-1 do
v <- A[i]j <- i-1while j ≥ 0 and A[j] > v do
A[j+1] <- A[j]j <- j-1
A[j+1] <- v
What is the worst case scenario ?A[j] > v executes highest # of times
When does that happen ?A[j] > A[i] for j = i-1, i-2, …, 0
Worst case input: An array of strictly decreasing values
Cworst(n) = = = є Θ(n2)
What is the best case ?A[i-1] ≤ A[i] for i = 1, 2, …, n-1
Cbest(n) = n-1 є Θ(n)
For almost sorted files,insertion sort’s performance
is excellent!
Cavg(n) ≈ є Θ(n2)
In place? Stable ?
Can you improve it ?
Topological Sorting• Ordering of the vertices of a directed
graph such that for every edge uv, u comes before v in the ordering
• First studied in 1960s in the context of PERT (Project Evaluation and Review Technique) for scheduling in project management.– Jobs are vertices, there is an edge from x to
y if job x must be completed before job y can be started
– Then topological sorting gives an order in which to perform the jobs
Directed Graph or Digraph• Directions for all edges
a b
c
d
e
0 1 1 0 0
1 0 1 0 0
0 0 0 0 0
0 0 1 0 1
0 0 0 0 0
a
bc
de
a b c d e
abcde
b ca c
c e
Not symmetric
One node for one edge
Digraph (contd.)a b
c
d
e
abcde
b ca c
c e
a
b
c
d
e
DFS forest
Tree edgeBack edgeForward edgeCross edgeDirected cycle: a, b, a
If no directed cycles, the digraph is called “directed acyclic graph” or “DAG”
Topological Sorting Scenario
Consider a robot professor Dr. Botstead. He gets dressed in the morning. The professor must put on certain garments before others (e.g., socks before shoes). Other items may beput on in any order (e.g., socks and pants).
undershorts
pants
belt
socks
shoes
shirt
tie
jacket
watch
Let us help him to decideon a correct order usingtopological sorting!
Wont workif directedcycle exists!
Method 1: DFS u
p
b
so
sho
shi
t
j
w
u1,
u -> p -> shop -> sho -> bb -> jshi -> b -> tt -> jso -> show -> nullsho -> nullj -> null p2,
sho3,
b4,
j5,
shi6,
t7,
so8,
w9,
1
2
3
45
6
7
8
9
t6 u5 p4 b3 j2 sho1w9 so8 shi7
Reverse the pop order!
Method 2: Decrease (by 1) and Conquer
u
p
b
so
sho
shi
t
j
w
u, p, shi, b, t, j, so, sho, w
Topological Sorting Scenario 2
• Set of 5 courses: { C1, C2, C3, C4, C5 }
• C1 and C2 have no prerequisites• C3 requires C1 and C2
• C4 requires C3
• C5 requires C3 and C4C1
C4
C5
C2
C3
C1, C2, C3, C4, C5 !
Topological Sorting: Usage
• A large project – e.g., in construction, research, or software development – that involves a multitude of interrelated tasks with known prerequisites– Schedule to minimize the total
completion time• Instruction scheduling in program
compilation, resolving symbol dependencies in linkers,
etc.
Decrease and Conquer: Generating Permutations
• Have to generate all n! permutations of the set of integers from 1 to n– They can be indices of n-element set
{a1, a2, …, an}• Assume that we already have (n-
1)! permutations of integers 1, 2, …, n-1
• Insert n in each of the n possible positions among elements of every permutation of n-1 elements
Generating Permutations (contd.)
• We shall generate all permutations of {1, 2, 3, 4}, 4! = 24 permutations in total
1Start
Insert 2 in 2 possible positions
21, 12
Insert 3 in 3 possible positions ofeach previouspermutation
321, 231, 213,312, 132, 123
Insert 4 in 4 possible positions of each previouspermutation
4321, 3421, 3241, 3214,4231, 2431, 2341, 2314,4213, 2413, 2143, 2134,
4312, 3412, 3142, 3124,4132, 1432, 1342, 1324,4123, 1423, 1243, 1234
Generating Permutations (contd.)
a b
dc
2
3
1
5 8 7
a b d c aa c b d aa c d b aa d b c aa d c b a
a b c d a 2+8+1+7 = 182+3+1+5 = 115+8+3+7 = 235+1+3+2 = 117+3+8+5 = 237+1+8+2 = 18
If we can produce a permutationfrom the previous one just by exchangingtwo adjacent elements, cost calculation takesconstant time!
This is possible and calledminimal-change requirement
Every time we process one of theold permutations to get a bunch of newones, we reverse the order of insertion
112,21Right to Left
R to L 123, 132, 312,L to R 321, 231, 213
R2L:1234, 1243, 1423, 4123,L2R:4132, 1432, 1342, 1324,R2L:L2R:R2L:L2R:
3124, 3142, 3412, 4312,4321, 3421, 3241, 3214,
4213, 2413, 2143, 21342314, 2341, 2431, 4231,
Generating Permutations (contd.)
• It is possible to get the same ordering of permutations of n elements without explicitly generating permutations for smaller values of n
• Associate a direction with each element k in a permutation
Element k is mobile if its arrow points toa smaller number adjacent to it.
mobile
ALGORITHM JohnsonTrotter(n)//Output: A list of all permutations of {1, …, n}initialize the first permutation with while the last permutation has a mobile element do
find its largest mobile element kswap k with the adjacent element k’s arrow points toreverse direction of all the elements that are larger than
kadd the new permutation to the list
Generating Permutations (contd.)
1́ 2́�́� 1́ �́� 2́ 3́ 1́ �́� �⃗� 2́ 1́ 2́ �⃗� 1́ 2́ 1́ 3⃗ should have been the
last one!Lexicographic
orderThis version is by Shimon EvenCan be implemented in Θ(n!) time
Generating Permutations: in
Lexicographic Order ALGORITHM LexicographicPermute(n)initialize the first permutation with 12…nwhile last permutation has two consecutive elements in increasing order do
let i be its largest index such that find the largest index j such that swap ai with aj
reverse the order from ai+1 to an inclusiveadd the new permutation to the list
Narayana Pandita in 14th century India
1 2 3i i+1=j
1 3 2i i+1 j
2 3 1i i+1 j
2 1 3i i+1=j
2 3 1i i+1=j3 2 1
i i+1=j3 1 2
i i+1=j3 2 1
Generating Subsets• Generate all 2n subsets of the set A = {a1, a2, …, an}Can divide the subsets into two groups:
Those that do not contain an Those that do contain an
It’s the subsets of {a1, a2, …, an-1}
Take each subset of {a1, a2, …, an-1}and add an to it
Generating Subsets (contd.)
• Let us generate 23 subsets of {a1, a2, a3}
Ø
Ø {a1}
Ø {a1} {a2} {a1, a2}
Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}
n = 0
n = 1
n = 2
n = 3
Generating Subsets (contd.)
• There is a one-to-one correspondence between all 2n subsets of an n element set A = {a1, a2, …, an} and all 2n bit strings b1b2…bn of length n.
Bit strings
Subsets
000 001 010 011 100 101 110 111
Ø {a3} {a2} {a2, a3} {a1} {a1, a3} {a1, a2} {a1, a2, a3}
How to generate subsets in “squashed order” from bitstrings?
Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}
Generating Subsets (contd.)
Is there a minimal-change algorithm for generatingbit strings so that every consecutive two differ by
only a single bit ?In terms of subsets, every consecutive two differ by either an addition or deletion, but not both of
a single element.Yes, there is!
000 001 011 010 110 111 101 100For n = 3,
It is called “Binary reflected Gray code”
Generating Subsets (contd.)
ALGORITHM BRGC(n)//Generates recursively the binary reflected Gray code of order n//Output: A list of all bit strings of length n composing the Gray codeif n = 1
make list L containing bit strings 0 and 1 in this orderelse
generate list L1 of bit strings of size n-1 by calling BRGC(n-1)
copy list L1 to list L2 in reverse orderadd 0 in front of each bit string in list L1add 1 in front of each bit string in list L2append L2 to L1 to get list L
return LL1 : 0 1L2 : 1 0 L1 : 00 01
L2 : 11 10
L : 00 01 11 10
Binary Reflected Gray Codes (contd.)
0 110 11
00 01000 001
101100
011111110
010
Decrease-by-a-Constant-Factor Algorithms
• Binary Search– Highly efficient way to search for a
key K in a sorted array A[0..n-1]
Compare K with A’s middle element A[m]If they match, stop.
Else if K < A[m] do the same for the first half of AElse if K > A[m] do the same for the second half of A
Binary Search (contd.)
A[0] … A[m-1] A[m] A[m+1] … A[n-1]
K (key)
Search here if K < A[m]
Search here if K > A[m]
Let us apply binary search for K = 70 on the following array:
3, 14, 27, 31, 39, 42, 55, 70, 74, 81, 85, 93, 980 121 2 3 4 5 6 7 8 9 10 11
Binary Search (contd.)ALGORITHM BinarySearch(A[0..n-1], K)//Input: A[0..n-1] sorted in ascending order and a search key K//Output: An index of A’s element equal to K or -1 if no such elementl <- 0r <- n-1while l ≤ r do
m <- if K = A[m]
return melse if K < A[m]
r <- m-1else
l <- m+1return -1
Input size: nBasic operation: “comparison”Does C(n) depend on input type?
YES!Let us look at the worst-case…
Worst-case input: K is absent or someK is at some special position…
Cworst(n) = Cworst() + 1Cworst(1) = 1
To simplify, assume n = 2k
Then Cworst(n) є Θ(lgn)
For any integer n > 0, Cworst(n) = =
Variable-Size-Decrease• Problem size decreases at each
iteration in variable amount• Euclid’s algorithm for computing
the greatest common divisor of two integers is one example
Computing Median and the Selection Problem• Find the k-th smallest element in a
list of n numbers• For k = 1 or k = n, we could just
scan the list• More interesting case is for k =
– Find an element that is not larger than one half of the list’s elements and not smaller than the other half; this middle element is called the “median”
– Important problem in statistics
Median Selection ProblemAny idea ?
Sort the list and select the k-th element
Time efficiency is ?O(n lgn)It is an overkill, the problem needs only the
k-th smallest, does not ask to order the entire list!
We shall take advantage of the idea of “partitioning”a given list around some value p of, say the list’s first
element. This element is called the “pivot”.p pall are < p all are ≥ p
Lomuto partitioning & Hoare’s partitioning
Lomuto Partitioning61 93 56 90 11p
s
i
A[i] not less than p
61 93 56 90 11p
s
i
A[i] less than p
i
s
61 56 93 90 11p i
s
61 56 93 90 11p i
A[i] not less than p
s
i
i
61 56 93 90 11p
A[i] less than p
s
i
s
61 56 11 90 93p i
s
11 56 61 90 93
Lomuto Partitioning (contd.)
ps i
< p ≥ p ?
A[i] < p?
p
s< p ≥ p
swap
p< p ≥ ps
Lomuto Partitioning (contd.)
ALGORITHM LomutoPartition(A[l..r])//Partition subarray by Lomuto’s algo using first element as pivot//Input: A subarray A[l..r] of array A[0..n-1], defined by its //left and right indices l and r (l ≤ r)//Output: Partition of A[l..r] and the new position of the pivot
p <- A[l]s <- lfor i <- l+1 to r do
if A[i] < ps <- s+1swap(A[s], A[i])
swap(A[l], A[s])return s
Time complexity is Θ(r-l)
Length of subarray A[l..r]
Median Selection Problem (contd.)
• We want to use the Lomuto partitioning to efficiently find out the k-th smallest element of A[0..n-1]
• Let s be the partition’s split position• If s = k-1, pivot p is the k-th smallest• If s > k-1, the k-th smallest (of entire array) is
the k-th smallest of the left part of the partitioned array
• If s < k-1, the k-th smallest (of entire array) is the [(k-1)-(s+1)+1]-th smallest of the right part of the partitioned array
Median Selection Problem (contd.)
ALGORITHM QuickSelect(A[l..r], k)//Input: Subarray A[l..r] of array A[0..n-1] of orderable elements and integer k (1 ≤ k ≤ r-l+1)//Output: The value of the k-th smallest element in A[l..r]s <- LomutoPartiotion(A[l..r])if s = k-1
return A[s]else if s > l+k-1
QuickSelect(A[l..s-1], k)else
QuickSelect(A[s+1..r], k-1-s)
Median Selection: QuickSelect
4 1 10 8 7 12 9 2 15
4 1 10 8 7 12 9 2 15
4 1 10 8 7 12 9 2 15
4 1 2 8 7 12 9 10 15
4 1 2 8 7 12 9 10 15
2 1 4 8 7 12 9 10 15
s i
s i
s i
s i
s i
s = 2 < k-1 = 4, so we proceed with the right part…
k = = 5
s
Median Selection: QuickSelect (contd.)
8 7 12 9 10 15
2 1 4 8 7 12 9 10 15
k = = 5
8 7 12 9 10 15
8 7 12 9 10 15
7 8 12 9 10 15
s i
s i
s i
Now s (=4) = k-1 (=5-1=4), we have found the median!
s
Median Selection (contd.)ALGORITHM QuickSelect(A[l..r], k)//Input: Subarray A[l..r] of array A[0..n-1] of orderable elements and integer //k (1 ≤ k ≤ r-l+1)//Output: The value of the k-th smallest element in A[l..r]s <- LomutoPartiotion(A[l..r])if s = k-1
return A[s]else if s > l+k-1
QuickSelect(A[l..s-1], k)else
QuickSelect(A[s+1..r], k-1-s)
What is the time efficiency ? Does it depend on input-type?
Partitioning A[0..n-1] takes n-1 comparisonsIf now s = k-1, this is the best-case, Cbest(n) = n є Θ(n)
What’s the worst-case ?
Say k = n and we have a strictlyincreasing array!
Then, Cworst(n) = (n-1)+(n-2)+……+1 = =
Cworst(n) is worse than sorting based solution!
Fortunately, careful mathematical analysishas shown that the average-case is linear
Why is QuickSelectin variable-size-decrease ?
Thank you !