Faculty of Science for Women( SCIW), University of Babylon, Iraq
By
Ass. Prof. Dr. Samaher Al_Janabi
LECTURE NOTES OF ALGORITHMS:
DESIGN TECHNIQUES AND
ANALYSIS
Department of Computer Science University of Babylon
15 March 2017
Outlines• Divide and Conquer Algorithm and design paradigm to various problems
• O(n log n) Algorithm for Counting Inversions I
• O(n log n) Algorithm for Counting Inversions II
• Strassen's Sub cubic Matrix Multiplication Algorithm
• O(n log n) Algorithm for Closest Pair I
• O(n log n) Algorithm for Closest Pair II
• QUZI
• Homwork
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Both merge sort and quicksort employ a common algorithmic paradigm based on recursion. Thisparadigm, divide-and-conquer, breaks a problem into subproblems that are similar to the originalproblem, recursively solves the subproblems, and finally combines the solutions to the subproblemsto solve the original problem. Because divide-and-conquer solves subproblems recursively, eachsubproblem must be smaller than the original problem, and there must be a base case forsubproblems. You should think of a divide-and-conquer algorithm as having three parts:
• Divide the problem into a number of subproblems that are smaller instances of the same problem.
• Conquer the subproblems by solving them recursively. If they are small enough, solve the subproblems as base cases.
• Combine the solutions to the subproblems into the solution for the original problem.
You can easily remember the steps of a divide-and-conquer algorithm as divide, conquer, combine. Here's how to view one step, assuming that each divide step creates two subproblems (though some divide-and-conquer algorithms create more than two):
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
If we expand out two more recursive steps, it looks like this:
Because divide-and-conquer creates at least two subproblems, a divide-and-conquer algorithm
makes multiple recursive calls.
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
When the divide and conquer become useful or not useful ?
1. Divide and conquer is useful if and only if the main problem can be divide into sub problems
form the same type (i.e., natural) of original problem. At this case time complexity become
Where, f(n) the time require to divided the original problem into many sub problems
g(n) the time require to solve solution of small problems
t(n) time require divide and conquer for the input have size n
2. Divide and conquer is not useful into two cases
a. Problem have the size n divide into two or more sub problems each one have approximation
the size n (such as fib. Problem) because this lead to exponational complexity
b. Problem have the size n divide into sub problems each one have approximation the size (n/c)
where c is constant. because this lead to the complexity of algorithm nΘ(log n)
otherwise,)n(f)n(t...)n(t)n(t
smallnif,)n(g)n(t
k21
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Merge + Sort =
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Examples and Motivation
Sol:
=(6(6-1))/2= (6*5)/2=30/2=15
What is the largest-possible number of inversions that a 6-element array can have?
a. 15
b. 21
c. 36
d. 64
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Q: Suppose the input array A has no split inversions.
What is the relationship between the sorted subarrays B and C?
a. B has the smallest element of A, C the second-smallest, B the third-smallest, and so on.
b. All elements of B are less than all elements of C.
c. All elements of B are greater than all elements of C.
d. There is not enough information to answer this question.
Pseudocode for Merge:
D = output [length = n]
B = 1st sorted array [n/2]
C = 2nd sorted array [n/2]
i = 1
j = 1
for k = 1 to n
if B(i) < C(j)
D(k) = B(i)
i++
else [C(j) < B(i)]
D(k) = C(j)
j++
End(ignores end cases)
Merge_and_CountSplitInv
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Strassen’s Subcubic Matrix Multiplication Algorithm
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Appling Divide and Conquer
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
What is the asymptotic running time of the straightforward iterative algorithm for matrix
multiplication?
𝜃(𝑛 log 𝑛)
𝜃(𝑛2)
𝜃(𝑛3)
𝜃(𝑛4)
Correct answer
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
The Closest Pair Problem
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
The Closest Pair Problem
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
ClosestPair(𝑃𝑥 , 𝑃𝑦)
Q: Suppose we can correctly implement the ClosestSplitPair subrouine in 𝑂(𝑛) time. What will be
the overall running time of the Closest Pair algorithm? (Choose the smallest upper bound that
applies.)
𝑂(𝑛)
𝑂(𝑛 log 𝑛)
𝑂(𝑛 (log 𝑛)2)
𝑂(𝑛2)
Correct answer
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
minDist = infinity
for i = 1 to length(P) - 1
for j = i + 1 to length(P)
let p = P[i], q = P[j]
if dist(p, q) < minDist:
minDist = dist(p, q)
closestPair = (p, q)
return closestPair
Summary and Analysis of the 2-D Algorithm
ClosestPair of a set of points:
1. Divide the set into two equal sized parts by the line l, and recursively compute the minimal distance in
each part.
2. Let d be the minimal of the two minimal distances.
3. Eliminate points that lie farther than d apart from l
4. Sort the remaining points according to their y-coordinates
5. Scan the remaining points in the y order and compute the distances of each point to its five neighbors.
6. If any of these distances is less than d then update d.
Steps 2-6 define the merging process which must be repeated logn times because this is a divide and
conquer algortithm:
• Step 2 takes O(1) time
• Step 3 takes O(n) time
• Step 4 is a sort that takes O(nlogn) time
• Step 5 takes O(n) time (as we saw in the previous section)
• Step 6 takes O(1) time
Hence the merging of the sub-solutions is dominated by the sorting at step 4, and hence takes O(nlogn) time.
This must be repeated once for each level of recursion in the divide-and-conquer algorithm,
Quiz
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Q1: 3-way-Merge Sort : Suppose that instead of dividing in half at each step of Merge Sort, you
divide into thirds, sort each third, and finally combine all of them using a three-way merge
subroutine. What is the overall asymptotic running time of this algorithm? (Hint: Note that the
merge step can still be implemented in O(n) time.)
a. n
b. n2log(n)
c. n(log(n))2
d. nlog(n)
Q2: You are given functions f and g such that f(n)=O(g(n)). Is f(n)∗log2(f(n)c)=O(g(n)∗log2(g(n))) ?
(Here c is some positive constant.) You should assume that f and g are non decreasing and always
bigger than 1.
a. Sometimes yes, sometimes no, depending on the functions f and g
b. False
c. Sometimes yes, sometimes no, depending on the constant c
d. True
Quiz
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Q3: Assume again two (positive) non decreasing functions f and g such that f(n)=O(g(n)).
Is 2f(n)=O(2g(n)) ? (Multiple answers may be correct, you should check all of those that apply.)
a. Yes if f(n)≤g(n) for all sufficiently large n
b. Sometimes
c. Never
d. Always
Q4: The Fibonacci of number (11 ) is
a. 12
b. 13
c. 20
d. 19
Q5: The main advantages of Asymptotic Analysis are
a. it's Sweet spot for discussing the high level performance(reasoning) of algorithms
b. it's sharp enough to be useful. In particular, to make predictive comparisons between different high
level algorithmic approaches to solving a common problem.
c. it is a mathematical concept
d. coarse enough to suppress all of the details that you want to ignore. Details that depend on the
choice of architecture, the choice of programming language, the choice of compiler} .
Quiz
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Q7 : Give Suitable word for each the following :
a. algorithm +data structure =
b. information + interactive with environment =
c. pseudo =
d. Person can give active solution for any problem in any domain =
e. Any algorithm can be analysis based on two domains =
Q6: k-way-Merge Sort. Suppose you are given k sorted arrays, each with n elements, and you want
to combine them into a single array of kn elements. Consider the following approach. Using the
merge subroutine taught in lecture, you merge the first 2 arrays, then merge the 3rd given array
with this merged version of the first two arrays, then merge the 4th given array with the merged
version of the first three arrays, and so on until you merge in the final (kth) input array. What is
the running time taken by this successive merging algorithm, as a function of k and n? (Optional:
can you think of a faster way to do the k-way merge procedure ?)
Quiz
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
Q8: Arrange the following functions in increasing order of growth rate
(with g(n) following f(n) in your list if and only if f(n)=O(g(n))).
• Write your 5-letter answer, i.e., the sequence in lower case letters in the space provided. For
example, if you feel that the answer is a->b->c->d->e (from smallest to largest), then type abcde
in the space provided without any spaces in between the string.
• You can assume that all logarithms are base 2 (though it actually doesn't matter).
Homwork
Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #4
15 March 2017
• You are given as input an unsorted array of n distinct numbers, where n is a power of 2.
Give an algorithm that identifies the second-largest number in the array, and that uses at
most n+log2n−2 comparisons.
• You are a given a unimodal array of n distinct elements, meaning that its entries are in
increasing order up until its maximum element, after which its elements are in
decreasing order. Give an algorithm to compute the maximum element that runs in O(log
n) time.
• You are given a sorted (from smallest to largest) array A of n distinct integers which can
be positive, negative, or zero. You want to decide whether or not there is an index i such
that A[i] = i. Design the fastest algorithm that you can for solving this problem.