Divide and ConquerApplications

Sanghyun Park

Fall 2002

CSE, POSTECH

Merge Sort

Sort the first half of the array using merge sort.

Sort the second half of the array using merge sort.

Merge the first half of the array with the second half.

Merge Algorithm

Merge is an operation that combines two sorted arrays. Assume the result is to be placed in a separate array called

result (already allocated). The two given arrays are called front and back. front and back are in increasing order. For the complexity analysis,

the size of the input, n, is the sum nfront + nback.

Merge Algorithm

For each array keep track of the current position. REPEAT until all the elements of one of the given

arrays have been copied into result:– Compare the current elements of front and back.– Copy the smaller into the current position of result

(break the ties however you like).– Increment the current position of result and the array

that was copied from. Copy all the remaining elements of the other given

array into result.

Merge Algorithm - Complexity

Every element in front and back is copied exactly once. Each copy is two accesses,so the total number of accessing due to copying is 2n.

The number of comparisons could beas small as min(nfront, nback) or as large as n-1.Each comparison is two accesses.

Merge Algorithm - Complexity

In the worst casethe total number of accesses is2n +2(n-1) = O(n).

In the best casethe total number of accesses is2n + 2min(nfront,nback) = O(n).

The average case is between the worst and best case and is therefore also O(n).

Merge Sort Algorithm

Split anArray into two non-empty parts anyway you like.For example,front = the first n/2 elements in anArrayback = the remaining elements in anArray

Sort front and back by recursively calling MergeSort.

Now you have two sorted arrays containing all the elements from the original array.Use merge to combine them, put the result in anArray.

MergeSort Call Graph (n=7)

Each box represents one invocation of MergeSort. How many levels are there in general

if the array is divided in half each time?

0~6

0~2 3~6

3~4

3~3 4~4

5~6

5~5 6~6

1~2

1~1 2~2

0~0

MergeSort Call Graph (general)

Suppose n = 2k. How many levels? How many boxes on level j? What values is in each box at level j?

n

n/2 n/2

n/4 n/4n/4n/4

1 1 1 1 1 1 1 1

MergeSort: Complexity Analysis

Each invocation of mergesort on p array positionsdoes the following:– Copies all p positions once (#accesses = O(p))– Calls merge (#accesses = O(p))

Observe that p is the same for all invocations at the same level, therefore total number of accesses at a given level j is O((#invocations at level j) * pj).

MergeSort: Complexity Analysis

The total number of accesses at level j isO((#invocations at level j) * pj)= O(2j-1 * n/2j-1)= O(n)

In other words,the total number of accesses at each level is O(n).

The total number of accesses for the entire mergesort is the sum of the accesses for all the levels.(#levels) * O(n)= O(log n) * O(n)= O(n log n)

MergeSort: Complexity Analysis

Best case: O(n log n) Worst case: O(n log n) Average case: O(n log n)

Quick Sort

Quicksort can be seen as a variation of mergesortin which front and back are defined in a different way.

Quicksort Algorithm

Partition anArray into two non-empty parts.– Pick any value in the array, pivot.– small = the elements in anArray < pivot– large = the elements in anArray > pivot– Place pivot in either part,

so as to make sure neither part is empty. Sort small and large by recursively calling QuickSort. You could use merge to combine them, but because th

e elements in small are smaller than elements in large, simply concatenate small and large, and put the result into anArray.

Quicksort: Complexity Analysis

Like mergesort,a single invocation of quicksort on an array of size phas complexity O(p):– p comparisons = 2*p accesses– 2*p moves (copying) = 4*p accesses

Best case: every pivot chosen by quicksort partitions the array into equal-sized parts. In this case quicksort is the same big-O complexity as mergesort – O(n log n)

Quicksort: Complexity Analysis

Worst case: the pivot chosen is the largest or smallest value in the array. Partition creates one part of size 1 (containing only the pivot), the other of size p-1.

n

1 n-1

n-21

1

1

Quicksort: Complexity Analysis

Worst case:There are n-1 invocations of quicksort (not counting base cases) with arrays of size:p = n, n-1, n-2, …, 2

Since each of these does O(p),the total number of accesses isO(n) + O(n-1) + … + O(1) = O(n2)

Ironically the worst case occurs when the list is sorted (or near sorted)!

Quicksort: Complexity Analysis

The average case must be betweenthe best case O(n log n) and the worst case is O(n2).

Analysis yields a complex recurrence relation. The average case number of comparisons turns out to be

approximately 1.386*n*log n – 2.846*n. Therefore the average case time complexity is

O(n log n).

Quicksort: Complexity Analysis

Best case O(n log n) Worst case O(n2) Average case O(n log n) Note that the quick sort is inferior to insertion sort and merge

sort if the list is sorted, nearly sorted, or reverse sorted.

Closest Pair Of Points

Given n points in 2D, find the pair that are closest.

Applications

We plan to drill holes in a metal sheet. If the holes are too close,

the sheet will tear during drilling. Verify that two holes are closer than a threshold

distance(e.g., holes are at least 1 inch apart.)

Air Traffic Control

3D --- Locations of airplanes flying in the neighborhood of a busy airport are known.

We want to be sure that no two planes get closer than a given threshold distance.

Simple Solution

For each of the n(n-1)/2 pairs of points,determine the distance between the points in the pair.

Determine the pair with the minimum distance.

O(n2) time

Divide and Conquer Solution

When n is small, use simple solution. When n is large,

– Divide the point set into two roughly equal parts A and B.

– Determine the closest pair of points in A.– Determine the closest pair of points in B.– Determine the closest pair of points such that one point

is in A and the other in B.– From the three closest pairs computed,

select the one with least distance.

Example

Divide so thatpoints in A have x coordinate <= that of points in B.

Example

Find the closest pair in A. Let d1 be the distance between the points in this pair.

Example

Find the closest pair in B. Let d2 be the distance between the points in this pair.

Example

Let d = min{d1,d2}. Is there a pair with one point in A and the other in B,

and their distance < d?

Example

Candidates lie within d of the dividing line. Call these regions RA and RB, respectively.

Example

Let q be a point in RA.

q need to be paired only with those points in RB

that are within d of q.y.

Example

Points that are to be paired with q are in d x 2d rectangle of RB (comparing region of q).

Points in this rectangle are at least d apart.(What is the maximum number of points in this rectangle?)

Example

So the comparing region of q has at most 6 points. So number of pairs to check is <= 6 * |RA| = O(n).

Time Complexity

Create a sorted list of points (by x-coordinate):O(n log n) time

Create a sorted list of points (by y-coordinate):O(n log n) time

Using these two lists, the required pairs of points from RA and RB can be constructed in O(n) time.

Let n < 4 define a small instance.

Time Complexity

Let t(n) be the time to find the closest pairexcluding the time to create the two sorted lists.

t(n) = c, n < 4, where c is a constant. when n >= 4,

t(n) = t(ceil(n/2)) + t(floor(n/2)) + dn,where d is a constant.

To solve the recurrence,assume n is a power of 2 and use repeated substitution.

t(n) = O(n log n)

Closest Pair Of Points

Given n points in 2D, find the pair that are closest.

C++ routines are given on pp. 695-699 of the text.

The complete program is also available athttp://www.mhhe.com/engcs/compsci/sahni/source.mhtml.