Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | madalyn-hillin |
View: | 217 times |
Download: | 0 times |
1
Algorithms & Pseudocode Orders of Magnitude Sequential Searching
Finding Minimum value Linear Search
Sorting Insertion Sort Selection Sort
CSE 30331CSE 30331Lecture 3 – Algorithms ILecture 3 – Algorithms I
2
Algorithms & Pseudocode Algorithm
Well defined computational process (sequence of steps) that takes some input(s) and produces some output(s)
Why discuss algorithms? Facilitate design of software modules Analyze costs (time and space) of software module in a manner
that is independent of platform
Pseudocode Method of writing algorithms that is independent of
programming language
3
Constant Time Algorithms
An algorithm is Θ(1) when its running time is independent of the number of data items. The algorithm runs in constant time.
The storing of the element involves a simple assignment statement and thus has efficiency Θ(1).
fro nt rear
D irec t In se r t a t R ear
4
Linear Time Algorithms
An algorithm is Θ(n) when its running time is proportional to the size of the list.
When the number of elements doubles, the number of operations doubles.
S equential S e arch fo r the Minim um E lem ent in an A rray
32 46 8 12 3
m in im u m elem en t fou n d in th e list a fter n com p a rison s
n = 51 2 3 4 5
5
Less Efficient Algorithms
Quadratic Algorithms: Θ(n2) practical only for relatively small values of n. When n
doubles, running time quadruples.
Cubic Algorithms: Θ(n3) efficiency is generally poor; doubling the size of n
increases the running time eight-fold.
Exponential: Θ(2n) Awful, but some algorithms have no better algorithmic
solution
6
Algorithm EfficiencyAlgorithm Efficiency
n log2n n log2n n2 n3 2n 2 1 2 4 8 4 4 2 8 16 64 16 8 3 24 64 512 256 16 4 64 256 4096 65536 32 5 160 1024 32768 4294967296 128 7 896 16384 2097152 3.4 x 1038
1024 10 10240 1048576 1073741824 1.8 x 10308
65536 16 1048576 4294967296 2.8 x 1014 Forget it!
7
Logarithmic Time Algorithms
The logarithm of n, base 2, is commonly used when analyzing computer algorithms.
Ex. lg(2) = log2(2) = 1
lg(1024) = log2(1024) = 10
When compared to the functions n and n2, the function lg(n) grows very slowly.
nn 2
lo g 2n
8
A Simple Linear Algorithm
Example: Find the minimum element in an array A Assume A contains n elements (n >= 1) Assume elements are not ordered Basic technique is sequential
min(A,n) cost times smallest = A[0] c1 1 for i = 1 to (n-1) c2 n if A[i] < smallest c3 n-1 smallest = A[i] c4 0..n-1 return smallest c5 1
9
Analysis of min(A,n) min(A,n) cost times smallest = A[0] c1 1 for i = 1 to (n-1) c2 n if A[i] < smallest c3 n-1 smallest = A[i] c4 0..n-1 return smallest c5 1
Best: A[0] is min T(n) = c1 + c2(n) + c3(n-1) + 0 + c5 T(n) = (c2+c3)(n) + (c1+c5-c3) = an + b linear or Θ(n)
Worst: A[n-1] is min and A[i] > A[i+1] for all i=[0,n) T(n) = c1 + c2(n) + c3(n-1) + c4(n-1) + c5 T(n) = (c2+c3+c4)(n) + (c1+c5-c3-c4) = cn + d linear or Θ(n)
10
Linear Search
Example: Find the target element t in an array A Assume A contains n elements (n >= 1) Assume elements are not ordered Basic technique is linear search Return value is index of smallest element in range 0..n, where n
indicates not found
linearSearch(A,n,t) cost times i = 0 c1 1 while (i < n) and (A[i] != t) c2 1..n+1 i = i + 1 c3 0..n return i c4 1
11
Linear Search linearSearch(A,n,t) cost times i = 0 c1 1 while (i < n) and (A[i] != t) c2 1..n+1 i = i + 1 c3 0..n return i c4 1
Worst: t is not in A T(n) = c1 + c2(n+1) + c3(n) + c4 T(n) = (c2+c3)(n) + (c1+c2+c4) = an + b linear or Θ(n)
Average: A[k] is t, where k = n/2 T(n) = c1 + c2(k+1) + c3(k) + c4 T(n) = (c2+c3)(k) + (c1+c2+c4) = cn + d T(n) = ((c2+c3)/2)(n) + (c1+c2+c4) = cn + d linear or Θ(n)
12
Insertion Sort
On each pass the next element is inserted into the portion of the array previously sorted
The sort is “in place”
The array is logically separated into two parts [..sorted..] [..unsorted..]
The index j marks the location of the key and separates sorted from unsorted portions of the array
13
Example of Insertion Sort
[ ] [ 6 2 5 4 3 ] original array [ 6 ] [ 2 5 4 3 ] prior to 1st iteration [ 2 6 ] [ 5 4 3 ] after 1st iteration [ 2 5 6 ] [ 4 3 ] after 2nd iteration [ 2 4 5 6 ] [ 3 ] after 3rd iteration [ 2 3 4 5 6 ] [ ] after last iteration
14
Insertion Sort
Invariant: A[0..j-1] is the same collection of elements as in
the previous iteration BUT they are now sorted On each iteration
A[j] is the element that is inserted into A[0..j-1] producing a sorted A[0..j]
A[j+1..n-1] remains unsorted
15
Insertion Sort Algorithm Sort n elements of A into non-decreasing order
insertionSort(A,n) cost times for j = 1 to (n-1) c1 n key = A[j] c2 n-1 // insert A[j] into sorted sequence A[0..j-1] i = j – 1 c3 n-1 while (i >= 0) and (A[i] > key) c4 f A[i+1] = A[i] c5 g i = i – 1 c6 g A[i+1] = key c7 n-1
f = g =
1
1
n
j
jt
1
1
)1(n
j
jt
16
Insertion Sort Analysis Best: already presorted
tj is number of times the while loop checks limits and pairs of elements during shifting of elements in A[0..j-1]
The loop terminates on first check since A[j] <= A[j+1] for all j T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(f) + (c5+c6)(g)
tj = 1, so f = = (n-1) and g = = 0
T(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7)
Linear Θ(n)
1
1
n
j
jt
1
1
)1(n
j
jt
17
Insertion Sort Analysis
Worst: presorted in reverse order Shifting loop has to compare and shift every element in A[0..j-1]
because initial order ensures A[k] > A[j] for all k < j
T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(f) + (c5+c6)(g)
tj = j, so f = = n(n-1)/2 and g = = (n-1)(n-2)/2
T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(n(n-1)/2 ) + (c5+c6)((n-1)(n-2)/2) T(n) = an2 + bn + c
Quadratic Θ(n2)
1
1
n
j
jt
1
1
)1(n
j
jt
18
Selection Sort On each pass the smallest element in the unsorted
sub-array is selected for appending to the sorted sub-array
The sort is “in place”
The array is logically separated into two parts [..sorted..] [..unsorted..]
The index k separates sorted from unsorted portions of the array
19
Example of Selection Sort
[ ] [ 6 2 5 4 3 ] original array [ 2 ] [ 6 5 4 3 ] after 1st iteration [ 2 3 ] [ 5 4 6 ] after 2nd iteration [ 2 3 4 ] [ 5 6 ] after 3rd iteration [ 2 3 4 5 ] [ 6 ] after last iteration
20
Selection Sort
Invariant: A[0..k-1] is sorted and all elements in A[0..k-1] are
less than or equal to all elements in A[k..n-1] On each iteration
A[smallIndex] is the element that is appended to A[0..k-1] producing a sorted A[0..k]
A[k+1..n-1] remains unsorted
21
Selection Sort Algorithm
Sort n elements of A into non-decreasing order
selectionSort(A,n) cost times for k = 0 to n-2 c1 n // scan unsorted sublist to find smallest value smallIndex = k c2 n-1 for j = k+1 to n-1 c3 f if A[j] < A[smallIndex] c4 g smallIndex = j c5 h // swap smallest value with leftmost if smallIndex != k c6 n-1 swap(A[k],A[smallIndex]) c7 0..n-1 f = g = h =
1
1
n
k
kt
1
1
)1(n
k
k
1
1
n
k
k
22
Selection Sort Analysis Best: already presorted
tk is 0, since the leftmost element of the unsorted sub-array is always the smallest element
No swaps are performed, since the array is already in order T(n) = c1(n) + (c2+c6)(n-1) + c3(f) + c4(g)
f = = n(n-1)/2 and g = = (n-1)(n-2)/2
T(n) = (c1+c2+c6)n – c6 + c3(n(n-1)/2) + c4((n-1)(n-2)/2) T(n) = an2 + bn + c
Quadratic Θ(n2)
1
1
n
k
k
1
1
)1(n
k
k
23
Selection Sort Analysis
Worst: presorted in reverse order Searching for smallest requires changing smallIndex on every
comparison in A[0..j-1] because initially A[k] > A[j] for all k < j A swap occurs n-1 times
T(n) = c1(n) + (c2+c6+c7)(n-1) + c3(f) + c4(g) + c5(h)
tk = k, so f = = n(n-1)/2 and g = h = = (n-1)(n-2)/2
T(n) = c1(n) + (c2+c6+c7)(n-1) + c3(n(n-1)/2) + (c4+c5)((n-1)(n-2)/2 ) T(n) = an2 + bn + c
Quadratic Θ(n2)
1
1
n
j
k
1
1
)1(n
j
k
24
Binary Search Algorithm
Case 1: A match occurs. The search is complete and mid is the index that locates the target.
if (midValue == target) // found match return mid;
m idfirs t
target
C as e 1: target = m id valueS earc h is d o ne
las t-1 las t
25
Binary Search Algorithm
Case 2: The value of target is less than midvalue and the search must continue in the lower sublist.
if (target < midvalue) // search lower sublist
<reposition last to mid>
<search sublist arr[first]…arr[mid-1]
las t-1firs t
target
C as e 2: target < m id valueS earc h lo w er s ub lis t
m id -1 las t
26
Binary Search Algorithm
Case 3: The value of target is greater than midvalue and the search must continue in the upper sublist .
if (target > midvalue)// search upper sublist
<reposition first to mid+1>
<search sublist arr[mid+1]…arr[last-1]>
C as e 3: target > m id valueS earc h up p er s ub lis t
las t-1new firs t = m id + 1
firs t
target
las t
27
Successful Binary Search
Search for target = 23
Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.
Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
28
Successful Binary Search
Step 2:
Indices first = 5, last = 9, mid = (5+9)/2 = 7.
Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
29
Successful Binary Search
Step 3:
Indices first = 5, last = 7, mid = (5+7)/2 = 6.
Since target = midvalue = 23, a match is found at index mid = 6.
m id
-7 3 5 8 12 16arr0 1 2 3 4 5
23 33 55
6 7 8 9
30
Unsuccessful Binary Search
Search for target = 4.
Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.
31
Unsuccessful Binary Search
Step 2:
Indices first = 0, last = 4, mid = (0+4)/2 = 2.
Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.
m id
-7 3 5 8 12 16arr0 1 2 3 4 5
23 33 556 7 8
32
Unsuccessful Binary Search
Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1.
Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index last = 9.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
33
Binary Search Implementationint binSearch (const int arr[], int first, int last,
int target)
{
int origLast = last; // original value of last
while (first < last) {
int mid = (first+last)/2;
if (target == arr[mid])
return mid; // a match so return mid
else if (target < arr[mid])
last = mid; // search lower sublist
else
first = mid+1; // search upper sublist
}
return origLast; // target not found
}
34
Binary Search Analysis
Each comparison divides the problem size in half
Assume for simplicity that n = 2k
So, worst case … How many times do we divide n in half before
there are no more values to check? T(n) = 1+ k = 1 + lg n Logarithmic Θ(lg n)