Post on 20-Jan-2021
transcript
. . . . . . 1/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Algorithms and Data Structures1st Lecture: Getting Started
Yutaka Watanobe, Jie Huang, Yan Pei,Wenxi Chen, Qiangfu Zhao, Wanming Chu
University of Aizu
Last Updated: 2020/11/30
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 2/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Outline
AlgorithmsData StructuresPseudocodeInsertion Sort
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 3/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Algorithms
Informally, an algorithm is any well-defined computationalprocedure that takes some value, or set of values, as input andproduces some value, or set of values, as output.An algorithm is thus a sequence of computational steps thattransform the input into the output.Example: Sorting problem
Input: A sequence of n numbers (a1, a2, ..., an).Output: A permutation (i.e., reordering) (a′
1, a′2, ..., a
′n) of the input
sequence such that a′1 ≤ a′
2 ≤ ... ≤ a′n.
Such an input sequence is called an instance of the sortingproblem.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 4/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Correct Algorithms
An algorithm is said to be correct if, for every input instance, ithalts with the correct output. We also say that a correct algorithmsolves the given computational problem.An algorithm can be specified in English, as a computerprogram, or even as a hardware design. The only requirement isthat the specification must provide a precise description of thecomputational procedure to be followed.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 5/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Data Structures
A data structure is a way to store and organize data in order tofacilitate access and modifications.No single data structure works well for all purposes, and so it isimportant to know the strengths and limitations of several ofthem.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 6/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Data Structures: Examples
Array 2D-Array List
Stack TreeQueue Graph
ND-Array
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 7/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Pseudocode Convention (1)
We typically describe algorithms as programs written in apseudocode that is similar in many respects to C, C++ or Java.In pseudocde, we employ whatever method is most clear andconcise to specify a given algorithm.The symbols /* ... */ indicate that the statement is a comment.The symbols // also indicate that the statement located on theright side of them is a commnet.Indentation indicates block structure.The symbol← or = indicates variable assignment.Parameters are passed to a prodecure by value.The Boolean operators are short-circuiting.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 8/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Pseudocode Convention (2)
A[i] indicates the i-th element of an array A.A.length indicates the length of the array A.A[1, ... j] indicates a sub-sequence of A including A[1], A[2], ...A[j]We use both 0-origin and 1-origin depending on the situation.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 9/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort
Consider the previous sorting problem.Input: A sequence of n numbers (a1, a2, ..., an).Output: A permutation (i.e., reordering) (a′
1, a′2, ..., a
′n) of the input
sequence such that a′1 ≤ a′
2 ≤ ... ≤ a′n.
Let us consider the Insertion Sort algorithm.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 10/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (into nondecreasing order)
0 1 32 4 5
0 1 32 4 5
0 1 32 4 5
0 1 32 4 5
0 1 32 4 5
0 1 32 4 5
0 1 32 4 5
8 3 51 2 1
8
3
51 2 1
5
1
2 1
1 3
5
8 2 1
2
1
1 2
1
1 1 32 5 8
3 8
3 85
53 8
0.
1.
2.
3.
4.
5.
6.
13
1
5
2
1
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 11/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (0)
0 1 32 4 5
8 3 51 2 1
0.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 12/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (1)
0 1 32 4 5
8
3
51 2 1
1.
3
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 13/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (2)
0 1 32 4 5
5
1
2 13 8
2.
1
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 14/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (3)
0 1 32 4 5
1 3
5
8 2 1
3.
5
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 15/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (4)
0 1 32 4 5
2
13 85
4.
1 2
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 16/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (5)
0 1 32 4 5
1 2
1
53 8
5.
1
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 17/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Insertion Sort (6)
0 1 32 4 5
1 1 32 5 8
6.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 18/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Implementation
Pseudo code Insertion Sort (Note that array indices are based on0-origin, and the number of elements is n)
01. for j ← 1 to n-1
02. key ← A[j]
03. // insert A[j] into the sorted sequence A[0,...,j-1]
04. i ← j - 1
05. while i >= 0 and A[i] > key
06. A[i+1] ← A[i]
07. i ← i-1
08. A[i+1] ← key
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 19/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Analyzing Algorithms
Analyzing an algorithm has come to mean predicting theresources that the algorithm requires.Occasionally, resources such as memory, communicationbandwidth, or computer hardware are of primary concern, butmost often it is computational time that we want to measure.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 20/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Input Size and Running Time
The best notion for input size depends on the problem beingstudied.For many problems, the most natural measure is the number ofitems in the input. For many other problems, the best measure isthe total number of bits needed to represent the input in ordinarybinary notation.The running time of an algorithm on a particular input is thenumber of primitive operations or ”steps” executed.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 21/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Analysis of Insertion Sort
InsertionSort(A) cost times01. for j ← 1 to n-1 c1 n02. key ← A[j] c2 n − 103. // comment line c3 n − 104. i ← j - 1 c4 n − 105. while i >= 0 and A[i] > key c5
∑nj=2 tj
06. A[i+1] ← A[i] c6∑n
j=2(tj − 1)07. i ← i-1 c7
∑nj=2(tj − 1)
08. A[i+1] ← key c8 n − 1
Let tj be the number of times the while loop test in line 5 is executedfor the value of j .
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 22/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Calculation of Running Time
The running time of the algorithm is the sum of running times foreach statement executed.We sum the product of the ”cost” and ”times” columns.Let T (n) be the running time of InsertionSort(A) on an inputsequence of size n.
T (n) = c1n + c2(n − 1) + c4(n − 1)
+ c5
n∑j=2
(tj) + c6
n∑j=2
(tj − 1) + c7
n∑j=2
(tj − 1) + c8(n − 1)
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 23/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Calculation (Best Case)
The best case occurs if the array is already sorted.For each j = 2,3, ..., n, we then find that A[i] ≤ key in line 5 wheni has its initial value of j − 1.Thus, tj = 1 for j = 2,3, ..., n.
The best-case running time is given as follows:
T (n) = c1n + c2(n − 1) + c4(n − 1) + c5(n − 1) + c8(n − 1)= (c1 + c2 + c4 + c5 + c8)n − (c2 + c4 + c5 + c8)
it is thus a linear function of n.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 24/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Calculation (Worst Case)
The worst case occurs if the array is in reverse sorted order.We must compare each element A[j] with each element in theentire sorted subarray A[1..j-1].Thus, tj = j for j = 2,3, ..., n.
The worst-case running time is given as follows:
T (n) = c1n + c2(n − 1) + c4(n − 1) + c5(n(n + 1)
2− 1)
+ c6(n(n − 1)
2) + c7(
n(n − 1)2
) + c8(n − 1)
= (c5
2+
c6
2+
c7
2)n2 + (c1 + c2 + c4 +
c5
2− c6
2
− c7
2+ c8)n − (c2 + c4 + c5 + c8)
it is thus a quadratic function of n.Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 25/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Worst-Case and Average-Case Analysis
We usually concentrate on finding only the worst-case runningtime, that is, the longest running time for any input of size n.In some particular cases, we shall be interested in theaverage-case (or expected) running time of an algorithm.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 26/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Worst-Case Analysis
Three reasons for using the worst case:The worst-case running time of an algorithm is an upper boundon the running time for any input. Knowing it gives us aguarantee that the algorithm will never take any longer.For some algorithms, the worst case occurs fairly often. Forexample, in searching a database for a particular piece ofinformation, the searching algorithm’s worst case will often occurwhen the information is not present in the database.The average case is often roughly as bad as the worst case.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures
. . . . . . 27/27
Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm
Reference
1 Introduction to Algorithms (third edition), Thomas H.Cormen,Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. TheMIT Press, 2012.
Y. Watanobe, J. Huang, Y. Pei, W. Chen, Q. Zhao, W. Chu University of Aizu
Algorithms and Data Structures