HKOI 2005 TrainingIntroduction to Algorithms
Alan, Tam Siu Lung
What is Algorithm?• Algorithm
– A set of data manipulations instructions to solve a problem (informatics problem)
• Problem– A mapping from each input (in a specific format) to an
output (in a specific format)• Pseudo-Code
– Realization of Algorithm in the current domain• Mapping from problem input to algorithm input• Mapping from algorithm output to problem output
• Program– Realization of Pseudo-Code given
• A compiler and the language supported• Data in plain text format and the IO routines supported
Visualization of Program
A BAlgorithm1
Coding Pseudo-Code
Source-Codeint main() {
A a = Read();
B b = Algorithm1(a);
C c = Algorithm2(b);
D d = Algorithm3(c);
Write(d);
}
B CAlgorithm2
C DAlgorithm3
Algorithm Example• Sorting Problem
– Given an array of N comparable elements with comparable keys
– Output a permutation of the elements– such that no element is has a key greater
than that of the one there before• Note
– Array = a sequence of elements– Element = a key-value pair– Permutation = ...– Greater than = …
Realization Example• To print in sorted order a set of points in a
cetacean plane by the distance from origin:• Realization contains
– Key = distance from origin– Value = coordinates = (x, y)– Comparison of 2 elements = comparison of the
numerical values of distances• Pseudo-Code contains
– Store the set in an array– Sort them– Output the array in order
Why Algorithms?
• Computational Biology• Database• Cryptography• Quantitative Analysis• Artificial Intelligence• Scheduling
Example – Multiple Sequence Alignment
• Input– GCTAGCTAAC– AACTAGC– AGCGCTAGCTA– TAACGACTAT
• Find a string with minimum which contains all these sub-strings
Example – Multiple Sequence Alignment
• Find a string with minimum which contains all these sub-strings
• OutputAACTAGCGCTAGCTAACGACTAT GCTAGCTAACAACTAGC AGCGCTAGCTA TAACGACTAT
Example – Sorting By Reversal
• Given a sequence of integers 1..N• Every step you can reverse a
continuous sub-subsequence• Find the minimum number of steps
to sort it
Example – Sorting By Reversal
• Input– 4 3 1 5 6 2
• Output– 4 3 1 5 6 2 – 1 3 4 5 6 2– 1 6 5 4 3 2– 1 2 3 4 5 6
To know what algorithms are
• Mathematics– Symbols and notations– Methods
• Data Structures– Specification of input and output– Internal storage during processing
• You can recite them.• It is science. You can also derive
them.
To know how to use algorithms
• Techniques– Common ways that algorithms are built
upon• Formulations
– How to map the problem domain into entities in algorithms
• It is art. We have no way to teach you!
• That’s why you need exercises.
Techniques• Recursion
– Reduction into sub-problems• Divide and Conquer
– Reduction into >1 sub-problems– Collect the results
• Exhaustion– Expand all reductions into sub-problems
• Greedy– Use only 1 reduction into sub-problem(s)
• Dynamic Programming– Try all and remember the one
To know why algorithms work
• Complexity Analysis• Proof of correctness• Mathematics-oriented• Out of our scope, but …• To “invent” new algorithms …
HKOI concerns• All basics and techniques specified
above– Mathematics, Data Structures– Recursion, Exhaustion, Greedy,
• Algorithms which implement the techniques– Sorting, Searching, DFS, BFS, Prim, Kruskal– Dijkstra, Bellman-Ford, Warshall-Floyd
• Famous IO models used by algorithms– Tree, Graph– Search Space (Graph Searching)– State Space (Constraint Satisfaction)
Introduction to Algorithms
Questions?
HKOI also concerns
• Modifications to algorithms– To accommodate the situation
• Multi-state BFS
– For speed• Branch & Bound
– For memory• Bit-wise States
• Other paradigms– Open test data (known input)– Interactive (hidden input)
Complexity
• An approximation to the runtime and memory requirement of a program.
• Note that there are– Best-case complexity– Average-case complexity– Worst-case complexity
• In most cases, we concern runtime only.
Measuring Runtime• Definition
– An operation– Problem size
• Consider bubble sort:for i := N - 1 downto 1 dofor j := 1 to i doif A[j] > A[j+1] thenswap(A[j], A[j+1]);
• In worst case how many “if” operations?“swap” operations? “for j” operations?“for j” operations? Integer comparisons?
Complexity
• Give a name to a group of functions with similar growth property, e.g. use O(n2) to mean n2 + n, 2n2 – 10n, 10n2, …
• Since the constant won’t be too big practically, it is a good indicator of expected runtime.
• We uses “worst case runtime complexity” (a.k.a. order) extensively to describe runtime of a program
Graph
1
2
3
4
5
76
8
Graph
WA
NT
SA
Q
NSW
V
T
Graph
• A directed graph G=(V,E), where– V = the set of Node/Vertex– E = the set of Edge/Arc, where
• E=(v1,v2) where v1 and v2 are in V
• Extreme: |E| ≤ |V|2
• So we have real algorithms with order:– O(|V|3)– O(|V| + |E|)– O(|V| log |V| + |E|)– O(|E| log(log* |E| – log* |E|/|V|))
Difficulty of Problem• Definitions (INCORRECT)
– A problem with order being a polynomial is called polynomial-time solvable (P)
– A problem whose solution is verified in polynomial time is said to be polynomial-time verifiable (NP)
– A problem with no known polynomial-time solution to date is called NP-hard
• Difficulty of problems are roughly classified as:– Easy: in P (of course all P problems are also in NP)– Hard: in NP but not in P (NP-complete)– Very Hard: not even in NP
Complexity
• When you have a problem, given input (output) constraints and runtime constraints, you can guess whether an algorithm works or not.
• But sometimes the stated order of an algorithm depends on some tricky data structures, which we won’t have skills or time to implement!
IOI/NOI Problems Requires
• Understanding of basic algorithms• Innovation on ways to realize the
algorithms• Technical capability of the
realization• Remember: Different people care
different things in different situations
Finally
Information, What
Knowledge, How
Understanding, Why InductionDed
uction