Computability and Complexity

Algorithms,Computability

andComplexity

Overview of mathematical concepts

Review of concepts already discussed

Algorithm

• You have a task• You want to formulate an algorithm to

perform the task• Three basic questions:

– What Is an algorithm?– Will an algorithm be able to compute the task?– How long will the algorithm take?

• how complex is the algorithm?

Three questions about algorithms

• These are actually non trivial questions– What Is an algorithm?– Will an algorithm be able to compute the task?– How long will the algorithm take?

• how complex is the algorithm?

• Important to mathematically formalize these questions – to create better algorithms– To know where the limitations of algorithms lie

Questions• What Is an algorithm?

– To make an algorithm, you have to know “exactly” what it is.. Its components and how to put these components together.

– To ask questions, you have to know what you are asking questions about

• Will an algorithm be able to compute the task?– Is the task I want impossible?– Is it possible to formulate an impossible task– What is the consequences for other algorithms

• How complex is the algorithm?• Will it take “too much” time? (for cryptography, this is a good thing)• Can I make a faster algorithm for a given type of task?

DEFINITION

What Is an algorithm?To make an algorithm, you have to know “exactly” what it is.. Its components and how to put these components together.To ask questions, you have to know what you are asking questions about

Algorithm Definition

• Part of the research is to formulate an exact workable definition of an algorithm

• There are several/many definitions– The do have common characteristics


• A logical sequence of steps for solving a problem, …

» From http://Dictionary.msn.com

• Dale and Lewis:» a plan of solution for a problem» Algorithm – An unambiguous (and precise) set of steps

for solving a problem (or sub-problem) in a finite amount of time using a finite amount of data.

http://dictionary.msn.com/


• Shackelford, Russell L. in Introduction to Computing and Algorithms – – “An algorithm is a specification of a behavioral

process. It consists of a finite set of instructions that govern behavior step-by-step.”

Definition• An algorithm is a finite sequence of step by

step, discrete, unambiguous instructions for solving a particular problem

– has input data, and is expected to produce output data

– each instruction can be carried out in a finite amount of time in a deterministic way

Keywords

• Notice the term finite. – Algorithms should lead to an eventual solution.– The algorithm should stop (halt).

• Sequence of steps – Each step should do one logical action.

COMPUTABILITY

Will an algorithm be able to compute the task?Is the task I want impossible?Is it possible to formulate an impossible taskWhat is the consequences for other algorithms

Computability vs Complexity

• Computability refers to whether of not in principle it is possible to evaluate f(n) by following a set of instructions.

• We are not for the moment worried if this computation requires 1,000,000 or more consecutive steps.

• The latter refers to complexity which we will return to later.

EnsheidungsproblemDecision problem

Hilbert 1928

The Entscheidungsproblem asks for an algorithm

that takes as input a statement of a first-order logic and

answers "Yes" or "No" according to whether the statement is universally valid

(valid in every structure satisfying the axioms).

An algorithm to ask whether something is trueIs it computable

EnsheidungsproblemDecision problem

Hilbert 1928

• Before the question could be answered, the notion of "algorithm" had to be formally defined. – This was done by Alonzo Church in 1936 with the

concept of "effective calculability" based on his λ calculus

– and by Alan Turing in the same year with his concept of Turing machines.

– It was recognized immediately by Turing that these are equivalent models of computation.

15

Alan Turing (1912 – 1954)

Alonzo Church (1903-1995)

Turing Machine Lambda calculus

Two mathematical ways to ask questions about “computability”

Equivalent Computers

z z zz z z z

1

Start

HALT

), X, L

2: look for (

#, 1, -

), #, R

(, #, L

(, X, R

#, 0, -

Finite State Machine

...

Turing Machine

term = variable | term term | (term) | variable . term

y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus

Alan Turing, 1936

• "On computable numbers, with an application to the Entscheidungsproblem [decision problem]”

• addressed a previously unsolved mathematical problem posed by the German mathematician David Hilbert in 1928:

Turing, 1936

Is there, in principal, any definite mechanical method or process

by which all mathematical questions

could be decided?

The Decision Problem

Turing, 1936

• To answer the question (in the negative), • Turing proposed a simple abstract computing

machine, • modeled after a mathematician with a pencil,

an eraser, and a stack of pieces of paper• He asserted that any function is a computable

function if it is computable by one of these abstract machines.

Turing, 1936

• He then investigated whether there were some mathematical problems which could not be solved by any such abstract machine.

• Such abstract computing machines are now called "Turing machines".

• One particular Turing machine, called a "Universal Turing Machine", served as the model for designing actual programmable computers.

Why a Turing Machine

Because Turing machines are very simple compared with computers in practical use,

conceptually easier to prove impossibility results

for Turing machines. These impossibility results

apply as well to all known computers

Turing, 1936

• Startling result of Turing's 1936 paper• assertion that there are well-defined

problems that cannot be solved by any computational procedure.

What is Computable?Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps.

An “effectively computable” function is one that can be computed in a finite amount of time using finite resources.

………………………

input

yes

no

outputEffectively

computable function

Unsolvable

• problems are formulated as functions, we call unsolvable functions are

• noncomputable;• Problems formulated as predicates,• they are called • undecidable.• Using Turing's concept of the abstract machine, we

would say that a function is noncomputable if there exists no Turing machine that could compute it.

25

............Tape

Read-Write headControl Unit

-Dept. of CSIE, CCU, Taiwan- 26

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The tape

OR


............

Read-Write head

The head at each time step:

1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right


............

Example: Time 0

............Time 1

a a cb

a b k c

1. Reads a2. Writes k 3. Moves Left

a


............Time 1

a b k c

............Time 2

a k cf

1. Reads b2. Writes f3. Moves Right

b

Add 1 to a sequence of ones

If 1, then move right If 1, then move left

Loop

If 0, replace with 1

Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row

End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row

So what?

You can add a 1 to a sequence of ones

From a set of a few (a manageable number) of primitive operations

If a number is represented by a sequence of onesthen you have the x+1 operator, i.e addition

This is a/(the most) basic mathematical operator

Powerful enough to represent complex mathematics

You can derive other (all) mathematic operations and objects

Powerful “language”From a set of a few (a manageable number) of

primitive operations

Easier enough To formulate Proofs about algorithms

Powerful enough To represent complex mathematics

All the ingredients are there Only have to deal with a few basic operations

Not worried about complexity…. Worried about, for example, “can” we compute?


The Input String

............

Blank symbol

head

a b ca

Head starts at the leftmost positionof the input string

Input string


States & Transitions

1q 2qLba ,

Read Write Move Left

1q 2qRba ,

Move Right


Turing machine for the language }{ nnba

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

ba

0q

a bTime 0


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

bx

1q

a b Time 1


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

x

2q

a b Time 2 b


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 3


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 4


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

a b Time 5


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 6


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 7


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 8


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 9


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

x y Time 10


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 11


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 12


0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

4q

x y

Halt & Accept

Time 13

What is Computable?Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps.

An “effectively computable” function is one that can be computed in a finite amount of time using finite resources.

………………………

input

yes

no

outputEffectively

computable function

Uncomputable

Using Turing's concept of the abstract machine, we would say that a function

is noncomputable

if there exists no Turing machine that could compute it.

Computability

Function f is computable if some program P computes it:

For any input x, the computation P(x) halts with output f(x)

Finite number of steps

Algorithm for 𝚷• To calculate the area of a circle, • we need the value of 𝚷• Can we find an algorithm to compute ?𝚷

𝚷• 3.14159265358979323846264338327950288419716939937510• 58209749445923078164062862089986280348253421170679• 82148086513282306647093844609550582231725359408128• 48111745028410270193852110555964462294895493038196• 44288109756659334461284756482337867831652712019091• 45648566923460348610454326648213393607260249141273• 72458700660631558817488152092096282925409171536436• 78925903600113305305488204665213841469519415116094• 33057270365759591953092186117381932611793105118548• 07446237996274956735188575272489122793818301194912• 98336733624406566430860213949463952247371907021798• 60943702770539217176293176752384674818467669405132• 00056812714526356082778577134275778960917363717872• 14684409012249534301465495853710507922796892589235• 42019956112129021960864034418159813629774771309960• 51870721134999999837297804995105973173281609631859• 50244594553469083026425223082533446850352619311881• 71010003137838752886587533208381420617177669147303• 59825349042875546873115956286388235378759375195778• 18577805321712268066130019278766111959092164201989…….

The first 1000 digits of 𝚷

Goes on as long as you want… Π 10,000 digitsΠ 100,000 digits

http://www.math.utah.edu/~pa/math/pi.html


http://www.geom.uiuc.edu/~huberty/math5337/groupe/digits.html

http://www.geom.uiuc.edu/~huberty/math5337/groupe/digits.html


Algorithm for 𝚷An algorithm (in general)is a finite sequence of step by step, discrete, unambiguous instructions for solving a particular problem

Problem 𝚷 is a infinite sequence of numbers…

cannot calculate with a finite number of steps

But there are algorithms????

Calculating 𝚷• We can’t

– Calculate the EXACT value of 𝚷• However, we can

– Calculate to whatever accuracy we desire𝚷– Finite number of digits, finite number of steps

Probabilistic AlgorithmsCalculate pi with a dart board

Area of square

d2

Area of Circle:

Probability dart will be in circle

d

number darts in circledivided by

number of darts in totaltimesIs π

Monte Carlo MethodAlways Gives an answerBut not necessarily CorrectThe probability of correctness goes up with time

Calculate

All do not stop (halt)…..

They can be an algorithm if only an approximation is needed

Halting function

• Decide whether program halts on input– Given program P and input x to P,

Halt (P,x) =

Fact: There is no algorithm for Halt

yes if P(x) haltsno otherwise

ClarificationsAssume program P requires one string input xWrite P(x) for output of P when run in input xProgram P is string input to Halt

Unsolvability of the halting problem

• Suppose P solves variant of halting problem– On input Q, assume P(Q) =

• Build program D

– D(Q) =

• Does this make sense? What can D(D) do? – If D(D) halts, then D(D) runs forever.– If D(D) runs forever, then D(D) halts.– CONTRADICTION: program P must not exist.

yes if Q(Q) haltsno otherwise

run forever if Q(Q) haltshalt if Q(Q) runs forever

Lambda Calculus

Alonzo Church (1903-1995)

Another formulationofcomputability

Equivalent Computers

z z zz z z z

1

Start

HALT

), X, L

2: look for (

#, 1, -

), #, R

(, #, L

(, X, R

#, 0, -

Finite State Machine

...

Turing Machine


y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus

What is Calculus?

• In High School:d/dx xn = nxn-1 [Power Rule]d/dx (f + g) = d/dx f + d/dx g [Sum Rule]

Calculus is a branch of mathematics that deals with limits and the differentiation and integration of functions of one or more variables...

Real Definition• A calculus is just a bunch of rules for

manipulating symbols.• People can give meaning to those symbols,

but that’s not part of the calculus.• Differential calculus is a bunch of rules for

manipulating symbols. There is an interpretation of those symbols corresponds with physics, slopes, etc.

-calculusAlonzo Church, 1940


-calculus

variable . term

F(variable) = term

Why?

• Once we have precise and formal rules for manipulating symbols, we can use it to reason with.

• Since we can interpret the symbols as representing computations, we can use it to reason about programs.

Universal Computer• Lambda Calculus can simulate a Turing Machine

– Everything a Turing Machine can compute, Lambda Calculus can compute also

• Turing Machine can simulate Lambda Calculus– Everything Lambda Calculus can compute, a Turing

Machine can compute also• Church-Turing Thesis: this is true for any other

mechanical computer also

Normal Steps• Turing machine:

– Read one square on tape, follow one FSM transition rule, write one square on tape, move tape head one square

• Lambda calculus:– One beta reduction

• Your PC:– Execute one instruction (?)

• What one instruction does varies

-Reduction (the source of all computation)

(x. M)N

This is the function definitionM is an expression with the variable x

N is used at the value of x

N substitutes x in the expression M

(x. x)a = a

-Reduction (the source of all computation)

Identity operationFunction gives the argument as output

(x. xx)a = aa(y.(x. xy)a)b =(y.ay)b=ab (y.(x. xy) z.zy)b =(y.(z.zy)y)b = (y. yy)b =bb

-calculusUseful and expression enough that a programming language was

modeled after it

LISP was developed from -calculus

not the other way round.)

This was the “first” general language ofArtificial intelligence

AlsoThe “first” functional programming language

ComplexityAlgorithm:An algorithm is a finite sequence of step by step, discrete, unambiguous instructions for solving a particular problem

Computable: Does an algorithm exist??

Complexity:The algorithm existsHow “complex” is it?How many “descrete steps” does it have?how much does each step cost?

How does complexity depend on size of input?

Vector/Array of size n

Efficient for the nth element

A single arithmetic calculationComplexity does not increase as the vector gets bigger

O(1)

76 8 9 10 11 120 1 2 3 4 5

Pos(v[0]) Pos(v[0]) + 5

Find ith element of vector

This is exactly an example for what a vector is designed for

Cost/Complexity

Compute: Pos(v[0]) + n

Find the nth element

Cost:C1: Cost of determining the address of the first element Pos(v[0])

Does not depend on the size of the array of numbers

C2: Cost of adding n to this address… cost of addition operation

Does not depend on the size of the array of numbers

Total cost: C1 + C2

Cost/Complexity Compute: Pos(v[0]) + nFind the nth element

Cost: Does not depend on the size of the array of numbers

Total cost = T = C1 + C2

N: size of array

T: To

tal C

ost

Cost(n) = T a constant amount

In the O notation

O(1)The constant factor is not so important

It is only important that there in no dependence on the size of the array

Vector/Array of size nInsert before ith element in vector

76 8 9 10 11 120 1 2 3 4 5

1. Allocate vector of size n+1

2. Copy element 0 to i-1 to places 0 to i-1 and copy elements i to n-1 to places i+1 to n

76 8 9 10 11 120 1 2 3 4 5

1 operation

n operations

3. Set in element

76 8 9 10 11 120 1 2 3 4 5 13

1 operation

Vectors are not designed to be used with insertion operationsAs the vector gets larger, the insertion takes more time/operations

O(n)

Cost of insertion

• C1: Allocate array size n+1 • C2: Copy I elements from old to new array

• C2=(i)*c (c: the cost to copy an element)• C3: Copy the element to insert

• C3= c (c: the cost to copy an element)• C4: Copy n-I elements from old to new array

• C4=(n-i)*c

Insert a new element at position I in array of length n

Total Cost = T = C1 + C2 + C3 + C4 = C1 + i*c + c + (n-i)*c = C1 + (n+1)*c

Total Cost = T = C1 + (n+1)*c

Cost of insertionInsert a new element at position I in array of length n

Total Cost = T(n) = C1 + (n+1)*c

Cost: Depends on the size of the array of numbers

N: size of array

T: To

tal C

ost

Cost(n) = C1 + (n+1)*c

In the O notation

O(n)The constant factor is not so important

It is only important that there in no dependence on the size of the array

ComplexityO(1) Time/operation complexity does not increase

with the size of the problem

O(n) Time/operation complexity does increases linearly with the size of the problem

Find ith element of vector

Insert before ith element in vector

N: size of array

T: To

tal C

ost

N: size of array

T: To

tal C

ost

Complexity of Bubble Sort

Bubble Sort: Idea

• Idea: bubble in water.– Bubble in water moves upward. Why?

• How?– When a bubble moves upward, the

water from above will move downward to fill in the space left by the bubble.

Bubble Sort Example9, 6, 2, 12, 11, 9, 3, 76, 9, 2, 12, 11, 9, 3, 76, 2, 9, 12, 11, 9, 3, 76, 2, 9, 12, 11, 9, 3, 76, 2, 9, 11, 12, 9, 3, 76, 2, 9, 11, 9, 12, 3, 76, 2, 9, 11, 9, 3, 12, 76, 2, 9, 11, 9, 3, 7, 12The 12 is greater than the 7 so they are exchanged.

The 12 is greater than the 3 so they are exchanged.

The twelve is greater than the 9 so they are exchanged

The 12 is larger than the 11 so they are exchanged.

In the third comparison, the 9 is not larger than the 12 so no exchange is made. We move on to compare the next pair without any change to the list.

Now the next pair of numbers are compared. Again the 9 is the larger and so this pair is also exchanged.

Bubblesort compares the numbers in pairs from left to right exchanging when necessary. Here the first number is compared to the second and as it is larger they are exchanged.

The end of the list has been reached so this is the end of the first pass. The twelve at the end of the list must be largest number in the list and so is now in the correct position. We now start a new pass from left to right.

Bubble Sort Example

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 11, 9, 3, 7, 122, 6, 9, 9, 11, 3, 7, 122, 6, 9, 9, 3, 11, 7, 122, 6, 9, 9, 3, 7, 11, 126, 2, 9, 11, 9, 3, 7, 12

Notice that this time we do not have to compare the last two numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons.

First Pass

Second Pass

Bubble Sort Example

2, 6, 9, 9, 3, 7, 11, 122, 6, 9, 3, 9, 7, 11, 122, 6, 9, 3, 7, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

This time the 11 and 12 are in position. This pass therefore only requires 5 comparisons.

Bubble Sort Example

2, 6, 9, 3, 7, 9, 11, 122, 6, 3, 9, 7, 9, 11, 122, 6, 3, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

Each pass requires fewer comparisons. This time only 4 are needed.

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

Bubble Sort Example

2, 6, 3, 7, 9, 9, 11, 122, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

The list is now sorted but the algorithm does not know this until it completes a pass with no exchanges.

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

2, 6, 3, 7, 9, 9, 11, 12Fifth Pass

Bubble Sort Example

2, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

2, 6, 3, 7, 9, 9, 11, 12Fifth Pass

Sixth Pass 2, 3, 6, 7, 9, 9, 11, 12This pass no exchanges are made so the algorithm knows the list is sorted. It can therefore save time by not doing the final pass. With other lists this check could save much more work.

Bubble Sort: Example

• Notice that at least one element will be in the correct position each iteration.

40 2 1 43 3 65 0 -1 58 3 42 4

652 1 40 3 43 0 -1 58 3 42 4

65581 2 3 40 0 -1 43 3 42 4

1 2 3 400 65-1 43 583 42 4

1

2

3

4

1 0 -1 32 653 43 5842404

Bubble Sort: Example

0 -1 1 2 653 43 58424043

-1 0 1 2 653 43 58424043

6

7

8

1 2 0 3-1 3 40 6543 584245

Bubble Sort Complexity

1 2 3 4 5 n-1 comparisons Total Cost= (n-1) * c

Even the lowest cost is a function of nCost = (n-1)*c = -c + n*c

O(n)

Remember: not so interested in the constant factors

N: size of array

T: To

tal C

ost

Bubble Sort Complexity5 4 3 2 1 Worse case: exactly opposite

4 5 3 2 1 Compare (c ) and swap (s)




3 2 1 4 5



Compare (c ) and swap (s)


Move 5 to place

4* (c+s)

Move 5 to place

3* (c+s)

Move 5 to place

1* (c+s)



Move 5 to place

2* (c+s)


Total Cost:

5*(c+s) + 4*(c+s) + 3*(c+s) + 2*(c+s) 1*(c+s)

(5+4+3+2+1)*(c+s)

15*(c+s)

3*5*(c+s)

In general the sum of numbers from 1 to n is:

If n odd: n*(n+1)/2If n even n*n/2


Size of array to sort

Cost

:

In general the sum of numbers from 1 to n is:

If n odd: n*(n+1)/2= n2/2 + n/2If n even n*n/2 = n2/2

Approx: n2/2

For a large n…. n2 is much much larger than n

Complexities

Size of object for algorithm

Cost

:

O(n)

O(1)

O(n2)

Tractability

• Some problems are intractable: as they grow large, we are unable to solve them in reasonable time

• What constitutes reasonable time? – Standard working definition: polynomial time– On an input of size n the worst-case running time is

O(nk) for some constant k– O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)– Polynomial time: O(n2), O(n3), O(1), O(n lg n) – Not in polynomial time: O(2n), O(nn), O(n!)

Polynomial-Time Algorithms

• Are some problems solvable in polynomial time?– Of course: many algorithms we’ve studied provide

polynomial-time solutions to some problems• Are all problems solvable in polynomial time?

– No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given

• Most problems that do not yield polynomial-time algorithms are either optimization or decision problems.

Optimization/Decision Problems

• An optimization problem tries to find an optimal solution• A decision problem tries to answer a yes/no question• Many problems will have decision and optimization versions

– Eg: Traveling salesman problem• optimization: find hamiltonian cycle of minimum

weight• decision: is there a hamiltonian cycle of weight k

• Some problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time– Is there a polynomial-time algorithm that solves the problem?

The Class P

P: the class of decision problems that have polynomial-time deterministic algorithms. – That is, they are solvable in O(p(n)), where p(n) is a polynomial on n– A deterministic algorithm is (essentially) one that always computes the

correct answer

Why polynomial?– if not, very inefficient– nice closure properties

• the sum and composition of two polynomials are always polynomials too

Sample Problems in P

• Fractional Knapsack• MST • Sorting• Others?

The class NPNP: the class of decision problems that are solvable in polynomial

time on a nondeterministic machine (or with a nondeterministic algorithm)

• (A determinstic computer is what we know)• A nondeterministic computer is one that can “guess” the right

answer or solution – Think of a nondeterministic computer as a parallel machine that can

freely spawn an infinite number of processes• Thus NP can also be thought of as the class of problems

– whose solutions can be verified in polynomial time • Note that NP stands for “Nondeterministic Polynomial-time”

Review: P And NP Summary

• P = set of problems that can be solved in polynomial time– Examples: Fractional Knapsack, …

• NP = set of problems for which a solution can be verified in polynomial time– Examples: Fractional Knapsack,…, Hamiltonian Cycle, CNF

SAT, 3-CNF SAT• Clearly P NP• Open question: Does P = NP?

– Most suspect not– An August 2010 claim of proof that P ≠ NP, by Vinay

Deolalikar, researcher at HP Labs, Palo Alto, has flaws

NP-complete problems

• A decision problem D is NP-complete iff1. D NP2. every problem in NP is polynomial-time reducible

to D

• Cook’s theorem (1971): CNF-sat is NP-complete

NP-Complete“NP-Complete” comes from:

– Nondeterministic Polynomial– Complete - “Solve one, Solve them

all”

There are more NP-Complete problems than provably intractable problems.

Date post:	26-Jan-2017
Category:	Education
Upload:	edward-blurock
View:	314 times
Download:	0 times

Computability and Complexity

Education