Download - The axiomatic power of Kolmogorov complexity

.

.

.

How useful are random axioms?

Laurent Bienvenu (CNRS & University of Paris 7)Andrei Romashchenko (University of Montpellier 2)Alexander Shen (University of Montpellier 2)Antoine Taveneaux (University of Paris 7)Stijn Vermeeren (University of Leeds)

Séminaire Logique

October28, 2012

.

.

..

1. Random axioms

.

.

Peano arithmetic

In this talk, we work in the axiomatic system of Peano arithmetic, PA, over thelanguage (0, 1,+,×). Its standard model N is the set N of natural numbers,with standard addition and multiplication.

As it is well known, Gödel’s theorem applies to this theory: there are sentencesthat are true (in the standard model) but not provable.

PA ⊢ φ ⇒⇍ N |= φ

1. Random axioms 3/31

.

.

Peano arithmetic

In this talk, we work in the axiomatic system of Peano arithmetic, PA, over thelanguage (0, 1,+,×). Its standard model N is the set N of natural numbers,with standard addition and multiplication.As it is well known, Gödel’s theorem applies to this theory: there are sentencesthat are true (in the standard model) but not provable.

PA ⊢ φ ⇒⇍ N |= φ


.

.

Random axiomsNow imagine that a formula φ has the following properties:

1. PA ⊬ φ(n̄) for any n ≤ N

2. At least 99.9% of integers n ≤ N satisfy φ3. Item 2. is provable inside PA

Then, if we pick an integer r ≤ N at random and consider the theory

PA+φ(̄r)

with error probability at most 0.1% our new theory is coherent (assuming PA is)and true.

Now ifPA+φ(̄r) ⊢ ψ

we have (in some sense) “evidence” that ψ is likely to be true.


.

.


1. PA ⊬ φ(n̄) for any n ≤ N2. At least 99.9% of integers n ≤ N satisfy φ

3. Item 2. is provable inside PA


PA+φ(̄r)





.

.


1. PA ⊬ φ(n̄) for any n ≤ N2. At least 99.9% of integers n ≤ N satisfy φ3. Item 2. is provable inside PA


PA+φ(̄r)





.

.




PA+φ(̄r)





.

.




PA+φ(̄r)





.

.

The canonical example

The canonical example:

C(x) ≥ L− 20 for binary strings x of length ≤ L

where C denotes Kolmogorov complexity (to which we will return shortly).

..Theorem (Chaitin)The above statement is true for a fraction ≥ 1 − 10−6 of strings x (and this wecan prove!), but is not provable in PA for any x (assuming L is large enough).



On this particular example, we could choose a binary string r of length L atrandom (say by flipping a coin), add to PA the new axiom C(r) ≥ L− 20 and tryto prove new statements.


.

.










.

.










.

.

Probabilistic proofs (1)

This yields the natural idea of probabilistic proof.

Fix an error threshold δ in advance (margin of error that you find acceptable)which will be treated as a initial capital. Start with the theory T0 = PA.


.

.


This yields the natural idea of probabilistic proof.

Fix an error threshold δ in advance (margin of error that you find acceptable)which will be treated as a initial capital. Start with the theory T0 = PA.


.

.


At step n, we have two options:

• We can use the statements of our theory Tn to prove (in the normal way) anew fact ψ and take Tn+1 = Tn ∪ {ψ}

• Or, if we have in Tn a statement of type

#{n ∈ A | φ(n̄)} ≥ (1 − ε) · |A|

for some finite set A, we can choose some r ∈ A at random, pay ε fromour capital, and take Tn+1 = Tn ∪ {φ(̄r)} (unless our capital becomesnegative in which case this step is not allowed)


.

.





#{n ∈ A | φ(n̄)} ≥ (1 − ε) · |A|



.

.





#{n ∈ A | φ(n̄)} ≥ (1 − ε) · |A|



.

.


Such a proof scheme (or strategy) is probabilistic: if we run it twice we mightprove completely different statements!

Still, if a fixed statement ψ could be proven with high probability with thisscheme, our intuition tells us that ψ is very likely to be true. This raises twoquestions:

1. Is this intuition correct? i.e., does this proof scheme make sense?2. Is it useful? i.e., can we prove more things than we classically could?


.

.






.

.




1. Is this intuition correct? i.e., does this proof scheme make sense?

2. Is it useful? i.e., can we prove more things than we classically could?


.

.






.

.


The answer: Yes, it makes sense.

..TheoremIf a proof strategy with initial capital δ proves ψ with probability > δ, then ψ istrue.



No, it is not (really) useful.

..TheoremIf a proof strategy with initial capital δ proves ψ with probability > δ, then in factψ is already provable in PA.




.

.











.

.











.

.


Probabilistic proofs are still useful in the following sense: in the determinizationargument (transforming a probabilistic proof into a deterministic one), there is anexponential blow-up, i.e., the deterministic proof is in general exponentiallylonger than the probabilistic one.

In fact, this cannot be avoided:

..TheoremUnless NP=PSPACE, there are statements with polynomial-size probabilisticproofs and only exponential-size deterministic ones.




.

.


Probabilistic proofs are still useful in the following sense: in the determinizationargument (transforming a probabilistic proof into a deterministic one), there is anexponential blow-up, i.e., the deterministic proof is in general exponentiallylonger than the probabilistic one.

In fact, this cannot be avoided:





.

.

..

2. The axiomatic powerof Kolmogorov complexity

.

.

Kolmogorov complexity (1)

The Kolmogorov complexity C(x) of a finite binary string x is the length of theshortest program (written in binary) that generates x. In a sense, it measuresthe amount of information contained in the string x.

For any given string x, we have

0 ≤ C(x) ≤ |x|+ O(1)

C(x) ≈ 0 meaning that x is simple (= far from random), while C(x) ≈ |x| meansthat x is quite random.

Most strings are quite random: there is only a fraction ≤ 2−c of strings of a givenlength such that C(x) ≤ |x|− c.

2. The axiomatic powerof Kolmogorov complexity 12/31

.

.




0 ≤ C(x) ≤ |x|+ O(1)




.

.




0 ≤ C(x) ≤ |x|+ O(1)




.

.


Variation 1: K(x) is the prefix-free Kolmogorov complexity, i.e. the Kolmogorovcomplexity one gets if the set of valid programs is prefix-free.

Variation 2: C(x | y) and K(x | y) denote the conditional and prefix-freeconditional Kolmogorov complexity, i.e., the length of the shortest program thatproduces x given y as input.


.

.


Variation 1: K(x) is the prefix-free Kolmogorov complexity, i.e. the Kolmogorovcomplexity one gets if the set of valid programs is prefix-free.

Variation 2: C(x | y) and K(x | y) denote the conditional and prefix-freeconditional Kolmogorov complexity, i.e., the length of the shortest program thatproduces x given y as input.


.

.


Kolmogorov complexity is not a computable function. It is howeverupper-semicomputable: one can computably enumerate pairs (x, n) such thatC(x) ≤ n.

To summarize:• If C(x) ≤ n1 we will find out eventually• If C(x) ≥ n2 we might never know for sure


.

.


Kolmogorov complexity is not a computable function. It is howeverupper-semicomputable: one can computably enumerate pairs (x, n) such thatC(x) ≤ n.

To summarize:• If C(x) ≤ n1 we will find out eventually• If C(x) ≥ n2 we might never know for sure


.

.


Translation into the language of logic:

..

Theorem (Chaitin)• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”• No matter what the actual value of C(x) is, PA ⊬ “C(x) ≥ n2” (provided n2 islarge enough).

..Theorem (Chaitin)• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”• No matter what the actual value of C(x) is, PA ⊬ “C(x) ≥ n2” (provided n2 islarge enough).


Proof: Berry’s paradox → otherwise, given n, look for an xn such thatPA ⊢ “C(xn) ≥ n”. Since n is enough to describe xn, C(xn) ≤ O(log n), acontradiction.


.

.


Translation into the language of logic:

..

Theorem (Chaitin)• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”• No matter what the actual value of C(x) is, PA ⊬ “C(x) ≥ n2” (provided n2 islarge enough).



Proof: Berry’s paradox → otherwise, given n, look for an xn such thatPA ⊢ “C(xn) ≥ n”. Since n is enough to describe xn, C(xn) ≤ O(log n), acontradiction.


.

.

Gödel’s theorem

In the previous argument, PA can be replaced by any stronger, recursive andcoherent theory. Thus, we have obtained a proof of Gödel’s first incompletenesstheorem using a Kolmogorov complexity interpretation of Berry’s paradox.

Note: one can also get a very elegant proof of Gödel’s secondincompleteness theorem via a Kolmogorov complexity interpretation of thesurprise examination paradox (Kritchman and Raz).


.

.

Gödel’s theorem

In the previous argument, PA can be replaced by any stronger, recursive andcoherent theory. Thus, we have obtained a proof of Gödel’s first incompletenesstheorem using a Kolmogorov complexity interpretation of Berry’s paradox.

Note: one can also get a very elegant proof of Gödel’s secondincompleteness theorem via a Kolmogorov complexity interpretation of thesurprise examination paradox (Kritchman and Raz).


.

.

Adding axioms about C (1)

Since PA does not prove them, a natural question:

How useful is the set C of true axioms of type “C(x) ≥ n”?

..TheoremThe statements provable from PA + C are exactly those of Π1-Th(N), i.e., thestatements φ that only contain ∀ as unbounded quantifiers and are true in N.



Proof. This is a direct translation into logic of the fact that if one can compute Cthen one can compute the halting problem ∅ ′.


.

.









.

.









.

.


Interestingly, among the axioms “C(x) ≥ n”, some are more useful than others:

..

TheoremThere is a constant c and sequence of strings (xm) such that |xm| = m,C(xm) ≥m − c, and adding to PA all axioms “C(xm) ≥ m − c” proves all statements ofΠ1-Th(N).

..TheoremThere is a constant c and sequence of strings (xm) such that |xm| = m,C(xm) ≥m − c, and adding to PA all axioms “C(xm) ≥ m − c” proves all statements ofΠ1-Th(N).


On the other hand...

..

TheoremIf φ is a statement of Π1-Th(N) which is not provable in PA, it is possible toadd, for each length m, 99% of the true statements of type “C(y) ≥ m− c”, with|y| = m, and still be unable to prove φ.

..TheoremIf φ is a statement of Π1-Th(N) which is not provable in PA, it is possible toadd, for each length m, 99% of the true statements of type “C(y) ≥ m− c”, with|y| = m, and still be unable to prove φ.



.

.


Interestingly, among the axioms “C(x) ≥ n”, some are more useful than others:

..

TheoremThere is a constant c and sequence of strings (xm) such that |xm| = m,C(xm) ≥m − c, and adding to PA all axioms “C(xm) ≥ m − c” proves all statements ofΠ1-Th(N).



On the other hand...

..

TheoremIf φ is a statement of Π1-Th(N) which is not provable in PA, it is possible toadd, for each length m, 99% of the true statements of type “C(y) ≥ m− c”, with|y| = m, and still be unable to prove φ.




.

.


This raises the question:

..Open questionCharacterize the “useful” xn, i.e., those such that the collection “C(xn) ≥ n− c”is true and proves all of Π1-Th(N).



One last important point: axioms need to be precise to be useful.

..TheoremThere is no sequence (xn) such that C(xn) ≥ n − c and such that adding thecollection of axioms “C(xn) ≥ n/2 − c” proves all of Π1-Th(N).




.

.











.

.











.

.











.

.


Another example: consider for a given k the set CCk of true axioms of type

“C(x | y) ≥ k”

..TheoremFor any k large enough, the statements provable fromPA+CCk are exactly thoseof Π1-Th(N).



Proof. We show that being able to decide the truth of “C(x | y) ≥ k” allows usto decide the halting problem, and translate the proof into logic.


.

.



“C(x | y) ≥ k”






.

.



“C(x | y) ≥ k”






.

.


More interesting: the translation into logic does not necessarily go through!

..Theorem (BBFFGLMST)If one is given for each x a pair of values n1 and n2 such that C(x) = n1 orC(x) = n2, then one can compute ∅ ′.



But...

..

TheoremLetφ be any statement not provable in PA. Then it is possible to add for each xa true axiom of type “C(x) = n1 ∨ C(x) = n2” and still have a theory too weakto prove φ.

..TheoremLetφ be any statement not provable in PA. Then it is possible to add for each xa true axiom of type “C(x) = n1 ∨ C(x) = n2” and still have a theory too weakto prove φ.



.

.






But...

..





.

.






But...

..





.

.

..

3. Infinite binary sequences

.

.

Random sequences

Kolmogorov complexity measures the amount randomness for finite binarysequences. For infinite binary sequences, it even allows us to provide an exactdefinition of randomness.

A sequence X = x1x2x3 . . . is said to be (Martin-Löf) random with constant c if

K(x1x2 ... xn) ≥ n− c

3. Infinite binary sequences 23/31

.

.

Random sequences

Kolmogorov complexity measures the amount randomness for finite binarysequences. For infinite binary sequences, it even allows us to provide an exactdefinition of randomness.

A sequence X = x1x2x3 . . . is said to be (Martin-Löf) random with constant c if

K(x1x2 ... xn) ≥ n− c


.

.

Randomness of a sequence, axiomatized (1)

Some random sequences do compute ∅ ′, such as Chaitin’s Omega number.

Could it then be that expressing the randomness of some sequences wouldprove all of Π1-Th(N)?

Interestingly, it depends on the way we express the randomness of X.


.

.






.

.






.

.


Given a sequence X random with constant c, the obvious way is to take the setMLRc(X) of all axioms of type

K(x1x2 ... xn) ≥ n− c

In that case, we cannot prove all of Π1-Th(N):

..

Theorem• If MLRc(X) is consistent, adding it to PA is never sufficient to proveΠ1-Th(N).• Moreover, MLRc(X) is asymptotically useless: if φ is not provable in PA,then PA+MLRc(X) does not prove φ for any c large enough.

..Theorem• If MLRc(X) is consistent, adding it to PA is never sufficient to proveΠ1-Th(N).• Moreover, MLRc(X) is asymptotically useless: if φ is not provable in PA,then PA+MLRc(X) does not prove φ for any c large enough.



.

.


Given a sequence X random with constant c, the obvious way is to take the setMLRc(X) of all axioms of type

K(x1x2 ... xn) ≥ n− c

In that case, we cannot prove all of Π1-Th(N):

..

Theorem• If MLRc(X) is consistent, adding it to PA is never sufficient to proveΠ1-Th(N).• Moreover, MLRc(X) is asymptotically useless: if φ is not provable in PA,then PA+MLRc(X) does not prove φ for any c large enough.




.

.


A very rough sketch of the proof. Suppose for the sake of contradiction thatMLRc(X) is true and proves Π1-Th(N).

We use an interesting back-and-forth between logic and computability.• Note that X can compute the set of axioms MLRc(X), hence X can

compute Π1-Th(N), thus X ≥T ∅ ′.• Now, consider the set of complete coherent extensions of PA, coded as

infinite binary sequences.• This forms a Π0

1 class P , so by the basis theorem for randomness, there isA ∈ P such that X is random relative to A.


.

.



We use an interesting back-and-forth between logic and computability.

• Note that X can compute the set of axioms MLRc(X), hence X cancompute Π1-Th(N), thus X ≥T ∅ ′.

• Now, consider the set of complete coherent extensions of PA, coded asinfinite binary sequences.

• This forms a Π01 class P , so by the basis theorem for randomness, there is

A ∈ P such that X is random relative to A.


.

.




compute Π1-Th(N), thus X ≥T ∅ ′.

• Now, consider the set of complete coherent extensions of PA, coded asinfinite binary sequences.




.

.





infinite binary sequences.




.

.





infinite binary sequences.• This forms a Π0

1 class P , so by the basis theorem for randomness, there isA ∈ P such that X is random relative to A.


.

.


• Let T be the logical theory coded by A. Argue that X being A-randomimplies that X is seen as random in the theory T .

• Argue that therefore, in the theory T , the collection of axioms MLRc(X) is(almost) true.

• Thus the theory T proves Π1-Th(N).• Thus A ≥T ∅ ′.• Thus X is random relative to ∅ ′, a contradiction with X ≥T ∅ ′.


.

.






.

.






.

.




• Thus the theory T proves Π1-Th(N).

• Thus A ≥T ∅ ′.• Thus X is random relative to ∅ ′, a contradiction with X ≥T ∅ ′.


.

.




• Thus the theory T proves Π1-Th(N).• Thus A ≥T ∅ ′.

• Thus X is random relative to ∅ ′, a contradiction with X ≥T ∅ ′.


.

.






.

.


An alternative way: take the set MLR ′c(X) of all axioms of type

(∀N)(∃σ ≻ x1...xn) |σ| = N and K(σ) ≥ N− c

In this case, MLR ′c(X) may be powerful.

..

Theorem• There exists a consistent set of axioms MLR ′

c(X) such thatPA+MLR ′c(X)

proves all of Π1-Th(N) [take Chaitin’s Omega].• Still, MLR ′

c(X) is asymptotically useless as before.

..Theorem• There exists a consistent set of axioms MLR ′









.

.


An alternative way: take the set MLR ′c(X) of all axioms of type

(∀N)(∃σ ≻ x1...xn) |σ| = N and K(σ) ≥ N− c

In this case, MLR ′c(X) may be powerful.

..

Theorem• There exists a consistent set of axioms MLR ′













.

.

Non-random sequencess can help too!

We finish on this intriguing fact: while MLRc(X) is never very useful for arandom X, there is a non-random sequence Z whose complexity gives usefulinformation:

..

TheoremThere is a sequence Z = z1z2... and constant c such that K(z1 ... zn) ≥ n − cfor infinitely many n, and such that if we take these infinitely many statements asaxioms, we can prove all of Π1-Th(N).

..TheoremThere is a sequence Z = z1z2... and constant c such that K(z1 ... zn) ≥ n − cfor infinitely many n, and such that if we take these infinitely many statements asaxioms, we can prove all of Π1-Th(N).

..TheoremThere is a sequence Z = z1z2... and constant c such that K(z1 ... zn) ≥ n − cfor infinitely many n, and such that if we take these infinitely many statements asaxioms, we can prove all of Π1-Th(N).


.

.

Open questions




..Open questionWhat axioms about Kolmogorov should we look at if we want to say somethingabout Πn-Th(N)?




.

.

Open questions








.

.

Open questions








.

.Thank You !
