Post on 28-Mar-2020
transcript
Artificial Intelligence in Theorem Proving
Cezary Kaliszyk
VTSA
Overview
Last Lecturetheorem proving problems
premise selection
deep learning for theorem proving
state estimation
Today
automated reasoning
learning in classical ATPs
learning for tableaux
reinforcement learning in TP
longer proofs
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 2 / 72
What about ATPs
Proof by contradiction
Assume that the conjecture does not hold
Derive that axioms and negated conjecture imply ⊥
SaturationConvert problem to CNF
Enumerate the consequences of the available clauses
Goal: get to the empty clause
RedundanciesSimplify or eliminate some clauses (contract)
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 3 / 72
Calculus
Ordered
Resolution
Aσ strictly maximal wrt Cσ and B maximal wrt Dσ.
Equality axioms?
Ordered
Paramodulation
(s = t)σ and L[s ′]σ′ maximal in their clauses.
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 4 / 72
Calculus
Ordered Resolution
Aσ strictly maximal wrt Cσ and B maximal wrt Dσ.
Equality axioms?
Ordered
Paramodulation
(s = t)σ and L[s ′]σ′ maximal in their clauses.
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 4 / 72
Calculus
Ordered Resolution
Aσ strictly maximal wrt Cσ and B maximal wrt Dσ.
Equality axioms?
Ordered
Paramodulation
(s = t)σ and L[s ′]σ′ maximal in their clauses.
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 4 / 72
Calculus
Ordered Resolution
Aσ strictly maximal wrt Cσ and B maximal wrt Dσ.
Equality axioms?
Ordered Paramodulation
(s = t)σ and L[s ′]σ′ maximal in their clauses.
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 4 / 72
Calculus
Ordered Resolution
Aσ strictly maximal wrt Cσ and B maximal wrt Dσ.
Equality axioms?
Ordered Paramodulation
(s = t)σ and L[s ′]σ′ maximal in their clauses.
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 4 / 72
Completion
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 5 / 72
Superposition Calculus
Basis ofE, Vampire, Spass, Prover9, ≈Metis
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 6 / 72
Beyond the Calculus
Tautology Deletion
a ∨ b ∨ ¬a ∨ d
Subsumption (forward and backward)e.g. E uses Feature Vector Indexing
{C1, C2, C3, C4}
{C3, C4}0
{C2}1
{C1}
2
{C3}0
{C4}2
{C2}0
{C1}1
{C3}0
{C4}0
{C2}1
{C1}0
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 7 / 72
Still...
fof(6, axiom,![X1]:![X2]:![X4]:gg(X1,sup_sup(X1,X2,X4)),file(’i/f/1/goal_138__Q_Restricted_Rewriting.qrstepfof(32, axiom,![X1]:![X2]:gg(set(product_prod(X1,X1)),transitive_rtrancl(X1,X2)),file(’i/f/1/goal_138__Q_Refof(55, axiom,![X1]:![X19]:![X20]:(member(product_prod(X1,X1),X19,X20)=>member(product_prod(X1,X1),X19,tranfof(68, axiom,![X1]:![X5]:![X3]:![X36]:![X20]:![X37]:![X16]:(ord_less_eq(set(product_prod(X1,X3)),X36,X20)=fof(70, axiom,![X1]:![X20]:transitive_rtrancl(X1,transitive_rtrancl(X1,X20))=transitive_rtrancl(X1,X20),filfof(74, axiom,![X1]:![X24]:![X34]:![X33]:((~(member(X1,X24,X34))=>member(X1,X24,X33))=>member(X1,X24,sup_sufof(78, axiom,![X1]:![X11]:![X13]:transitive_rtrancl(X1,sup_sup(set(product_prod(X1,X1)),transitive_rtranclfof(79, axiom,![X1]:![X22]:![X39]:(member(X1,X22,collect(X1,X39))<=>pp(aa(X1,bool,X39,X22))),file(’i/f/1/gofof(85, axiom,![X1]:(semilattice_sup(X1)=>![X23]:![X24]:![X22]:(ord_less_eq(X1,sup_sup(X1,X23,X24),X22)<=>(fof(86, axiom,![X1]:![X11]:relcomp(X1,X1,X1,transitive_rtrancl(X1,X11),transitive_rtrancl(X1,X11))=transitifof(98, axiom,![X1]:![X33]:![X34]:(gg(set(X1),X34)=>(ord_less_eq(set(X1),X33,X34)<=>sup_sup(set(X1),X33,X34fof(99, axiom,![X1]:![X33]:![X34]:ord_less_eq(set(X1),X33,sup_sup(set(X1),X33,X34)),file(’i/f/1/goal_138__Qfof(100, axiom,![X3]:![X1]:supteq(X1,X3)=sup_sup(set(product_prod(term(X1,X3),term(X1,X3))),supt(X1,X3),id(fof(102, axiom,![X1]:![X34]:![X33]:ord_less_eq(set(X1),X34,sup_sup(set(X1),X33,X34)),file(’i/f/1/goal_138__fof(103, axiom,![X1]:![X33]:![X18]:![X34]:(ord_less_eq(set(X1),X33,X18)=>(ord_less_eq(set(X1),X34,X18)=>ordfof(109, axiom,![X1]:![X34]:![X33]:(gg(set(X1),X33)=>(ord_less_eq(set(X1),X34,X33)=>sup_sup(set(X1),X33,X34fof(114, axiom,![X1]:![X33]:![X18]:![X34]:![X48]:(ord_less_eq(set(X1),X33,X18)=>(ord_less_eq(set(X1),X34,X4fof(116, axiom,![X1]:![X33]:ord_less_eq(set(X1),X33,X33),file(’i/f/1/goal_138__Q_Restricted_Rewriting.qrstefof(125, axiom,![X1]:![X24]:![X33]:![X34]:(member(X1,X24,X33)=>(~(member(X1,X24,X34))=>member(X1,X24,minus_fof(127, axiom,![X1]:![X24]:![X33]:![X34]:(member(X1,X24,minus_minus(set(X1),X33,X34))=>~((member(X1,X24,X3fof(131, axiom,![X1]:![X33]:(gg(set(X1),X33)=>collect(X1,aTP_Lamp_a(set(X1),fun(X1,bool),X33))=X33),file(’ifof(134, axiom,![X1]:(order(X1)=>![X35]:![X49]:((gg(X1,X35)&gg(X1,X49))=>(ord_less_eq(X1,X35,X49)=>(ord_lesfof(136, axiom,![X1]:(preorder(X1)=>![X35]:![X49]:![X50]:(ord_less_eq(X1,X35,X49)=>(ord_less_eq(X1,X49,X50)fof(143, axiom,![X1]:![X33]:![X34]:(ord_less_eq(set(X1),X33,X34)<=>![X52]:(gg(X1,X52)=>(member(X1,X52,X33)=fof(160, axiom,![X1]:![X39]:![X35]:![X33]:(pp(aa(X1,bool,X39,X35))=>(member(X1,X35,X33)=>?[X30]:(gg(X1,X30)fof(171, axiom,![X1]:![X65]:![X66]:(pp(aa(X1,bool,aTP_Lamp_a(set(X1),fun(X1,bool),X65),X66))<=>member(X1,X6fof(186, axiom,![X67]:semilattice_sup(set(X67)),file(’i/f/1/goal_138__Q_Restricted_Rewriting.qrsteps_comp_sfof(187, axiom,![X67]:preorder(set(X67)),file(’i/f/1/goal_138__Q_Restricted_Rewriting.qrsteps_comp_supteq_sfof(188, axiom,![X67]:order(set(X67)),file(’i/f/1/goal_138__Q_Restricted_Rewriting.qrsteps_comp_supteq_subsfof(207, conjecture,ord_less_eq(set(product_prod(term(a,b),term(a,b))),relcomp(term(a,b),term(a,b),term(a,b
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 8 / 72
Still the search space is huge: What can we learn?
What has been learnedCASC: Strategies
AIM: Hints
Hammers: Premises
What can be chosen in Superposition calculus
Term ordering
(Negative) literal selection
Clause selection
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 9 / 72
E-Prover given-clause loop
Most important choice: unprocessed clause selection [Schulz 2015 ]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 10 / 72
Learning for E: Data Collection
Mizar top-level theorems [Urban 2006 ]
Encoded in FOF
32,521 Mizar theorems with ≥ 1 proof
training-validation split (90%-10%)
replay with one strategy
Collect all CNF intermediate steps
and unprocessed clauses when proof is found
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 11 / 72
Deep Network Architectures
Clause EmbedderNegated conjecture
embedder
Concatenate
Fully Connected(1024 nodes)
Fully Connected(1 node)
Logistic loss
Clause tokens Negated conjecture tokens
Conv 5 (1024) + ReLU
Input token embeddings
Conv 5 (1024) + ReLU
Conv 5 (1024) + ReLU
Max Pooling
Overall network Convolutional Embedding
Non-dilated and dilated convolutionsCezary Kaliszyk Artificial Intelligence in Theorem Proving 12 / 72
Recursive Neural Networks
Curried representation of first-order statements
Separate nodes for apply, or, and, not
Layer weights learned jointly for the same formula
Embeddings of symbols learned with rest of network
Tree-RNN and Tree-LSTM models1
1Relation to graphsCezary Kaliszyk Artificial Intelligence in Theorem Proving 13 / 72
Model accuracy
Model Embedding Size Accuracy: 50-50% splitTree-RNN-256×2 256 77.5%Tree-RNN-512×1 256 78.1%
Tree-LSTM-256×2 256 77.0%Tree-LSTM-256×3 256 77.0%Tree-LSTM-512×2 256 77.9%
CNN-1024×3 256 80.3%?CNN-1024×3 256 78.7%
CNN-1024×3 512 79.7%CNN-1024×3 1024 79.8%
WaveNet-256×3×7 256 79.9%?WaveNet-256×3×7 256 79.9%WaveNet-1024×3×7 1024 81.0%
WaveNet-640×3×7(20%) 640 81.5%?WaveNet-640×3×7(20%) 640 79.9%
? = train on unprocessed clauses as negative examples
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 14 / 72
Improving Proof Search inside E
Overview
Processed ClausesUnprocessed
Clauses
Select oneUsing a deep neural
network
Superposition
ProblemDeep neural network evaluation is slow
Slower than combining selected clause with all processed clauses2
2State of 2016Cezary Kaliszyk Artificial Intelligence in Theorem Proving 15 / 72
Hybrid heuristic
Optimizations for performance
Batching
Combining TF with auto
102 103 104 105
Processed clause limit
0%
20%
40%
60%
80%
100%
Perc
ent
unpro
ved
Pure CNN
Hybrid CNN
Pure CNN; Auto
Hyrbid CNN; Auto
102 103 104 105
Processed clause limit
0%
20%
40%
60%
80%
100%
Perc
ent
unpro
ved
Auto
WaveNet 640*
WaveNet 256
WaveNet 256*
WaveNet 640
CNN
CNN*
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 16 / 72
Harder Mizar top-level statements
Model DeepMath 1 DeepMath 2 Union of 1 and 2
Auto 578 581 674?WaveNet 640 644 612 767?WaveNet 256 692 712 864
WaveNet 640 629 685 997?CNN 905 812 1,057
CNN 839 935 1,101
Total (unique) 1,451 1,458 1,712
Overall proved 7.4% of the harder statements
Batching and hybrid necessary
Model accuracy unsatisfactory
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 17 / 72
Harder Mizar top-level statements
Model DeepMath 1 DeepMath 2 Union of 1 and 2
Auto 578 581 674?WaveNet 640 644 612 767?WaveNet 256 692 712 864
WaveNet 640 629 685 997?CNN 905 812 1,057
CNN 839 935 1,101
Total (unique) 1,451 1,458 1,712
Overall proved 7.4% of the harder statements
Batching and hybrid necessary
Model accuracy unsatisfactory
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 17 / 72
ENIGMA [Jakubuv,Urban 2017 ]
Evaluation on AIM
E’s auto-schedule: 261
Single best strategy: 239
Different trained models: 337
Accuracy: 97.6%
Looping and boosting
Still in 30s: best trained strategy: 318
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 18 / 72
ENIGMA [Jakubuv,Urban 2017 ]
Evaluation on AIM
E’s auto-schedule: 261
Single best strategy: 239
Different trained models: 337
Accuracy: 97.6%
Looping and boosting
Still in 30s: best trained strategy: 318
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 18 / 72
ENIGMA [Jakubuv,Urban 2017 ]
Evaluation on AIM
E’s auto-schedule: 261
Single best strategy: 239
Different trained models: 337
Accuracy: 97.6%
Looping and boosting
Still in 30s: best trained strategy: 318
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 18 / 72
ENIGMA [Jakubuv,Urban 2017 ]
Evaluation on AIM
E’s auto-schedule: 261
Single best strategy: 239
Different trained models: 337
Accuracy: 97.6%
Looping and boosting
Still in 30s: best trained strategy: 318
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 18 / 72
Automated Theorem Proving
Historical dispute: Gentzen and Hilbert
Today two communities: Resolution (-style) and Tableaux
Possible answer: What is better in practice?
Say the CASC competition or ITP libraries?
Since the late 90s: resolution (superposition)
But still so far from humans?We can do learning much better for Tableaux
And with ML beating brute force search in games, maybe?
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 19 / 72
leanCoP: Lean Connection Prover [Otten 2010 ]
Connected tableaux calculusGoal oriented, good for large theories
Regularly beats Metis and Prover9 in CASC (ATP Systems Competition)
despite their much larger implementation
Compact Prolog implementation, easy to modify
Variants for other foundations: iLeanCoP, mLeanCoP
First experiments with machine learning: MaLeCoP
Easy to imitate
leanCoP tactic in HOL Light
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 20 / 72
Lean connection Tableaux
Very simple rules:
Extension unifies the current literal with a copy of a clause
Reduction unifies the current literal with a literal on the path
{}, M, PathAxiom
C , M, Path ∪ {L2}C ∪ {L1}, M, Path ∪ {L2}
Reduction
C2 \ {L2}, M, Path ∪ {L1} C , M, Path
C ∪ {L1}, M, PathExtension
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 21 / 72
Example lean connection proof
Clauses:
c1 : P(x)
c2 : R(x , y) ∨ ¬P(x) ∨ Q(y)
c3 : S(x) ∨ ¬Q(b)
c4 : ¬S(x) ∨ ¬Q(x)
c5 : ¬Q(x) ∨ ¬R(a, x)
c6 : ¬R(a, x) ∨ Q(x)
Tableau: P(a)
R(a, b)
¬R(a, b) Q(b)
¬Q(b) ¬R(a, b)
¬P(a) Q(b)
S(b)
¬S(b) ¬Q(b)
¬Q(b)
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 22 / 72
leanCoP Example [Otten’15 ]
Formula to prove:
DNF:
Matrix:
Tableaux:
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 23 / 72
leanCoP: Basic Code
1 prove([Lit|Cla],Path ,PathLim ,Lem ,Set) :-
23 (-NegLit=Lit;-Lit=NegLit) ->
4 (
567 member(NegL ,Path),unify_with_occurs_check(NegL ,NegLit)
8 ;
9 lit(NegLit ,NegL ,Cla1 ,Grnd1),
10 unify_with_occurs_check(NegL ,NegLit),
11121314 prove(Cla1 ,[Lit|Path],PathLim ,Lem ,Set)
15 ),
1617 prove(Cla ,Path ,PathLim ,Lem ,Set).
18 prove([],_,_,_,_).
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 24 / 72
leanCoP: Actual Code (Optimizations, No history)
1 prove([Lit|Cla],Path ,PathLim ,Lem ,Set) :-
2 \+ (member(LitC ,[Lit|Cla]), member(LitP ,Path),LitC==LitP),
3 (-NegLit=Lit;-Lit=NegLit) ->
4 (
5 member(LitL ,Lem), Lit==LitL
6 ;
7 member(NegL ,Path),unify_with_occurs_check(NegL ,NegLit)
8 ;
9 lit(NegLit ,NegL ,Cla1 ,Grnd1),
10 unify_with_occurs_check(NegL ,NegLit),
11 ( Grnd1=g -> true ;
12 length(Path ,K), K<PathLim -> true ;
13 \+ pathlim -> assert(pathlim), fail ),
14 prove(Cla1 ,[Lit|Path],PathLim ,Lem ,Set)
15 ),
16 ( member(cut ,Set) -> ! ; true ),
17 prove(Cla ,Path ,PathLim ,[Lit|Lem],Set).
18 prove([],_,_,_,_).
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 25 / 72
First experiment: MaLeCoP in Prolog [Tableaux 2011 ]
Select extension stepsUsing external advice
Slow implementation
1000 less inf per second
Can avoid 90% inferences!
Important: Strategies
leanCoP
cache
advisor
Otherprovers
SNoWlearningsystem
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 26 / 72
What about efficiency: FEMaLeCoP [LPAR 2015 ]
Advise the:selection of clause for every tableau extension step
Proof state: weighted vector of symbols (or terms)
extracted from all the literals on the active path
Frequency-based weighting (IDF)
Simple decay factor (using maximum)
Consistent clausificationformula ?[X]: p(X) becomes p(’skolem(?[A]:p(A),1)’)
Predictor: Custom sparse naive Bayes
association of the features of the proof states
with contrapositives used for the successful extension steps
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 27 / 72
FEMaLeCoP: Data Collection and Indexing
Extension of the saved proofs
Training Data: pairs (path, used extension step)
External Data Indexing (incremental)
te num: number of training examples
pf no: map from features to number of occurrences ∈ Qcn no: map from contrapositives to numbers of occurrences
cn pf no: map of maps of cn/pf co-occurrences
Problem Specific Data
Upon start FEMaLeCoP reads
only current-problem relevant parts of the training data
cn no and cn pf no filtered by contrapositives in the problem
pf no and cn pf no filtered by possible features in the problem
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 28 / 72
Efficient Relevance (1/2)
Estimate the relevance of each contrapositive ϕ by
P(ϕ is used in a proof in state ψ | ψ has features F(γ))
where F (γ) are the features of the current path.
Assuming the features are independent, this is:
P(ϕ is used in ψ’s proof)
·∏
f∈F (γ)∩F (ϕ)P(ψ has feature f | ϕ is used in ψ’s proof
)·∏
f∈F (γ)−F (ϕ)P(ψ has feature f | ϕ is not used in ψ’s proof
)·∏
f∈F (ϕ)−F (γ)P(ψ does not have f | ϕ is used in ψ’s proof
)
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 29 / 72
Efficient Relevance (1/2)
Estimate the relevance of each contrapositive ϕ by
P(ϕ is used in a proof in state ψ | ψ has features F(γ))
where F (γ) are the features of the current path.
Assuming the features are independent, this is:
P(ϕ is used in ψ’s proof)
·∏
f∈F (γ)∩F (ϕ)P(ψ has feature f | ϕ is used in ψ’s proof
)·∏
f∈F (γ)−F (ϕ)P(ψ has feature f | ϕ is not used in ψ’s proof
)·∏
f∈F (ϕ)−F (γ)P(ψ does not have f | ϕ is used in ψ’s proof
)
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 29 / 72
Efficient Relevance (2/2)
All these probabilities can be estimated (using training examples):
σ1 ln t +∑
f∈(f∩s)
i(f ) lnσ2s(f )
t+ σ3
∑f∈(f−s)
i(f ) + σ4
∑f∈(s−f )
i(f ) ln(1−s(f )
t)
where
f are the features of the path
s are the features that co-occurred with ϕ
t = cn no(ϕ)
s = cn fp no(ϕ)
i is the IDF
σ∗ are experimentally chosen parameters
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 30 / 72
Inference speed ... drops to about 40%
Prover Proved (%)
OCaml-leanCoP 574 (27.6%)FEMaLeCoP 635 (30.6%)together 664 (32.0%)
(evaluation on MPTP bushy problems, 60 s)
On various datasets, 3–15% problems more solved than leanCoP(run for double the time)
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 31 / 72
What about stronger learning?
Yes, but... [Michalewski 2017 ]
If put directly, huge times needed
Still improvement small
NBayes vs XGBoost on 2h timeout
Preliminary experiments with deep learning [Olsak 2017 ]
So far too slow
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 32 / 72
Is theorem proving just a maze search?
Yes and NO!The proof search tree is not the same as the tableau tree!
Unification can cause other branches to disappear.
Can we provide a tree search like interface?
Two functions sufficestart : problem→ state
action : action→ state
wherestate = 〈action list× remaining goal-paths〉
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 33 / 72
Is theorem proving just a maze search?
Yes and NO!The proof search tree is not the same as the tableau tree!
Unification can cause other branches to disappear.
Can we provide a tree search like interface?
Two functions sufficestart : problem→ state
action : action→ state
wherestate = 〈action list× remaining goal-paths〉
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 33 / 72
Is it ok to change the tree?
Most learning for games sticks to game dynamics
Only tell it how to do the moves
Why is it ok to skip other branches
Theoretically ATP calculi are complete
Practically most ATP strategies incomplete
In usual 30s – 300s runsDepth of proofs with backtracking: 5–7 (complete)
Depth with restricted backtracking: 7–10 (more proofs found!)
But with random playouts: depth hundreds of thousands!
Just unlikely to find a proof → learning
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 34 / 72
Monte Carlo First Try: monteCoP
Use Monte Carlo playouts to guide restricted backtracking
Improves on leanCoP, but not by a big margin
Potential still limited by depth
0 20 40 60 80 100 120 140
450
460
470
480
490
500
maxIterations
Pro
ble
ms
solv
ed
0 20 40 60 80 100 120 140100
150
200
250
300
350
smax
Pro
ble
ms
solv
ed
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 35 / 72
“Simple” learning in leanCoP
FEMaLeCoP: Speed: 40%On various datasets, 3–15% problems more solved than leanCoP
XGBoost: Speed: 8%But more precise and again small improvement
Monte CarloImproves on leanCoP, but not by a big margin
Change in game moves
More inspiration from games?
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 36 / 72
AlphaZero (1/3) [Silver et al.]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 37 / 72
AlphaZero (2/3) [Silver et al.]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 38 / 72
AlphaZero (3/3) [Silver et al.]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 39 / 72
How to select the best action? [Szepesvari 2006 ]
IntuitionGiven some prior probabilities
And having done some experiments
Which action to take?
(later extended to sequences of actions in a tree)
Monte Carlo Tree Search with Upper Confidence Bounds for Trees
Select node n maximizingwi
ni+ c · pi ·
√lnN
ni
wherewi
niaverage reward pi action i prior
N number of experiments ni action i experiments
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 40 / 72
How to select the best action? [Szepesvari 2006 ]
IntuitionGiven some prior probabilities
And having done some experiments
Which action to take?
(later extended to sequences of actions in a tree)
Monte Carlo Tree Search with Upper Confidence Bounds for Trees
Select node n maximizingwi
ni+ c · pi ·
√lnN
ni
where
wi
niaverage reward pi action i prior
N number of experiments ni action i experiments
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 40 / 72
How to select the best action? [Szepesvari 2006 ]
IntuitionGiven some prior probabilities
And having done some experiments
Which action to take?
(later extended to sequences of actions in a tree)
Monte Carlo Tree Search with Upper Confidence Bounds for Trees
Select node n maximizingwi
ni+ c · pi ·
√lnN
ni
wherewi
niaverage reward pi action i prior
N number of experiments ni action i experiments
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 40 / 72
MCTS tree for WAYBEL 0:28
r=0.3099n=1182
Tableaux starting axiom
p=0.24r=0.3501n=548
p=0.21r=0.1859n=28...
p=0.10r=0.2038
n=9...
p=0.13r=0.2110n=14...
p=0.14r=0.2384n=21...
p=0.14r=0.3370n=181...
p=0.20r=0.3967n=279
p=0.30r=0.1368n=14...
p=0.15r=0.0288
n=2...
p=0.56r=0.4135n=262
p=0.66r=0.4217n=247
36 more MCTS tree levels until proved
Subset(c2, powerset(carrier(c1))p=0.18r=0.2633
n=8...
p=0.17r=0.2554
n=6...
Subset(union(c2),carrier(c1))
upper(c1)p=0.08r=0.1116
n=3...
RelStr(c1)p=0.19r=0.2289n=58...
p=0.22r=0.1783n=40...
p=0.35r=0.2889n=536...
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 41 / 72
Learn Policy and Value
Policy: Which actions to take?
Proportions predicted based on proportions in similar states
Explore less the actions that were “bad” in the past
Explore more and earlier the actions that were “good”
Value: How good (close to a proof) is a state?
Reward states that have few goals
Reward easy goals
Where to get training data?
Explore 1000 nodes using UCT
Select the most visited action and focus on it for this proof
A sequence of selected actions can train both policy and value
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 42 / 72
Learn Policy and Value
Policy: Which actions to take?
Proportions predicted based on proportions in similar states
Explore less the actions that were “bad” in the past
Explore more and earlier the actions that were “good”
Value: How good (close to a proof) is a state?
Reward states that have few goals
Reward easy goals
Where to get training data?
Explore 1000 nodes using UCT
Select the most visited action and focus on it for this proof
A sequence of selected actions can train both policy and value
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 42 / 72
Learn Policy and Value
Policy: Which actions to take?
Proportions predicted based on proportions in similar states
Explore less the actions that were “bad” in the past
Explore more and earlier the actions that were “good”
Value: How good (close to a proof) is a state?
Reward states that have few goals
Reward easy goals
Where to get training data?
Explore 1000 nodes using UCT
Select the most visited action and focus on it for this proof
A sequence of selected actions can train both policy and value
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 42 / 72
Mizar TPTP problems: train (29272) and test (3252) sets
BaselineSystem leanCoP playouts UCT
Train 10438 4184 7348
Test 1143 431 804
10 iterations
Iteration 1 2 3 4 5
6 7 8 9 10
Train 12325 13749 14155 14363 14403 14431 14342 14498 14481 14487
Test 1354 1519 1566 1595 1624
1586 1582 1591 1577 1621
More TimeleanCoP, 4M inferences, strategies 1396
rlCoP union 1839
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 43 / 72
Mizar TPTP problems: train (29272) and test (3252) sets
BaselineSystem leanCoP playouts UCT
Train 10438 4184 7348
Test 1143 431 804
10 iterations
Iteration 1 2 3 4 5
6 7 8 9 10
Train 12325 13749 14155 14363 14403 14431 14342 14498 14481 14487
Test 1354 1519 1566 1595 1624
1586 1582 1591 1577 1621
More TimeleanCoP, 4M inferences, strategies 1396
rlCoP union 1839
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 43 / 72
Mizar TPTP problems: train (29272) and test (3252) sets
BaselineSystem leanCoP playouts UCTTrain 10438 4184 7348Test 1143 431 804
10 iterations
Iteration 1 2 3 4 5 6 7 8 9 10
Train 12325 13749 14155 14363 14403 14431 14342 14498 14481 14487Test 1354 1519 1566 1595 1624 1586 1582 1591 1577 1621
More TimeleanCoP, 4M inferences, strategies 1396
rlCoP union 1839
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 43 / 72
Mizar TPTP problems: train (29272) and test (3252) sets
BaselineSystem leanCoP playouts UCTTrain 10438 4184 7348Test 1143 431 804
10 iterations
Iteration 1 2 3 4 5 6 7 8 9 10
Train 12325 13749 14155 14363 14403 14431 14342 14498 14481 14487Test 1354 1519 1566 1595 1624 1586 1582 1591 1577 1621
More TimeleanCoP, 4M inferences, strategies 1396
rlCoP union 1839
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 43 / 72
n0
p=1.00r=0.3944v=1386
p=0.08r=0.0621
v=3
p=0.08r=0.1048
v=5
p=0.08r=0.0544
v=3
p=0.09r=0.0712
v=4
p=0.08r=0.0443
v=3
p=0.26r=0.4016v=1356
p=0.07r=0.0504
v=2
p=0.07r=0.0628
v=3
p=0.08r=0.0544
v=3
p=0.08r=0.0660
v=3
p=0.38r=0.0518
v=1
p=0.47r=0.0774
v=1p=0.14
..58.. ..61..
p=0.37r=0.0942
v=2
p=0.47r=0.1391
v=2p=0.16
..57..p=0.03
r=0.0615v=1
..58..
p=0.02r=0.1099
v=1..60..
..62..
p=0.29r=0.0448
v=1
p=0.36r=0.0613
v=1..2..
..58.. ..61..
p=0.75r=0.0367
v=3p=0.25
p=0.02r=0.0214
v=2..60..
p=0.03 ..61..
p=0.27p=0.73
r=0.0353v=2
p=0.03r=0.0212
v=1..60..
..62..
p=0.68r=0.4067v=1322
p=0.32r=0.2054
v=33
p=0.31r=0.1136
v=28
p=0.09r=0.0297
v=2
p=0.16r=0.4148v=1288
p=0.11r=0.0595
v=3..5..
p=0.52r=0.1158
v=27..6..
p=0.89r=0.1145
v=26p=0.11
p=0.09p=0.91
r=0.1138v=25
p=0.91r=0.1131
v=24p=0.09
p=0.10p=0.90
r=0.1109v=23
p=0.88r=0.1086
v=22p=0.12
p=0.89r=0.1055
v=21p=0.11
p=0.89r=0.1021
v=20p=0.11
p=0.90r=0.0959
v=19p=0.10
p=0.11p=1.00
r=0.0838v=17
STEPS:2
p=0.07r=0.0144
v=1
p=0.50r=0.0703
v=10
p=0.23r=0.0250
v=2
p=0.21r=0.1899
v=3
..4..p=0.64
r=0.0922v=7
p=0.36r=0.0109
v=2
p=0.58p=0.42
r=0.1138v=5
..8..p=0.32
r=0.1340v=3
p=0.32r=0.1593
v=1
p=0.57r=0.1177
v=2p=0.43
p=0.38 p=0.62
..2..
..8..p=0.06
r=0.0121v=1
..7..
..3..p=0.04
r=0.0097v=1
..4..
p=0.56r=0.1736
v=2p=0.44
p=0.57 p=0.43
p=0.23r=0.0347
v=1..2..
..6..
..6..p=0.05
r=0.0376v=1
p=0.45r=0.4158v=1284
p=0.10r=0.0641
v=2
..7..p=0.13
r=0.4095v=32
p=0.87r=0.4161v=1251
p=0.09r=0.0166
v=1
p=0.91r=0.4329
v=30
..2.. ..2..p=0.09
r=0.5188v=1
p=0.71r=0.4312
v=28
..3.. ..2..p=0.80
r=0.4263v=27
p=0.10p=0.90
r=0.4194v=26
p=0.90r=0.4119
v=25p=0.10
p=0.10p=0.90
r=0.4029v=24
p=0.14p=0.86
r=0.3925v=23
p=0.87r=0.3791
v=22p=0.13
p=0.13p=0.87
r=0.3636v=21
p=0.88r=0.3429
v=20p=0.12
p=0.13p=0.87
r=0.3194v=19
p=0.27r=0.1444
v=2
p=0.73r=0.3383
v=16
p=0.50 p=0.50p=0.38
r=0.3240v=15
..8..
..6..p=0.08
r=0.4829v=1
p=0.50r=0.3151
v=13
..9.. p=0.14p=1.00
r=0.3028v=11
STEPS:2
p=0.19p=0.05
r=0.2163v=1
p=0.46r=0.3041
v=6
p=0.30r=0.3181
v=3
..4..p=0.69
r=0.2876v=5
..2..
p=0.18p=0.82
r=0.2565v=4
p=0.30r=0.3362
v=1
p=0.70r=0.1728
v=2
..58.. ..2..p=0.11
r=0.1136v=1
..58..
p=0.38p=0.62
r=0.2401v=2
p=0.62 p=0.38
p=0.10r=0.1064
v=2
p=0.90r=0.4168v=1248
..3..p=0.07
r=0.0220v=1
..4..
p=0.12r=0.0475
v=2
p=0.88r=0.4175v=1245
p=0.92p=0.08
r=0.0132v=1
..2..
p=0.10r=0.0883
v=2
p=0.90r=0.4182v=1242
p=0.05r=0.0185
v=1..3..
..4..
p=0.85r=0.4193v=1217
p=0.15r=0.3694
v=24
p=0.14r=0.3774
v=23
p=0.86r=0.4202v=1193
p=0.09r=0.0128
v=1
p=0.91r=0.4059
v=21
..2..p=0.75
r=0.4018v=20
..2..
p=0.75r=0.3887
v=19..2..
p=0.13p=0.87
r=0.3712v=18
p=0.12p=0.88
r=0.3472v=17
p=0.87r=0.3197
v=16p=0.13
p=0.27r=0.1444
v=2
p=0.73r=0.3430
v=13
p=0.50 p=0.50p=0.38
r=0.3255v=12
..8..
..6..p=0.08
r=0.4829v=1
p=0.50r=0.3141
v=10
..9..p=1.00
r=0.2971v=8
STEPS:2
p=0.14
p=0.05r=0.2163
v=1
p=0.46r=0.3041
v=6..2..
..4.. ..2..p=0.69
r=0.2876v=5
p=0.82r=0.2565
v=4p=0.18
p=0.30r=0.3362
v=1
p=0.70r=0.1728
v=2
..58..p=0.11
r=0.1136v=1
..2..
..58..
p=0.85r=0.4211v=1171
p=0.15r=0.3782
v=21
p=0.88r=0.4218v=1159
p=0.12r=0.3576
v=11
p=0.87r=0.4227v=1148
p=0.13r=0.3300
v=10
p=0.72r=0.4248v=1010
p=0.28r=0.4098
v=137
p=0.37r=0.3275
v=180
p=0.19r=0.4483
v=822
p=0.12r=0.1899
v=7..6..
..6..p=0.09
r=0.2099v=2
p=0.49r=0.3301
v=177
..8..p=0.07
r=0.1942v=1
..8..
p=1.00r=0.3316
v=173
STEPS:2
p=0.14r=0.1451
v=2
p=0.05r=0.1507
v=1
p=0.46r=0.3531
v=138
p=0.19r=0.2429
v=10
p=0.30r=0.2492
v=23
..4..p=0.70
r=0.3574v=134
p=0.16r=0.1781
v=3p=0.14
p=0.82r=0.3605
v=130
p=0.18r=0.2057
v=3
p=0.30r=0.1158
v=7
p=0.70r=0.3747
v=122
p=0.03r=0.0797
v=6..57..
..56.. p=0.03 p=1.00
STEPS:5
p=0.11r=0.1120
v=1
p=0.84r=0.3781
v=120p=0.05
..58..p=0.83
r=0.3776v=119
..2..
p=0.90r=0.3774
v=118p=0.10
p=0.90r=0.3764
v=117p=0.10
p=0.10p=0.90
r=0.3755v=116
p=0.09p=0.91
r=0.3743v=115
p=1.00r=0.3727
v=113
STEPS:2
p=0.10
p=0.05r=0.2690
v=1
p=0.46r=0.3990
v=87
p=0.22r=0.2926
v=10
p=0.27r=0.2758
v=14
..4.. p=0.13p=0.70
r=0.4046v=84
p=0.18r=0.1837
v=2
p=0.17p=0.83
r=0.4039v=83
p=0.31r=0.2517
v=7
p=0.69r=0.4182
v=75
p=0.03r=0.2353
v=6..57..
p=0.03p=0.04
r=0.2789v=4
..56..
p=0.02r=0.2712
v=3..60..
..56.. p=0.02 p=1.00
STEPS:2
p=0.11r=0.2201
v=1
p=0.84r=0.4230
v=73p=0.05
..58..p=0.86
r=0.4221v=72
..2..
p=0.90r=0.4216
v=71p=0.10
p=0.90r=0.4206
v=70p=0.10
p=0.90r=0.4196
v=69p=0.10
p=0.90r=0.4179
v=68p=0.10
p=0.10p=0.90
r=0.4161v=67
p=0.90r=0.4157
v=66p=0.10
p=0.10p=0.90
r=0.4151v=65
p=0.16p=1.00
r=0.4137v=63
STEPS:2
p=0.07r=0.3008
v=1
p=0.54r=0.4289
v=49
p=0.20r=0.4200
v=7
p=0.19r=0.2850
v=5
..4..p=0.72
r=0.4296v=48
..2..
p=0.83r=0.4283
v=47p=0.17
p=0.31r=0.3160
v=5
p=0.69r=0.4420
v=41
p=0.03r=0.2981
v=4..57..
p=0.03p=0.04
r=0.4316v=2
..56..
p=0.02r=0.5279
v=1..60..
..58..
p=0.05p=0.11
r=0.2474v=1
p=0.84r=0.4506
v=39
..58.. ..2..p=0.86
r=0.4487v=38
p=0.10p=0.90
r=0.4465v=37
p=0.90r=0.4442
v=36p=0.10
p=0.10p=0.90
r=0.4404v=35
p=0.90r=0.4364
v=34p=0.10
p=0.10p=1.00
r=0.4308v=32
STEPS:2
p=0.05r=0.3183
v=1
p=0.46r=0.4650
v=24
p=0.22r=0.3632
v=1
p=0.27r=0.3042
v=5
..4.. ..2..p=0.75
r=0.4671v=23
p=0.18p=0.82
r=0.4653v=22
p=0.29r=0.2719
v=3
p=0.71r=0.4984
v=18
..57..p=0.03
r=0.1886v=2
p=0.02 ..57..
p=0.05p=0.11
r=0.2887v=1
p=0.84r=0.5227
v=16
..58.. ..2..p=0.86
r=0.5215v=15
p=0.90r=0.5185
v=14p=0.10
p=0.10p=0.90
r=0.5161v=13
p=0.10p=0.90
r=0.5126v=12
p=0.90r=0.5043
v=11p=0.10
p=0.10p=0.90
r=0.4943v=10
p=0.91r=0.4919
v=9p=0.09
p=0.10p=1.00
r=0.4719v=7
STEPS:2
..2..p=0.06
r=0.4340v=1
p=0.52r=0.4765
v=5
..4.. ..2..p=0.71
r=0.4841v=4
p=0.20p=0.80
r=0.4556v=3
p=0.30r=0.4804
v=1
p=0.70r=0.3744
v=1
..58.. ..3..
..4..p=0.55
r=0.2420v=4
p=0.45
p=0.55p=0.45
r=0.2074v=2
..6.. p=0.12
p=0.11p=0.04
r=0.4998v=1
p=0.43r=0.4249
v=4
p=0.42r=0.3528
v=1
..4..p=0.91
r=0.4022v=3
p=0.09
p=0.04r=0.3513
v=1
p=0.43r=0.3622
v=1..2..
..4.. ..2..
..2..
p=0.39p=0.61
r=0.2249v=4
p=0.61p=0.39
r=0.1872v=2
p=0.13 ..6..
p=0.55 p=0.45
..2..p=0.05
r=0.4794v=1
p=0.53r=0.2601
v=8
..4.. p=0.09p=0.91
r=0.2298v=7
p=0.05r=0.2465
v=1
p=0.54r=0.1778
v=5..2..
..4..p=0.90
r=0.1585v=4
p=0.10
..4.. p=0.12p=0.26
r=0.1691v=2
p=0.58 p=0.42
p=0.66r=0.2021
v=9
p=0.34r=0.3890
v=4
p=0.66p=0.34
r=0.1903v=7
..5.. p=0.12p=0.34
r=0.2037v=5
..2.. p=0.10p=0.69
r=0.2328v=3
p=0.35p=0.65
r=0.1757v=2
p=0.43 p=0.57
p=0.04r=0.4636
v=1
p=0.54r=0.3893
v=2..7..
..8..p=0.83
r=0.3519v=1
p=0.17
..2..
p=0.86r=0.1706
v=2p=0.14
p=0.10r=0.1576
v=1..2..
..58..
p=0.49p=0.51
r=0.1531v=1
..10..
p=0.04r=0.0813
v=1
p=0.42r=0.2220
v=4
p=0.41r=0.2948
v=4p=0.12
..4.. p=0.09p=0.91
r=0.2154v=3
p=0.04r=0.0533
v=1
p=0.42r=0.1284
v=1..2..
..4.. ..2..
p=0.91r=0.2885
v=3p=0.09
..2..p=0.04
r=0.2072v=1
p=0.41r=0.3459
v=1
..4.. ..2..
p=0.62r=0.2172
v=15
p=0.38r=0.2867
v=7
p=0.62p=0.38
r=0.2144v=13
..5.. p=0.11p=0.35
r=0.2260v=11
p=0.10p=0.66
r=0.2410v=9
..2..
p=0.67r=0.2318
v=8p=0.33
p=0.61p=0.39
r=0.2561v=6
..7..p=0.04
r=0.2875v=1
p=0.49r=0.2169
v=4
..8.. p=0.17p=0.83
r=0.1984v=3
p=0.30r=0.2021
v=1
p=0.70r=0.1930
v=1
..58.. ..3..
..7..p=0.04
r=0.3833v=1
p=0.52r=0.2645
v=5
..8.. p=0.17p=0.83
r=0.2306v=4
p=0.31r=0.2910
v=1
p=0.69r=0.1461
v=2
..58.. ..2..p=0.11
r=0.1104v=1
..58..
..3..p=0.06
r=0.0439v=1
..4..
..3.. p=0.16p=0.11
r=0.0468v=2
p=0.20r=0.4570
v=774
p=0.10r=0.0338
v=2
p=0.18r=0.3394
v=42
p=0.28r=0.0390
v=1..2..
..6..
..6..p=0.05
r=0.0394v=1
p=0.51r=0.4594
v=759
p=0.13r=0.3648
v=13
..8..p=0.81
r=0.4580v=699
p=0.19r=0.4742
v=59
p=0.76r=0.4613
v=690
p=0.24r=0.1710
v=8
p=0.07p=0.83
r=0.4625v=686
p=0.10r=0.1745
v=3
p=0.09p=0.79
r=0.4623v=666
p=0.12r=0.4638
v=19
p=0.12p=0.88
r=0.4622v=665
p=0.88r=0.4620
v=664p=0.12
p=0.12p=0.88
r=0.4618v=663
p=0.10p=0.90
r=0.4615v=662
p=0.90r=0.4613
v=661p=0.10
..5..p=0.39
r=0.4666v=607
p=0.20r=0.3947
v=53
p=0.27r=0.4797
v=98
p=0.73r=0.4644
v=508
p=0.50p=0.50
r=0.4866v=96
p=0.23r=0.5009
v=36
p=0.36r=0.4737
v=59..6..
..7..p=0.04
r=0.6056v=1
p=0.54r=0.4924
v=34
..8.. p=0.10p=0.90
r=0.4889v=33
p=0.89r=0.4835
v=32p=0.11
p=0.09p=0.91
r=0.4863v=31
..2..p=0.68
r=0.4798v=30
p=0.89r=0.4719
v=29p=0.11
p=0.89r=0.4627
v=28p=0.11
p=0.91r=0.4631
v=27p=0.09
p=0.90r=0.4529
v=26p=0.10
p=0.43r=0.4423
v=25..8..
..4.. p=0.10p=0.30
r=0.4541v=23
p=0.65p=0.35
r=0.4681v=21
p=0.13r=0.3356
v=3
p=0.27r=0.4818
v=17..6..
p=0.02r=0.2698
v=2..35..
p=0.02 ..35..
..8..p=0.02
r=0.4200v=1
p=0.44r=0.4819
v=15
..3..p=0.08
r=0.1894v=1
p=0.92r=0.4960
v=13
..2..p=0.92
r=0.5004v=12
p=0.08
p=0.08p=0.92
r=0.4982v=11
p=0.08p=0.92
r=0.4919v=10
p=0.90r=0.4798
v=9p=0.10
p=0.90r=0.4649
v=8p=0.10
p=0.10p=0.90
r=0.4430v=7
p=0.09p=0.91
r=0.4138v=6
p=0.90r=0.3729
v=5p=0.10
p=0.59r=0.2968
v=4p=0.41
p=0.59p=0.41
r=0.2550v=2
..6.. p=0.13
p=0.05p=0.41
r=0.4980v=36
p=0.39r=0.4428
v=21..2..
p=0.52r=0.4551
v=24
p=0.26r=0.5671
v=11p=0.22
..4..p=0.10
r=0.3057v=1
p=0.42r=0.4537
v=22
..3.. ..8..p=0.49
r=0.4586v=21
..4..p=0.09
r=0.1651v=1
p=0.43r=0.4650
v=19
..3..p=0.19
r=0.5221v=15
p=0.23r=0.1725
v=3..6..
..6..p=0.21
r=0.5380v=8
p=0.26r=0.4791
v=6
p=0.01r=0.5267
v=7..35..
..7..p=0.27
r=0.5151v=6
p=0.04r=0.3664
v=1
p=0.56r=0.5148
v=4..7..
..8..p=0.90
r=0.5338v=3
p=0.10
p=0.89r=0.5478
v=2p=0.11
p=0.09p=0.91
r=0.4984v=1
..3..
p=0.03r=0.5425
v=3
p=0.32r=0.3149
v=2..5..
..7..p=0.27
r=0.5156v=2
..8..p=0.04
r=0.3664v=1
..8..
p=0.82p=0.18
r=0.0274v=1
..2..
..4.. p=0.06 p=0.43
p=0.19p=0.41
r=0.6284v=8
p=0.40r=0.3018
v=2
p=0.25r=0.7791
v=1
p=0.75r=0.5984
v=6
..6.. ..7..p=0.15
r=0.6117v=5
p=0.35r=0.5958
v=4..9..
p=0.33r=0.6144
v=3..6..
p=0.68p=0.32
r=0.5316v=2
p=0.11r=0.2808
v=1..5..
..3..
p=0.39r=0.2222
v=1..8..
..8..
..3..p=0.35
r=0.4115v=9
p=0.34r=0.4492
v=11
..3.. p=0.31p=0.29
r=0.4237v=7
p=0.37p=0.63
r=0.3672v=6
p=0.21p=0.79
r=0.3910v=4
p=0.39r=0.4036
v=3..8..
..6..p=0.10
r=0.5081v=1
p=0.53r=0.2632
v=1
..10.. ..2..
..3..p=0.26
r=0.4252v=6
p=0.28r=0.4656
v=4
p=0.37p=0.63
r=0.3578v=5
p=0.63r=0.2875
v=4..2..
p=0.23r=0.2858
v=3..6..
p=0.54p=0.46
r=0.2154v=2
p=0.54 p=0.46
p=0.63r=0.3692
v=3p=0.37
p=0.61r=0.2491
v=2..2..
p=0.23r=0.2618
v=1..6..
..2..
p=0.17r=0.3871
v=28
p=0.40r=0.4731
v=467
p=0.17r=0.2982
v=12..6..
p=0.15r=0.4362
v=13
p=0.21r=0.3464
v=13p=0.18 ..5..
..8..p=0.35
r=0.3554v=5
p=0.35r=0.4581
v=7
..6..p=0.40
r=0.3600v=4
p=0.24r=0.3528
v=1
p=0.76r=0.3870
v=2
..2.. ..8..p=0.42
r=0.2055v=1
..8..
p=0.38r=0.4185
v=6..6..
p=0.26r=0.4174
v=1
p=0.74r=0.4484
v=4
..2..p=0.41
r=0.3908v=3
..8..
..6..p=0.06
r=0.4177v=1
p=0.56r=0.3502
v=1
..8.. ..2..
..7..p=0.09
r=0.2261v=1
p=0.24r=0.3351
v=11
..3..p=0.33
r=0.3285v=2
p=0.67r=0.3064
v=8
p=0.58 p=0.42p=0.38
r=0.2979v=7
..8..
p=1.00r=0.2606
v=5
STEPS:2
..6..
p=0.05r=0.2286
v=1
p=0.51r=0.2492
v=3..2..
..4..p=0.58
r=0.0981v=2
p=0.42
p=0.59 p=0.41
p=0.50r=0.4747
v=462
p=0.10r=0.3024
v=4..6..
p=0.14r=0.2577
v=3
p=0.86r=0.4762
v=458
..2..p=0.06
r=0.2723v=1
p=0.52r=0.2484
v=1
..4.. ..2..
p=0.88r=0.4766
v=457p=0.12
p=0.88r=0.4768
v=456p=0.12
p=0.90r=0.4770
v=455p=0.10
p=0.10p=0.90
r=0.4772v=454
p=0.16r=0.2766
v=5
p=0.84r=0.4794
v=448
..2..p=0.05
r=0.3036v=1
p=0.45r=0.2648
v=3
..4..p=0.71
r=0.2583v=2
p=0.29
p=0.11r=0.2203
v=1..5..
..3..
p=0.88r=0.4796
v=447p=0.12
p=0.88r=0.4797
v=446p=0.12
p=0.90r=0.4798
v=445p=0.10
p=0.89r=0.4798
v=444p=0.11
p=0.13p=0.87
r=0.4795v=443
p=0.15r=0.4817
v=16
p=0.85r=0.4789
v=426
..2..p=0.05
r=0.4097v=1
p=0.45r=0.4884
v=14
..4.. ..2..p=0.07
r=0.4209v=1
p=0.70r=0.5033
v=12
..8..p=0.89
r=0.4805v=11
p=0.11
p=0.88r=0.4517
v=10p=0.12
p=0.62r=0.4663
v=9..2..
p=0.48r=0.4790
v=4
p=0.52r=0.3705
v=4
p=0.48p=0.52
r=0.5651v=2
p=0.05r=0.5643
v=1..8..
..8..
..8..p=0.21
r=0.3091v=3
..4.. p=0.09p=0.28
r=0.4870v=1
..10..
p=0.88r=0.4783
v=425p=0.12
p=0.88r=0.4776
v=424p=0.12
p=0.15r=0.4962
v=36
p=0.62r=0.4786
v=343
p=0.23r=0.4584
v=44
p=0.63r=0.4992
v=35..2..
p=0.48r=0.3582
v=11
p=0.52r=0.5532
v=23
p=0.48p=0.52
r=0.3486v=9
p=0.27r=0.3236
v=6
p=0.23r=0.3026
v=2..7..
p=0.09p=0.28
r=0.3753v=4
..4..
p=0.32r=0.3381
v=3..9..
p=0.09r=0.1868
v=1
p=0.29r=0.3751
v=1..4..
..3.. ..2..
p=0.09 ..4..
..7..p=0.05
r=0.5643v=1
p=0.56r=0.5504
v=21
..8.. p=0.12p=0.88
r=0.5430v=20
p=0.12p=0.88
r=0.5334v=19
p=0.10p=0.90
r=0.5315v=18
..2..p=0.66
r=0.5182v=17
p=0.12p=0.88
r=0.5033v=16
p=0.88r=0.4849
v=15p=0.12
p=0.90r=0.4754
v=14p=0.10
p=0.12p=0.88
r=0.4531v=13
p=0.65r=0.3579
v=9
p=0.35r=0.6200
v=3
p=0.48p=0.52
r=0.3481v=7
p=0.26r=0.2994
v=4
p=0.26r=0.3124
v=2..7..
p=0.09p=0.28
r=0.4084v=2
..4..
..9..p=0.32
r=0.4097v=1
..6..
p=0.09 ..4..
..7..p=0.26
r=0.6335v=2
p=0.21r=0.6346
v=1..7..
..7..
p=0.48r=0.4476
v=135
p=0.52r=0.4972
v=207
p=0.48p=0.52
r=0.4484v=133
p=0.05r=0.4632
v=5
p=0.17r=0.4672
v=14
p=0.56r=0.4444
v=113..6..
..7..p=0.15
r=0.4379v=4
..6..p=0.31
r=0.3920v=3
p=0.22r=0.2971
v=2..7..
p=0.27 ..6..
..3.. p=0.07p=0.59
r=0.4976v=12
p=0.45r=0.4610
v=7
p=0.36r=0.5306
v=4p=0.19
p=0.45r=0.3315
v=2
p=0.55r=0.4756
v=4
p=0.45 p=0.55 ..8..p=0.53
r=0.4835v=3
p=0.55p=0.45
r=0.3938v=2
p=0.45 p=0.55
p=0.37r=0.4981
v=3..2..
p=0.45r=0.4014
v=2p=0.55
p=0.41 p=0.59
p=0.88r=0.4425
v=112p=0.12
p=0.88r=0.4402
v=111p=0.12
p=0.90r=0.4393
v=110p=0.10
p=0.12p=0.66
r=0.4427v=103
p=0.22r=0.3332
v=6
p=0.88r=0.4399
v=102p=0.12
p=0.12p=0.88
r=0.4368v=101
p=0.10p=0.90
r=0.4354v=100
p=0.88r=0.4320
v=99p=0.12
p=0.65r=0.4143
v=68
p=0.35r=0.4597
v=30
p=0.48p=0.52
r=0.4150v=66
p=0.18r=0.2796
v=5
p=0.31r=0.4195
v=41
p=0.18r=0.4321
v=19..6..
..4.. p=0.09p=0.28
r=0.3391v=3
p=0.32r=0.3051
v=2..9..
p=0.09r=0.2005
v=1..5..
..3..
p=0.09p=0.39
r=0.3547v=18
p=0.24r=0.4870
v=21..2..
p=0.65r=0.3728
v=13
p=0.35r=0.2334
v=4
p=0.43p=0.57
r=0.3850v=11
p=0.24r=0.3291
v=6
p=0.24r=0.4441
v=4..6..
p=0.17r=0.3258
v=4p=0.28 ..6..
p=0.32p=0.23
r=0.3173v=2
..5..
..3.. p=0.13
..4.. p=0.05p=0.43
r=0.5692v=2
p=0.89r=0.5473
v=1p=0.11
..2..
p=0.32p=0.15
r=0.2967v=2
..8..
..4.. p=0.13
p=0.26p=0.39
r=0.5096v=19
..4..
p=0.81r=0.5073
v=18p=0.19
p=0.24r=0.4398
v=2
p=0.76r=0.5182
v=15
..57..p=0.03
r=0.4113v=1
..58..
p=0.11r=0.4318
v=1
p=0.82r=0.5337
v=13p=0.07
..58.. ..2..p=0.80
r=0.5301v=12
p=0.10p=0.90
r=0.5233v=11
p=0.88r=0.5130
v=10p=0.12
p=0.12p=0.88
r=0.4977v=9
p=0.89r=0.4709
v=8p=0.11
p=0.13p=0.87
r=0.4316v=7
p=0.50r=0.4425
v=3
p=0.50r=0.3067
v=3
p=0.56p=0.44
r=0.5536v=1
..8..
p=0.18r=0.1904
v=2..8..
p=0.12 ..5..
..8..p=0.36
r=0.4042v=7
p=0.36r=0.4354
v=11
p=0.45r=0.4313
v=3
p=0.55r=0.2543
v=3
p=0.50p=0.50
r=0.5116v=1
..8..
..8..p=0.19
r=0.1257v=2
..5.. p=0.12
p=0.48r=0.5025
v=5
p=0.52r=0.2932
v=5
p=0.54p=0.46
r=0.5626v=3
..7..p=0.23
r=0.5773v=2
p=0.19r=0.5922
v=1..7..
..7..
..7..p=0.18
r=0.2067v=2
p=0.29r=0.2670
v=2
p=0.11 ..5.. p=0.09 ..4..
p=0.26r=0.3885
v=8
p=0.39r=0.4821
v=21..6..
p=0.21r=0.3587
v=7..7..
p=0.26p=0.27
r=0.3822v=5
..5..
p=0.11p=0.68
r=0.3970v=3
..2..
p=0.25p=0.75
r=0.3033v=2
p=0.44 p=0.56
p=0.03p=0.29
r=0.4745v=5
p=0.46r=0.5120
v=14..3..
p=0.43r=0.4471
v=4..2..
p=0.47r=0.3399
v=2
p=0.53r=0.4263
v=1
p=0.53 p=0.47 ..9..
p=0.87r=0.5089
v=13p=0.13
p=0.12p=0.88
r=0.4913v=12
p=0.90r=0.4879
v=11p=0.10
p=0.68r=0.4640
v=10..2..
p=0.12p=0.88
r=0.4357v=9
p=0.12p=0.88
r=0.3975v=8
p=0.90r=0.3674
v=7p=0.10
p=0.88p=0.12
r=0.3637v=5
p=0.08r=0.4549
v=1
p=0.92r=0.2570
v=3
..2.. p=0.11 p=0.74 p=0.15
p=0.77r=0.2613
v=5p=0.23
p=0.03r=0.2015
v=4..57..
..56.. p=0.02 p=1.00
STEPS:3
..6..p=0.21
r=0.2660v=9
p=0.20r=0.5418
v=107
p=0.25r=0.4666
v=90
p=0.09p=0.28
r=0.3002v=5
p=0.25r=0.2262
v=2..3..
..9..p=0.32
r=0.2535v=4
p=0.09r=0.1868
v=1
p=0.29r=0.1876
v=2..4..
..3.. p=0.56 p=0.44
p=0.54 p=0.46
..8..p=0.35
r=0.5811v=57
p=0.35r=0.4949
v=49
p=0.43r=0.5304
v=21
p=0.57r=0.6067
v=35
p=0.43p=0.57
r=0.5480v=19
p=0.05r=0.5454
v=1
p=0.53r=0.5508
v=17..7..
..8.. p=0.12p=0.88
r=0.5441v=16
p=0.12p=0.88
r=0.5330v=15
p=0.10p=0.90
r=0.5321v=14
p=0.66r=0.5183
v=13..2..
p=0.88r=0.5024
v=12p=0.12
p=0.88r=0.4814
v=11p=0.12
p=0.10p=0.90
r=0.4699v=10
p=0.12p=0.88
r=0.4365v=9
p=0.56r=0.3855
v=4
p=0.44r=0.4045
v=4
p=0.45p=0.55
r=0.3815v=2
p=0.19r=0.2514
v=1..8..
..6..
..7..p=0.25
r=0.3706v=3
p=0.20r=0.2880
v=2..7..
..6.. p=0.27
..6..p=0.21
r=0.1182v=2
p=0.21r=0.7704
v=20
p=0.25r=0.4261
v=12
..5.. p=0.10p=0.06
r=0.5509v=1
p=0.55r=0.7935
v=18..7..
..8..p=0.54
r=0.8765v=14
p=0.46r=0.4268
v=3
p=0.39r=0.8980
v=13..9..
p=0.68r=0.9059
v=12p=0.32
p=0.63r=0.9143
v=11..2..
p=0.12p=0.88
r=0.9208v=10
p=0.86r=0.9207
v=9p=0.14
p=0.91r=0.9471
v=8p=0.09
..2..p=0.68
r=0.9487v=7
p=0.86r=0.9496
v=6p=0.14
p=0.14p=0.86
r=0.9497v=5
p=0.09p=0.91
r=0.9889v=4
p=0.11p=0.89
r=0.9996v=3
p=0.20r=0.9999
v=2..3..
p=0.62 ..2..
p=0.46p=0.54
r=0.5401v=1
..9..
..3.. p=0.09p=0.37
r=0.4607v=10
p=0.62r=0.4713
v=8
p=0.38r=0.2794
v=1
p=0.41p=0.59
r=0.5299v=6
p=0.04r=0.6329
v=1
p=0.56r=0.5331
v=4..7..
..8..p=0.89
r=0.5264v=3
p=0.11
p=0.11p=0.89
r=0.5044v=2
p=0.09p=0.91
r=0.5703v=1
..3..
..10..
p=0.46r=0.5184
v=24
p=0.54r=0.4590
v=24
p=0.46p=0.54
r=0.5284v=22
..7..p=0.05
r=0.6141v=1
p=0.51r=0.5219
v=20
..8..p=0.88
r=0.5137v=19
p=0.12
p=0.12p=0.88
r=0.5019v=18
p=0.90r=0.4980
v=17p=0.10
p=0.66r=0.4808
v=16..2..
p=0.12p=0.88
r=0.4644v=15
p=0.12p=0.88
r=0.4440v=14
p=0.90r=0.4306
v=13p=0.10
p=0.12p=0.88
r=0.3980v=12
p=0.59r=0.3268
v=7
p=0.41r=0.4100
v=4
p=0.48p=0.52
r=0.2932v=5
..7..p=0.19
r=0.2067v=2
p=0.30r=0.2670
v=2
..5.. p=0.11 ..4.. p=0.09
..7..p=0.24
r=0.3754v=3
..7..p=0.19
r=0.2916v=2
..6.. p=0.25
..7..p=0.20
r=0.2671v=4
p=0.24r=0.4948
v=19
p=0.08p=0.33
r=0.3064v=2
..4..
p=0.59 p=0.41
p=0.09p=0.38
r=0.5191v=17
..3..
p=0.65r=0.5614
v=14
p=0.35r=0.1698
v=2
p=0.44p=0.56
r=0.6002v=12
..7..p=0.04
r=0.6957v=1
p=0.55r=0.6023
v=10
..8.. p=0.11p=0.89
r=0.6070v=9
p=0.89r=0.6107
v=8p=0.11
p=0.91r=0.6342
v=7p=0.09
p=0.70r=0.6437
v=6..2..
p=0.89r=0.6526
v=5p=0.11
p=0.11p=0.89
r=0.6590v=4
p=0.09p=0.91
r=0.7128v=3
p=0.09p=0.91
r=0.7399v=2
p=0.49p=0.51
r=0.7784v=1
..2..
..9.. p=0.28
..2.. p=0.09p=0.38
r=0.4301v=47
p=0.24r=0.5173
v=41
p=0.65r=0.4512
v=39
p=0.35r=0.2848
v=7
p=0.44p=0.56
r=0.4584v=37
p=0.04r=0.5810
v=1
p=0.56r=0.4556
v=35..7..
..8..p=0.89
r=0.4548v=34
p=0.11
p=0.89r=0.4523
v=33p=0.11
p=0.09p=0.91
r=0.4536v=32
..2..p=0.70
r=0.4500v=31
p=0.11p=0.89
r=0.4454v=30
p=0.11p=0.89
r=0.4395v=29
p=0.91r=0.4388
v=28p=0.09
p=0.10p=0.90
r=0.4320v=27
p=0.55r=0.4619
v=17
p=0.45r=0.3506
v=9
p=0.57p=0.43
r=0.4738v=15
p=0.34r=0.4636
v=14..9..
p=0.35r=0.5304
v=6
p=0.65r=0.3451
v=7
p=0.40p=0.60
r=0.5728v=4
p=0.21r=0.5134
v=3..7..
p=0.16r=0.3842
v=2..7..
..6.. p=0.25
p=0.20r=0.2794
v=6..8..
..4.. p=0.11p=0.23
r=0.3131v=4
..9..p=0.31
r=0.2711v=3
..5..p=0.11
r=0.1859v=2
p=0.26r=0.1393
v=1..2..
..62..
p=0.24r=0.3234
v=6
p=0.25r=0.3098
v=2..6..
p=0.18r=0.3215
v=4p=0.26 ..6..
..5.. p=0.32p=0.23
r=0.3129v=2
..3.. p=0.12
p=0.05 ..5..
..8.. p=0.32p=0.15
r=0.3391v=5
p=0.13p=0.40
r=0.3833v=3
..3..
p=0.44r=0.3183
v=2p=0.56
p=0.57 p=0.43
..4.. p=0.26p=0.39
r=0.5295v=39
p=0.81r=0.5270
v=38p=0.19
p=0.24r=0.3417
v=3
p=0.76r=0.5424
v=34
p=0.03r=0.2369
v=2..57..
..57.. p=0.02
p=0.07p=0.10
r=0.5146v=1
p=0.83r=0.5463
v=32
..58.. ..2..p=0.80
r=0.5428v=31
p=0.10p=0.90
r=0.5396v=30
p=0.12p=0.88
r=0.5358v=29
p=0.12p=0.88
r=0.5309v=28
p=0.11p=0.89
r=0.5236v=27
p=0.87r=0.5150
v=26p=0.13
p=0.50r=0.5754
v=17
p=0.50r=0.3538
v=8
p=0.50p=0.50
r=0.5997v=15
..7..p=0.05
r=0.5662v=1
p=0.54r=0.6051
v=13
..8.. p=0.12p=0.88
r=0.6026v=12
p=0.88r=0.5970
v=11p=0.12
p=0.10p=0.90
r=0.6067v=10
..2..p=0.66
r=0.5961v=9
p=0.12p=0.88
r=0.5831v=8
p=0.12p=0.88
r=0.5631v=7
p=0.90r=0.5648
v=6p=0.10
p=0.88r=0.5304
v=5p=0.12
p=0.33p=0.67
r=0.4605v=4
p=0.50p=0.50
r=0.5287v=2
p=0.18r=0.3844
v=1..8..
..6..
p=0.20r=0.2202
v=2
p=0.27r=0.3675
v=5..7..
p=0.10 ..5.. p=0.09p=0.38
r=0.4202v=3
..3..
p=0.35p=0.65
r=0.3193v=2
p=0.44 p=0.56
p=0.17r=0.3480
v=4
p=0.38r=0.4657
v=39..6..
p=0.24r=0.2757
v=3..8..
..4.. p=0.09p=0.27
r=0.3378v=1
..10..
..2.. p=0.05p=0.30
r=0.5426v=22
p=0.23r=0.3243
v=5
p=0.28r=0.3998
v=10
..7.. p=0.03p=0.38
r=0.5688v=20
p=0.86r=0.5702
v=19p=0.14
p=0.89r=0.5736
v=18p=0.11
p=0.12p=0.88
r=0.5755v=17
p=0.09p=0.91
r=0.5744v=16
p=0.91r=0.5732
v=15p=0.09
p=0.16p=0.84
r=0.5719v=14
p=0.12p=0.88
r=0.5709v=13
p=0.12p=0.88
r=0.5691v=12
p=0.09p=0.91
r=0.5629v=11
p=0.90r=0.5533
v=10p=0.10
p=0.13p=0.87
r=0.5348v=9
p=0.85r=0.5020
v=8p=0.15
p=0.89r=0.4503
v=7p=0.11
p=0.88p=0.12
r=0.4582v=5
..2..p=0.04
r=0.5282v=1
p=0.44r=0.3586
v=3
..4.. p=0.34p=0.66
r=0.2698v=2
p=0.58 p=0.42
p=0.39r=0.3230
v=2
p=0.61r=0.1446
v=2
p=0.42 p=0.58 p=0.27 ..9..
p=0.41r=0.3842
v=9..4..
p=0.32p=0.30
r=0.4076v=7
..3..
..2..p=0.50
r=0.3654v=6
p=0.36r=0.3542
v=2
p=0.64r=0.2617
v=3
p=0.64 p=0.36p=0.30
r=0.2430v=2
..8..
..5.. p=0.09
..9..p=0.33
r=0.2461v=3
..6..p=0.45
r=0.2882v=2
p=0.86r=0.2676
v=1p=0.14
..2..
..2.. p=0.09p=0.36
r=0.2978v=7
p=0.28r=0.3034
v=3
p=0.33r=0.3285
v=2
p=0.67r=0.2133
v=4
p=0.58 p=0.42 ..8.. p=0.29p=0.27
r=0.2428v=2
p=0.09 ..5..
p=0.06r=0.2109
v=1p=0.26 ..7..
..3..
p=0.11r=0.2994
v=1
p=0.67r=0.3888
v=51..2..
..2..p=0.27
r=0.5332v=18
p=0.73r=0.3019
v=32
p=0.42p=0.58
r=0.5636v=16
..7..p=0.30
r=0.5660v=15
..7..p=0.04
r=0.4323v=1
p=0.55r=0.5621
v=13
..8.. p=0.10p=0.90
r=0.5586v=12
p=0.89r=0.5502
v=11p=0.11
p=0.91r=0.5366
v=10p=0.09
p=0.68r=0.5247
v=9..2..
p=0.11p=0.89
r=0.5064v=8
p=0.11p=0.89
r=0.4792v=7
p=0.91r=0.4308
v=6p=0.09
p=0.90r=0.3776
v=5p=0.10
..8..p=0.30
r=0.3111v=4
p=0.10p=0.30
r=0.3111v=2
..4..
p=0.65 p=0.35
..8..p=0.36
r=0.2984v=31
..6..p=0.11
r=0.3128v=2
p=0.53r=0.2924
v=28
p=0.33r=0.1606
v=1..9..
..10..
p=0.80r=0.2903
v=27p=0.20
p=0.89r=0.2920
v=26p=0.11
p=1.00r=0.2929
v=24
STEPS:2
p=0.11
p=0.04r=0.2870
v=1
p=0.42r=0.2902
v=10
p=0.41r=0.3011
v=12p=0.13
..4..p=0.46
r=0.2763v=4
p=0.54r=0.2852
v=5
p=0.46p=0.54
r=0.3663v=2
..9..p=0.35
r=0.4165v=1
..2..
..9..p=0.39
r=0.2844v=4
p=0.48r=0.2118
v=2
p=0.52r=0.2904
v=1
p=0.48 p=0.52 ..7..
p=1.00r=0.3055
v=10
STEPS:2
..2..
p=0.17p=0.04
r=0.3126v=1
p=0.34r=0.2824
v=3
p=0.46r=0.3225
v=5
..4..p=0.46
r=0.2118v=2
p=0.54
p=0.46 p=0.54
p=0.80r=0.3336
v=4..2..
p=0.19p=0.81
r=0.3243v=3
p=0.89r=0.3300
v=2p=0.11
p=1.00
STEPS:2
p=0.11
..7..p=0.53
r=0.4589v=18
p=0.05p=0.62
r=0.4852v=16
p=0.33
p=0.88r=0.4794
v=15p=0.12
p=0.88r=0.4703
v=14p=0.12
p=0.12p=0.88
r=0.4604v=13
p=0.10p=0.90
r=0.4468v=12
p=0.90r=0.4291
v=11p=0.10
..6..p=0.40
r=0.3957v=10
p=0.27r=0.3028
v=1
p=0.73r=0.4193
v=8
..2.. ..8..p=0.41
r=0.3977v=7
..6..p=0.10
r=0.4657v=1
p=0.52r=0.3791
v=5
..10.. p=0.14p=0.86
r=0.3718v=4
p=0.88r=0.3868
v=3p=0.12
p=0.12p=0.88
r=0.4018v=2
p=0.09p=0.91
r=0.4054v=1
..2..
p=0.03r=0.1194
v=2..57..
p=0.02 ..57..
p=0.03r=0.1127
v=6p=1.00
STEPS:2
..56..
..56.. p=0.02 p=1.00
STEPS:5
p=0.85r=0.4725
v=58p=0.15
p=0.10r=0.3364
v=1
p=0.83r=0.4726
v=56p=0.07
..58.. ..2..p=0.79
r=0.4721v=55
p=0.12p=0.88
r=0.4704v=54
p=0.88r=0.4679
v=53p=0.12
p=0.12p=0.88
r=0.4656v=52
p=0.10p=0.90
r=0.4625v=51
p=0.10p=0.90
r=0.4591v=50
p=0.39r=0.4628
v=39
p=0.20r=0.4148
v=10..5..
p=0.27r=0.1514
v=2
p=0.73r=0.4847
v=36
p=0.50 p=0.50 ..8..p=0.40
r=0.4823v=35
..6..p=0.10
r=0.3167v=2
p=0.50r=0.4946
v=32
p=0.33r=0.1620
v=1..9..
..7..
p=0.86r=0.4975
v=31p=0.14
p=0.12p=0.88
r=0.5031v=30
p=0.88r=0.5080
v=29p=0.12
p=0.10p=0.90
r=0.5120v=28
p=0.90r=0.5160
v=27p=0.10
p=0.84r=0.5184
v=26p=0.16
p=0.12p=0.88
r=0.5227v=25
p=0.88r=0.5271
v=24p=0.12
p=0.90r=0.5299
v=23p=0.10
p=0.89r=0.5330
v=22p=0.11
p=0.13p=0.87
r=0.5295v=21
p=0.85r=0.5206
v=20p=0.15
p=0.88r=0.5083
v=19p=0.12
p=0.88r=0.4939
v=18p=0.12
..2..p=0.62
r=0.5040v=17
p=0.48r=0.5852
v=9
p=0.52r=0.3577
v=7
p=0.48p=0.52
r=0.6401v=7
..7..p=0.05
r=0.5643v=1
p=0.56r=0.6701
v=5
..8.. p=0.12p=0.88
r=0.6714v=4
p=0.12p=0.88
r=0.6631v=3
p=0.10p=0.90
r=0.7294v=2
..2..p=0.66
r=0.7299v=1
..2..
p=0.21r=0.3449
v=4
p=0.25r=0.2848
v=2..7..
p=0.09p=0.28
r=0.4696v=2
..4..
p=0.32r=0.4523
v=1..9..
..6..
..4.. p=0.09
..2..p=0.11
r=0.2994v=1
p=0.67r=0.3821
v=8
..2..p=0.27
r=0.2901v=2
p=0.73r=0.3813
v=5
p=0.58 p=0.42p=0.36
r=0.3742v=4
..8..
..6..p=0.11
r=0.4650v=1
p=0.53r=0.2979
v=2
..10.. p=0.20p=0.80
r=0.2459v=1
..2..
p=0.37p=0.19
r=0.3564v=11
..5..
..2.. p=0.12p=0.68
r=0.3451v=9
p=0.30r=0.2872
v=2
p=0.70r=0.3279
v=6
p=0.42 p=0.58p=0.36
r=0.3423v=5
..8..
p=0.07r=0.4707
v=1
p=0.55r=0.2771
v=3..6..
..9.. p=0.20p=1.00
r=0.2435v=1
STEPS:2
..4..
..5..p=0.19
r=0.0194v=1
..3..
p=0.06r=0.1037
v=1
p=0.37r=0.3390
v=37
p=0.15r=0.3697
v=3..7..
..3.. p=0.10p=0.90
r=0.3339v=36
p=0.90r=0.3289
v=35p=0.10
p=0.10p=0.90
r=0.3235v=34
p=0.90r=0.3162
v=33p=0.10
p=0.14p=0.86
r=0.3085v=32
p=0.87r=0.2974
v=31p=0.13
p=0.87r=0.2838
v=30p=0.13
p=0.12p=0.88
r=0.2690v=29
p=0.87p=0.13
r=0.2609v=27
p=0.08r=0.1900
v=1
p=0.92r=0.2529
v=25
..2.. p=0.71p=0.14
r=0.2545v=22
p=0.15
..5.. p=0.08p=0.39
r=0.2552v=20
..2.. p=0.08p=0.68
r=0.2553v=18
p=0.65p=0.12
r=0.2659v=15
p=0.23
..8.. p=0.26p=0.25
r=0.2849v=13
..3.. p=0.10p=0.21
r=0.3350v=6
p=0.28r=0.2273
v=5
..8..p=0.54
r=0.3410v=5
p=0.09r=0.3196
v=2
p=0.29r=0.2573
v=2..4..
..2..p=0.37
r=0.2994v=1
..62..
p=0.43 p=0.57
p=0.57p=0.43
r=0.1933v=3
..6..p=0.14
r=0.0320v=1
p=0.23r=0.2604
v=1
..36.. ..10..
p=0.30r=0.5445
v=1
p=0.70r=0.0838
v=1
..2.. ..9..
p=0.10p=0.37
r=0.2067v=5
..3..
p=0.30r=0.4085
v=1
p=0.70r=0.0481
v=3
..2.. ..8.. p=0.25p=0.26
r=0.0463v=1
..6..
p=0.48p=0.52
r=0.4143v=135
..6..p=0.30
r=0.4105v=77
p=0.25r=0.4189
v=57
p=0.05r=0.1446
v=1
p=0.47r=0.4160
v=75..7..
..8..p=0.90
r=0.4137v=74
p=0.10
p=0.11p=0.89
r=0.4105v=73
p=0.09p=0.91
r=0.4115v=72
p=0.66r=0.4196
v=65
p=0.25r=0.2813
v=6p=0.09
p=0.11p=0.89
r=0.4153v=64
p=0.11p=0.89
r=0.4106v=63
p=0.91r=0.4101
v=62p=0.09
p=0.10p=0.90
r=0.4050v=61
..8..p=0.39
r=0.3984v=60
..2.. p=0.10p=0.28
r=0.4193v=48
p=0.16r=0.3520
v=9p=0.19
p=0.65p=0.35
r=0.4242v=46
..5..p=0.13
r=0.2911v=3
p=0.28r=0.4391
v=41p=0.19
p=0.02r=0.2692
v=2..35..
p=0.02 ..35..
p=0.02r=0.4187
v=1
p=0.44r=0.4608
v=32
p=0.25r=0.3259
v=7..7..
..3..p=0.08
r=0.0542v=1
p=0.92r=0.4700
v=30
..2.. p=0.08p=0.92
r=0.4710v=29
p=0.08p=0.92
r=0.4692v=28
p=0.08p=0.92
r=0.4658v=27
p=0.10p=0.90
r=0.4622v=26
p=0.90r=0.4568
v=25p=0.10
p=0.10p=0.90
r=0.4501v=24
p=0.10p=0.90
r=0.4429v=23
p=0.90r=0.4350
v=22p=0.10
p=0.58r=0.3075
v=9
p=0.42r=0.5105
v=12
p=0.58p=0.42
r=0.2988v=7
..5.. p=0.13p=0.36
r=0.3165v=5
..2.. p=0.10p=0.68
r=0.3537v=3
p=0.69r=0.2659
v=2p=0.31
p=0.40 p=0.60
p=0.04r=0.5645
v=1
p=0.56r=0.5052
v=10..7..
..7..p=0.92
r=0.5109v=9
p=0.08
p=0.08p=0.92
r=0.5169v=8
p=0.92r=0.5127
v=7p=0.08
p=0.92r=0.5075
v=6p=0.08
p=0.10p=0.90
r=0.5005v=5
p=0.10p=0.90
r=0.4709v=4
p=0.90r=0.4252
v=3p=0.10
p=0.10p=0.90
r=0.3189v=2
p=0.90 p=0.10
p=0.62r=0.2735
v=6p=0.38
p=0.62p=0.38
r=0.2499v=4
..5.. p=0.27p=0.32
r=0.2506v=2
p=0.12 ..3..
p=0.31p=0.28
r=0.3647v=7
..5..
..2.. p=0.12p=0.66
r=0.3871v=5
p=0.09r=0.3487
v=2
p=0.32r=0.3096
v=2..4..
p=0.37r=0.2580
v=1..2..
..62..
p=0.62 p=0.38
p=0.80r=0.2138
v=5p=0.20
..57..p=0.03
r=0.1371v=4
..56.. p=0.02 p=1.00
STEPS:3
..2.. p=0.05p=0.40
r=0.4399v=32
p=0.40r=0.3964
v=23
p=0.54r=0.4314
v=24
p=0.23r=0.4427
v=7p=0.22
..4..p=0.10
r=0.1949v=1
p=0.42r=0.4313
v=22
..3.. ..8..p=0.45
r=0.4451v=21
p=0.08r=0.1305
v=1
p=0.47r=0.4523
v=19..4..
..3.. ..6..p=0.16
r=0.5161v=15
p=0.24r=0.1730
v=3
p=0.20r=0.5430
v=6
p=0.28r=0.4791
v=8..6..
p=0.01r=0.5272
v=5..35..
..7..p=0.28
r=0.5086v=4
p=0.04r=0.3769
v=1
p=0.56r=0.4931
v=2..7..
..8..p=0.90
r=0.5171v=1
p=0.10
..2..
..5..p=0.03
r=0.5499v=3
p=0.32r=0.3902
v=4
..7..p=0.28
r=0.5240v=2
p=0.04r=0.3769
v=1..8..
..8..
p=0.18r=0.0166
v=1
p=0.82r=0.4682
v=2
..2..p=0.91
r=0.5030v=1
p=0.09
..2..
p=0.06 p=0.44 ..4..
p=0.42r=0.4670
v=5
p=0.39r=0.1447
v=1p=0.19
p=0.26r=0.7836
v=1
p=0.74r=0.2994
v=3
..6..p=0.18
r=0.2650v=2
..7..
..8..p=0.03
r=0.0100v=1
..8..
..9..
..3..p=0.35
r=0.3900v=10
p=0.33r=0.4177
v=12
..3.. p=0.31p=0.29
r=0.3934v=8
p=0.61r=0.3713
v=6
p=0.39r=0.2209
v=1
p=0.21p=0.79
r=0.3868v=4
..8..p=0.36
r=0.4444v=3
p=0.07r=0.5024
v=1
p=0.54r=0.4031
v=1..6..
..9.. ..2..
..8..
..3..p=0.26
r=0.4085v=6
p=0.28r=0.4255
v=5
p=0.39p=0.61
r=0.3550v=5
p=0.62r=0.2738
v=4..2..
..6..p=0.16
r=0.2710v=3
p=0.37p=0.63
r=0.2042v=2
p=0.59 p=0.41
p=0.39p=0.61
r=0.3374v=4
..2..p=0.60
r=0.2417v=3
p=0.16r=0.2486
v=2..6..
p=0.64r=0.2486
v=1p=0.36
..2..
p=0.11r=0.0603
v=1
p=0.89r=0.3809
v=8
..2.. ..2..p=0.71
r=0.3464v=7
p=0.25r=0.2615
v=1
p=0.75r=0.3745
v=5
..2.. ..8..p=0.38
r=0.3496v=4
p=0.09r=0.4333
v=1
p=0.49r=0.3791
v=2..6..
..9.. p=1.00
STEPS:2
p=0.14
p=0.09r=0.0228
v=1
p=0.91r=0.4129
v=9
..2.. ..2..p=0.71
r=0.3935v=8
p=0.87r=0.3464
v=7p=0.13
p=0.25r=0.2615
v=1
p=0.75r=0.3745
v=5
..2.. ..8..p=0.38
r=0.3496v=4
p=0.09r=0.4333
v=1
p=0.49r=0.3791
v=2..6..
..9.. p=0.14 p=1.00
STEPS:2
p=0.09r=0.0128
v=1
p=0.91r=0.4098
v=19
..2.. ..2..p=0.74
r=0.4019v=18
..2..p=0.75
r=0.3864v=17
p=0.12p=0.88
r=0.3618v=16
p=0.87r=0.3334
v=15p=0.13
p=0.27r=0.1444
v=2
p=0.73r=0.3621
v=12
p=0.50 p=0.50 ..8..p=0.38
r=0.3448v=11
..6..p=0.08
r=0.4829v=1
p=0.50r=0.3364
v=9
..9.. p=0.14p=1.00
r=0.3233v=7
STEPS:2
p=0.05r=0.2163
v=1
p=0.46r=0.3422
v=5..2..
..4.. ..2..p=0.69
r=0.3310v=4
p=0.82r=0.3041
v=3p=0.18
p=0.30r=0.3362
v=1
p=0.70r=0.2320
v=1
..58.. ..3..
p=0.10r=0.0131
v=1
p=0.90r=0.3989
v=22
..2.. ..2..p=0.76
r=0.4052v=21
p=0.75r=0.4037
v=20..2..
p=0.13p=0.87
r=0.3887v=19
p=0.13p=0.87
r=0.3712v=18
p=0.88r=0.3472
v=17p=0.12
p=0.87r=0.3197
v=16p=0.13
p=0.27r=0.1444
v=2
p=0.73r=0.3430
v=13
p=0.50 p=0.50p=0.38
r=0.3255v=12
..8..
..6..p=0.08
r=0.4829v=1
p=0.50r=0.3141
v=10
..9.. p=0.14p=1.00
r=0.2971v=8
STEPS:2
p=0.05r=0.2163
v=1
p=0.46r=0.3041
v=6..2..
..4.. ..2..p=0.69
r=0.2876v=5
p=0.18p=0.82
r=0.2565v=4
p=0.30r=0.3362
v=1
p=0.70r=0.1728
v=2
..58..p=0.11
r=0.1136v=1
..2..
..58..
p=0.08 ..4..
..3.. p=0.06p=0.50
r=0.0985v=1
..2..
p=0.21p=0.79
r=0.2156v=31
..5..p=0.22
r=0.2637v=22
p=0.13r=0.0392
v=2
p=0.21r=0.1267
v=6
..7..p=0.06
r=0.0320v=1
p=0.46r=0.2857
v=20
..8.. p=0.13p=0.87
r=0.2910v=19
p=0.11p=0.89
r=0.2966v=18
p=0.10p=0.90
r=0.3095v=17
p=0.09p=0.56
r=0.3231v=13
p=0.35r=0.1771
v=3
p=0.12p=0.88
r=0.3040v=12
p=0.11p=0.89
r=0.2788v=11
p=0.10p=0.90
r=0.2970v=10
p=0.11p=0.89
r=0.2485v=9
..8..p=0.37
r=0.1795v=8
..3.. p=0.10p=0.19
r=0.1188v=1
p=0.27r=0.1712
v=5
..7.. p=0.61p=0.39
r=0.1168v=3
p=0.13r=0.0386
v=1
p=0.27r=0.1806
v=1..6..
..36.. ..10..
p=0.19p=0.81
r=0.0301v=2
..57..p=0.03
r=0.0365v=1
..58..
..7..p=0.39
r=0.0362v=1
..9..
..2.. p=0.06p=0.38
r=0.1479v=1
p=0.41r=0.1610
v=3
..3..p=0.33
r=0.2246v=2
..4..
p=0.29 ..4..
p=0.06r=0.0760
v=1..26..
..61..
..26..p=0.06
r=0.0656v=2
..60..p=0.02
r=0.0410v=1
..62..
p=0.37r=0.0448
v=1
p=0.47r=0.0613
v=1p=0.16
..58.. ..61..
p=0.29r=0.0486
v=1
p=0.37r=0.1091
v=1..2..
..58.. ..61..
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 44 / 72
RL-CoP setup summary
1. Representation:a search in the treeshould correspondto a tableaux
<=
>
3. Explore thenode and backupthe found rewardto all nodes above
6. Repeat100 times
9. Repeat!
2. Playout: followmaximum UCT untilunexplored node
5. Focus onmost visitednode
4. Repeat1000 times
7. Do this for alltheorems. We getmany sequencesof focused steps
8. Train newpredictors forpolicy and valueusing the seqs.
Theorem proving requiring significant hardware
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 45 / 72
RL-CoP setup summary
1. Representation:a search in the treeshould correspondto a tableaux
<=
>
3. Explore thenode and backupthe found rewardto all nodes above
6. Repeat100 times
9. Repeat!
2. Playout: followmaximum UCT untilunexplored node
5. Focus onmost visitednode
4. Repeat1000 times
7. Do this for alltheorems. We getmany sequencesof focused steps
8. Train newpredictors forpolicy and valueusing the seqs.
Theorem proving requiring significant hardwareCezary Kaliszyk Artificial Intelligence in Theorem Proving 45 / 72
ATP versus learned ATP
ATPs tend to find short proofs.
Learning helps only minimally
Graph Representations for Higher-Order Logic andTheorem Proving [A. Paliwal et.al., 2019 ]
Cumulative proof lengths of rlCoP on the MizarMathematical Library [NeurIPS 2018 ]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 46 / 72
Main aims of FLoP [Zombori’2019 ]
Build an internal guidance system that can find long proofs
Find a domain that is simple enough to analyse the inner workings of the prover
At first, try to learn from very few problems (with given or without given proofs)
Try to generalize to harder problems (longer proofs) with a similar structure
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 47 / 72
Domain: Robinson Arithmetic
Prove simple ground equalities
Proofs are non trivial, but have a strong shared structure
Proof lengths can get very long as numbers increase
See how little supervision is required to learn some proof types
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 48 / 72
Challenges for RL for TP
Theorem proving as a 1 person game
Meta-Learning task: each problem isa new maze: train on some, evaluateon others
Sparse, binary rewards
Defining good features
Action space not fixed:different across steps and acrossproblems
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 49 / 72
Challenges for RL for TP
Theorem proving as a 1 person game
Meta-Learning task: each problem isa new maze: train on some, evaluateon others
Sparse, binary rewards
Defining good features
Action space not fixed:different across steps and acrossproblems
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 49 / 72
FLoP
External guidance based on RL
Theorem Prover encapsulated as an environment
Use curriculum learning
Applicable when we know the proof of a problem
More efficient use of training signals
Start learning from the end of the proof
Gradually move starting step towards the beginning of proof
Proximal Policy Optimization (PPO)
Actor learns a policy (what steps to take)
Critic learns a value (how promising is a proof state)
Actor is confined to change slowly to increase stability
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 50 / 72
Datasets
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 51 / 72
Evaluation
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 52 / 72
Learning for ATPs: Summary and next steps
For some calculi major improvement
Learning for Resolution-style systems open
Learn features
RL prefers shorter proofs
but they may not be the ones that generalize best
Evaluate with backtracking
Scale to more interesting domains
Bolzano–Weierstrass
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 53 / 72
Communication with a Proof Assistant
The prover does not get what I mean
Completely clear things need to be fully expanded
Even if I said it 100 times, I have to say it again
(or implement the expansion)
Compared to a student
Proof assistant does not get what I mean
Cannot repeat a simple action
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 54 / 72
Proof assistant – assistant
Given some text, the assistant can say
What you wrote
What you wanted to write
(What I think you meant)
Does it make sense
Can I be convinced of this(Can I prove this∗)
TasksUnderstand LATEX formulas, as well as some text
Translate it to logic (of a/the proof assistant)
Report on the success
Questions
Can we (a computer) learn how to state lemmas formally?
Can we (a computer) learn to prove?
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 55 / 72
Proof assistant – assistant
Given some text, the assistant can say
What you wrote
What you wanted to write
(What I think you meant)
Does it make sense
Can I be convinced of this(Can I prove this∗)
TasksUnderstand LATEX formulas, as well as some text
Translate it to logic (of a/the proof assistant)
Report on the success
Questions
Can we (a computer) learn how to state lemmas formally?
Can we (a computer) learn to prove?
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 55 / 72
Demo
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 56 / 72
Why don’t we have this? (1/2)
Claus Zinn and others tried and have not arrived very far because:
lack of background knowledge
lack of powerful automated reasoning
lack of self-adapting translation
But huge machine learning progress
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 57 / 72
Why don’t we have this? (1/2)
Claus Zinn and others tried and have not arrived very far because:
lack of background knowledge
lack of powerful automated reasoning
lack of self-adapting translation
But huge machine learning progress
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 57 / 72
Why don’t we have this? (2/2)
Controlled languages
Ranging from Naproche and MathLang to Mizar
Easy start but huge number of patterns100 most frequent patterns cover half of 42,931 ProofWiki sentences [CICM’14 ]
5829 Let $?$ be [?].
2688 Let $?$.
774 Then $?$ is [?].
736 Let $?$ be [?] of $?$.
724 Let $?$ and $?$ be [?].
578 Let $?$ be the [?] of $?$.
555 Let $?$ be the [?].
But can go very far
Thousands of manually entered patterns [Matsuzaki+’16,’17 ]
Better than humans on university entrance exams (some domains) [Arai+’18 ]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 58 / 72
Learning data: Aligned corpora
Dense Sphere Packings: A Blueprint for Formal Proofs
400 theorems and 200 concepts mapped [Hales13 ]
IsaFoR [SternagelThiemann14 ]
most of “Term Rewriting and All That” [BaaderNipkow ]
Compendium of Continuous Lattices (CCL)
60% formalized in Mizar [BancerekRudnicki02 ]
high-level concepts and theorems aligned
Feit-Thompson theorem by Gonthier [Gonthier13 ]
Two graduate books
detailed proofs and symbol linking in Wikipedia, ProofWiki, PlanetMath, ...
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 59 / 72
Aligned corpora: Kepler ExampleDocument:
Informal Formal
Definition of [fan, blade] DSKAGVP (fan) [fan FAN]
Let be a pair consisting of a set and a set of unordered pairs of distinct elementsof . The pair is said to be a fan if the following properties hold.
(CARDINALITY) is finite and nonempty. [cardinality fan1]1.(ORIGIN) . [origin fan2]2.(NONPARALLEL) If , then and are not parallel. [nonparallel fan6]3.(INTERSECTION) For all , [intersection fan7]4.
When , call or a blade of the fan.
basic properties
The rest of the chapter develops the properties of fans. We begin with a completely trivialconsequence of the definition.
Informal Formal
Lemma [] CTVTAQA (subset-fan)
If is a fan, then for every , is also a fan.
Proof
This proof is elementary.
Informal Formal
Lemma [fan cyclic] XOHLED
[ set_of_edge] Let be a fan. For each , the set
is cyclic with respect to .
Proof
If , then and are not parallel. Also, if , then
Article Raw Log in
↔
(V , E) V ⊂ R3 EV
V ↔0 ∉ V ↔
{v, w} ∈ E v w ↔ε, ∈ E ∪ {{v} : v ∈ V }ε′ ↔
C(ε) ∩ C( ) = C(ε ∩ ).ε′ ε′
ε ∈ E (ε)C0 C(ε)
(V , E) ⊂ EE ′ (V , )E ′
E(v) ↔ (V , E) v ∈ V
E(v) = {w ∈ V : {v, w} ∈ E}
(0, v)
w ∈ E(v) v w w ≠ ∈ E(v)w′
Document:
Informal Formal
#DSKAGVP?
let FAN=new_definition`FAN(x,V,E) <=> ((UNIONS E) SUBSET V) /\ graph(E) /\ fan1(x,V,E) /\ fan2(x,V,E)/\fan6(x,V,E)/\ fan7(x,V,E)`;;
basic properties
The rest of the chapter develops the properties of fans. We begin with a completely trivial consequence ofthe definition.
Informal Formal
let CTVTAQA=prove(`!(x:real^3) (V:real^3->bool) (E:(real^3->bool)->bool) (E1:(real^3->bool)->bool).FAN(x,V,E) /\ E1 SUBSET E==>FAN(x,V,E1)`,
REPEAT GEN_TACTHEN REWRITE_TAC[FAN;fan1;fan2;fan6;fan7;graph]THEN ASM_SET_TAC[]);;
Informal Formal
let XOHLED=prove(`!(x:real^3) (V:real^3->bool) (E:(real^3->bool)->bool) (v:real^3).FAN(x,V,E) /\ v IN V==> cyclic_set (set_of_edge v V E) x v`,
MESON_TAC[CYCLIC_SET_EDGE_FAN]);;
Informal Formal
Remark [easy consequences of the definition] WCXASPV (fan)
Let be a fan.
The pair is a graph with nodes and edges . The set
is the set of edges at node . There is an evident symmetry: if and only if .
1.
[ sigma_fan] [ inverse1_sigma_fan] Since is cyclic, each has an azimuthcycle . The set can reduce to a
2.
singleton. If so, is the identity map on . To make the notation less cumbersome, denotes the value of the map at .
The property (NONPARALLEL) implies that the graph has no loops: .1.
The property (INTERSECTION) implies that distinct sets do not meet. This property of fansis eventually related to the planarity of hypermaps.
2.
Article Raw Log in
(V , E)
(V , E) V E
{{v, w} : w ∈ E(v)}
v w ∈ E(v) v ∈ E(w)σ ↔ σ(v)−1 ↔ E(v) v ∈ V
σ(v) : E(v) → E(v) E(v)
σ(v) E(v) σ(v, w)σ(v) w
{v, v} ∉ E
(ε)C0
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 60 / 72
Aligned corpora: Kepler Example
596 formulas from the Flyspeck book extracted with LATEXML
Translation to HOL Light based on a small table
17% same as formal ones
Too hardmake more precise examples or [ongoing ]
start with simpler ones [ITP’15 +]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 61 / 72
Informalization
22000 Flyspeck statements informalized
72 overloaded instances like “+” for vector add
108 infix operators
forget all “prefixes”
real , int , vector , nadd , hreal , matrix , complex
ccos, cexp, clog, csin, ...
vsum, rpow, nsum, list sum, ...
Deleting all brackets, type annotations, and casting functors
Cx and real of num (which alone is used 17152 times).
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 62 / 72
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 63 / 72
CYK and parsing — just a little
Induce PCFG (probabilistic context-free grammar) from term trees
inner nodes → rules frequencies → probabilities
Binarize PCFG grammar for efficiency
CYK parses ambiguous sentences
outputs most probable parse trees
tweak: small probability for each symbol to be a variable
Pruning
Compatible types for free variables in subtrees
HOL type-checking
Hammer
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 64 / 72
Example tree inducing grammar
"(Type bool)"
! "(Type (fun real bool))"
Abs
"(Type real)" "(Type bool)"
Var
A0
"(Type real)" = "(Type real)"
real_neg "(Type real)"
real_neg "(Type real)"
Var
A0
Var
A0
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 65 / 72
Statistics
Just PFG [ITP’15 ]
39.4% of the Flyspeck sentences parsed correctly
average rank: 9.34
Problems with PCFG and CYK
1 ∗ x + 2 ∗ x
Use deeper trees [ITP 2017 ]
semantic pruning + subtree depth 4-8 + substitution trees
83% sentences parsed correctly
average rank: 1.93
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 66 / 72
Statistics
Just PFG [ITP’15 ]
39.4% of the Flyspeck sentences parsed correctly
average rank: 9.34
Problems with PCFG and CYK
1 ∗ x + 2 ∗ x
Use deeper trees [ITP 2017 ]
semantic pruning + subtree depth 4-8 + substitution trees
83% sentences parsed correctly
average rank: 1.93
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 66 / 72
Types helped us - what about no types?
MizarDeveloped by mathematicians for mathematicians
Many features significantly different from the usual
How would you formalize:
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 67 / 72
Mizar Statistics
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 68 / 72
Sequence-to-sequence models: decoder/encoder RNN
[Luong et al’15 ]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 69 / 72
Sequence-to-sequence models: decoder/encoder RNN
[Luong et al’15 ]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 69 / 72
Neural Auto-formalization [CICM’18 ]
IdenticalStatements
0
≤ 1 ≤ 2 ≤ 3
Best Model- 1024 Units
69179 (total)22978 (no-overlap)
65.73%47.77%
74.58%59.91%
86.07%70.26%
88.73%74.33%
Top-5 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units
78411 (total)28708 (no-overlap)
74.50%59.68%
82.07%70.85%
87.27%78.84%
89.06%81.76%
Top-10 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units- 2-Layer Adam Bi. Res.- 256 Units- 5-Layer Adam Res.- 6-Layer Adam Res.- 2-Layer Bi. Res.
80922 (total)30426 (no-overlap)
76.89%63.25%
83.91%73.74%
88.60%81.07%
90.24%83.68%
Union of All 39 Models 83321 (total)32083 (no-overlap)
79.17%66.70%
85.57%76.39%
89.73%82.88%
91.25%85.30%
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 70 / 72
Neural Auto-formalization [CICM’18 ]
IdenticalStatements
0
≤ 1 ≤ 2 ≤ 3
Best Model- 1024 Units
69179 (total)22978 (no-overlap)
65.73%47.77%
74.58%59.91%
86.07%70.26%
88.73%74.33%
Top-5 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units
78411 (total)28708 (no-overlap)
74.50%59.68%
82.07%70.85%
87.27%78.84%
89.06%81.76%
Top-10 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units- 2-Layer Adam Bi. Res.- 256 Units- 5-Layer Adam Res.- 6-Layer Adam Res.- 2-Layer Bi. Res.
80922 (total)30426 (no-overlap)
76.89%63.25%
83.91%73.74%
88.60%81.07%
90.24%83.68%
Union of All 39 Models 83321 (total)32083 (no-overlap)
79.17%66.70%
85.57%76.39%
89.73%82.88%
91.25%85.30%
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 70 / 72
Neural Auto-formalization [CICM’18 ]
IdenticalStatements
0 ≤ 1 ≤ 2 ≤ 3
Best Model- 1024 Units
69179 (total)22978 (no-overlap)
65.73%47.77%
74.58%59.91%
86.07%70.26%
88.73%74.33%
Top-5 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units
78411 (total)28708 (no-overlap)
74.50%59.68%
82.07%70.85%
87.27%78.84%
89.06%81.76%
Top-10 Greedy Cover- 1024 Units- 4-Layer Bi. Res.- 512 Units- 6-Layer Adam Bi. Res.- 2048 Units- 2-Layer Adam Bi. Res.- 256 Units- 5-Layer Adam Res.- 6-Layer Adam Res.- 2-Layer Bi. Res.
80922 (total)30426 (no-overlap)
76.89%63.25%
83.91%73.74%
88.60%81.07%
90.24%83.68%
Union of All 39 Models 83321 (total)32083 (no-overlap)
79.17%66.70%
85.57%76.39%
89.73%82.88%
91.25%85.30%
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 70 / 72
Machine Learning applied to informal LaTeX
[Karpathy’16 ]
Linguistic methodsWord2Vec on the Arxiv [Alemi’16,...]
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 71 / 72
Final Summary / Take Home
Proofs are hard
Machine learning key to most powerful proof assistant automation
Older but very efficient algorithms with significant adjustments
Many other learning problems and scenarios
Not coveredLearning strategy selection [Jakubuv,Urban]
Kernel methods [Kuhlwein]
Deep Prolog [Rocktaschel ]
Semantic Features, Conecturing
Tactic selection [Nagashima,...]
SVM [Holden]
Adversarial Networks [Szegedy ]
Human proof optimization
Theory exploration [Bundy+]
Concept Alignment [Gauthier ]
...
Cezary Kaliszyk Artificial Intelligence in Theorem Proving 72 / 72