Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | gabriella-knotts |
View: | 215 times |
Download: | 1 times |
MOTIVATION High-level LP languages, e.g.,
SILK http://silk.semwebcentral.org Flora-2 http://flora.sourceforge.net
are designed to be suitable for knowledge engineers, who are
not programmers KBs created by engineers typically
are complex, large stress the capabilities of the underlying engine=> Non-termination happens often – very hard to debug
To address this, we developed Terminyzer – a non-Termination analyzer (this extends our previous work in PADL-13)
OUTLINE Preliminaries: causes of non-termination,
tabling, and forest logging Adding ids to rules Terminyzer: analyses causes, repairs
problems Experiments Conclusion and future work
CAUSES OF NON-TERMINATION Cause 1: loops in SLD-resolution Example
p(X) :- p(X).?- p(a).
Solution: tabling [SW12] Caches calls to subgoals, which cuts recursive
loops If enough predicates are tabled then
Each subgoal is tabled once Each answer is tabled once Evaluation terminates if there are finitely many
subgoals and answers
[SW12] Terry Swift and David S. Warren. XSB: Extending prolog with tabled logic programming. TPLP’12.
CAUSES OF NON-TERMINATION (CONT’D) Cause 2:
Engine supports tabling But the program generates infinitely many tabled
subgoals Example
p(X) :- p(f(X)).?- p(a).Subgoals to be tabled: p(a), p(f(a)), p(f(f(a))), ...
Solution: subgoal abstraction [RS] Abstracts subgoals that are deeper than a
threshold Assuming threshold = 2, p(f(f(f(a)))) would be
abstracted to p(f(f(X))), X = f(a) Guarantees that there will be only a finite number
of tabled subgoals[RS] F. Riguzzi and T. Swift. Terminating evaluation of logic programs with finite three-valued models. ACM on Computational Logic. To appear.
CAUSES OF NON-TERMINATION (CONT’D) Cause 3:
Engine supports both tabling and subgoal abstraction But infinitely many answers
Examplep(a). p(f(X)) :- p(X).?- p(X).Answers to be derived: p(a), p(f(a)), p(f(f(a))), …
Solution: does not exist Halting problem is undecidable Whether a program has a finite number of answers is
undecidable We can only try to help the user to deal with the issue
If user really intended the program to be the way it is We need a way to limit output. E.g., bounded rationality [BS13]
Our focus: Unexpected non-termination (ie, when it’s a bug)
[BS13] B. Grosof and T. Swift. Radial restraint: A semantically clean approach to bounded rationality for logic programs. AAAI’13
TABLING AND FOREST LOGGING Tabling needs no introduction Forest logging is a new tracing facility in XSB
Events Logs
Calls to tabled subgoalsE.g. parent calls child
tabled_call(child,parent,status,timestamp)neg_tabled_call(child,parent,status,timestamp)
Answer derivationsE.g. ansr is derived for sub
new_answer(ansr,sub,timestamp)new_delayed_answer(ansr,sub,delayed_lits,timestamp)
Return answers to consumersE.g. ansr for child is retuned to parent
answer_return(ansr,child,parent,timestamp)delayed_answer_return(ansr,child,parent,timestamp)
Subgoal completionsE.g. sub is completed
completed(sub,scc_num,timestamp)completed(sub,early_completed,timestamp)
Other events Irrelevant to our discussion
where Timestamp preserves the order of events status = new, complete, incomplete
ADDING IDS TO RULES – KEY INSIGHT Add unique ids to rules s.t. tabled subgoals
remember their host rules:
Each tabling declaration :- table p/n is changed to :- table p/(n+1)
For a query ?- p(x1, …, xn) If p/n is tabled, then
Change it to ?- p(x1, …, xn, Newvar) Chop off the last arguments of returned answers
Otherwise, the query stays the same
TERMINYZER OVERVIEW Two versions Version 1 (most precise) requires:
Tabling Forest logging Subgoal abstraction
Version 2 requires only Tabling Forest logging
Currently only XSB has all three features, but: Several systems have tabling Forest logging info exists internally in all of them
– just needs to be exposed to the user => at least Version 2 is easily portable
TERMINYZER OVERVIEW
Suppose a query does not terminate, Terminyzer then Analyzes an execution forest log Determines the causes of non-termination
The exact sequence of unfinished tabled subgoals, and the host rule id of each subgoal
Subgoals forming recursive cycles Rectifies some causes of misbehavior
(heuristically)
CALL SEQUENCE ANALYSIS Identifies the exact sequence of unfinished calls
and their host rule ids that lead to a non-termination Find unfinished subgoals – whose answers have not
been completely derived – by unfinished(Child,Parent,Timestamp) :-
tabled_call(Child,Parent,new,Timestamp),
not_exists(completed(Child,SCCNum,Timestamp1)).
unfinished(child,parent,timestamp) says that Subgoal parent calls subgoal child, and it happened at timestamp Neither child nor parent have been completely evaluated
Sort unfinished calls by their timestamps Host rule ids are kept in the last arguments of child-
subgoals
not_exists is the XSB well-founded negation operator; existentially quantifies SCCNum and Timestamp1.
CALL SEQUENCE ANALYSIS (CONT’D) Example 1
@!r1 p(a). @!r5 r(X) :- r(X).@!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- p(X). ?- r(X).where @!ruleid is the syntax to assign rule ids
Its unfinished calls – the red ones form a non-terminating loopunfinished(r(_h9900,_h9908), root, 0) – root is for intial query
unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)
Next, we will find recursive cycles
CALL SEQUENCE ANALYSIS (CONT’D) Unfinished calls can be represented using an unfinished-call
graph UCG = (N,E) N: the set of unfinished subgoals E: {(parent,child) |unfinished(child,parent,ts) is true}
For the above exampleunfinished(r(_h9900,_h9908), root, 0)unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)
where: Each node is represented by the timestamp when it is first called -1 represents root Edges are labeled with timestamps of calls
Loops in UCG represent recursive cycles However, not all cycles are causing non-termination
E.g. [8, 8]
Can be represented as
thousandsof subgoals
CALL SEQUENCE ANALYSIS (CONT’D) Assume all predicates are tabled + subgoal abstraction Theorem (Soundness of the call sequence analysis)
If there are unfinished calls in a query’s complete trace, then Call sequence analysis finds the exact sequence of
unfinished calls that caused non-termination, and The ids of the rules that issued these calls
Theorem (Completeness of the call sequence analysis) If the evaluation of a query does not terminate, then There is at least one loop in the UCG for its complete trace, and
the loop’s subgoals are responsible for generating infinite number of answers, and
The last argument of each of these subgoals specifies the rule ids from whose bodies these subgoals were called.
The complete trace is infinite due to non-termination so, practically speaking We work with only a prefix of the trace by limiting term depth/size or
execution time/space It may produce false negatives, but they are also useful for
identifying computational bottlenecks.
ANSWER FLOW ANALYSIS Recall that not all cycles in UCG are causing
non-termination, so we need to refine call sequence analysis
Answer flow analysis does precisely that: it identifies the cycles that actually cause non-termination
Non-termination happens if and only if a subset of subgoals keeps: Receiving answers from producers, Deriving new answers, and Returning answers to callers
Answer flow analysis looks for repeated patterns of answer returns
ANSWER FLOW ANALYSIS (CONT’D) Compute answer-flow patterns (AFP)
Answer-return sequence (ARS): the sequence of (child,parent) pairs where child returns answers to parent
Candidate AFP: a sequence cafp s.t. cafp2+ is a suffix of ARS AFP: the shortest candidate AFP cafp s.t. its repetition forms
the maximal suffix of ARS among all candidate AFP’s In previous Example 1
ARS = [ (p(_h599,r4),q(_h599,r2)), (q(_h619,r2),p(_h619,r4)), (p(_h639,r4),q(_h639,r2)), (q(_h659,r2),p(_h659,r4)), (p(_h679,r4),q(_h679,r2)), (q(_h699,r2),p(_h699,r4)), (p(_h719,r4),q(_h719,r2)), (q(_h739,r2),p(_h739,r4)), (p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))].
where (child,parent) indicates child returns answers to parent Candidate AFPs are:
cafp1 = [(p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))] cafp2 = cafp1• cafp1
AFP is cafp1 AFP captures information flow pattern without redundancy
@!r2 p(f(X)) :- q(X).@!r4 q(g(X)) :- p(X).
ANSWER FLOW ANALYSIS (CONT’D) An AFP can be represented as an answer-flow
graph AFG = (N,E), where N: the set of subgoals in afp E: {(child,parent) | (child,parent) ∈ afp}
Loops in AFG represent cycles that cause non-termination
ANSWER FLOW ANALYSIS (CONT’D) As before, we assume all predicates are tabled +
subgoal abstraction Theorem (Soundness of the answer flow analysis)
If the complete trace of a query has an AFP then the query does not terminate.
Theorem (Completeness of the answer flow analysis) If the query evaluation does not terminate, then: There is an AFP in its complete trace, AFG = (N, E) contains at least one loop, Every sub ∈ N appears in at least one loop, and Each edge (sub1,sub2)∈E, where sub1=pred(...,
ruleid), tells us that sub2 calls sub1 from the body of a rule whose id is ruleid.
MORE ON UCG AND AFG Theorem (Relationship between UCG and
AFG)Consider the UCG and AFG for a non-terminating forest log, we have: nodes(AFG) ⊂ nodes(UCG) edges(AFG) ⊂ reverse-edges(UCG) loops(AFG) ⊆ loops(UCG)
Theorem (No false results for finite traces) If the evaluation of a query, Q, terminates, then both the UCG and the AFG for Q’s trace are empty.
AUTO-REPAIR OF RULES A query does not terminate if
It has infinitely many answers, or It has a finite number of answers, but one of its
subqueries has an infinite number of them In this case: a different evaluation order may terminate
the query This case is targeted by our auto-repair heuristic
For each unfinished(child,parent,timestamp) We know
the host rule for this call, and the common set of the unbound arguments of parent and
child – the arguments whose bindings are to be derived Thus, to reduce the possibility that parent receives
infinite number of bindings from child, one can delay issuing a child-call from its host rule until these arguments are bound
AUTO-REPAIR OF RULES (CONT’D) Example
@!r1 p(a). @!r5 r(X) :- r(X).@!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- p(X). ?- r(X).
Its unfinished calls are:unfinished(r(_h9900,_h9908), root, 0)unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)
Applying auto-repair@!r1 p(a). @!r5 r(X) :- wish(ground(X))^r(X).@!r2 p(f(X)) :- wish(ground(X))^q(X). @!r6 r(X) :- wish(ground(X))^p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- wish(ground(X))^p(X). ?- wish(ground(X))^r(X).
Then the query will terminate with X = f(b)
TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION
Additional cause of non-termination: infinite number of subgoals
Steps Compute the sequence of unfinished subgoals Compute simplified subgoal sequence (SSS) out of
unfinished subgoal sequence Each unfinished subgoal, predicate(…, ruleid), is
simplified to predicate(ruleid) Find the SSS pattern, as in the case of answer flow
pattern SSS pattern contains the predicates and their rule
ids that recursively call one another to form increasingly deep subgoals
TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION (CONT’D)
Example@!r1 p(a). @!r4 r(X) :- r(X).@!r2 p(X) :- q(f1(X)). @!r5 r(X) :- p(X), s(X).@!r3 q(X) :- p(f2(X)). @!r6 s(a).?- r(a).
Its unfinished calls are:unfinished(r(a,_h46), root, 0).unfinished(r(a,r4), r(a,_h27), 8).unfinished(r(a,r4), r(a,r4), 11).unfinished(p(a,r5), r(a,r4), 12).unfinished(q(f1(a),r2), p(a,r5), 16).unfinished(p(f2(f1(a)),r3), q(f1(a),r2), 19).unfinished(q(f1(f2(f1(a))),r2), p(f2(f1(a)),r3), 22).unfinished(p(f2(f1(f2(f1(a)))),r3), q(f1(f2(f1(a))),r2), 25).unfinished(q(f1(f2(f1(f2(f1(a))))),r2), p(f2(f1(f2(f1(a)))),r3), 28).unfinished(p(f2(f1(f2(f1(f2(f1(a)))))),r3), q(f1(f2(f1(f2(f1(a))))),r2), 31).unfinished(q(f1(f2(f1(f2(f1(f2(f1(a)))))),r2), p(f2(f1(f2(f1(f2(f1(a)))))),r3), 34).……
SSS = [root, r(_), r(r4), r(r4), p(r5), q(r2), p(r3), q(r2), p(r3), q(r2), p(r3), q(r2)] SSS pattern = [p(r3), q(r2)] – says that it is predicate p of r3 and predicate q of
r2 that recursively call each other, thus forming increasingly deep nested subgoals
STATUS
Unfinished-call/answer flow implemented in SILK and Flora-2
SILK has a GUI, Flora-2’s underway Rule Ids are crucial for practicality Auto-repair: not implemented yet
EXPERIMENTS System
Dual core 2.4GHz Lenovo X200 with 3GB RAM Ubuntu 11.04 with Linux kernel 2.6.38
Small programs They took a tiny fraction of a second to analyze Correctness of analyses is manually verified
Large programs: one biology ontology from SILK KB size:
Flora-2 program with 4,774 rules and 919 facts Compiled into XSB’s 5,500+ rules and 1,000+ facts
Logs produced until evaluation consumed all memory Size: ~2GB Number of records: ~14M
Took 170 seconds
CONCLUSIONS Terminyzer – a tool for analyzing non-
termination Future work:
Implement auto-repair better auto-repair algorithms
Comparison with others: All other work deals with underpowered logic
engines that are so last Century (Prolog) Or with trying to find sufficient conditions for
termination (different focus)