Intro to MC Solver Interface Invariants Beyond Safety
SAT-Based Model Checking
Fabio Somenzi
Department of Electrical, Computer, and Energy Engineering
University of Colorado at Boulder
SAT-SMT Summer School, 14 June 2012
Intro to MC Solver Interface Invariants Beyond Safety
Outline
1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking
2 SAT Solver InterfaceTo The SolverFrom The Solver
3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification
4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL
Intro to MC Solver Interface Invariants Beyond Safety
Outline
1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking
2 SAT Solver InterfaceTo The SolverFrom The Solver
3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification
4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL
Intro to MC Solver Interface Invariants Beyond Safety
Simple Synchronous Arbiter (Verilog)
module arbsim (input clock, input [1:2] r, output reg [1:2] g);initial g <= 0;always @ (posedge clock) begin
g[1] <= r[1] & (∼r[2] | g[2]);g[2] <= r[2] & (∼r[1] | ∼g[1]);
endendmodule // arbsim
Intro to MC Solver Interface Invariants Beyond Safety
Mutual Exclusion for the Simple Arbiter
DFF
r1
r2
g1
g2
DFF
00
10
11
01
00
00
0110
10, 11
00
10, 11
1001, 11
01, 11
0001
I (g) = ¬g1 ∧ ¬g2
∃r1, r2 .T (r , g , g ′) = ¬g ′1 ∨ ¬g ′
2
P(g) = ¬g1 ∨ ¬g2
Intro to MC Solver Interface Invariants Beyond Safety
The Model Checking Question
Given a structure S and a property ϕ, is S a model of ϕ?
Written S |= ϕ
More in detail: does ϕ hold for all computations of S?
From all initial states
Intro to MC Solver Interface Invariants Beyond Safety
Finite-State Transition Systems
Symbolic representation of a system:
S : (i , x , I (x), T (i , x , x ′))
i : primary inputs
x : state variables
x ′: next state variables
I (x): initial states
T (i , x , x ′): transition relation
I and T define a finite transition structure (Kripke structure)
Every valuation of x is a state
∃i .T (i , x , x ′) = T (x , x ′) defines the transitions
Intro to MC Solver Interface Invariants Beyond Safety
Composition
Complex systems are composed of several modules
Each module is described as a finite state structure Si
The overall Kripke structure is obtained as the product of thestructures
State explosion!
The product can be either synchronous or asynchronous(interleaving)
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Temporal Logic Properties
G p: p is invariably true (always along all paths)
p is an atomic propositionG is a temporal operator
F p: p is inevitably true (sometimes true along all paths)
p U q: q eventually holds and p holds up until then
G(p → X q): every p is immediately followed by a q
Only allowed if time is discrete
GF(p → q): if p is persistent, then q is inevitable
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Temporal Logic Properties
G p: p is invariably true (always along all paths)
p is an atomic propositionG is a temporal operator
F p: p is inevitably true (sometimes true along all paths)
p U q: q eventually holds and p holds up until then
G(p → X q): every p is immediately followed by a q
Only allowed if time is discrete
GF(p → q): if p is persistent, then q is inevitable
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Temporal Logic Properties
G p: p is invariably true (always along all paths)
p is an atomic propositionG is a temporal operator
F p: p is inevitably true (sometimes true along all paths)
p U q: q eventually holds and p holds up until then
G(p → X q): every p is immediately followed by a q
Only allowed if time is discrete
GF(p → q): if p is persistent, then q is inevitable
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Temporal Logic Properties
G p: p is invariably true (always along all paths)
p is an atomic propositionG is a temporal operator
F p: p is inevitably true (sometimes true along all paths)
p U q: q eventually holds and p holds up until then
G(p → X q): every p is immediately followed by a q
Only allowed if time is discrete
GF(p → q): if p is persistent, then q is inevitable
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Temporal Logic Properties
G p: p is invariably true (always along all paths)
p is an atomic propositionG is a temporal operator
F p: p is inevitably true (sometimes true along all paths)
p U q: q eventually holds and p holds up until then
G(p → X q): every p is immediately followed by a q
Only allowed if time is discrete
GF(p → q): if p is persistent, then q is inevitable
Intro to MC Solver Interface Invariants Beyond Safety
Properties
Properties are sets of behaviors
Various specification mechanisms are in use: Temporal logicsand automata are popular
The examples we have seen are formulae of the temporal logicLTL (Linear-Time Logic)
Syntactic sugar often useful (e.g., PSL, Property SpecificationLanguage)
Intro to MC Solver Interface Invariants Beyond Safety
Properties
Properties are sets of behaviors
Various specification mechanisms are in use: Temporal logicsand automata are popular
The examples we have seen are formulae of the temporal logicLTL (Linear-Time Logic)
Syntactic sugar often useful (e.g., PSL, Property SpecificationLanguage)
Intro to MC Solver Interface Invariants Beyond Safety
Properties
Properties are sets of behaviors
Various specification mechanisms are in use: Temporal logicsand automata are popular
The examples we have seen are formulae of the temporal logicLTL (Linear-Time Logic)
Syntactic sugar often useful (e.g., PSL, Property SpecificationLanguage)
Intro to MC Solver Interface Invariants Beyond Safety
Properties
Properties are sets of behaviors
Various specification mechanisms are in use: Temporal logicsand automata are popular
The examples we have seen are formulae of the temporal logicLTL (Linear-Time Logic)
Syntactic sugar often useful (e.g., PSL, Property SpecificationLanguage)
Intro to MC Solver Interface Invariants Beyond Safety
Linear Time
Linear time logics reason about sets of computation paths
3
1 2
3
4
5
3 4
1
2
3
221
1
5
1
1
1
1
1
1
1
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time
Branching time logics reason about computation trees
5
1 2
3
4
5 1 2 2
1
1
2
3 4
3
Intro to MC Solver Interface Invariants Beyond Safety
Invariance, Safety, and Progress
Invariance properties say that certain states are unreachable
Reachability analysis
Safety properties say that certain events never happen
Generalize invariants and can be reduced to them
Progress properties are the non-safety properties
Cycle detection (for finite state systems)
This can be made (a lot) more formal
Why is G(p → X q) a safety property, but G(p → F q) is not?Borel hierarchy, (Landweber [1969])
Intro to MC Solver Interface Invariants Beyond Safety
Invariance, Safety, and Progress
Invariance properties say that certain states are unreachable
Reachability analysis
Safety properties say that certain events never happen
Generalize invariants and can be reduced to them
Progress properties are the non-safety properties
Cycle detection (for finite state systems)
This can be made (a lot) more formal
Why is G(p → X q) a safety property, but G(p → F q) is not?Borel hierarchy, (Landweber [1969])
Intro to MC Solver Interface Invariants Beyond Safety
Invariance, Safety, and Progress
Invariance properties say that certain states are unreachable
Reachability analysis
Safety properties say that certain events never happen
Generalize invariants and can be reduced to them
Progress properties are the non-safety properties
Cycle detection (for finite state systems)
This can be made (a lot) more formal
Why is G(p → X q) a safety property, but G(p → F q) is not?Borel hierarchy, (Landweber [1969])
Intro to MC Solver Interface Invariants Beyond Safety
Invariance, Safety, and Progress
Invariance properties say that certain states are unreachable
Reachability analysis
Safety properties say that certain events never happen
Generalize invariants and can be reduced to them
Progress properties are the non-safety properties
Cycle detection (for finite state systems)
This can be made (a lot) more formal
Why is G(p → X q) a safety property, but G(p → F q) is not?Borel hierarchy, (Landweber [1969])
Intro to MC Solver Interface Invariants Beyond Safety
Automata
Properties can be described by automata that take thecomputation of the system as input and either accept it orreject it
For non-terminating computations and linear-time propertieswe need ω-automata
For linear-time model checking we need the automaton for thenegation of the property of interest
Model checking reduced to checking language emptiness of anω-automaton
Intro to MC Solver Interface Invariants Beyond Safety
Automata
Properties can be described by automata that take thecomputation of the system as input and either accept it orreject it
For non-terminating computations and linear-time propertieswe need ω-automata
For linear-time model checking we need the automaton for thenegation of the property of interest
Model checking reduced to checking language emptiness of anω-automaton
Intro to MC Solver Interface Invariants Beyond Safety
Automata
Properties can be described by automata that take thecomputation of the system as input and either accept it orreject it
For non-terminating computations and linear-time propertieswe need ω-automata
For linear-time model checking we need the automaton for thenegation of the property of interest
Model checking reduced to checking language emptiness of anω-automaton
Intro to MC Solver Interface Invariants Beyond Safety
Omega-Automata
ω-automata describe linear-time properties
In fact, nondeterministic Buchi automata recognize allω-regular properties
Examples of Buchi automata (an accepting run visits someaccepting state infinitely often)
⊤
p q
p
p
p
⊤
⊤
They are more expressive than LTL
Intro to MC Solver Interface Invariants Beyond Safety
From Formula to Buchi Automaton
ψUϕ = ϕ ∨ [ψ ∧ X(ψ Uϕ)]
⊤
ϕ
ψ
Expansion produces a DNF whose every term is theconjunction of:
1 a propositional formula that must hold now and2 a temporal formula that must hold from the next step
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Branching Time Temporal Logic
Add path quantifiers to LTL to obtain CTL∗
A: for all pathsE: for at least one path
AGEF p: resetability
LTL is embedded in CTL∗ by prepending A to all formulae
AG(p → F q)
AG(p → F q) is equivalent to AG(p → AF q), but. . .
AFAG p is not equivalent to A FG p
Maidl [2000] for more info
In CTL every temporal operator must be immediatelypreceded by a path quantifier
Intro to MC Solver Interface Invariants Beyond Safety
Linear vs. Branching Time
Branching time is more powerful, but also trickier
ResetabilityA FGϕ vs. AFAGϕ
Structure equivalence is finer-grained for branching time:
Linear time ↔ language (trace) equivalenceBranching time ↔ simulation relations
Linear time is more suitable for compositional verification andBounded Model Checking
Counterexample generation simpler for linear time
Intro to MC Solver Interface Invariants Beyond Safety
Linear vs. Branching Time
Branching time is more powerful, but also trickier
ResetabilityA FGϕ vs. AFAGϕ
Structure equivalence is finer-grained for branching time:
Linear time ↔ language (trace) equivalenceBranching time ↔ simulation relations
Linear time is more suitable for compositional verification andBounded Model Checking
Counterexample generation simpler for linear time
Intro to MC Solver Interface Invariants Beyond Safety
Linear vs. Branching Time
Branching time is more powerful, but also trickier
ResetabilityA FGϕ vs. AFAGϕ
Structure equivalence is finer-grained for branching time:
Linear time ↔ language (trace) equivalenceBranching time ↔ simulation relations
Linear time is more suitable for compositional verification andBounded Model Checking
Counterexample generation simpler for linear time
Intro to MC Solver Interface Invariants Beyond Safety
Linear vs. Branching Time
Branching time is more powerful, but also trickier
ResetabilityA FGϕ vs. AFAGϕ
Structure equivalence is finer-grained for branching time:
Linear time ↔ language (trace) equivalenceBranching time ↔ simulation relations
Linear time is more suitable for compositional verification andBounded Model Checking
Counterexample generation simpler for linear time
Intro to MC Solver Interface Invariants Beyond Safety
Characteristic Functions
Let V contain all states with either the first or the last bit setto 1
χV = x1 ∨ xn
Set V has 3 · 2n−2 elements
Great, but let us not get carried away, because it is notpossible to find a representation that is compact for mostfunctions
Intro to MC Solver Interface Invariants Beyond Safety
Characteristic Functions
Let V contain all states with either the first or the last bit setto 1
χV = x1 ∨ xn
Set V has 3 · 2n−2 elements
Great, but let us not get carried away, because it is notpossible to find a representation that is compact for mostfunctions
Intro to MC Solver Interface Invariants Beyond Safety
Characteristic Functions
Let V contain all states with either the first or the last bit setto 1
χV = x1 ∨ xn
Set V has 3 · 2n−2 elements
Great, but let us not get carried away, because it is notpossible to find a representation that is compact for mostfunctions
Intro to MC Solver Interface Invariants Beyond Safety
Symbolic vs. Explicit Algorithms
Symbolic model checking uses characteristic functions
BDDs and CNF are the most popular choices
No explicit loop on the states or the transitions is used
Implicit enumeration
The cost of implicit enumeration is more affected by the sizeof the representation than by the cardinality of the set
Intro to MC Solver Interface Invariants Beyond Safety
Symbolic vs. Explicit Algorithms
Symbolic model checking uses characteristic functions
BDDs and CNF are the most popular choices
No explicit loop on the states or the transitions is used
Implicit enumeration
The cost of implicit enumeration is more affected by the sizeof the representation than by the cardinality of the set
Intro to MC Solver Interface Invariants Beyond Safety
Symbolic vs. Explicit Algorithms
Symbolic model checking uses characteristic functions
BDDs and CNF are the most popular choices
No explicit loop on the states or the transitions is used
Implicit enumeration
The cost of implicit enumeration is more affected by the sizeof the representation than by the cardinality of the set
Intro to MC Solver Interface Invariants Beyond Safety
Outline
1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking
2 SAT Solver InterfaceTo The SolverFrom The Solver
3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification
4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL
Intro to MC Solver Interface Invariants Beyond Safety
From Hardware Description Language to CNF
From source code to CDFG
From CDFG to formulae over bit vectors and finite-domainvariables
May involve abstraction
Bit-blasting (binary encoding) to Boolean circuit plus memoryelements
Optimization of Boolean circuit
Often uses And-Inverter Graphs (AIGs) or similar datastructures
Conversion of circuit to CNF
Intro to MC Solver Interface Invariants Beyond Safety
From Hardware Description Language to CNF
From source code to CDFG
From CDFG to formulae over bit vectors and finite-domainvariables
May involve abstraction
Bit-blasting (binary encoding) to Boolean circuit plus memoryelements
Optimization of Boolean circuit
Often uses And-Inverter Graphs (AIGs) or similar datastructures
Conversion of circuit to CNF
Intro to MC Solver Interface Invariants Beyond Safety
From Hardware Description Language to CNF
From source code to CDFG
From CDFG to formulae over bit vectors and finite-domainvariables
May involve abstraction
Bit-blasting (binary encoding) to Boolean circuit plus memoryelements
Optimization of Boolean circuit
Often uses And-Inverter Graphs (AIGs) or similar datastructures
Conversion of circuit to CNF
Intro to MC Solver Interface Invariants Beyond Safety
From Hardware Description Language to CNF
From source code to CDFG
From CDFG to formulae over bit vectors and finite-domainvariables
May involve abstraction
Bit-blasting (binary encoding) to Boolean circuit plus memoryelements
Optimization of Boolean circuit
Often uses And-Inverter Graphs (AIGs) or similar datastructures
Conversion of circuit to CNF
Intro to MC Solver Interface Invariants Beyond Safety
From Hardware Description Language to CNF
From source code to CDFG
From CDFG to formulae over bit vectors and finite-domainvariables
May involve abstraction
Bit-blasting (binary encoding) to Boolean circuit plus memoryelements
Optimization of Boolean circuit
Often uses And-Inverter Graphs (AIGs) or similar datastructures
Conversion of circuit to CNF
Intro to MC Solver Interface Invariants Beyond Safety
Distributivity
Apply (a ∧ b)∨ c = (a ∨ c)∧ (b ∨ c) systematically along withsimplifications
Preserves equivalence and does not introduce new variables
Size may blow up
(a ∧ b) ∨ (c ∧ d) = (a ∨ c) ∧ (a ∨ d) ∧ (b ∨ c) ∧ (b ∨ d)(x1 ∧ x2 ∧ x3) ∨ (x4 ∧ x5 ∧ x6) ∨ · · ·
Seldom applied in its pure form
Intro to MC Solver Interface Invariants Beyond Safety
Distributivity
Apply (a ∧ b)∨ c = (a ∨ c)∧ (b ∨ c) systematically along withsimplifications
Preserves equivalence and does not introduce new variables
Size may blow up
(a ∧ b) ∨ (c ∧ d) = (a ∨ c) ∧ (a ∨ d) ∧ (b ∨ c) ∧ (b ∨ d)(x1 ∧ x2 ∧ x3) ∨ (x4 ∧ x5 ∧ x6) ∨ · · ·
Seldom applied in its pure form
Intro to MC Solver Interface Invariants Beyond Safety
Distributivity
Apply (a ∧ b)∨ c = (a ∨ c)∧ (b ∨ c) systematically along withsimplifications
Preserves equivalence and does not introduce new variables
Size may blow up
(a ∧ b) ∨ (c ∧ d) = (a ∨ c) ∧ (a ∨ d) ∧ (b ∨ c) ∧ (b ∨ d)(x1 ∧ x2 ∧ x3) ∨ (x4 ∧ x5 ∧ x6) ∨ · · ·
Seldom applied in its pure form
Intro to MC Solver Interface Invariants Beyond Safety
Distributivity
Apply (a ∧ b)∨ c = (a ∨ c)∧ (b ∨ c) systematically along withsimplifications
Preserves equivalence and does not introduce new variables
Size may blow up
(a ∧ b) ∨ (c ∧ d) = (a ∨ c) ∧ (a ∨ d) ∧ (b ∨ c) ∧ (b ∨ d)(x1 ∧ x2 ∧ x3) ∨ (x4 ∧ x5 ∧ x6) ∨ · · ·
Seldom applied in its pure form
Intro to MC Solver Interface Invariants Beyond Safety
Equisatisfiability
Two formulae F and G are equisatisfiable if
1 F is satisfiable iff G is satisfiable.
2 If ηF (ηG ) is a satisfying assignment for F (G ), there exists asatisfying assignment ηG (ηF ) for G (F ) that agrees with ηF(ηG ) on all the variables that F and G have in common.
A common case occurs when one of the two formulae, say G ,contains all the variables in the other formula. Then a satisfyingassignment for F can be easily derived from one for G by droppingthe extra variables.
Intro to MC Solver Interface Invariants Beyond Safety
Tseitin
Use definitions for subformulae
f ↔ g ∨ h
g ↔ a ∧ b
h ↔ c ∧ d
Then, from (a ∧ b) ∨ (c ∧ d), we get
(a ∨ ¬g) ∧ (b ∨ ¬g) ∧ (¬a ∨ ¬b ∨ g)
∧ (c ∨ ¬h) ∧ (d ∨ ¬h) ∧ (¬c ∨ ¬d ∨ h)
∧ (¬g ∨ f ) ∧ (¬h ∨ f ) ∧ (g ∨ h ∨ ¬f ) ∧ f
Intro to MC Solver Interface Invariants Beyond Safety
Simpler Equisatisfiable CNF Formulae
If the formula is in negation normal form, Tseitin’s translation canbe simplified (Plaisted and Greenbaum [1986])
f → g ∨ h
g → a ∧ b
h → c ∧ d
Then, from (a ∧ b) ∨ (c ∧ d), we get
(a ∨ ¬g) ∧ (b ∨ ¬g)
∧ (c ∨ ¬h) ∧ (d ∨ ¬h)
∧ (g ∨ h ∨ ¬f ) ∧ f
Intro to MC Solver Interface Invariants Beyond Safety
More Conversions to CNF
Wilson, Sheridan
Nice DAGs
Cut-based
BDD-based
SAT preprocessor
Intro to MC Solver Interface Invariants Beyond Safety
More Conversions to CNF
Wilson, Sheridan
Nice DAGs
Cut-based
BDD-based
SAT preprocessor
Intro to MC Solver Interface Invariants Beyond Safety
Proofs of Unsatisfiability
Different verification techniques require
Resolution proofs
UNSAT cores
Assumptions (unit clauses) in UNSAT cores
Can be extracted with minimal overhead (Een and Sorensson[2003])
Intro to MC Solver Interface Invariants Beyond Safety
Incremental Solving
Solve sequences of related SAT instances
Ability to push and pop clauses (efficiently)
Keep learned clauses that are still valid
All learned clauses remain valid if no clause is popped
Keep variable scores
Multiple solver objects
Intro to MC Solver Interface Invariants Beyond Safety
Outline
1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking
2 SAT Solver InterfaceTo The SolverFrom The Solver
3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification
4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
A technique to falsify invariants (“bug finding”)
Based on unrolling the transition relation
Looks for counterexamples of certain lengths
Foundation for complete methods
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
A technique to falsify invariants (“bug finding”)
Based on unrolling the transition relation
Looks for counterexamples of certain lengths
Foundation for complete methods
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
A technique to falsify invariants (“bug finding”)
Based on unrolling the transition relation
Looks for counterexamples of certain lengths
Foundation for complete methods
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
A technique to falsify invariants (“bug finding”)
Based on unrolling the transition relation
Looks for counterexamples of certain lengths
Foundation for complete methods
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
Checks for a counterexample to a property of a model
We assume finite state
Encodes the property checking problem as propositionalsatisfiability (SAT)
Constructs a propositional formula that is satisfiable iff thereexits a length-k counterexample, e.g.,
I (x0) ∧∧
0≤i<k
T (i i , x i , x i+1) ∧ ¬P(xk)
If no counterexample is found, BMC increases k until
a counterexample is found,the search becomes intractable, ork reaches a certain bound
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
Checks for a counterexample to a property of a model
We assume finite state
Encodes the property checking problem as propositionalsatisfiability (SAT)
Constructs a propositional formula that is satisfiable iff thereexits a length-k counterexample, e.g.,
I (x0) ∧∧
0≤i<k
T (i i , x i , x i+1) ∧ ¬P(xk)
If no counterexample is found, BMC increases k until
a counterexample is found,the search becomes intractable, ork reaches a certain bound
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
Checks for a counterexample to a property of a model
We assume finite state
Encodes the property checking problem as propositionalsatisfiability (SAT)
Constructs a propositional formula that is satisfiable iff thereexits a length-k counterexample, e.g.,
I (x0) ∧∧
0≤i<k
T (i i , x i , x i+1) ∧ ¬P(xk)
If no counterexample is found, BMC increases k until
a counterexample is found,the search becomes intractable, ork reaches a certain bound
Intro to MC Solver Interface Invariants Beyond Safety
Bounded Model Checking
Checks for a counterexample to a property of a model
We assume finite state
Encodes the property checking problem as propositionalsatisfiability (SAT)
Constructs a propositional formula that is satisfiable iff thereexits a length-k counterexample, e.g.,
I (x0) ∧∧
0≤i<k
T (i i , x i , x i+1) ∧ ¬P(xk)
If no counterexample is found, BMC increases k until
a counterexample is found,the search becomes intractable, ork reaches a certain bound
Intro to MC Solver Interface Invariants Beyond Safety
Proving Properties with BMC
The original BMC algorithm (Biere et al. [1999]), althoughcomplete for finite state, is limited in practice to falsification
BMC can prove that an invariant ψ holds on a model S only ifa bound, κ, is known such that:
if no counterexample of length up to κ is found, then S |= ψ
Several methods exist to compute a suitable κ
The optimum value of κ, however, is usually very expensive toobtain
Finding it is at least as hard as checking whether S |= ψ
(Clarke et al. [2004])
Intro to MC Solver Interface Invariants Beyond Safety
Proving Properties with BMC
The original BMC algorithm (Biere et al. [1999]), althoughcomplete for finite state, is limited in practice to falsification
BMC can prove that an invariant ψ holds on a model S only ifa bound, κ, is known such that:
if no counterexample of length up to κ is found, then S |= ψ
Several methods exist to compute a suitable κ
The optimum value of κ, however, is usually very expensive toobtain
Finding it is at least as hard as checking whether S |= ψ
(Clarke et al. [2004])
Intro to MC Solver Interface Invariants Beyond Safety
Proving Properties with BMC
The original BMC algorithm (Biere et al. [1999]), althoughcomplete for finite state, is limited in practice to falsification
BMC can prove that an invariant ψ holds on a model S only ifa bound, κ, is known such that:
if no counterexample of length up to κ is found, then S |= ψ
Several methods exist to compute a suitable κ
The optimum value of κ, however, is usually very expensive toobtain
Finding it is at least as hard as checking whether S |= ψ
(Clarke et al. [2004])
Intro to MC Solver Interface Invariants Beyond Safety
Proving Properties with BMC
The original BMC algorithm (Biere et al. [1999]), althoughcomplete for finite state, is limited in practice to falsification
BMC can prove that an invariant ψ holds on a model S only ifa bound, κ, is known such that:
if no counterexample of length up to κ is found, then S |= ψ
Several methods exist to compute a suitable κ
The optimum value of κ, however, is usually very expensive toobtain
Finding it is at least as hard as checking whether S |= ψ
(Clarke et al. [2004])
Intro to MC Solver Interface Invariants Beyond Safety
Finding The Bound κ
Compute diameter of graph
Minimum d such that, if there is a path of length d + 1between two states, then there is a path of length at most dbetween the same states∀x0, . . . , xd+1 .
∧0≤i≤d T (x i , x i+1) →
∃x ′0, . . . , x′d .(
∧0≤i<d T (x ′i , x
′i+1)∧x
′0 = x0∧
∨0≤i≤d x
′i = xd+1)
If one end of the path is constrained to an initial (target)state, one obtains the forward (backward) recursive radius ofthe graph
Restrict search to simple paths (next slide)
Intro to MC Solver Interface Invariants Beyond Safety
Finding The Bound κ
Compute diameter of graph
Minimum d such that, if there is a path of length d + 1between two states, then there is a path of length at most dbetween the same states∀x0, . . . , xd+1 .
∧0≤i≤d T (x i , x i+1) →
∃x ′0, . . . , x′d .(
∧0≤i<d T (x ′i , x
′i+1)∧x
′0 = x0∧
∨0≤i≤d x
′i = xd+1)
If one end of the path is constrained to an initial (target)state, one obtains the forward (backward) recursive radius ofthe graph
Restrict search to simple paths (next slide)
Intro to MC Solver Interface Invariants Beyond Safety
Finding The Bound κ
Compute diameter of graph
Minimum d such that, if there is a path of length d + 1between two states, then there is a path of length at most dbetween the same states∀x0, . . . , xd+1 .
∧0≤i≤d T (x i , x i+1) →
∃x ′0, . . . , x′d .(
∧0≤i<d T (x ′i , x
′i+1)∧x
′0 = x0∧
∨0≤i≤d x
′i = xd+1)
If one end of the path is constrained to an initial (target)state, one obtains the forward (backward) recursive radius ofthe graph
Restrict search to simple paths (next slide)
Intro to MC Solver Interface Invariants Beyond Safety
Simple Paths
A counterexample to an invariant is a finite prefix path to astate that satisfies ¬P (bad state)
If a counterexample exists, then there is a simple path from aninitial state to a bad state that goes through no other initialor bad state
An invariant holds (Sheeran et al. [2000]) if:
there is no counterexample of length k to ¬P , andno simple path of length k + 1 to ¬P that does not gothrough any other states satisfying ¬P , orno simple path of length k + 1 from an initial state that doesnot go through any other initial states
Intro to MC Solver Interface Invariants Beyond Safety
Simple Paths
A counterexample to an invariant is a finite prefix path to astate that satisfies ¬P (bad state)
If a counterexample exists, then there is a simple path from aninitial state to a bad state that goes through no other initialor bad state
An invariant holds (Sheeran et al. [2000]) if:
there is no counterexample of length k to ¬P , andno simple path of length k + 1 to ¬P that does not gothrough any other states satisfying ¬P , orno simple path of length k + 1 from an initial state that doesnot go through any other initial states
Intro to MC Solver Interface Invariants Beyond Safety
Simple Paths
A counterexample to an invariant is a finite prefix path to astate that satisfies ¬P (bad state)
If a counterexample exists, then there is a simple path from aninitial state to a bad state that goes through no other initialor bad state
An invariant holds (Sheeran et al. [2000]) if:
there is no counterexample of length k to ¬P , andno simple path of length k + 1 to ¬P that does not gothrough any other states satisfying ¬P , orno simple path of length k + 1 from an initial state that doesnot go through any other initial states
Intro to MC Solver Interface Invariants Beyond Safety
Checking for Simple Paths
Simple-minded check produces quadratic formula
∧
0<i≤k
∧
0≤j<i
(x i 6= x j)
Using a bitonic sorting network (Kroning and Strichman[2003]) reduces the complexity to O(k log2 k)
Lazy checking is more effective in practice (Sorensson’s thesis)
Intro to MC Solver Interface Invariants Beyond Safety
Checking for Simple Paths
Simple-minded check produces quadratic formula
∧
0<i≤k
∧
0≤j<i
(x i 6= x j)
Using a bitonic sorting network (Kroning and Strichman[2003]) reduces the complexity to O(k log2 k)
Lazy checking is more effective in practice (Sorensson’s thesis)
Intro to MC Solver Interface Invariants Beyond Safety
Checking for Simple Paths
Simple-minded check produces quadratic formula
∧
0<i≤k
∧
0≤j<i
(x i 6= x j)
Using a bitonic sorting network (Kroning and Strichman[2003]) reduces the complexity to O(k log2 k)
Lazy checking is more effective in practice (Sorensson’s thesis)
Intro to MC Solver Interface Invariants Beyond Safety
k-Induction
Sheeran et al. call their method k-induction
If all states on length-k paths from the initial states satisfy p,and
k consecutive states satisfying p are always followed by a statesatisfying p, then
all states reachable from the initial states satisfy p
The second premise is verified when there are no simple pathsof length k + 1
Intro to MC Solver Interface Invariants Beyond Safety
Abstraction Refinement
Assume abstract model Sa and abstraction of property ϕa
such that Sa |= ϕa implies S |= ϕ
Use complete method on abstract model Sa, but use BMC onthe concrete model S when a counterexample is found in Sa
Use the counterexample(s) found in Sa to constrain search in S
If concretization fails, use UNSAT core to refine abstractionOne-to-one and one-to-many concretization possible
It is possible to reverse the order: proof-based abstraction(Amla and McMillan [2004])
Use BMC and periodically extract abstract model fromUNSAT core and check it with complete model
Intro to MC Solver Interface Invariants Beyond Safety
Abstraction Refinement
Assume abstract model Sa and abstraction of property ϕa
such that Sa |= ϕa implies S |= ϕ
Use complete method on abstract model Sa, but use BMC onthe concrete model S when a counterexample is found in Sa
Use the counterexample(s) found in Sa to constrain search in S
If concretization fails, use UNSAT core to refine abstractionOne-to-one and one-to-many concretization possible
It is possible to reverse the order: proof-based abstraction(Amla and McMillan [2004])
Use BMC and periodically extract abstract model fromUNSAT core and check it with complete model
Intro to MC Solver Interface Invariants Beyond Safety
Abstraction Refinement
Assume abstract model Sa and abstraction of property ϕa
such that Sa |= ϕa implies S |= ϕ
Use complete method on abstract model Sa, but use BMC onthe concrete model S when a counterexample is found in Sa
Use the counterexample(s) found in Sa to constrain search in S
If concretization fails, use UNSAT core to refine abstractionOne-to-one and one-to-many concretization possible
It is possible to reverse the order: proof-based abstraction(Amla and McMillan [2004])
Use BMC and periodically extract abstract model fromUNSAT core and check it with complete model
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation (McMillan [2003])
SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable
Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)
Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
Interpolant I1(x1) is such that
F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable
I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable
∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant
set of states reachable from I (x0) in one step
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation
I1(x1) is a superset of the states reachable in one step suchthat no member state has a path of length k − 1 to a badstate
Replace I (x0) with I (x0) ∨ I1(x0) and repeat
If formula still unsatisfiable, interpolant I2(x1) is a superset ofstates reachable in one or two steps such that no memberstate has a path of length k − 1 to a bad state
A converging sequence of interpolants means that no statessatisfying ¬p (bad states) are reachable
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation
I1(x1) is a superset of the states reachable in one step suchthat no member state has a path of length k − 1 to a badstate
Replace I (x0) with I (x0) ∨ I1(x0) and repeat
If formula still unsatisfiable, interpolant I2(x1) is a superset ofstates reachable in one or two steps such that no memberstate has a path of length k − 1 to a bad state
A converging sequence of interpolants means that no statessatisfying ¬p (bad states) are reachable
Intro to MC Solver Interface Invariants Beyond Safety
Interpolation
I1(x1) is a superset of the states reachable in one step suchthat no member state has a path of length k − 1 to a badstate
Replace I (x0) with I (x0) ∨ I1(x0) and repeat
If formula still unsatisfiable, interpolant I2(x1) is a superset ofstates reachable in one or two steps such that no memberstate has a path of length k − 1 to a bad state
A converging sequence of interpolants means that no statessatisfying ¬p (bad states) are reachable
Intro to MC Solver Interface Invariants Beyond Safety
Preimage Computation by Solution Enumeration
Let Pre(Q(x)) be the predicate describing the states that arepredecessors of the states described by Q
Repeated application of Pre from ¬P corresponds tobackward breadth-first search from the error states
Common approach with BDDs
Can be adapted to CNF (McMillan [2002])
Introduced the use of blocking clauses
Intro to MC Solver Interface Invariants Beyond Safety
Preimage Computation by Solution Enumeration
Let Pre(Q(x)) be the predicate describing the states that arepredecessors of the states described by Q
Repeated application of Pre from ¬P corresponds tobackward breadth-first search from the error states
Common approach with BDDs
Can be adapted to CNF (McMillan [2002])
Introduced the use of blocking clauses
Intro to MC Solver Interface Invariants Beyond Safety
Preimage Computation by Solution Enumeration
Let Pre(Q(x)) be the predicate describing the states that arepredecessors of the states described by Q
Repeated application of Pre from ¬P corresponds tobackward breadth-first search from the error states
Common approach with BDDs
Can be adapted to CNF (McMillan [2002])
Introduced the use of blocking clauses
Intro to MC Solver Interface Invariants Beyond Safety
Preimage Computation by Solution Enumeration
Let Pre(Q(x)) be the predicate describing the states that arepredecessors of the states described by Q
Repeated application of Pre from ¬P corresponds tobackward breadth-first search from the error states
Common approach with BDDs
Can be adapted to CNF (McMillan [2002])
Introduced the use of blocking clauses
Intro to MC Solver Interface Invariants Beyond Safety
Back to The Simple Arbiter
00
10
11
01
00
00
0110
10, 11
00
10, 11
1001, 11
01, 11
0001
I (g) = ¬g1 ∧ ¬g2
∃r1, r2 .T (r , g , g ′) = ¬g ′1 ∨ ¬g ′
2
P(g) = ¬g1 ∨ ¬g2
Intro to MC Solver Interface Invariants Beyond Safety
Inductive Proofs for Transition Systems
Prove initiation (base case)
I (x) ⇒ P(x)All initial states satisfy P
(¬g1 ∧ ¬g2) ⇒ (¬g1 ∨ ¬g2)
Prove consecution (inductive step)
P(x) ∧ T (i , x , x ′) ⇒ P(x ′)All successors of states satisfying P satisfy P
(¬g1 ∨ ¬g2) ∧ (¬g ′1 ∨ ¬g ′
2) ⇒ (¬g ′1 ∨ ¬g ′
2)
If both pass, all reachable states satisfy the property
S |= P
Intro to MC Solver Interface Invariants Beyond Safety
Visualizing Inductive Proofs
00
10
11
01
The inductive assertion (yellow) contains all initial (blue) statesand no arrow leaves it (it is closed under the transition relation)
Intro to MC Solver Interface Invariants Beyond Safety
Counterexamples to Induction: The Troublemakers
00 01 11 10
Intro to MC Solver Interface Invariants Beyond Safety
Counterexamples to Induction: The Troublemakers
00 01 11 10
CTI
Intro to MC Solver Interface Invariants Beyond Safety
Invariant Strengthening
00 01 11 10
CTI
Intro to MC Solver Interface Invariants Beyond Safety
Invariant Strengthening
00 01 11 10
Intro to MC Solver Interface Invariants Beyond Safety
Invariant Strengthening
00 01 11 10
Intro to MC Solver Interface Invariants Beyond Safety
Invariant Strengthening
00 01 11 10
Intro to MC Solver Interface Invariants Beyond Safety
Strong and Weak Invariants
000 001
011010
110
111
100
101
Induction is not restricted to:
the strongest inductive invariant (forward-reachable states)
. . . or the weakest inductive invariant (complement of thebackward-reachable states)
¬x1 is simpler than ¬x1 ∧ (¬x2 ∨ ¬x3) (strongest) and(¬x1 ∨ ¬x3) (weakest)
Intro to MC Solver Interface Invariants Beyond Safety
Completeness for Finite-State Systems
CTIs are effectively bad states
If a CTI is reachable so is at least one bad state
Remove CTI from P and try again
Eventually either:
An inductive strengthening of P resultsAn initial state is removed from P
In the latter case, a counterexample is obtained
Intro to MC Solver Interface Invariants Beyond Safety
Examples of Strengthening Strategies
Removing one CTI at a time is very inefficient!
Several strategies in use to avoid that
Fixpoint-based invariant checking: if νZ . p ∧ AXZ convergesin n > 0 iterations, then
∧0≤i<n AX
i p is an inductiveinvariant
In fact, the weakest inductive invariant
k-induction: if all states on length-k paths from the initialstates satisfy p, and k distinct consecutive states satisfying p
are always followed by a state satisfying p, then all statesreachable from the initial states satisfy p.
fsis algorithm: try to extract an inductive clause from CTI toexclude multiple CTIs
Intro to MC Solver Interface Invariants Beyond Safety
Relative Induction
010 000
001011
100
101
110
111
ϕ = ¬x1 ∧ (x1 ∨ ¬x2)
Intro to MC Solver Interface Invariants Beyond Safety
Relative Induction
010 000
001011
100
101
110
111
¬x1 is not inductive
Intro to MC Solver Interface Invariants Beyond Safety
Relative Induction
010 000
001011
100
101
110
111
x1 ∨ ¬x2 is inductive
Intro to MC Solver Interface Invariants Beyond Safety
Relative Induction
010 000
001011
100
101
110
111
¬x1 is inductive relative to x1 ∨ ¬x2
Intro to MC Solver Interface Invariants Beyond Safety
Shortcoming of Relative Induction
010 100
101011
000
001
110
111
P = (x1 ∨ x2 ∨ x3) ∧ (¬x1 ∨ ¬x2 ∨ x3)
ϕ = (x1 ∨ x2) ∧ (¬x1 ∨ ¬x2)
Intro to MC Solver Interface Invariants Beyond Safety
Shortcoming of Relative Induction
010 100
101011
000
001
110
111
(x1 ∨ x2) ∧ P ∧ T 6⇒ (x ′1 ∨ x ′2)
Intro to MC Solver Interface Invariants Beyond Safety
Shortcoming of Relative Induction
010 100
101011
000
001
110
111
(¬x1 ∨ ¬x2) ∧ P ∧ T 6⇒ (¬x ′1 ∨ ¬x ′2)
Intro to MC Solver Interface Invariants Beyond Safety
Shortcoming of Relative Induction
010 100
101011
000
001
110
111
(x1 ∨ x2) ∧ (¬x1 ∨ ¬x2) ∧ P ∧ T ⇒ (x ′1 ∨ x ′2) ∧ (¬x ′1 ∨ ¬x ′2)
Intro to MC Solver Interface Invariants Beyond Safety
Shortcoming of Relative Induction
010 100
101011
000
001
110
111
(x1 ∨ x2) and (¬x1 ∨ ¬x2) are mutually inductive
Intro to MC Solver Interface Invariants Beyond Safety
IC3: Basic Algorithm
IC3 (Bradley [2011]) stands for
1 Incremental Construction of
2 Inductive Clauses for
3 Indubitable Correctness
IC3 is an Incremental Inductive Verification (IIV) algorithm
Intro to MC Solver Interface Invariants Beyond Safety
Basic Tenets
Approximate reachability assumptions
Fi : contains at least all the states reachable in i steps or lessIf S |= P , Fi eventually becomes inductive for some i
Approximation is desirable: IC3 does not attempt to get themost precise Fi ’s
Stepwise relative induction
Learn useful facts via induction relative to reachabilityassumptions
Clausal representation
Learn clauses (lemmas) from CTIsA form of abstract interpretation
Intro to MC Solver Interface Invariants Beyond Safety
IC3 Invariants
The four main invariants of IC3:
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Established if there are no counterexamples of length 0 or 1
The implicit invariant of the outer loop: no counterexamplesof length k or less
Intro to MC Solver Interface Invariants Beyond Safety
Reasonable Invariants
I ⇒ F0: F0 overapproximates the initial condition. (Inpractice, I = F0.)
Fi ⇒ Fi+1: a state believed to be reachable in i steps or lessis also believed to be reachable in i + 1 steps or less
Fi ⇒ P : no state believed to be reachable in i steps or lessviolates P
Fi ∧ T ⇒ F ′i+1: all the immediate successors of a state
believed to be reachable in i steps or less are believed to bereachable in i + 1 steps or less
Intro to MC Solver Interface Invariants Beyond Safety
Pseudo-Pseudocode
bool IC3 {if (I 6⇒ P or I ∧ T 6⇒ P ′)
return ⊥F0 = I ; F1 = P ; k = 1repeat {
while (there are CTIs in Fk) {either find a counterexample and return ⊥or refine F1, . . . ,Fk
}k ++set Fk = P and propagate clausesif (Fi = Fi+1 for some 0 < i < k)
return ⊤}
}
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No counterexamples of length 0 or 1
00 01 11 10I = ¬x1 ∧ ¬x2
P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Does F1 ∧ T ⇒ P ′?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Found CTI s = x1 ∧ x2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F1?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No. Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F0?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Yes. Generalize ¬s at level 0 in one of the two possible ways: either¬x1 or ¬x2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Update F1
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No more CTIs in F1. No counterexamples of length 2. InstantiateF2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x2
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Propagate clauses from F1 to F2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x2
F2 = (¬x1 ∨ x2) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
F1 and F2 are identical. Property proved
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x2
F2 = (¬x1 ∨ x2) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
What happens if we generalize ¬s = ¬x1 ∨ ¬x2 at level 0 in theother way (¬x1)?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Update F1
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x1
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No more CTIs in F1. No counterexamples of length 2. InstantiateF2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No clauses propagate from F1 to F2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = (¬x1 ∨ x2) ∧ ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Remove subsumed clauses
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Does F2 ∧ T ⇒ P ′?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Found CTI s = x1 ∧ x2 (same as before)
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F1?
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
No. We know it is inductive at level 0.
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
If generalization produces ¬x1 again, the CTI is not eliminated
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Find predecessor t of CTI x1 ∧ x2 in F1 \ F0
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Found t = ¬x1 ∧ x2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
The clause ¬t = x1 ∨ ¬x2 is inductive at all levels
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Generalization of ¬t = x1 ∨ ¬x2 produces ¬x2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1
F2 = P = ¬x1 ∨ x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
Update F1 and F2
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1 ∧ ¬x2
F2 = (¬x1 ∨ x2) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Passing Property
F1 and F2 are equivalent. Property (almost) proved
00 01 11 10F0 = I = ¬x1 ∧ ¬x2
F1 = ¬x1 ∧ ¬x2
F2 = (¬x1 ∨ x2) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
No counterexamples of length 0 or 1
000
001
100
101
110
111
011
010
I = ¬x1 ∧ ¬x3 ∧ ¬x3
P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Does F1 ∧ T ⇒ P ′?
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Found CTI s = ¬x1 ∧ x2 ∧ x3
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
The clause ¬s = x1 ∨ ¬x2 ∨ ¬x3 generalizes to ¬x2 at level 0
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = (¬x1 ∨ ¬x2 ∨ ¬x3) ∧ ¬x2
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
No CTI left: no counterexample of length 2. F2 instantiated, butno clause propagated
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
The clause ¬s = x1 ∨ ¬x2 ∨ ¬x3 generalizes again to ¬x2 at level 0
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Suppose IC3 recurs on t = ¬x1 ∧ ¬x2 ∧ x3 in F1 \ F0
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Clause ¬t = x1 ∨ x2 ∨ ¬x3 is not inductive at level 0: the propertyfails
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Suppose now IC3 recurs on t = x1 ∧ ¬x2 ∧ x3 in F1 \ F0
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Clause ¬t = ¬x1 ∨ x2 ∨ ¬x3 is inductive at level 1
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2
F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Generalization of ¬t adds ¬x1 to F1 and F2
000
001
100
101
110
111
011
010
F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3
F1 = ¬x2 ∧ ¬x1
F2 = (¬x1 ∨ ¬x2 ∨ ¬x3) ∧ ¬x1
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
Only t = ¬x1 ∧ ¬x2 ∧ x3 remains in F1 \ F0
000
001
100
101
110
111
011
010
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Example: Failing Property
The same counterexample as before is found
000
001
100
101
110
111
011
010
I ⇒ F0
Fi ⇒ Fi+1 0 ≤ i < k
Fi ⇒ P 0 ≤ i ≤ k
Fi ∧ T ⇒ F ′i+1 0 ≤ i < k
Intro to MC Solver Interface Invariants Beyond Safety
Clause Generalization
A CTI is a cube (conjunction of literals)
e.g., s = x1 ∧ ¬x2 ∧ x3
The negation of a CTI is a clause
e.g., ¬s = ¬x1 ∨ x2 ∨ ¬x3
Conjoining ¬s to a reachability assumption Fi excludes theCTI from it
Generalization extracts a subclause from ¬s that excludesmore states that are “like the CTI”
e.g., ¬x3 may be a subclause of ¬s that excludes states that,like the CTI, are not reachable in i stepsEvery literal dropped doubles the number of states excluded bya clauseGeneralization is time-consuming, but critical to performance
Intro to MC Solver Interface Invariants Beyond Safety
Generalization
Crucial for efficiency
Generalization in IC3 produces a minimal inductive clause(MIC)
The MIC algorithm is based on DOWN and UP.
DOWN extracts the (unique) maximal subclause
UP finds a small, but not necessarily minimal subclause
MIC recurs on subclauses of the result of UP
Intro to MC Solver Interface Invariants Beyond Safety
Minimal Inductive Clause
234
1234
123 124
13 14 23 24
4
⊥
1 2 3
3412
134
Intro to MC Solver Interface Invariants Beyond Safety
Minimal Inductive Clause
234
1234
123 124
13 14 23 24
4
⊥
1 2 3
3412
134
Intro to MC Solver Interface Invariants Beyond Safety
Minimal Inductive Clause
234
1234
123 124
13 14 23 24
4
⊥
1 2 3
3412
134
Intro to MC Solver Interface Invariants Beyond Safety
Minimal Inductive Clause
234
1234
123 124
13 14 23 24
4
⊥
1 2 3
3412
134
Intro to MC Solver Interface Invariants Beyond Safety
Minimal Inductive Clause
234
1234
123 124
13 14 23 24
4
⊥
1 2 3
3412
134
Intro to MC Solver Interface Invariants Beyond Safety
Maximal Inductive Subclause (DOWN)
000
101
¬x1 ∨ x2 ∨ ¬x3
Intro to MC Solver Interface Invariants Beyond Safety
Maximal Inductive Subclause (DOWN)
000
101
¬x1 ∨ x2 ∨ ¬x3
001
Intro to MC Solver Interface Invariants Beyond Safety
Maximal Inductive Subclause (DOWN)
000
101
x2 ∨ ¬x3
001
Intro to MC Solver Interface Invariants Beyond Safety
Maximal Inductive Subclause (DOWN)
000
101
x2 ∨ ¬x3
001
100
Intro to MC Solver Interface Invariants Beyond Safety
Maximal Inductive Subclause (DOWN)
000
101
x2
001
100
Intro to MC Solver Interface Invariants Beyond Safety
Use of UNSAT Cores
¬s ∧ Fi ∧ T ⇒ ¬s ′ if and only if ¬s ∧ Fi ∧ T ∧ s ′ isunsatisfiable
The literals of s ′ are (unit) clauses in the SAT query
If the implication holds, the SAT solver returns anunsatisfiable core
Any literal of s ′ not in the core can be removed from s ′
because it does not contribute to the implication . . .
and from ¬s because strengthening the antecedent preservesthe implication
Intro to MC Solver Interface Invariants Beyond Safety
Use of UNSAT Core Example
¬s ∧ F0 ∧ T ⇒ ¬s ′ with
¬s = ¬x1 ∨ ¬x2
F0 = ¬x1 ∧ ¬x2
T = (¬x1 ∧ ¬x2 ∧ ¬x ′1 ∧ ¬x ′2) ∨ · · ·
The SAT query, after some simplification, is
¬x1 ∧ ¬x2 ∧ ¬x ′1 ∧ ¬x ′2 ∧ x ′1 ∧ x ′2
Two UNSAT cores are
¬x ′1 ∧ x ′1
¬x ′2 ∧ x ′2
from which the two generalizations we saw before follow
Intro to MC Solver Interface Invariants Beyond Safety
Clause Clean-Up
As IC3 proceeds, clauses may be added to some Fi thatsubsume other clauses
The weaker, subsumed clauses no longer contribute to thedefinition of Fi
However, a weaker clause may propagate to Fi+1 when thestronger clause does not
Weak clauses are eliminated by subsumption only betweenmajor iterations and after propagation
Intro to MC Solver Interface Invariants Beyond Safety
More Efficiency-Related Issues
State encoding determines what clauses are derived
Incremental vs. monolithic
Reachability assumptions carry global information. . . but are built incrementally
Semantic vs. syntactic approach
Generalization “jumps over large distances”
Long counterexamples at low k
Typically more efficient than increasing k
Consequences of no unrolling
Many cheap (incremental) SAT calls
Ability to parallelize
Clauses are easy to exchange
Intro to MC Solver Interface Invariants Beyond Safety
Outline
1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking
2 SAT Solver InterfaceTo The SolverFrom The Solver
3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification
4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Translation from LTL
Various techniques have been devised to translate an LTLformula ϕ into a propositional formula that expresses theconstraints on a path that is a model of ¬ϕ. For instance:
[[¬FG¬p]] =∨
0≤l≤k
(T (xk , x l) ∧∨
l≤i≤k
p(x i))
k-induction can be extended to provide a termination criterion
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Translation from LTL
Various techniques have been devised to translate an LTLformula ϕ into a propositional formula that expresses theconstraints on a path that is a model of ¬ϕ. For instance:
[[¬FG¬p]] =∨
0≤l≤k
(T (xk , x l) ∧∨
l≤i≤k
p(x i))
k-induction can be extended to provide a termination criterion
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Liveness to Safety
Checking progress properties requires cycle detection
Augment model with shadow register
The augmented model can nondeterministically save asnapshot of the current state in the shadow register
If a state is subsequently reached that is identical to the onesaved, a cycle has been detected
Constraints can be added to make sure the cycle is anaccepting one
With this transformation an invariant checker suffices for allLTL properties
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Liveness to Safety
Checking progress properties requires cycle detection
Augment model with shadow register
The augmented model can nondeterministically save asnapshot of the current state in the shadow register
If a state is subsequently reached that is identical to the onesaved, a cycle has been detected
Constraints can be added to make sure the cycle is anaccepting one
With this transformation an invariant checker suffices for allLTL properties
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Liveness to Safety
Checking progress properties requires cycle detection
Augment model with shadow register
The augmented model can nondeterministically save asnapshot of the current state in the shadow register
If a state is subsequently reached that is identical to the onesaved, a cycle has been detected
Constraints can be added to make sure the cycle is anaccepting one
With this transformation an invariant checker suffices for allLTL properties
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Liveness to Safety
Checking progress properties requires cycle detection
Augment model with shadow register
The augmented model can nondeterministically save asnapshot of the current state in the shadow register
If a state is subsequently reached that is identical to the onesaved, a cycle has been detected
Constraints can be added to make sure the cycle is anaccepting one
With this transformation an invariant checker suffices for allLTL properties
Intro to MC Solver Interface Invariants Beyond Safety
BMC: Liveness to Safety
Checking progress properties requires cycle detection
Augment model with shadow register
The augmented model can nondeterministically save asnapshot of the current state in the shadow register
If a state is subsequently reached that is identical to the onesaved, a cycle has been detected
Constraints can be added to make sure the cycle is anaccepting one
With this transformation an invariant checker suffices for allLTL properties
Intro to MC Solver Interface Invariants Beyond Safety
FAIR: Finding Rechable Fair Cycles
Check language nonemptiness of the composition of structureS and generalized Buchi automaton for ¬ϕ
Generalized means that multiple acceptance conditions (akafairness constraints may be given: each must be satisfied
FAIR (Bradley et al. [2011]) looks for a reachable fair cycle
Intro to MC Solver Interface Invariants Beyond Safety
FAIR: Finding Rechable Fair Cycles
Check language nonemptiness of the composition of structureS and generalized Buchi automaton for ¬ϕ
Generalized means that multiple acceptance conditions (akafairness constraints may be given: each must be satisfied
FAIR (Bradley et al. [2011]) looks for a reachable fair cycle
Intro to MC Solver Interface Invariants Beyond Safety
FAIR: Finding Rechable Fair Cycles
Check language nonemptiness of the composition of structureS and generalized Buchi automaton for ¬ϕ
Generalized means that multiple acceptance conditions (akafairness constraints may be given: each must be satisfied
FAIR (Bradley et al. [2011]) looks for a reachable fair cycle
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
Strongly Connected Components
A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints
A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph
A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it
A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .
Every cycle of a graph is contained in some SCC-closed set
Maintain a partition of the states into SCC-closed set
Refine it until a reachable fair cycle is found or none is provedto exist
Intro to MC Solver Interface Invariants Beyond Safety
FAIR: Finding Reachable Fair Cycles
Reduce search for reachable fair cycle to a set of safety problems:
Skeleton:•
◦ •
•
States of skeleton together satisfy all fairness constraints.
Task: Connect states to form lasso.•
◦ •
•
Intro to MC Solver Interface Invariants Beyond Safety
Reach Queries
Each connection task is a reach query.
Stem query: Connect initial condition to a state:
•
◦ •
•
Cycle query: Connect one state to another:
•
◦ •
•
(To itself if skeleton has only one state.)
Intro to MC Solver Interface Invariants Beyond Safety
Witness to Nonemptiness
If all queries are answered positively:
•
◦ •
•
Witness to nonemptiness of C.
Intro to MC Solver Interface Invariants Beyond Safety
Global Reachability
If a stem query is answered negatively: new inductive globalreachability information.
•
◦ •
•
Constrains subsequent selection of skeletons.
Constrains subsequent reach (stem and cycle) queries.
Improve proof by strengthening (using ideas from IC3).
Intro to MC Solver Interface Invariants Beyond Safety
Barriers: Discovering SCC-Closed Sets
If a cycle query is answered negatively: new information aboutSCC structure of state graph.
•
◦ •
•
Inductive proof: “one-way barrier”
Each “side” of the proof is SCC-closed.
Constrains subsequent selections of skeletons: all states onone side.
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
000 001
101100
010
110
011
111
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk1 010 110
000 001
101100
010 011
111110
s0
s1
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk1 010 110
000 001
101100
010 011
111110
s0
s1
stem query produces x1 ∨ ¬x2
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
000 001
101100
010
110
011
111
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
stem query passes
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
reach(S , (x1 ∨ ¬x2), s0, s1) passes
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
reach(S , (x1 ∨ ¬x2), s1, s0) produces x2
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
because x1 ∧ x2 ∧ ¬x3 ⇒ x2 . . .
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk2 101 110
states satisfyx1 ∨ ¬x2
000 001
101100
010
110
011
111
s0 s1
and x2 ∧ (x1 ∨ ¬x2) ∧ T ⇒ x ′2
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
000 001
101100
010
110
011
111
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk3 101 100
states satisfy(x1 ∨ ¬x2) ∧ ¬x2
000 001
101100
010
110
011
111
s0
s1
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk3 101 100
states satisfy(x1 ∨ ¬x2) ∧ ¬x2
000 001
101100
010
110
011
111
s0
s1
stem query passes
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
s0 s1sk3 101 100
states satisfy(x1 ∨ ¬x2) ∧ ¬x2
000 001
101100
010
110
011
111
s0
s1
reach(S , (x1 ∨ ¬x2) ∧ ¬x2, s0, s1) produces x2 ∨ x3
Intro to MC Solver Interface Invariants Beyond Safety
Example: Empty Language
no skeletons left000 001
101100
010
110
011
111
Intro to MC Solver Interface Invariants Beyond Safety
Key Insights
Inductive assertions describe SCC-closed sets.
Arena: Set of states all on the same side of each barrier.
Unlike previous symbolic methods:
Barrier constraints on the transition relationcombined with the over-approximating nature ofIC3 enable the simultaneous (symbolic)consideration of all arenas.
A proof can provide information about many arenas eventhough the motivating skeleton comes from one arena.
Intro to MC Solver Interface Invariants Beyond Safety
Methodological Parallels with IC3
IC3 FAIR
Seed: CTI Skeleton
Lemma: Inductive clause Global reachability proofOne-way barrier
Relative to previously discovered lemmas.
CEX: CTI sequence Connected skeletonDiscovery guided by lemmas. Not minimal.
Proof: Inductive strengthening All arenas skeleton-freeSufficient set of lemmas.
Intro to MC Solver Interface Invariants Beyond Safety
IICTL: Incremental Inductive CTL Model Checking
Task-directed strategy
Maintains upper and lower bounds on states satisfying eachsubformula
States in between the bounds are undecided
Typically don’t need to decide all states to decide the property(Traditional symbolic CTL algorithms do)
Decide states by executing appropriate query:
EX: SAT queryEU: Safety model checker (e.g., IC3)EG: Fair cycle finder (e.g., FAIR)
Generalizing decisions (proofs or counterexamples) to otherstates and refining the bounds
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
[[EF p]]
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
[[EF¬EF p]]
[[EF p]]
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
[[EF¬EF p]]
[[EF¬EF p]]
[[EF p]]
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⊇ initial states?[[EF¬EF p]]
[[EF¬EF p]]
[[EF p]]
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
Yes: Property holdsNo: Property fails
⊇ initial states?[[EF¬EF p]]
[[EF¬EF p]]
[[EF p]]
[[EF p]]
[[p]]
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
I ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
No: Property failsI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
I ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
Yes: Property holdsI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
can s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
Yes: s can also reach ψ2can s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
can s reach U2?
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
No: s cannot reach ψ2can s reach U2?
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
Yescan s reach U2?
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
t |= U2Yescan s reach U2?
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
t
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
t
t
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
can t reach L4 (or U4)?t
t
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
p ∨ tcan t reach L4 (or U4)?
t
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
¬p ∧ ¬t
p ∨ tcan t reach L4 (or U4)?
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
u
¬p ∧ ¬t
p ∨ tcan t reach L4 (or U4)?
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Example
Property: AGEF p = ¬EF¬EFp
u
u
¬p ∧ ¬t
p ∨ tcan t reach L4 (or U4)?
t is undecided for node 2t 6|= L2
t |= U2Yescan s reach U2?
Nocan s reach L2?s
s s is undecided for node 0s |= U0
s 6|= L0NoI ∧ ¬L0?
YesI ∧ ¬U0?⊤
⊥
⊤
⊥
¬p
⊥
⊤
p
p
p
ψ1
ψ4
ψ3
ψ2
ψ0
1
2
3
4
0
EF
EF
p
¬
¬
Intro to MC Solver Interface Invariants Beyond Safety
IICTL Algorithm
IICTL
1 Construct the parse-graph of the formula
2 Initialize bounds
3 Are all initial states in lower bound of root node?Yes: property holds
4 Is any of the initial states not in upper bound of root?Yes: property fails
5 There is an undecided state s. Decide s recursively andgeneralize.
6 Repeat step 3