Radu Grosu Vienna University of Technology
Hybrid Systems Modeling, Analysis and Control
Lecture 11 Constraint Satisfaction
Using Interval Approximation
Linear Classifiers with Hard Threshold x 2
: sur
face
-wav
e m
agni
tude
x1: body-wave magnitude
−Linear: A hyperplane. E.g. − 4.9 + 1.7x1 − x2 = 0
−Explosions: − 4.9 + 1.7x1 − x2 > 0
−Earthquake: − 4.9 + 1.7x1 − x2 < 0 Decidable!
! eartquake • nuclear explosion
Decision boundary: Surface that separates two classes
Polynomial Constraints
Theorem (Tarski 1949): The first-order theory of real-closed ordered fields is decidable.
Proof: FOT (R, ≤ ,+,×) admits quantifier elimination.
x2 + y2 − 1 = 0 ∧ y − x2 = 0
x
y
Polynomial Constraints
Theorem (Tarski 1949): The first-order theory of real-closed ordered fields is decidable.
Complexity: ExpExp in # alternations, Exp in # vars
x2 + y2 − 1 = 0 ∧ y − x2 = 0
x
y
What About Certificates?
Certificate: Satisfying valuation (solution) Quantifier-free case: Eg. x2 = 2
Question: How to represent the solution?
−Linear case: Rational numbers. Eg. 3x = 2 ⇒ x = 2 / 3
−Polynomial case: Real algebraic numbers (unintuitive)
−Non-polynomial case f(x) = 0: An approx algorithm
Given: desired precision k ∈NReturns r ∈Q: ∃x. |x − r| ≤ 10−k ∧ f(x) = 0
What About Certificates?
Certificate: Satisfying valuation (solution) Quantifier-free case: Eg. x2 = 2
Question: How to represent the solution?
−Linear case: Rational numbers. Eg. 3x = 2 ⇒ x = 2 / 3
−Polynomial case: Real algebraic numbers (unintuitive)
−Non-polynomial case f(x) = 0: An approx algorithm
Given: desired precision k ∈NReturns r ∈Q: |f(r)| ≤ 10−k
Non-Polynomial Constraints (NPC)
What about: FOT (R, ≤ ,+,×,sin) ?
−Undecidable: A Would allow encoding of polynomial diophantine equations (PDE)
f(x,y) > 0
f(x,y) < 0
x
y
−PDE: Known to be undecidable (Matiyasevich, 1970)
Fixing Precision of NPC
x
y
f(x,y) < 0
f(x,y) > 0
What about: FOT (R, ≤ ,+,×,sin) ?
Exponential: In the number of boxes!
Top-Down Algorithm
x
y
f(x,y) < 0
f(x,y) > 0
What about: FOT (R, ≤ ,+,×,sin) ?
Infinite precision: May not terminate!
Robust Constraints
Real CPS (biology/engineered): Satisfy robust properties
−Satisfiability: Does not change under perturbations
c2
! eartquake • nuclear explosion
c1
x1: body-wave magnitude
x 2: s
urfa
ce-w
ave
mag
nitu
de
Robust Constraints
Constraint ϕ is robust iff
Real CPS (biology/engineered): Satisfy robust properties
−Satisfiability: Does not change under perturbations
x2 ≤ 0 ⇒ x2 ≤ −0.00001 −Not robust:
x2 ≤ 1 ⇒ x2 ≤ +0.99999 −Robust:
−Constraint distance: d(ϕ,ϕ') ! (ϕ ≡upto cst c
ϕ') ? c : ∞
−There is an ε such that
−For all ϕ ' with d(ϕ,ϕ') ≤ ε
ϕ and ϕ ' are equi-satifiable
Quasi Deciadable Problems
A problem is quasi decidable iff:
−There is an algorithm that
−Correctly checks satifiability and −Terminates on all robust instances
Theorem (Ratschan 2002):
−FOT (R, ≤ +, ×, exp, sin, ...) is quasi decidable
Assumptions:
−All variables are bounded (in some interval)
−Shortcut: f = 0 ≡ (f ≤ 0 ∧ f ≥ 0)
Implementation: http://rsolver.sourceforge.net
Not Quasi Deciadable Problems
Theorem (Clarke/Gao 2011):
−δ-Satisfiability is decidable
x
y
f(x,y) < 0
f(x,y) > 0
δ do not know
Algorithm: Quantifier-Free Conjunctive
Box (hyper-rectangle): Cartesian product of intervals
−Given ϕ: Conjunction of ≤ using +, ×, exp, sin
Uses algorithms for both sat and unsat: Why?
−Due to undecidability failure to prove sat
−Box B: Providing an interval for each variable
Quasi-Satisfiability Algorithm
−Decide: Sat or Unsat
−Does not imply unsat and vice versa −Later on can use information from each other
Search for Sat
Satisfiability: Statement over one valuation −Good search method: Suffices
−Local search examples: Newton-type methods
−Iteratively refine the grid
−Local search: Start from this points
x
y
f(x,y) < 0
f(x,y) > 0
−Approximation errors: Floating-point rounding OK
Search for Unsat: Branch and Bound
Non-Satisfiability: Statement over an uncountable set
−Symbolic representation: Necessary
S = test(ϕ,B) if (S = unsat) return S else
−Use: test(ϕ,B) ∈{unsat, unknown}
let B = B1 ∪B2 non-overlapping
Algorithm BB(ϕ,B): Either returns unsat or runs forever
if (BB(ϕ,B1) = BB(ϕ,B2) = unsat) return unsat
Can be interleaved: With satisfiability test
Test for Unsat
Works recursively: On the structure of ϕ −Special case: One single equality
−Input: f(x1,...,xn) = 0, B = intervals (I1,...,In)
{f(x1,...,xn) | x1 ∈I1,...,xn ∈In} ⊆ [f] (I1,...,In)
if (0 ∉[f](I1,...,In)) return unsat else unknown
Compute: [f] (I1,...,In), an inclusion of f, s.t:
Example:
x × y + 1 = 0
x ∈[2,3], y ∈[4,7] B = ([2,3],[4,7])
[2,3] [4,7]
x y
[8,21]
[×] [1,1]
[9,22]
[+]
1
Interval Arithmetic: Addition
Function [+] on intervals [a,a] and [b,b] is an inclusion:
[a,a] [+] [b,b] =
∅ if [a,a] = ∅ ∨ [b,b] = ∅[a + b,a + b] otherwise
⎧⎨⎪
⎩⎪
Example:
[8,21] [+] [1,1] = [9,22] [8,21] [1,1]
[9,22]
[+]
Proof: + does not have any local minimum Corners suffice
Interval Arithmetic: Multiplication
Function [×] on intervals [a,a] and [b,b] is an inclusion:
[a,a] [×] [b,b] =
∅ if [a,a] = ∅ ∨ [b,b] = ∅ [min{ab,ab,ab,ab}, max{ab,ab,ab,ab}] else
⎧⎨⎪
⎩⎪
Example:
[2,3] [×] [4,7] = [8,21] [2,3] [4,7]
[8,21]
[×]
Proof: × does not have any local minimum Corners suffice
Multiplication Improved
Do we really need all corners?
b ≥ 0 0 ∈[b,b] b ≤ 0a ≥ 0 [ab, ab] [ab,ab] [ab,ab]
0 ∈[a,a] [ab,ab] [min{ab,ab}, max{ab,ab}] [ab,ab]a ≤ 0 [ab,ab] [ab,ab] [ab,ab]
Example:
[8,21] [+] [1,1] = [9,22] [2,3] [4,7]
[8,21]
[×]
−Observation: Fixed monotonicity in quadrants
−For: [a,a] = [0,0]∨ [b,b] = [0,0] return [0,0] else
Other Operations
What about exp, sin, cos?
−Observation: Exploit monotonicity!
Interval arithmetic: Given term t in vars x1 ∈I1,...,xn ∈In
[t](I1,...,In) =
Ii if t = xi
[c,c] if t = c
[s]([t1](I1,...,In),...,[tk ](I1,...,In)) if t = s(t1,..., tk )s ∈{+,×,exp,...}
⎧
⎨⎪⎪
⎩⎪⎪
Fundamental theorem of interval arithmetic:
−For a term e, its interval extension [e] is an inclusion.
Properties of Inclusions
Width of a box: w([a1,b1] × ... × [an,bn]) = max
i∈{1,...,n}| bi − ai |
−Thin: iff w(B) = 0 ⇒ w([f](B)) = 0 An inclusion [f] of a function f is:
where: f(B) ! {f(x) | x ∈B}, [A] ! [inf A, sup A]
−Convergent: iff ∀B1 ⊇ ... ⊇Bn.limn→∞ w(Bn) = 0 ⇒ w([f](Bn)) = 0
−Monotonic: iff B1 ⊆ B2 ⇒ [f](B1) ⊆ [f](B2)
−Optimal: iff [f](B) = [f(B)]
Theorem:
If f: Rn → R does not contain any variable more than once then [f] is optimal.
Dependency Problem
Interval arithmetic forgets equality between variables:
[x − x]([-1,1]) = [x]([-1,1]) [−] [x]([-1,1])
Diff ways of writing the same exp, diff results:
[a × (b + c)](B) ⊆ [a × b + a × c)](B)
There are systematic methods for dealing with the dependency problem: [Neumaier 1990, Moore 2009]
[x × y + x]([1,2],[-1,-1]) = ([x]([1,2]) [×] [y]([-1,-1])) [+] [x]([1,2])
= [-1,1] [−] [-1,1] = [-2,2] ⊃ [0,0]
= ([1,2] [×] [-1,-1]) [+] [1,2]
= [-2,-1] [+] [1,2] = [-1,1] ⊃ [0,0]
Wrapping Effect
Rotation matrix:
At =cos(t) sin(t)−sin(t) cos(t)
⎛
⎝⎜
⎞
⎠⎟
Appears in several contexts especially ODE solving
Exponential blow-up
−Smaller t makes things worse
−In the limit box eplodes by a
−A factor of e2π ≈ 535 per revolution
Partial solution: Coordinate transfomations
Efficiency of Branch-And-Bound
Curse of dimensionality:
−Halving boxes in n-dim: 2n boxes
−Branching does not scale in problem dim
−Can one deduce more without branching?
Contractor Method (Davis 1987,Clearly 1987, Jaulin 2001)
Istead of an inclusion use a contractor:
−For given constraint with solution p ⊆ Rn and box B ⊆ Rn
−Find box C ⊆ Rn such that (B∩p) ⊆ C ⊆ B
x − 99 = 0 ∧ y − 99 = 0, x ∈[0,100], y ∈[0,100]
x
y
100
100
Contractor Method (Davis 1987,Clearly 1987, Jaulin 2001)
x
y
100
100
Istead of an inclusion use a contractor:
−For given constraint with solution p ⊆ Rn and box B ⊆ Rn
−Find box C ⊆ Rn such that (B∩p) ⊆ C ⊆ B
x − 99 = 0 ∧ y − 99 = 0, x ∈[0,100], y ∈[0,100]
Contractor Method (Davis 1987,Clearly 1987, Jaulin 2001)
Istead of an inclusion use a contractor:
−For given constraint with solution p ⊆ Rn and box B ⊆ Rn
−Find box C ⊆ Rn such that (B∩p) ⊆ C ⊆ B
x − 99 = 0 ∧ y − 99 = 0, x ∈[0,100], y ∈[0,100]
A contractor ⌢p of a constraint p is called optimal iff:
−For every box B: ⌢p(B) = (B∩p)
−In practice: Optimal up to rounding
Computing Contractors: Sum
sum ! {(x,y,z) | x + y = z}
Contractor:
Need hull of projection to 3 axes within B = Ix × Iy × Iz
i π1(sum∩ Ix × Iy × Iz ) = Ix ∩ (Iz − Iy )
i π2(sum∩ Ix × Iy × Iz ) = Iy ∩ (Iz − Ix )
i π3(sum∩ Ix × Iy × Iz ) = Iz ∩ (Ix + Iy )
i s⌢um(Ix × Iy × Iz ) = Ix ∩ (Iz − Iy ) × Iy ∩ (Iz − Ix ) × Iz ∩ (Ix + Iy )
Computing Contractors: Product
prod ! {(x,y,z) | x × y = z}
where:
Need hull of projection to 3 axes within B = Ix × Iy × Iz
i π1(prod∩ Ix × Iy × Iz ) = Ix ∩ (Iz / Iy )
i π2(prod∩ Ix × Iy × Iz ) = Iy ∩ (Iz / Ix )
i π3(prod∩ Ix × Iy × Iz ) = Iz ∩ (Ix × Iy )
i [a,b] / [c,d] = {z | ∃x∃y. y × z = x ∧ x ∈[a,b]∧ y ∈[c,d]}
Contractor:
i p⌢rod(Ix × Iy × Iz ) = Ix ∩ (Iz / Iy ) × Iy ∩ (Iz / Ix ) × Iz ∩ (Ix × Iy )
i This is not an interval! [1,1] / [-1,1]
i sin, exp: similar
Contractors: Conjunction c1∧...∧cn
C = {c1,...,cn}
while (∃c ∈C. ⌢c(B) ≠ B) B =
⌢c(B)
Termination? Optimal?
Contractors: Conjunction c1∧...∧cn
C = {c1,...,cn}
while (∃c ∈C. ⌢c(B) ≠ B) B =
⌢c(B)
Termination? Optimal? x − y = 0 ∧ x + y = 0