Post on 27-Dec-2015
transcript
Christoph M. Wintersteiger
Floating-point arithmetic
• Types• Half (16), Float(32), Double (64), Quadruple (128 bit)• double x = 1.5;
• Variables• E.g., float x; double y;
• Operations
• Special values:
• 5 Rounding Modes• Ties toward odd, Ties toward even, to negative, to positive, to zero
Floating-point arithmetic
• Operator semantics are defined on reals• Rounding when necessary• (predefined rounding modes only)
• E.g., (fp.add rm a b)
• Verification• Rounding introduces error• Unintuitive and unexpected results• Lacks `nice’ mathematical properties (e.g., associativity)• E.g.
FPA semantics
• Approximation of real numbers• Standards: IEEE754 vs SMT
• Special exponents (IEEE754)• = 0…0 for ±zero and denormal/subnormal numbers• = 1…1 for ±oo and NaN
FPA representation
sign exponent significand
(−1 ) sign ⋅significand ⋅2exponent
• TypesAny combination sbits/ebits: (_ FloatingPoint ebits sbits)Single: (_ FloatingPoint 8 24), also Float32Double: (_ FloatingPoint 11 53), also Float64
• Variables(define-fun x () (_ FloatingPoint 8 24))(define-fun y () Float32)
SMT floating-point arithmetic
(fp.add rm x y)(fp.sub rm x y)(fp.mul rm x y)(fp.div rm x y)(fp.sqrt rm x)(fp.rem x y)(fp.fma rm x y z)…
Fused Multiply-Add:
+ ,− ,∗ , ÷ ,% ,𝑎𝑏𝑠 ,√❑ , 𝑓𝑚𝑎 ,𝑚𝑖𝑛 ,𝑚𝑎𝑥 ≤<¿>≥SMT floating-point arithmetic
(fp.isNormal x)(fp.isSubnormal x)(fp.isZero x)(fp.isInfinite x)(fp.isNegative x)…
(fp.leq x y)(fp.lt x y)(fp.geq x y)(fp.gt x y)(fp.eq x y)
• Special values:
• Rounding modesRNE, roundNearestTiesToEvenRNA, roundNearestTiesToAwayRTP, roundTowardPositiveRTN, roundTowardNegativeRTZ, roundTowardZero
SMT floating-point arithmetic
(_ NaN ebits sbits)(_ -zero ebits sbits), (_ +zero ebits sbits)(_ -oo ebits sbits), (_ +oo ebits sbits)
double Kp=1.0, Ki=0.25;double set_point=20.0, integral=0.0;double error, in, out;
for (int i=0; i < N; i++) { in = read_input(); error = set_point – in; integral = integral + error; out = Kp*error + Ki*integral; set_output(out);}
An example
then out
If in
• Verification• Unwind loops• No quantifiers
• Convert• QF_FP -> QF_BV• QF_BV -> SAT
Example strategy
• Good approximate results• Real arithmetic• Fixed-point arithmetic• Reduced precision FPA• Interval analysis
(declare-fun Kp () (_ FloatingPoint 5 11))(declare-fun Kp () (_ FloatingPoint 5 11))(declare-fun …
(define-fun rm () RoundingMode RNE)
(assert (= c18 (fp #b0 #b10011 #b0010000000))) ;; = 18.0(assert (= c22 (fp #b0 #b10011 #b0110000000))) ;; = 22.0(assert (= cp3 (fp #b0 #b10000 #b1000000000))) ;; = +3.0(assert (= cn3 (fp #b1 #b10000 #b1000000000))) ;; = -3.0
(assert (= Kp (fp #b0 #b01111 #b0000000000))) ;; = 1.0(assert (= Ki (fp #b0 #b01101 #b0000000000))) ;; = 0.25(assert (= set_point (fp #b0 #b10011 #b0100000000))) ;; = 20.0(assert (= integral (_ +zero 5 11))) ;; = +0.0
An example in SMT2
(assert (not (=> (and(fp.leq c18 in)(fp.leq in c22)(fp.eq error_post (fp.sub rm set_point in))(fp.eq integral_post (fp.add rm integral error_post))(fp.eq out (fp.add rm (fp.mul rm Kp error_post)
(fp.mul rm Ki integral_post))))(and
(fp.leq cn3 out)(fp.leq out cp3))
)))
> z3 pi_controller_1.smt2unsat
An example in SMT2
• fp.add is commutative?(assert (not (fp.eq (fp.add RNE x y) (fp.add RNE y x))))
> z3 add_is_commutative.smt2unsat
• fp.add is associative?(assert (not (fp.eq (fp.add RNE (fp.add RNE x y) z) (fp.add RNE x (fp.add RNE y z)))))
> z3 add_is_associative.smt2satmodel (define-fun z () (_ FloatingPoint 5 11) (fp #b0 #b11010 #b1101111011)) ;; 1.8701171875 * 2^11…
Checking properties
• Approximations• Real arithmetic• Fixed-point arithmetic• Reduced precision FPA• Abstract interpretation-based
(Astrée)
• Precise• Translation to bit-vectors and/or
SAT(MathSAT, Z3, Sonolar, CBMC)
• Mixed abstractions (CBMC)• Abstract CDCL (MathSAT)• Translation to reals (Realizer)
Related techniques• Theorem prover-based (Gappa)• Proof assistants (Coq, HOL)• Often not sound or complete
1. Translate to QF_BV(build circuits)
2. Translate to SAT (e.g., Tseitin translation, bit-blasting)
3. Run SAT Solver4. Translate models/proofs
(back to QF_BV, then QF_FP)
Solving QF_FP via SAT
Circuit for and method of providing a floating-point adder
US 8463835 B1 (Xilinx)
• Conversion QF_FP -> QF_BV -> SAT
Example performance
Bad
1. Mixed abstractions (CBMC)2. Non-conservative approximations (Z3)3. Abstraction into interval arithmetic
(MathSAT)4. Translation into non-linear reals (Realizer)
Advanced QF_FP solving
Abstraction refinementSolve abstraction(s)
Check model
Refine abstraction(s)(model-guided)
Model
satRefine abstraction(s)(proof- or core-guided)
Proof
unsat
no refinement possiblefailed
• Abstract the problem• By reduction of significand precision• Adjust rounding modes• Replacing by fresh Boolean yields over-approximation• Restrict to non-rounded results yields under-approximation
• Over- and under-approximation• Solve them at the same time
• Refinement• Removal of under-approximation constraints• Increase of significand precision
Mixed abstractions
Brillout, Kroening, Wahl: Mixed Abstractions for Floating-Point Arithmetic. FMCAD 2009
Mixed abstractions
Brillout, Kroening, Wahl: Mixed Abstractions for Floating-Point Arithmetic. FMCAD 2009
sign exponent significand
sign exponent significand
ORMURM
RM
Small-float approximation
sign exponent significand
sign exponent significandRM
RM
Zeljic, Wintersteiger, Rümmer: Approximations for Model Construction. IJCAR 2014
For the theory of floating-point numbers:
Small-floats inclusion
𝑠≤𝑠 ′∧𝑒≤𝑒′⇒𝐹 𝑃 𝑠 ,𝑒⊆𝐹 𝑃𝑠 ′ ,𝑒′
Smaller bit-width makes subsets of numbers.
Zeljic, Wintersteiger, Rümmer: Approximations for Model Construction. IJCAR 2014
Approximation framework
𝑀⊨𝑇Φ
�̂�⊨𝑇Φ̂
liftreconstruct
No implication!May fail
Zeljic, Wintersteiger, Rümmer: Approximations for Model Construction. IJCAR 2014
Lifting
𝑦=𝑥+1.75∧𝑦>0∧(𝑥=2.0∨𝑥=−4.0)
∧
¿ ¿ ∨
𝑦 +¿ 0𝑝0
𝑥
𝑝1 ¿ ¿𝑦
1.75𝑝2 𝑥 2.0𝑝3 𝑥 −4.0𝑝4
Example precision terms
(fp.op a_1 … a_n)
(fp.op p a_1 … a_n)And
Refinement schemeSolve approximation
Reconstruct model
Refine approximation(model-guided)
Model
satRefine approximation(proof- or core-guided)
Proof
unsat
no refinement possiblefailed
• Search for• Approximate model• With all precisions assigned to values• (not necessarily the same)• Implementation: precision 0 =
• Could also• Optimize toward fewer rounding occurrences• Search for small values first• Use real numbers• And many more…
Approximate model construction
Model-guided refinementSolve approximation
Reconstruct model
Refine approximation(model-guided)
Model
satRefine approximation(proof- or core-guided)
Proof
unsat
no refinement possiblefailed
Precise model reconstruction
𝑦=𝑥+1.75∧𝑦>0∧(𝑥=2.0∨𝑥=−4.0)
∧
¿ ¿ ∨
𝑦 +¿ 00
𝑥
0 ¿ ¿𝑦
1.750 𝑥 2.00 𝑥 −4.00
2.0 3.5(
𝜔
𝜔
𝜔
𝜔 𝜔
𝑦≔3.5
3.75
• Choose terms to refine• Check which terms introduce imprecision•
• Increase precision of highest ranked term(s)• Error • propagated from a term’s children• or newly introduced by the term
Model-guided refinement
Proof-guided refinementSolve approximation
Reconstruct model
Refine approximation(model-guided)
Model
satRefine approximation(proof- or core-guided)
Proof
unsat
no refinement possiblefailed
• Various levels of granularity• Increase precision of all terms• … those that appear in the unsat core• … those that appear in the proof• … as expensive as you like• Trade-off between precision and efficiency
Proof-guided refinement
• Built upon existing decision procedure• Preserved: soundness, completeness
• Solving more but smaller problems often faster
• Candidate models provide hints for the refinement
• Also preserves termination• If is a well-founded relation• And there is progress in every iteration
Algorithm properties
Fröhlich, Kovásznai, Biere: More on the Complexity of Quantifier-Free Fixed-Size Bit-Vector Logics with Binary Encoding. CSR 2013Kovásznai, Fröhlich, Biere: On the Complexity of Fixed-Size Bit-Vector Logics with Binary Encoded Bit-Width. SMT 2012
A note on complexity
∃𝑥 𝑓∈𝐹𝑃𝐴 (11 ,53) .𝜙 (𝑥)
∃𝑥𝑏𝑣∈𝐵𝑉 64 .𝜙 ′ (𝑥)
∃𝑥0 ,…,𝑥31∈𝔹 .𝜙 ′ ′(𝑥0 ,…,𝑥31)
𝑛
2𝑛
𝑛
QF_BV: NEXPTIME
SAT: NP
QF_FP: (NEXPTIME)
• Crucial for success• “Close” models • Effective model reconstruction
• Incomplete solvers• Can be efficient when starting “close” to a solution• Can find solutions quickly (or we abort them early)• Examples:• Local-search, “cheap” optimization• Perhaps Horn or 2-CNF abstractions
Non-conservative approximation
Fröhlich, Biere, Wintersteiger, Hamadi: Stochastic Local Search for Satisfiability Modulo Theories. AAAI 2015
• Decision procedure framework• Concepts from abstract interpretation• Abstract values from abstract domains• Fixed-point computation over abstract values• Meet irreducibles (~ invertible constraints)• Very nice theory
• For FP-ACDCL• Half-open floating-point intervals, e.g., • Combined into intervals, e.g., • MathSAT
Abstract CDCL
Brain, D’Silva, Griggio, Haller, Kroening: Interpolation-Based Verification of Floating-Point Programs with Abstract CDCL. SAS 2013Brain, D’Silva, Griggio, Haller, Kroening: Deciding floating-point logic with abstract conflict driven clause learning. FMSD 45(2) 2014
FP-ACDCL
Decision Propagation
Analysis
𝑥 (…∧𝑥 )→ 𝑦
(… )→¬𝑥
𝑥∈[1.0 ,2.0] (… )→ 𝑦∈[4.0 ,5.0]
(… )→ 𝑥∈[2.0 ,3.0]
Realizer• Translates QF_FP into QF_NIRA• Satisfiability-equivalent• Eliminates all floating-point terms• Uses for real-valued • Normalization• Rounding
Leeser, Mukherjee, Ramachandran, Wahl: Make it real: Effective floating-point reasoning via exact arithmetic. DATE 2014
Mixed abstraction performance
Brillout, Kroening, Wahl: Mixed Abstractions for Floating-Point Arithmetic. FMCAD 2009
FP-ACDCL performance
Brain, D’Silva, Griggio, Haller, Kroening: Interpolation-Based Verification of Floating-Point Programs with Abstract CDCL. SAS 2013Brain, D’Silva, Griggio, Haller, Kroening: Deciding floating-point logic with abstract conflict driven clause learning. FMSD 45(2) 2014
• Comparison• Bit-blasting (in Z3)• FP-ACDCL (MathSAT)• Small-float approximation (in Z3)
• Benchmark sets• Verification of C programs• Randomly generated systems of
polynomial inequalities
Experimental evaluation
SAT UNSAT
Z3 76 56
MathSAT 76 76
Ours 86 46
Data (SAT)
Data (UNSAT)
Complexity of operations5.3 Homogeneous general-computational operations 5.3.0
5.3.1 General operations 5.3.1.0
Implementations shall provide the following homogeneous general-computational operations for all supported arithmetic formats; …
― sourceFormat remainder(source, source)When y ≠ 0, the remainder r = remainder(x, y) is defined for finite x and y regardless of the …
… FPREM1 … can reduce the exponent of ST(0) by no more than 63 in one executionof the instruction … Software can re-execute the instruction (using the partial remainder in ST(0) as the dividend) until …IEEE 754-2008, Sec 5.3.
Intel® 64 and IA-32 Architectures Software Developer’s Manual. May 2012
Conclusion• SMT2 Floating-point standard• Solvers are starting to support it• Various solving approaches
• Near future: challenging problems• Out-of-the-box performance• Theory combination• Quantifiers• Real error bounds• Non-linearity
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.