Provably Optimal Test Cube Generation usingQuantified Boolean Formula SolvingASP-DAC 2013
Albert-Ludwigs-Universität Freiburg
Matthias Sauer, Sven Reimer, Ilia Polian, Tobias Schubert, Bernd BeckerChair of Computer Architecture01.24.13
Motivation – Test pattern relaxation
Test cube:Parts of the pattern are unspecified (Don’t Care)Test requirements still hold
Used for:RefillingMinimizing power consumptionCompaction (e.g., Embedded Deterministic Test)
All known techniques are approximativeOur approach:
Test cube generation with maximum number of Don’t Cares⇒ Optimal test cube
Measure the quality of heuristic methods
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 2 / 17
Outline
1 Motivation2 Preliminaries
Circuit encodingUnspecified valuesSensitizable paths + small delay faults
3 Optimal test cube generation4 Experimental results5 Conclusion
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 3 / 17
Circuit encoding
A
C
B
E
D
G
F
D
Boolean satisfiability (SAT) formulation in CNF:Tseitin encoding [Tseitin ’68]Additional variables for each gateLinear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 4 / 17
Circuit encoding
A
C
B
E
D
G
F
D
Boolean satisfiability (SAT) formulation in CNF:Tseitin encoding [Tseitin ’68]Additional variables for each gateLinear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 4 / 17
Circuit encoding
A
BE
Boolean satisfiability (SAT) formulation in CNF:Tseitin encoding [Tseitin ’68]Additional variables for each gateLinear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 4 / 17
Circuit encoding
A
BE
E ↔ ¬(A ∧B)
Boolean satisfiability (SAT) formulation in CNF:Tseitin encoding [Tseitin ’68]Additional variables for each gateLinear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 4 / 17
Circuit encoding
A
BE
E ↔ ¬(A ∧B)(A ∨ E) ∧
Clause︷ ︸︸ ︷(B ∨ E)∧(
Literal︷︸︸︷¬A ∨¬B ∨ ¬E)
Boolean satisfiability (SAT) formulation in CNF:Tseitin encoding [Tseitin ’68]Additional variables for each gateLinear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 4 / 17
Unspecified values – 01X logic [Jain et al. ’00]
A = 101X
C = 101X
B = X01X
E = X01X
D = X01X
G = 101X
F = X01X
D = X01X
Three-valued logic:001X (logic 0), 101X (logic 1), X01X (unknown)01X in SAT: 001X = (0,1), 101X = (1,0), X01X = (0,0)SAT encoding for 01X doubles size of the formulaIn example: Output F is unknown if input B is unspecified
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 5 / 17
Unspecified values – Exact formulation
A = 1
B
C = 1D
E
F
G
= 0= 1
= 1= 0
= 0= 1
= 1= 1
= 1= 1
D= 0= 1
Simulation for B= 0= 1
But: F can be set to 1, even if B is unspecified
⇒ QBF: Universally quantified variables for unknown values∃{A,C}∀{B}∃{D,E ,F ,G}︸ ︷︷ ︸
Prefix
. ϕ(A, . . . ,G)︸ ︷︷ ︸Tseitin encoding
∧(A)∧ (C)∧ (F )︸︷︷︸property
QBF: reconvergent paths are resolved by formulation01X : reconvergent paths may block propagation of values
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 6 / 17
Unspecified values – Exact formulation
A = 1
B = ?
C = 1D
E
F = 1
G
But: F can be set to 1, even if B is unspecified⇒ QBF: Universally quantified variables for unknown values∃{A,C}∀{B}∃{D,E ,F ,G}︸ ︷︷ ︸
Prefix
. ϕ(A, . . . ,G)︸ ︷︷ ︸Tseitin encoding
∧(A)∧ (C)∧ (F )︸︷︷︸property
QBF: reconvergent paths are resolved by formulation01X : reconvergent paths may block propagation of values
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 6 / 17
Unspecified values – Exact formulation
A = 1
B
C = 1D
E
F
G
= 0= 1
= 1= 0
= 0= 1
= 1= 1
= 1= 1
D= 0= 1
Simulation for B= 0= 1
But: F can be set to 1, even if B is unspecified⇒ QBF: Universally quantified variables for unknown values∃{A,C}∀{B}∃{D,E ,F ,G}︸ ︷︷ ︸
Prefix
. ϕ(A, . . . ,G)︸ ︷︷ ︸Tseitin encoding
∧(A)∧ (C)∧ (F )︸︷︷︸property
QBF: reconvergent paths are resolved by formulation
01X : reconvergent paths may block propagation of values
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 6 / 17
Unspecified values – Exact formulation
A = 101X
C = 101X
B = X01X
E = X01X
D = X01X
G = 101X
F = X01X
D = X01X
But: F can be set to 1, even if B is unspecified⇒ QBF: Universally quantified variables for unknown values∃{A,C}∀{B}∃{D,E ,F ,G}︸ ︷︷ ︸
Prefix
. ϕ(A, . . . ,G)︸ ︷︷ ︸Tseitin encoding
∧(A)∧ (C)∧ (F )︸︷︷︸property
QBF: reconvergent paths are resolved by formulation01X : reconvergent paths may block propagation of values
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 6 / 17
Unspecified values – Exact formulation
A = 101X
C = 101X
B = X01X
E = X01X
D = X01X
G = 101X
F = X01X
D = X01X
But: F can be set to 1, even if B is unspecified⇒ QBF: Universally quantified variables for unknown values∃{A,C}∀{B}∃{D,E ,F ,G}︸ ︷︷ ︸
Prefix
. ϕ(A, . . . ,G)︸ ︷︷ ︸Tseitin encoding
∧(A)∧ (C)∧ (F )︸︷︷︸property
QBF: Exact formulation for Don’t Cares01X : Approximative formulation for Don’t Cares
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 6 / 17
Sensitizable paths + small delay faults
FF1
FF2
FF3
FF4
Sensitizable path: Transition from input to outputLength of a path according to sum of gate delays
Small delay faults: Assume additional delay for one gateOutput transition too late for clockTwo-pattern delay testThe longer the path the higher the detection quality
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 7 / 17
Sensitizable paths + small delay faults
02
4
4
1
Clock
22
6
7Length 6Length 2
FF1
FF2
FF3
FF4
Sensitizable path: Transition from input to outputLength of a path according to sum of gate delays
Small delay faults: Assume additional delay for one gateOutput transition too late for clockTwo-pattern delay testThe longer the path the higher the detection quality
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 7 / 17
Sensitizable paths + small delay faults
02
4
4
1
Clock
22
6
7Length 6Length 2
+δ+δ
+δ+δ
+δ
(erroneous behavior)(works fine)
Additional delay of δ = 2
FF1
FF2
FF3
FF4
Sensitizable path: Transition from input to outputLength of a path according to sum of gate delaysSmall delay faults: Assume additional delay for one gateOutput transition too late for clockTwo-pattern delay testThe longer the path the higher the detection quality
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 7 / 17
Optimal test cube generation
Primary inputs
Two-pattern delay test
Fli
p-Fl
ops
Fli
p-F
lops
p1 ... pipi+1 ... pm
x
x
x
x
Small delay faults over two timeframesTest cube with maximum number of unspecified inputs using QBFQuantify unspecified inputs universally, specified ones existentiallyIf path for small delay fault is sensitizable:Universally quantified inputs: excluded from test cubeExistential quantified inputs: test cubeBut: The quantifier of a variable cannot be changed in QBFUnspecified inputs are unidentified a-prioriWhich inputs have to be quantified universally?
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 8 / 17
Multiplexed inputs
Primary inputsE1 A1
MUX1S1
...
En An
Sn MUXn..
.
Multiplexed Inputs Two-pattern delay test
0 1
0 1
Flip
-Flo
ps
Flip
-Flo
ps
p1 ... pipi+1 ... pm
x
x
x
x
ψ = ∃{S1, . . . ,Sn,E1, . . . ,En}∀{A1, . . . ,An}∃ . . .ϕCircuit ∧ϕProperty ∧ϕMUX
Dynamic choice of (un-)specified input with multiplexer
Select input Si switches between specified (Si = 0 : ∃Ei ) and unspecified(Si = 1 : ∀Ai ) for any primary input IiFind the maximum number of select inputs that can be set to 1
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 9 / 17
Maximization
Primary inputsE1 A1
MUX1S1
...
En An
Sn MUXn..
.
...
Multiplexed Inputs Two-pattern delay testMaximization
0 1
0 1
BitonicSortingNetwork
SOn
SO1
Flip
-Flo
ps
Flip
-Flo
ps
p1 ... pipi+1 ... pm
x
x
x
x
0100101
1110000
Sort select-inputs Si with Bitonic sorting network [Batcher ’68]
Circuit size of sorter: O(n logn)
Input vector−→S is sorted by 1’s and 0’s
⇒ Sorted output vector−→SO
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 10 / 17
Optimal test cube generation
Primary inputsE1 A1
MUX1S1
...
En An
Sn MUXn
...
Multiplexed Inputs Two-pattern delay testMaximization
0 1
0 1
BitonicSortingNetwork
SOn
SO1
Flip
-Flo
ps
Flip
-Flo
ps
p1 ... pipi+1 ... pm
x
x
x
x
1110000
SOkSOk+1
ψ(j) = ∃{SO1, . . . ,SOn,S1, . . . ,Sn,E1, . . . ,En}∀{A1, . . . ,An}∃ . . .ϕCircuit ∧ϕProperty ∧ϕMUX ∧ϕSorter ∧ (SOj )
⇒ Binary search over jSearch for k , such that: path is sensitizable with k unspecified inputs(SOk = 1), but not with k +1 (SOk+1 = 0)QBF solver returns assignment for outermost existential variables:S1, . . . ,Sn: unspecified inputs; remaining E1, . . . ,En: test cubeOptimal test cube, i.e., maximum number of Don’t Cares
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 11 / 17
01X -Optimal test cube generation
Primary inputs
...
...
...
Trigger X Two-pattern delay testMaximization
BitonicSortingNetwork
SOn
SO1
Flip
-Flo
ps
Flip
-Flo
ps
p1 ... pipi+1 ... pm
x
x
x
x
T1
Tn
ϕ(j) = ϕCircuit︸ ︷︷ ︸01X encoding
∧ϕProperty ∧ϕTrigger ∧ϕSorter ∧ (SOj )
⇒ Binary search over jSearch for k , such that: path is sensitizable with k unspecified inputs(SOk = 1), but not with k +1 (SOk+1 = 0)If Ti = 1, corresponding input Ii is set to X01X
SAT solver returns assignment for all variables:T1, . . . ,Tn: unspecified inputs; remaining input variables: test cube01X -Optimal test cube, i.e., optimal for 01X encoding
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 12 / 17
Experimental setup
Sequential versions of ISCAS 89 and ITC 99 benchmarksSAT-based path generator PHAETON [Sauer et al. ’11]:100 longest broadside testable paths of each circuitIn-house SAT solver antom [Schubert et al. ’10] andQBF solver quantom
Cone-of-influence (COI) reductionAverage percentage of Don’t Cares (DC)
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 13 / 17
Results for ISCAS 89 & ITC 99 circuits
s000
27
s002
08
s002
98
s003
44
s003
49
s003
82
s003
86
s004
00
s004
20
s004
44
s005
10
s005
26
s006
41
s007
13
s008
20
s008
32
s008
38
s009
53
s011
96
s012
38
s014
23
s014
88
s014
94
s053
78
s132
07
s359
32
s384
17
b01
b02
b03
b04
b05
b06
b07
b08
b09
b10
b11
b12
b13
b15
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Cone-of-Influence Optimal test cube
DC
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 14 / 17
Comparison of QBF-optimal result
Static (initial test pattern needed):1. Lifting [Ravi, Somenzi ’04] (best case QBF-optimal)2. Simulation (best case 01X -optimal)
Average over 100 random initial test patterns
Dynamic (find test cube directly with given test requirements):3. 01X -optimal
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 15 / 17
Comparison
0.1
1
10
100
1000
10000
QBF-opt. 01X-opt. Lifting Simulation
aver
age
runt
ime
in s
econ
ds
0%
1%
2%
3%
4%
5%
6%
7%
QBF-opt. 01X-opt. Lifting Simulation
qual
itiy
loss
*
∗Loss(Method) = 1− DCMethod−DCCOIDCOptimal−DCCOI
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 16 / 17
Conclusion
Novel technique for generation test cubes with QBFFirst approach producing test cubes with maximum numberof Don’t CaresFramework adaptable to any task that maximizes number ofunspecified linesCompare heuristic approaches with true optimumNew and fast method for 01X encoding (01X -optimal)
Future workAdapt framework to other applications and fault modelsIncrease scalability of QBF-solver
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 17 / 17
SAT + QBF
Satisfiability problem or SAT problem:Given propositional formula ϕ. Is there an assignment to thevariables, such that ϕ is satisfied?ϕ in conjunctive normal form (CNF), e.g.,ϕ(x1, . . . ,xn) = (x1∨ ¬x2︸︷︷︸
literal
)∧ (x2∨x3∨¬x4)︸ ︷︷ ︸clause
∧ . . .
Notation: ϕ(x1, . . . ,xn) = {{x1,¬x2},{x2,x3,¬x4}, . . .}Properties of CNF:Clause is satisfied iff at least one literal is assigned to 1.CNF is satisfied iff all clauses are satisfied.Combinational circuits can be transformed into CNF in linearsize of the circuit (Tseitin encoding)Well known NP-complete problem with enormousimprovements in the last decades
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 18 / 17
SAT + QBF
Quantified Boolean formula (QBF) is an extension of SAT:variables are quantified existentially (∃) or universally (∀)Example for a QBF ψ in prenex normal form:ψ(x1, . . . ,xn) = ∃{x1}∀{x2,x3}∃{x4} . . .∃{xn}︸ ︷︷ ︸
prefix
.ϕ(x1, . . . ,xn)︸ ︷︷ ︸matrix (CNF)
Semantics (for this example):ψ is satisfied iff there exists one assignment for x1 such thatfor every assignment of x2 and x3, there exists oneassignment for x4 and so forth, such that ϕ is satisfied.PSPACE-complete problem with increasing interest in thelast decade
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 19 / 17
Circuit encoding
Circuit to propositional formulae in CNF viaTseitin encoding [Tseitin ’68]
Introduces additional Tseitin variablesResulting formula is linear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 20 / 17
Circuit encoding
A
B
CD
D ↔ B ∧ C{{B,¬D}, {C,¬D}, {¬B,¬C,D}}
Circuit to propositional formulae in CNF viaTseitin encoding [Tseitin ’68]Introduces additional Tseitin variables
Resulting formula is linear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 20 / 17
Circuit encoding
A
B
CD
D ↔ B ∧ C{{B,¬D}, {C,¬D}, {¬B,¬C,D}}
F ↔ D ∨ E{{¬D,F}, {¬E,F}, {D,E,¬F}}
E
F
G
G ↔ A ∨ E{{¬A,G}, {¬E,G}, {A,E,¬G}}E ↔ ¬(A ∧B)
{{A,E}, {B,E}, {¬A,¬B,¬E}}
Circuit to propositional formulae in CNF viaTseitin encoding [Tseitin ’68]Introduces additional Tseitin variablesResulting formula is linear in circuit size
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 20 / 17
Encode small delay faults
A
B
CD
D ↔ B ∧ C{{B,¬D}, {C,¬D}, {¬B,¬C,D}}
F ↔ D ∨ E{{¬D,F}, {¬E,F}, {D,E,¬F}}
E
F
G
G ↔ A ∨ E{{¬A,G}, {¬E,G}, {A,E,¬G}}E ↔ ¬(A ∧B)
{{A,E}, {B,E}, {¬A,¬B,¬E}}
Encode both timeframes . . .
. . . and trigger path with unit clauses(in this example: {{¬C1},{C2},{¬D1},{D2},{¬F1},{F2}})
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 21 / 17
Encode small delay faults
A
B
C[D1, D2]
{{B2,¬D2}, {C2,¬D2}, {¬B2,¬C2, D2}}
{{¬D2, F2}, {¬E2, F2}, {D2, E2,¬F2}}
[F1, F2]
[G1, G2]
{{¬A2, G2}, {¬E2, G2}, {A2, E2,¬G2}}
{{B1,¬D1}, {C1,¬D1}, {¬B1,¬C1, D1}}
{{¬D1, F1}, {¬E1, F1}, {D1, E1,¬F1}}
{{¬A1, G1}, {¬E1, G1}, {A1, E1,¬G1}}
[E1, E2][A1, A2]
[B1, B2]
[C1, C2]
Encode both timeframes . . .
. . . and trigger path with unit clauses(in this example: {{¬C1},{C2},{¬D1},{D2},{¬F1},{F2}})
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 21 / 17
Encode small delay faults
A
B
C[D1, D2]
{{¬D2, F2}, {¬E2, F2}, {D2, E2,¬F2}}
[F1, F2]
{{¬D1, F1}, {¬E1, F1}, {D1, E1,¬F1}}[B1, B2]
[C1, C2]
Encode both timeframes . . .
. . . and trigger path with unit clauses(in this example: {{¬C1},{C2},{¬D1},{D2},{¬F1},{F2}})
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 21 / 17
Literature
[Batcher ’68] K. E. Batcher, “Sorting networks and their applications,” in AFIPS Spring
Joint Computing Conference, pp. 307–314, ACM, 1968.
[Jain et al. ’00] A. Jain, V. Boppana, R. Mukherjee, J. Jain, M. Fujita, and M. S. Hsiao, “Tes-
ting, Verification, and Diagnosis in the Presence of Unknowns,” in VLSI Test
Symp., pp. 263–269, 2000.
[Ravi, Somenzi ’04] K. Ravi and F. Somenzi, “Minimal assignments for bounded model
checking,” in Tools and Algorithms for the Construction and Analysis of Sys-
tems, vol. 2988, pp. 31–45, Springer, 2004.
[Sauer et al. ’11] M. Sauer, A. Czutro, T. Schubert, S. Hillebrecht, I. Polian, and B. Becker,
“SAT-based analysis of sensitisable paths,” in IEEE Design and Diagnostics
of Electronic Circuits and Systems, pp. 93–98, 2011.
[Schubert et al. ’10] T. Schubert, M. Lewis, and B. Becker, “antom — Solver Description,” in SAT
Race, 2010.
[Tseitin ’68] G.S. Tseitin, “On the complexity of derivations in propositional calculus,” in
Studies in Constructive Mathematics and Mathematical Logics, 1968.
01.24.13 Sven Reimer – Provably Optimal Test Cube Generation using Quantified Boolean Formula Solving 22 / 17