Large-Scale Knowledge Processing (1st part, lecture-4)minato/LSKP2017/lskp2017... · 2017-12-13 ·...

Large-Scale Knowledge Processing (1st part, lecture-4)

Shin-ichi Minato

Division of Computer Science and Information Technology,Graduate School of Information Science and Technology,

Hokkaido University

Review of the last lecture

– Examples to show the effect of variable ordering.– Two empirical rules.

• Weight assignment method when generating BDDs from Boolean expressions or logic circuits.

– Minimization for a given BDD.• Exact methods: all enumeration, dynamic programming.• Heuristic methods: adjacent exchange, minimum-width.

– Dynamic variable ordering• “Built-in” ordering method for BDD package.• Locality of adjacent exchange.• Effect and computation time

Variable ordering for BDDs

2017.12.13 2Large-Scale Knowledge Processing

Topics of this lecture

• BDD applications– VLSI design automation

• Verification, logic optimization, test generation– Combinatorial problems, optimization

• Knapsack problem, 8-queens, Traveling salesman, etc.– Comparison with problem-specific method

• Representation of sets of combinations and ZDDs (Zero-suppressed BDDs)

• Sets of combinations• ZDD reduction rule• Basic properties of ZDDs

BDD applications and ZDDs


BDD application to LSI design systems

• BDD is originally developed for hardware (logic circuit) design. – Logic simulation, verification– Logic synthesis, optimization– Test pattern generation

• LSI logic design is hard problem to deal with very large-scale Boolean function data.– Too large to solve by hand.– No error allowed. ( Case of Pentium’s bug.)– Hard deadline of design time. (company’s competition.)– Big market.


Verification of combinational circuits• Equivalence checking of two combinational circuits.

(not including loops and registers.)

For any input pattern, the output becomes the same value or not?(naïve method requires 2n steps.)

If we can generate BDDs for the primary outputs, equivalence checking can be done just by comparing pointers to the BDDs.

F1

x1x2x3x4x5

Y1

Y2

F2

x1x2x3x4x5

Y1

Y2


Exprimental result of BDD generation• Evaluated by the practical benchmark circuits.

– Even for the circuit with hundreds of gates, we can generate BDDs in a few minutes, if the function is “BDD-friendly.”

Size of circuitsName Inputs outputs gates BDD nodes time(sec)sel8 12 2 29 78 0.3enc8 9 4 31 56 0.3adder8 18 9 65 119 0.4adder16 33 17 129 239 0.7mult4 8 8 97 524 0.5mult8 16 16 418 66161 24.8c432 36 7 203 131299 55.5c499 41 32 275 69217 22.9c880 60 26 464 54019 17.5c1355 41 32 619 212196 89.9c1908 33 25 938 72537 33.0c5315 178 123 2608 60346 31.3

(using Sun3, 24MByte)


Verification of sequential circuits• Equivalence checking of sequential circuits, including

registers (memory units, flip-flops).

F2

x1x2x3x4x5

Y1

Y2

F1

x1x2x3x4x5

Y1

Y2

T1

T2

T1

T2

For any input pattern sequence, the output becomes the same sequence or not? (much moredifficult than combinational one.)

“State” of the circuits are decidedby the values of registers. If wecan find correspondence of the states of the two circuits, we may apply combinational equiv. checking for each state.


Model checking for sequential circuits

• To check whether a given condition (=model) is satisfied or not in the designed circuit.

Also used for verifying software and communication protocols.

F1

x1x2x3x4x5

Y1

Y2

T1

T2

cond1: T1and T2 neverbecome 1 together.

cond2: Y2 infinitelyoften becomes 1.

Check contradiction

- All reachable state set of T1, T2 is represented by BDDs.- In the reachable state set, check the conditions by BDDs.


BDD application to logic synthesis/optimization• Automatic generation of a good (less logic gates and

delay) circuit to satisfy a given specification.• Generate a certain quality of initial circuit, and then

minimize/optimize the circuit.• A method of optimization by repeating local

transformation of the circuit. ( transduction method)– Local transformation must not change the global logic

function of the circuit.– Using BDD for checking global logic change, hundred

times larger circuits become possible to be optimized.• A method of generating an SOP expression as initial

circuit, then factorizing SOPs into multi-level SOPs to generate compact circuit.– Fast BDD-based method for SOP generation and

factorization is known.2017.12.13 9Large-Scale Knowledge Processing

Transduction method

0101

0011

11001010

1011

1101

0110

• Repeating addition and deletion of lines, reduce the number of gates and lines.

• For each gate output, we compute the condition of addition and deletion without changing the output function.


Transduction method• Repeating addition and deletion of lines, reduce

the number of gates and lines.• For each gate output, we compute the condition of

addition and deletion without changing the output function.

addinglines










add

delete




0101

0011

1011

1101

0110

1110


Test pattern generation• In manufacturing LSI,

if tiny dust falls into it,erroneous circuit appears.– Occurs in some % ratio.– Erroneous ones must be

detected before shipping.• We cannot directly see the internal status of the circuits.

– Providing some input pattern and if we observe output value different from expected one, then we can detect the error.

– For each error point, we have to prepare an input pattern (= test pattern) to detect the error.

– To reduce testing time, we want to reduce the number of test patterns. (we may detect many errors with a same pattern.)

• Test pattern generation requires large-scale logic operations.– BDD is useful for this problem.

LSI circuit

x1x2x3x4x5

Y1

Y2×


BDD application to combinatorial problems• There are many studies to apply BDDs to solve basic

problems in graph theory.– ex. maximum clique, maximum independent set, etc.

• Basically, from a set of edges (or nodes) of the given graph, we find a subset of them satisfying a condition.

• Using a logic variable for each edge (or node) of the graph, and possible combinations are represented by a BDD.

• If the condition to be satisfied is written in logic expressions, and if BDD can be constructed, then minimum cost solution can be obtained.– However, BDD size may become too large to construct in a

limited memory size. Not always easily done.


Example of comb. problem (Knapsack problem)• There are n items. Each item has a price and a weight.

Up to the weight capacity of my knapsack, I want to buy a subset of items whose total price is maximum. – Each variable xi represents that buy i-th item (=1) or not (=0).– Let item’s weight: ai, price: ci, and knapsack capacity: b,

then under the costraint Σai xi ≦ b, find a combination of xi’s to make Σci xi maximum.

• Constraint Σai xi ≦ b can be a Boolean function with input variables x1…xn .– If the function is represented by a BDD each path from the

root to 1-terminal corresponds to a possible item combination.

– The path of maximum total price can be found in a linear time for the BDD size.


Example of comb. problem (8-queens problem)• In 8×8 chess board, find a layout of 8 queens with

no collisions to each other. • We prepare 64 logic variables.

– Each variable representsexistence (1/0) of a queenat each position.

• Put constraints by logicexpressions.– Just 1 queen on each column.– Just 1 queen on each row.– 0 or 1 queen on each

diagonal line.• BDD for conjunction

(AND) of all constraintsrepresent the set of solutions.

QQ

QQ

QQ

QQ


Experimental results for N-Queens problem

• Layout of N queens on N×N chess board.• We generated BDDs and counted the number of

solutions.– N=13 is upper limit in a practical time.– Specific (non-BDD) method for N-Queens finds a solution for

N=100. (However, they do not enumerate all solutions.)

N variables BDD nodes solutions time(sec)8 64 2,450 92 6.19 81 9,556 352 18.3

10 100 25,994 724 68.811 121 94,821 2,680 1081.9

(SPARC station2, 40MByte)


Other combinatorial problems• Knight’s tour problem

– To find a way of a knight (八方桂馬） to visit all positions on a chess board, and finally return to the original position.

– BDD-based method [Lobbing et al.1996] is reported.• Traveling salesman problem

– To find a shortest path visiting N cities once and return home.– BDD-based method:

There are N(N-1)/2 edges connecting any pair of cities. We assign a logic variable for each edge (use=1, not use=0). Generate a BDD for all visiting tour (Hamilton paths), and then find a minimum cost solution. (Experimental result as follows.)

N variables BDD nodes solutions time(sec)8 28 2,054 2,520 8.79 36 64,72 20,160 28.8

10 45 19,972 181,440 216.5

(SPARC station2, 40MByte)2017.12.13 20Large-Scale Knowledge Processing

Various problems in graph theory• Problems which BDD-based method applicable:

– Maximum cliques （最大クリーク問題）

– Maximum k-cover （最大kカバー問題）

– Minimum α-covering （最小α被覆問題）

– Maximum independent set （最大独立集合）

– Minimum vertex cover （最小節点カバー）

– Minimum coloring （最小彩色問題）

– Minimum clique partition （最小クリーク分割）

– Minimum clique cover （最小クリークカバー）

– Minimum dominant set （最小支配集合）

• However, sometimes problem-specific non-BDD method is more efficient. It depends on the property of the functions. We should be careful.


Comparison with problem-specific method• BDD-based method solves any problem automatically

if we can describe the constraints in logic expressions.– Fast and easy programming for a given problem.– BDD enumerates large number of solutions at once.

• However, for many popular problems, some specific (non-BDD) algorithms already have been developed.– Using problem-specific properties, they solves more

efficiently than BDD-based general method.– Especially, problem-specific method may be stronger when

finding only one optimum solution. For example, a solution of traveling salesman can be found by a specific algorithm for n=1000. BDD-based enumerative method is limited to around n=15.


BDD-based knowledge DB and inference

• BDDs are also used in the area of artificial intelligence. • Inference/proof of propositional logic can be handled

by Boolean function manipulation. ( using BDDs)– BDD-based method has the following limitations:

• Variable ordering should be fixed in all propositions.• All data should be represented in the main memory.

– When the property of the problem matches with BDDs, great acceleration will be obtained.


Summary of BDD applications• In many commercial tools (especially in LSI design),

BDD techniques are used as “internal engine.”• BDD is “efficient enumeration technique.”• If the problem is written in Boolean expressions, then it

can be solved straightforward by BDDs.• BDD works only in the main memory.

– Memory overflow may occur for large-scale problems.• BDD-based method may be slower than a problem-

specific, state-of-the-art algorithm.– However, solver development may be much easier and fast.

• Not so strong for finding one solution, but very powerful for enumerating all the solutions and checking no counterexample exists.

• ZDDs are used for manipulating sets of combinations. Applications are spreading to Data mining and AI.


Boolean function and combinatorial itemset

Boolean function:F = (a b ~c) V (~b c)

Combinatorial itemset:F = {ab, ac, c}

a b c F0 0 0 01 0 0 00 1 0 01 1 0 10 0 1 11 0 1 10 1 1 01 1 1 0

c ab

ac• Operations of combinatorial itemsets

can be done by BDD-based logic operations.– Union of sets logical OR– Intersection of sets logical AND– Complement set logical NOT

(customer’s choice)


• Improvement of BDDs for representing combinatorial iemsets.– Proposed and named by Minato in 1993.– Based on different

reduction rules.– Effective for sets

of sparse comb.(Super marketdisplays thousandsof items but eachcustomer chooseonly tens.)

• ZDDs are now widely used in many applications as well as BDDs.– Recently, used for data mining applications and remarkable results are

reported. (Hundred times data compression and acceleration.)– Application areas are still now spreading.

ZDD: Zero-suppressed BDD

x

f f

(jump)

Ordinary BDDs

x

f f

(jump)

0

Zero-suppressed BDDs


Comparison of BDD and ZDD

2017.12.13 Large-Scale Knowledge Processing 27

(BDD and ZDD reduced froma same binary decision tree.)

Features of ZDDs

• Nodes of Irrelevant items are automatically eliminated.– In ordinary BDDs, irrelevant nodes still remain and they may

spoil the reduction effect of sharing similar nodes.• ZDDs are especially effective for sparse combinations.

– For instance, sets of combinations selecting 10 out of 1000 items, up to 100 times compact than ordinary BDDs.

• Number of the paths from the root node to the 1-terminal node corresponds to the cardinality of the set.– In ordinary BDDs, this property does not always hold.


ZDD and BDD for a set of sparse comb.


ZDD and BDD for a set of dense comb.


Comparison with linear list and ZDDs

• When no nodes can be shared, the ZDD explicitly enumerates all combinations as well as linear linked list. – Namely, ZDD size never

exceeds the (order of) explicit representation.

– More nodes are shared, more compact than linear list.

– Ordinary BDDs have larger overhead to represent sparser combinations. ZDDs have no such overhead.

{ abcd, bc, cde }

d

0 1

a0 1

bc

d

b

c

e

c0 1

01


Frequent itemset mining problem• Basic and well-known problem in database analysis.• Enumerate all sub-itemsets which frequently appear more times than a given

threshold in the database records.– Hard problem since the number of all itemsets becomes exponential.– Enumeration results may also become large-scale size.

Record ID itemset

1 a b c2 a b3 a b c4 b c5 a b6 a b c7 c8 a b c9 a b c10 a b11 b c

α = 8 { ab, a, b, c }

α = 7 { ab, bc, a, b, c }

α = 5 {abc, ab, bc, ac, a, b, c }

α = 10 { b }

α = 1 {abc, ab, bc, ac, a, b, c }

α：min. freq.


“LCM-ZDD” [Minato et al. 2008]

• LCM: [Uno2003]Output-linear time algorithm of frequent itemset mining.

• ZDD: [Minato93]A compact graph-based representation for large-scale sets of combinations.

Combination of the two techniques

Generates large-scale frequent itemsets on the main memory, with a very small overhead from the original LCM.

( Sub-linear time and space to the number of solutions when ZDD compression works well.)


A ZDD representing Frequent itemsets

Min. freq. α = 7

{ ab, bc, a, b, c }

LCM-ZDD

F

a

b b

c c

0 1

0

0

0

0 0

1

1 1

11

Record ID itemset

1 a b c2 a b3 a b c4 b c5 a b6 a b c7 c8 a b c9 a b c10 a b11 b c



Original LCMLCM-ZDD# solutions


All Freq. Itemsets

Post Processing after LCM-ZDD

• We can extract distinctive itemsets by comparing frequent itemsets for multiple sets of databases. – Various ZDD algebraic operations can be used for the

comparison of the huge number of frequent itemsets.

Dataset 1

Dataset 2

LCM-ZDD

LCM-ZDD

ZDD

ZDD

All Frequent Itemsets

?ZDD algebraicoperation

ZDD

Distinctive Frequent Itemsets

• The latest Knuth’s book fascicle (Vol. 4-1) includes a BDD section with 140 pages and 236 exercises.

• In this section, Knuth used 30 pages for ZDDs, including more than 70 exercises.– Knuth recommended to use “ZDD” instead of “ZBDD.”

( important as well as BDD)– He reorganized ZDD

operations and named“Family Algebra.”

– It is the first Japaneseresearcher’s results writtenin this famous book series as a separated section.

– Honored to serve proofreading of the draft version of his article.

– His recent lecture at Oxford and Stanford was titled “Fun with ZDDs.

BDDs/ZDDs in the Knuth’s book


38

Checks from Knuth (Certificate of error-correction)

(until mid-Oct. 2008)Check of real bank account.Error correct: 2.56 $Improvement: 0.32 $(No one cash the check.)

（after mid-Oct. 2008）Certificate of a virtual bank.Error correct:

0x1.00 $ (hexadecimal)

2017.12.13 Large-Scale Knowledge Processing

39

Algebraic operations for ZDDs

• Knuth evaluated not only the data structure of ZDDs, but more interested in the algebra on ZDDs.

φ, {1} Empty and singleton set. (0/1-terminal)P.top Returns the item-ID at the top node of P.P.onset(v)P.offset(v)

Selects the subset of itemsets including or excluding v.

P.change(v) Switching v (add / delete) on each itemset.∪, ∩, ＼ Returns union, intersection, and

set difference.P.count Counts number of combinations in P.P * Q Cross-product set of P and Q. P / Q Quotient set of P divided by Q.P % Q Remainder set of P divided by Q.

Basic operations(Corresponds toBoolean algebra)

New operationsintroduced by Minato.

Formerly I called this “unate cube set algebra,”but Knuth reorganized as “Family algebra.”

Useful for many practical applications.

2017.12.13 Large-Scale Knowledge Processing

1

{λ}

change(c)

0 1

c10

{c}

1

c10

{λ, c}0 1

b

c

10

0 1

{b, bc}

0 1

a10

{a} 1

b

c

10

0 1

{λ, b, bc, c}

1

b

c

10

0 1

a1

0

{λ, a, b, bc, c}

change(a)

union

change(b)

union

union

Example of ZDD construction


Boolean function vs. combinatorial itemset

Boolean function itemset(set of combinations)

Boolean algebra:AND/OR/EXOR

set algebra: union /intersection / diff.

BDD（二分決定グラフ）

ZDD（ゼロサプレス型BDD）

logic design data mining

datastructure

basicoperation

mainapplication

data model



Comparison of BDDs and ZDDs

Naïve representation

Quasi-reduced BDD

BDDZDD

Sub-graph sharing (Dictionary-based)

Symmetric reduction Asymmetric reduction

- VLSI Logic Design- Formal Verification

(Symmetric world)

- etc.

(Asymmetric world)- Data mining & Knowledge discovery- Market data analysis - Web data analysis

- Risk analysis- Calculation with Bayesian networks- Machine learning & clustering

- Formal concept analysis

etc.

Many of real-lifeproblems are likely asymmetric.

x

f f

(jump)x

f f

(jump)

x

f f

(jump)

0

x

f f

(jump)

0

x

f0 f1

x x

f0 f1

(share)x

f0 f1

x x

f0 f1

(share)

Summary

• BDD applications– VLSI design automation

• Verification, logic optimization, test generation– Combinatorial problems, optimization

• Knapsack problem, 8-queens, Traveling salesman, etc.– Comparison with problem-specific method

• Representation of sets of combinations and ZDDs (Zero-suppressed BDDs)

• Sets of combinations• ZDD reduction rule• Basic properties of ZDDs

BDD applications and ZDDs


Exercises

• Confirm that BDDs’ logical OR operation algorithm can be applied for ZDDs as well, and it corresponds the Union operation for sets of combinations.

• Similarly, BDDs’ logical AND operation algorithm can be applied for ZDDs as well, and it corresponds the Intersection operation for sets of combinations.


Date post:	06-Aug-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Large-Scale Knowledge Processing (1st part, lecture-4)minato/LSKP2017/lskp2017... · 2017-12-13 ·...

Documents