Jan Lemeire December 19 th 2007

transcript

Learning Causal Models of Multivariate Systemsand the Value of it for the Performance Modeling of Computer Programs

Jan LemeireDecember 19th 2007

Supervisor: Prof. dr. ir. Erik Dirkx

2Causal Inference & Performance Analysis

Pag.Jan Lemeire / 49

OverviewLearning causal models for the performance analysis of programs executed on various computer systems.

Intermezzo I: Causal inference.Practical deployment of the causal learning algorithms.Philosophical and theoretical study of causal inference.

Intermezzo II: Kolmogorov Minimal Sufficient Statistics.The importance of qualitative properties.

Intermezzo I: Causal Inference.Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference.

Intermezzo II: Kolmogorov Minimal Sufficient StatisticsThe importance of qualitative properties.

What is Parallel Processing?

Ideally: Speedup = number of processors

Computational work:

Parallel system

Parallel Overhead

Speedup = 2.55 Overhead = time the processors are not spending on useful work

= lost processor cycles

Overhead Analysis

ratiosoverheadprocessorsofnumberSpeedup

runtimetimeoverheadratioOverhead

Impact of overhead on speedup

Experimental Parallel Performance Analysis: Data Acquisition

Parallel Program

EPPADatabase

EPPA instrumentation

library

Executable

EPDA: Multivariate Analysis

User-defined variables

Database

1.5 2 1842.5 4 8360.9 1 1043

Specify context

Modeling

Causal Model

Curve fitting

Analytical Model

CPT compression

Causal Inference

Derivatives of variables

Visualization

Outlier identification

Augmented Model

Intermezzo I: Causal Inference

System under study

Data A B C D Eexperiment 1 2 12 0.42 TRUE blueexperiment 2 1 73 1.93 FALSE greenexperiment 3 4 8 0.03 TRUE redexperiment 4 2 27 2.84 TRUE black

CACausal model

Experiments

Causal Inference for PerformanceAnalysis

Utility based on the following properties:1. Dependency analysis: how variables relate.2. Markov property.3. A causal model corresponds to a decomposition.

fclock

#instrop

array size

cache misses

memory

element typeCinstr

element size Cmem

PROGRAM#op

fclock

array size

cache misses

memory

element type

Cinstr

#instrop

??element size

PERFORMANCE

Execution of program gives cache misses

Datastructure

PROGRAMx?

datatype (integer, float, double,…)

data size in Bytes

Markov Property

cache missesdatatype

cache missesdatatype data size

data sizedatatype cache misses

Provides explanationsDifferentiate direct from indirect relations

Correlated

With information about the data size:

Cannon angle

Can we Observe Causal Relations?

distance~OK, but: or ???

What is Causality?A causal relation denotes a mechanism, that a variable

is `produced’ by its causes. However… not directly observable.

Causality is a relic of a bygone age

Bertrand RussellJudea Pearl

But: we want to learn something about underlying system (goal of statistics)

gunpowder distance~

V-structure Propertycannon angle

distance

gunpowder

angle independent from gunpowder

but dependent when distance is known

Conditional Independencies Make Causal Inference Possible

From a causal structure follow conditional independencies, irrespective of the mechanisms.– Markov– V-structure

Graph is a Description of Independencies

Graphical criterion: d-separation– Intuitive

Faithfulness property: independencies independencies in graph in reality

Causal Structure Learning

In two steps:1. Undirected

graph2. Orientation

C D B(c)

DA DA D B

DA CA C B

Result

Partially directed acyclic graph

“We know what parts are unknown.”Faithfulness assumption: all independencies follow from the causal structure

Experimental Results(1) Automatic learning of accurate performance models(2) Model validation(3) Identification ofunexpected dependencies(4) Explanations for outliers

Contribution 1Figuur opnieuw in png, zonder losless compression

Intermezzo I: Causal Inference.Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference.

Practical Causal Inference

The following limitations had to be overcome:Non-linear relations: form-free independence testMixture of continuous, discrete and categorical data: general independence testDeterministic relations: augmented causal model and extended learning algorithms

Form-Free and General Dependency Test

Mutual information

Example

Kernel density estimation

Pearson:Rxy=0.083 => X and Y linearly independent

I(X;Y)=0.90 bits => dependent

P(X, Y)

Deterministic Relations

data sizedatatype cache misses

Data size and data type are information equivalent with respect to cache missesDuring learning connect least complex relation

Complexity Criterion

Correct models are learned under the Complexity Increase Assumption

Contribution 2a

mech1 mech2X Y Z

Complexity( X – Z ) ≥ Complexity( X – Y )Complexity( X – Z ) ≥ Complexity( Y – Z )

Reestablishment of Faithfulness

Consequences are consideredInformation equivalences

Independence and simplicity

D-separation extension

Faithful model: represents all independencies

Contribution 2b

X and Y eq. for A

Information is added to the model Basic information equivalences

Y Z X Y Z XS

Y Z X Y Z Xeq

Dit moet erbij!!Details misschien niet?

Extension of PC Learning Algorithm

Detection of information equivalencesAmong information equivalent relations, the simplest one is chosenOrientation rules remain the same

Correct models are learned from data containing deterministic relations.

Contribution 2c

Intermezzo I: Causal Inference.

Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference.

Inductive Inference

Occam’s Razor“Among equivalent models

choose the simplest one.”William of Ockham

Jaartallen van scientists erbij zetten

BUT: Objective measure of complexity?

2.cmE 3. HFmE hyx vhm

22yx ddH

vvgF yx

Kolmogorov Complexity

Andrey Kolmogorov

REPEAT 11 TIMES PRINT "001"

PROGRAM

001001001001001001001001001001001

Universal Turing Machine

Kolmogorov Complexity of a binary string: the length of the shortest program that

computes the string and halts

Shortest Programs

REPEAT 11 TIMES PRINT "001"

PROGRAM

PRINT "01100011010110 1010111001001101000"

PROGRAM

regularity of repetition allows compression

011000110101101010111001001101000

random information = incompressible

Randomness versus Regularity

001001001001001001001001001001001

011000110101101010111001001101000Only random information (incompressible)

Kolmogorov Minimal Sufficient Statistics (KMSS): formal separation

Meaningful informationregularities

Accidental information randomness

repetition 11 times, 001

Learning = finding regularities = maximal compression

regularitiesrandom

Structure of a diamond Exact size

random

P(A)P(B)P(C|A, B)P(D|B)P(E|C, D)

CPDsGRAPH

Meaningful Information of Probability Distributions

meaningful information (Theorem 1)

Kolmogorov Minimal Sufficient Statistic if graph and CPDs are incompressible (Theorem 2)

Contribution 3a

a graph with random CPDs is faithful (Theorem 4)

Joint Probability Distribution

P(A, B, C, D, E)= E

P(A)P(B)P(C|A, B)P(D|B)P(E|C, D)

CPDsGRAPH

Causal Aspect of Causal Models = Decomposition

Canonical decomposition: quasi-unique and minimal decomposition into atomic and independent components (the CPDs)Corresponds to reality (mechanisms)

Causal Component Relies on Reductionism

When DAG of Bayesian network is a complete graph

no meaningful information holism

The world can be studied in parts.Or, even more:

The world is made up of indivisible parts.

Figuurtje toevoegen van holisme en reductionisme

Validity of Causal Inference

Do CPD components correspond to physical mechanisms?

Contribution 3b

Minimal model?Faithful?Other regularities?

How OK is the learned causal model?

Unfaithful

Other regularities

Conformreality

Wrongdecomposition

Counterexamples from literature

3, 6 2, 5, 7, 8Causal model ≠

minimal model{

Well-known Example of Unfaithfulness

’Normally’:A and D correlate

A and D get independent if influences along paths 1 and 2 cancel each other out

Mechanisms are relatedRegularity among them

Intermezzo I: Causal Inference.

Practical deployment of the causal learning algorithms. Philosophical and theoretical study of causal inference.

Regularities are Qualitative Properties

Different from quantitative information.

Allow for qualitative reasoning.

Qualitative properties determine behavior.

Communication Schemes on Network Topologies

Communication Scheme

Network Topology

Communication time?

Generic Performance Model

Good predictions for combinations of random schemes and random topologies

Contribution 4a

Scheme1

Topo-logy1

model Tcomm

Scheme2 Scheme3

Topo-logy2

Topo-logy3

Combinations of Patterns

broadcast shift

Communication Schemes

star ring

Network Topologies

Performance depends onmatch!

Contribution 4b

Qualitative Properties

Faithfulness: ”graph should describe all independencies”

KMSS: ”model should describe all regularities”

Qualitative information Quantitative informationcontains no more regularities

explicitly describe regularities

Explicitly Mention Qualitative Properties!

(12,61)

(9,41)

(19,24)

(2,12)

(5,21)

??(12,61)

(12,61)

(9,41)

Conclusions

Contribution to performance analysis.Automatic causal analysis.Useful add-on in combination with other techniques.

The value of causal inference is underlined.The importance of regularities or qualitative properties.

Future Work

Application of the learned performance models for optimization.Is the failure of generic performance models only due to regularities?Augment models with qualitative properties.But: how define, recognize and reason with regularities?

Jan Lemeire December 19 th 2007

Documents