Download - Luca de Alfaro Thomas A. Henzinger Ranjit Jhala UC Berkeley Compositional Methods for Probabilistic Systems.

Luca de AlfaroThomas A. Henzinger

Ranjit Jhala

UC Berkeley

Compositional Methods forCompositional Methods forProbabilistic SystemsProbabilistic Systems

Compositional Methods for Probababilistic Systems 2

Introduction

• Compositional Model : – Construct large systems from models of components

• Shallow Compositionality: Syntactic– Given P, Q can construct PkQ

• Deep Compositionality: Semantic– |[ P k Q ]| a function of |[P]| , |[Q]|


Deep Compositionality : Example

• Transition systems with Trace Semantics

• Variable-based version: – System made of variables X– X-State: A valuation of the variables in X– X-Trace: A sequence of X-States, corresponding to a run– |[P]| : Set of X-Traces corresponding to all possible runs

– Private variables projected away

• Given components P, Q: – Read variables written by each other

– |[P k Q]| = |[P]| Å |[Q]|


Deep Compositionality

• Composition of properties– Allows decomposition of large verification tasks

• Simple Refinement Decomposition:– To check: P1 k P2 ¹ Q1 k Q2

– Suffices that: P1 ¹ Q1 and P2 ¹ Q2

• Assume-Guarantee Decomposition:– To check: P1 k P2 ¹ Q1 k Q2

– Suffices that: P1 k Q2 ¹ Q1 and Q1 k P2 ¹ Q2

• Crucial for non-deterministic systems– Even more beneficial in the probabilistic setting


Our Contribution

• First Deeply compositional model for systems with both Probabilistic and Non-deterministic choice

• Generalise semantic properties of trace-based models to the probabilistic setting

• First Assume-Guarantee rule for decomposing refinement checks for such systems


Previous Work

• A large body of work on the modelling and verification of probabilistic systems– Vardi 85, Courcoubetis & Yannakakis 89– Basic Model : Markov Decision Processes– Defining the behaviour using schedulers

• “Branching-time” models based on Process Algebras: Jonson & Larsen 91

• Probabilistic Process Algebras– Performance properties

• Models based on I/O Automata by Segala 95– Semantics described as Trace Distributions– Refinement as trace distribution inclusion


Plan

• Systems with Probabilistic and Non-determinisitic choice

• Why is deep compositionality tricky ?– Atoms, the solution to the scheduler problem

• Concrete Model : Probabilistic Modules

• Bundle Algebra

• Theorems

• Conclusions etc.


Probabilistic Systems

• We wish to model transition systems that can make both Probabilistic and Non-deterministic choice

¼ ¾ ½ ½

• At a state, the system does the following:

1. Picks one of several available distributions (or moves) over next state non-deterministically

2. Picks a next state randomly out of the chosen distribution


Prob. Systems: Example

¼ ¾ ½ ½

There are 2 possible behaviors arising from the non-deterministic choice at

• ¼ , ¾

• ½ , ½


Semantics: dealing with choices

• Non-deterministic, Probabilistic choice are “orthogonal”

• Factor out non-determinism using schedulers[Derman70, Vardi 1985, Courcoubetis & Yannakakis 1989]

• Given a scheduler, the execution is fully probabilistic – Outcome: A sequence of bundles of length i, 8 i

> 0– Semantics: Sum of the outcomes for all the

different schedulers


Schedulers: Example

1/2

1/2

4 Possible Schedulers, one outcome (bundle) for each

½ : , ½ :

½ : , ½ :

½ : , ½ :

½ : , ½ :

Outcomes (Bundles)Schedulers


Non-Det. Choice Vs Prob. Choice

1/2

1/2

A B

• Non-deterministic choice is more flexible than probabilistic choice

• We want A ¹ B, but …

Bundle of A

½ , ½

Bundles of B

1 1

1


, 1-

Non-Det. Choice Vs Prob. Choice

1/2

1/2

A B

• Solution: Let the scheduler be randomized

• The scheduler of B can flip a coin to select nondeterministic choice

•The move of B is then the convex combination of its simple moves

Bundles of B: For every 2 [0,1]

In particular = ½ matches A’s bundle


Semantics of Probabilistic Systems

X-State: A valuation of the variables in X

1/3 2/3X-Move: A probability distribution over X-States

Given a set of variables X:

X-Trace: A sequence of X-States

X-Bundle: A probability distribution over X-Traces 1/2

1/3

1/6X-Probabilistic Language: A set of X-Bundles



Refinement corresponds to bundle inclusion:– P ¹ Q if |[ P ]| µ |[ Q ]|

Given a Probabilistic system P with variables X,

semantics |[ P ]| is an X-Probabilistic language

X-Probabilistic Language: A set of X-Bundles


Plan




• Bundle Algebra

• Theorems



Why is it tricky ? (1)

P-1

X0

Y0

P0

X0

Y0

P1

X0

Y0

P0

X0

Y0

P1

X1

Y1

PPriv: PCtr : XExtl: Y

Q-1

X0

Y0

Q0

X0

Y0

Q1

X0

Y0

Q0

Y0

X0

Q1

Y1

X1

QPriv: QCtr : YExtl: X

PkQ Priv: P, Q Ctr : X , Y

Q-1

X0

Y0

P-1

Q1

X0

Y0

P1

Q0

X0

Y0

P1

Q1

X0

Y0

P0

Q0

X0

Y0

P0

Q0

X0

Y0

P0

Q1

X0

Y1

P0

Q0

X1

Y0

P1

Q1

X1

Y1

P1

1/2 1/2 1/4

This is the ONLY bundle of P k Q ) |[P]| Å |[Q]| ¾ |[ P k Q ]| !!

A bundle in |[P]| and |[Q]|



|[P]| Å |[Q]| ¾ |[ P k Q ]| !!

P-1

X0

Y0

P0

X0

Y0

P1

X0

Y0

P0

X0

Y0

P1

X1

Y1

PPriv: PCtr : XExtl: Y

Q-1

X0

Y0

Q0

X0

Y0

Q1

X0

Y0

P0

Y0

X0

Q1

Y1

X1

QPriv: QCtr : YExtl: X

PkQ Priv: P, Q Ctr : X , Y

Q-1

X0

Y0

P-1

Q1

X0

Y0

P1

Q0

X0

Y0

P1

Q1

X0

Y0

P0

Q0

X0

Y0

P0

Q0

X0

Y0

P0

Q1

X0

Y0

P0

Q0

X0

Y0

P1

Q1

X0

Y0

P1

• External variable was scheduled looking at private variable …

• … this breaks compositionality

) must have two schedulers

1. CONTROLLED-VAR scheduler: can look at private variables

2. EXTERNAL-VAR scheduler: cannot look at private variables



P Ctr : X, non-det

Extl: Y

Q Ctr : Y, non-det

Extl: X

PkQ Ctr : X , Y

X,Y are non-det. set With a single scheduler we

get :

• No matching bundle in |[P]| or |[Q]|• |[P]| Å |[Q]| ½ |[ P k Q ]| !!

• ) A composed system must be made up of schedulers for individual components

X:=1

X:=0

1-

Y:=1

Y:=0

1-X0

Y0

X1

X0

X1

Y0

Y1

X1

(1-)(1-) (1- )

(1-)

1/2

X0

Y0

X1

X0

X1

Y0

Y1

X1

0 0 1/2


Ex 2: After composition, joint scheduling breaks compos.

Ex 1: Environment must not see private variables !

Schedulers and Compositionality

Q: Why are previous models not deeply compositional ?

A: Monolithic Schedulers are bad !!

Module P

Interface x

Private p

External y

Module Q

Interface y

Private q

External x

Module P k Q

Interface x

Private p

Interface y

Private qCompose


Atoms : The Solution to the Scheduler Problem

A single scheduler associated with each atom

- Module Scheduler is the “composition” of atomic schedulers

Atomic (scheduling) structure preserved after parallel composition

Module P

Reads x,p,y…

Writes x,p… External y,…

Writes y

Reads ObsReads x,…

Writes …

Module Q

Reads y,p,x…

Writes x,p… External x,…

Writes x


Writes …

Compose

Module P k Q

Reads x,p,y…

Writes x,p…

Reads x,…

Writes …

Reads y,p,x…

Writes x,p…

Reads x,…

Writes …

External …

Writes …

Reads Obs

Atoms : Units of Scheduling

Variables written by the atom

Variables read : on whose history non-det. is resolved


The Importance of Atoms

Module A

Atom Axy controls x,y

Init

[] true-> x,y:=0,0

[] true-> x,y:=0,1

[] true-> x,y:=1,0

[] true-> x,y:=1,1

Module B

Atom Bx controls x

Init

[] true-> x:=0

[] true-> x:=1

Update

[] . . .

Atom By controls y

Init

[] true-> y:=0

[] true-> y:=1

Update

[] . . .

• |[A]| |[B]| because:

• A has a bundle where x,y have correlated values { ½: 0,0 ½: 1,1}

• In B’s bundle it is not possible to get correlation, despite complete non-det in each atom, as the schedulers are independent


Plan




• Bundle Algebra

• Theorems



Probabilistic Modules

Module A

Interface x,w Private y External z

Atom AXY control x,y read x,y,z

Init

[] true-> ½ x,y:=0,0 ½ x,y:=1,1

Update

[] true-> x’,y’:= x,x

[] y ->’¼ x’y:=:z,z ¾ x’y’= z,:z

Atom Aw control w read y,z

Init

[] true-> w:=0

[] true-> w:=1

Update

[] true-> w’:= z

Update : To each state, associate a set of distributions (moves), for next state

Z1

X1

Y1

X1

1

Move 1Y1

X0

1/4

Y1

X1

Y0

3/4

Move 2

The atom scheduler

Chooses between moves


Operations : Parallel Composition

Module P

Reads x,p,y…

Writes x,p…External y,…

Writes y

Reads Obs Reads x,…

Writes …

Module Q

Reads y,p,x…

Writes x,p… External x,…

Writes x


Writes …


Operations : Parallel Composition

Module PkQ

Reads x,p,y…

Writes x,p…

External y,…

Writes y

Reads Obs

Reads x,…

Writes …

Reads y,p,x…

Writes x,p…

Reads x,…

Writes …


Module A

Interface x,w Private y External z

Atom AXY control x,y read x,y,z

Init

[] true-> ½ x,y:=0,0 ½ x,y:=1,1

Update

[] true-> x’,y’:= x,x

[] y ->¼ x’y’:=:z,z ¾ x’y’= z,:z

Atom Aw control w read y,z

Init

[] true-> w:=0

[] true-> w:=1

Update

[] true-> w’:= z

Module Semantics


Module Semantics

Module A

Reads x,y,z

Writes x,y

External z

Writes z

Reads x,w

Reads y,z…

Writes w…1 2

env

Schedulers for every atom

Each Scheduler takes a trace, returns a move

:1/3 2/3

Every triple (1,2,env) generates a bundle

1/2

1/3

1/6

|[A]| = Union over all triples (1,2,env) :


Composing Atomic Schedulers

XP

CtrP

XQ

CtrQ

XP

CtrP

XQ

CtrQ

Project Project

P

CtrP Move

Q

CtrQ Move

P£Q=P||Q

£

CtrP [ CtrQ = XPkQ Move


Semantics: Atomic Schedulers

Composing Atom Schedulers:

For schedulers 1 from X1 to Y1, 2 from X2 to Y2, s.t. Y1 Å Y2 = ?,

(1 £ 2) : from X1 [ X2 to Y1 [ Y2 s.t. (1 £ 2)(t) = 1(t[X1]) £ 2(t[X2])

For sets of schedulers 1 from X1 to Y1, 2 from X2 to Y2,

1 £ 2 = { 1 £ 2 | 1 2 1, 2 2 2}


Module Semantics

Schedulers of P • extl(P) = set of all schedulers from extlX(P) [ intfX(P) to

extlX(P)

• mod(P) = extl(P) £ A 2 Atoms(P) atom(A)

Language of P • L(P) = [ 2 mod(P) Outcome()

Trace Semantics of P• |[ P ]| = L(P)[obsX(P)]

– the language projected to the observables


Plan




• Bundle Algebra

• Theorems




X-State: A valuation of the variables in X

1/3 2/3X-Move: A probability distribution over X-States

Given a set of variables X:

X-Trace: A sequence of X-States

X-Bundle: A probability distribution over X-Traces 1/2

1/3

1/6X-Probabilistic Language: A set of X-Bundles


Bundle Algebra

For reasoning about parallel composition

Decomposing : ProjectionGiven sets of variables X, X’ s.t. X’ µ X– X-Bundle X’-Bundle

Composing : ProductGiven sets of variables X, Y– X-Bundle £ Y-Bundle (X [ Y) – Bundle


Projection : StatesX’

X

X State

X’State


Projection : Moves

1/9 1/9 1/9 1/6 1/6 1/9 1/91/9

1/3 1/31/3

X

X Move

X’ Move

X’

X’


Projection : Bundles1/8 1/12 1/12 1/24 1/61/6 1/9 1/91/9

1/3 1/3 1/3

X Bundle

X’ Bundle


Product : States

X

XY X Z

X [ Y State X [ Z State

X [ Y [ Z State

Y X Z


Product : Moves, Bundles

£

X [ Y Move

X [ Z Move X [ Y [ Z Move

Y X

=

X Z

1

2

1

.25

.25

.5

1

2

3

1

2

.166

.166

.166

.25

.25

1 1

1 2

1 3 2 3

2 2

2 1

1 1

1 2

.25 x.166 / .5

.5 x.25 / .5


Operations : Product

Product:Given 2 sets of variables X1, X2 :– Given an X1-State s1, a X2-State s2:

s1, s2 can be multiplied if s1 [X1 Å X2] = s2[X1 Å X2]

– Same condition for for Traces and Bundles

– Given an X1-Bundle b1, X2-Bundle b2:

(b1 £ b2): X1 [ X2 – Bundle s.t.

(b1 £ b2)(t) = b1 (t[X1]) £ b2 (t[X2]) / b1 (t[X1 Å X2])

– Given an X1-Language L1, X2-Language L2:

L1 £ L2 = { b1 £ b2 | b1 2 L1 and b2 2 L2 can be multiplied }


Plan




• Bundle Algebra

• Theorems



Compositional Semantics

Theorem: |[ P1 k P2 ]| = |[ P1 ]| Å |[ P2 ]|

• This is because L(P1 k P2) = L(P1) £ L(P2)

• For every b1 2 L(P1), b2 2 L(P2),

s.t. b1[X(P1) Å X(P2)] = b2[X(P1) Å X(P2)] … are multipliable

b1 £ b2 2 L(P1 k P2)

• For every b 2 L(P1 k P2)

b[X(P1)] 2 L(P1) and b[X(P2)] 2 L(P2)


Recall : Probabilistic Refinement

Refinement corresponds to bundle inclusion:– P ¹ Q if |[ P ]| µ |[ Q ]|

Given a Probabilistic system P with variables X,

semantics |[ P ]| is an X-Probabilistic language

X-Probabilistic Language: A set of X-Bundles


Refinement Is Compositional

Module Refinement: P ¹ Q iff |[ P ]| µ |[ Q ]|

Theorem: Refinement is Compositional • P k Q ¹ P• If P ¹ Q , then P k R ¹ Q k R

– Follows from deep compositionality

Theorem: Assume-GuaranteeIf P1 k Q2 ¹ Q1 and Q1 k P2 ¹ Q2,

then P1 k P2 ¹ Q1 k Q2– Deep compositionality– Induction


Conclusions

• Deeply compositional semantics for systems with Non-deterministic and Probabilistic choice

• Assume-Guarantee rule

• Only possible by restricting the visibility and influence of schedulers

• Checking Bundle Inclusion– Simulation based approach

• Adding combinational (0-delay) dependencies

• Logics for Specification:– Correctness and performance properties– Compositional reasoning