Verier Finite Horizon Analysis of Markov Chains with the Murmelatti/relazioni/MEFISTO2003.pdf ·...

MEFISTO- 11/2003

Finite Horizon Analysis of Markov Chains with the Mur � Verifier

Giuseppe Della Penna Benedetto Intrigila Igor Melatti

Dip. di Informatica, Universita di L’Aquila

Enrico Tronci Marisa Venturini Zilli

Dip. di Informatica, Universita di Roma “La Sapienza”

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –1–

MEFISTO- 11/2003 Probabilistic Model Checking

Markov Chain analysis

Given the description of a Markov Chain, it verifies a PCTL property

PCTL: Probabilistic CTL

– true

– true

Very few available probabilistic model checkers

– PRISM

– Two Towers

– FHP-Mur (new)


MEFISTO- 11/2003 Probabilistic Model Checking� Markov Chain analysis

Given the description of a Markov Chain, it verifies a PCTL property


– true

– true


– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-a–


� Given the description of a Markov Chain, it verifies a PCTL property


– true

– true


– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-b–



� PCTL: Probabilistic CTL

– � � � �� true � �– � � � � true ��


– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-c–




– � � � �� true � �– � � � � true ��

� Very few available probabilistic model checkers

– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-d–




– � � � �� true � �– � � � � true ��


– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-e–




– � � � �� true � �– � � � � true ��


– PRISM

– Two Towers

– FHP-Mur (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-f–




– � � � �� true � �– � � � � true ��


– PRISM

– Two Towers

– FHP-Mur � (new)

Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –2-g–

MEFISTO- 11/2003 PRISM

PRISM Probabilistic Symbolic Model Checker

State-of-the-art probabilistic model checker

Implicit verification algorithm (MTBDD-based)

It allows to verify three types of Markov Chains:

DTMC, with PCTL are the “classic” ones, here we will deal with these

only

MDP, with PCTL non-determinism added

CTMC, with CSL continuous time managed

Three verification modalities:

– totally MTBDD-based (calculating fix points)

– algebraic (on the Markov Chain transition matrix)

– an hybrid modality between the two previous ones




� State-of-the-art probabilistic model checker

Implicit verification algorithm (MTBDD-based)



only











� Implicit verification algorithm (MTBDD-based)



only












� It allows to verify three types of Markov Chains:


only












� It allows to verify three types of Markov Chains:


only



� Three verification modalities:





MEFISTO- 11/2003 FHP-Mur �

� FiniteHorizonProbabilistic-Mur �

Explicit probabilistic model checker

– symbolic and explicit verification are not comparable in non-probabilistic

model checking

– we will show that this holds also for probabilistic model checking

Mur modified in the input language and in the verification algorithm

Specialized in verifying a particular type of PCTL properties

– true Path

– is a boolean function defined on states

– If models an error, we are asking if the error probability is acceptable




� Explicit probabilistic model checker


model checking




– true Path








model checking




– true Path








model checking




– true Path








model checking


� Mur � modified in the input language and in the verification algorithm


– true Path








model checking



� Specialized in verifying a particular type of PCTL properties

– �� true � � � �� Path � � � ��

– � is a boolean function defined on states







model checking



� Specialized in verifying a particular type of PCTL properties

– �� true � � � �� Path � � � ��

– � is a boolean function defined on states

– If � models an error, we are asking if the error probability is acceptable


MEFISTO- 11/2003 FHP-Mur � ’s input language� We added finite precision real numbers and probabilities:

– on the initial states (initial probability distribution)

initial states with probability

has always to hold

– on the rules (they now define a Markov Chain transition function)

successor states of with probability

has always to hold

– on the invariant to be verified

property to be verified: is the probability of the event “an error state

(i.e., not satisfying the invariant) is reachable within a given number of

steps” less than a given ?

i.e., does is a Markov Chain path

hold?

equivalent to the PCTL formula true




! " initial states with probability #$ %'& & & % #(

! ( )* $ # ),+ - has always to hold


successor states of with probability

has always to hold






hold?








! .$ %'& & & % .( successor states of . with probability # $ %'& & & % #(







hold?








! .$ %'& & & % .( successor states of . with probability # $ %'& & & % #(



! property to be verified: is the probability of the event “an error state



! i.e., does � � �� / � �� is a Markov Chain path � �

hold?

! equivalent to the PCTL formula �� true ��


MEFISTO- 11/2003 FHP-Mur � ’s verification algorithm� Let 01 12 1 34 be the probability of true � � � �

Initially,

is incremented whenever a state is reached such that holds

in



� Initially, 01 1 2 1 34 + 5

is incremented whenever a state is reached such that holds

in



� Initially, 01 1 2 1 34 + 5

� 01 1 2 1 34 is incremented whenever a state . is reached such that � holds

in .



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 :



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 :

ErrProb=ErrProb +; <>= 8 ?= 8



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 :

ErrProb=ErrProb +; <>= 8 ? +; <>= 9 ?

= 9= 8



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 :

ErrProb=ErrProb +; <>= 8 ? +; <>= 9 ? +; <>= : ?

= 9= 8 = :



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 :

ErrProb=ErrProb +; <>= 8 ? +; <>= 9 ? +; <>= : ? +; <>= @ ?

= 9= 8 = :

= @



� Initially, 01 1 2 1 34 + 5


in .6 7

6 86 9

6 : ErrProb=ErrProb+; <A ?

ErrProb=ErrProb +; <>= 8 ? +; <>= 9 ? +; <>= : ? +; <>= @ ?

A= 9

= 8 = := @


MEFISTO- 11/2003 FHP-Mur � ’s verification algorithm� 01 1 2 1 34 + 01 12 1 34 B # where # is the probability to reach . from the

initial states

– If there are C paths to . , # is the sum of the probabilities of these C

paths

Already visited states are not to be discarded, since they can be reached

via different paths

It is necessary to compute paths probabilities

– The initial states are reached with a given probability

– If is reached with probability , and goes to with probability , then

is reached with probability

– The additive property for holds for every reachable state

The reachability analysis is stopped after the -th step

States that satisfy are not expanded



initial states


paths

� Already visited states are not to be discarded, since they can be reached

via different paths

It is necessary to compute paths probabilities









initial states


paths


via different paths

� It is necessary to compute paths probabilities









initial states


paths


via different paths










initial states


paths


via different paths



– If . is reached with probability # , and . goes to D with probability E , then

D is reached with probability # E






initial states


paths


via different paths





– The additive property for 01 12 1 34 holds for every reachable state





initial states


paths


via different paths






� The reachability analysis is stopped after the� -th step




initial states


paths


via different paths






� The reachability analysis is stopped after the� -th step

� States that satisfy � are not expandedIgor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –12-g–

MEFISTO- 11/2003 FHP-Mur � ’s verification algorithm

F 8 :F 8 9

F 8 8F 8 7

F GF H

F IF J

F K

F 9F :

F @

F 8

Uniform probability

. L % . M % . N % .$ N are the states in which � holds (error states)



F 8 :F 8 9

F 8 8F 8 7

F GF H

F IF J

F K

F 9F :

F @

F 8

Uniform probability

. L % . M % . N % .$ N are the states in which � holds (error states)

� [ true �� L � ] + $ N$ N B $ N$ N B $ N$ N B $ N$ L B $ N$ L B $ N$ O + NOIgor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –14–


F 8 :

F 9F :

F JF G

F 8 :

F 9F : F :

F 9

F 8 9F 8 8

F 8 7F H

F IF K

F 8

F @ F @ F 8 9F 8 8

F 8 7

F 8

If . is such that � � . � holds then the Markov Chain starting from . is forced

to cycle on .

� [ true �� L � �+ $ N - B $ N - B $ N$ O + NO againIgor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –15–

MEFISTO- 11/2003 FHP-Mur � ’s BFS

...

Queue

s , qs’, q’

rear

front

. : state to be expanded

E : probability of reaching . in P�Q - levels

.SR next state to be expanded

State explosion virtually never occurs: if the queue grows too much, disk

storage is used



...

Queue

s , qs’, q’

rear

front

. . . . .

T UsT V

W UW V

. : state to be expanded

#$ %'& & & % #( : rules whose probability is strictly positive in .

X �& # ) � 5 % - �

( )* $ # ),+ -Y

Z[

Conditions to have a Markov Chain



... ...

Cache Queue

fronts , qs’, q’

rear. . . . .

T UsT V

W UW V \ ]\ V

Cache: limits the number of states enqueued more than once

X �& ^ ) is empty or stores a pair (state, probability)

.`_$ %'& & & % .�_ a , # _$ %'& % # _ a states among the . ) in which � holds (error states)

and their transition probabilities

.cb _$ %'& & & % .cb _(ed a , # b _$ %'& % # b _(ed a “correct” states (all the other ones) and their

transition probabilitiesIgor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –18–


... ...

Cache Queue

fronts , qs’, q’

rear. . . . .

T UsT V

W UW V \ ]\ V

.f_$ %'& & & % .f_(ed a update ErrProb B + ) # _ ) EAt the end of the . expansion X �& � g ) / ^ hi + � .jb _) % E ) �

X �& E )+k

l m

# b _) E if .jb _) was not in the Cache

# b _) E B Cache[ n � . b _) � ].prob otherwise



...

...

...

Cache Queue

rear

s , qs’, q’

. . . . .T UsT V

W V W U \ ]\ V

\ o 8\ o p

front

swap All non-empty cache entries ( ^ qi ) are enqueued

All cache entries will now result empty

.SR next state to be expanded after the enqueue of . b _a

BFS levels as before; each level changing is always preceeded by a swap


MEFISTO- 11/2003 Experimental results

Probabilistic dining philosophers Pnueli-Zuck (PZ) and Lehmann-Rabin

(LR) protocols

PZ is there a positive probability that a philosopher

become hungry

choose the left fork first

LR the same as PZ, but

is there a positive probability that a philosopher puts down the left fork

first

no philosopher will never wait more than a fixed number (N) of actions

made by the other philosopher before making an action himself

Hybrid systems Verification of a turbogas control system, assuming a

probability distribution on the user demand




(LR) protocols


become hungry

choose the left fork first



first








(LR) protocols


� become hungry

� choose the left fork first



first








(LR) protocols


� become hungry



� is there a positive probability that a philosopher puts down the left fork

first

� no philosopher will never wait more than a fixed number (N) of actions







(LR) protocols


� become hungry



� is there a positive probability that a philosopher puts down the left fork

first

� no philosopher will never wait more than a fixed number (N) of actions





MEFISTO- 11/2003 Experimental results: PZ protocol

NPHIL MAX WAIT Probability Mur � Memory (MB) PRISM Memory (MB) Mur � Time PRISM Time

3 3 7.335194164e-05 200 0.9057 51.970 s 1.487 s

3 4 6.883132778e-10 200 1.6844 52.610 s 2.507 s

4 3 1.88985976e-06 200 28.1066 4 min 28.72 s

4 4 2.910383046e-12 200 66.2659 4 min 1 min

5 3 9.164495139e-08 200 916.8246 23 min 17 min

5 4 4.194304e-14 200 N/A 23 min N/A

8 3 1.210429649e-10 1000 N/A 2 89 days N/A

�� $r s [true �� L s a philosopher has waited for MAX WAIT transitions]

Results on a 2-processors (both INTEL Pentium III 500Mhz) computer with

2GB of RAM

NPHIL: number of philosophers

MAX WAIT: max waiting time for every philosopher


MEFISTO- 11/2003 Experimental results: an explanation

Implicit vs Explicit sometimes the former performs better than the latter,

sometimes not

Probabilistic verification We showed that this holds for probabilistic

verification

Termination is not all, also time is important

PRISM, if terminates, terminates faster than FHP-Mur

FHP-Mur virtually always terminates (thanks to the disk storage of the

queue), but it could require too much time

– if the horizon is too much long, the verification will take a great amount

of time

– PRISM execution time is not dependent from the horizon




sometimes not


verification






of time





sometimes not


verification






of time





sometimes not


verification






of time





sometimes not


verification


� PRISM, if terminates, terminates faster than FHP-Mur �




of time





sometimes not


verification



� FHP-Mur � virtually always terminates (thanks to the disk storage of the



of time





sometimes not


verification






of time





sometimes not


verification






of time


Igor Melatti, Finite Horizon Analysis of Markov Chains with the Mur � Verifier –23-g–

MEFISTO- 11/2003 PRISM or FHP-Mur � ?

Not comparable There are cases in which Mur is better, other in which

PRISM is

PCTL formulas Only of a certain type in FHP-Mur

FHP-Mur better when

the transition function is based on (complex) mathematical operations

the horizon is not too long

PRISM better in the other cases

FHP-Mur is however a probabilistic model checker to be taken into account



Not comparable There are cases in which Mur � is better, other in which

PRISM is

PCTL formulas Only of a certain type in FHP-Mur

FHP-Mur better when








PRISM is

PCTL formulas Only of a certain type in FHP-Mur �

FHP-Mur better when








PRISM is


FHP-Mur � better when

� the transition function is based on (complex) mathematical operations

� the horizon is not too long






PRISM is










PRISM is






FHP-Mur � is however a probabilistic model checker to be taken into account


MEFISTO- 11/2003 Future works

More features for FHP-Mur � and then comparison with PRISM

Handling of PCTL formulas like true

Infinite horizon

– Some precomputations will be necessary in these two cases

Continuous Markov Chains

– Approximable to Discrete Time Markov Chain with an exponential

distribution

– The smaller the sampling step

the lowest the approximation error

the higher the execution time




� Handling of PCTL formulas like � � � tu � true ��

Infinite horizon




distribution








� Infinite horizon




distribution








� Infinite horizon


� Continuous Markov Chains


distribution


! the lowest the approximation error

! the higher the execution time


MEFISTO- 11/2003 Publicationsv G. Della Penna, B. Intrigila, I. Melatti, E. Tronci, and M. V. Zilli Finite Horizon Verification of

Markov Chains with the Mur w Verifier, CHARME, L’Aquila, 2003

v G. Della Penna, B. Intrigila, I. Melatti, E. Tronci, and M. V. Zilli Integrating RAM and Disk

based Verification within the Mur w Verifier, CHARME, L’Aquila, 2003

v G. Della Penna, B. Intrigila, I. Melatti, E. Tronci, and M. V. Zilli Finite Horizon Verification of

Stochastic Process with the Mur w Verifier, ICTCS, Bertinoro (FC), 2003

v G. Della Penna, B. Intrigila, I. Melatti, M. Minichino, E. Ciancamerla, A. Parisse, E. Tronci,

and M. V. Zilli Automatic Verification of a Turbogas Control System with the Mur w Verifier,

HSCC, Prague, 2003

v G. Della Penna, B. Intrigila, E. Tronci, and M. Venturini Zilli Exploiting Transition Locality in

the Disk based Mur w Verifier, FMCAD, Portland 2002

v E. Tronci, G. Della Penna, B. Intrigila, and M. Venturini Zilli Exploiting Transition Locality in

Automatic Verification, CHARME, Edinburgh 2001

v http://www.dsi.uniroma1.it/ x tronci/cached.murphi.html

v http://vv.cs.byu.edu/mug (Mur w users group)


Date post:	22-Aug-2019
Category:	Documents
Upload:	phambao
View:	213 times
Download:	0 times

Verier Finite Horizon Analysis of Markov Chains with the Murmelatti/relazioni/MEFISTO2003.pdf ·...

Documents