Concur 2001 August 21, 2001

Performance Evaluation := (Process Algebra + Model

Checking)x Markov Chains

Holger Hermanns and Joost-Pieter Katoen

with contributions ofChristel Baier, Ed Brinksma, Boudewijn Haverkort, Ulrich Herzog, Joachim Meyer-Kayser, Markus Siegle

22

A reactive, embedded system:The ‘Hubble Space Telescope’A reactive, embedded system:The ‘Hubble Space Telescope’

and its stabilising

unit

33

s

r

56 4 23 1 crash

f f f f f f

sleep sleep ff

r

s

A simple model of the Hubble

The base station prepares a shuttle mission to repair the telescope (r).

Each gyroscope may fail (f).

The telescope turns into sleep mode if less than 3 gyroscopes remain operational (s).

Without operational gyro the telescope eventually crashes.

44

What is this? What is it good for?

A model

A stochastic model

A continuous-time Markov model

Prediction of the system behaviour

Computer-assisted analysis of

CorrectnessPerformanceDependability

on the basis of a model, instead of the real system

s

r

56 4 23 1 crash

sleep sleepf

fr

s

f f f f f f

55

Quantitative Verification

Information technology is finally reaching a scale where

probabilistic methodsprobabilistic methods should play a larger role in system design.

D. Tennenhouse, director research Intel Corp.

Proactive Computing, Communications of the ACM, May 2000

66

Why probabilities?practically relevant for

deterministically unsolvable problems:randomised distributed algorithms.

unreliable and unpredictable system behaviour:fault tolerant systems, ...

performance and dependability analysis:‘quality of service’, ...

wheighting important (likely/frequent) and unimportant (unlikely/rare) aspects in the specification.

approximating large ‘populations’ of discrete structures

77

s

r

56 4 23 1 crash

6 f 5 f 4 f 3 f 2 f f

sleep sleep2 ff

r

s

A Markov model of the Hubble

The base station prepares a shuttle mission to repair the telescope (r).

Each gyroscope posesses a failure rate f.

To turn on sleep mode requires some time (s).

Without operational gyroscope the telescope eventually crashes.

88

Specification formalisms for CTMCs

stochastic Petri nets [Molloy]

Markovian queueing networks [Muppala & Trivedi]

stochastic automata networks [Plateau]

stochastic process algebra [Herzog et al]

probabilistic I/O automata [Stark et al]

and many variants/combinations thereof.

99

Continuous-time Markov chains (CTMCs)

(finite state) automata,

all times are exponentially distributed,

sojourn time in states are memory-less,

very well investigated class of stochastic processes,

widely used in practice,

best guess, if only mean values are known,

efficient and numerically stable algorithms for stationary and transient analysis are available.

00.10.2

0.30.40.50.60.7

0.80.9

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

h

PrPr(X (X >>t) = t) = ee--hhtt

1010

1

1

2

33

Transient and Stationary Behaviour of CTMCs

transient probability

stationary (‘steady state’) probability

)0( s, )10( s, )20( s, )30( s, )40( s, )60( s, )70( s, )80( s, )90( s, )100( s, )110( s, )120( s, )130( s, )140( s, )150( s, )160( s, )170( s, )180( s, )190( s, )200( s, )210( s, )220( s,

s

)( s,

1111

1212

Model Checking CTMCs

Continuous Stochastic Logic

Fixpoint Characterisations

Model Checking Algorithms

Extensions and Applications

1313

Model Checking

Automated verification technique

Checks whether a given finite-state model satisfies a given requirement, by

systematic state-space explorationeffective means to combat the state-space explosion

Some model checkers: Spin, SMV, Mur, Uppaal

Application areas:hardware verification (VHDL-code, ...)software validation (storm surge barrier, ...)software bug hunting (web server design, e-commerce, ...)

1414

CTL - Computation Tree Logic

state-formula :

true

a atomic proposition’

1 2 ‘and’

‘not’

‘for All paths’

‘there Exists a path’

path-formula :

X ‘neXt’

1 U 2 ‘Until’

‘eventually’

‘invariantly’

a branching-timetemporal logic

powerful specification language for requirements

widely used

true U =

=

[Clarke & Emerson 83]

1515Sat(6) Sat(6) Sat(sleep)

Model checking CTL by example

Given: a finite-state model and a CTL state-formula :

Strategy: calculate recursively the sets for all sub-formulas of

| sSsSat

= ( 6 U sleep)

56 4 23 1 crash

sleep sleep

initialisation first iterationsecond iterationthird iterationfourth iteration

Sat()

s satisfies

fifth iteration

fixed point!

1616

Basic idea specify a desired performance/reliability property using appropriate extension oftemporal logic, e.g.,

P<0.01(<10 error) , S<10-6(error) ,

or similar

probability that an error occurs within 10 years is less than 1 %probability that an error occurs in equilibrium is less than 10-6.

interpret and check these formulas on CTMCs

1717

state-formula :

true

a atomic proposition

1 2 and

not

for all paths

there is a path

CSL - Continuous Stochastic Logic

path-formula :

X neXt

1 U 2 Until

CTL plus probabilistic path-quantifier [Hansson and Jonsson]

probabilistic ‘time-bounded until’ [Aziz et al]

stationary probability quantifier

[Baier et al]

0

,

1,0

I

p

state-formula :

true

a atomic proposition

1 2 and

not

S~p() stationary probability

P~p() path probability

path-formula :

XI timed neXt

1 UI 2 timed Until

1818

A few requirements for the Hubble

availability? S>p( (sleep crash))

gyroscope failure between 1993 and 1997? P>q([3,7] 6)

sleep mode between 1997 and September 1999?

Pr( sleep U[7,9.8]sleep)

risk of a crash before 2010? P<10-2([0,20] crash)

56 4 23 1 crash

0.6 0.5 0.4 0.3 0.2 0.1

sleep sleep0.20.1

6

6

100 100

1990

1919

State formulas:

s a iff a L(s) s 1 2 iff s i , i=1,2

s iff s /

state in at time t

probability that “on the long run” the system is in a -state (when starting in s)

requires -algebra

and probability measure

Prob on paths of CTMC

ptst

~ @| lim

PathsProb s S~p() iff

ps ~ | PathsProb s P~p() iff

Formal semantics of CSL (1)

2020

Path formulas:interpretation over the paths (from state ) in a CTMC

state wins the race after time units, and so on

33

22

110

ts

ts

ts

ts o

0 s

01 ts

kk

k

ttytt

sy

010

with@ where

2

1

@

@ . ,0

.

x

yxy

Ix

1 UI 2 iff

Formal semantics of CSL (2)

XI iff s1 and It 0

2121

Model Checking CTMCs

Continuous Stochastic Logic

Fixpoint Characterisations

Model Checking Algorithms

Extensions and Applications

2222

For the non-probabilistic fragment: as for CTL

Model checking CSL Given: a CTMC and a CSL state-formula :

Strategy: recursively compute the sets for all sub-formulas of

| sSsSat

2323

Model checking CSL Given: a CTMC and a CSL state-formula :

Strategy: recursively compute the sets for all sub-formulas of

Steady-state operator requires slight adaptations of standard methods for steady-state probabilities

S~p() ps,s'ss

~ '

iff

steady state probability for s’ in the BSCC Bsystem of linear equations

graph algorithm

system of

linear equations

matrix-vector multiplication

Bs sBs

s

tstsss

B ' ' ,Pr

' 0

'@|',

BSCC

BSCC

PathsProb

if

if

for

where

| sSsSat

2424

BSCC B1

BSCC B2

{stable}{unstable}

{initial}{stable} 1

1

2

33

S 0.5 (P 0.98 ( 1.5 stable) )

s

5.03

15.015.0

5.0 ,Pr ,Pr 21 BsBs

3

1

1

2

1

B

B

An example

2525

Model checking CSL Given: a CTMC and a CSL state-formula :

Strategy: recursively compute the sets for all subformulas of

P~p()

,Pr ps s iff

Probabilistic state-formula with ‘neXt step’ X and ‘until’ U are treated as in the discrete-time case [Hansson & Jonsson]

vector U is the least fixed point in [0,1] of

if s 2 then

if s / 1 2 then

if s 1 2 then

ss F ,Pr 21 s's,s's

s

s

s'

FF

0F

1F

P

'

,Prs

s,s's P X matrix-vector multiplication

system of linear equations

iterative solution

| sSsSat

2626

dxxts't

es,s'ts

ts

ts

s'

xs ,F ,F

0,F

1,F

0

EQ

tss ,F,Pr 21 values Ut are the least solution in [0,1] of

if s 2 then

if s / 1 2 then

if s 1 2 then

Model checking ‘time-bounded until’

21 ,Pr s' U t-x

s’s

1 2 2

t

t0 x

t-x

system of integral equations

probability to move from s to s’ at time x

2727

Model Checking CTMCs

Continuous Stochastic Logic

Fixpoint Characterisations

Model Checking Algorithms

Extensions and Applications

2828

Model checking ‘time bounded until’ Pr(s, 1 UI 2) via transient

analysis

transient analysis determines a snapshot of the state probabilities at time t (if starting in state s at time 0)

state-of-the-art: uniformisation

numerically stable

(relatively) easy to implement: boils down to iterative matrix-vector multiplications

a priori calculation of number of iterations based on user-given accuracy

on-the-fly steady-state detection possible

)( s,t

2929

calculating transient probabilities:

Transient analysis of CTMCstransient probability distribution (s,t ):

the (snapshot)

probability at time t when starting

in state s at time 0

'@|)( ' stss,ts PathsProbin CSL expressed as:

P~p([t,t] ats’ )

and

S~p(ats’)

),(lim)( '' tss st

s

steady-state probability (s):

EQQ Diagˆ i.e.

CTMC, of matrix generator ˆ),()( Q tss

dt

d

Chapman-Kolmogorov equation

3030

Transient analysis of CTMCs

to rise gives ˆ),()( Q tssdt

d

Techniques: Runge-Kutta and (more efficient and accurate):

Uniformisation (“Jensen’s Method”)

Basic idea of uniformisation:

transform CTMC into a corresponding DTMC,

normalise transition rates w.r.t. shortest (average) residence time

!

)ˆ(

0

i

i

i

tQas compute

otherwise 0 and

i.e. ies,probabilit initial

,(s,t)πs 1

)0()( ˆ tes,s,t Q

ˆ~

*

QIP

ˆ iii* qmaxwith

3131

ˆ

~

*

QIP

Uniformisation

different outgoing rates per stateno self-loops*= +

same outgoing “rate” * per state branching probabilities self-loops (mimic delays)

10

2

CTMC*,ˆ Q

DTMC

P~

/ ( +)

/ ( +)

/ ( +)

/ ( +)

0 1 2 +

+

3232

(given stepping rate *)

Uniformisation

0 Pr)( s,n,tns,t in steps probability distribution

in DTMC after n steps,

starting from state s

P

~,1,

)0,,0,1,0,0(0,

nsπnsπ

sπ

matrix-vector

multiplication

Round-off error can be calculated a priori:

probability of n arrivals in [0,t]in a Poisson process with rate *

!

**

n

nte

compute

recursively

(Fox-Glynn)

k

n

t

n

ntetss,t

0

*

!1,

*crequired

accurac

y

number of steps in

DTMC

exact compute

d

3333

Reduction to transient analysis

Aim: Compute Pr(s, 1 UI 2) via (...,... )

1 2

1 2

1 2

s

1 2

3434

1 2

1 2

1 2

s

1 2

s’ (s,t)2 's

1 2

1 2

1 2

s

1 2

s’ (s,t)2 's

Lemma A

Pr(s, 1 U[0,t0,t] 2) =

1 2

1 2

1 2

s

1 2

Assume all 2-states are absorbing

3535

Pr(s, 1 U[0,t0,t] 2)

1 2

1 2

1 2

s

1 2

Pr(s, 1 U[0,t0,t] 2)

1 2

1 2

1 2

s

1 2

Pr(s, 1 U[0,t0,t] 2)

1 2

1 2

1 2

s

1 2

Theorem 1

Pr(s, 1 U[0,t0,t] 2) =

then apply Lemma A

1 2

1 2

1 2

s

1 2

= s’ (s,t )2 's

3636

Model checking CSL

‘Bottom-up’ strategy along the property of interest,

recursively collects states satisfying sub-formulae

Ingredients:

graph algorithms, and matrix-vector multiplication

solvers for linear equation systems

model transformations and uniformisation

Worst-case time complexity:

O(|formula| x (M.q.tmax + N2.81))

number of transitions Muniformisation rate qmaximal time-bound tmax

number of states N

3737

Lumping

Two CTMCsCTMCs are lumping equivalentlumping equivalent, if they can mimic their

cumulated ratescumulated rates stepwise, and stay bisimilar in doing so

22

if then ,

and vice versa, and so on

such that = ,

Lumping ensures that cumulated (transient/steady)-state probabilities of

equivalent states can be computed on the quotient CTMC

3838

Lumping and CSL

Two states in a CTMC are lumping equivalentlumping equivalent

if and only if

they satisfy the same CSL-formulas

(... if the bisimulation respects the state labelling)

3939

Model Checking CTMCs

Continuous Stochastic Logic

Fixpoint Characterisations

Model Checking Algorithms

Extensions and Applications

4040

The model checker

implemented in JAVA (version 1.2 with Swing)

about 8,000 lines of code, 15 man months

implements iterative numerical algorithms to solvelinear system of equations (standard)

uses backwards uniformisation for UI

uses dedicated algorithms for P=1() and P=0()

uses sparse data structures for matrices

www7.informatik.uni-erlangen.de/etmcc/TE MC2

4141

The model checker TE MC2

GUIGUI

VerificationparametersVerificationparameters

ModelinputModelinput

ResultoutputResultoutput

PropertyManagerPropertyManager

Tool DriverTool Driver CSL parserCSL parser

S~p() P~p() S~p() P~p()

State Space ManagerState Space Manager

SatSat States States TransitionsTransitions RatesRates

Analysis Engine

( 1 U 2) ( 1 U 2)

BSCC

Analysis Engine

( 1 U 2) ( 1 U 2)

BSCC

FilterFilter

Numerical Engine

Linear systems of equationsNumerical integration

Backwards uniformisation

Numerical Engine

Linear systems of equationsNumerical integration

Backwards uniformisation

4242

Current developments

Application/case studies:performance assessment of cyclic polling systemdependability analysis of a workstation clusterperformance and availability analysis of distributed database server

Extensions towards CTMCs with costs (rewards): “with probability at most 0.01 at most 10 jobs have been processed before the first error occurs”

extension of CSL has been definedmodel checking combined reward- and time-bounded formulas?

Using symbolic data structures (MTBDDs) in Prism

Extension of model checking algorithms for Markov decision processes

4343

Summary

CTMC algebra:

compositional and abstract specificationautomated generation of CTMCsreduction and comparison of performance models

CTMC model checking:

specification language for performance propertiesautomated verification technique with property-driven transformationallows model reduction cross-fertilisation of formal

specification and performance modeling techniques

cross-fertilisation of formalverification and performance

analysis techniques