Formal Testing withInput-Output

Transition Systems

Ed BrinksmaCourse 2004

© Ed Brinksma/Jan Tretmans

Formal Testing

exec : TESTS IMPS (OBS)

der : SPECS (TESTS)

Ts TESTS

s SPECS

IUT IMPS

imp

iIUT MODS

obs : TESTS MODS (OBS)

t: (OBS) {fail,pas

s}

OBS

pass

fail

Proof soundness and exhaustivess:iMODS . ( tder(s) . t(obs(t,i)) = pass ) i imp s

Test hypothesis :IUTIMPS . iIUT MODS . tTESTS . exec(t,IUT) = obs(t,iIUT)

© Ed Brinksma/Jan Tretmans

Input-Output Transition Systems

dub

coffee

kwart

tea

S1 S2

S3 S4

S0

LI = { ?dub, ?kwart }

LU = { !coffee, !tea }

dub, kwart coffee, tea

from user to machine from machine to userinitiative with user initiative with machinemachine cannot refuse user cannot refuse

input outputLI LU

LI LU = LI LU = L

!

??

!

© Ed Brinksma/Jan Tretmans

Input-Output Transition Systems

LI = { ?dub, ?kwart }

LU = { !coffee, !tea }

Input-Output Transition Systems IOTS (LI ,,LU ) LTS (LI , LU )

IOTS is LTS with Input-Outputand always enabled inputs:

for all states s,for all inputs ?a LI :

?dub?kwart

?dub?kwart

?dub?kwart

?dub?kwart

?dub

!coffee

?kwart

!tea

S ?a

© Ed Brinksma/Jan Tretmans

?kwart

?dub?kwart

?kwart

?dub?kwart

?dub?kwart

?kwart?dub

?dub?kwart

?kwart?dub

?kwart

?kwart?dub

?dub?kwart

?dub?kwart

Input-Output Transition Systems

!coffee

?dub

!tea !coffee

?dub

?dub

!coffee

?dub

!tea

© Ed Brinksma/Jan Tretmans

Labelled Transition System Testing

SPECS LTS ( LI LU ) LTS MODS IOTS (LI , LU ) LTS TESTS TTS ( LU , LI ) LTS

OBS traces obs t || i der der : LTS ( LTS )

Which imp ? (strong, weak, branching, ... ) bisimulation trace-, testing-, refusal - preorder / equivalence conf, conf*, aconf, ioconf, ioco, miocoF

ioco

© Ed Brinksma/Jan Tretmans

Formal Correctness

testing equivalences

refusal testing

canonical tester

Input Output Automataquiescence

ioco

© Ed Brinksma/Jan Tretmans

Preorders on Transition Systems

implementationi

specifications

environmente

environmente

? ? ?

i s e E . obs ( e, i ) obs (e, s )

i LTS s LTS

© Ed Brinksma/Jan Tretmans

Preorders onInput-Output Transition Systems

implementationi

specifications

environmente

environmente

imp

i IOTS(LI,LU) s LTS(LILU)

imp IOTS (LI,LU) x LTS (LILU)Observing IOTS where system inputsinteract with environment outputs, and v.v.

© Ed Brinksma/Jan Tretmans

Preorders onInput-Output Transition System

implementationi

specifications

environmente

environmente

IOTS(LU,LI)

imp

i imp s e E . obs ( e, i ) obs (e, s )

i IOTS(LI,LU) s LTS(LILU)

© Ed Brinksma/Jan Tretmans

Input-Output Testing Relation

implementationi

specifications

environmente

environmente

obs ( e, p ) = ( traces (e||i ), Ctraces (e||i ) )

iot

i iot s e IOTS(LU,LI) . obs ( e, i ) obs (e, s )

i IOTS(LI,LU) s LTS(LILU)

© Ed Brinksma/Jan Tretmans

Input-Output Refusal Relation

implementationi

specifications

environmente

environmente

obs ( e, p ) = ( traces (e||i ), Ctraces (e||p) )

ior

i ior s e IOTS(LU,LI {} ) . obs ( e, i ) obs (e, s )

i IOTS(LI,LU) s LTS(LILU)

© Ed Brinksma/Jan Tretmans

Input-Output Testing Relation

i,s LTS : i te s e LTS . obs ( e, i ) obs (e, s )

inputs can never be refused by ioutputs can never be refused by e :

FP ( i ) FP ( s )

FP ( p ) = { , A | A L, traces(p), p afer refuses A }

i IOTS(LI,LU) : i iot s e IOTS(LU,LI) . obs ( e, i ) obs (e, s )

i afer refuses A A = or A = LU

© Ed Brinksma/Jan Tretmans

Input-Output Testing Relation

FP ( i ) FP ( s )

i IOTS(LI,LU) : i iot s e IOTS(LU,LI) . obs ( e, i ) obs (e, s )

{ | traces(i), i afer refuses } { | traces(s), s afer refuses } and { | traces(i), i afer refuses LU } { | traces(s), s afer refuses LU } traces(i) traces(s) and Qtraces(i) Qtraces(s)

Qtraces : Quiescent traces = traces ending in quiescence i i = i i = !x LU {} : i !x LU

© Ed Brinksma/Jan Tretmans

Input-Output Refusal Relation

Ftraces( i ) Ftraces ( s )

i IOTS(LI,LU) : i ior s e IOTS(LU,LI {}) . obs ( e, i ) obs (e, s )

( L ( L ) )* : i Failure trace :Failure traces of i : Ftraces ( i ) = { ( L ( L ) )* | i

}

Failure A : A { } : i Ai i where:

inputs can never be refused by ioutputs can never be refused by e :

i afer refuses A A = or A = LU

© Ed Brinksma/Jan Tretmans

Input-Output Refusal Relation

Ftraces( i ) Ftraces ( s )

i IOTS(LI,LU) : i ior s e IOTS(LU,LI {}) . obs ( e, i ) obs (e, s )

Straces : Suspension traces

= Failure traces restricted to refusals quiescence LU =

Straces( i ) Straces ( s )

Straces ( i ) = Ftraces ( i ) ( L { LU } )*

= { ( L { } )* | i }

© Ed Brinksma/Jan Tretmans

Input-Output Refusal Relation

i IOTS(LI,LU) : i ior s e IOTS(LU,LI {}) . obs ( e, i ) obs (e, s )

Straces( i ) Straces ( s )

( L { } )*: out ( i after ) out ( s after )

where:

out ( I ) = { !x LU | i !x , iI } { | i i, i S }

!xout ( i after ) = { !x LU { } | i }

© Ed Brinksma/Jan Tretmans

Implementation Relation ioco

i IOTS(LI,LU) : i ior s ( L { } )*: out ( i after ) out ( s after )

To allow under-specification :

i ioco s Straces( s ) : out ( i after ) out ( s after )

© Ed Brinksma/Jan Tretmans

i ioco s =def Straces (s) : out (i after ) out (s after )

Implementation Relation ioco

Correctness expressed by implementation relation ioco:

Intuition:i ioco-conforms to s, iff• if i produces output x after trace , then s can produce x after • if i cannot produce any output after trace , then s cannot produce any output after ( quiescence )

© Ed Brinksma/Jan Tretmans

i ioco s =def Straces (s) : out (i after ) out (s after )

Implementation Relation ioco

p p = p p = !x LU {} : p

!x LU

Straces (s) = Ftraces (s) ( L { LU } )*

= { ( L { } )* | s }

p after = { p’ | p p’ }

out ( P ) = { !x LU | p , p P }

{ | p p, pP }

!x

© Ed Brinksma/Jan Tretmans

Implementation Relation iocoout ( i after )

=out ( i after ?dub ) =out ( i after ?dub.?dub )

=out ( i after ?dub.!coffee) =out ( i after ?kwart ) =out ( i after !coffee ) =out ( i after ?dub.!tea )

=out ( i after ) =

!coffee

?dub

?dub?kwart

?dub?kwart

i ?kwart

{ }{ !coffee }{ !coffee }{ }

{ }

{ }

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

!coffee

?dub

?dub

?dub

i

!coffee

?dubs

out (i after ) = { }out (i after ?dub) = { !coffee }out (i after ?dub.!coffee) = { }

out (s after ) = { }out (s after ?dub) = { !coffee }out (s after ?dub.!coffee) = { }

ioco

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

!coffee

?dub

?dub

?dub

i

!coffee

?dubs

!tea

out (i after ?dub) = { !coffee } out (s after ?dub) = { !coffee, !tea }

ioco

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

!coffee

?dubs

?dub

?dub

!coffee

?dubi

!tea

?dub

out (i after ?dub) = { !coffee, !tea } out (s after ?dub) = { !coffee}

ioco

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

out (i after ?dub) = { !coffee, !tea } out (s after ?dub) = { !coffee, !tea}

ioco?dub

?dub

?dub

!coffee

?dubi

!tea

?dub

?dub

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

!coffee

?dubs

!tea

© Ed Brinksma/Jan Tretmans

ioco

?dub

?dub?kwart

!coffee

?kwarti

!tea !coffee

?dubs

out (i after ?dub) = { !coffee }out (i after ?kwart) = { !tea }

out (s after ?dub) = { !coffee }out (s after ?kwart) =

But ?kwart Straces ( s )

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

?dub

?dub?kwart

!coffee

?kwarti

!tea

ioco

s?dub

!coffee

?kwart

!tea

out (i after ?dub) = { !coffee }out (i after ?kwart) = { !tea }

out (s after ?dub) = { !coffee }out (s after ?kwart) = { !tea }

© Ed Brinksma/Jan Tretmans

!coffee

?dub

i ?kwart

?dub?kwart

?dub?kwart

out (s after ?kwart) = { !tea }out (i after ?kwart) = { }

ioco

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

s?dub

!coffee

?kwart

!tea

© Ed Brinksma/Jan Tretmans

out (s after ?dub) = { !coffee }out (i after ?dub) = { , !coffee }

ioco

?dub

?dub

?dub

!coffee

?dubi

?dub

!coffee

?dubs

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

© Ed Brinksma/Jan Tretmans

out (s after ?dub) = { , !coffee }out (i after ?dub) = { , !coffee }

ioco

!coffee

?dub

s

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after )

?dub

?dub

?dub

!coffee

?dubi

?dub

© Ed Brinksma/Jan Tretmans

out (i after ?dub.?dub) = out (s after ?dub.?dub) = { !tea, !coffee }

i ioco s

Implementation Relation ioco

i ioco s =def Straces (s) : out (i after ) out (s after ) i

?dub

?dub

?dub ?dub

!tea

?dub

?dub

!coffee

?dub

s

!coffee

?dub

?dub

?dub ?dub

!tea

?dub

?dub

?dub

?dub

!tea

s ioco i

out (i after ?dub.?dub) = { !coffee } out (s after ?dub..?dub) = { !tea, !coffee }

© Ed Brinksma/Jan Tretmans

?kwart

!coffee

?dub

!tea

!coffee

?dub?kwart

?dub?kwart

?dub?kwart !coffee

?dub

!tea

ioco

ioco

ioco

Implementation Relation ioco

ioco

ioco

ioco

© Ed Brinksma/Jan Tretmans

i ioco s =def Straces (s) : out (i after ) out (s after )

? x (x >= 0)! x

? x (x < 0)

? y

implementation i

! -x? x (x >= 0)! x

? x (x < 0)

? y

specification s

i ioco ss ioco i

Implementation Relation ioco

equation solver for y2 =x :

© Ed Brinksma/Jan Tretmans

Genealogy of ioco Labelled Transition Systems

IOTS (IOA, IOSM, IOLTS)

Testing Equivalences(Preorders)

Refusal Equivalence(Preorder)

Canonical Testerconf

Quiescent Trace Preorder

Repetitive QuiescentTrace Preorder

(Suspension Preorder)

ioco

ioconf

© Ed Brinksma/Jan Tretmans

Formal Testing with Transition Systems

t: (traces){fail,pass}

exec : TESTS IMPS (OBS)

traces

der : LTS (TTS)

Ts TTS

s LTS

IUT IMPS

ioco

iIUT IOTS pass

fail

obs : TTS IOTS (traces)

Soundness and exhaustivess proved:iIOTS . ( tder(s) . t(obs(t,i)) = pass ) i ioco s

Test hypothesis :IUTIMPS . iIUT IOTS . tTTS . exec(t,IUT) = obs(t,iIUT)