CSE 555Protocol Engineering

Dr. Mohammed H. SqalliComputer Engineering Department

King Fahd University of Petroleum & Minerals

Credits: Dr. Abdul Waheed (KFUPM)

Spring 2004 (Term 032)

Correctness Requirements(Cont.)

Term 032 4-3-3 CSE555-Sqalli

Validation Case Studies

Alternating bit protocol

Flight software

Term 032 4-3-4 CSE555-Sqalli

Specification of a Process Instantiation

A process of type A is explicitly specified as:A[N]

where N is the PID of the process Processes are assigned PIDs in the order in which they are

instantiates The init process has a PID of 0

If process of type A is to be specified that it is in state labeled as P0:A[N] : P0

A reference to the current value of local variable any is:A[N].any

For example: assert (A[N].any < 0)

Term 032 4-3-5 CSE555-Sqalli

Alternating Bit Protocol Revisited

Message loss is possible Retransmit in that case

Term 032 4-3-6 CSE555-Sqalli

Example: Alternating Bit Protocol

Processes Sender and Receiver communicate via message channels named sender and receiver

Each channel can hold one message of type byte Message loss is modeled explicitly in sender and receiver processes

Receiver process may steal messagesbefore Sender process can receive

Term 032 4-3-7 CSE555-Sqalli

Example (Cont’d) Possible correctness claim:

It is always true that when the sender transmits a message, the receiver will eventually accept it

To express it as a temporal claim, we need to find the undesirable property:

We can use following 4-state claim:never {

do:: skip; /* allow any time delay */:: receiver?[msg0] -> goto accept0:: receiver?[msg1] -> goto accept1od;

accept0:do:: !Receiver[2] : P0od;

accept1:do:: !Receiver[2] : P1od

}

Term 032 4-3-8 CSE555-Sqalli

Example (Cont’d)

The 4 state in this claim: The initial state The two states with labels accept0 and accept1 The normal end state

Initial state At least one of three conditions should be true Claim remains in this state as long as receiver is empty Moves to accept0 or accept1 depending on message

received Second and third states

Claim remains in these states as long as Receiver process never accepts a message with the same sequence number

Term 032 4-3-9 CSE555-Sqalli

Case Study

Using SPIN model checking for flight software verification

From paper by Peter R. Glück and Gerard J. Holzmann

Application of model based checking to the verification of a mission-critical flight software

Uses SPIN for verification A practical example of techniques we are studying

We consider following issues: Problem background and motivation Approach Main results

Term 032 4-3-10 CSE555-Sqalli

Software Application Flight software is a concurrent system

Multiple threads of execution active at the same time Interleaved execution due to CPU sharing

Interleaving depends on thread scheduler May be different for different executions

Manually testing a multi-threaded system is difficult Limited controllability

Tester cannot control the specifics of thread interleaving Very large state space

Limited observability Error can be detected But reconstructing sequence of events that lead to error is hard!

Model checking techniques are applicable for such cases

Term 032 4-3-11 CSE555-Sqalli

Suitability of Model Checking Techniques Model provides complete control

All salient aspects of system execution Other aspects can be abstracted out

Model allows a thorough analysis Not possible otherwise

What model checking can do? Earliest techniques provided reachability analysis

A systematic exploration of the reachable states of the graph Reachability analysis extended by two developments:

Dedicated logics to allow reasoning about correctness of systems that evolve over time (e.g., Linear Temporal Logic)

Mechanical translation of temporal logic formulae to test drivers that can be used inside model checking tools for automated correctness proofs

Verification systems can be based on source code directly

Term 032 4-3-12 CSE555-Sqalli

How Does Spin Work

SPIN requires the definition of “closed” systems All potential input sources must be defined as part of the

system Conservative approximations can be made “Test drivers” to model the impact of environment

SPIN works with global system state Begins by computing a state vector to define an initial

state Correctness properties are defined for the system

Model checker objective: Compute minimal number of reachable states to prove or

disprove the correctness property Supports both safety and liveness properties

Term 032 4-3-13 CSE555-Sqalli

Model Extraction Traditional approach

Construct an accurate model formulate correctness properties check that properties hold for the model

Problem I: property itself is inaccurate Problem II: model is inaccurate (due to incomplete

understanding or incorrect model construction) Problem III: model is accurate only for a version of the

application Problem IV: it may take days or weeks to construct an

accurate model Model may never be up-to-date Results may be discarded by

the developers A more recent approach used by SPIN and other tools

Mechanically extract model from application source code SPIN model extractor is called FEAVER

Term 032 4-3-14 CSE555-Sqalli

Model Extractor: FEAVER It works like a compiler front-end:

A parser constructs a parse tree from source code (C language)

Parse tree is converted into standard control-flow graph One control-flow graph for each procedure

Each control flow graph is cast in the modeling language of SPIN

Statements and expressions remain in native C language An extension of SPIN allows using embedded C code in models Data can also be encapsulated similarly as embedded data

Model is executed by SPIN SPIN retains complete control over thread interleaving It keeps track of system state through a state vector

Model extraction process is instantaneous Process can be repeated as the system changes

Term 032 4-3-15 CSE555-Sqalli

Model Extractor: Test Harness Accuracy and level of detail can be adjusted with a test

harness

Test harness: It is a small definitions file Model extractor uses this file to decide which portions of the

source code are to be converted to model fragments Also defines how the model fragments should be joined

Tester can exclude large portions of applications from model checking process

Replaced by simple stubs Useful when test with model checker is run on a platform where

all components of actual application system are not available

Term 032 4-3-16 CSE555-Sqalli

Goal of This Study

Apply model checking to legacy flight software from NASA’s Deep Space One (DS1) mission

Implemented in C Contained some known defects at launch

Goal: To reproduce a known defect mechanically with model

extractor and model checker

Term 032 4-3-17 CSE555-Sqalli

DS1 Downlink Packet Handling Module

There were 16 C source files and 18 header files A total of 5166 lines of C code

Two separate, coordinated tasks

Term 032 4-3-18 CSE555-Sqalli

The Approach Abstraction

Can’t have actual space craft or DS1 module on VxWorks interacting with one another

Instead, use a small library of stubs Randomly emits possible responses (e.g., success/failure) when

specific hardware/software functions from VxWorks or spacecraft are invoked

Consists of 352 lines of C code

FEAVER test harness: It is 146 lines of text Defines three components:

Set of data objects in the application that hold basic state info Two asynchronous threads of execution that read and dispatch

messages in priority order: DownFifo controller and DownLink handler

A test driver that emits a random stream of valid commands for DS1

Term 032 4-3-19 CSE555-Sqalli

Correctness Criteria

Property that was known to fail: When the Downlink purge command was given a

command, a particular error scenario will prevent it from executing

This criteria can be formalized by LTL requirements on a variable v as:

[](v > 0 <>v = 0) “It is always the case that whenever the value of v become

greater than zero, eventually its value must return to zero at least once”

Variable of interest is: Downlink_waitingToPurge

Term 032 4-3-20 CSE555-Sqalli

Validation Results

Validation with SPIN resulted in errors Provided counter-examples Helped trace back to the cause

Two variations of the errors: Loss of a message sent to a full message buffer

Caused purge command to be lost Module left in a state where command is issued but not

executed This is precisely the known defect

Persistent stream of high priority messages Postponed the execution of purge command indefinitely

Both of these scenarios were found in few seconds

Term 032 4-3-21 CSE555-Sqalli

Conclusions of the Study

Used model checking: Identified a known error in the launch version of the DS1

spacecraft flight software

Discovered a second scenario of the same error

Discovered a rare race condition in another module

Causes of violation should be determined Real error vs. unrealistic input data or improper property

specification