2007sttt Rips

8/14/2019 2007sttt Rips

1/13

Software Tools for Technology Transfer manuscript No.(will be inserted by the editor)

Formal Verification of the NASA Runway Safety Monitor

Radu I. Siminiceanu1, Gianfranco Ciardo2

1 National Institute of Aerospace, Hampton, VA 23666, e-mail: [email protected] University of California, Riverside, CA 92521, e-mail: [email protected]

The date of receipt and acceptance will be inserted by the editor

Abstract. The Runway Safety Monitor (RSM) designedby Lockheed Martin is part of NASAs effort to reduceaviation accidents. We developed a Petri net model ofthe RSM protocol and used the model checking func-tions of our tool SmArT to investigate a number of safetyproperties for the RSM. To mitigate the impact of state-space explosion, we built a highly discretized model ofthe system, obtained by partitioning the monitored run-way zone into a grid of smaller volumes and by consider-ing scenarios involving only two aircraft. The model alsoassumes that there are no communication failures, such

as bad input from radar or lack of incoming data, thus itrelies on a consistent view of reality by all participants.In spite of these simplifications, we were able to exposepotential problems in the conceptual design of RSM. Ourfindings were forwarded to the design engineers, who un-dertook corrective action. Additionally, the results stressthe efficiency attained by the new model checking algo-rithms implemented in SmArT, and demonstrate theirapplicability to real-world systems. Attempts to verifyRSM with similar NuSMV and SPIN models have faileddue to excessive memory consumption.

Key words: Formal verification, model checking, avia-tion safety, collision avoidance protocols

1 Introduction

As the avionics systems that are put into operation growin complexity every year, an increasing share of the func-tionality in modern aircraft is shifted to computer-based,

Work supported in part by the National Aeronautics andSpace Administration under grant NAG-1-02095 and by the Na-

tional Science Foundation under grants CCR-0219745 and ACI-0203971.

automated devices. However, this rapid advance in so-phistication makes the verification and certification ofthe deployed devices more difficult. This is due to thetremendous amounts of resources, measured in time, hu-man expertise, and money, required for the analysis ofcomplex systems.

The field of formal methodsoffers an alternative totraditional testing approaches that can explore only alimited number of scenarios. Formal verification uses rig-orous mathematical techniques toexhaustivelycheck thata model of the system satisfies a set of desired properties.

Model checking [13], which has gained increased pop-ularity since the early 90s, is an automatic techniquethat relies on discovering the set of reachable states ofthe model and evaluating whether a given property, ex-pressed in atemporal logic, is satisfied or not. The modelis usually specified in a modeling language, such as au-tomata, Petri nets, or pseudo-code, rather than usingmathematical notation. If a temporal property holds,model checking attests it with 100% confidence. When aproperty does not hold, the model checking tool providesa counterexample, in the form of an execution path inthe model, which can illustrate the source of the errors.

When using these computerized tools to verify mod-

ern protocols, the major obstacle is usually the state-space explosionphenomenon. As the size and complex-ity of a model increases, the size of the state-space (innumber of reachable states) also increases, sometimesexponentially (in the number of components, variables,or parts of the system). Nevertheless, advances in modelchecking techniques, particularly insymbolicmodel check-ing [23], have made it possible to analyze systems withextremely large state spaces.

Model checking has been successful in the verificationof complex, mostly synchronous, circuit designs, whileverifying large asynchronous protocols and software has

been generally viewed as more difficult. For the last sev-eral years, our research has targeted the class ofglobally-

8/14/2019 2007sttt Rips

2/13

2 Radu I. Siminiceanu, Gianfranco Ciardo: Formal Verification of the NASA Runway Safety Monitor

asynchronous locally-synchronous systems[5], consistingof loosely coupled systems (homogeneous or heteroge-neous) evolving somewhat independently of each other.Recently, NASA and Lockheed Martin have begun de-veloping a protocol to detect runway incidents, calledthe Runway Safety Monitor (RSM) [16], which repre-sents an excellent candidate for testing and evaluatingour techniques. Although the verification of RSM waschallenging and pushed our computational resources tothe limit, we were able to discover several obscure scenar-ios that constitute potential hazards. Equally significant,however, is the fact that so few hazards were discov-ered overall, compared to the total number of reachablestates, 6.7 1042. This is strong evidence that RSM isrobust and safe.

The rest of the paper is structured as follows. Sec-tions 2 and 3 describe RSM and our tool SmArT, which

we used for this study. Section 4 gives the details of theRSM model we developed and Section 5 reports the re-sults of our analysis. Finally, Section 6 summarizes ourwork and discusses ideas for future extensions.

2 The Runway Safety Monitor

The Runway Safety Monitor (RSM) is a component ofNASAs Runway Incursion Prevention System (RIPS)research [20]. Designed and implemented by LockheedMartin engineers, RSM is intended to be incorporated inthe Integrated Display System (IDS) [1], a suite of cock-pit systems which NASA has been developing since 1993.IDS also includes other conflict detection and preventionalgorithms, such as TCAS II [22]. The IDS design en-ables RSM to exploit existing data communications facil-ities, displays, Global Positioning System (GPS), groundsurveillance system information, and data-links.

Collision avoidance protocols are already in opera-tion. TCAS [22] has been in use since 1994 and is nowrequired by the Federal Aviation Administration (FAA)on all commercial US aircraft. TCAS has a full formalspecification, but it has been verified only partially, dueto its complexity [4,17].

The Small Aircraft Transportation System SATS [3],also under development at NASA Langley to help en-sure safe landings of general aviation craft at towerlessregional airports, has instead been formally verified [14].

Purpose of RSM. The goal of the Runway SafetyMonitor is to detect runway incursions, defined by theFAA as any occurrence at an airport involving an air-craft, vehicle, person, or object on the ground, that cre-

ates a collision hazard or results in the loss of separation

with an aircraft taking off, intending to take off, landing,

or intending to land.

Since most air safety incidents occur on or near run-

ways, the Runway Safety Monitor plays a key role inaccident avoidance. RSM is not intended to prevent in-

IDS RunwaySafety Monitor

Start/Stop/ContinueIncursion Testing

Track all Trafficin Incursion Zone

Test for Incursions

IDS TrafficMonitor

RETURN

Incursion detected:set alert flags/data

No Incursion: clearalert flags/data

Test Own-ship inIncursion Zone

Test Traffic inIncursion Zone

Get Own-ship andTraffic States.

Compute Traffic Data

(each target belowzone altitude)

Each traffic scan

Testingnot required,

Ownshipnot in Zone

No zone traffic

Traffic in Zone

Testing for Incursions

Operational StatesMatrix

Fig. 1. RSM Algorithm top-level design.

cursions, but to detect them and alert the pilots. Pre-vention is provided by other components of RIPS in theform of a number of IDS capabilities such as heads-updisplay, electronic moving map, cockpit display of trafficinformation, and taxi routing. Experimental studies con-ducted by Lockheed Martin [16,27] show that incursionsituations are less likely to occur when IDS technology isemployed on aircraft. RSM should greatly improve thispositive effect.

RSM design. Figure 1 shows the high-level architec-ture of the RSM algorithm. RSM runs on a device in-stalled in the cockpit and is activated prior to takeoff andlanding procedures at airports. An independent copy ofRSM runs on each aircraft and refers to the aircraft on

which it is operating as ownship and to other aircraft,ground vehicles using the same runway, or even physicalobstacles such as equipment, as targets.

RSM monitors traffic in a zone surrounding the run-way where the takeoff or landing is to take place. Thezone is a 3-D volume of space that runs up to 220 feetlaterally from each edge of the runway, up to 400 feet ofaltitude above the runway, and 1.1 nautical miles fromeach runway end (the 400 feet altitude corresponds to a3 glide slope for takeoff/landing trajectories).

The protocol, implemented as a C-language program,consists of a repeat-loop over three major phases. In thefirst phase, RSM gathers traffic information from radarupdates received through a data-link. It identifies eachtarget present in the monitored zone and stores its 3-Dphysical coordinates. The frequency of the updates maynot be constant, updates can be lost, and data mighteven be faulty. The implications of data-link errors oromissions are not addressed in this study, but presenta challenging task for future study. These errors havealready been the subject of some experimental measure-ments [27], and their analysis calls for astochasticflavornot captured in our present model, which is instead con-cerned only with logical errors.

In the second phase, the algorithm assigns a status

to each target, from a predetermined set of values thatincludes taxi, pre-takeoff, takeoff, climb out, landing, roll

8/14/2019 2007sttt Rips

3/13

Radu I. Siminiceanu, Gianfranco Ciardo: Formal Verification of the NASA Runway Safety Monitor 3

out, and fly-through modes. We discuss in detail themeaning of these states when we describe our model ofthe system.

The third phase is responsible for detecting incur-sions, and is performed for each target based on the spa-tial attributes (position, heading, acceleration) of own-ship and target, plus some logical conditions. Table 2,discussed later, shows the operational state matrix ofthis phase. Our analysis focuses on verifying that thisdecision procedure is able to detect all possible incur-sion scenarios, or on finding possible incursion scenarioswhere RSM fails to raise an alarm.

3 Overview of the SmArT tool

To model the Runway Safety Monitor, we employ our

tool SmArT (the Stochastic and Model checking Ana-lyzer for Reliability and Timing) [7], which we developedfor the logical and stochastic analysis of structured sys-tems. Given a formal description of a system as a Petrinet, SmArT can generate the state-space, verify temporal-logic properties, and provide efficient numerical solutionsfor timing and stochastic analysis. SmArT has several ad-vantages over most other model checkers:

Compact storage for states with Multiway DecisionDiagrams (MDDs) [24], a generalized version of theBinary Decision Diagrams (BDDs [2]) for multi-valuedvariables.

Extremely compact encoding of the transition rela-tion between states with Kronecker matrices [9]. Efficient symbolic state-space exploration algorithms

based on saturation [8], a novel fixed-point iterationstrategy.

Fast generation of counterexamples, based on Edge-Valued MDDs (EVMDDs) [10].

The SmArT input is a Petri net with Turing-equivalentextensions (immediate transitions and marking depen-dent arc cardinalities) [26], required to have a finite state-space. Each SmArT input file defines one or more struc-tured (i.e., partitioned into submodels) event-based mod-els. A model can be parameterized and defines a set ofmeasures, which, in our case, can be thought of as logicalqueries to be evaluated by systematic state exploration.

SmArTimplements a wide range of explicit as well asimplicit exploration methods. Use of the most advancedtechniques requires a partitionof the model to exploitthe system structure. A partition of the Petri net modelintoKsubmodels is equivalent to partitioning its places(representing the system variables) into subnets. Subse-quently, a system state (a marking of the places withtokens) can be written as the concatenation ofK localstates (submarkings) and thus be encoded as an MDD.In particular, a partition is Kronecker-consistent if any

global system behavior can be expressed as a functionalproduct of local behaviors for each subsystem. From a

logical point of view, for example, an event in the modelis globally enabled if it is locally enabled in each of the Ksubmodels in isolation. Similar consistency requirementscan be defined for transition guard expressions or, froma stochastic point of view, for transition firing rates (butonly the logical interpretation of Kronecker consistencyis needed in this study). A more detailed discussion ofthe implications of consistency requirements follows inSection 4.

A comprehensive SmArT User Manual is available on-line [6].

There are several reasons for using a tool designedfor the analysis of discrete-state systems to model andverify an embedded (hybrid) system. Even though thereexist tools for the verification of hybrid systems, suchas HyTech [18], their focus is on the integration of dis-crete and continuous aspects of the systems. The dis-

crete aspects of a large application like RSM are wellbeyond the scope of the state-of-the-art hybrid modelcheckers. Moreover, it is clear that the actual RSM algo-rithm is implicitly using a discretized view of time: theradar updates are not fed continuously to the RSM de-vice, but only at a given frequency (nominally, at every0.5 seconds). This suggests that an abstraction schemefor the other continuous-type variables (location andspeed of aircraft) is adequate in this case. Last but notleast, the discretized RSM model belongs to the class ofglobally asynchronous/locally synchronous systems, forwhich the saturation-based algorithms for analysis im-plemented in SmArT excel.

4 The SmArT model of RSM

To model RSM, we first identify the variables represent-ing the system state and the events describing the po-tential state-to-state transitions. Then, we translate thisinformation into a Petri net for input into SmArT. Wepartition the model into n + 1 submodels where n isthe number of targets moving inside the zone. The vari-ables of the first submodel (indexed 0) describe the stateof ownship. The variables of the other n submodels de-

scribe the state of each target. For submodeli, 0 i n,the relevant attributes are:

Location:a 3-D vector (xi, yi, zi), where theX-axis isacross the width of the runway, the Y-axis is alongthe length, and the Z-axis is on the vertical.

Speed and heading: a second 3-D vector (vxi, vyi, vzi).Acceleration along the runway:ayi.Status:an enumerated type variable, statusi.Alarm flag: a boolean variable, alarmi.Phase:an integer variable, phasei.

All other variables are deemed irrelevant to our study

and can be abstracted away from the model to reducethe size of its state space.

8/14/2019 2007sttt Rips

4/13


As mentioned earlier, SmArT requires a partitioningof the model variables in order to apply the most ad-vanced symbolic model checking techniques. A naturalchoice is to group variables referring to the same targettogether. However, assigning all variables to the samepartition leads to extremely large local state spaces foreach submodel, which is unacceptable. A better choiceis to further split the subnets into even smaller ones. Wearranged the variables inn + 1 clusters, as follows:

subnet 5i+ 1 : phaseisubnet 5i+ 2 : statusi, alarmisubnet 5i+ 3 : xi, vxisubnet 5i+ 4 : yi, vyi, ayisubnet 5i+ 5 : zi, vzi

Domain of the state variables. Since SmArT operates

on discrete-type systems, abstraction by discretization isnecessary to cope with the continuous-type variables ofthe RSM algorithm. To come up with a good represen-tation of the variable domains, we start with the rough-est possible discretization that can be extracted fromthe protocol specifications, and then manually refine itfurther as needed. We had to take into consideration abalanced solution between a very rough discretization(which potentially hides too many meaningful behav-iors by merging distinct states into a single representa-tive), and a discretization that is too fine for an efficientstate-space generation (which prevents analysis due tothe state-space explosion problem). In the end, we chosethe following domains (the subscriptiis omitted for clar-ity):

The coordinatesx,y, z could be as simple asx, y,z {0, 1, 2}, where 0 means out of the monitored zone,1 means in the vicinity, and 2 means on the run-way. However, we chose a finer parametric repre-sentation:x {0,..., maxx}, y {0,..., maxy}, andz {0,..., maxz}, where 0 means outside the zone,and the constants maxx, maxy, and maxz can beadjusted to the modelers preference. In other words,location (0, 0, 0) represents all positions outside thezone. A target that exits the zone, or has not yet en-tered it, is assigned this location. As an alternative,we could have used an outer layer of locations sur-rounding the monitored zone, but this would unnec-essarily increase the state space with entries of theform (0, y , z), (x, 0, z), and (x,y, 0), all representingthe same circumstance: the target is not being mon-itored.

The speed values vx, vy,vzcould be assigned the do-main {0, 1, 2}, where 0 means not moving, 1means moving slowly (below the predetermined taxispeed thresholdT Sof 45 knots), and2 means mov-ing fast (above T S). Again, a better representation

is vx,vy,vz {maxspeed,..., 0,..., maxspeed}, usinganother parameter maxspeed.

1 2 3 4

1

2

Real trajectory: (51.5, 161.3), (128.5, 93.6), (220.1, 80.3), (318.5, 111.2), ...

Discretized trajectory: (1,2), (2,1), (3,1), (4,2), ...

t=2.0

t=2.5 t=3.0

t=3.5

t=4.0

Discretized speed: (+1,-1), (+1,-1), (+1,0), (+1,+1), ...

Fig. 2. Example projection of a continuous trajectory in 2-D.

Since, in Petri nets, places cannot hold a negativenumber of tokens, we have to offset the values of thespeed variables by maxspeed.

The acceleration ay has only two relevant values:non-negative or strictly negative.

The statusis one of{out, taxi, takeoff, climb, land,rollout,flythru}.

Thephaseis one of{radar update, set status, detect}.

The variable phaseworks like a program counter forthe execution of the algorithm on each participant, whichloops through three steps:

A. Update location of targets (phase= radar update).

B. Update status of targets (phase= set status).C. Set or reset alarm (phase=detect).

We next discus in detail the modeling decisions thatwere taken for each of these three steps.

A. The 3-D motion of targets. Our discretizationdivides the monitored space into a number of volumesarranged in a 3-D grid. As a result, the possible positionsof the aircraft are identified by a finite number of gridcells, from the discrete domain (0, 0, 0) {1,..., maxx} {1,..., maxy} {1,..., maxz}. Similarly, continuous tra-

jectories have to be represented by abstract, discretizedtrajectories through the cells of the 3-D grid. This is areasonable compromise when modeling continuous vari-ables with discrete-state approaches. Regarding the pos-sible trajectories allowed in the model, there are threealternatives to be considered.

First is the projection method, which assigns to everycontinuous trajectory its corresponding discrete path inthe grid. An example of such a projection is given inFigure 2 (in a 2-D space, for the sake of readability).The grid cells in the figure have the size of 100 units (infeet), and the snapshots are taken after each 0.5 timeunits (in seconds). The speed units are measured in thenumber axis divisions traveled after each update. The

major challenge in putting in practice this method isthe difficulty in discerning between physical possibilities

8/14/2019 2007sttt Rips

5/13


1 2 3

0

1

2

3

0 ...

x

y

x++, vx>0any vy

x++, vx>0y--, vy0

any vxany vy

current state possible next stateLegend:

Fig. 3.Possible 2-D movements of a target (free-motion model).

and impossibilities. There is no efficient way of rulingout all anomalies. For example, a target could changeits real location, while its discretized location might not.The dependency between the speed and the number oftime units a target may remain in one grid cell is alsovery difficult to establish: it could be one move (at highspeed), or more (at low speed), but no upper bound on

the number of time units allowed within one grid cellcan be computed in the discretized model.

Therefore, we have considered a different approach tomodeling the motion of targets, that proved to be morepractical. One alternative allows nearly free movementof a target, in the sense that a move to an adjacent cell isalways allowed. In principle, a target is free to remain inthe current cell or to move to any of the neighboring 26cells, corresponding to a nondeterministic decrease, nochange, or increase in the coordinates x,y , andz . How-ever, the changes must be consistent with the heading.This allows for almost random movements. On the onehand, the restriction to allow transitions only betweenadjacent cells excludes a large number of trajectories,most of which are truly physically impossible. On theother hand, we have to argue that no realistic trajectoryis excluded by the model. This is indeed true when thecell size is large (corresponding to a rough discretiza-tion of the space, into a small number of cells). In oursimplest model, which captured all the interesting prop-erties, the size of a grid cell is 900 feet. Given that thelocation updates arrive on the data-link every 0.5 sec-onds, a target can skip a grid cell and move to a cell twodiscrete positions away only if traveling at speeds ex-ceeding 1800 ft/sec 1227 mph (or 1975 km/h). This

is over 1.6 times the speed of sound. Although it is notentirely safe to assume that these speeds are not encoun-

set_statusradar_update

same_xinc_x

pos_vx any_vx neg_vx

dec_x

x

vx

[#(x) < maxx] [#(x) > 1]

0

doneupdate

ux uzuy

maxspeed-12maxspeed

maxspeed+1

2maxspeed 0

#(vx)

Fig. 4. Subnet for updating the variables x and vx in each sub-model

tered at civil airports, their exclusion from our model isreasonable and helps simplify the analysis. Moreover, arough discretization also serves the purpose of mitigat-ing the state-space explosion problem, as the number ofpossible states becomes manageable. Figure 3 shows the

possible moves of a target in this second model (also ina 2-D space, for clarity).

To achieve the non-deterministic choice, we modelthe 3-D motion not via 27 concurrent and mutually ex-clusive events, but rather via the composition of a non-deterministic choice to decrease, not change, or increaseeach coordinate. More precisely, the next position is com-puted as the composite effect of firing three concur-rent and independent events, non-deterministically cho-sen from among a set of three mutually exclusive optionsfor each axis, for a total of just nine events. The updateof the coordinatexiand speed componentvxifor targetiis modeled by the subnet shown in Figure 4 (updates forthey andz coordinates are analogous, they are triggeredby the arrival tokens in placesuy anduz , respectively).Petri net places are drawn as circles, transitions as rect-angles, and immediate transitions as thick bars. Transi-tions inc xanddec xhave associated guard expressions,and arc cardinalities (other than the default value 1) areshown on each arc. Note that #(a) indicates the currentnumber of tokens in place a, thus the effect of the arcfrom place vx to transition update is to reset the oldvalue ofvxin preparation for its new setting. The threearrays of immediate transitions are needed to assign avalue to vxi: random positive (1 vxi maxspeed,

i.e., from maxspeed+ 1 to 2maxspeed tokens), any value(maxspeed vxi maxspeed, i.e., from 0 to 2maxspeed

8/14/2019 2007sttt Rips

6/13


1 2 3

0

1

2

3

0 ...

x

y

vx=3, vy=3


Fig. 5. Possible movements from a state satisfying vx= 3, vy = 3(restricted model).

tokens), or random negative (maxspeed vxi 1,i.e., from 0 to maxspeed 1 tokens).

Also, when a target enters the zone, its position isnondeterministically chosen on the frontier of the mon-itored volume, i.e., x {1, maxx}, y {1, maxy}, orz= maxz (but notz = 1, since no entry is possible from

below ground). The entry speed parameters are also cho-sen nondeterministically, but consistently with the direc-tion of entry. For example, a target cannot enter fromthe left with a negative vx.

This second model might still include unrealistic tra-jectories. Examples of typically abnormal behaviors al-lowed in the model are: oscillating back and forth be-tween two adjacent squares (when the correspondingspeed components alternate from positive to negativeand back) or staying forever in one square, even witha positive speed. This is still acceptable in the verifi-cation process as long as the model covers all realisticbehaviors. If a property holds globally in the abstractmodel, then it will also hold in the realistic model. How-ever, if a property does not hold globally, we must checkthe corresponding counterexample generated by SmArTto determine whether it represents a realistic scenario.

If a more thorough elimination of unwanted trajec-tories is desired, a third alternative that forbids abruptvariations in speed can be considered. In other words,both the coordinates x, y,z and the speed componentsvx,vy,vz can change by at most one in absolute value.This further restriction can be achieved by allowing onlythe increase, decrease, and no change of speed at eachtimestep, together with a consistent update of the coor-

dinates: for example, the variablexcannot be decreasingwhen the speed component vx is non-negative.

1 2 3

0

1

2

3

0 ...

x

y


vx=1, vy=0

Fig. 6. Possible movements from a state satisfying vx= 1, vy = 0(restricted model).

In comparison to the free-motion model, Figure 5shows the possible next states (in 2-D space) for a targetwhose speed components are v x= 3 and vy = 3 in thecurrent state. In this case, only four new locations arepossible, corresponding to the no change or increase in xand, independently,y. The reduced number of choices is

due to the strictly positive value of the speed, which doesnot allow any move in the negative axis direction. Onlywhen one speed component is 0 in the current state, thetarget can move in both directions of the correspondingaxis, as seen in Figure 6. In this model, at least twosteps are required to go from positive to negative speed(and vice versa). This implies that zigzagging is notpossible, a fact that could have a significant importancein the analysis, as seen in Section 5.

We implemented both versions, with free or restrictedmovement between adjacent cells, in SmArT.

B. Status definitions. In the second phase of the exe-

cution loop, the status variable of each aircraft is deter-ministically updated using the other state information.In our model, the status values are:

out: not in the monitored zone (x= 0) (y = 0) (z = 0)

taxi: on the ground, either at low speed or not with arunway heading (z = 1)

((|vx| T S |vy| T S) (vx = 0))

takeoff: on the ground, with a runway heading, accel-erating (z = 1) (|vy|> T S) (vx = 0) (ay 0)

rollout: on the ground, with a runway heading, decel-

erating (z = 1) (|vy|> T S) (vx = 0) (ay < 0)

8/14/2019 2007sttt Rips

7/13


grid size Number of states in the model SS gen. time (sec) SS gen. memory (MBytes)

maxspeed 1 tar. 2 tar. 3 tar. 4 tar. 1 tar. 2 tar. 3 tar. 4 tar. 1 tar. 2 tar. 3 tar. 4 tar.

353 2 1.01013 3.41019 1.11026 3.51032 2.01 2.93 3.93 4.91 2.00 3.00 4.00 4.99

5105 2 4.11014 8.41021 1.71029 3.51036 5.52 8.27 11.19 13.91 7.21 10.81 14.41 18.01

101010 2 7.61015 6.61023 5.81031 5.01039 13.62 20.58 27.50 34.42 20.86 31.29 41.72 52.15

353 5 2.71014 4.41021 7.21028 1.21036 4.41 6.51 8.77 10.98 4.22 6.33 8.44 10.55

5105 5 8.31015 7.61023 6.91031 6.31039 12.91 19.07 25.42 32.05 15.48 23.21 30.95 38.69

101010 5 1.41017 5.01025 1.81034 6.71042 28.45 42.84 57.25 71.75 39.73 59.59 79.45 99.31

Table 1. State-space generation results for our model, for 1, 2, 3, or 4 targets and as a function ofmaxx maxy maxz and maxspeed.

climbout: airborne, with a runway heading, strictly pos-itive vertical speed (z >1) (vx = 0) (vz >0)

land: airborne, with a runway heading, negative verticalspeed

(z >1) (vx = 0) (vz 0)flythru: airborne, not in climbout or land mode

(z >1) (vx = 0)

The predicates z = 1 and z > 1 used above also implyx > 0 and y > 0, by the way we designed the non-monitored zone to be represented by a single cell, notby a rim of states. Also, the acceleration ay, needed todiscern betweentakeoff androlloutstatus, does not needto be modeled directly, since its value is computed on thespot based on the variation of the variable vy.

The partial model constructed so far can be used asa building block for further analysis, since it captures

the free movement of targets in 3-D space (phase oneof RSM) and the target status assignments (phase twoof RSM). This model exhibits strong event locality, i.e.,each event depends and affects only a few levels; thisis an essential property exploited by the saturation al-gorithm we employ. To evaluate its complexity, we col-lected measurements of the state spaces generated fordifferent input parameters of this core model: number oftargets,n, grid size maxx, maxy, and maxz, and speedthresholds,maxspeed. The state-space size, runtime, andmemory consumption are listed in Table 1. The resultsshow that the state space can be generated for multipletargets, a fairly large size of grid, and multiple thresholdsof speed, in a few minutes using under 100 MBytes.

C. Setting the alarm. The third and most importantphase of the RSM algorithm is setting the alarm flag forevery target. In pseudocode, this corresponds to a singlevariable assignment statement: set the (boolean) valueof each alarmi based on different combinations of thecurrent values of the other variables, as listed in the op-erational state matrix of Table 2. We can either modelthe third phase directly, by adding transitions to thePetri net, or define queries that use a combination ofstatus and position variables to determine whether the

alarm would have been set correctly. We choose the for-mer approach.

Target taxi takeoff climbout land rollout flythru

Ownship

taxi af af af acf

takeoff af de de de ad bc

climbout af de de de de bc

land af de de de ad bcrollout acf ad ad ad de bc

flythru bc bc bc bc

a: Distance closing

b: In takeoff/landing path

c: Distance less than min. sep.

d: Takeoff/landing in same direction, less than min. sep.

e: Takeoff/landing in opposite direction, closing

f: Taxi/stationary on or near runway

Table 2. Operational state matrix for setting the alarm.

Modeling this rather complex assignment statementin a Petri net is difficult because of two factors. First,predicates such as distance is closing or in the takeoffpath potentially involve geometry and linear equationsand are difficult to express in a discretized model. How-ever, certain factors help make our task easier: the de-signers kept the concepts simple and trigonometry canbe circumvented on a case by case basis. For exam-ple, distance to target i is closing should normallybe evaluated by comparing the value of the expression

(x0 xi)2 + (y0 yi)2 + (z0 zi)2 in the current andprevious states. This could further imply that the pre-

vious location of each target should be stored in a setof auxiliary variables, say oldxi, oldyi, oldzi, further in-creasing the state space. However, this can be avoided byexploiting the information derived from each aircraftsstatus. For example, if ownship is taxiing and target i istaking off, we know that z0 = 1, vx0, vy0 T S, zi = 1,vxi = 0, |vyi| > T S, and vz0 =vzi = 0, i.e., the targetis on the ground, lined up with the runway and movingfaster than the taxi speed limit. For the distance to beclosing, it is enough for ownship to be in front of the tar-get, depending on which direction this is moving. Hence,in this situation, the predicate can be expressed as

a (vyi > 0 y0 > yi) (vyi < 0 y0 < yi).

8/14/2019 2007sttt Rips

8/13


maxspeed 2 3 4 5

grid size State-space generation time (sec.)

3 5 4 75.92 105.17 179.28 252.253 7 4 195.54 324.65 604.23 805.95

3 10 5 995.18 2212.24 4668.55 7348.27

5 10 7 48257.3 - - -Memory consumption (MB)

3 5 4 11.19 21.20 32.58 49.393 7 4 18.27 36.02 56.91 87.25

3 10 5 42.59 83.53 138.56 218.855 10 7 246.22 - - -

Table 3. Results from state-space generation on the completemodel.

We can similarly express the other predicates as follows:

b c (vy0 > 0 y0 yi y0+ 1 |x0xi| 1 zi 2)

(vy0 < 0 y0 1 yi y0 |x0xi| 1 zi 2)d vy0 vyi > 0 |x0 xi| 1 |y0 yi| 1e (vy0 > 0 vyi < 0 yi y0)

(vy0 < 0 vyi > 0 yi y0)f 1 < xi< maxx,

where the above example formulae are derived for thefollowing pairs of states, respectively: b c for takeoff-flythru,d and e for takeoff-takeoff.

The roughness of the discretization can also help sim-plify the model. Ifmaxx = 3 (which is a reasonable as-sumption given that there is usually no room for two air-craft on the runway side-by-side, anyway), then xi = 2for any aircraft taking off or landing. In this case, thepredicate |x0 xi| 1 is equivalent to 1 x0 3,which is always true when ownship is not out.

Kronecker consistency requirements. A second chal-lenge in modeling the third phase is that the Kroneckerconsistency requirements force us to split events intomultiple finergrain events. For example, the predicatetarget i is in takeoff/landing path of ownship can beexpressed as:

(xi = x0)(((vy0 > 0) (yi > y0)) ((vy0 < 0) (yi < y0))) .

However, since variables yi andy0 are described by dif-

ferent local states of the model, each term involving thetwo must be split to satisfy Kronecker consistency, bydomain enumeration:

1Cymaxy((xi = x0) ((vy0 > 0 yi = Cy Cy > y0)

(vy0 < 0 yi = Cy Cy < y0)))

The same procedure must be applied to x0 and xi, byfurther splitting terms:

1Cxmaxx

1Cymaxy

((xi = Cx)(x0 = Cx)(vy0 > 0)(yi = Cy)(Cy > y0))

((xi = Cx)(x0 = Cx)(vy0 < 0)(yi = Cy)(Cy < y0))

This generates 2 maxx maxy events from a single origi-nal event, one for each term of the disjunction. The split

EXp EFp EGp E[pUq]

AXp AFp AGp A[pUq]

pholds qholdsLEGEND: dont care

Fig. 7. The semantics of CTL operators.

events require over 2000 lines of additional SmArTcode,compared to just 500 lines needed to model the first two

phases. At the end of this process, the most significantchange in the model was a severe loss of event local-ity, leading to a slowdown in generation time and, mostimportantly, a much higher memory consumption. Thepeak MDD size increased to over 1000 times larger thanthe final, causing the SmArT model checker to run out ofmemory for large parameters, including multiple targets.However, we were still able to build the state space forone target and a medium size of the grid, within 1GB ofmemory and in less than five minutes. This was enoughto expose several potential problems with the decisionprocedure of the protocol. Table 3 shows the state-spacemeasurements on this final SmArT model with just one

target. Missing entries in the table correspond to param-eter choices that required excessive runtime or memory.

To compare our saturation technique with other ex-isting methods, we tried using the symbolic model checkerNuSMV [11] and the explicit model checker SPIN [19]on equivalent models of RSM. However, neither tool wasable to construct the state-space successfully. NuSMVruns out of memory even before starting the generation,as the BDD encoding of the transition relation is toolarge, while SPIN explores a very small fraction of thestate-space (less than 1/106, even when using partial or-der reduction) to be able to expose any problem. Weneed to mention here that other techniques, such as com-

positional state-space minimization (CADP/CWB) [15]or probabilistic model checking (PRISM) [21], have alsobeen successfully used to analyze large state-spaces, buttime contraints prevented their analysis in this project.

5 Model checking RSM

Model checking is concerned with verifying temporal logicproperties of discrete-state systems evolving in time.

SmArT implements the branching time ComputationTree Logic(CTL) [12], widely used in practice due to its

simple yet expressive syntax. In CTL, operators occur inpairs: the path quantifier, either A(on all future paths)

8/14/2019 2007sttt Rips

9/13


LEGEND:

ownship position

target position

min. separation

Fig. 8. Scenario for the memory-less property (ground level).

or E (there exists a path), is followed by the tense op-erator, one of X (next), F (future, finally), G (globally,

generally), andU

(until). Their semantics is informallyshown in Fig. 7, where system states are depicted asthe nodes of the trees and arcs represent transitions be-tween states, so that a node precedes in temporal orderthe nodes it can reach. In each case, the root node islabeled with a CTL formula it satisfies.

Notation and formal definitions. The operationalstate matrix in Table 2 lists the alarm setting criteria, asgiven in the documentation of the RSM algorithm [16].Our study aims at exhaustively checking whether thisoperational matrix is able to detect all incursion sce-narios. A situation where two aircraft get too close toeach other (within the minimum separation distance of

900 feet) without the alarm variable having been set isfrom now on called a missed alarmscenario. The follow-ing predicates are used to describe properties of interest(for sake of clarity, subscripts o and t refer to ownshipand target, respectively):

detectphaseo = detect phaset = detect

sep distance(o, t)> min. sep.

alarmalarmt =true

track statuso {taxi,flythru} statust {taxi,flythru}

We begin with asking the most simple safety property.

A safety property. Is there a tracked state where min-imum separation is lost and the alarm is off ?

CTL syntax: EF(detect track sep alarm)

The omission of the predicate detectfrom the querycan lead to false positives since, if a targets coordinatesare inspected in the middle of the radar updates or itsstatus is queried before it is modified accordingly, thedata can be inconsistent. Therefore, all queries should beasked only at the right moment: when ownship executesphase C of its algorithm.

A scenario that satisfies the query arises when thecondition distance is closing is not satisfied in the cur-

rent state. This is the case in the third snapshot of Figure8. However, this might not correspond to an unwanted

LEGEND:

ownship position

target position

min. separation

in landing path

Fig. 9.Scenario 2, airborne: flythru target in conflict with landingaircraft.

behavior, since the alarm might have been set in a pre-

vious state, when the minimum separation was lost. Thevalue of the alarm variable also depends on whether thealarm is aged or not for a few more cycles. Never-theless, the situation is still of potential concern, evenwith aging of the alarm, since the target can maintaina constant distance (at less than minimum separation)for longer than the duration of the aging, eventually re-sulting in a bad state in the round after the alarmexpires.

The memory-less nature of the query influences theresult. We looked at the property in a particular snap-shot of time, without considering the sequence of eventsleading to the current state. To get a better understand-

ing of the system, we next investigate the states of thesystem immediately after the minimum separation dis-tance between two aircraft is lost.

Analysis of the transition that causes loss of sep-

aration. Is there a state where minimum separation islost by transitioning to the current state while the alarm

is off ?

CTL: EF(detecttracksep E[(detect) U (detecttrack sep alarm)])

The nested EU operator in the query (instead of EX)is due to the fact that several transitions are needed to

complete the update of the coordinates, 3, and of thestatus, 1, and to set the alarm again, 1. A witness forthis query (see Figure 9) has ownship in a landing orclimbout state, the target flying across the runway fasterthan ownship, moving within separation distance fromthe side at an angle. The condition for setting an alarmin this circumstance is distance less than minimum sep-arationand target in takeoff/landing path. The secondterm is not satisfied, hence no alarm is raised. Aircraftcan actually collide (trajectories intersect in Figure 9),while none of the participants are ever warned.

The above scenario is the only one satisfying this

query, a fact attesting to the robustness of RSM. Thissituation can be corrected by adding distance less than

8/14/2019 2007sttt Rips

10/13


LEGEND:

ownship position

target position

min. separation

Fig. 10. Scenario 3 (ground level): taxiing target interferes withownship taking off.

minimum separation as part of the criterion for thiscombination of states.

We included the predicate track in both states (be-fore and after the transition), as we are interested inscenarios involving only takeoff and/or landing trajec-tories. However, this additional constraint could masksome other undesired behaviors. Therefore, we next aska more general question.

A stronger safety property. Is there a tracked statewhere minimum separation is lost, reachable withoutever

previously setting the alarm?

CTL: E[(alarm) U (detecttracksep alarm)];

Several scenarios satisfy this query.

Example 1. As shown in Figure 10, actors enter themonitored area taxiing fast, not aligned to the runway,and already at close distance to each other. Note thatthe RIPS specifications explicitly ignore this situation,as the algorithm is only active when ownship is takingoff or landing. However, once on the runway, say own-ship changes direction and aligns itself to the runway.Thereafter, it is categorized as takeoff (or climbout, if itbecomes airborne). The other aircraft stays within mini-mum separation, but it does not close in: it can be eitherbehind ownship or, more dangerously, in front of it. Noalarm is raised because the criterion distance is closingis, again, not satisfied. If the distance between aircraft

at entry is very small, there might not be enough timefor an escape maneuver, even if, later on, the alarm isset by closing in.

Figure 10 shows an abstract trace that contradictsthis safety property. The trajectories are shown for ahorizontal section in the monitored zone at ground level.The third snapshot illustrates the bad state of thesystem: the two aircraft are within minimum separationdistance, but no alarm has been issued either for thecurrent state or any of the previous states in the scenario.

Example 2. An identical scenario exists for airbornestates that are not tracked (status flythru).

Example 3. Additional scenarios do not satisfy thissafety property, where events develop immediately after

...

...

LEGEND:

ownship position

target position

min. separation

Fig. 11.Scenario 4 (ground level): target zigzags near the ownship

taking off.

both planes enter the monitored zone. The bad behav-ior in these cases is caused by the fact that the previousposition is unknown coordinates (0, 0, 0) in our model for both planes, hence the distance cannot be closingin the next state. If the airplanes enter the zone at po-sitions very close to each other (e.g., both are trying toland), the alarm will not be raised. However, this behav-ior is exhibited only in our theoretical model, due to ourchoice of modeling conventions, and not in the actual

implementation of the protocol on the RSM device.

Summarizing the common characteristics of the abovescenarios, we observe that the key factor is that bothaircraft are in the taxi or flythru status when minimumseparation is lost. The situation is not tracked, hence apotentially bad occurrence is masked by a protocol spec-ification. The predicate distance closing is not satisfiedand no alarm is issued, although the distance is less thanminimum separation.

To further extend our discussion, we look at possi-ble continuations of the scenario after the bad state isreached. If the distance is closing in the next state, awarning will be issued and the missed alarm situationwill cease to exist. The only way for a malicious agentto perpetuate the problem is shown in Figure 11, whichis an extension of Scenario 3. The target can stay withinminimum separation radius for a longer period of timeif it zig-zags in front of ownship and at each radarupdate has the same distance to ownship. The targethas to zig-zag to maintain the distance, since followinga parallel path to ownship will cause RSM to considerit as taking off. The alarm criterion for the new com-bination of operational states is taking off in the samedirection and distance less than minimum separation.

Therefore, an alarm will be issued as soon as the targetstops zig-zagging.

8/14/2019 2007sttt Rips

11/13


LEGEND:

ownship position

target position

min. separation

Fig. 12. Scenario 5: aircraft vs. ground vehicle.

The case when the target is not an aircraft (vehicle,service truck, etc.) adds an extra degree of freedom for a

malicious behavior of the target (see Scenario 5, in Fig-ure 12). Initially, ground vehicles were always consideredin taxi mode by the protocol, regardless of their speed,heading, and physical coordinates. Therefore, as in Sce-nario 4, the target may follow ownship at close distance,and even continue chasing ownship after it is lined upfor takeoff and accelerating. No flag will be raised forthe same reasons as in Scenario 4.

For the most recent implementation of RSM, the de-signers took into account our findings and eliminatedthe special treatment given to ground vehicles. This ad-dresses the situation in Scenario 5.

The situation in Scenario 4 is of less concern, since

it is extremely difficult to realize in practice, even inten-tionally by a saboteur. At the same time, there is somebenefit in exposing it: the designer is aware of this low-probability event. Also, by the fact that it is the onlyremaining unwanted behavior in the system, it serves asa validation for phase 3 of the RSM algorithm.

6 Conclusions and future work

Human effort vs machine effort. The modeling andanalysis of RSM was conducted in a relatively short pe-riod of time: twelve months. Up to three people workedin various stages of the project, for an estimated to-tal of nine man months of work. Of this, approximatelysix months were dedicated to modeling decisions, duringwhich three regular meetings with the design engineerswere scheduled. The final three man months were spentwith the verification per se: formulating queries, analyz-ing, and synthesizing the answers. The understandingof RSM was facilitated by the the designers strive tokeep the specification and implementation as simple aspossible.

Overall, while the human effort was predominant inall stages of the analysis, mostly due to the lack of au-

tomation in the modeling process, the impact of effi-cient model checking tools was crucial in completing the

study. At the same time, for a short-time collaborationlike this, it was only natural that the experience of themodelers was often preferred to certain automated pro-cesses, like automatic model extraction and automatedabstraction/refinement schemes.

Lessons learned. Several lessons were learnedfrom our analysis, first and foremost that formal verifica-tion has an undeniable value. We presented the designerswith a list of important findings which were not exposedduring the testing activities involving real aircraft, al-ready underway at different airport locations. The meritof our technique is that, besides being considerably lessexpensive than testing, it is exhaustive.

We were able to analyze all possible scenarios in ourmodel and found situations of potential concern thathappen with extremely low probability. These are al-most impossible to expose during either testing proce-

dures, which usually afford no more than a dozen testflights a day, or simulation sessions. When compared tothe actual state space sizes of the order of 1013 1042

states, this shows the need for exhaustive analysis.The second outcome of our experiments was that,

after identifying the problems and suggesting modifi-cations to the protocol to eliminate them, we have in-creased the level of assurance of the design in what con-cerns missed alarms. All the findings were related to situ-ations when only one aircraft is landing or taking off andthe other is not. The section of the decision table dealingwith both aircraft landing or taking off was validated inthe original form.

With respect to the dual analysis, of false alarms, thisis still on the list of future plans. From a practical pointof view, pilots are equally concerned with both types ofsituations. Individual reports indicate that frequent falsealarms can become a distraction or, in the best case, anuisance factor in operating an airplane. It is also thecase that a system with too many false alarms will tendto be switched off or ignored, thus rendering it useless.Therefore the occurrence rate of false alarms has to bereduced, even though these are not as critical as missedalarms, which should be completely eliminated. The de-signers of the protocol had to come up with a balancedsolution trading the simplicity of very loose require-

ments that raise too many alarms for the complexity ofstricter conditions that decrease the number of falsealarms, but make the analysis more difficult.

Another aspect not discussed here is fault tolerance.We assumed that all scenarios happen in the absenceof communication faults, meaning that the radars anddata-links provide accurate and timely updates to allparticipants. A natural extension of our analysis is to in-clude faulty behaviors, of either benign nature (missed orlate updates) or malicious/Byzantine (inconsistent databetween participants). This type of analysis requires theinclusion of probabilistic aspects in the model, and will

be the subject of further research. While our work veri-fies the correct operation of RSM under no-fault assump-

8/14/2019 2007sttt Rips

12/13


tions, the presence of faults on the data-link may signif-icantly impact the correct operation of the algorithm.On the one hand, if all data is faulty, RSM will be ofno help whatsoever in avoiding incursions. On the otherhand, if no data is faulty, we have already demonstratedthe correctness of the algorithm. The task of realisticallymodeling faulty data that falls in between these extremesis a major challenge.

Finally, this case study has inspired ideas of theo-retical nature that can result in improvements and ex-tensions to our technique. The observation that a singletemporal logic query can have more than one counterex-ample (thus correcting it alone will not entirely rid thesystem of the error) suggests that generating and stor-ing all counterexamples is beneficial. This is not com-monly done by model checkers. Also, even though theKronecker matrix encoding of the transition relation has

been found to be more efficient than the traditional BDDencoding in all our previous studies, this particular appli-cation has revealed a situation when the need to satisfythe consistency requirements could be detrimental, byproducing an excessive number of split events. A satura-tion approach that does not require the model to be Kro-necker consistent has been proposed in [25], but it cansuffer from poor performance due to excessively large lo-cal state spaces; we are currently working on extendingit so that it is completely general, yet with an efficiencyapproaching that of the Kronecker-consistent case.

References

1. Sharon O. Beskenis, David F. Green, Paul V. Hyer, andEdward J. Johnson, Jr. Integrated Display System forLow Visibility Landing and Surface Operations. NASALangley Contractor Report 208446, July 1998.

2. R. E. Bryant. Graph-based algorithms for boolean func-tion manipulation. IEEE Trans. Comp., 35(8):677691,Aug. 1986.

3. Victor Carreno, Hanne Gottliebsen, Rick Butler, andSaraswati Kalvala. Formal Modeling and Analysis of aPreliminary SATS Concept. NASA Langley TechnicalReport 12999, March 2004.

4. William Chan, Richard J. Anderson, Paul Beame, Steve

Burns, Francesmary Modugno, David Notkin, and JonD. Reese. Model checking large software specifications.IEEE Transactions on Software Engineering, 24(7):498520, July 1998.

5. Daniel M. Chapiro, Globally-Asynchronous Locally-Synchronous Systems. Ph.D. Thesis, Stanford Univer-sity, 1984.

6. Gianfranco Ciardo, Robert L. Jones, Andrew S. Miner,and Radu I. Siminiceanu. SmArT: Stochastic Modelchecking Analyzer for Reliability and Timing, User Man-ual. http://cs.ucr.edu/ciardo/SMART/.

7. Gianfranco Ciardo, Robert L. Jones, Andrew S. Miner,and Radu I. Siminiceanu. Logical and stochastic mod-eling with SmArT. In Modeling Techniques and Tools

for Computer Performance Evaluation, LNCS 2794, pp.7897, September 2003.

8. Gianfranco Ciardo, Gerald Luttgen, and Radu Si-miniceanu. Saturation: An efficient iteration strategy forsymbolic state space generation. In Proc. Tools and Al-gorithms for the Construction and Analysis of Systems(TACAS 2001), LNCS 2031, pp. 328342, Genova, Italy,

April 2001.9. Gianfranco Ciardo and Andrew S. Miner. A data struc-

ture for the efficient Kronecker solution of GSPNs. In Pe-ter Buchholz, editor, Proc. 8th Int. Workshop on PetriNets and Performance Models (PNPM99), pp. 2231,Zaragoza, Spain, September 1999. IEEE Computer Soci-ety Press.

10. Gianfranco Ciardo and Radu Siminiceanu. Using edge-valued decision diagrams for symbolic generation ofshortest paths. Proc. Fourth International Conferenceon Formal Methods in Computer-Aided Design (FMCAD02), LNCS 2517, pp. 256273, Portland, November 2002.Springer-Verlag.

11. Alessandro Cimatti, Edmund M. Clarke, Fausto

Giunchiglia, and Marco Roveri. NuSMV: a new Sym-bolic Model Verifier. Proc. Computer-Aided Verification(CAV99), N. Halbwachs and D. Peled, editors, LNCS1633, pp. 495499, Trento, Italy, July 1999. Springer-Verlag.

12. Edmund M. Clarke and E. Allen Emerson. Design andsynthesis of synchronization skeletons using branchingtime temporal logic. IBM Workshop on Logics of Pro-grams, LNCS 131, pp. 5271. Springer-Verlag, 1981.

13. Edmund M. Clarke, Orna Grumberg, and Doron A.Peled. Model Checking. MIT Press, 1999.

14. Gilles Dowek, Cesar Munoz, and Victor Carreno. Ab-stract Model of the SATS Concept of Operations: InitialResults and Recommendations. NASA Langley Techni-

cal Report 213006, March 2004.15. J.-C. Fernandez, H. Garavel, A. Kerbrat, and L.

Mounier. CADP: a protocol validation and verificationtoolbox. LNCS 1102, pp. 437446, Springer-Verlag, 1996.

16. David F. Green, Jr. Runway safety monitor algorithm forrunway incursion detection and alerting. NASA LangleyContractor Report 211416, Jan 2002.

17. Mats P.E. Heimdahl. Experiences and lessons from theanalysis of TCAS II. Proc. ACM SIGSOFT Int. Sympo-sium on Software Testing and Analysis (ISSTA 96), pp.7983, 1996.

18. Thomas A. Henzinger, Pei-Hsin Ho, and Howard Wong-Toi. HYTECH: a model checker for hybrid systems.International Journal on Software Tools for Technology

Transfer (STTT), 1(12):110122, December 1997.19. Gerard J. Holzmann. The model checker SPIN. IEEE

Transactions on Software Engineering, 23(5):279295,1997.

20. Denise R. Jones. Runway Incursion Prevention SystemFact Sheet. October 2000.

21. Marta Kwiatkowska, Gethin Norman, and David Parke.PRISM: Probabilistic Symbolic Model Checker. Proc.Computer Performance Evaluation / TOOLS, pp. 200204, Apr 2003.

22. Carolos Livadas, John Lygeros and Nancy A. Lynch.High-Level Modeling and Analysis of TCAS. RTSS 99,IEEE Press, 1999.

23. Kenneth L. McMillan. Symbolic Model Checking: An

Approach to the State-explosion Problem. PhD thesis,Carnegie-Mellon Univ., 1992.

8/14/2019 2007sttt Rips

13/13


24. Andrew S. Miner and Gianfranco Ciardo. Efficient reach-ability set generation and storage using decision dia-grams. Proc. 21th Int. Conf. on Applications and Theoryof Petri Nets (ICATPN 99), LNCS 1639, pp. 625, June1999.

25. Andrew S. Miner. Saturation for a general class of mod-els. In G. Franceschinis, J.-P. Katoen, and M. Wood-side, editors,Proc. QEST, pages 282291, Enschede, TheNetherlands, Sept. 2004.

26. Tadao Murata. Petri Nets: properties, analysis and ap-plications. Proc. IEEE 77(4):541579, April 1989.

27. J. Timmerman. Runway Incursion Prevention System,ADS-B and DGPS data link analysis, Dallas Ft. WorthInternational Airport. NASA Contractor Report 211242,NASA Langley, Hampton, VA, November 2001.

Date post:	04-Jun-2018
Category:	Documents
Upload:	sfofoby
View:	222 times
Download:	0 times

2007sttt Rips

Documents