Inspection of Safety-Critical SoftwareCase study: Darlington nuclear power plant shutdown system

Prepared by: Ke Tang

The Darlington Nuclear Generating Station

The Darlington Nuclear Generating Station Provides 20% of Ontario’s electricity Four reactor station Each has 2 independent shutdown systems 1. Drop neutron-absorbing rods into the core 2. Injects liquid poison into the moderator Decided to computerize Design Diversity

Why computerize

Software solutions more complex than

hardware, but offer more intelligent monitoring

Offer diagnostic aids, periodic testing can be performed more often

Requires less equipment

Could improve safety, worth a try

hardware solutions Simple, easily studied,

Offer a great deal confidence

Couldn’t perform sophisticated checking

Requires large amount equipment

Original Software Development 1 Work begin 1983, multiple versions were

development, mostly by experts of safety system engineering and control system

Software hires for second and third versions without backgrounds in nuclear engineering

Surprisingly small amount of code- 6,000 lines of assembly code- 7,000 lines of FORTRAN, 13,000 Pascal

Unit testing, integrating test, random testing. Software assessment, hazard analysis

Original Software Development 2 Review by AEBC in 1987, License not granted There exited too much doubt about whether the software

implemented the requirements correctly Functions performed by well-understood hardware devices

were performed by much more difficult understood software routines

Program components can’t physically separated and worked “in parallel”

“ …there was no widely accepted definition of what constituted ‘good enough’ for software used in safety-critical applications such as shutdown system”

- John Harauz

Software Inspection AECB hired David Parnas of Queen’s University

in mid of 1987 The document written in nature language,

information was hard to find, no confidence that all of the problem has been found

Parnas proposed a formal mathematical inspection in Jan. 1989

Reverse engineering, code existed, but formal specification was need

Two dimension expressions, tables, were used. Helped both designers and reviewers

Darlington Inspection

Uses two documents:– Software requirement document contains description of

actions that must be taken on a single pass through the periodic loop (the shutdown system is designed as a periodic loop).

– Program function descriptions describe effects of the execution of the loop body.

Goal:– Both documents describe the same functions.

– Confidence they are the right functions.

Software requirement document Nuclear plant is complex, but only a small portion

of the information is actually relevant to understanding shutdown system

System requirement document separate the complex details of outside word from the complex details of the design of the computer system

Precisely document, the requirement of software must satisfy. Which is needed by programmers and QA group

Nature language is not adequate for the task of precise requirement specification

Example Description and 3 reasonable interpretation Shut off the pumps if the water level

remains above 100 m for more than 4 s”

100])(4

1[

100))](())(([2

1

100])([4

1

2/1

4

2

],4[],4[

4

T

T

TTTT

T

T

dttWL

tWLMintWLMax

dttWL

Example Description and the analogous ambiguity Shut off the pumps f the water level remains

above 100 m for more than 4 s” Shut off the pumps if the minimum water

level over past 4 s was above 100 m”

100))]((],4[ tWLMin TT

Rigorous Approach Correctness can be achieved with highly formal

methods. Mathematical specifications offer the promise of

concise, precise description Rigorous Construction – Methods for deriving

programs from precise specification. Verification – methods for showing that a program

satisfies its specification. Approach based on control theory models have

been proven practical

Documentation Of Computer System design

System Requirement

treats the computer system as a black-box, and gives a description of the environment in terms of environment state variables. It describes the relations among these state variables result from physical, natural or other constraints. It also specifies the additional relations that must be established and maintained by the computer system itself when it will operate on this environment.

Documentation Of Computer System design

System design document

Identifies the computers within the system, and specifies the communications between the computers and the environment by describing the relations that will exist between the values of the environmental state variables and the values of the computer

Documentation Of Computer System design

The System Requirement Document and Design Document together determine the Software Requirements.

Software Function Specification

Record additional design decisions and describes the exact behavior of the software

Environmental State Variables State variables: characterize the environment in

which the system will have to operate. Classified as monitored or controlled, or both. Monitored variables are those the system has to

measure (pressure, temperature, pushbutton states, etc ,..);

Controlled variables are those the values of which the system is intended to modify and operate upon (switch, valve states, ...).

Functional Relations

The contents of the documents are a representation of the following mathematical relations

1) REQ(mt,ct)

2) IN(mt, it)

3) OUT(ot,ct)

4) NAT(mt,ct)

5) SOF(it,ot) mt, it ,ot ,ct [IN(mt, it)^ SOF(it,ot)^

OUT(ot,ct)^NAT(mt,ct)]REQ(mt,ct)

The use of Tabular Representations

software programs usually have a large number of discontinuities that can occur at arbitrary points in the domain.These functions are discrete or piecewise continuous.

Tabular formats have been found to be more practical for the description and communication of these functions between system designers and software engineers than conventional mathematical formulations.

Parnas Tables

Parnas Tables use tabular constructs to organize mathematical expressions, where

rows and columns separate an expression into cases

each table entry specifies either the result value for some case or a condition that partially identifies some case

Design and Documentation of Modules

Modularization is the key to control of software complexity

Divided modules respect to processing -> strong coupled modules->shared resources, hard to understand

“information hiding” modules hide( or abstract) a data structure,device interface, mathematical model of some physical system.

Must accompanied by precisely documented module interfaces

Program Function Documentation

Describes the precise effect of the program without describing the intermediated states

Each document contains representation of certain relations, otherwise if is incomplete.

Additional information makes the document faulty --> information not directly related should go elsewhere.

Program Function Documentation

Need precise notation for relations. Conditional expressions is too complex. Multi-dimensional notation is the author’s

choice. Program function tables have a practical

advantage.

Program Function Documentation

Tables control the complexity of the expressions

1) Tables parses the expression, made it easy to understand for the reader;

2) Tables eliminates many repetitions of sub expressions

3) Each table entry applies to a small part of the functions domain, the expression in that entry can be simplified

Program Function Tables

Quality Control

Quality Control for safety-critical software have three components:– Testing of the product– Inspection of the code.– Evaluation of the design and the methods that

they use

Testing and verification are not alternative approach, but complementary

Testing and review Planned testing and random testing are needed

– Planned Testing ensure coverage of all “interesting” cases

– Random testing test cases that will be encountered in practical respected to the statistical distributions.

Planned testing is more effective in uncovering errors, but often miss subtle cases.

Review: Check a section of the program to see if it actually did what they should.

A formal review based on the formal documents.

Inspection Process

Designed around the following ideas:– Inspectors need quiet time to “think.”– Inspection results must be scrutinized carefully.– Lengthy inspections must be done is smaller

sessions (ie. multiple days)– Inspections must focus on small sections of the

program at one time.– All cases must be considered.– All parts of the programs must be inspected.

AECB Inspection Teams Requirements team( designer of CANDU) –

produces tabular representation of the requirements. Code inspection team( consultants without

knowledge of nuclear field ) – produces program-function tables from the code.

Comparison team(Ontario Hydro employees) – finds equivalence between requirements tables & program-function tables by showing step-by-step transformations from one to the other.

Audit team(AECB employees and consultants) – checks the work of the other three teams

Testing and review using tables

Tables can facilitate both random and selective testing. The tables identify the discontinuities in the program functions and clearly identify the limiting and boundary cases which are known to be major sources of programming faults.

Tables also identify intervals between discontinuities and help verify that all intervals have been covered by a test set.

Conclusions It is certain that formal - documentation is a key

issue in the specification, design, and review of software for critical control and safety functions.

Tabular forms present great advantages over other forms of documentation techniques. Make the systematic inspection advanced.

This approach(Tabular forms) is practical when reliability and trustworthiness are extremely important and the codes are relatively small

Darlington project: costly, but built the Canada Standard

References: D.L. Parnas, G. Asmis and J. Madey, Assessment of

Safety-Critical Software in Nuclear Power Plants, Nuclear Safety, vol. 32, no. 2, April-June 1991, pp. 189-198.

P. J. Courtois, D. L. Parnas, Documentation for safety Critical software, Processing of the 15th international conference on Software Engineering, p.315-323, May 17-21, 1993

Parnas D. L., Van Schouwen, A.J., Shu Po Kwan (1990) Evaluation of Safety Critical Software. Communications of the ACM, 33,6,636-648.

References: D Parnas, Inspection of Safety-Critical Software Using

Program-Function Tables, in Report No. 288, Communications Research Laboratory, McMaster University, Canada, June 1994

Jeffrey Smith, Richard Bruno, Vince Fumo Inspection of safety-critical software using program-function tables..

J. Dennis Lawrence, Warren L. Persons, and G. Gary Preckshot (Lawrence Livermore National Laboratory) Evaluating Software for Safety Systems in Nuclear Power Plants

Canada Nuclear