Collins Advanced Technology Center / Iowa State University / SEC

PM/PI: David Statezni, co-PI: Gary Daugherty, Rockwell CollinsSubcontract: Dr. Suresh Kothari, Iowa State University

Contract: F33615-00-C-1624

Advanced Computing Systems

Slide 2

Pattern driven analysis & adaptation of HA SW

Problem Description & Program Objectives

Technical Approach

Schedule

Milestones in next 6 months

Technology transition

Slide 3

Project web site

This presentation and other information is available from:

http://dirac.ee.iastate.edu/sec/index.htm

User Name: sec

Password: skgd5103

Slide 4

Problem Description*

Large body of complex software (Boeing OCP)

High assurance application (SEC)

Software is critical (faults may result in mission failure, loss of aircraft, equivalent to DO-178B level A)

Concern over a number of issues (83 in Issues List)

Complete manual review impractical for small team, esp. with respect to global issues

Testing is expensive, and does not cover all cases

Slide 5

Program Objectives*

Cost effective, automated approach (driven by static analysis)

Analyze/adapt OCP to establish confidence, eliminate problems, avoid potential faults in flight

Address the most important issues

Flexible with respect to definition of problems, specification of changes, ability to keep developer in the loop - based on user specified “Software Patterns”

Support for multiple modeling and programming languages (to amortize costs, address a broader user base)

Test cases generated from interface specs to meet MC/DC coverage criteria – for changed modules, "high risk” modules

Slide 6

Technical Approach

Catalog

of patterns for high assurance software

KCStoolset

(for translation, analysis, and adaptation)

Modeling tools

(GME, Rose, etc.)

OCP and other software source (in C++, Java, other languages)

HA SW

XML/XMI models

Patterns in XML/XMI

Adapted OCP, and other software source (in C++, Java, other languages.)

Models

SW

HA Models

Adapted XML/XMI models

Developer

Review/editIssues List

T-VEC

Regression testing

T-VEC formal specs

Slide 7

Issues List: Complete

Compiled from a variety of sources

Identifies 83 issues related to software for high assurance systems

Includes both design level and language level issues

And issues related to verification and testing List available from the Issues page of our web site

http://dirac.ee.iastate.edu/sec/HA-SW-Issues.htm

Slide 8

Issues List: Sources*

Compiled from a wide variety of sources, including: Handbook for Object-Oriented Technology in Aviation (OOTiA),

Proceedings of FAA/NASA Workshops 1 & 2 on Object-Oriented Technology in Aviation, 2003

Guide to the use of the Ada Programming Language in high integrity systems, ISO/IEC TR 15942, 2000

John Barnes. High Integrity Ada: The SPARK Approach, Addison-Wesley, Harlow, England, ISBN 0-201-17517-7, 1997

Use of the C++ Programming Language, Certification Authorities Software Team (CAST), position paper, CAST-8, January, 2002

David W. Binkley. C++ in Safety Critical Systems, NIST, November 1995

Boeing, BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801, 2002

Slide 9

Bertrand Meyer. “Applying design by contract.” IEEE Computer 25(10):40-51, October 1992.

Barbara Liskov and Jeanette Wing. “A Behavioral Notion of Subtyping”, ACM Transactions on Programming Languages and Systems, 16(6): 1811-1841, November 1994.

Scott Meyers. Effective C++, 2nd edition, Addison-Wesley, Reading, MA, 1998

Scott Meyers. More Effective C++, Addison-Wesley, Reading, MA, 1996

Marshall P. Cline and Greg A. Lamow. C++ FAQs, Addison-Wesley, Reading, MA, 1995.

Jeff Offutt, et al. A Fault Model for Subtype Inheritance and Polymorphism, Twelfth IEEE International Symposium on Software Reliability Engineering (ISSEE ’01), 2001

Robert V. Binder. Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley, Reading, MA, 2000.

Issues List: Sources*

Slide 10

XCIL: Complete*

eXtensible Common Intermediate Language (XCIL) Target language independent XML representation

Based on UML, with actions defined by Microsoft Intermediate Language (MS-IL) and the Java Virtual Machine (JVM)

Extended to include full C++

Precise semantics

Supports UML models and all .NET and JVM languages

Basis for integration with other tools

See http://dirac.ee.iastate.edu/sec/XCIL.htm

Slide 11

XPSL: Complete*

eXtensible Pattern Specification Language (XPSL) Patterns described in GOF-like format

But formal (XPSL) – in order to drive underlying tools

An extension of Aspect-Oriented concepts

But at the level of actions, with support for transformations (in addition to insert/replace)

And semantically defined pointcuts identified by analysis

Anticipates OMG Query/View/Transformation standard, which we plan to use as a foundation

See http://dirac.ee.iastate.edu/sec/XPSL.htm

Slide 12

Pattern catalogs: Complete

40 patterns (10 abstract, 30 concrete)

Addressing 26 of the 83 issues

Build on FAA/NASA, AVSI patterns defined by RC, Boeing, Honeywell, Goodrich and others

Few other high assurance software patterns available

Developed incrementally

Most important issues first

See http://dirac.ee.iastate.edu/sec/PatternCatalogs.htm

Slide 13

KCS toolset: To date

C++ to XCIL translator complete

Entire OCP (version 2.2) translated to XCIL

Pattern analysis/adaptation is also being developed incrementally

Initial focus on finding problems

Analysis support for 8 patterns, addressing 10 issues

Working on adaptation framework for two of the patterns

With more to follow, based on ranking of problems

Slide 14

Ranking of problems*

Hard run time error A fault may occur when the erroneous code is executed

Lurking run time error A hard run time error may result if additional correct code is

introduced

Obstacle to analysis (potential masking of existing errors) The code cannot be analyzed in some manner typically required

for “certification”

Maintenance risk (potential source of future errors) The code is correct, but subsequent changes are likely to introduce

errors

Slide 15

Analysis results: To date*

For the 8 patterns and 10 issues addressed so far: Matches on 9 of the 10 issues were found when the patterns

were applied to the OCP

Typically several hundred matches per issue

Problems range from maintenance risks to lurking and potential run time errors

A few of these are discussed in the following slides

Summary given by OCP Results link on home page of web site http://dirac.ee.iastate.edu/sec/HomePage.htm

Detailed results (with mapping to source code) given on OCP Results page

Slide 16

Pattern catalog

Increment one

Development of eXtensible Pattern Specification Language (XPSL/XML)

Development of patterns for:

Control flow,

Data integrity,

Pointer integrity,

Unintended side effects,

Multiple inheritance

Slide 17

Tool support

Increment one

Development of XCIL

Translation of C++ to XCIL,

Analysis framework (queries and views),

Analysis support for selected increment one patterns,

Focus on “local” patterns, affecting individual OCP components

Slide 18

Pattern: Safe Type Conversion

Addresses issue 4: Implicit type conversion Implicit type conversion may involve a potential loss of data or

precision, or semantics [OOTiA Handbook]

“Type conversions shall not be made from a longer to a shorter (number of bits) type. Type conversion shall not be made from an unsigned type to a signed type of the same size” - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801

Matches

No implicit conversions between integer and float

58,675 implicit conversions of a larger value to a smaller value

352 implicit conversions of a signed value to an unsigned value

87 implicit conversions of an unsigned value to a signed value

Matches represent potential run time errors

See OCP Results and Increment One Patterns for details

Slide 19

Adaptations

Subject to review

Need report on types converted from/to

Need report on writes to variables involved in conversions

Need to consider whether the actual range of values of the source type fits within the range of values permitted by the destination type

Need to consider whether the converted values are treated as numeric values (whether signed/unsigned matters)

Introduce user defined types or typedefs

Use bool instead of integer

Use single size, avoid conversions

Use explicit conversions

Introduce run-time range checks (not performed by C++)

Slide 20

Pattern: Compiler generated operations

Addresses issue 25: Compiler generated operations Compiler generated operations have default meanings that may be

inappropriate for a given class

Analysis results 1,451 classes with compiler generated default constructors

2,078 classes with compiler generated copy constructors

2,100 classes with compiler generated “=” (assignment) operators

2,218 classes with compiler generated “&” (address of) operators

Matches are subject to application of the pattern’s rules

Violations of these rules represent likely run time errors and lurking run time errors

Slide 21

Pattern: Compiler generated operations

Rules Law of The Big Three: “If a class needs a destructor, or a copy

constructor, or an assignment operator, it needs them all” … “During code reviews and debugging sessions, we have traced many core dumps back to violations of The Law” [Cline and Lomow, C++ FAQs]

“Declare a copy constructor and an assignment operator for classes with dynamically allocated memory” [Scott Meyers, Effective C++, 2nd edition, item 11]

Declare a constructor for all classes with an user defined invariant [OOTiA Handbook]

All fields without initial values must be set by a user defined constructor [OOTiA Handbook]

See OCP Results for details, associated adaptations

Slide 22

Adaptations

The user is responsible for introducing the necessary operations, while the tool is responsible for checking the user supplied code

In addition the tool may introduce run-times checks of constructor postconditions, to ensure the class invariant is established

Checks of user supplied constructor

Run time check of invariant as postcondition

Check that all fields are initialized

Checks of user supplied copy constructor

Run time check of invariant as postcondition

Check that all dynamically allocated components are replicated (deep copy)

Check of user supplied assignment operator

Check that all dynamically allocated components are replicated (deep copy)

Check of user supplied destructor

Check that all dynamically allocated components are deallocated

Slide 23

Pattern: No pointer arithmetic

Addresses issue 30: Use of pointer arithmetic “Operational software shall not include pointer arithmetic. In general, there

is little justification for going to the effort of designing code to do pointer arithmetic. The performance gained by doing pointer arithmetic is seldom worth the increased risk of error.” - Explicit pointer value calculation is prohibited for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801

Matches

470 instances of pointer arithmetic

Matches represent potential run time errors and obstacles to analysis

Rules A pointer can be used only as a substitute for an opaque reference

See OCP Results and Increment One Patterns for details/adaptations

Slide 24

Adaptations

Replace each array element pointer with an array base pointer and index, performing all arithmetic on indices rather than pointer values

Selectively introduce run-time bounds checks (typically not performed by C++ compilers)

Restrict other uses of pointers in XCIL to the cases given (in which the use of a pointer is compatible with the use of an opaque reference), requiring the user to make associated changes

Slide 25

Pattern: Goto-free execution

Addresses issue 19: Use of goto’s “The use of goto is regarded as both unnecessary, and undesirable

compared to the use of other control structures. It should not appear in well written code.” – SPARK

The use of goto should be excluded because it complicates flow analysis (FA), symbolic analysis (SA), timing analysis (TA), and structural testing (ST). - Guide for the Use of the Ada Programming Language in High Integrity Systems, ISO/IEC TR 15942

Matches 1,744 gotos

Matches represent obstacles to analysis

See OCP Results and Increment One Patterns for details/adaptations

Slide 26

Adaptations

Subject to review

Unless used as a substitute for exceptions, refactor code to:

Replace goto with structured control flow (conditional, loop, break, continue, error return, other common forms)

If used as a substitute for exceptions, apply exception/error handling patterns:

Enforcing the disciplined use of exceptions (per Bertrand Meyer)

Ensuring all reported errors (thrown exceptions) have handlers – as in Java

Eliminating macros in favor of pattern substitution, allowing a variety of error reporting/error handling strategies

Slide 27

Pattern: Fall through switch case

Addresses issue 17: Break in switch “Each case in a switch, which contains any executable code, shall be

terminated with a break. This eliminates the dubious practice of falling through a case to use code in a succeeding case for common processing. It is too easy to lose track of the correct program flow in code maintenance.” Mandatory for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801

Matches 461 fall through cases

Matches represent maintenance risks

See OCP Results and Increment One Patterns for details/adaptations

Slide 28

Adaptations

Refactor code to either:

introduce a method called by both cases (when the shared code is cohesive)

introduce an inlined call (when the shared code is short and cohesive)

introduce a conditional (if statement or nested case) when the shared code is not cohesive

introduce replicated code (when the shared code is short but not cohesive)

Slide 29

Other increment one patterns

Non-discriminated union (Issue 47)

Implicit method return (Issue 56)

Loop termination (Issue 44)

Enforce invariant (Issue 13)

Closed argument list (Issue 15)

No dangling pointers to local variables

No assignment statement arguments (Issue 40)

No user thrown exceptions (Issue 49)

Top level handler for predefined exceptions (Issue 49)

Slide 30

Pattern catalog

Increment two

Development of patterns for:

Contracts and subtyping,

Coupling and cohesion,

OO metrics,

Dynamic resource allocation,

Safety critical language subsets

Slide 31

Tool support

Increment two

Adaptation framework (transformations),

Adaptation support for selected increment one patterns,

Analysis support for selected increment two patterns,

Focus on “global” patterns, requiring an analysis of the entire OCP

Slide 32

Pattern: Formal interfaces*

Addresses Issue 1: Lack of precise requirements/interface specifications “If we don’t state what a module should do, there is little likelihood that it will

do it (The law of excluded miracles)” – Bertrand Meyer

Programming by contract: “The relationship between a class and its clients [can be] viewed as a formal agreement [contract]. Only through such a precise definition of every module’s claims and responsibilities can we hope to attain a significant degree of trust in large software systems” [Bertrand Meyer, Object-oriented Software Construction]

MC/DC test coverage criteria [DO-178B], Unit level subtyping tests

Matches All class interfaces

See Increment Two Patterns for details, discussion of generation of pre/post from code

Slide 33

Pattern: Subtyping*

Addresses Issue 6: Supertype/subtype compatibility Issue 23: Use of dynamic dispatch Issue 5: Dynamic dispatch during object construction

In particular “The single largest source of design errors is improper inheritance” [C+

+ FAQs, Issue 119]

Errors may result from Faulty intuition Failure to implement to inherited specifications Misuse of inheritance to share code/data Programmer specified optimizations Accidental overriding/hiding Missing override

Slide 34

Pattern: Subtyping*

Analysis results 938 subtyping relationships

1,392 cases in which an operation is implemented by multiple methods (OMM pointcut)

3,112 instances of “true dynamic dispatch” (TDD pointcut)

Review results (sample of ~40 subtyping relationships) 24 violations of subtyping rules/LSP (including conflicting postconditions,

less visible subclass operations, disabled methods, null method implementations, and weaker postconditions) - overlapping

Violations represent either lurking or hard run time errors

See Increment Two Patterns and OCP Subtyping Relationships for details/adaptations

Slide 35

Pattern: Multiple implementation inheritance*

Addresses issue 8: Ambiguity resulting from the use of multiple implementation inheritance

Multiple inheritance session of OOTiA workshop unattended because no one thought MII appropriate for high assurance applications

Review results (sample of ~40 subtyping relationships)

4 instances of multiple implementation inheritance

Matches represent maintenance risks and potential run time errors

Rules Replace with delegation for DO-178B levels A through C

See OCP Results and Increment One Patterns for details/adaptations

Slide 36

Other increment two patterns*

Safety critical language subsets (“Safe” C++?) (Issue 71)

Synchronization (Issue 3)

Multiple interface inheritance (Issue 7)

Replacement of multiple implementation inheritance with delegation (Issue 8)

No dynamic allocation after initialization (Issue 9)

Initialization before use (Issue 12)

Class coupling (Issues 13, 67, 68)

Dynamic dispatch test coverage

Method extension

Deep hierarchy

Slide 37

Regression/subtype testing*

Formal interface specifications (pre/post) are being developed for all classes we expect to change

And for all classes involved in subtype relationships that require LSP compliance testing

Test cases are generated from these formal interface specs by T-VEC

This approach helps us address issue 1: Lack of precise requirements/interface specifications

As well as providing a basis for regression testing

With MC/DC coverage

Slide 38

Lessons learned*

Tool support has been invaluable in identifying potential problems Much more effective than reviews in terms of hours spent,

accuracy, and counts of problems found

Valuable as a driver/assistant in the review process

Necessary due to size of OCP

Need to be able to interactively refine problem description, tune patterns

Slide 39

Schedule

Slide 40

Next Milestones*

Regression testing of 1st pass adapted software (pointer arithmetic, type conversion, etc.) 6/27/2003

Complete work on analysis and adaptation (strategy = most important first) 7/31/2003

Regression testing of 2nd pass adapted software 8/29/2003

Final Report 12/19/2003

Slide 41

Technology Transition*

Web site has latest versions of the Issues list, XCIL papers, XPSL papers, Pattern catalogs, and the OCP analysis results

Reports identifying problems found (by both reviews and the toolset) will be posted to the web site

Change log + T-VEC interface specs, test cases, and test results for changed modules will be provided

Revised OCP code will be posted when all regression testing is complete

The tools developed by ISU and the contents of the web site will be distributed on CD

The KCS tools will be made commercially available through Ensoft Corporation at ISU Research Park

Slide 42

Program Issues

Maximize benefit in time remaining Extended subcontract with ISU to end of July

Space and time required to run KCS analysis and adaptations Faster machine, parallelization, incremental

Use of the ISU toolset by others requires licensing of EDG parser (from Edison Software) University agreement with Edison (similar to ISU)

Licensing from Ensoft (as part of XCIL translator for C++)