2003 NASA OSMA Software Assurance SymposiumSARP Initiative 583

The Use of a Virtual System Simulator & Executable

Specifications to Enhance SW Validation, Verification, and Safety

Assurance

TRIAKIS Corporation 31 July 2003

An Introductory BriefingBy Ted Bennett & Paul Wennberg

TRIAKIS Corporation 2

DefinitionsExecutable Specification (ES): Description of the dynamic behavior of a system or system element in an executable language, through the execution of which its behavior may be tested, validated & verified. The ES’ used in this project are bounded with virtual interfaces analogous to the real parts they specify.

Detailed Executable (DE): Virtual embedded control system element simulation running the unmodified object software developed for its real-world counterpart.

Virtual System Integration Laboratory (VSIL): Virtual environment wherein embedded system element executable specifications and detailed executables may be interconnected and tested for verification & validation purposes.

TRIAKIS Corporation 3

These Are the ProblemsMost embedded SW faults traceable to ambiguities & errors in system rqmts. [1,2,3] Poor comm. of rqmts. changes and poor comm. between teams during development also implicated as a major source of SW faults and significant schedule & budget overruns [1, et al]

Conventional fault injection-based testing limited by cost and schedule constraintsPresent methods of collecting dynamic SW metrics are intrusive - typically requiring instrumentation of operating system or target software itself

TRIAKIS Corporation 4

Project ObjectivesEvaluate viability & benefit of maintaining test consistency between VSIL using ES, & VSIL using DE running executable SW

Evaluate metric capabilities of VSILNew types of dynamic metrics,easier capture methodsReliability, accuracy benefit of noninvasive metric capture

TRIAKIS Corporation 5

Project Plan1. Create simplified simulation of Shuttle Robotic

Manipulator System (Robotic Arm)

2. VSIL simulation to comprise multiple ES’ and one

computerized subsystem developed into DE

3. Write test suite to V&V system design

4. Develop DE and control SW from target ES

5. Rerun all system tests with DE substituted for ES

6. Use VSIL to investigate metric objectives

TRIAKIS Corporation 6

ES’ in IcoSim VSIL

Hierarchical

Highly BoundedFirmly Anchored in Reality

Shuttle Computer

RMA Control Computer

Remote Manipulator Arm

RMS Power

RMA I/O e.g.:Analogs, Discretes,

Ethernet…

RMA Control Panel

Panel Power

Panel I/O e.g.:Analogs, Discretes, Serial Databus…

RMA PartsExample

TRIAKIS Corporation 7

RMA Control Computer ES

Central Maint Computer ES

RMAMaintenance Data

Remote Manipulator Arm ES Shoulder

Upper Arm Signal

Converter ES

Strain Gauge

CMC Output Data

RMA EthernetData bus

Test Sequence ExampleUpper_Arm_Strain_Gauge_1->

OpenFault();

Delay(0.15); // delay 150 ms

Maint_Computer->GetStatus(status_data, UA_SG_1);

Verify("UA SG 1 Fault Detected", status_data->fault, FAIL_OPEN);

ES/DE Test Consistency

Substitute DE for ESRun same test sequencesWhen DE SW passes tests,it correctly implementsfunctionality verified in ES

DE

MPC555Ethernet

Controller

RAM

ROM

RMAMaintenance Data

RMA EthernetData bus

RMA Control Computer

Unmodified Object SW

DirectSubstitution

TRIAKIS Corporation 8

Simulator Abstraction Charts

Simulink,MATLAB,Easy 5

Triakis IcoSim

VHDL/Spice Simulation

Ab

stra

ctio

n L

evel

Fidelity

EDA Simulation Tools

Simulator Tool Comparisons

Ab

stra

ctio

n L

evel

Weather, Threats, Airplane,

Landing Gear,Engines

Sensors, Drivers, Actuators,

Pumps,

A/D and D/A Converters,

Serial Interface Devices

Ports, Counters, Timers,

Interrupt ControllersCPU

Fidelity

Typical IcoSim Avionics VSIL

TRIAKIS Corporation 9

IcoSim Part CharacteristicsHighly modularBoundedHierarchicalRecursiveAbstract or DetailedSimple or ComplexDefinable intrinsic failure modes

PART

ES/DE

ES

DE

ESES

ES/DE

DEDE

ESES/DE

DE

Hierarchical yes, modular yes… but

definitely not a Venn diagram!

TRIAKIS Corporation 10

Potential ContributionsReduce interpretation induced SW faults due to ambiguities in system requirementsImprove ability for dynamic, noninvasive test of system & SW response to failure conditions

Known behavioral characteristics & failure modes of real part are intrinsic to virtual part and manifested under test control

TRIAKIS Corporation 11

Potential Contributions (Cont’d)

Reduce SW faults caused by breakdown in communication of system Rqmts changes

Systems Engineering &VSIL Development Team

SW Development Team 1

SW Development Team n

Integration/Test Team

Project Network File Server

•System design, ES, and DE changes verified in VSIL by Systems Engineering & VSIL Development team

•Updates to VSIL and all tests maintained under configuration control and distributed as they occur to all team members

TRIAKIS Corporation 12

Potential Contributions (Cont’d)

New capacity for empirical SW V&V in cases where analysis was only viable means

Realistic fault injection & failure mode testingComplex digital signal processor designs

Complete VSIL also provides useful tool for:

Post deployment command testingPost deployment SW change testingAnomaly/mishap analysis & problem solving

TRIAKIS Corporation 13

Potential Contributions (Cont’d)

Reduce project development costs & time by doing bulk of integration testing in VSIL during SW development phase

System, SW Dev & Test Integration TestingConventional Methods

System & VSIL Dev, SW Dev & Test in VSIL Env. Integ. tstIcoSim© VSIL

Project Effort Emphasis Comparison:Conventional Methods vs. IcoSim© VSIL

TRIAKIS Corporation 14

Project StatusProject started in AprilES-based simulator operationalSystem-level tests in development

Create DE and write control SWVerify control SW passes system testsCollect dynamic metrics

Next Steps

TRIAKIS Corporation 15

Quest

ions?

Please stop by Friday for a demo

TRIAKIS Corporation 16

References[1] Lutz, Robyn R. 1994. Analyzing Software Errors in Safety-

Critical, Embedded Systems. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA.

[2] Ellis, A. 1995. Achieving Safety in Complex Control Systems. Proceedings of the Safety-Critical Systems Symposium. pp. 2-14. Brighton, England. Springr-Verlag. ISBN 3-540-19922-5

[3] Leveson N. G. 1995. Safeware - System, Safety and Computers. Addison Wesley 1995. ISBN 0-201-11972-2