Validation of Safety-Critical Systems with AADL
© 2006 Carnegie Mellon University
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213
Peter H Feiler
April 11, 2008
Outline
Multiple aspects of system validation
System & software engineers working together
Multi-fidelity model-based analysis
Property preserving transformations
Conclusions
2Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Conclusions
Dimensions of System Validation
The system
System modelsModel-based vof
Validation of models against system
3Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
System models
System implementation
Model-based vofsystem
Validation of implementation against
system models
Single Source Annotated Architecture Model
Predictive Analysis Across Engineering Dimensions
SecurityIntrusion
Integrity
Confidentiality
Availability & Reliability
MTBF
FMEA
Hazard analysis
Architecture Model
4Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
4
Real-timePerformance
Execution time/Deadline
Deadlock/starvation
Latency
ResourceConsumption
Bandwidth
CPU time
Power consumption
Data precision/accuracy
Temporal correctness
Confidence
Data Quality
Low incremental cost for additional analyses &
simulations
Fewer independently developed models
reduces model validation
Architecture-Driven Modeling
Automatically derived
analytical models
5Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Annotated architecture
System generation
from validated models Validation of generators
AADL and Safety-Criticality
Fault management
• Architecture patterns in AADL
— Redundancy, health monitoring, …
• Fault tolerant configurations & modes
Dependability
• Error Model Annex
• Specification of fault occurrence and fault propagation information
6Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
6
• Use for hazard and fault effect modeling
• Reliability & fault tree analysis
Behavior validation
• Behavior Annex
• Model checking
• Source code validation
Outline
Multiple aspects of system validation
System & software engineers working together
Multi-fidelity model-based analysis
Property preserving transformations
Conclusions
7Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Conclusions
Traditional Embedded System Engineering
System Engineer Control Engineer
System
Under
Control
Control
System
8Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Software-Intensive Embedded Systems
System Engineer Control Engineer
Ap
plic
atio
n D
eve
lop
er
Ha
rdw
are
En
gin
ee
r
System
Under
Control
Control
System
9Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Ap
plic
atio
n D
eve
lop
er
Ha
rdw
are
En
gin
ee
r
Compute
Platform
Runtime
Architecture
Application
Software
Embedded SW System Engineer
Mismatched Assumptions
System Engineer Control Engineer
Ap
plic
atio
n D
eve
lop
er
Ha
rdw
are
En
gin
ee
r
SystemUnder Control
ControlSystem
Physical Plant Characteristics
Data Stream Characteristics
Precision Units
10Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Ap
plic
atio
n D
eve
lop
er
Ha
rdw
are
En
gin
ee
r
ComputePlatform
RuntimeArchitecture
ApplicationSoftware
Embedded SW System Engineer
Characteristics
Concurrency Communication
Distribution Redundancy
Predictable Embedded System Engineering
Document the Runtime
ArchitectureNavigation
System
AirbagDeploymentParking
Assistance
EmissionManagement
CruiseControl
AntilockBrakingSystem
ElectronicFuel
Injection
System Analysis
• Schedulability
• Performance
• Reliability
• Fault Tolerance
• Dynamic Configurability
11Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Execution Platform
. . . . . . . . . .
Abstract, but Precise
Application Software
System Construction
• AADL Runtime System
• Application SoftwareIntegration
ExternalEnvironment
Working Together
Conceptual architecture
• UML-based component model
• Architecture views (DoDAF, IEEE1471)
• Platform independent model (PIM)
System engineering
• SysML as standardized UML profile
• Focus on system architecture and operational environment
12Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
12
Embedded software system engineering
• SAE AADL
• OMG MARTE profile based on AADL
• AADL as MARTE sub-profile
• Non-functional properties require deployment on platform
Data modeling
• UML, ASN,, …
Outline
Multiple aspects of system validation
System & software engineers working together
Multi-fidelity model-based analysis
Property preserving transformations
Conclusions
13Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Conclusions
Impact of Sampling Latency Jitter
Impact of Scheduler Choice on Controller Stability
• A. Cervin, Lund U., CCACSD 2006
Sampling jitter due execution time jitter and application-driven send/receive
14Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Latency Contributors
System Engineer Control Engineer
System
Under
Control
Control
System
Operational
Environment
15Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
• Processing latency
• Sampling latency
• Physical signal latency
ARINC 653 Partitions & Communication
Frame-delayed inter-partition communication
Timing semantics are insensitive to partition order
Partition A Partition B
16Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
t0 t1 t2
T4
T1T2
T3
Partition APartition B
Partition APartition B
T1T2
T3T4
T1
T2
T3
T4
Latency Impact of Partitions
Display Manager
Sensor Request for new page
New page content
Latency contribution:
17Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
FlightManager
FlightDirector
Page ContentManager
Latency contribution:
Partition period per partition hop
Lower bound on worst-case latency
Intended Data Flow in Task Architecture
Navigation Sensor
Processing
Integrated Navigation
20Hz
10Hz
From other Partitions
Decre
asin
g P
rio
rity
Periodic I/O
20Hz
To otherPartitions
Shared data area
Pr 1
Pr 2
Pr 3
18Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Navigation
Guidance Processing
Flight PlanProcessing
Aircraft Performance Calculation
20Hz
5Hz
2Hz
Decre
asin
g P
rio
rity
Pr 4
Pr 6
Pr 9Priority assignment achieves desired data
flow
Preemption & concurrency affect read/write order
Frame-level Latency Jitter of Data Stream
Example: Non-deterministic downsampling
• Desired sampling pattern 2X: n, n+2, n+4 (2,2,2,…)
• Worst-case sampling pattern: n, n+1, n+4 (1,3,…)
NavSensor Processing
Integrated Navigation
20Hz 10Hz
19Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Timeline
Thread NavSensorProcessing
Thread IntegratedNavigation
Processing Navigation
Write
Read
Managed Latency Jitter through Deterministic Sampling
Navigation Sensor
Processing
Integrated Navigation Guidance
Processing
20Hz
10Hz 20Hz
From
Partitions
To
Partitions
Guidance
Nav
sensor
data
Nav signal
data
Nav
dataNav sensor
Periodic I/O
20Hz
20Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Processing
Flight PlanProcessing
Aircraft Performance Calculation
5Hz
2Hz
Fuel Flow
FP data
Performance
data
data
Nav data
FP data
Immediate and delayed data port connections for
deterministic sampling
Input-compute-outputAADL thread semantics
Rate Group Optimization
Logical threads to execute at a specific rate
Multiple logical threads to execute with the same rate
Placement of units with same rate in same operating
system thread
Reduced number of threads and context switches
21Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Reduced number of threads and context switches
Rate Group Order Can Affect Latency
Data flow from sensor Ts to control Tc to actuator Ta with mid-
frame communication
Effect of rate groups: Tc to Ta becomes delayed
Occurs when pairwise immediate connections in opposite
direction
22Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
t0 t50 t100
Ts Ta
Tc
OST 50ms
OST 100ms
Ts Ta
Ts
Ta
OS Thread 50ms
Tc
OS Thread 100ms
Software-Based Latency Contributors
Execution time variation: algorithm, use of cache
Processor speed
Resource contention
Preemption
Legacy & shared variable communication
23Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Legacy & shared variable communication
Rate group optimization
Protocol specific communication delay
Partitioned architecture
Migration of functionality
Fault tolerance strategy
Latency and Age of Data
Latency: the amount of time between a sensor reading and an output to an actuator based on the sensor reading
Age: amount of time that has passed since the sensor reading
Age Contributors
• Oversampling
24Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
• Missing sensor readings
• Failed processing
• Missed deadlines
Outline
Multiple aspects of system validation
System & software engineers working together
Multi-fidelity model-based analysis
Property preserving transformations
Conclusions
25Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Conclusions
Efficient Runtime System Generation
Navigation Sensor
Processing
Integrated Navigation Guidance
Processing
20Hz
10Hz 20Hz
From
Partitions
To
Partitions
Guidance
Nav
sensor
data
Nav signal
data
Nav
dataNav sensor
Periodic I/O
20Hz
Preserve timing semantics of execution
and communication
26Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Processing
Flight PlanProcessing
Aircraft Performance Calculation
5Hz
2Hz
Fuel Flow
FP data
Performance
data
data
Nav data
FP data
Immediate and delayed data port connections for
deterministic sampling
Input-compute-outputAADL thread semantics
Will This Implementation Work?
Navigation Sensor
Processing
Integrated Navigation
20Hz
10Hz
From other Partitions
Periodic I/O
20Hz
To otherPartitionsBuffer
Variable
Pr 1
Pr 2
Pr 3Buffer
VariableBuffer Variable
Buffer
27Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Navigation
Guidance Processing
Flight PlanProcessing
Aircraft Performance Calculation
20Hz
5Hz
2Hz
Pr 4
Pr 6
Pr 9
Buffer Variable
Buffer VariableSimulink: single variable
per connection
Overlapping Message Lifespan
Periodic thread MP and MC
MP ->> MC
Need for double buffering
MP MP
28Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MPi
MCi
MPi+1
MCi-1
Optimization of General Port Buffer Model
MPj
Producer
Send
Xfer
MPk
Consumer/Producer
MCj
Receive Send
Consumer
MCk
Xfer
Receive
29Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MSj MRj
Xfer
MSk MRk
Xfer
MP: producer copy
MS: send copy
MR: receive copy
MC: consumer copy
• Send/receive with or without copy
• Transfer with or without copy
• Processing with or without copy
Message Streaming Lifespan Framework
MSi
MPiProducer task
Send
Xfer
MPi+1
Send buffer MSi+1
TP, Mi+1DP, Mi+1
SMi
XMi
B E
30Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MRi
Xfer
MCi
Receive
Consumer task MCi+1
MRi+1Receive buffer
TC, MiDC, Mi
RMi
XMi
TX
Message Lifespan Properties
MC input-compute-output guarantee
TC, Mi≤ RMi
= BMCi≤ EMCi
≤≤≤≤ TC, Mi+1≤ Rmi+1
Message operation ordering condition
SMi< XMi
< RMi
MP bounded by producer dispatches
31Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MP bounded by producer dispatches
TP, Mi≤ BMPi
≤ EMPi= SMi
≤ TP, Mi+1
MS bounded by sends and transfer
SMi= BMSi
≤ X*Mi
≤ EMSi< SMi+1
MR bounded by transfers and receive
X**Mi
≤ BMRi≤ EMRi
= R***Mi
< XMi+1 * Completion of transfer
** Start of transfer
*** Latest of multiple receivers
Sequential Execution of Periodic Tasks
(τ P ; τ C )*
Collapse to single buffer
MPi MPi+1
32Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MS
MR
MCi
MS
MR
MCi-1
Application-based Send and Receive (ASR)
MP
MR
αP ΩPS&X
TP ≤ αP ≤ S ≤ ΩP ≤ DP
(ττττ P | ττττ C )*
3 buffers
33Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MR
MC
αC ΩCR
αP - ΩP ∩ αC - ΩC ≠ ∅ ⇒ non-deterministic S/R order
TC ≤ αC ≤ R ≤ ΩC≤ DC
α : actual execution start time
Ω : actual completion time
Dispatch-based Send and Receive (DSR)
MP
MR
αP ΩPS&XTP ≤ αP ≤ S ≤ ΩP ≤ DP
DP ≤ R ≤ TC
(ττττ P | ττττ C )*
2 buffersMP
34Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
MR
MC
R
αP - ΩP ∩ DP - TC = ∅ ⇒ deterministic S/R
α : actual execution start time
Ω : actual completion time
MC
Buffer Optimization Considerations
Send and receive execution
• As part of application (ASR)
• As part of task dispatch/completion (DSR)
Task execution order
• Concurrent: τ | τ
Periodic & aperiodic task dispatch
35Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
• Concurrent: τC | τP
• Atomic non-deterministic: τC ≠ τP
• Ordered: τC ; τP or τP ; τC
Message transfer
• Immediate to consumer (IMT)
• Direct to delayed consumer (DMT)
• Period-delayed to consumer (PMT)
Periodic Task Communication Summary
Periodic Same period
ASRIMT | PMT
DSRIMT | PMT
DMT
τP ; τCMF:1B PD:2B
S∨X∨R
PD:2B
R
PD:2B
S∨X/R
MF:1B
τC ; τPPD:1B PD:1B PD:1B PD:1B PD:1B
τ ≠ τ ND:1B PD:2B PD:2B PD:2B ND:1B
36Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
τP ≠ τCND:1B PD:2B
X
PD:2B
R
PD:2B
X/R
ND:1B
τP | τCND:3B
S/XC
RC
PD:2B
X
PD:2B
R
PD:2B
X/R
NDI:2B
S/X/RC
1B: Single buffer
2B: Two buffers
3B: Three buffers
4B: Four buffers
S, X, R : data copy
S/X : IMT combined send/xfer
S/X/R : DMT combined S, X, R
X/R: DSR/PMT combined X, R
o1∨∨∨∨o2 : One operation copy
MF: Mid-Frame
PD: Period Delay
ND: Non-Deterministic
NDI: No Data Integrity
Outline
Multiple aspects of system validation
System & software engineers working together
Multi-fidelity model-based analysis
Property preserving transformations
Conclusions
37Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Conclusions
Predictable Model-based Engineering
Reduce the risks
• Analyze system early and throughout life cycle
• Understand system wide impact
• Validate assumptions across system
Increase the confidence
• Validate models to complement integration testing
38Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
• Validate models to complement integration testing
• Validate model assumptions in operational system
• Evolve system models in increasing fidelity
Reduce the cost
• Fewer system integration problems
• Fewer validation steps through use of validated generators
Software
System
Design
System
Test
Acceptance
Test
Requirements
Engineering
Traditional Development Model
39Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Software
Architectural
Design
Component
Software
Design
Code
Development
Unit
Test
Integration
Test
Software
System
Design
System
Test
Acceptance
Test
Top-Level
Verification Items
High-level
AADL Model
Detailed
Low fidelity
Adequate confidence
High fidelity
Strong confidence
Requirements
Engineering
Virtual System Integration
Benefits of Predictive Architecting
40Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Software
Architectural
Design
Component
Software
Design
Code
Development
Unit
Test
Integration
Test
Detailed
AADL Model
Specify Model-
Code Interfaces
→ generation of test cases
← updating models with actual data
Industrial Embedded Systems Initiatives
SAE AADLStandardNov 2004
Automotive
OSATEToolset
SEIAADL Meta
Avionics
MBE
TOPCASEDOpen Source EmbeddedSystems Tool Framework
28 partners €20+M 2005-2008
ITEA SPICESModel-Driven Embedded
EAST ADLConsortium
AutoSAROpenGroup
Real-Time ForumEU + US partners
COTREAviation Systems
2002-2004
41Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
SEIAADL Meta Model & XMIJune 2006
AADL Error Annex Standard
June 2006
AADL UML Profile Std
2008
Aerospace
Model-Driven Embedded Systems Engineering
15 partners €16M 2006-2009
US AVSI Avionics ConsortiumAnalysis-based System Validation
12+ partners $40+M 2008-2011
EC ASSERTProof-based Satellite
Architectures ESA + 30 partners€15M 2004-2007
IST ARTIST2Embedded SystemsCenter of Excellence
2007-2011
IST ARTISTEmbedded Systems
2001-2006
ESA SatelliteArchitectures
2002-2004
A Research Transition Platform
MetaHVestal
Honeywell
RMALehoczky
Klein
SimplexDependable
UpgradeSha
QRAMRajkumar
EDCS
INSERT/SimplexSha Lehoczky
Klein Feiler
EDCS
RTQTLehoczkyDASADA
TimeWeaverRajkumar
MoBIES
Sporadic serverRTQTKleinPACC
Dynamic QRAMRajkumar Feiler
DASADA
Predictable CachingIn Embedded SystemsFeiler Hansson DeNiz
AADL LatencyARINC653
Feiler Hansson
QRAM/RMAFeiler
ConfigurationConsistency
Krogh Feiler Li
EDCS
MetaH/AcmeFeiler
AMRDEC
Alloy-basedArchitecture Verification
DeNiz Garlan
SEI SCS
OSATEBinpacker RMA
ARINC653Feiler DeNiz
AlloyVerification
Jackson (MIT)
Model Validation
Resource Management
PartitionedArchitectures
MBE
42Safety-Critical Systems & AADLFeiler, April 2008
© 2008 Carnegie Mellon University
Formalized Execution SemanticsRolland
IRIT
Runtime SystemCode Verification
Verimag/IRIT
ReliabilityFault Tree
Vestal
Honeywell
Honeywell
Runtime SystemGeneration & Verification
AADL/PetriNetENST (Paris)
System Fault ImpactFeiler Sha
SEI UIUCRuntime System
VerificationHybrid Automata
Vestal
Honeywell
Process algebraACSR
Sokolsky (U.Penn)
ReliabilityAnalysis
GSPNLAAS
Reliability ModelingMobiusUIUC
ReliabilityAnalysisMarkov
Embry-Riddle
Sensornet Resources
ANDESStankovic Son
UVA
Fault PropagationFPTC
Wallace (York U.)
Formalized AADL Temporal
SemanticsIRIT (Toulouse)
ResourceScheduling
Singhoff (Brest)
NetworkCalculusVestal
Honeywell
ConfidentialityIn AADL
Feiler HanssonSEI IR&D
WirelessSecurity
ISIS Vanderbilt
Dependability
Security
Slack Stealer In MetaH
Vestal Binns
Honeywell Aging in AsynchronousArchitectures
Vestal Honeywell
SAE AADLStandardNov 2004
OSATEToolset
SEI
AADL Error Annex Standard
June 2006
Runtime SystemGeneration Verification