August 20, 2008 FAA Software & AEH Conference 1
SAE S18 & EUROCAE WG-63
Presented bySteve Beland
Associate Technical FellowBoeing Commercial Airplanes
Assignment of Assignment of Design Assurance LevelsDesign Assurance Levels
August 20, 2008 FAA Software & AEH Conference 2
Status of New Draft GuidanceSAE S-18 Airplane Safety Assessment Committee updating SAE ARP 4754
Subcommittee: Fellowship of the DAL (FotDAL)
Regular coordination with EUROCAE WG-63Mature text inserted as Sec 5.2 in Draft G2, S18 meeting Jan ’08
Delicate consensus, WG-63 participated remotelyEstablished baseline
Draft H2 contains April joint meeting effortFlowchart addedConsensus strengthened, FotDAL done
August 20, 2008 FAA Software & AEH Conference 3
FotDAL Problem StatementInitial DAL has not always been based on rigorous safety analysisDelineation of the architectural containment boundaries were not always properly definedInconsistency between guidance sourcesDifficult to delineate the subtleties between “independence” and “dissimilarity”Probabilities have often been improperly linked to development assurance levelsPresent ARP 4754 Section 5.4 does not clearly distinguish between loss vs. erroneous output of a function (but it is there “between the lines” of Table 4)
August 20, 2008 FAA Software & AEH Conference 4
SituationUse of ARP4754 is becoming more common
ARP4754 was published in 1996Referenced in AMC 25.1309, AC 23.1309-1C and draft 25.1309 (Arsenal) published in 2002
FAA Policy Memo allows ESF for Part 25Section 5.4 permits reduced level of process rigor for development assurance activities provided proper safety analysis shows architectural containmentSome perceived inconsistencies between ARP4754 and DO-178B and DO-254 exist (Ref. FAA Policy Statement ANM-10-117-09, Jan 2004)
August 20, 2008 FAA Software & AEH Conference 5
Related Changes to ARP4754Revision A restructured to include both the aircraft and system development life cycle process modelsDocument outline has a smoother flow Sections 4 through 10 combined into 2 sections
Section 4 outlines the aircraft/system development process modelSection 5 addresses the integral processes used during aircraft/system development
New Section 5.2 is influenced by existing ARP 4754 Table 4 and the Functional Failure Path Analysis concept in RTCA DO-254 Appendix B
Includes constraints to preclude extreme FFPA interpretations
August 20, 2008 FAA Software & AEH Conference 6
Development vs. Design AssuranceDevelopment Assurance:
All of those planned and systematic actions used to substantiate, at an adequate level of confidence, that errors in requirements, design and implementation have been identified and corrected such that the system satisfies the applicable certification basis. Note: The development assurance of an item has sometimes been called design assurance, such as in RTCA/EUROCAE DO-254/ED-80.
Development Assurance Level (DvAL)Applies to aircraft and systemsV&V Process defined in SAE ARP4754AIncludes Safety AnalysisInfluences Architectures
August 20, 2008 FAA Software & AEH Conference 7
Development vs. Design AssuranceDesign Assurance:
All of those planned and systematic actions used to substantiate, at an adequate level of confidence, that design errors have been identified and corrected such that the items (hardware, software) satisfy the applicable certification basis.
Design Assurance Level (DsAL) Applies to items (software / hardware)V&V Process defined in RTCA DO-178 & DO-254
August 20, 2008 FAA Software & AEH Conference 8
n Sy
stem
s in
Airp
lane
m It
ems
in n
Sys
tem
sA
irpla
ne S
yste
ms
Layered Development Life Cycle
Aircraft Function
DvAL
System Function
DvAL
Item DsAL
August 20, 2008 FAA Software & AEH Conference 9
Assurance Activities
Assurance ActivityAt system level ARP 4754/ED-79
At Hardware Level DO-254/ED-80
At Software Level DO-178B/ED-12B
FHA Yes None None
PSSA/SSA Yes None None
FMEA Yes Implied via Functional Failure Path Analysis.
None
Common Cause Analysis (including SW and complex HW common mode failures)
Yes None None
Requirement Capture Yes Yes YesRequirement Validation Yes Yes, for HW specific
requirementsYes, for SW specific
requirements
Implementation Verification
Yes Yes Yes
Configuration management
Yes Yes Yes
Process assurance Yes Yes Yes
August 20, 2008 FAA Software & AEH Conference 10
FotDAL Approach to Assigning LevelsExisting ARP explicitly addresses system level (with only some mention of airplane level)
Level assignment/reduction applied to items defined from the system architecture
New ARP addresses airplane & system levels explicitlyDvAL is effectively new for the new ARP and should be assigned to the systems from the aircraft architecture using the PASA/PSSADsAL assignment should be similar to existing ARP
Discusses “independence” rather than “dissimilarity”Emphasize assigning levels rather than reducing levels
“Reduction” is a misnomer, but arises when a function has a level lower than its parent function
August 20, 2008 FAA Software & AEH Conference 11
ARP4754A Draft Outline
4 AIRCRAFT & SYSTEM DEVELOPMENT PROCESS4.1 Conceptual Aircraft/System Development Process4.2 Aircraft Function Development4.3 Allocation of Aircraft Functions to Systems4.4 Development of System Architecture4.5 System Implementation
5 INTEGRAL PROCESSES5.1 Safety Assessment5.2 Development Assurance Level (DvAL) Assignment5.3 Requirements Capture5.4 Requirements Validation5.5 Implementation Verification5.6 Configuration Management5.7 Process Assurance5.8 Certification and Regulatory Authority Coordination
August 20, 2008 FAA Software & AEH Conference 12
System Development Process Model
CONCEPTDESIGN
FUNCTIONDEVELOPMENT
FUNCTIONALLOCATION
ARCHITECTUREDEVELOPMENT
REQUIREMENTALLOCATION
SYSTEMIMPLEMENTATION
PLANNING
- 5.1 SAFETY ASSESSMENT- 5.2 DEVELOPMENT ASSURANCE- 5.3 REQUIREMENTS CAPTURE- 5.4 REQUIREMENTS VALIDATION- 5.6 CONFIGURATION MANAGEMENT- 5.7 PROCESS ASSURANCE- 5.8 CERTIFICATION & REGULATORY GUIDANCE COORDINATION
AIRCRAFT/SYSTEMDEVELOPMENT PROCESS
4.1
4.2 4.3 4.4 4.5 4.6
3.0
DATA &DOCUMENTATION
6.0
5.5 IMPLEMENTATION VERIFICATION
INTEGRAL PROCESSES
Figure 4-2
August 20, 2008 FAA Software & AEH Conference 13
5.2 Assurance Level Assignment
5.2.1 Introduction 5.2.2 General Principles5.2.3 Independence Attributes5.2.4 DvAL & DsAL Assignment Guidelines
August 20, 2008 FAA Software & AEH Conference 15
TerminologyFunctional Failure Set:
A single member, or a specific group of members (not necessarily limited to one system) whose anomalous behavior (random failure or systematic error) leads to a top level Failure Condition. An FFS member can be related to a function, a sub-function, or an item.
August 20, 2008 FAA Software & AEH Conference 16
DvAL Assignment General PrinciplesCatastrophic Failure Condition:
At least 1 development process is Level A, or at least 2 independent development processes are Level B, but none any lower than their individual hazard and no lower than Level COverall integration process is Level A
Hazardous Failure Condition:At least 1 development process is Level B, or at least 2 independent development processes are Level C, but none any lower than their individual hazard and no lower than Level DOverall integration process is Level B
August 20, 2008 FAA Software & AEH Conference 17
Independence AttributesFunctional: members with different fcns & reqts
Common requirements errorsRequirements interpretation errors
Design: members with different designsHardware or software design errorsSoftware language or HDL errorsDesign tool errors
Others: do not influence DvAL/DsAL assignmentPhysical
Redundancy, installationProcess
Between independent designs or functionsBetween development/design vs. verif/validation
August 20, 2008 FAA Software & AEH Conference 18
Independence AttributesDvAL considers the functional independence of the aircraft (or system) functionsDsAL considers the design independence of itemsOnce the DsALs are assigned to items, they should be fed back to the system and aircraft processes to ensure that no common mode is inadvertently introduced that violates any claimed functional independence.The assertion of independence needs to be substantiated & address potential common-modes One type of independence does not necessarily imply the other
August 20, 2008 FAA Software & AEH Conference 19
DvAL/DsAL Assignment ProcessDevelopment Assurance Level (DvAL)
Aircraft and system levels assigned within Aircraft Level FHA & PASAValidated with Aircraft & System level Safety Analysis
Design Assurance Level (DsAL)Assigned from System Level FHAs & PSSAsValidated per System level Safety Analysis and Component Functional Failure AnalysisMust trace up to upper level functions’ DvAL so that it is not decomposed/assigned more than once (e.g. 4 Level D items assuring a Level A aircraft function).Non-complex items that are fully and deterministically tested and analyzed may be considered Level A
August 20, 2008 FAA Software & AEH Conference 20
Table 5-2: DvAL Assignments (paraphrased)
Two or more Independent FFS MembersOne FFS
Members Option 1 Option 2
CAT A A for 1 member, more members assigned per its failure cond (but ≥C).
B for 2 members, more members assigned per its failure cond (but ≥C).
HAZ B B for one member, more members assigned per its failure condition (≥D)
C for 2 members, more members assigned per its failure condition (≥D)
MAJ C C for one member, more members assigned per its failure condition
D for 2 members, more members more per its failure condition
MIN D D for one member, more members assigned per its failure condition
members assigned per its failure condition
None E E E
Top Event
August 20, 2008 FAA Software & AEH Conference 21
DEVELOPMENT ASSURANCE LEVEL
functional failure sets with multiple independent members(note 2)
functional failure sets with a single member or with non-independent members
Option 1 (NOTE 3)
Option 2
Catastrophic DvAL A(NOTE 1)
DvAL A for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level C for the additional members).
DvAL B for two of the members leading to top level failure condition. The other member(s) at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level C for the additional member(s)).
Hazardous DvAL B DvAL B for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level D for the additional members).
DvAL C for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level D for the additional members).
Major DvAL C DvAL C for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.
DvAL D for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.
Minor DvAL D DvAL D for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.
DvAL of each member at the level associated with the most severe individual effects of an error in their development process
No Safety Effect DvAL E DvAL E DvAL E
Top Event / Failure Condition
Classification
August 20, 2008 FAA Software & AEH Conference 22
Notes after DvAL/DsAL Table 1. When a FFS has a single member and the mitigation strategy for
systematic errors is to be DvAL A alone, then the applicant may be required to substantiate to the Certification Authorities that the applicant’s development assurance activities have the rigor to provide for an acceptable level of safety.
A single Level A member is OK, but may get more scrutiny2. When decomposing an aircraft function it is necessary to stay in the
same row no matter the number of functional decompositions performed (e.g. any degree of decomposition from a level A function should include at least one level A or 2 level B functions
3. If there is a large disparity on the numerical availability of the members in the functional failure set, the higher level DvAL should generally be assigned the higher availability
4. Some classes of 14CFR Part 23 /CS-23 aircraft will allow DvALslower than derived in this process. See the current FAA AC23.1309 and EASA policy for guidance.
August 20, 2008 FAA Software & AEH Conference 23
DsAL Assignment Cases: 0 No independence:
Assign DvAL & DsAL for top FC (DvAL = DsAL)1 Both functional & design independence:
Assign DvAL using Table 2, then DsAL using Table 2 staying in the same row as the top failure conditionWatch cross-products of one member’s DvAL and another member’s DsAL; re-assess assignment for FFS not meeting general principles
2 Functional but no design independence:DsAL of common items per top-level failure conditionEnsure partitioning ensure functional independence? May lead to re-evaluation of DvAL if not
3 Design but no functional independence:Assign DsAL using Table 2, staying in the same row as the top-level failure condition
August 20, 2008 FAA Software & AEH Conference 24
Level Assignment
Flow Fig 5-5
(1 of 2, DvAL)CAN
FUNCTIONAL INDEPENDENCE
BE CLAIMED? (5.2.4.1)
FAILURE CONDITIONS FROM
THE FHA
SELECT ONE (OR NEXT) FC AND ITS FFS
CAN DESIGN
ASSIGN DvAL = TOP DvAL PER COL 2 OF
TABLE 2
ASSIGN DvAL PER OPTION 1 OR 2 OF
TABLE 2
ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO
HAVE ALL FCs FOR
THIS FUNCTION BEEN
ASSESSED?
FUNCTION DvAL
ITEM DsAL
COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs.
NO
NO
YES
YES
ASSIGN DvAL TO AIRCRAFT-LEVEL FUNCTION (TOP DvAL) PER TABLE 1
Function DvAL
Item DsAL
August 20, 2008 FAA Software & AEH Conference 25
Level Assignment
Flow Fig 5-5
(2 of 2, DsAL)
CAN DESIGN
INDEPENDENCE BE CLAIMED?
(5.2.4.2)
ASSIGN ITEM DsAL = TOP DvAL (5.2.4.2.2)
ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO
DETERMINE DvAL; CHECK CROSS-PRODUCTS OF DvAL
& DsAL IF DsAL DIFFERS FROM ITS DvAL (5.2.4.2.1, .3)
THIS FUNCTION BEEN
ASSESSED?
HAVE ALL FCs FOR THIS
ITEM BEEN ASSESSED?
FUNCTION DvAL
ITEM DsAL
COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs.
FINAL DsAL MUST SATISFY ALL FCs (AND MAY NEED TO BE REASSIGNED FOR COMBINATION OF ALL APPLICABLE FCs)
NO
NO
NO
YES
YES
YES
Function DvAL
Item DsAL
August 20, 2008 FAA Software & AEH Conference 26
CA
N
FUN
CTI
ON
AL
IND
EP
EN
DE
NC
E
BE
CLA
IME
D?
(5.2
.4.1
)
FAIL
UR
E
CO
ND
ITIO
NS
FR
OM
TH
E F
HA
SEL
EC
T O
NE
(OR
NEX
T) F
C
AN
D IT
S F
FS
CA
N
DES
IGN
IN
DE
PE
ND
EN
CE
B
E C
LAIM
ED
? (5
.2.4
.2)
ASS
IGN
DvA
L =
TOP
D
vAL
PE
R C
OL
2 O
F TA
BLE
2
ASS
IGN
DvA
L P
ER
OP
TIO
N 1
OR
2 O
F TA
BLE
2
AS
SIG
N IT
EM
DsA
L =
TOP
DvA
L (5
.2.4
.2.2
)
ASS
IGN
DsA
L P
ER O
PTIO
N 1
O
R 2
OF
TAB
LE 2
US
ING
SA
ME
RO
W A
S U
SE
D T
O
DE
TER
MIN
E D
vAL;
CH
EC
K
CR
OS
S-P
RO
DU
CTS
OF
DvA
L &
DsA
L IF
DsA
L D
IFFE
RS
FR
OM
ITS
DvA
L (5
.2.4
.2.1
, .3)
HA
VE
ALL
FC
s FO
R
THIS
FU
NC
TIO
N
BE
EN
ASS
ESS
ED?
HA
VE
ALL
FC
s FO
R T
HIS
IT
EM
BE
EN
ASS
ESS
ED?
FUN
CTI
ON
DvA
L
ITE
M D
sAL
CO
MP
ILE
A L
IST
OF
ALL
FU
NC
TIO
NS
AN
D T
HEI
R D
vALs
FI
NA
L D
vAL
MU
ST
SAT
ISFY
ALL
APP
LIC
ABLE
FC
s.
FIN
AL
DsA
L M
US
T S
ATIS
FY A
LL F
Cs
(AN
D M
AY
NE
ED
TO
BE
RE
ASS
IGN
ED
FO
R
CO
MB
INA
TIO
N O
F A
LL A
PP
LIC
ABL
E F
Cs)
NO
NO
NO N
O
YE
S
YES
YES
YES
ASS
IGN
DvA
L TO
AIR
CR
AFT
-LE
VEL
FUN
CTI
ON
(TO
P D
vAL)
PE
R T
AB
LE 1
Whole Figure 5-5 For Your Handouts
August 20, 2008 FAA Software & AEH Conference 27
Special Cases:Non-complex items (e.g. mechanical parts, relays, electro-mechanical devices, electro valves, servo valves, simple logic devices, etc.) when the requirements that are applicable to the design have been validated and the analysis and testing of the design for verification with those requirements are considered to provide a level of confidence equivalent to DsAL A. However, this is only when the non complex item is fully tested or fully analyzed relative to the identified failure conditions. Independent functions using common resources based on the same COTS designs (e.g. computers, networks, interfaces) are likely to require careful consideration if assigned a DsAL of A. A practical case of functional and design independence is whereby independent functions are implemented in designs that are independent from one another.
August 20, 2008 FAA Software & AEH Conference 28
DvAL Considerations for Conditional Events
Arises for protection systems for environmental or intrinsic threats
e.g. windshear, fire
DvAL should be consistent with reduction in safety marginsDoes not apply to operations intentionally planned (e.g. autolands, ETOPS missions)
August 20, 2008 FAA Software & AEH Conference 29
DvAL Assignment as a Function of an External Event
A
B
C
10-910-710-510-3
DvAL
Probability of the external event
Legend: CAT Top Level Failure Condition HAZ Top Level Failure Condition
10-4 10-6
Figure 5-5
August 20, 2008 FAA Software & AEH Conference 30
Outputs of DvAL/DsAL AssignmentDescribe FFS members and explain their DvALs in Cert Plan and PASA/PSSADescribe items and explain their DsALs in applicable plans and PSSADescribe interactions between FFS members in PASA/PSSA
Functional interactionsSubstantiate claimed functional & design independenceCapture assumptions made in the supporting analysesDerive requirements needed to support assumptions and analyses.
Substantiate that all planned activities are successfully accomplished in the ASA/SSA and Cert Summary.
August 20, 2008 FAA Software & AEH Conference 31
CAST Comments on Draft HComments received are in-work by S18 & WG-63Some clarifications are neededA couple significant misunderstandings arising:
2 Level Bs don’t easily make a Level A functionEven if the item (DsAL) can be Level B, the system & aircraft functions still need to be developed & integrated at DvAL ANot intended to be endorsement of n-version programming
Concern of entering Table 5-2 more than once may lead to 4 Cs to meet Level A, etc.
Text disallows moving from row to row, each use must stay in the same row.
S18/WG-63 will disposition comments & reply
August 20, 2008 FAA Software & AEH Conference 32
More Difficult Than Algebra
First we had Algebra in math classThen Boolean Algebra in collegeIntroducing……..
August 20, 2008 FAA Software & AEH Conference 33
DALgebraA = A*C = B*B B = B = B*D = C*CSubstituting…….B*D*C*C = A?
August 20, 2008 FAA Software & AEH Conference 34
ClosingDvAL and DsAL are based on safety analysis; do it early (and revisit it often as the system evolves)!Emphasize assigning levels rather than reducinglevelsPASA and PSSA should be used to derive requirements including DvAL/DsALDevelopment/Design Assurance can be an enabler to focus resources on aspects that matter most. Aim is to have consistent guidance in one place for use across a systemThe DsAL assigned to software or hardware should be about the same as determined when using existing ARP 4754.