Security Architecture Cost/Benefit Model:
December 10, 2013
Mark O. Scott Technical Fellow for System Security
Architecture and Engineering
Layered Assurance Workshop
Assessment of Two Embedded System Approaches
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
2
• Why is this a significant topic?
• What is the scope of today’s presentation?
• How are the two approaches similar? How do they differ?
• What assumptions are built into the cost/benefit model?
• How will the costs and benefits be characterized?
• What results emerged from this model?
• How can these results be applied to current endeavors?
• What additional work can profitably be done in this area?
Security Architecture Cost/Benefit
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
3
A Precarious Position …
Dr. Roger R. Schell
From ACSAC 2001: Information Security: Science, Pseudoscience, and Flying Pigs
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
4
A Bold Assertion … Dr. Roger R. Schell
From ACSAC 2001: Information Security: Science, Pseudoscience, and Flying Pigs
“The greatest achievement in the science of computer and network security is the ability to build and deploy truly bulletproof systems having verifiable protection. And this remains the most powerful solution available for many of today’s hard problems.”
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
5
Bob Blakley, In Workshop on the Economics of Information Security (WEIS, 2002)
The Measure of Information Security is Dollars
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
6
Bob Blakley, In Workshop on the Economics of Information Security (WEIS, 2002)
The traditional approach to information security has failed.
The information security community does not currently have a viable technical alternative to the failed model.
The Measure of Information Security is Dollars
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
7
Bob Blakley, In Workshop on the Economics of Information Security (WEIS, 2002)
The traditional approach to information security has failed.
The information security community does not currently have a viable technical alternative to the failed model.
A. Information about both losses and product effectiveness is the prerequisite for the formation of a viable information security solution market.
B. The value of security solutions is impaired by customers’ inability to distinguish between effective and ineffective offerings.
C. The act of accepting liability for losses could be a powerful economic signal of a vendor’s belief in the effectiveness of its security solutions.
The Measure of Information Security is Dollars
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
8
7th Layered Assurance Workshop
LAW has provided a forum for vital exchange, as well as a maturing source of information, focused on
key issues relating to the effective and efficient modular construction and certification of assured
systems from assured components.
It is widely recognized that such an approach is the most promising way to achieve diverse and flex ible systems that can be certified quick ly
and cost effectively.
LAW is concerned with the theoretical, engineering, and certification challenges to be met before this
goal can be fully realized.
http://www.acsac.org/2013/workshops/law
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
9
The Objective of D-MILS is …
• To provide an environment for the design, analysis, verification, compositional implementation and certification of scalable, interoperable, and affordable trustworthy architectures.
Distributed MILS (D-MILS) Project
http://www.d-mils.org/page/overview
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
10
• Alternatives to “failed” information security solutions do exist – The science of information security is rich with solutions to solve the hard
problems – Ability to build and deploy truly bulletproof systems having verifiable
protection
• The market for these solutions must be built upon effective offerings
• What does the Layered Assurance Workshop bring to the equation?
– A forum focused on effective and efficient modular construction and certification of assured systems from assured components
• What about the Distributed MILS Project? – An environment for the implementation of affordable trustworthy architectures
Why is this a significant topic?
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
11
• Embedded System: Composed of Segments with Connections – Embedded Segment
– Operations Segment
– Support Segment
– Command and Control Center
– Mission Entities
– External Entities
• Embedded System: Security Architecture Requirements – Support Multiple Security Processing Domains (aka Security Enclaves)
– Support Cross-Domain interactions and data flows between Security Domains
• Embedded System: Security Architecture Approaches – System High / Multiple Single Levels of Security Processing Domains
– Multi-level Secure (MLS) Operating System throughout system (not the subject of analysis here)
– Multiple Independent Levels of Security (MILS) platform supporting MLS components
• Components of an emerging MILS Ecosystem featured in what follows: – MILS Separation Kernel (MSK)
– MILS Network System (MNS)
– MLS File System (MFS)
– MLS Console System (MCS)
– MILS SK-based Cross-Domain Solution (CDS) – Software- vs. Hardware-based design/implementation
Scope of This Presentation
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
12
Terminology and Diagram Schema
Hardware SoftwareNetwork
TerminologyBLP – Bell and La Padula (Security Model)CDS – Cross-Domain Solution (MLS Component)HMI – Human Machine Interface (Display Console, Keyboard, Mouse)HW – HardwareIDS – Intrusion Detection SystemMCS – MLS Console SystemMFS – MLS File SystemMILS – Multiple Independent Levels of SecurityMLS – Multi-Level Secure (see BLP Security Model)MNS – MILS Network SystemMSK – MILS Separation KernelMSLS – Multiple Single-Level Security (System)OS – Operating SystemPNI – Protected Network Interface (Boundary Security Functions, e.g., Firewall, IDS, Malware Protection)Security Classification – Security Level (Hierarchical) + Optional Compartment (aka Category or Caveat)SH – System High: Aggregate of security classifications of all data potentially in the SystemSK – Separation KernelSL – Single-Level (Processing, Network, Storage)SW – SoftwareSWaP – Size, Weight, and Power
Embedded System Diagram SchemaSecurity ClassificationEncrypted (Data in Transit or at Rest)S1 – Security Level 1 (e.g., Unclassified)S2 – Security Level 2 (e.g., Secret)S3A – Security Level 3, Compartment AS3B – Security Level 3, Compartment BSH Processing, Network, StorageMILS Processing, Network, StorageMLS Processing or Component
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
13
1. SH / MSLS Embedded System
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
14
2. MILS / MLS Embedded System
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
15
• Information security marketplace … – Values product and system assurance – Values product effectiveness, flexibility, and extensibility
• MILS (mostly future) Ecosystem Marketplace – MILS / MLS products are available, mature, and cost-effective – MILS / MLS products are interoperable and certified – Feature-rich MILS development tools are available – Robust MILS / MLS integration frameworks are available
• Cost/Benefit Model Maturity – Costs are relative and lack support of actuals at this time – Benefit Weighting Criteria are relative to approaches under analysis – Benefits – and hence the value of a solution – are difficult to quantify
Cost/Benefit Model Assumptions
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
16
Cost/Benefit Evaluation Criteria
Architecture Feature / Model Criteria SH / MSLS Architecture
MILS / MLS Architecture
Benefit Weighting
Criteria
Core Processing: SH or Single-Level Primary processing at SH Primary SL processing at S3B
Separation
SL SW Instances
Assurance
Certification
SWaP
Performance
Flexibility
Extensibility
Core Processing: Other Security Domains Multiple SL processing enclaves SL SW partitions on MILS SK
CDS: Platform: HW, OS, CDS Engine, Interfaces Two HW CDS platforms; MLS OS Two MILS SK platforms
Processing Backplane SL backplane per enclave Single backplane; MNS used
Network System SL network per enclave Single MILS Network System
Disk Storage SL disk per enclave Single MLS File System (MFS)
Disk Encryption SL encryption per enclave Single SH encryption of MFS
HMI / Displays & Controls SH console Single MLS Console System
Data Labeling As necessary for off-board CDS Not required
Complexity
Assurance
Certification
High-level CDS Rule Set SH-S3A and SH-S2 rule sets S3A-S2 and S3B-S2 rule sets
Mid-level CDS Rule Set S2-S1 rule sets S2-S1 rule sets (fewer rules)
CDS Protocol Support Larger rule set may inhibit Smaller rule set may enable
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
17
Cost/Benefit Model Parameters Parameter Parameter Values
Program Phase EMD, Production, Operations
Segment Embedded, Operations, System, Support, etc.
Architecture Feature / Model Criteria
Evaluation criteria used within the Cost/Benefit Model to compare the two architectural approaches. Consists of relevant components that include: Core Processing, CDS Platform, CDS Rule Sets, Development Activity, Network, Storage, Console, Process, etc.
Benefit Weighting Criteria Weighting factor (in Percent) applied to cost to designate a relative benefit. A single value based on multiple criteria: Separation, Single-level SW Instances, Assurance, Certification, SWaP, Performance, Flexibility, Extensibility, etc.
Number of Enclaves Number of Security Domains required in the Embedded Segment: 1, 2, 3, 4, 5, 6, 7
Architectural Approach SH / MSLS or MILS / MLS
Unit Qty Quantity of Component as incorporated into the Architectural Approach
Unit Cost ($M) Cost of component (Quantity 1) in $Millions
Cost ($M) Cost = (Unit Qty) X (Unit Cost) in $Millions
Benefit Weight A weighting factor (in percent) used to capture the relative benefit of a given component cost. See Benefit Weighting Criteria above.
Benefit ($M) Benefit = (Cost) X (Benefit Weight) in $Millions
Value (=B/C) Value = (Sum[Benefits]) / (Sum[Costs]) in percent. For a specified number of enclaves and a given architectural approach, Value is intended to quantity the benefits relative to costs at the overall program, program phase, and segment levels.
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
18
Cost/Benefit Model Structure Program Phase Benefit
Weighting Criteria
SH/MSLS Architecture: 4 Enclaves Configuration MILS/MLS Architecture: 4 Enclaves Configuration Segment Unit
Qty Unit Cost
Cost ($M)
Benefit Weight
Benefit ($M)
Value (=B/C)
Unit Qty
Unit Cost
Cost ($M)
Benefit Weight
Benefit ($M)
Value (=B/C) Architecture Feature / Model Criteria
EMD Phase $126 $90 71% $107 $84 78% Embedded Segment $43 $31 73% $36 $29 80% Core Processing: SH or Primary SL Processing Separation
SL SW Instances Assurance Certification SWaP Performance Flexibility Extensibility
7 $0.4 $3 85% $2 6 $0.4 $2 90% $2 Core Processing: Other Security Domains 3 $0.4 $1 20% $0 4 $0.4 $2 80% $1 CDS: Platform: HW, OS, CDS Engine, Interfaces 2 $3.0 $6 80% $5 2 $3.0 $6 85% $5 Processing Backplane 3 $2.0 $6 60% $4 1 $4.0 $4 100% $4 Network System 4 $1.0 $4 60% $2 1 $4.5 $5 80% $4 Disk Storage 3 $1.2 $4 70% $3 1 $4.5 $5 80% $4 Disk Encryption 3 $1.2 $4 70% $3 1 $2.0 $2 80% $2 HMI / Displays & Controls 1 $3.0 $3 100% $3 1 $6.0 $6 60% $4 Data Labeling
Complexity Assurance Certification
25 $0.2 $5 80% $4 0 $0.2 $0 100% $0 High-level CDS Rule Set 15 $0.3 $5 80% $4 9 $0.3 $3 80% $2 Mid-level CDS Rule Set 5 $0.3 $2 80% $1 4 $0.3 $1 80% $1 CDS Protocol Support 7 $0.2 $1 70% $1 4 $0.2 $1 80% $1 Operations Segment $48 $32 67% $32 $23 74% Core Processing: SH or Primary SL Processing Separation 2 $1.0 $2 75% $2 2 $1.0 $2 70% $1 . . . . . . 10 $0.2 $2 60% $1 5 $0.2 $1 80% $1 System $35 $26 75% $40 $32 80% Integration Complexity 1 $25.0 $25 75% $19 1 $25.0 $25 80% $20 Certification & Accreditation (C&A) Complexity 1 $10.0 $10 75% $8 1 $15.0 $15 80% $12 Production Phase $191 $134 70% $150 $119 79% Embedded Segment $73 $52 71% $51 $41 82% Operations Segment $57 $37 65% $35 $26 74% System $60 $45 75% $65 $52 80% Operations Phase $126 $90 71% $107 $84 78% Embedded Segment $43 $31 73% $36 $29 80% Operations Segment $48 $32 67% $32 $23 73% System $35 $26 75% $40 $32 80%
Program Totals (All Segments) $442 $314 71% $365 $287 79%
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
19
Cost/Benefit Model Detailed Results Program Phase Segment Solution Description Benefit
Weighting Criteria
SH/MSLS Architecture: 4 Enclaves Configuration MILS/MLS Architecture: 4 Enclaves Configuration
Segment SH / MSLS Architecture
MILS / MLS Architecture
Unit Qty
Unit Cost
Cost ($M)
Benefit Weight
Benefit ($M)
Value (=B/C)
Unit Qty
Unit Cost
Cost ($M)
Benefit Weight
Benefit ($M)
Value (=B/C) Architecture Feature / Model Criteria
EMD Phase $126 $90 71% $107 $84 78% Embedded Segment $43 $31 73% $36 $29 80% Core Processing: SH or S3B Primary processing at SH Primary SL processing at S3B
Separation SL SW Instances Assurance Certification SWaP Performance Flexibility Extensibility
7 $0.4 $3 85% $2 6 $0.4 $2 90% $2
Core Processing: Other Security Domains Multiple SL processing enclaves SL SW partitions on MILS SK 3 $0.4 $1 20% $0 4 $0.4 $2 80% $1
CDS: Platform: HW, OS, CDS Engine, Interfaces Two HW CDS platforms; MLS OS Two MILS SK platforms 2 $3.0 $6 80% $5 2 $3.0 $6 85% $5
Processing Backplane SL backplane per enclave Single MILS Network System 3 $2.0 $6 60% $4 1 $4.0 $4 100% $4
Network Hub SL network per enclave Single MILS Network System 4 $1.0 $4 60% $2 1 $4.5 $5 80% $4
Disk Storage SL disk per enclave Single MLS File System 3 $1.2 $4 70% $3 1 $4.5 $5 80% $4
Disk Encryption SL encryption per enclave Single SH encryption 3 $1.2 $4 70% $3 1 $2.0 $2 80% $2
HMI / Displays & Controls SH console Single MLS Concole System 1 $3.0 $3 100% $3 1 $6.0 $6 60% $4
Data Labeling As necessary for off-board CDS Not required
Complexity Assurance Certification
25 $0.2 $5 80% $4 0 $0.2 $0 100% $0
High-level CDS Rule Set SH-S3A and SH-S2 rule sets S3A-S2 and S3B-S2 rule sets 15 $0.3 $5 80% $4 9 $0.3 $3 80% $2
Mid-level CDS Rule Set S2-S1 rule sets S2-S1 rule sets 5 $0.3 $2 80% $1 4 $0.3 $1 80% $1
CDS Protocol Support Larger rule set may inhibit Smaller rule set may enable 7 $0.2 $1 70% $1 4 $0.2 $1 80% $1
Operations Segment $48 $32 67% $32 $23 74% Server Processing: SH or S3B Primary processing at SH Primary SL processing at S3B
Separation SL SW Instances Assurance Certification Performance Flexibility Extensibility
2 $1.0 $2 75% $2 2 $1.0 $2 70% $1
Server Processing: Other Security Domains Multiple SL processing enclaves SL SW partitions on MILS SK 4 $1.0 $4 40% $2 2 $1.0 $2 90% $2
CDS: Platform: HW, OS, CDS Engine, Interfaces Two HW CDS platforms; MLS OS Two MILS SK platforms 2 $2.0 $4 55% $2 2 $2.0 $4 65% $3
Network Hub SL network per enclave Single MILS Network System 5 $0.6 $3 40% $1 1 $3.0 $3 65% $2
Disk Storage SL disk per enclave Single MLS File System 5 $0.6 $3 60% $2 1 $3.0 $3 65% $2
Disk Encryption SL encryption per enclave Single SH encryption 4 $1.0 $4 70% $3 1 $1.5 $2 65% $1
HMI / Displays & Controls SH and MSLS consoles Single MLS Concole System 5 $1.0 $5 60% $3 1 $4.5 $5 75% $3
High-level CDS Rule Set SH-S3A, SH-S3B, and SH-S2 rule sets S3A-S2 and S3B-S2 rule sets Complexity Assurance Certification
50 $0.3 $15 80% $12 25 $0.3 $8 80% $6
Mid-level CDS Rule Set S2-S1 rule sets S2-S1 rule sets 20 $0.3 $6 80% $5 10 $0.3 $3 80% $2
CDS Protocol Support Larger rule set may inhibit Smaller rule set may enable 10 $0.2 $2 60% $1 5 $0.2 $1 80% $1
System $35 $26 75% $40 $32 80% Integration Interoperability between SL enclaves Integration of MILS Ecosystem products Complexity 1 $25.0 $25 75% $19 1 $25.0 $25 80% $20
Certification & Accreditation (C&A) CDS and SL enclaves CDS, SL enclaves, and MILS products Complexity 1 $10.0 $10 75% $8 1 $15.0 $15 80% $12
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
20
Cost/Benefit Results Summary ($M)
(%)
Value = Benefit / Cost
Number of Enclaves
Number of Enclaves
$100
$200
$300
$400
$500
$600
$700
1 2 3 4 5 6 7
SH / MSLS Cost
MILS / MLS Cost
SH / MSLS Benefit
MILS / MLS Benefit
50
60
70
80
90
100
1 2 3 4 5 6 7
SH / MSLS Value
MILS / MLS Value
Cost Benefit
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
21
• What’s been tried before? – Useful, but not secure – Secure, but not useful – Build it, they will come …
• What works? – A market based on effective offerings – Solutions that solve the hard problems – Assured systems from assured components – Affordable trustworthy architectures
• MILS Ecosystem components with potential – MILS Separation – MILS Network System – MLS File System – MLS Console System – MILS SK-based CDS
Cost/Benefit in the Security Arena …
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13
22
• Better incorporate the following qualities into the Cost/Benefit Model – Relevant – Complete – Consistent – Transparent – Accurate – Conservative – Insightful
• Architecture Analysis and Design Language (AADL) Modeling
• Cost/Benefit Model Maturity – Include costs based on actuals – Quantify and detail Benefit Weighting Criteria – Automate the model to include “submodel” characterization functions – Automate scenario, analysis, and reporting functions
Further Investigation …
From 2007 Workshop on the Economics of Information Security, A Framework for Classifying and Comparing Models of Cyber Security Investment to Support Policy and Decision-Making, Rachel Rue et al.
See Software Engineering Institute, Architectural Security Modeling with the AADL, Jorgen Hansson et al.
Approved for Public Release: Northrop Grumman Aerospace Systems Case 13-2400, 12/9/13