Security Attribute Evaluation Method: A Cost Benefit Analysis

Shawn A. ButlerComputer Science Department

Carnegie Mellon University27 November 2001

MS

Hey Boss, we need more security. I think we should get the new Acme 2000 Hacker

Abolisher

We always seem to need more security!

Don’t we have enough?

MS

Trust me, we will be more secure!

What are my alternatives?

What is it going to cost?

What is the added value?

Value?

Alternatives?

S

Problem

• Security managers lack structured cost-benefit methods to evaluate and compare alternative security solutions.

Security Architecture Development Process

Risk Assessment

Risk Assessment

Outcomes

Threats

PrioritizedRisks Select

CountermeasuresSelect

Countermeasures

System Design

Policies Requirements

AvailableCountermeasures

Security Components Develop

Security Architecture

Develop Security

Architecture

Security Architecture

The Multi Attribute Risk Assessment

1. Determine threats and outcomes2. Assess outcome attribute values3. Assess weights4. Compute threat indices5. Sensitivity Analysis

Risk Assessment

Risk Assessment

Outcomes

Threats

PrioritizedRisks

ThreatsScanningProcedural ViolationBrowsingDistributed Denial of

ServicePassword NabbingPersonal AbuseSignal Interception : :29 Threats

Determine Threats and Outcomes

Outcome AttributesLost ProductivityLost RevenueRegulatory PenaltiesReputationLives LostLawsuits : :

Scanning in More Detail Outcomes

Attacks

Lost Producti-vity (hrs)

Lost Revenue($$)

Regulatory Penalties(scale 0-6)

Reputation(scale 0-6)

Scanning10,220/yr

Low .3 0 0 1Expected .5 2 0 1

High 1 1,000 0 4.01 = plow (j=attributesWj Vj(xj

low)).07 = pexpected (j=attributesWj Vj(xj expected))

.00 = phigh (j=attributesWj Vj(xj

high)) 10,220 (.01 +.07 +.00) 886.57

Risk Assessment Results

ThreatFrequency Low Expected High Total

Scanning 10,220 .0084 .0750 .0034 886.57

Procedural Violation 4380 .0000 .0773 .0065 367.03

Browsing 2920 .0000 .0742 .0035 226.71

Dist Denial of Service 156 .0085 .1530 .0060 26.12

Password Nabbing 365 .0001 .0008 .0009 .62

Personal Abuse 110 .0000 .0003 .0009 .13

TOTAL 1,507.18

Risks as a Percentage of Threat Index Total

Scanning36%

Procedural Violation27%

Signal Interception19%

Browsing9%

Other1%

Cryptographic Compromise

1%

Trojan Horse1%

Compromise1%

DDoS1%

Virus2%

Alteration2%

But what about the numbers?

Sensitivity Analysis is Key!!

• How sensitive are the answers to estimation errors?

• Does it matter if the estimates are not accurate?

• How accurate do they have to be before the decision changes?

• When is it important to gather additional information?

Security Attribute Evaluation Method (SAEM)

• Evaluation Method1. Assess security technology benefits2. Evaluate security technology benefits3. Analyze Costs 4. Assess coverage5. Sensitivity Analysis

Select Countermeasures

Select Countermeasures

System Design

Policies Requirements

AvailableCountermeasures

Security ComponentsPrioritized

Risks

Assess Security Technology Benefits

Scanning 50% 75% 66% 66% 33% 33%50%

Procedural Violation

50% 40%25%

Browsing 30%

Dist Denial of Service

75%

Password Nabbing

50%

Personal Abuse

40%

Effectiveness Percentages

Threat Secu

rity

Tech

PF

Fire

wall

Prx

y F

irew

all

Net

IDS

Audit

ing

Host

ID

S

Vuln

Ass

ess

Hard

ened O

S

Auth

Polic

y S

erv

Vir

tual Pri

v N

et

Net

Monit

ors

Prioritized Technologies

Technology

Value Threat Index

Overall Rank

PKI/Cert .24 28

Auditing 241 11

Auth Policy Server

161 15

Host-IDS 589 2

Net-IDS 293 10

Smart Cards 103 16

One Time Psswrd

340 7

Single Sign-on 0 35

Analyze Costs

0

589

$0

Host IDS

Single Sign-on

Smart Cards

Net IDS Auditing

PKI Cert$20,000

Auth Policy Server

Th

reat

Ind

ex

Purchase Cost

Assess Coverage

Host Intrusion Detection Coverage

Auditing Coverage

Preliminary Results

• Risk Assessment threat indices reflect security manager’s concerns– based on interviews and feedback

• Security managers are able to estimate technology benefits – based on experience, organizational skill

levels, and threat expectations

• Sensitivity Analysis is key to method– based on uncertainty of assumptions