of 143
7/30/2019 Bila r Phd Thesis
1/143
Quantitative Risk Analysis of Computer Networks
A Thesis
Submitted to the Facultyin partial fulfillment of the requirements for thedegree of
Doctor of Philosophy
by
Daniel Bilar
Thayer School of EngineeringDartmouth College
Hanover, New Hampshire
June, 2003
Examining Committee:
George Cybenko(Chairperson)
Robert Morris(Member 1)
Robert Gray(Member 2)
Sue McGrath(Member 3)
Carol L. FoltDean of Graduate Studies
c 2003 Trustees of Dartmouth College
7/30/2019 Bila r Phd Thesis
2/143
7/30/2019 Bila r Phd Thesis
3/143
ii
Thayer School of EngineeringDartmouth College
Quantitative Risk Analysis of Computer Networks
Daniel Bilar
Doctor of Philosophy
George CybenkoRobert MorrisRobert GraySue McGrath
ABSTRACT
Quantitative Risk Analysis of Computer Networks (QSRA) addresses the problem of risk opacity
of software in networks. It allows risk managers to get a detailed and comprehensive snapshot of
the constitutive software on the network, assess its risk with assistance of a vulnerability database,
and manage that risk by rank ordering measures that should be taken in order to reduce it, subject
to cost, functionality and risk constraints. A theoretical methodology is proposed and a prototype
implementation has been developed. Six out-of-the-box popular operating systems were studied using
the methodology and the prototype.
We find that around 75% of discovered vulnerabilities are patchable within two weeks, and around
90% within 40 days after initial discovery. We find a statistically significant time window difference
between security-audited and non-security audited software. Across the operating systems, the majority
of faults give rise to availability and full compromise consequences. There is a statistically significant
difference between fault types: Input validation faults are proportionally over-represented. There is a
statistically significant difference between consequence types: Full compromise consequences are propor-
tionally over-represented. There is, however, no statistically significant fault or consequence proportion
difference between the audited systems.
QSRAs risk assessment model calculated that for all audited systems, four to six months after their
respective release date, the probabilities are very high (66% to 99%) that an attacker can conduct a full
consequence compromise, remotely and locally. Risk management analysis for remote risk probabilities
indicates that, given a moderate fault count, QSRAs highest risk analytic risk mitigation strategy
consistently outperforms the simpler strategy of choosing software with the highest vulnerability count.
Highest risk outperforms the undifferentiated highest count strategy for at least four out of the six
tested operating systems and for four out of five fault consequences.
7/30/2019 Bila r Phd Thesis
4/143
iii
I would like to thank everybody who helped me. George Cybenko as my adviser and mentor who
showed me that one can routinely have two brilliant ideas a day and still be a decent and kind human
being. My tender, kind and loving girlfriend Daniella Hirschfeld who brought me sandwiches when I was
starving writing this thesis. Biiiiig kiss, my sweetheart! NH Senator Gregg, Dean Duncan and George
again for getting Washington, DC to finance my excellent work environment. Susan McGrath who runs
the IRIA research group with great success and a winning presence. Bob Gray, a kind man for always
finding time to field questions as he is preparing four papers at the same time. Guofei Jiang whose
unassuming demeanor belies his immense work ethic and sparkling intelligence. Robert Morris as an
example that truth can be far stranger than fiction. The excellent teachers I had at Thayer, including
but not limited to Prof. Eric Hansen and Prof. Ulf Oesterberg. And last but not least, all those people
who follow Wordsworths dictum:
That best portion of a good mans life; his little, nameless, unremembered acts of kindness
and love
I remember.
7/30/2019 Bila r Phd Thesis
5/143
Contents
1 Introduction 1
1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.1 Fault Tree Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.2 Previous work on cybersecurity risk . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Major findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.1 Parameters and Vulnerability Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.2 Faults and consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.3 Design and Implementation of the QSRA system . . . . . . . . . . . . . . . . . . . 8
1.4.4 QSRA empirical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Design of the QSRA system 10
2.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1 Software quality, faults, failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Process sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.1 Preliminary example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.2 QSRA risk calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.5 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.1 Optimization problem setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.6.1 ISTS ICAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.6.2 NETWORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
iv
7/30/2019 Bila r Phd Thesis
6/143
CONTENTS v
2.7 A QSRA example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Implementation of QSRA system 28
3.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3 QSRA Event Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.1 Vulnerability assessment: Software inventorying . . . . . . . . . . . . . . . . . . . . 32
3.3.2 Vulnerability assessment: Software identification . . . . . . . . . . . . . . . . . . . 34
3.3.3 Risk assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3.4 Risk management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4 Analysis 37
4.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1.1 Parameters and Vulnerability Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1.2 Faults and consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1.3 QSRA empirical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2 Parameter and vulnerability data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.1 Vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.2 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2.3 Timing data set findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2.4 Attacker expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.1 Audited environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.2 Data and analytical framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4 Vulnerability assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.5 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.5.1 Risk calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.5.2 Fault and Consequences distributions . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.6 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.7 Possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.7.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.7.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
7/30/2019 Bila r Phd Thesis
7/143
CONTENTS vi
5 Future Work 82
5.1 Database update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.1.1 Vulnerability Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.1.2 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.1.3 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
A Numeric analysis 90
A.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
A.2 ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
B Databases 92
B.1 ISTS ICAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
B.2 NETWORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
C Supplemental data 101
C.1 Average time window from discovery of the vulnerability to posted patch . . . . . . . . . 101
C.2 Software inventorying experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
C.3 Faults and Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
C.3.1 Vulnerability count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
C.3.2 Fault count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
C.4 Risk management scenario results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
D Glossary 124
D.1 Boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7/30/2019 Bila r Phd Thesis
8/143
List of Tables
1.1 Virus spread profile 1990-2003 [Kes00][Bek03] . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Software fault densities [Bin97] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 Hypothetic risk of a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Fault types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Consequence types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Functionality (Class) groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5 Vulnerabilities data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.6 Risk management results for George . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.7 Remote risk limits for George . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.8 Risk assessment: Consequence losses and consequence probabilities for George . . . . . . 27
2.9 Risk management: Consequence losses and consequence probabilities for George . . . . . 27
2.10 Vulnerabilities of Apache 2.0.35 and MySQL 4.0.5 . . . . . . . . . . . . . . . . . . . . . . 27
2.11 Vulnerabilities of Apache 1.3.26 and MySQL 3.23.50 . . . . . . . . . . . . . . . . . . . . . 27
3.1 QSRA event sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Sample uname and ver output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1 Questionnaire: Vulnerability lifecycle and attacker ability . . . . . . . . . . . . . . . . . . 40
4.2 Categorical and continuous data sources: decision criteria . . . . . . . . . . . . . . . . . . 41
4.3 Vulnerability lifecycle events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4 ANOVA results of time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . 44
4.5 Experts estimate for time window from discovered vulnerability to posted patch [tdesc, tpatch] 49
4.6 Overview of questionnaire results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.7 Vulnerability advisories vs attack in 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.8 Standard installed software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
vii
7/30/2019 Bila r Phd Thesis
9/143
LIST OF TABLES viii
4.9 Experimental factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.10 Timing experiment: Software count, database size, OS . . . . . . . . . . . . . . . . . . . . 54
4.11 Timing experiment: Connection type, ARP cache . . . . . . . . . . . . . . . . . . . . . . . 55
4.12 Timing experiment: Database size, number of inventoried hosts . . . . . . . . . . . . . . . 55
4.13 Stepwise regression results, two factors: Number of software packages, database size . . . 57
4.14 Stepwise regression results, two factors: Number of hosts, database size . . . . . . . . . . 57
4.15 OS ANOVA: Software inventorying timing data . . . . . . . . . . . . . . . . . . . . . . . . 60
4.16 Size of software inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.17 Parameters: Attacker ability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.18 Parameters: Time window till posted exploit . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.19 OS consequence probability calculation results . . . . . . . . . . . . . . . . . . . . . . . . 65
4.20 Proportion of OS to consequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.21 Proportion of consequences to OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.22 Proportion of consequences to fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.23 Proportion of faults to consequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.24 Proportion of OS to fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.25 Proportion of faults to OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.26 ANOVA results of proportions of faults, OS and consequences . . . . . . . . . . . . . . . . 68
4.27 Fault type breakdown 2002 [oSN02] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.28 Consequence magnitudes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.29 Individual OS remote consequence risk probability scenario results . . . . . . . . . . . . . 72
4.30 Individual OS remote vulnerability count scenario results . . . . . . . . . . . . . . . . . . 73
4.31 OS family average remote risk probability scenario results . . . . . . . . . . . . . . . . . . 74
4.32 OS family average remote vulnerability count scenario results . . . . . . . . . . . . . . . . 74
4.33 Relevant libraries used by Apache 1.3.26 and Apache 1.3.27 . . . . . . . . . . . . . . . . . 81
5.1 Revised vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2 New vulnerability consequence types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
B.1 System aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
B.1 System aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
B.1 System aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
B.1 System aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
B.2 Vulnerabilities aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7/30/2019 Bila r Phd Thesis
10/143
LIST OF TABLES ix
B.2 Vulnerabilities aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
B.3 Relations aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
B.4 NCC aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
B.5 Risk Assessment aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
B.5 Risk Assessment aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
B.6 Risk Management aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
B.6 Risk Management aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
C.1 Security advisories as of September 30th, 2002 . . . . . . . . . . . . . . . . . . . . . . . . . 101
C.2 Average time window (days) of discovered vulnerability to posted patch . . . . . . . . . . 103
C.3 ANOVA group mean rank comparison: OS . . . . . . . . . . . . . . . . . . . . . . . . . . 106
C.4 ANOVA results group mean rank comparison: consequences vs fault type . . . . . . . . . 112
C.5 ANOVA results group mean rank comparison: consequences vs OS . . . . . . . . . . . . . 112
C.6 ANOVA results group mean rank comparison: fault type vs OS . . . . . . . . . . . . . . . 113
C.7 Vulnerability count by OS, class and access: Debian 3.0 and Mandrake 8.2 . . . . . . . . . 115
C.8 Vulnerability count by OS, class and access: Suse 8.0, W2K SP2, OpenBSD 3.1 and
RedHat 7.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
C.9 Fault count of standard OS installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7/30/2019 Bila r Phd Thesis
11/143
List of Figures
1.1 Fault tree example of event No power on the 415V bus [www03] . . . . . . . . . . . . . 5
2.1 Software view of the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 QSRA approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Fault tree for consequence c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Escalation exploits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1 Inventorying the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Sample lsof output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Sample fport output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Specifying the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.5 Assessing the software risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.6 Managing software risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 Timeline of events in vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3 Histogram of time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . . . . 46
4.4 Group mean comparison of [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.5 Scatterplot: Software inventorying time for one host . . . . . . . . . . . . . . . . . . . . . 56
4.6 Scatterplot: Software inventorying time for multiple hosts . . . . . . . . . . . . . . . . . . 57
4.7 Failure probability of independent components in series . . . . . . . . . . . . . . . . . . . 63
4.8 Fault count data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.1 Vulnerability database input mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
C.1 Boxplot of [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
C.2 Software inventorying time ANOVA: Boxplots for connection type and ARP size . . . . . 105
x
7/30/2019 Bila r Phd Thesis
12/143
LIST OF FIGURES xi
C.3 Software inventorying time ANOVA: Boxplots for database size and software count . . . . 108
C.4 Software inventorying time ANOVA: Boxplots for database size and software count . . . . 109
C.5 Fault count ANOVA: Boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
C.6 Risk probability scenario reduction: Consequences . . . . . . . . . . . . . . . . . . . . . . 120
C.7 Risk probability scenario reduction: Consequences . . . . . . . . . . . . . . . . . . . . . . 121
C.8 Risk probability scenario reduction: Individual OS . . . . . . . . . . . . . . . . . . . . . . 122
C.9 Risk probability scenario reduction: OS families . . . . . . . . . . . . . . . . . . . . . . . . 123
D.1 Sample boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7/30/2019 Bila r Phd Thesis
13/143
Chapter 1
Introduction
1.1 Problem statement
At the end of the 20th century, we have witnessed the massive transition from isolated, disconnected
computers to networked computer clusters all over the world. At the time of this writing (Spring 2002),
there are an estimated 195 million networked hosts world-wide [Tec02].
This global pervasive connectivity has been a boon for consumers, businesses and governments alike
due to the ease, convenience and speed of electronic data exchange. However, the ease of use and
relative anonymity that the Internet affords has been leveraged by criminal elements, as well. The
National Infrastructure Protection Centers (http://www.nipc.gov/) Congressional statement of May
25th, 2000, highlights the risks we face today:
Ninety percent of respondents detected security breaches over the last 12 months. At least
74% of respondents reported security breaches including theft of proprietary information,
financial fraud, system penetration by outsiders, data or network sabotage, or denial of
service attacks. Many companies experienced multiple attacks; 19% of respondents reported
10 or more incidents. Information theft and financial fraud caused the most severe financial
losses, estimated by the respondents at $68 million and $56 million respectively. The losses
from 273 respondents totaled just over $265 million. Notably, this survey does not include
harm caused by recent destructive episodes such as the Distributed Denial of Service attacks
on e-commerce sites in February, and the ILOVEYOUor Love Bug virus earlier this month.
No private, commercial or government agency is completely safe or has been unaffected by the
proliferation of this kind of cyber-crime. E-Commerce Times reported that the ILOVEYOUvirus affected
1
7/30/2019 Bila r Phd Thesis
14/143
CHAPTER 1. INTRODUCTION 2
Table 1.1: Virus spread profile 1990-2003 [Kes00][Bek03]Virus Year Type Time to prevalence 1 Estimated damagesJerusalem, 1990 .exe File, Boot Sector 3 years $50MCascade,Form,Concept 1995 Word Macro 4 months $50MMelissa 1999 E-mail enabled, Word Macro 4 days $93M-$385MLove Bug 2000 E-mail enabled, .vbs 5 hours $700M - $6.7BSlammer 2003 self-propagating worm 10 minutes $1B
Table 1.2: Software fault densities [Bin97]Application Faults per thousand lines
of source codeNASA Space Shuttle Avionics 0.1
Leading-edge software companies 0.2Reliability Survey 1.4Best, Military system survey 5.0Worst, Military system survey 55.0
45 million hosts and inflicted monetary damages to the tune of estimated $2.6 billion [Eno00]. The
infamous Melissa macro virus caused an estimated $300 million in damage in 1999 and several prominent
e-commerce sites were hit by Distributed Denial of Service attacks in the beginning of 2000 [SoT00].
The estimated worldwide damage caused by automated digital attacks over $30 billion for 2002 [mi202b].
These estimated damage figures have to be taken with a grain of salt, but the trend is clear. Moreover,
in just a dozen years time, the propagation speed, as well as the estimated damages have increased by
five, and two orders of magnitude, respectively (see Table 1.1).
One of the persistent factors is faulty software. Software fault density remains between 0.1 and 55
faults per thousand lines of source code (see Table 1.2). Exacerbating circumstances include increases in
code size and in the number of interaction pathways between the various software subsystems. Perhaps
the most aggravating fact is the unwillingness and unawareness of designers and implementers alike to
use proven techniques to build secure code. The net result has been an increase in the number of poten-
tially serious software faults. In the absence of larger, long-term efforts and initiatives (vendor liability,
legal regulations, widespread adoption of safe design practices and responsible user behavior), the risk
posed by faulty software has to be managed somehow.
1Time it took to become the #1 most prevalent computer virus in history
7/30/2019 Bila r Phd Thesis
15/143
CHAPTER 1. INTRODUCTION 3
1.2 Feasibility
There are two main objections that have been raised to the feasibility of cyberrisk analysis and manage-
ment:
Unknown enemies with unknown skills, knowledge, resources, authority, motives, and objectives
(SKRAMO) attacking unknown vulnerabilities make risk measurement and management impossi-
ble.
Dealing with known SKRAMO and known vulnerabilities does little to reduce risk.
The first argument is rather like Lord Kelvins 1895 remark in front of the Royal Society: Heavier
than air flying machines are impossible. Incomplete information does not mean no information at
all. There are studies of software defects in code [MV96][Rea03], theoretical modelling of software
defects [MIO90] and attack proportions/damages among operating systems [mi202c][mi202a]; as well as
extensive historical vulnerability databases [oSN02][Str02]. These sources can be used to hypothesize
about future unknown vulnerabilities, attack frequencies, and attack targets. It is correct, however,
that there is a conspicuous lack of attacker studies - their knowledge, resources, and skills. The short
answer is that in light of incomplete information, cyber risk, like all risks, cannot be eliminated, but
can be managed. Various organizations, diligent ones like the US DoD among them, have been doingthis for some time now [Gor03]. The issue is subtler when one tries to evaluate cyberrisk in the context
of formulating insurance policies and setting insurance premiums. Insurance companies have to guard
against two phenomena when setting premiums: adverse selection, when someone chooses to insure
against a particular loss because of private information not available to the insurance companies at the
time of contracting (like somebody getting life insurance before going to Liberia), and moral hazard,
which denotes the lack of incentives by the insured to take actions that reduce the probability of a loss
subsequent to purchasing the insurance [GLS03]. The moral hazard problem can be fruitfully addressed
by QSRA. Extensive software auditing and subsequent risk analysis can pinpoint problem areas to the
insurers, which in turn can offer discounts for companies that take (QSRA specified) security measures.
The short answer is that the QSRA methodology is not the whole picture, but is just part of the premium
assessing process.
The second argument has been shown to be false. Known vulnerabilities are the main entry point
into most systems, as security groups have maintained and recent incidents have shown [SASS02,
/top20][CC03, VU484891]. Mitigating these risks would go a long way towards improving security.
In addition, the issue is not known vulnerabilities per se. The issue is one of locating the software on the
7/30/2019 Bila r Phd Thesis
16/143
CHAPTER 1. INTRODUCTION 4
network that exhibits these vulnerabilities and patching them. Few administrators are diligent about
installing the latest patches and upgrades. Almost no one has a good overview of the software running
on their network in the first place. An extensive software inventorying is a pre-requisite for dealing with
known vulnerabilities possible, and is one of the most useful features of the implemented QSRA system.
1.3 Risk
1.3.1 Fault Tree Analysis
Fault tree analysis was developed by Bell Telephone Laboratories in 1961, to improve the reliability of
the Minuteman Launch Control System [II99]. Since then, it has been extensively used for evaluatingsystem safety and concomitant risk in many engineering disciplines (power, nuclear, electric, source code
analysis) [oRBC75] [LM00]. Fault tree analysis is used to model failure situations in (typically) multi-
component systems. The goal of fault tree analysis is to investigate dependencies between failures of
components that lead to the fault trees top-event. A fault tree is a logical diagram representing the
consequences of component failures on the failure of the so-called top-event, the root of the tree. The
top-event occurrence depends on a specific combination of basic events, which are combined with two
primary types of gates, AND and OR.
An example of a fault tree is given in Figure 1.1. The top-event is 415VBUS, indicating that there
is no power on the 415V bus. The sub-events are ISO-A, ISO-D (and its negation NOT(ISO-D)), CON-A,
DIESEL, and EXT-GRID-REPAIR. The following Boolean expression illustrates their relationship:
415VBUS = ((ISO-A NOT(ISO-D)) + CON-A + DIESEL) (EXT-GRID-REPAIR + ISO-D) (1.1)
Systems
A systems reliability function r(t) is the probability that the top event does not occur over time period
(0, t). The systems failure function f(t) is equal to 1 r(t). Systems can be arranged in different
configurations. Hillier and Lieberman identify three structures: series, parallel, and k-out-of-n-systems
[HL90]. A serial system is a system which does not fail only if all components do not fail. A parallel
system is a system which does not fail only if at least one component does not fail. A k-out-of-n-system
fails if at least (n k) components fail.
For serial systems with n independent components, each with reliability ri(t), the systems reliability
rsystem(t) and the systems failure fsystem(t) are given by Equations 1.2 and 1.3.
7/30/2019 Bila r Phd Thesis
17/143
CHAPTER 1. INTRODUCTION 5
Figure 1.1: Fault tree example of event No power on the 415V bus [www03]
rsystem(t) =
ni=1
ri(t) (1.2)
fsystem(t) = 1 ni=1
(1 fi(t)) (1.3)
This is the basic risk modelling setup that is going to be used by QSRAs calculations.
1.3.2 Previous work on cybersecurity risk
There exist a number of suites that attempt to evaluate and manage cybersecurity risk (Strategic Tech-
nology Institutes Cost-Benefit-Analysis Tool C-BRAT [Ins01], SAINT Corporations Security Admin-
istrators Integrated Network Tool [Cor01], Network Associates CyberCop Scanner [Ass01]), proposed
measures to quantify the risk of vulnerability exposure (Northcutts severity measure system [Nor99])
and approaches for assessing vulnerabilities in computer networks (Sandias computer-attack graph gen-
eration tool [PS98] [SPEC01]).
The risk calculations in the aforementioned approaches are either opaque (in the case of the com-
7/30/2019 Bila r Phd Thesis
18/143
CHAPTER 1. INTRODUCTION 6
mercial products) or very crude (Northcutts measure), or gloss over how the probability/risk measure
will actually be calculated (Sandia). Furthermore, practical, specific, granular risk management is a
key aspect of an integrated methodology of mitigating cybersecurity risk - and there has been a paucity
of tools and approaches in that respect. Lastly, since todays networks are increasingly ad-hoc, mobile
and wireless (with the invariably concomitant increase in risk), no effective methodology can afford to
be static - it must constantly adjust itself or be quickly adjustable to reflect the addition, removal and
modifications of hosts and their constitutive software. Ideally, the assessment, analysis and management
recommendations must be real-time or near-real time (on the order of minutes). QSRA was designed
with six principles in mind:
1. Generality: QSRA is general in its design and can be deployed to assess and manage the risk posed
by any IP device on a network
2. Adaptiveness: QSRA can be run in a semi-automated or manual fashion on a network of interest;
mobile, wireless, static, or combinations
3. Comprehensiveness: QSRAs risk metric calculations take machine configuration, functionality
and value, attacker ability, software installed/running, time flow, and compromise dependencies
into account
4. Granularity: The granularity of the data used in the risk analysis step is on the level of the
individual program, broken down into different compromise consequences of different severity
5. Transparency: QSRA is an open methodology, as well as a toolset,that is largely platform inde-
pendent with exclusive use of COTS components
6. Solutions: QSRA offers risk management, i.e. rank ordered practical steps to reduce risk such
as upgrading this web server on IP 127.0.0.2 to a newer version, or disabling service A on IP
127.0.0.7 subject to cost, compatibility and functionality constraints.
1.4 Major findings
Vulnerability data was collected for six operating systems: Red Hat Linux 7.3, SuSe Linux 8.0, Mandrake
Linux 8.2, OpenBSD 3.1, Debian Linux 3.0, and Windows 2000 Professional SP2. The standard, out-of-
the-box workstation installation was chosen when installation options were presented. Since the Linux
and OpenBSD distributions come with a wealth of applications, MS Office was assumed to be part of
the standard out-of-the-box Windows 2000 distribution.
7/30/2019 Bila r Phd Thesis
19/143
CHAPTER 1. INTRODUCTION 7
1.4.1 Parameters and Vulnerability Data
Vulnerability lifecycle
Empirical research on 58 vulnerabilities over a three months period showed that in over 60% of cases
where data were available, exploits were obtainable before and at the time the patch was released. Most
exploits were obtainable the day the patch was released, and some preceded the patch by as much as
two months.
Time window data finding
For six audited operating system, we find that around 75% of vulnerabilities are patchable within two
weeks, and around 90% within 40 days after initial discovery.
There is no statistical significant difference between locally and remotely exploitable vulnerabilities,
nor between vulnerability consequence types. No statistical inference can be drawn regarding fault types.
There is no statistical significant difference between open source and closed source software. There is,
however, a statistically significant difference between security-audited and non-security audited software.
Attacker expertise
Automated exploit tools and straight-forward exploit descriptions are readily found on the Internetand make acquisition of attacker skills easy. A corollary of this finding is that one cannot count on
vulnerabilities and associated exploits to remain secret.
1.4.2 Faults and consequences
Faults were mapped to a vulnerability consequence, a fault type and an operating system. The over-
whelming majority of fault occurrences lead to availability and full compromise consequences. 75% of
the 3-tuples (OS, consequence and fault type) have a count of 0, and around 25% fall within a count
range of four to one.
Within availability and full compromise consequences, around 15% percent have a count of one or
two, and around 25% fall within a count range of four to one. Input validation faults are proportionally
over-represented.
There is a statistically significant difference between consequence types. Full compromise conse-
quences are proportionally over-represented. There is no statistically significant fault or consequence
proportion difference between the audited hosts.
7/30/2019 Bila r Phd Thesis
20/143
CHAPTER 1. INTRODUCTION 8
1.4.3 Design and Implementation of the QSRA system
Java as an implementation language proved its worth in cross-platform development. It cannot be
stressed enough how much time is saved by the lack of the tedious memory leaks and pointer manip-
ulations that plague C and C++. Text parsing, however, is not Javas forte, and hence QSRA client
development took a disproportional amount of time.
The decision to design a relational, as opposed to a flat database was found to have been very
fortunate. Having all data relations in third normal form is advisable, since this enables additions and
mutations to be accommodated with few if any changes to the existing design.
The ists_icat database holds data about software characteristics and its associated vulnerabilities.
It is used to provide choices for the software identification, data and parameters for risk assessment and
options for risk management. It has two main functional aggregates: Systems which holds data on
software programs, and Vulnerabilities, which holds data on the softwares associated vulnerabilities.
Its comprehensiveness is key. Lower estimate for necessary risk management data for 40 software class
categories with 5 substitutes programs each across 3 families (Windows, BSD, Linux) is 600 entries. The
lower estimate for risk analysis is far higher, since there are thousands of software packages for each
family. The lower bound record size of the Systems aggregate, which holds data data on software
programs, for production quality deployment is estimated to be around 3000 entries, namely for three
OS families with fifty functionality groups containing twenty records each. This number is necessary so
that the risk management optimizer has a sufficiently large pool from which to assemble an alternative
software makeup. In steady-state, and with an input mask, it takes no more than 5 minutes to amass
and assess one entry and another 3 minutes to write it to the database, making the time commitment
to populate the Systems database to be around 400 man-hours, ar at least two months for a single
person.
The time commitment to keep the Vulnerability aggregate current is continuous and substantial.
Researching and scrubbing a single vulnerability takes at least ten minutes, and in many cases may take
an hour or more. CERT reported over four thousand new vulnerabilities in 2002 [CC03], so at a rate of
eighty vulnerablities/week, one should schedule at a minimum of 12 man-hours a week, although twenty
to thirty man-hours a week would be more advisable.
1.4.4 QSRA empirical results
As implemented, software inventorying time grows linearly with the number of audited hosts and the
number of installed software packages and the number of records in the database networks which records
7/30/2019 Bila r Phd Thesis
21/143
CHAPTER 1. INTRODUCTION 9
the software makeup of the network. Database throughput to networks becomes a non-negligible factor
when the number of records in the database is high.There is no statistically significant time difference
between auditing wireless hosts and landline hosts.
QSRAs risk assessment model calculated that across all the original audited operating systems, four
to six months after their respective release data, the probabilities are very high (66% to 99%) that
an attacker can conduct a full consequence compromise, remotely and locally. One or two faults are
enough to cause the probabilities to rise to 50% and more. Almost all, if not all, of the faults have to
be eliminated to have an appreciable effects on risk probabilities.
Risk management analysis for remote risk probabilities indicates that, given a moderate fault count,
QSRAs highest risk analytic risk mitigation strategy consistently outperforms the simpler strategy of
choosing software with the highest vulnerability count. Highest risk outperforms the undifferentiated
highest count strategy for at least four out of the six tested operating systems and for four out of five
fault consequences. The most compelling effects are found on the Windows system, probably due to the
comprehensiveness of Windows-style patches. The effects on Linux-family systems are less pronounced.
Both strategies have minimal effect on risk probability reduction across all audited hosts when the
vulnerability count is high.
7/30/2019 Bila r Phd Thesis
22/143
Chapter 2
Design of the QSRA system
2.1 Findings
The ists_icat database holds data about software characteristics and its associated vulnerabilities. It
is used to provide choices for the software identification, data and parameters for risk assessment and
options for risk management. It has two main functional aggregates: Systems which holds data on
software programs, and Vulnerabilities, which holds data on the softwares associated vulnerabilities.
Its comprehensiveness is key. Lower estimate for necessary risk management data for 40 software class
categories with 5 substitutes programs each across 3 families (Windows, BSD, Linux) is 600 entries. The
lower estimate for risk analysis is far higher, since there are thousands of software packages for each
family. The lower bound record size of the Systems aggregate, which holds data data on software
programs, for production quality deployment is estimated to be around 3000 entries, namely for three
OS families with fifty functionality groups containing twenty records each. This number is necessary so
that the risk management optimizer has a sufficiently large pool from which to assemble an alternative
software makeup. In steady-state, and with an input mask, it takes no more than 5 minutes to amassand assess one entry and another 3 minutes to write it to the database, making the time commitment
to populate the Systems database to be around 400 man-hours, ar at least two months for a single
person.
The time commitment to keep the Vulnerability aggregate current is continuous and substantial.
Researching and scrubbing a single vulnerability takes at least ten minutes, and in many cases may take
an hour or more. CERT reported over four thousand new vulnerabilities in 2002 [CC03], so at a rate of
eighty vulnerablities/week, one should schedule at a minimum of 12 man-hours a week, although twenty
10
7/30/2019 Bila r Phd Thesis
23/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 11
to thirty man-hours a week would be more advisable.
2.2 Overview
Conceptually for QSRA, the network is not a collection of hardware. From QSRAs point of view, the
network is a collection of IP addresses with associated software. Figure 2.1 is an illustration of this view.
Figure 2.1: Software view of the network
The software on each IP-enabled device can be classified into three groups.
Services Services are running programs that have one or more sockets bound to listening ports. They
react to connection requests from the other devices. An example would be a httpd daemon listening on
port 80, the common web service port.
7/30/2019 Bila r Phd Thesis
24/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 12
Applications The main difference between services and applications is that the latter are not listening
on any port. Rather, the term encompasses installed software on the device. This software may or may
not be actively executing. An example would be MS Office.
Operating system The operating system is the most important software on a device, since it provides
the operating environment for other software.
2.2.1 Software quality, faults, failures
Software code quality has not markedly improved. Although it is impossible, both on theoretical and
practical grounds, to build completely secure systems, commercial off-the-shelf (COTS) software has been
exhibiting the same source code fault density for the past twenty years. Software fault density remains
between 0.1 and 55 faults per thousand lines of source code (see Table 1.2 on page 2). Exacerbating
circumstances include increases in code size (Windows 2000 has an estimated 35 million lines of source
code [Whe02]) and in the number of interaction pathways between the various software subsystems.
Perhaps the most aggravating fact is the unwillingness of designers and implementers alike to use proven
techniques to build secure code [HL02, pp.19-57]. The net result has been an increase in the number of
potentially serious software faults.
A fault is a defect in a program that, given certain conditions, causes a failure [MIO90, p. 6]. A fault
may cause many failures; likewise, a failure may be caused by more than one fault. Some failures can
be exploited by attackers. Faults, as well as failures, may be classified according to type. Drawing from
the taxonomy used by ICAT, SecurityFocus, Landwehr and Bishop [oSN02] [Sec02] [LBMC94] [Bis00],
the following fault types can be distinguished:
1. Input validation error: A fault is characterized as an Input Validation Error if the input being
received by the software is not properly checked, thereby enabling a vulnerability to be exploited
by a certain input sequence. An example of this would be a buffer overflow. A different example
would be a boundary condition error - the canonical case being the Y2K problem, where the 2
digit year assumption was exceeded. Another example would be insufficient validation of HTTP
requests that may allow execution of code.
2. Access validation error: A fault is characterized as a Access Validation Error if software is vul-
nerable because the access control mechanism is faulty. The problem lies not with the configuration
of the access control mechanism but with the mechanism itself. An example would be an error
that enabled authentication to an FTP service using a non-existent username/password.
7/30/2019 Bila r Phd Thesis
25/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 13
3. Exceptional condition handling error: A fault is characterized as an Exceptional Condition Han-
dling error if software becomes vulnerable due to an exceptional condition that has arisen. The
handling (or mishandling) of the exception by the software enables a vulnerability. An example of
this would be an error in router software that allows remote attackers to cause a denial of service
via a special BOOTP packet that is forwarded to broadcast MAC addresses.
4. Environmental error: A fault is characterized as an Environmental Error if unanticipated syn-
ergies causes the software to be vulnerable. This may be due, for example, to an unexpected
interaction between an application and the operating system or between two applications on the
same host. An example of this would be an interaction between encryption software in conjunction
with encrypted file systems, which result in the creation of cleartext temporary files that cannot
be wiped or deleted due to the strong permissions of the file systems.
5. Configuration error: A fault is characterized as a Configuration Error if user-controllable settings
cause the software to become vulnerable. This fault is caused by controllable configuration, not
inherent design errors. An example of this error would be unchanged, well-known default user
accounts on remotely accessible software.
6. Race condition: A fault is characterized as a Race condition if the non-atomicity of a security
check causes the existence of a vulnerability. An example would be software checking the validity
of an operation against a security model. However, between the time of the security check and
the actual operation, some changes occur that invalidate the security model. Attackers can then
perform illegal operations like writing to the password file.
7. Design error: A fault is characterized as a Design Error if there exists no errors in the software
implementation or configuration. Rather, the initial design is faulty. A example of this error would
be a default root password that cannot be changed.
The assumption is made that some fault types are harder to exploit than others; taking advantage of
a race condition requires on the whole more expertise and finesse than exploiting an input validation
error, for instance.
Faults cause failures, which will henceforth be called consequences. Consequences may be one of five
different types (see Table 2.3 on page 17) .
Each of these consequences entails a potential loss, which is dependent on the IP-enabled device on
which the software resides. The determinants are, intuitively, data and/or functionality of the IP-enabled
device. An availability breach on a public Web server may be much more serious than an identical breach
7/30/2019 Bila r Phd Thesis
26/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 14
on an internal Web server, for instance. Similarly, a confidentiality breach on a database hosting customer
data may spell ruin, whereas the impact of the same confidentiality breach on a database hosting archival
information may be minimal. Presently, there are no known ways to automate the generation of the
loss functions; there has to be a risk manager who can gauge the value of each IP-enabled device for the
networks information/business/mission purpose and set the appropriate magnitudes for the consequence
losses.
2.3 Process sequence
The software information of the network is gathered and processed in three steps, as shown in Figure
2.2:
Figure 2.2: QSRA approach
1. Vulnerability Assessment: Conduct an exhaustive and detailed inventory and identification of
software on a network of interest (operating systems, services, and applications)
7/30/2019 Bila r Phd Thesis
27/143
7/30/2019 Bila r Phd Thesis
28/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 16
2.4 Risk Assessment
2.4.1 Preliminary example
Risk is commonly defined as the product of probability and severity of adverse effects, and the most
common approach to quantify risk is a single figure - its expected value [Hai98, p. 29]. Mathematically
speaking, given a random variable X with probability function px(x) and loss function lx(x), the expected
risk value in the discrete case is equal to
E(x) =
pX(x)lX(x) (2.1)
It is apparent that these are generic probability weighed averaging formulas. Its semantic specializa-
tion into an expected value of risk occurs through the loss function. The unit of the expected risk value
is the unit used by the loss function and could be downtime, cost, credibility, etc.
As a preliminary example, the simplified risk of attack consequences on a host that is running one
application is shown in Table 2.1. The assumption is that an attacker can exploit three vulnerabilities,
with the following consequences for the host: Full compromise (equivalent to root access), Availability
(commonly known as Denial of Service) and Confidentiality (as an example, reading a password file).
Let X be a discrete variable denoting the individual fault consequence. Let the probability function
be the probability that the vulnerability is targeted by the hacker. Let the loss function be defined as
general damage, in US$ to the function the host serves.
Table 2.1: Hypothetic risk of a hostX pX(x) lX(x) RiskX(x)Full 0.1 $100 $10Availability 0.8 $10 $8Confidentiality 0.1 $50 $5
With the probabilities and expected values shown in Table 2.1, the expected value of risk E(X) of ahacker attack is $10 + $8 + $5 = $23.
2.4.2 QSRA risk calculation
Since the unit of analysis is software, this step gauges the risk of the deployed software on the network.
The first section Basic Risk describes the software risk without taking interactions into account. Basic
remote and local consequence risk is calculated this way. The second section Extended Risk takes
interactions between programs into account. This refines the risk of remote attack risk by including
7/30/2019 Bila r Phd Thesis
29/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 17
Table 2.2: Fault typesType DescriptionInput validation error improper input sequence validation
Buffer overflow error expected buffer length exceededBoundary condition error assumed boundary exceeded
Access validation error faulty access control mechanismExceptional condition mishandling incorrect exception handlingEnvironmental error discrepancy between test and installation en-
vironmentConfiguration error unsafe configuration by userRace condition non-atomicity of security checkDesign error faulty basic design
Table 2.3: Consequence types
Type DescriptionAvailability Some software or data is unavailable for legitimate useConfidentiality Unintended read privilege to dataIntegrity Unintended write privilege to dataProcess User access over software and dataFull Full access over all software and data on a device
interactions between services and applications.
Basic Risk
Basic risk calculates the basic remote and local risk, without taking interactions between programs into
account. The risk equations presented in this section focus on services (remotely accessible software),
but can be applied without loss of generality to applications (locally accessible software).
Let C be the set of consequences. Let V be the access venue (remotely exploitable or locally ex-
ploitable); V {vr, vl}. Let loss(i,c) be the consequence loss for a specific IP-enabled device i. Let
pv,c(t) be the probability of consequence c C, v V. Then, we have
Risk(v,i,c)(t) = pv,c(t) loss(i,c) (2.2)
where c C, i I
The consequence losses, loss(i,c), are domain-dependent on the data and the functionality of the IP-
enabled device and have to be set by the risk manager. The probability of consequence c is the com-
bined failure probability of the services residing on the device that have faults that potentially entails
consequence c. This probability is time dependent and assumed to be increasing monotonically.
For remotely exploitable risk (access venue v = vr), let Kc be the set of services with faults that
entail consequence c. Let pvr ,c(t), c C be the aggregate probability of the set of services Kc leading
7/30/2019 Bila r Phd Thesis
30/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 18
Figure 2.3: Fault tree for consequence c
to consequence c. Let pc,k(t) be the exploit probability of consequence c of a specific service k. Then,
we have
pvr ,c(t) = 1 kKc
(1 pc,k(t)) (2.3)
where c C
Equation 2.3 represents the failure of independent components in series [Ros93, pp. 418, 434]. The
components are the software in Kc and 1
(1 pc,k(t)) is the combined failure probability of the
services residing on the device that have faults that potentially entail consequence c. This is intuitively
reasonable, since the faulty services constitute independent pathways toward consequences. The more
pathways you have, the more probable that this consequence will materialize. Figure 2.3 illustrates a
consequence fault tree with two service pathways toward consequence c.
The same pathway argument applies to multiple faults in a single service as well. Service k may
have many faults which may lead to consequence c. For instance, an input validation fault and a race
condition fault both could lead to a full (root) compromise. Let Fc,k be the set of faults in service k
that lead to consequence c. Let qf(t) be the exploit probability of fault f, f Fc,k, at time t . Then,
we have
pc,k(t) = 1
fFc,k
(1 qf(t)) (2.4)
(2.5)
qf(t) denotes the probability that an attacker will be able to exploit fault type f at time t. Let t0 be
the time when a fault has been discovered. pautomated(t,f) be a probability function that an automated
tool to exploit this fault has been developed since tdesc, the time the vulnerability was discovered. The
7/30/2019 Bila r Phd Thesis
31/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 19
function is assumed to increase monotonically from time tdesc till tposted, the time the exploit tools are
publicly available and then stabilize. Let Fc,k be the set of faults in service k that lead to consequence
c Let pmanual(f) , f Fc,k, be the proportion of attackers that can exploit this fault without an
automated tool. These are skilled attackers who do not need an automated tools. Let ptool(f), f Fc,
be the additional proportion of attackers that can exploit this fault with an automated tool. As time
goes by, it becomes more likely that this set of attackers learn how to exploit this vulnerability. Then
we have
qf(t) = pautomated(t,f)proptool(f) + propmanual(f) (2.6)
The basic remote and local risk over all consequences can be calculated this way
Extended Risk: Services and Applications
So far, remote and local consequence risk have been calculated without taking interactions between
programs into account. Interactions between programs enable escalation exploits. Escalation exploits are
a manifestation of negative synergies between vulnerabilities. These are attacks that use vulnerabilities in
one software program - usually a service - as a stepping stone to exploit vulnerabilities in another software
program, usually an application. Figure 2.4 illustrates the pathways of two types of escalation exploits.Inclusion of escalation exploits affects the consequence probability pvr,full by potentially increasing the
risk of a remote full consequence attack.
Remote2Local-User2Root Also known as R2L-U2R, a pathway in which a service is remotely ex-
ploited that has a fault entailing a process, and/or integrity privilege compromise. This privilege is then
used by an attacker to access an application a with a fault that can entail a full consequence compromise.
Remote2Local-Remote2Root Also known as R2L-R2R, it represents a pathway in which a service
is remotely exploited that has a fault entailing a process and/or integrity privilege compromise. This
compromise enables a changes in the state of some service, creating a configuration error fault within
that service that enables an attacker to compromise service k2, leading to a full consequence breach.
Let Escalation be the set of the two Remote2Local-Remote2Root and Remote2Local-Remote2Root
escalation events. Let P I be the set of consequences that affect non-root level of access control; P I =
Process
Integrity. Let svr,full(t) be the remote full consequence probability, resulting from services
with configuration faults. Let tvr ,full(t) be the remote full consequence probability, excluding services
with configuration faults. We have
7/30/2019 Bila r Phd Thesis
32/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 20
Figure 2.4: Escalation exploits
pEscalation(t) = pvr,PI(t)(pvl,full(t) + svr ,full(t) pvl,full(t) svr ,full(t)) (2.7)
Revising the full consequence probability pvr,full to include these escalation exploit pathways is
straightforward. Combined with the non-escalation attack event that leads to full compromise, we have
pvr,full(t) = tvr,full(t) + pEscalation(t) tvr,full(t) pEscalation(t) (2.8)
Aggregation Each consequence risk is measured in $US. The aggregate risk of the IP-enabled device
i is then the summation of 2.2 on page 17 for all consequences c C, C being the set of consequences
in Table 2.3 on page 17. The device risk is then
Risk(i) =cC
Risk(i,c) (2.9)
2.5 Risk Management
Once the risk metrics have been calculated for the networks constitutive IP-enabled devices, the next
step is to implement measures to reduce the networks risk profile. This is the risk management step.
Subject to cost, functionality, and risk limit constraints, the risk profile optimization procedure consists
of replacing, patching and/or removing software. The QSRA application, together with its optimization
module, works with the vulnerability database to assemble an alternative software makeup.
The conceived optimization problem is an instance of a Set Partitioning Problem (with base con-
7/30/2019 Bila r Phd Thesis
33/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 21
Table 2.4: Functionality (Class) groupsaccess control admin tool applicationapplication server authentication basic OS toolsboot server bridge (hardware) bridge (software)bug tracking communication compressiondatabase dhcp encryptionfile file sharing firewallfirmware IDS internet clientISDN library logmail mail client mail servername NAT network access (Ethernet)network access (non-Ethernet) network storage newsnon-TCP server OS kernel packet filterpacket sniffer printer spooler programming languageremote access router (hardware) router (software)
sandbox security module server daemonsuperdaemon switch (hardware) switch (software)task scheduler text editor timeunspecified web web proxywireless access point . . . . . .
straints). See Eso for an introduction [Eso99]. The problem statement is formulated as an Integer
Linear Program (ILP) [BHM77, Ch. 9]. The ILP for one IP-enabled device a is described, without loss
of generality.
There are four general sets of constraints: OS compatibility, functionality, risk and cost limits. The
QSRA OS compatibility ensures that the QSRA application solver module only chooses compatible
software. Functionality ensures that the software set chosen by the solution retains pre-optimization
functionality. If host a had a web server, a database server and a ftp server before optimization, it has
to offer this functionality afterward, too. Table 2.4 lists the functionality (class in QSRA vernacular)
groups in the prototype QSRA implementation. This list is not complete and can be amended; it is merely
a listing of functionality of the inventoried software in section 4.3. Risk limits are set by the risk manager
and reflect the upper acceptable bounds for the risk consequences. Cost limits are the costs associated
with software - monetarily, they include installation, configuration, maintenance, upgrade, training,
and acquisition costs, expressed in US$. Timewise, they include downtime, installation, configuration,
training and/or maintenance costs, expressed in hours. They are set by the risk manager, and specify
an upper bound. The risk manager should be work in conjunction with a senior network administrator
and a senior executive in order to accurately assess the risk and cost dimensions.
7/30/2019 Bila r Phd Thesis
34/143
7/30/2019 Bila r Phd Thesis
35/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 23
subject to
F(1)
x(1)
f(1)
rhs (2.15)...
F(k)x(k) f(k)rhs
V(1)E(1)x(1) + . . . +V(k)E(k)x(k) vrhs (2.16)
R(1)x(1) + . . . +R(k)x(k) rrhs (2.17)
such that
xi binary, 1 i k
The feasible solutions are returned as indicator vectors xi, 1 i k. If no feasible solution can be
found, the risk and cost limit constraints (2.12) (2.16)(2.13)(2.17) may have to be relaxed.
2.6 Database
QSRAs knowledge repository consists of two relational databases: ists_icat and networks. Both
databases were designed to be modularly expandable, easily mutable and free of the data redundancies
found in other databases. Hence, all relations were designed to be, at a minimum, in third normal form.
For more information on relational database design and normalization, see Rolland and Gillenson [Rol98,
pp.72-85] [Gil87].
2.6.1 ISTS ICAT
ists_icat holds data about software and its associated vulnerabilities. It is used to provide data and
parameters for software identification, risk assessment and risk management. The database contains 40
tables which can b e separated into four functional aggregates.
System This set of 17 tables holds data on software programs known as Systems. The inventoried
services and application executables are mapped to these Systems, down to the patch level.
Vulnerabilities This set of 13 tables holds data on vulnerabilities.
Relations This set of 5 tables links the Systems to Vulnerabilities.
7/30/2019 Bila r Phd Thesis
36/143
7/30/2019 Bila r Phd Thesis
37/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 25
Table 2.5: Vulnerabilities data sourcesResource Name URL(e),(l) SecuriTeam http://www.securiteam.com
(e),(l) Packet Storm http://packetstormsecurity.nl/(e),(l) New Order http://neworder.box.sk/(e),(l) Security Tracker http://www.securitytracker.com/(t),(a),(c), (e),(l) Network Security News http://www.security-update.com(t),(a),(c), (l) SecurityFocus http://online.securityfocus.comportal for (t),(a),(c),(e),(l) CVE http://cve.mitre.org(t),(a),(c) ISS X-Force Database http://www.iss.net/security_center(t),(a),(c) ICAT Metabase http://icat.nist.gov(t),(a),(c),(l) MARC http://www.theaimsgroup.com(t),(a),(c), (e),(l) @stake http://www.atstake.com/researchLegend (t)ype, (a)ccessibility, (c)onsequences, (e)xploits,(l)ifecycle
Risk Assessment This set of 5 tables holds entries pertaining to the calculated risk measure of a
network: The individual software risk, the risk of the IP-enabled device hosting that software, and
the aggregate network risk.
Risk Management This set of 8 tables holds the entries pertaining to risk management scenarios and
the alternative software makeup on the network
This database holds extremely detailed information about the software setup on the network. In
effect it is a gold mine for any attacker. As such, access to this database should be restricted to theQSRA application and senior trusted personnel. As a technical security measure, the database contents
as well as the communication stream between the QSRA application and networks should be encrypted.
A detailed makeup of the databases can be found in appendix B on page 92.
2.7 A QSRA example
Assume we have a host George that functions as a web and database server for the PUSH company. The
PUSH companys risk manager Susan knows that an immaculate, omnipresent George is vital for PUSHs
public image. The integrity of the data residing on it is important as well - but not its confidentiality,
since the information is public anyway. Of course, Susan would not want to see the server processes
taken over (and perhaps redirect the user to a rival company) or worse yet, have George be taken over
completely and used as a zombie in a DDoS attack against PUSH or another company (and be subjected
to potential liability suits). Susan thus assesses the loss consequences for George for the company. QSRA
calculates the risk (attack) probabilities after conducting the software inventorying and identification.
Software inventorying shows that George is a Debian Linux 3.0 host with service Apache 1.3.26 running,
7/30/2019 Bila r Phd Thesis
38/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 26
Table 2.6: Risk management results for GeorgeSoftware Functionality Replacement software Migration cost Remote riskMySQL 3.23.50 database server MySQL 4.0.5 $3,000 $5,700
Apache 1.3.26 web server Apache 2.0.35 $2,000 $15,200
Table 2.7: Remote risk limits for GeorgeConsequence Type Remote risk limit Risk for original software Risk for new softwareAvailability $3000 $9,900 $2,700Integrity $3000 $9,500 $0Confidentiality $500 $0 $0Process $8,000 $27,300 $5,700Full $14,000 $49,500 $12,500
tied to a back-end running MySQL 3.23.50 database server. The respective vulnerabilities of these
services is shown in Table 2.11. Risk assessment reveals that its attack probabilities are pretty high.
Table 2.8 shows the data from Susan and the QSRA process so far. The total remote and local attack
consequence risk is $96,200 and $61,000, respectively.
Susan wants to keep Georges functionality. After consulting with management and senior network
administration, she sets the acceptable risk limits for George (see Table 2.7). She is not that concerned
with insiders (personel hires trustworthy people), but is concerned with outside attackers. She is willing
to spend roughly half of her calculated remote attack risk fix to change her risk profile, around $50,000.
With the risk and cost limits set, Susan invokes the risk management process. The optimizer chooses
two new programs, Apache 2.0.35 and MySQL 4.0.5, to substitute for their respective (more vulnerable)
predecessors. The vulnerabilities of these new services is shown in Table 2.10. Their migration costs
and remote risk are shown in Table 2.6. The risk profile is calculated for these services, as shown in
Table 2.9. The total remote and local attack consequence risk is now $20,900 and $0, respectively. This
represents a 78% remote risk reduction.
All of Susans requirements and PUSHs organizational needs have thus been met: The costs of
migrating to the new software are $5,000, which meets the cost limit of $50,000. The consequencerisk limits have not been exceeded, as is shown in Table 2.7. Since functionality is preserved, PUSHs
managers are content with the decision, as well. If no feasible alternative software makeup had been
found, either the migration costs or the risk limits would have had to be relaxed, and the risk management
process re-invoked.
7/30/2019 Bila r Phd Thesis
39/143
CHAPTER 2. DESIGN OF THE QSRA SYSTEM 27
Table 2.8: Risk assessment: Consequence losses and consequence probabilities for GeorgeType Consequence loss remote attack local attack
for George probability risk probability riskAvailability $10,000 0.99 $9,900 0.95 $9,500Integrity $10,000 0.95 $9,500 0.40 $4,000Confidentiality $0 0.87 $0 0.40 $0Process $30,000 0.91 $27,300 $0Full $50,000 0.99 $49,500 0.95 $47,500
Table 2.9: Risk management: Consequence losses and consequence probabilities for GeorgeType Consequence loss remote attack local attack
for George probability risk probability riskAvailability $10,000 0.27 $2,700 0 $0Integrity $10,000 0 $0 0 $0Confidentiality $0 0 $0 0 $0Process $30,000 0.19 $5,700 0 $0Full $50,000 0.25 $12,500 0 $0
Table 2.10: Vulnerabilities of Apache 2.0.35 and MySQL 4.0.5Type Apache 2.0.35 MySQL 4.0.5
remote local remote localAvailability 3 Integrity Confidentiality Process 2 Full 1
Table 2.11: Vulnerabilities of Apache 1.3.26 and MySQL 3.23.50Type Apache 1.3.26 MySQL 3.23.50
remote local remote localAvailability 1 1 2 Integrity 1 Confidentiality 1 Process 2 3 1Full 1
7/30/2019 Bila r Phd Thesis
40/143
Chapter 3
Implementation of QSRA system
3.1 Findings
Java as an implementation language proved its worth in cross-platform development. It cannot be
stressed enough how much time is saved by the lack of the tedious memory leaks and pointer manipulation
that plague C and C++. Text parsing, however, is not Javas forte, and hence QSRA client development
took a disproportional amount of time.
The decision to design a relational, as opposed to a flat database was found to have been very
fortunate. Having all data relations in third normal form is advisable, since this enables additions and
mutations to be accommodated with few if any changes to the existing design.
3.2 Overview
As a principle, the QSRA application, clients and SQL database were implemented using royalty-free
software, with the simplest tools available. The implementation was to be portable and small. As such,
the following products were used to implement the QSRA application and client:
Language Suns Java 1.3.1, ensuring portability and ease of implementation[Mic01b]
Database MySQL ABs MySQL 3.23, the worlds most widely used open-source database [AB00]
Tools Victor Abells lsof 4.47, an software-to-port mapper for Unix to identify service executables;
Foundstones fport 1.33, a software-to-port mapper for Windows NT/2000/XP to identify service
executables; Michel Berkelaars lp_solve 2.0, a Java implementation of a mixed integer linear
program solver [Abe00] [Fou00] [Ber96]
28
7/30/2019 Bila r Phd Thesis
41/143
CHAPTER 3. IMPLEMENTATION OF QSRA SYSTEM 29
The current QSRA and database implementation was meant to set up the framework for the method-
ology, a proof-of-concept software engineering implementation. It took more than eighteen months of
developing, testing, redesigning and debugging. The application contains around 6000 lines of source
code (200 kB of data), and the client applications around 900 lines (30 kB of data). The whole archived
file with the GUI and other helper classes totals 980kB of data.
The database went through a few redesigns before and during implementation. Implementing the
schemata was straightforward; thanks to the initial design decision to put all the relations in third
normal form, additions and mutations in the schema were easier to make. Around 400 system and 200
vulnerability records were inserted into the prototype ists_icat database.
Not implemented QSRA was first implemented without escalation attacks (negative synergy effects
between services and applications) in mind. The necessity to introduce escalation attacks became ap-
parent in the course of the research as more attention was given to vulnerability exploits and their
interactions with installed applications. The decision was made to incorporate escalation attacks into
the methodology, but defer implementation in the prototype. A work-around numerical framework,
which realized the updated methodology, was designed and implemented using Microsoft Excel.
The prototype QSRA client implementation returns data on services and operating systems only,
applications have to be inventoried manually. Well-behaved applications follow recommended instal-lation procedures. In the case of Windows machines, proper installation means that meta-data about the
installed software is recorded in the Windows Registry under the key HKEY_LOCAL_MACHINE\SOFTWARE.
In the case of Linux and BSD machines, there are similar database-like structures (such as packages.rpm
for Linux or _pkg-name_ for BSD) that get updated when standard installation procedures are followed
properly.
As a concomitant to the client not auditing the applications, the prototype QSRA application does
not take escalation attacks into account in its risk analysis calculations. The lack of this feature in
the prototype did not limit the experimental approach, it just added manual labor to an otherwise
automated risk calculation process.
Implementation and testing of inventorying applications on a client should take four weeks. Tieing
it into the QSRA application and testing this feature is more time-consuming and is estimated to take
at least two months. For risk management, the prototype QSRA application implementation does not
take time cost constraints into consideration when performing the risk management optimization. The
reasons for this was the belated realization that time can just be another $US metric after conversion.
This avoided the added complication of having to weight two metrics in the formulation of the objective
7/30/2019 Bila r Phd Thesis
42/143
7/30/2019 Bila r Phd Thesis
43/143
VulnerabilityAssessment
PRTCL
IP
NAME
OS
PORT
tcp
1.1.1.2
httpd
Linux2
.4
80
tcp
1.1.1.2
xinted
Linux2
.4
21
tcp
1.1.1.4
svchost
Win2K
135
udp
1.1.1.4
lsass
Win2K
500
tcp
1.1.1.5
svchost
WinXP
5000
udp
1.1.1.5
WinXP
38037
msgsys
Unix
lsof
WinXP
fport
Win2K
fport
(TCP)4447
,
4448
(1)
(2)
(3)
MSIIS5
80
MSIIS5
.1
VulnerabilityAssignment
tcp
1.1.1.2
httpd
Linux2
.4
80
PRTCL
IP
NAME
OS
PORT
tcp
1.1.1.2
xinetd
Linux2
.4
21
port
version
patch
MSIIS3
80
MSIIS3
.0andprev.
Apache1
.3
80
Apache1
.3.2
0
MaphttpdtoApache1
.3.2
0
MySQL
ists
_icat
networks
SysVerPatch
commonP
ort
CP2SVP
IP2SVP
(1)
(2)
(4)
(3)
(5)(5)
MySQL
ists
_icat
networks
Vuln
Vuln2ExploitFactors
IPRisk
IndivRisk
RiskAssessment
NetworkRisk
IPRisk
IndivRisk
Full
Trust
Process
Integrity
Availability
Confidentiality
Consequence
Risk
340
200
12001
00 0 90
Full
Trust
Process
Integrity
Availability
Confidentiality
Consequence
Risk
600
260
2300
100
120
300
IPaddress
Risk
1.1.1.4
1290
1.1.1.2
3680
1.1.1.5
230
Apache1..3
..20
1.1.1.2
(1)
(2)
(4)
(3)
MySQL
ists
_icat
networks
Cost
Class
IPRisk
IndivRisk
RiskManagement
Scenario
IP
Risk
Improv
Scen2
1.1.1.2
3680
0%
Scen1
1.1.1.2
2750
25%
Scen1
1.1.1.4
800
38%
Full
Trust
Process
Integrity
Availability
Confidentiality
340 2001200
100 0 90
consequence
Lim
it
money
time
12000
148
resources
Lim
it
IP
Risk
1.1.1.2
3680
Scenarios
1.1.1.4
1290
1.1.1.5
230
Baseline
optimize
System
Scenario
(1)
(2)
(5)
(4)
(6)
(3)
QSRA
connectstoassessorinstalledinclient
onTCPport4447/4448.
(3)QSRA
parseslsof
andfportdataand
displaysthelistofser
vices(calledhere
CommunicationProcess,CP)forevery
IP-enableddeviceinventoried
Clients(IP-enab
le
devices)send
raw
lsof,
fport
data
about
open
ports
and
the
programsbound
tothem
(1)
Process
Port
Proto
Path
svchost
135
TCP
svchost.exe
persfw
44334
TCP
persfw.exe
(2)CommandDeviceSizeNodeName
xinetdTCP*:21
(LISTEN)
xinetdTCP*:44
3(LISTEN)
httpdTCP*:80
(LISTEN)
(1)QSRA
requestsselectionofsoftware
(calledhereSystemso
rSVPs)from
ists
icat.
(3)QSRA
displayslistofprogramsandwaits
forusertomapCPst
oSVPs
(4)Mappingdataissenttonetworks
(2)ists_
icat
returnsalistofprograms.
(5)networks
sto
resaScenario:
theCPs
andSVPs,aswellastheCP-to-SVP,SVP-
to-IP,IP-to-netw
orkmappings.
(1)QSRA
requeststh
enumericSVP
vulnerabilityinformat
ionforthecurrent
networkscenario.
(3)QSRA
calculatestheIndivRisk,IPRisk
andNetworkRiskofthescenario,usingdata
from