Research ArticleSensitivity of Importance Metrics for Critical Digital ServicesGraph to Service Operatorsrsquo Self-Assessment Errors
Mariusz Kamola
RampD Department NASK National Research Institute Kolska 12 01-045 Warszawa Poland
Correspondence should be addressed to Mariusz Kamola mariuszkamolanaskpl
Received 22 March 2019 Accepted 31 August 2019 Published 23 September 2019
Academic Editor Clemente Galdi
Copyright copy 2019 Mariusz Kamola is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Interdependency of critical digital services can be modeled in the form of a graph with exactly known structure but with edgeweights subject to estimation errors We use standard and custom centrality indexes to measure each service vulnerabilityVulnerability of all nodes in the graph gets aggregated in a number of ways into a single network vulnerability index for serviceswhose operation is critical for the state is study compares sensitivity of various centralities combined with various aggregationmethods to errors in edge weights reported by service operators We nd that many of those combinations are quite robust andcan be used interchangeably to reect various perceptions of network vulnerability We use graphs of source lesrsquo dependenciesfor a number of open-source projects as a good analogy for real critical services graph which will remain condential
1 Introduction
Correct operation of digital services and infrastructures hassince long become critical for societies and therefore de-mands coordinated actions for maintenance and incidentresponse e Directive on Security of Network and In-formation Systems (NIS [1]) by the European Parliamentprovides a framework for coherent implementation of se-curity measures by European Union member states Due tothe scale and dynamics of digital networks eective andecient protection of their operation must be assisted byintelligent decision support systems operating at nationallevel Such systems should be
(i) Complete ie possessing information about allcritical services in the country
(ii) Automated ie minimizing human factor in dailyoperations as well as in network model construction
(iii) Coupled ie exchanging information at in-ternational level
Researchers industry and regulators stay aware of theabove challenges and come up accordingly with ideas of suchsystems (cf eg [2 3] and references therein) NotablyPolish government is supporting National Cybersecurity
Platform (NPC) a RampD project whose goal is to address therst two of the above issues ie actually implement anddeploy a system supporting security operation centers(SOCs) A crucial phase of NPC operation is creation of agraph modeling interdependent digital services run byvarious operators is process is done semiautomaticallyfrom SOC perspective service dependencies are discoveredin depth-rst search fashion by interviewing subsequentoperators with online questionnaires
Apart from privacy and organizational obstacles lling aquestionnaire can be a challenge of its own for an operatorFor a given own service an operator is asked to reportservices preconditioning its correct operation and to pro-vide estimates of their impact on own service in terms ofavailability condentiality and integrity (CIA) [4] Whilethe earlier is quite straightforward (as it can be based oninspection of business contracts service level agreements(SLAs) and invoices or any other formal documents)measuring the magnitude of service dependencies is proneto errors and bias But on the other hand the nationalcritical services network model is built exactly with this infoe model includes routines for vulnerability calculation foreach service Vulnerabilities in turn get combined into ascalar index of overall network vulnerability
HindawiSecurity and Communication NetworksVolume 2019 Article ID 7510809 8 pageshttpsdoiorg10115520197510809
Our goal is to examine how the above process is sensitiveto incorrect information about mutual service impact asreported by operators with the assumption that the struc-ture of the network is known fully and correctly Such in-formation is crucial because that the scalar index value willbe reported to SOCs and consequently will play the role ofthe main threat indicator
We organized the paper as follows A network model ofservices is presented in the remaining part of this section Asuite of methods for calculation of service vulnerability and foraggregation of vulnerabilities into a scalar vulnerability indexare described in Section 2 It is followed by the section withdiscussion of results (Section 3) and we conclude in Section 4
e network of interdependent digital services ismodeled as a directed graph
G(V E) (1)
where V is a list of ordered vertices representing servicesV (v1 v2 v|V|) and E is a list of ordered edges eij isin E
if operation of service vi influences operation of service vje impact of such influence is defined by the operator ofservice vj on a discrete scale from 1 to 10 All the informationabout the graph structure and service impact can beexpressed conveniently by adjacency matrix A whose ele-ment aij is equal to the impact value or zero if there is noedge eij Here we assume to operate with respect to only oneimpact aspect eg howmuch the loss of service i availabilityinfluences service j availability ere can be nine suchaspects in total C I A times C I A It is possible to combinethem all into one scalar coefficient when some assumptionson their meaning are made eg if one considers them asprobabilities
Such graph model extension with edge weights repre-sented actually by a matrix of up to nine aspects of impactdemands developing new graph algorithmsmdashor picking upone of the aspects like it is done in this paper It makes themodel universal enough to accommodate both digital servicesand physical infrastructure elements In the latter case onerefers to just the availability aspect For example availability ofbackup power supply may influence availability and integrityof the physical access control system hence an operator hasto address the influence in two aspects A⟼A and A⟼ I
Topology of a service graph represents existence ofservice interdependencies while edge weights stand forintensity of those interdependencies When combined theymake it possible to calculate the overall vulnerability of eachservice ere are many ways such vulnerability could beformulated we express its definition as
r ≔ Φ(A) (2)
whereΦ is some function defined over adjacency matrix thatcomputes vector r of vulnerabilities for each servicerespectively
While r contains complete information about vulnera-bility of each service a single scalar index c of overall networkvulnerability would be much more convenient in everydayuse Like for individual vulnerabilities its calculation can beaccomplished in many ways we denote this process as
c ≔ Γ(r) (3)
where Γ is some function defined over vulnerability vectore major practical problem concerns credibility of c
which is computed indirectly from A whose values are notobjective ey come from the questionnaires and are aresult of self-assessment process by service operators whoseaccuracy depends on their cybersecurity awareness andmaturity of methodologies used in service impact estima-tion An objective approach to vulnerability estimationwould require excessive provocative tests on critical servicesor postmortem analyses both of which are costly andundesirable
erefore we must assume that contrary to structure ofservice dependencies that is known and correct the reportedimpact values 1113957A differ from true ones by some errors
1113957aij min 10max 1 aij + ξ1113872 11138731113872 1113873 if aij gt 0
0 otherwise
⎧⎨
⎩ (4)
where ξ is realization of a random variable with uniformdiscrete distribution U minus N N Here N is the maximumimpact estimation error in the ten-star rating scale Note thatin (4) we curb disturbed rating within the original scale ofone to ten stars Consequently we denote calculated vul-nerabilities of serivces for the reported values of 1113957A as
1113957r ≔ Φ(1113957A) (5)
Star ratings have been commonplace practice in manyfields where user feedback is required While facilitating thequestioning process from a psychological perspective itcomplicates analysis of statistical properties of responses asit has been reported in [5] e same authors claim thatscales with more than seven stars provide too many pos-sibilities and spoil the quality of a poll Likewise providingthe respondent a scale with odd number of stars promptshim a safe and lazy option to hit the middle of the scalewhich also reduces response quality
In our case we kept the original 10-star scale as proposedby the NPC risk-analysis team Such scale leaves operator noldquomiddlerdquo option unlike grade ldquo3rdquo on 5-star scale Indeed wedo not want operators to answer neutrally because oppositeto eg hotel ranking there is no ldquoneutralrdquo answer other thanabsence of the edge connecting the two services Moreoverfiner scale makes room for elaborating more precise in-structions on self-assessment and answering in the future Asregards the choice of distribution for ξ it came from papers[5 6] e cited authors applied disturbances of moderatescale of one to two stars only
e main aim of this paper is to evaluate sensitivity ofvarious definitions of service vulnerability Φ and of im-portance aggregation functions Γ to errors in user as-sessment of service impacts
2 Materials and Methods
21 Importance Definitions ere exist a number of rec-ognized and widely known definitions of vertex structuralimportance that can be used as candidates forΦ In parlance
2 Security and Communication Networks
of networks they are usually called node centralities [7]Some of them are trivial ones like node degreemdashthey areuseful but out of scope of this study as they do not considerlink weights ie impact values Some others are related tonetwork flow maximization problems [8] ey also areinappropriate here because software malfunctions unlikeflows are indivisible and on the contrary replicable is iswhy we decided to consider the following three ways tocalculate service vulnerability
(i) ΦPR Page Rank Values of r meet equation r HrwhereH is adjacencymatrixA normalized so that thesum of elements in each column of H equals oneVulnerability of a service calculated this way reflectstherefore vulnerability of all other services that ser-vice depends on Such was exactly the original idea ofweb page rank calculation by Google founders [9] Inour case a service is a counterpart of a web pageNote however that such normalization necessaryfrom theoretical point of view weakens impact ofvertices with high outdegree While reasonable for auser clicking through web pages this assumptiondoes not necessarily hold in case of eg spreadingfailures as they may affect dependent services equallystrongly independently of their number
(ii) ΦRC Reach Centrality Values of r represent frac-tion of all services whose operation may affect agiven service To account for service impact aweighted variant is used [10] Originally any vi
affecting vj increases rj by 1(1 minus |V|) In theweighted version this amount depends on averagelink weight on the shortest path from vi to vj inrelation to average link weight in the graph Withsuch approach a kind of weighted impact sum-mation is performed for each service howeverwithout concern for important structural proper-ties of the graph as for example existence ofbridges
(iii) ΦMI Maximum Input Values of r are solution ofthe following equation
rj min 10 1 +110
maxi
aijri1113874 1113875 (6)
e aim of the above formula is to calculate cen-tralities like for page rank however taking into ac-count only currently most important factorsAlgorithm (6) is repeated until convergence guar-anteed by curbing the outcome within lt1 10gt in-terval consistent with our rating scheme Finally astrongest impact path is created for each dependentservice which identifies most crucial parts of thegraph and service vulnerabilities accordinglyHowever it ignores all relations outside the path evenif they stay close to the path in terms of theirimportance
Service vulnerabilities calculated above are based onincoming edges and in fact have the meaning of servicesusceptibility to failure
22 Aggregation Functions Vulnerabilities can be aggre-gated by equation (3) into a single network vulnerabilityindex c in many ways Here we propose three of them
(i) ΓAV the mean of r it represents the total of servicevulnerabilities without regard for their distributionWhile providing a good measure of overall vul-nerability it hides the existence of extraordinaryvulnerable services in the network
(ii) Γ50 the median it represents the typical value ofservice vulnerability in the network ie it discardsextreme values
(iii) ΓMX the maximum contrary to Γ50 the service withbiggest vulnerability is picked up regardless ofvulnerability of the other ones
23 Sensitivity of Vulnerability to Self-Assessment ErrorsFor any instance of reported impact matrix 1113957Am we cancalculate corresponding 1113957rm and finally vulnerability index1113957cmmdashusing any combination of Φrsquos and Γrsquos provided aboveen we can calculate the difference between vulnerabilitiescalculated for reported and for real impact values
δm(Φ Γ) 1113957cm minus c (7)
In the context of difference between two sets of serviceswe may introduce yet another measure based on differencein ordering of the most important services there δm(Φ ΓL5)It uses Levenshtein distance [11] to compare the contentsand order of first five most important services in r and in 1113957re Levenshtein distance counts the number of edit oper-ations to apply to one sequence to convert it to anothersequence In our case five-element sequences are comparedEdit operations are insertion deletion and change of asingle element in a sequence For example if r [0 1 3
4 6 5] and rm [1 0 3 4 5 6] the five most importantservices would be (r5 r6 r4 r3 r2) and (r6 r5 r4 r3 r1)respectively It takes three operations to transform one setinto the other two for swapping of r5 with r6 and one forreplacement of r2 with r1mdashand therefore the edit distanceequals three
24 Used Networks In practice the service graph G andreported impact values 1113957A are compiled after a laboriousprocess of questioning service operators about their servicesrelationship structure and relationship intensity A samplereal graph of services made this way is presented in Figure 1Reconstruction of service dependencies between operators isparticularly hard since such information is often consideredconfidential Collected data are inherently sensitive becausethey may serve as well for improving network reliability asfor attacking its weakest points Such observation has beenmade previously in case of critical infrastructure modelingand holds also for digital services e papers [12 13] coversector-wise interdependency analysis and summarizemodeling approaches respectively All the authors expresstheir concern about privacy of the collected data conse-quently only a small fraction of interdependencies is
Security and Communication Networks 3
presented in [12] Similarly we decided to carry out ourstudy for networks whose operation is partially analogousto the interplay of digital services instead of the realnetwork
We found that networks of source code dependencies area close analogy First they represent software componentson a much smaller scale though Second the dependencybetween modules can be relatively easily tracked by staticcode analysis ird failure or malfunction of one softwaremodule influences the operation of all modules that dependon it although differently Fourth module dependencies inopen-source projects appear not in predefined way butrepresent current needs of programmers as already reportedin [14] Finally dependencies between source code modulesas well as between essential services can be relatively easilytraced while their intensity can not
All networks analyzed in this study describe softwaremodule dependencies in Javascript (JS) projects availablefrom hosting platform githubcom Dependencies have beenfound by using the static code analysis tool Madge httpwwwnpmjscompackagemadge Project properties aregiven in Table 1 Projects differ in size moreover some ofthem happen to have circular dependencies of the codewhich also happens for real digital services A sample graphof dependencies is shown in Figure 2
3 Results and Discussion
Formula (7) calculates the vulnerability estimation error fora single realization of 1113957A To assess the error in statisticalsense one would need to calculate analytically how ξ affects1113957A 1113957r and finally δ In this paper we rather present results ofcursory estimation of δ based on random sampling of δm fora number of M samples m isin 1 2 M We calculate thefollowing statistics from sample distributions of δ
(i) Mean average absolute error θAE (1M)1113936m|δm|
(ii) Mean average relative error θRE θAEc(iii) Standard deviation of error θAD stdev(δ)
(iv) Standard deviation of error relative to true valueθRD θADc
ey all are comprehensive measures of how errors ofoperators impact estimation affects errors of network vul-nerability given any of the proposed formulas of Φ and Γ
All the reasoning provided above concerns a single in-stance of A whose values are chosen randomly In order todraw more general conclusions about the properties ofchosen combination of Φ and Γ we need to repeat calcu-lations for a number of test cases Let us call them exper-imentsmdashnonzero values of new impact factor matrix A arechosen and disturbed using equation (4) in each experimentFinally all θrsquos are calculated accordingly Sample graphicalresults from two series of 1000 experiments each for Airbnbnetwork are given in Figure 3 In all our analyses from nowon the number of experiments will be equal to the numberof samples in each experiment M
Figures 3(a) and 3(b) show various characters of vul-nerability errors In some aspects the two demonstratedexamples bear similarity eg c and the average of δ isnegatively correlated (Intuitively the more high-score linksin the network the less important is error by one star inimpact estimation by the service operator) Next someconfigurations result in more discrete error distributionmdashasin case (b) where the switching nature of median manifestsin striped dot patterns Finally histograms show how muchvariable are vulnerability errors across experiments Forexample we see that in case (a) they are quite stableclustered closely around one value while in case (b) theyshow much bigger variability
Results in Figure 3 justify the need for deeper inspectionof the nature of observed errors However to comparesensitivity of many networks in multidimensional parameterspace of Φrsquos Γrsquos and Nrsquos we have to develop a simplerapproach We propose to calculate and compare averagevalues of θrsquos ie θAE θRE θAD and θRD over all performedexperiments Such averaged indicators are collected inTables 2ndash6 each table for a different project
Branch A
Branch B
Branch C
Operator 16
Operator 2
Operator 3
Operator 15
Operator 1
Operator 14
Figure 1 Graph of real dependencies between 33 services run by 17 operators in 3 branches of national economy
4 Security and Communication Networks
Processed with ΦMI ΓAV
200
175
150
125
100
75
50
25
0
θ AD
0026
0024
0022
0020
0018
0016
0014
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ13 14 15 16
γ13 14 15 16 0010 0012 0014 0016 0018
θRD
(a)300
250
200
150
100
50
001 02 03 04 05
θ AD
0018
0016
0014
0012
0010
0008
0006
0004
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ002 004 006 008 010 012
γ002 004 006 008 010 012
θRDProcessed with ΦRC Γ50
(b)
Figure 3 Scatter plots of vulnerability estimation error (left) and standard deviation (middle) vs true vulnerability Also (right)standard deviation histogrammdashfor experiments carried out for Airbnb network with N 1 and different importance and aggregationalgorithms (a) and (b)
Table 1 Properties of projects used for analysis
Project name Modules Number of circular dependencies Project urlAirbnb 22 0 httpgithubcomairbnbjavascriptFcc 426 18 httpgithubcomfreeCodeCampfreeCodeCampNodejs 9507 27 httpgithubcomnodejsnodeOmi 475 0 httpgithubcomTencentomiReact 507 0 httpgithubcomfacebookreactVue 419 8 httpgithubcomvuejsvue
Testtest-basejs
Testtest-react-orderjs
Basejs
Legacyjs
Indexjs
Rulesreact-allyjs
Rulesreactjs
Whitespacejs
Testrequiresjs
Figure 2 Screenshot of a sample exemplary graph of module dependencies in a part of Airbnb project displayed by Madge
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Our goal is to examine how the above process is sensitiveto incorrect information about mutual service impact asreported by operators with the assumption that the struc-ture of the network is known fully and correctly Such in-formation is crucial because that the scalar index value willbe reported to SOCs and consequently will play the role ofthe main threat indicator
We organized the paper as follows A network model ofservices is presented in the remaining part of this section Asuite of methods for calculation of service vulnerability and foraggregation of vulnerabilities into a scalar vulnerability indexare described in Section 2 It is followed by the section withdiscussion of results (Section 3) and we conclude in Section 4
e network of interdependent digital services ismodeled as a directed graph
G(V E) (1)
where V is a list of ordered vertices representing servicesV (v1 v2 v|V|) and E is a list of ordered edges eij isin E
if operation of service vi influences operation of service vje impact of such influence is defined by the operator ofservice vj on a discrete scale from 1 to 10 All the informationabout the graph structure and service impact can beexpressed conveniently by adjacency matrix A whose ele-ment aij is equal to the impact value or zero if there is noedge eij Here we assume to operate with respect to only oneimpact aspect eg howmuch the loss of service i availabilityinfluences service j availability ere can be nine suchaspects in total C I A times C I A It is possible to combinethem all into one scalar coefficient when some assumptionson their meaning are made eg if one considers them asprobabilities
Such graph model extension with edge weights repre-sented actually by a matrix of up to nine aspects of impactdemands developing new graph algorithmsmdashor picking upone of the aspects like it is done in this paper It makes themodel universal enough to accommodate both digital servicesand physical infrastructure elements In the latter case onerefers to just the availability aspect For example availability ofbackup power supply may influence availability and integrityof the physical access control system hence an operator hasto address the influence in two aspects A⟼A and A⟼ I
Topology of a service graph represents existence ofservice interdependencies while edge weights stand forintensity of those interdependencies When combined theymake it possible to calculate the overall vulnerability of eachservice ere are many ways such vulnerability could beformulated we express its definition as
r ≔ Φ(A) (2)
whereΦ is some function defined over adjacency matrix thatcomputes vector r of vulnerabilities for each servicerespectively
While r contains complete information about vulnera-bility of each service a single scalar index c of overall networkvulnerability would be much more convenient in everydayuse Like for individual vulnerabilities its calculation can beaccomplished in many ways we denote this process as
c ≔ Γ(r) (3)
where Γ is some function defined over vulnerability vectore major practical problem concerns credibility of c
which is computed indirectly from A whose values are notobjective ey come from the questionnaires and are aresult of self-assessment process by service operators whoseaccuracy depends on their cybersecurity awareness andmaturity of methodologies used in service impact estima-tion An objective approach to vulnerability estimationwould require excessive provocative tests on critical servicesor postmortem analyses both of which are costly andundesirable
erefore we must assume that contrary to structure ofservice dependencies that is known and correct the reportedimpact values 1113957A differ from true ones by some errors
1113957aij min 10max 1 aij + ξ1113872 11138731113872 1113873 if aij gt 0
0 otherwise
⎧⎨
⎩ (4)
where ξ is realization of a random variable with uniformdiscrete distribution U minus N N Here N is the maximumimpact estimation error in the ten-star rating scale Note thatin (4) we curb disturbed rating within the original scale ofone to ten stars Consequently we denote calculated vul-nerabilities of serivces for the reported values of 1113957A as
1113957r ≔ Φ(1113957A) (5)
Star ratings have been commonplace practice in manyfields where user feedback is required While facilitating thequestioning process from a psychological perspective itcomplicates analysis of statistical properties of responses asit has been reported in [5] e same authors claim thatscales with more than seven stars provide too many pos-sibilities and spoil the quality of a poll Likewise providingthe respondent a scale with odd number of stars promptshim a safe and lazy option to hit the middle of the scalewhich also reduces response quality
In our case we kept the original 10-star scale as proposedby the NPC risk-analysis team Such scale leaves operator noldquomiddlerdquo option unlike grade ldquo3rdquo on 5-star scale Indeed wedo not want operators to answer neutrally because oppositeto eg hotel ranking there is no ldquoneutralrdquo answer other thanabsence of the edge connecting the two services Moreoverfiner scale makes room for elaborating more precise in-structions on self-assessment and answering in the future Asregards the choice of distribution for ξ it came from papers[5 6] e cited authors applied disturbances of moderatescale of one to two stars only
e main aim of this paper is to evaluate sensitivity ofvarious definitions of service vulnerability Φ and of im-portance aggregation functions Γ to errors in user as-sessment of service impacts
2 Materials and Methods
21 Importance Definitions ere exist a number of rec-ognized and widely known definitions of vertex structuralimportance that can be used as candidates forΦ In parlance
2 Security and Communication Networks
of networks they are usually called node centralities [7]Some of them are trivial ones like node degreemdashthey areuseful but out of scope of this study as they do not considerlink weights ie impact values Some others are related tonetwork flow maximization problems [8] ey also areinappropriate here because software malfunctions unlikeflows are indivisible and on the contrary replicable is iswhy we decided to consider the following three ways tocalculate service vulnerability
(i) ΦPR Page Rank Values of r meet equation r HrwhereH is adjacencymatrixA normalized so that thesum of elements in each column of H equals oneVulnerability of a service calculated this way reflectstherefore vulnerability of all other services that ser-vice depends on Such was exactly the original idea ofweb page rank calculation by Google founders [9] Inour case a service is a counterpart of a web pageNote however that such normalization necessaryfrom theoretical point of view weakens impact ofvertices with high outdegree While reasonable for auser clicking through web pages this assumptiondoes not necessarily hold in case of eg spreadingfailures as they may affect dependent services equallystrongly independently of their number
(ii) ΦRC Reach Centrality Values of r represent frac-tion of all services whose operation may affect agiven service To account for service impact aweighted variant is used [10] Originally any vi
affecting vj increases rj by 1(1 minus |V|) In theweighted version this amount depends on averagelink weight on the shortest path from vi to vj inrelation to average link weight in the graph Withsuch approach a kind of weighted impact sum-mation is performed for each service howeverwithout concern for important structural proper-ties of the graph as for example existence ofbridges
(iii) ΦMI Maximum Input Values of r are solution ofthe following equation
rj min 10 1 +110
maxi
aijri1113874 1113875 (6)
e aim of the above formula is to calculate cen-tralities like for page rank however taking into ac-count only currently most important factorsAlgorithm (6) is repeated until convergence guar-anteed by curbing the outcome within lt1 10gt in-terval consistent with our rating scheme Finally astrongest impact path is created for each dependentservice which identifies most crucial parts of thegraph and service vulnerabilities accordinglyHowever it ignores all relations outside the path evenif they stay close to the path in terms of theirimportance
Service vulnerabilities calculated above are based onincoming edges and in fact have the meaning of servicesusceptibility to failure
22 Aggregation Functions Vulnerabilities can be aggre-gated by equation (3) into a single network vulnerabilityindex c in many ways Here we propose three of them
(i) ΓAV the mean of r it represents the total of servicevulnerabilities without regard for their distributionWhile providing a good measure of overall vul-nerability it hides the existence of extraordinaryvulnerable services in the network
(ii) Γ50 the median it represents the typical value ofservice vulnerability in the network ie it discardsextreme values
(iii) ΓMX the maximum contrary to Γ50 the service withbiggest vulnerability is picked up regardless ofvulnerability of the other ones
23 Sensitivity of Vulnerability to Self-Assessment ErrorsFor any instance of reported impact matrix 1113957Am we cancalculate corresponding 1113957rm and finally vulnerability index1113957cmmdashusing any combination of Φrsquos and Γrsquos provided aboveen we can calculate the difference between vulnerabilitiescalculated for reported and for real impact values
δm(Φ Γ) 1113957cm minus c (7)
In the context of difference between two sets of serviceswe may introduce yet another measure based on differencein ordering of the most important services there δm(Φ ΓL5)It uses Levenshtein distance [11] to compare the contentsand order of first five most important services in r and in 1113957re Levenshtein distance counts the number of edit oper-ations to apply to one sequence to convert it to anothersequence In our case five-element sequences are comparedEdit operations are insertion deletion and change of asingle element in a sequence For example if r [0 1 3
4 6 5] and rm [1 0 3 4 5 6] the five most importantservices would be (r5 r6 r4 r3 r2) and (r6 r5 r4 r3 r1)respectively It takes three operations to transform one setinto the other two for swapping of r5 with r6 and one forreplacement of r2 with r1mdashand therefore the edit distanceequals three
24 Used Networks In practice the service graph G andreported impact values 1113957A are compiled after a laboriousprocess of questioning service operators about their servicesrelationship structure and relationship intensity A samplereal graph of services made this way is presented in Figure 1Reconstruction of service dependencies between operators isparticularly hard since such information is often consideredconfidential Collected data are inherently sensitive becausethey may serve as well for improving network reliability asfor attacking its weakest points Such observation has beenmade previously in case of critical infrastructure modelingand holds also for digital services e papers [12 13] coversector-wise interdependency analysis and summarizemodeling approaches respectively All the authors expresstheir concern about privacy of the collected data conse-quently only a small fraction of interdependencies is
Security and Communication Networks 3
presented in [12] Similarly we decided to carry out ourstudy for networks whose operation is partially analogousto the interplay of digital services instead of the realnetwork
We found that networks of source code dependencies area close analogy First they represent software componentson a much smaller scale though Second the dependencybetween modules can be relatively easily tracked by staticcode analysis ird failure or malfunction of one softwaremodule influences the operation of all modules that dependon it although differently Fourth module dependencies inopen-source projects appear not in predefined way butrepresent current needs of programmers as already reportedin [14] Finally dependencies between source code modulesas well as between essential services can be relatively easilytraced while their intensity can not
All networks analyzed in this study describe softwaremodule dependencies in Javascript (JS) projects availablefrom hosting platform githubcom Dependencies have beenfound by using the static code analysis tool Madge httpwwwnpmjscompackagemadge Project properties aregiven in Table 1 Projects differ in size moreover some ofthem happen to have circular dependencies of the codewhich also happens for real digital services A sample graphof dependencies is shown in Figure 2
3 Results and Discussion
Formula (7) calculates the vulnerability estimation error fora single realization of 1113957A To assess the error in statisticalsense one would need to calculate analytically how ξ affects1113957A 1113957r and finally δ In this paper we rather present results ofcursory estimation of δ based on random sampling of δm fora number of M samples m isin 1 2 M We calculate thefollowing statistics from sample distributions of δ
(i) Mean average absolute error θAE (1M)1113936m|δm|
(ii) Mean average relative error θRE θAEc(iii) Standard deviation of error θAD stdev(δ)
(iv) Standard deviation of error relative to true valueθRD θADc
ey all are comprehensive measures of how errors ofoperators impact estimation affects errors of network vul-nerability given any of the proposed formulas of Φ and Γ
All the reasoning provided above concerns a single in-stance of A whose values are chosen randomly In order todraw more general conclusions about the properties ofchosen combination of Φ and Γ we need to repeat calcu-lations for a number of test cases Let us call them exper-imentsmdashnonzero values of new impact factor matrix A arechosen and disturbed using equation (4) in each experimentFinally all θrsquos are calculated accordingly Sample graphicalresults from two series of 1000 experiments each for Airbnbnetwork are given in Figure 3 In all our analyses from nowon the number of experiments will be equal to the numberof samples in each experiment M
Figures 3(a) and 3(b) show various characters of vul-nerability errors In some aspects the two demonstratedexamples bear similarity eg c and the average of δ isnegatively correlated (Intuitively the more high-score linksin the network the less important is error by one star inimpact estimation by the service operator) Next someconfigurations result in more discrete error distributionmdashasin case (b) where the switching nature of median manifestsin striped dot patterns Finally histograms show how muchvariable are vulnerability errors across experiments Forexample we see that in case (a) they are quite stableclustered closely around one value while in case (b) theyshow much bigger variability
Results in Figure 3 justify the need for deeper inspectionof the nature of observed errors However to comparesensitivity of many networks in multidimensional parameterspace of Φrsquos Γrsquos and Nrsquos we have to develop a simplerapproach We propose to calculate and compare averagevalues of θrsquos ie θAE θRE θAD and θRD over all performedexperiments Such averaged indicators are collected inTables 2ndash6 each table for a different project
Branch A
Branch B
Branch C
Operator 16
Operator 2
Operator 3
Operator 15
Operator 1
Operator 14
Figure 1 Graph of real dependencies between 33 services run by 17 operators in 3 branches of national economy
4 Security and Communication Networks
Processed with ΦMI ΓAV
200
175
150
125
100
75
50
25
0
θ AD
0026
0024
0022
0020
0018
0016
0014
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ13 14 15 16
γ13 14 15 16 0010 0012 0014 0016 0018
θRD
(a)300
250
200
150
100
50
001 02 03 04 05
θ AD
0018
0016
0014
0012
0010
0008
0006
0004
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ002 004 006 008 010 012
γ002 004 006 008 010 012
θRDProcessed with ΦRC Γ50
(b)
Figure 3 Scatter plots of vulnerability estimation error (left) and standard deviation (middle) vs true vulnerability Also (right)standard deviation histogrammdashfor experiments carried out for Airbnb network with N 1 and different importance and aggregationalgorithms (a) and (b)
Table 1 Properties of projects used for analysis
Project name Modules Number of circular dependencies Project urlAirbnb 22 0 httpgithubcomairbnbjavascriptFcc 426 18 httpgithubcomfreeCodeCampfreeCodeCampNodejs 9507 27 httpgithubcomnodejsnodeOmi 475 0 httpgithubcomTencentomiReact 507 0 httpgithubcomfacebookreactVue 419 8 httpgithubcomvuejsvue
Testtest-basejs
Testtest-react-orderjs
Basejs
Legacyjs
Indexjs
Rulesreact-allyjs
Rulesreactjs
Whitespacejs
Testrequiresjs
Figure 2 Screenshot of a sample exemplary graph of module dependencies in a part of Airbnb project displayed by Madge
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
of networks they are usually called node centralities [7]Some of them are trivial ones like node degreemdashthey areuseful but out of scope of this study as they do not considerlink weights ie impact values Some others are related tonetwork flow maximization problems [8] ey also areinappropriate here because software malfunctions unlikeflows are indivisible and on the contrary replicable is iswhy we decided to consider the following three ways tocalculate service vulnerability
(i) ΦPR Page Rank Values of r meet equation r HrwhereH is adjacencymatrixA normalized so that thesum of elements in each column of H equals oneVulnerability of a service calculated this way reflectstherefore vulnerability of all other services that ser-vice depends on Such was exactly the original idea ofweb page rank calculation by Google founders [9] Inour case a service is a counterpart of a web pageNote however that such normalization necessaryfrom theoretical point of view weakens impact ofvertices with high outdegree While reasonable for auser clicking through web pages this assumptiondoes not necessarily hold in case of eg spreadingfailures as they may affect dependent services equallystrongly independently of their number
(ii) ΦRC Reach Centrality Values of r represent frac-tion of all services whose operation may affect agiven service To account for service impact aweighted variant is used [10] Originally any vi
affecting vj increases rj by 1(1 minus |V|) In theweighted version this amount depends on averagelink weight on the shortest path from vi to vj inrelation to average link weight in the graph Withsuch approach a kind of weighted impact sum-mation is performed for each service howeverwithout concern for important structural proper-ties of the graph as for example existence ofbridges
(iii) ΦMI Maximum Input Values of r are solution ofthe following equation
rj min 10 1 +110
maxi
aijri1113874 1113875 (6)
e aim of the above formula is to calculate cen-tralities like for page rank however taking into ac-count only currently most important factorsAlgorithm (6) is repeated until convergence guar-anteed by curbing the outcome within lt1 10gt in-terval consistent with our rating scheme Finally astrongest impact path is created for each dependentservice which identifies most crucial parts of thegraph and service vulnerabilities accordinglyHowever it ignores all relations outside the path evenif they stay close to the path in terms of theirimportance
Service vulnerabilities calculated above are based onincoming edges and in fact have the meaning of servicesusceptibility to failure
22 Aggregation Functions Vulnerabilities can be aggre-gated by equation (3) into a single network vulnerabilityindex c in many ways Here we propose three of them
(i) ΓAV the mean of r it represents the total of servicevulnerabilities without regard for their distributionWhile providing a good measure of overall vul-nerability it hides the existence of extraordinaryvulnerable services in the network
(ii) Γ50 the median it represents the typical value ofservice vulnerability in the network ie it discardsextreme values
(iii) ΓMX the maximum contrary to Γ50 the service withbiggest vulnerability is picked up regardless ofvulnerability of the other ones
23 Sensitivity of Vulnerability to Self-Assessment ErrorsFor any instance of reported impact matrix 1113957Am we cancalculate corresponding 1113957rm and finally vulnerability index1113957cmmdashusing any combination of Φrsquos and Γrsquos provided aboveen we can calculate the difference between vulnerabilitiescalculated for reported and for real impact values
δm(Φ Γ) 1113957cm minus c (7)
In the context of difference between two sets of serviceswe may introduce yet another measure based on differencein ordering of the most important services there δm(Φ ΓL5)It uses Levenshtein distance [11] to compare the contentsand order of first five most important services in r and in 1113957re Levenshtein distance counts the number of edit oper-ations to apply to one sequence to convert it to anothersequence In our case five-element sequences are comparedEdit operations are insertion deletion and change of asingle element in a sequence For example if r [0 1 3
4 6 5] and rm [1 0 3 4 5 6] the five most importantservices would be (r5 r6 r4 r3 r2) and (r6 r5 r4 r3 r1)respectively It takes three operations to transform one setinto the other two for swapping of r5 with r6 and one forreplacement of r2 with r1mdashand therefore the edit distanceequals three
24 Used Networks In practice the service graph G andreported impact values 1113957A are compiled after a laboriousprocess of questioning service operators about their servicesrelationship structure and relationship intensity A samplereal graph of services made this way is presented in Figure 1Reconstruction of service dependencies between operators isparticularly hard since such information is often consideredconfidential Collected data are inherently sensitive becausethey may serve as well for improving network reliability asfor attacking its weakest points Such observation has beenmade previously in case of critical infrastructure modelingand holds also for digital services e papers [12 13] coversector-wise interdependency analysis and summarizemodeling approaches respectively All the authors expresstheir concern about privacy of the collected data conse-quently only a small fraction of interdependencies is
Security and Communication Networks 3
presented in [12] Similarly we decided to carry out ourstudy for networks whose operation is partially analogousto the interplay of digital services instead of the realnetwork
We found that networks of source code dependencies area close analogy First they represent software componentson a much smaller scale though Second the dependencybetween modules can be relatively easily tracked by staticcode analysis ird failure or malfunction of one softwaremodule influences the operation of all modules that dependon it although differently Fourth module dependencies inopen-source projects appear not in predefined way butrepresent current needs of programmers as already reportedin [14] Finally dependencies between source code modulesas well as between essential services can be relatively easilytraced while their intensity can not
All networks analyzed in this study describe softwaremodule dependencies in Javascript (JS) projects availablefrom hosting platform githubcom Dependencies have beenfound by using the static code analysis tool Madge httpwwwnpmjscompackagemadge Project properties aregiven in Table 1 Projects differ in size moreover some ofthem happen to have circular dependencies of the codewhich also happens for real digital services A sample graphof dependencies is shown in Figure 2
3 Results and Discussion
Formula (7) calculates the vulnerability estimation error fora single realization of 1113957A To assess the error in statisticalsense one would need to calculate analytically how ξ affects1113957A 1113957r and finally δ In this paper we rather present results ofcursory estimation of δ based on random sampling of δm fora number of M samples m isin 1 2 M We calculate thefollowing statistics from sample distributions of δ
(i) Mean average absolute error θAE (1M)1113936m|δm|
(ii) Mean average relative error θRE θAEc(iii) Standard deviation of error θAD stdev(δ)
(iv) Standard deviation of error relative to true valueθRD θADc
ey all are comprehensive measures of how errors ofoperators impact estimation affects errors of network vul-nerability given any of the proposed formulas of Φ and Γ
All the reasoning provided above concerns a single in-stance of A whose values are chosen randomly In order todraw more general conclusions about the properties ofchosen combination of Φ and Γ we need to repeat calcu-lations for a number of test cases Let us call them exper-imentsmdashnonzero values of new impact factor matrix A arechosen and disturbed using equation (4) in each experimentFinally all θrsquos are calculated accordingly Sample graphicalresults from two series of 1000 experiments each for Airbnbnetwork are given in Figure 3 In all our analyses from nowon the number of experiments will be equal to the numberof samples in each experiment M
Figures 3(a) and 3(b) show various characters of vul-nerability errors In some aspects the two demonstratedexamples bear similarity eg c and the average of δ isnegatively correlated (Intuitively the more high-score linksin the network the less important is error by one star inimpact estimation by the service operator) Next someconfigurations result in more discrete error distributionmdashasin case (b) where the switching nature of median manifestsin striped dot patterns Finally histograms show how muchvariable are vulnerability errors across experiments Forexample we see that in case (a) they are quite stableclustered closely around one value while in case (b) theyshow much bigger variability
Results in Figure 3 justify the need for deeper inspectionof the nature of observed errors However to comparesensitivity of many networks in multidimensional parameterspace of Φrsquos Γrsquos and Nrsquos we have to develop a simplerapproach We propose to calculate and compare averagevalues of θrsquos ie θAE θRE θAD and θRD over all performedexperiments Such averaged indicators are collected inTables 2ndash6 each table for a different project
Branch A
Branch B
Branch C
Operator 16
Operator 2
Operator 3
Operator 15
Operator 1
Operator 14
Figure 1 Graph of real dependencies between 33 services run by 17 operators in 3 branches of national economy
4 Security and Communication Networks
Processed with ΦMI ΓAV
200
175
150
125
100
75
50
25
0
θ AD
0026
0024
0022
0020
0018
0016
0014
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ13 14 15 16
γ13 14 15 16 0010 0012 0014 0016 0018
θRD
(a)300
250
200
150
100
50
001 02 03 04 05
θ AD
0018
0016
0014
0012
0010
0008
0006
0004
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ002 004 006 008 010 012
γ002 004 006 008 010 012
θRDProcessed with ΦRC Γ50
(b)
Figure 3 Scatter plots of vulnerability estimation error (left) and standard deviation (middle) vs true vulnerability Also (right)standard deviation histogrammdashfor experiments carried out for Airbnb network with N 1 and different importance and aggregationalgorithms (a) and (b)
Table 1 Properties of projects used for analysis
Project name Modules Number of circular dependencies Project urlAirbnb 22 0 httpgithubcomairbnbjavascriptFcc 426 18 httpgithubcomfreeCodeCampfreeCodeCampNodejs 9507 27 httpgithubcomnodejsnodeOmi 475 0 httpgithubcomTencentomiReact 507 0 httpgithubcomfacebookreactVue 419 8 httpgithubcomvuejsvue
Testtest-basejs
Testtest-react-orderjs
Basejs
Legacyjs
Indexjs
Rulesreact-allyjs
Rulesreactjs
Whitespacejs
Testrequiresjs
Figure 2 Screenshot of a sample exemplary graph of module dependencies in a part of Airbnb project displayed by Madge
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
presented in [12] Similarly we decided to carry out ourstudy for networks whose operation is partially analogousto the interplay of digital services instead of the realnetwork
We found that networks of source code dependencies area close analogy First they represent software componentson a much smaller scale though Second the dependencybetween modules can be relatively easily tracked by staticcode analysis ird failure or malfunction of one softwaremodule influences the operation of all modules that dependon it although differently Fourth module dependencies inopen-source projects appear not in predefined way butrepresent current needs of programmers as already reportedin [14] Finally dependencies between source code modulesas well as between essential services can be relatively easilytraced while their intensity can not
All networks analyzed in this study describe softwaremodule dependencies in Javascript (JS) projects availablefrom hosting platform githubcom Dependencies have beenfound by using the static code analysis tool Madge httpwwwnpmjscompackagemadge Project properties aregiven in Table 1 Projects differ in size moreover some ofthem happen to have circular dependencies of the codewhich also happens for real digital services A sample graphof dependencies is shown in Figure 2
3 Results and Discussion
Formula (7) calculates the vulnerability estimation error fora single realization of 1113957A To assess the error in statisticalsense one would need to calculate analytically how ξ affects1113957A 1113957r and finally δ In this paper we rather present results ofcursory estimation of δ based on random sampling of δm fora number of M samples m isin 1 2 M We calculate thefollowing statistics from sample distributions of δ
(i) Mean average absolute error θAE (1M)1113936m|δm|
(ii) Mean average relative error θRE θAEc(iii) Standard deviation of error θAD stdev(δ)
(iv) Standard deviation of error relative to true valueθRD θADc
ey all are comprehensive measures of how errors ofoperators impact estimation affects errors of network vul-nerability given any of the proposed formulas of Φ and Γ
All the reasoning provided above concerns a single in-stance of A whose values are chosen randomly In order todraw more general conclusions about the properties ofchosen combination of Φ and Γ we need to repeat calcu-lations for a number of test cases Let us call them exper-imentsmdashnonzero values of new impact factor matrix A arechosen and disturbed using equation (4) in each experimentFinally all θrsquos are calculated accordingly Sample graphicalresults from two series of 1000 experiments each for Airbnbnetwork are given in Figure 3 In all our analyses from nowon the number of experiments will be equal to the numberof samples in each experiment M
Figures 3(a) and 3(b) show various characters of vul-nerability errors In some aspects the two demonstratedexamples bear similarity eg c and the average of δ isnegatively correlated (Intuitively the more high-score linksin the network the less important is error by one star inimpact estimation by the service operator) Next someconfigurations result in more discrete error distributionmdashasin case (b) where the switching nature of median manifestsin striped dot patterns Finally histograms show how muchvariable are vulnerability errors across experiments Forexample we see that in case (a) they are quite stableclustered closely around one value while in case (b) theyshow much bigger variability
Results in Figure 3 justify the need for deeper inspectionof the nature of observed errors However to comparesensitivity of many networks in multidimensional parameterspace of Φrsquos Γrsquos and Nrsquos we have to develop a simplerapproach We propose to calculate and compare averagevalues of θrsquos ie θAE θRE θAD and θRD over all performedexperiments Such averaged indicators are collected inTables 2ndash6 each table for a different project
Branch A
Branch B
Branch C
Operator 16
Operator 2
Operator 3
Operator 15
Operator 1
Operator 14
Figure 1 Graph of real dependencies between 33 services run by 17 operators in 3 branches of national economy
4 Security and Communication Networks
Processed with ΦMI ΓAV
200
175
150
125
100
75
50
25
0
θ AD
0026
0024
0022
0020
0018
0016
0014
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ13 14 15 16
γ13 14 15 16 0010 0012 0014 0016 0018
θRD
(a)300
250
200
150
100
50
001 02 03 04 05
θ AD
0018
0016
0014
0012
0010
0008
0006
0004
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ002 004 006 008 010 012
γ002 004 006 008 010 012
θRDProcessed with ΦRC Γ50
(b)
Figure 3 Scatter plots of vulnerability estimation error (left) and standard deviation (middle) vs true vulnerability Also (right)standard deviation histogrammdashfor experiments carried out for Airbnb network with N 1 and different importance and aggregationalgorithms (a) and (b)
Table 1 Properties of projects used for analysis
Project name Modules Number of circular dependencies Project urlAirbnb 22 0 httpgithubcomairbnbjavascriptFcc 426 18 httpgithubcomfreeCodeCampfreeCodeCampNodejs 9507 27 httpgithubcomnodejsnodeOmi 475 0 httpgithubcomTencentomiReact 507 0 httpgithubcomfacebookreactVue 419 8 httpgithubcomvuejsvue
Testtest-basejs
Testtest-react-orderjs
Basejs
Legacyjs
Indexjs
Rulesreact-allyjs
Rulesreactjs
Whitespacejs
Testrequiresjs
Figure 2 Screenshot of a sample exemplary graph of module dependencies in a part of Airbnb project displayed by Madge
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Processed with ΦMI ΓAV
200
175
150
125
100
75
50
25
0
θ AD
0026
0024
0022
0020
0018
0016
0014
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ13 14 15 16
γ13 14 15 16 0010 0012 0014 0016 0018
θRD
(a)300
250
200
150
100
50
001 02 03 04 05
θ AD
0018
0016
0014
0012
0010
0008
0006
0004
Mea
n of
δm
0015
0010
0005
0000
ndash0005
ndash0010
ndash0015
γ002 004 006 008 010 012
γ002 004 006 008 010 012
θRDProcessed with ΦRC Γ50
(b)
Figure 3 Scatter plots of vulnerability estimation error (left) and standard deviation (middle) vs true vulnerability Also (right)standard deviation histogrammdashfor experiments carried out for Airbnb network with N 1 and different importance and aggregationalgorithms (a) and (b)
Table 1 Properties of projects used for analysis
Project name Modules Number of circular dependencies Project urlAirbnb 22 0 httpgithubcomairbnbjavascriptFcc 426 18 httpgithubcomfreeCodeCampfreeCodeCampNodejs 9507 27 httpgithubcomnodejsnodeOmi 475 0 httpgithubcomTencentomiReact 507 0 httpgithubcomfacebookreactVue 419 8 httpgithubcomvuejsvue
Testtest-basejs
Testtest-react-orderjs
Basejs
Legacyjs
Indexjs
Rulesreact-allyjs
Rulesreactjs
Whitespacejs
Testrequiresjs
Figure 2 Screenshot of a sample exemplary graph of module dependencies in a part of Airbnb project displayed by Madge
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
e figures given in Tables 2ndash6 cover all combinations offive graphs three importance indices Φ four importanceaggregation functions Γ and two amplitudes of estimationerror N Basically we search this space to find valuablecombinations of Φrsquos and Γrsquos A valuable combination ischaracterized by
(i) Small total error Δ for all considered projects andvalues of Nmdashwe want the approach to be in-dependent of graph structure
(ii) Big sensitivity S to change of N for all projects (pickthe worst case)mdashwe want operatorsrsquo errors of es-timation to really influence the value of overallmetrics θ
(iii) Small standard deviation Σ of error for all projects(pick the worst case)mdashwe want small variance of θrsquosin general
Candidate combinations ofΦ and Γ should therefore be ingeneral tolerant to imprecise information provided by op-erators but at the same time sensitive to the scale of such lackof precision Moreover it is desirable that errors in networkvulnerability calculated by such combination do not varywidelyWe check the last two requirements with respect to theworst results found for the analyzed projects Results of suchthree-criteria scoring are presented in Figure 4 projected on
three planes e axes have been selected or adjusted so thatmarkers located near an axis correspond to combinations thatperform better Visual comparison provided in Figure 4 doesnot determine strictly the optimum combination butmakes itpossible to observe that in general performance indices donot vary widelymdashat least so that using linear axis scaling willdo to reveal differences Secondly markers get clusteredmainly with respect to their color which means that thechoice of aggregation method Γ is more important than thechoice of algorithm for importance index calculation
As analyzed combinations form a cloud in 3D space wemay find a Pareto front ie a set of nondominated com-binations ey are
(i) (ΦRC ΓAV)mdashthe average of reach centrality(ii) (ΦPR ΓAV)mdashthe average page rank(iii) (ΦPR ΓAV)mdashthe median of page rank(iv) (ΦPR ΓMX)mdashthe maximum of page rank(v) (ΦMI ΓAV)mdashthe average of maximum input
importance
4 Conclusions
It should be reminded that research reported here is done incontext of a large project aiming to build a nation-wide
Δ
30
25
20
15
10
5
0
00 02 04 06 08 10 12sum
ndashS
00
ndash01
ndash02
ndash03
ndash04
0 5 10 15 20 25 30Δ
12
10
08
06
04
02
00
ndash04 ndash03 ndash02 ndash01 00
sum
ndashS
Figure 4 Values ofΔ the negative of S and Σ presented in pairs in separate graphs Shapes denotemethods used for importance calculationΦPRmdashplus ΦRCmdashdot ΦMImdashcross Colors denote aggregation methods used ΓAVmdashblack Γ50mdashblue ΓMXmdashred ΓL5mdashgreen
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
model of critical services network While integrity of theresulting graph can be obtained by careful automated in-spection of questionnaires filed by service operators theestimated reported impact between services will be biasedand inherently erroneous erefore it was worth to studysensitivity of some candidate synthetic metrics of overallnetwork vulnerability with respect to incorrect inputWe feltit correct to use networks of software module dependenciesbecause of their functional and structural similarity tonetwork of critical services let alone that such real networkswill probably remain confidential
e study shows that all three proposed formulas forindividual service vulnerability calculation are valuable isis rather a positive observation as each of them has its ownspecifics and can be used under various circumstances Alsoalmost all proposed ways of vulnerability aggregation into asingle vulnerability index are useful (except the Levenshteindistance which shows much variation and has turned out tobe useless) Naturally combinations of formulas appropriatefor capturing ldquoextremerdquo phenomena as (ΦMI ΓMX) willhave show variability
e main takeaway is that it is safe to apply mean ormedian aggregation of individual service vulnerability what-ever is the formula for importance calculation Such aggregatedvalue may serve as a single comprehensive vulnerability indexNote that being robust to errors in graph edge weights it will beaffected by major structural graph changesmdasheg edge removalas result of real-time detected failure Our previous work hasshown that networks of autonomous systems (AS) can be reallybadly affected by just one link failure contrary to widespreadbelief in Internet robustness [15]
One should remember that results reported here werebased on the sound assumption of analogy between criticalservices and software modules is assumption will even-tually get verified in practice once the national cybersecurityplatform is operational and filled with data We look forwardto compare properties of vulnerability calculation formulascalculated here by random sampling with careful expertjudgment and postmortem analyses for real services graph
Data Availability
e open source code used to support the findings of thisstudy is publicly available on httpgithubcom and can bedownloaded and processed with tools indicated in thispaper e proprietary Python code created by the author toanalyze data used to support the findings of this study isavailable from the corresponding author upon request
Conflicts of Interest
e author declares that there are no conflicts of interestregarding the publication of this paper
Acknowledgments
e work presented in this paper has been supported by thePolish National Centre for Research and Development grant(CYBERSECIDENT369195INCBR2017)
References
[1] e European Commission lte Directive on Security ofNetwork and Information Systems e European Commis-sion Brussels Belgium 2016
[2] J Hingant M Zambrano F J Perez I Perez and M EsteveldquoHybint a hybrid intelligence system for critical in-frastructures protectionrdquo Security and Communication Net-works vol 2018 Article ID 5625860 13 pages 2018
[3] G Settanni F Skopik Y Shovgenya et al ldquoA collaborativecyber incident management system for european inter-connected critical infrastructuresrdquo Journal of InformationSecurity and Applications vol 34 pp 166ndash182 2017
[4] W Stallings L Brown M D Bauer and A K BhattacharjeeComputer Security Principles and Practice Pearson Educa-tion Upper Saddle River NJ USA 2012
[5] M Medo and J R Wakeling ldquoe effect of discrete vscontinuous-valued ratings on reputation and ranking sys-temsrdquo EPL (Europhysics Letters) vol 91 no 4 Article ID48004 2010
[6] W W Moe and M Trusov ldquoe value of social dynamics inonline product ratings forumsrdquo Journal of Marketing Re-search vol 48 no 3 pp 444ndash456 2011
[8] U Brandes and D Fleischer ldquoCentrality measures based oncurrent flowrdquo in Annual Symposium on lteoretical Aspects ofComputer Science pp 533ndash544 Springer Berlin Germany2005
[9] L Page S Brin R Motwani and W Terry ldquoe pagerankcitation ranking bringing order to the webrdquo Tech RepStanford InfoLab Stanford CA USA 1999
[10] E Mones L Vicsek and T Vicsek ldquoHierarchy measure forcomplex networksrdquo PLoS One vol 7 no 3 Article ID e337992012
[11] V Levenshtein ldquoBinary codes capable of correcting deletionsinsertions and reversalsrdquo Soviet Physics Doklady vol 10no 8 pp 707ndash710 1966
[12] C-N Huang J J H Liou and Y-C Chuang ldquoA method forexploring the interdependencies and importance of criticalinfrastructuresrdquo Knowledge-Based Systems vol 55 pp 66ndash742014
[13] M Ouyang ldquoReview on modeling and simulation of in-terdependent critical infrastructure systemsrdquo Reliability En-gineering amp System Safety vol 121 pp 43ndash60 2014
[14] M Kamola ldquoHow to verify conwayrsquos law for open sourceprojectsrdquo IEEE Access vol 7 pp 38469ndash38480 2019
[15] K Mariusz and A Piotr ldquoNetwork resilience analysis reviewof concepts and a country-level case studyrdquo Computer Sci-ence vol 15 no 3 p 311 2014
