Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

Rationale for and Capabilities ofIT Security Assessment

Niklas Hallberg, Jonas Hallberg and Amund Hunstad

resources and could diminish the usability and use of systems.Abstract-The abundance of security threats makes IT security Hence, it is important to have just enough IT security.

a prerequisite for the use of information technology (IT). Striving The assessment of IT security has been recognized as afor appropriate security, costs for IT security controls should be fundamental ability to ensure adequate levels of IT security inrelated to their impact on the level of IT security. This requiresthe level of IT security to be assessed. However, this insight is to orgaization i tio s and nectworks. ever,general to guide the design of methods and tools for IT security there is a lack of tools and methods for security assessmentsassessments. Thereby, there is a necessity to explore what are the [1]-[5]. Hence, there is a need for development within thisrationale for IT security assessments, i.e., why, where, and when area.is it needed. To develop adequate tools and methods for IT security

The objective of this study is to explore the rationale for and assessment it is necessary to understand why, where and whencapabilities required of methods and tools for IT security they are needed. Hence, before these methods and tools can beassessment. The knowledge, about rationale and needed v ncapabilities, should constitute as a foundation for the future developed, it is essential to explore what is the rationale for ITdevelopment of methods and tools regarding IT security security assessment. Based on the rationale, the capabilities,assessment. The study was performed as a case study within the which tools and methods for IT security assessment need toSwedish Armed Forces. Based on interviews and relevant provide, can be determined.documents, statements directly or indirectly indicating the need The objective of this study is to explore the rationale for ITfor IT security assessments were identified. These statementswere carefully analyzed to identify IT security issues. Thereafter, curity essment and basedon that determine ththe IT security issues were categorized into six categories: (1) capabilities that the corresponding tools and methods need tosystems development, (2) system operation, (3) risk management, provide. For this purpose, a case study was performed within(4) communication and management of security work, (5) the Network Based Defense (NBD) development effort of thecompetence regarding IT security and (6) attainment and Swedish Armed Forces. The study is based on interviews withpreservation of trust. From these categories, 18 contributions to persons involved in and documents related to the developmentthe rationale for IT security assessments were identified and used pfrthnSiedi h NBD oto determine capabilities needed of tools and methods for ITsecurity assessments. These capabilities of IT security assessmentare presented by criteria ordered in the categories: security II. BACKGROUNDassessment domains, security relevant factors, characteristics of This section presents the area of IT security assessments andsecurity controls, and assessments results. the study context.

Index Terms- IT security, IT security assessment, case study. A. IT Security Assessment

Sufficient handling of security issues during theI. INTRODUCTION development and operation of IT systems is vital. Thereby,

Modern societies, organizations, and businesses depend on security management requires understanding of the impact andreliable information systems. This has made them vulnerable consequences of security-related decisions, such as, whichto, for instance, unauthorized use, IT frauds, and attacks security controls to include, the password policy, and thetargeting the corresponding IT systems. As a consequence, the routines for software patching. This evaluation of security-need for sufficient IT security increases. However, the controls related decisions requires the ability to assess the security offor protection of information systems and sensitive data cost IT systems. Software security has been recognized as an

especially cumbersome area regarding IT security assessment[1]. Unfortunately, there is a lack of efficient methods and

Dr. Nikias fallberg is with the Swedish Defence Research Agency, tools to accomplish IT security assessment [2]-[5]. A furtherLinkoping, SWEDEN and LinkdipingS University. SWEDEN. (e-mail: rao oadesteqatfcto fI ytmscrt stniklas.ha1lberggfoi.se). rao oadesteqatfcto fI ytmscrt st

Dr. Jonas Hallberg is withl the Swedish Defence Research Agency, provide a vital mean for the progress of the IT security area,Linkoping, SWEDEN (e-mail: jonas.hallberggfoi.se). that is, the founldation for scienltific research [6].

M.Sc. Amnund Hunlstad is withl the Svwrdishl Defeince Research Agenlcy. The fundamental objective of IT security assessment is toL ?kpn,SEE emi:ann.usa@o.e deliver quantitative or qualitative security values describing

1-4244-1304-4/07/$25.00 ©C2007 IEEE 159

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

the security qualities of systems and system components. Since today's society, e.g., participation in international peacesecurity cannot be directly measured, its assessment has to be keeping forces and defending against terrorist attacks.based on consequences or char-acteristics [7]. Assessment Therefore, the main objective of the reorganization is toapproaches regarding consequences can be based on accomplish a cost effective and flexible organization based onobservations or tests. Approaches based on observations the ideas of network-centric warfare (NCW) [9]. Modernconsider system output, or consequences, but ignore internal information technology is an important enabler, especially,characteristics of the system. Apart from studying the when it comes to the realization of command and control (C2)consequences of the security of the system or system systems.components, approaches based on tests involve provocation of Several projects have been initiated within the SwAF tothe system to obtain results for further analysis. Assessment develop these kinds of C2 systems. The projects oftenapproaches based on characteristics are divided into encounter problems, much due to: (1) the enormous extent andapproaches considering component, system-wide, or structural the complexity of the organization, (2) the heritage followingcharacteristics, where knowledge about the internals of the the historical division into three separate branches, i.e., army,system or system component is assumed. The difference navy, and air force, (3) the need to be interoperable withbetween component and system-wide characteristics depends different nations, organizations and systems in various settings,on how the system is modeled, since system-wide and (4) how to realize net-centric C2 systems is far fromcharacteristics include those properties not assigned to obvious.individual system components. Structural characteristicsregard the relations between the components of the system. III. METHOD

All assessment approaches are, more or less explicitly, This study was performed in five steps: (A) Data collection,based on models of the systems to assess. For example, if the (B) Identification of security issues, (C) Categorization ofassessment is based on interviews with system administrators, security issues, (D) Determination of rationale for securitythe system model coincides with their view of the system. If assessment, and (E) Determination of relevant securityavailable system documentation is used to extract knowledge assessment capabilities (Figure 1).about the system, the model is based on those documents.Thus, system modeling is an important element in security A. Data collectionassessment. The first step was to collect relevant data. This was

Whether the assessment is based on consequences, performed through interviews and a study of governing docu-characteristics, or both, basic measurements have to be ments. The aim was to identify statements that could be usedperformed and then transformed into presentable results. This to determine security issues for which IT security assessmentsrequires viable security metrics for both the results extracted could be of interest. The outcome was a set of statements.from the model and the security values resulting from the The interviews were based on a semi-structured set ofaggregation performed by the assessment method. There are questions. The interviews were recorded and, thereafter,several interpretations of the term security metric. Hallberg et transcribed. The interview transcriptions and a set of relevantal [8] state that "a security metric contains three main parts: a governing documents were carefully scrutinized to identifymagnitude, a scale and an interpretation. The security values of statements that could be of relevance for understanding thesystems are measured according to a specified magnitude and needs for IT security assessments.related to a scale. The interpretation prescribes the meaning of B. Identification qf'security issuesobtained security values."Commine useursi o security assessment results are other The objective of the second step was to identify the security

system-related proesses,csuch as,esrisktmanagementessthem issues contained in the collected statements. Therefore, eachdystev -relopmen dprocesystems,suchas,rimanagement.,Meanl statement was carefully analyzed by asking, who needs it, whatdevelopment, and systems management. Meantongdul securwty tassessment has to correspond to the needs of these users. Thus, do they want to do with it, when do they want to doita wherethe first step in providing viable methods and tools for security 't

assessment is to decide the user needs to be fulfilled, which is they like to do it. Thus, the analysis was to reveal securitythe topic of this text. The user needs are here discussed in the issues within the statements. The outcome of this step was a set

terms of rationale for and capabilities of IT security of security issues.assessment. C. Categorization of secur-ity issuiesB. Study context The objective of the third step was to produce a categorization

The wedshAmedForcs (wAF)arein te poces of of the identified security issues. The categories were con-major reorganizations to befter meet the new challenges facing structed by clustering of the related security issues. Aftcer anmiitr oraiain wit mor limited bugt fae. The. initial categorization, the categories were merged into newformer organization was structured as an invasion defense and caeoisa hgelvlofasrcinutlaoeetis thrfr o utdt adl h oedvretsso structure was achieved. During this analysis, duplicates were

1-4244-1304-4/07/$25.00 ©C2007 IEEE 160

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

Interviews Collection of identify security issues not relevant for the field of IT securitygoverning documents assessments, which were left unattended.

Interviewtranscription analysis Literature study E. Delermination ofrelevant security assessment

capabilitiesIn this step, the rationale was analyzed to determine features

Identified and capabilities that are needed of security assessment toolsstatements and methods. During this step, an additional analysis of the

collected data was performed, to obtain a richer understandingIdentification of of the criteria for security assessment and to make use of thosesecurity issues statements explicitly expressing needs for security assessments

capabilities. This analysis was performed by expertise insecurity assessments.

Identifiedsecurity issues IV. RATIONALE FOR SECURITY ASSESSMENTS

This section presents the categories of security issues andCategorization of the rationale for security assessment resulting from the study.security issues Six interviews were performed with nine interviewees in total.

Thereof, eight persons were engaged in the development of c2systems for the SwAF and one person was responsible for the

Categories of IT security in an agency related to the SwAF. The numbers ofsecurity issues respondents per interview varied between one and three. In

addition to the interviews, 13 relevant documents were

Determination of collected. The analysis of the interview transcriptions and therationale for security relevant documents resulted in 215 statements. Of those 11 0

assessment statements were found in the interview transcriptions and 105statements originated from the governing documents.

525 security issues were identified from the statements.Rationale for After categorization and removal of duplicates, six categories

security assessment of security issues remained. These categories are:* Systems development

Determination of relevant * System operationsecurity assessment * Risk management

capabilities * Communication and management of security work* Competence regarding information security

Rele vanto* Attainment and preservation of trustsecrit ass A. Systems development

The respondents stated that a considerable number ofsecurity issues arise during the development of systems. They

Figure 1: The method used to decide the rationale for and pointed at the necessity to consider security already from therelevant capabilities of security assessment. beginning of the development. It was experienced that security

controls often were selected with little attention to the usage ofthe system. This was stated to be particularly apparent when

removed and formulations of the security issues were security controls were introduced late during the development,improved. The security issues were analyzed iteratively until often affecting usability in a negative manner. Somethe quality of the set of security issues was found stable and respondents claimed that the security controls must besatisfactory. The outcome of this step was categories of integrated in systems to achieve efficient protection againstsecurity issues. various threats. There are also needs regarding support for the

D. Determination ofrationale jbr security azssessment development of more usable security solutions anld procedures.In this step, the categories of security issues were analyzed

to determine the rationale for secDurity assessments. During this Security assessment rationale regardin1g security con1trols:analysis expertise in security assessments participated to These staJtemen7ts accentuateS the need o)f moret contcursrent

contsideration of uxsability axnd secuXrity during the early phaxses

1-4244-1I304-4/07/$25.00 ©C2007 IEEE 161I

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

of the systems development. Hence, the ability to detection, and reaction. Moreover, the effects of integratingquantilatively consider the effects of security controls components into syslems need to be assessedregarding the security level taking the utsability qf systems intoconsideration, is needed. This ability is needed already from It was stated that there are needs to be able to verify andthe begi'nning of the systems development. Hence, there is a validate the fulfillment of security requirements regarding: (1)need to be able to consider security controls already during system architectures, (2) system designs, (3) systemthe initial modeling qf the syslems lo be developed. Further, implementations, (4) security solutions, and (5) the systemthere is a need to be able to consider the effects c?f various components. Thereby, security requirements must bedegrees qf integralion ofsecurity controls in the system. formulated with measurable qualities. Moreover, there are

needs for support to integrate verification and validationThe respondents stated that the security requirements must methods and to verify and validate different types of systems,

be based on and balanced with business needs, external for example, service-based systems and systems that handledemands and functional requirements, but also responding to multi-level security.external as well as internal threats. While specifying therequirements on IT security, they stated that it ought to be Security assessment rationale regarding verification andpossible to consider the operational context of systems and the validation: To peijbrm verification and validation requireslifespan of information. The security requirements must be ways to decide whether security requirements are fid/illed.formulated so that they can be used to verify the solutions. This implies the need to be able lo assess IT security based onHence, the security requirements should be based on: (1) the the architectures, designs, and implementations of systems asbusiness' needs and risk tolerance, (2) possible threats, (3) well asfijr various kinds ofsystems.laws, regulations, and directives defined by national andinternational authorities, (4) desired abilities to detect and Several organizations are only allowed to use informationreact on incidents, (5) the security classification of systems that have been accredited. Hence, there is a need to beinformation, and (6) the security services of the infrastructures able to accredit system, components, and services for specificand the specific requirements of applications. application. It was stated to be important that the processes

providing input to the accreditation are understandable and,Security assessment rationale regarding thle balance thereby, could be trusted by those responsible.between different requirements: To achieve balance belweenrequirements on security, on one hand, and business and Security assessment rationale regarding accreditation: Tofinctionality, on the other hand, the ability to assess the supporl accreditation the results of security assessments needeJffcts of the requiremenets is needed, considering bhusiness, to be reliable, valid, transparent, and comprehensible.fiJnctionality, and security. Moreover; the need to base B System operationsecurity requirements on business needs, external demandsand threats, external and internal, points at the need to assess During the operation of systems, it is important to be able tothe ejjkcts qf specific security requirements considering these monitor the security posture. Hence, there are needs toissues. identify: assess, and handle security related incidents,

including the assured detection of information integrityIt was asked for support to identify security controls that violations. Further, support to identify and assess security

fulfill the security and business requirements. To achieve the related events to be able to act before incidents occur wasright balance between prevention, detection, and reaction expressed as urgent. Proactive measures preventing incidentsregarding security incidents was also considered important. must be timely executed. The responders stated their need toFurther, there are needs to be able to integrate secured detect and collect data about security incidents, regarding: (1)components into secured system and exchange security security affecting events, (2) attacks against and incidents insolutions in systems without affecting other security solutions. networks, (3) attempts at information operations, (4)Security solutions must be connected to systems via open intrusions, (5) the manipulation of information, and (6)interfaces and protocols. security vulnerabilities.

Security assessment rationale regarding identification of Security assessment rationale regarding monitoring securityadequate security controls: To determine which security posture: The need to monitor the secrity postur ofsystemscovntrolIs aIre the most sufficient regar ding the speciied calls for security assessment. This must be done regardingsecur-ity requir-ements, the ability to assess the security c/fr cts detection mechanisms and the compilation qf threat pictures.ovf the respective solutiovns is dema;nded. To) determine thesecurity povsture of snystems, it is necessary to) be able to) assess The respondents stated that during system operation it isthe combined effects qf security controls jbr prevention, necessary to be able to adjust system security and functionality

to match the current threats, known weaknlesses, anld businless1-4244-1304-4/07/$25.00 ©C2007 IEEE 162

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

activities. It is also necessary to be able to handle the access Security assessmenit rationale regarding probabilitydemands, access control needs, and access rights of different estimates: Probability estimnations duiring risk analysis requireactors. the ability to assess the security posture ofsystems.

Security assessment rationale regardinig security It is essential to be able to make use of experiences fromadaptability: A continuous awareness of the current security incidents. The respondents stated the need to document thelevel of systems is needed to be able to adapt the security of results of risk analyses, so it could constitute the foundationsystems to match the current threats, knowvn wevaknesses, and for appropriate action plans. Risks ought to be analyzed:business activities. Further, there is a need to be able to assess * quantitatively and in terms of monetary value,the effects of different policies fbr the assignment of access * regarding the security consequences of networkrights and the use ofaccess control mechanisms. connections between organizations and connected

C. Risk management critical infrastructures,* regarding the increased use of net-based functionalityThe respondents pointed at the necessity to be able toansevc,

perform risk management, balancing between confidentiality, and services,integrity, and availability considering the consequences this * regarding the use of systems in new environments, and

will have for the business. Thereby, the risk tolerance of a regarding the use of information infrastructures ownedbusinesses must be identified. and managed by others.

Security assessment rationale regarding balanced security: Security assessment rationale regarding risk analysis:Tobeable to strike a balance between the security Quantitative risk analysis requir-es quantitative security

coharacteristic confidena i andavailabilcity their assessment lo decide the probabilities of events. The securitycaacetsrth iav tobei consequences qf nelwork connections, use qf net-basedrespectlive levels have to be known, i.e., functionality and services, and use of ifrastructures outside,able to assess those characteristics, the realm o,fcontrol ofthe organization need to be assessed.

Respondents claimed that risk management should provide Risk analysis needs to be performed on a regular basis, tothe foundation for the prioritization of IT security controls.Further, risk management must be possible to continuously prvdrevatifmtonsfudtonorheeciyperf ring opertions. work. Risk analysis includes threat analysis, vulnerability

analysis, and consequence analysis. Regarding vulnerabilitySecurity assessment rationale regardinig prioritization and analysis, needs for continuous identification of security flawsrmanagement The prioritization of IT secrity controls and dependencies within systems and infrastructures were

requires that the eJjfcts of the controls can be assessed. expressed.Continuous risk management requires continuous secur.i Security assessment rationale regarding vulnerabilityassessment to quanltij§ the vulnerabililies o?fsystems. an.alysis: Efa,lsl:Eficieni vuclnerazbility afnaflysis requires the abilily to

To obtain solid foundations for decision-making it i assess the security level of systems and informationin raXstruXcturFes incDluding the efecDts ov securiy lacws acndnecessary to have support for the analysis of security costs in 'gt J J ty f

relation to the gained advantage. In risk management, the dependencies.context of use must be considered based on the existing D. Communication and management ofsecurily workthreats, against which the use of security controls provides The respondents pointed at the need for support of theprotection. communication of security-related issues and management of

security work. It is necessary to be able to communicate theSecurity assessment rationale regarling cost-benefit need of security to the business management so that they baseanalysis: To peijbrm cost-benefit analysis of'secur;itycontr-ols, the allocation of resources on the needs to obtain adequatethe ability to assess the changes in security levels resulting levels of IT security. Hence, communication with the businessfrom the corresponding ejfjbrts is required management is needed about IT security, to attain

understanding of the need for information security and theThere are needs to act preventively before security problems need to invest in security. It is important that the Chief

arise. Risk analysis is needed to determine conceivable threats, Information Security Officer (CISO) is trusted by thecontexts and situations where risks occur. The risk analysis management to manage the security work and that theincorporates consequenlce anlalysis anld probability estimationls. management is informed about security issues and the present

status.

1-4244-1304-4/07/$25.00 ©C2007 IEEE 163

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

Security assessment rationale regarding businiess * how to improve IT security, including protectionsmanagement: Valid and illuxstrative data regarding the against malicious code, andsecurity posture of the system and the effects of security- * the connection between actions and securityrelated events and actions needs to be conveyed to the consequences.business management.

Security assessment rationale regardinig the illustration ofThe CISO must continually provide business-relevant security effects: In order lo decrease the abstract nature of

feedback from the security work and security functionality to security, there is a need to be able to clear-ly illustrate thethe system operation management. This communication should effects of system vulner-abilities, specific actions, or neglect oninclude: the security. Further, it is needed to truistvorthily illustr-ate

* how security is handled in the businesses, how security controls affect the security, lo motivate their use.* the relation between security and usability,* security functions as characteristics of IT systems, System developers have to understand the necessity to* which business activities and frunctional requirements consider security issues at an early stage of the development.

affect the security, Further, it is important to understand which security* changes in the threat pictures resulting from the requirements are necessary and how they relate to the security

increased use of net-based functionalities and services, functions and controls.and

* changes in the threat pictures resulting from the use of Security assessment rationale regarding requirements,systems in new environments. design, and concepts: The connection between security

requirements engineering and design decisions, on one hand,Security assessment rationale regarding system operational and secur;ity, on the other hand, has to be illustrated.management: Valid and illuxstrative data regarding the Assessments of security concepts and controls to provide thesecurity postutre vf the system and the cffcts qf security- foundation for how they affect the security in the actualrelated events and actions needs lo be conveyed to the system syslems are needed. This to attain understanding of the needoperational management. for security amongst those engaged in the development

It is also needed to communicate IT security-related issues The competence of the personnel configuring andwithin the organizations, from both a business and a security administrating IT systems needs to increase regarding:perspective, and in relation to incidents. It is necessary to * specification of security requirements,inform about security, security solutions, and consequences of * information classification,security related events and to present security requirements * security configurations in systems of systems,and restrictions. Hence, communication with the users of the * administration of access rights,systems about IT security is needed, including informing about * mitigation of incidents, andthe importance of using and not obstructing IT security * ability to get support for decisions about security actionssolutions. by the management.

Security assessment rationale regarding users of systerns: Security assessment rationale regardinig systemValid and illustralive dala regarding the securily posture of configuration and administration: Adequate securitythe system and the ejfizcls of securily-related events and assessments will increase the conmpetence of personnelactions needs to be conveyed to the users qf the systems. configuring and administrating IT syslems by providing

E. Competence r-egarding injbr-mation securi.ty Jfedback on the secur-ity postulre of systems; i. e. the lack ofsecurity doves not haxve to) be reveaxled the hard wccy.

The respondents stated that the human factor has a largeimpact on security. Hence, there is a need to decrease the F. Allainment andpreservation oftrustsecurity risks through increased competence, improved The respondents stated that it is important to obtain and pre-routines, increased security awareness, and the understanding serve the trust of users in information services and e-businessof that security enables rather than hinder business. More services, which request information to be provided. Increasedspecifically, there are needs to increase the competence of trust in e-services depends, among other things, on securedpersonnel and users regardinrg: availability. Further, there are eeds for organizations to prove

* information and IT security in general, their trustworthiness and ability to maintain and develop it.* vulnerabilities inl IT systems, There are nleeds to be able to measure how the concern of* IT security threats, including malicious code, individuals affects the development anld use of inlformation* consequences ofITsecurity breaches services. Thereby, there are needs to measure the relation

between trust as a subjective measure and technical security.

1-4244-1304-4/07/$25.00 ©C2007 IEEE 164

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

* user awareness,Security assessmenit rationale regardling trustworthiiness: * usability,Security assessments provide data supporling attainment and * dependencies between IT systems,preser-vation of the trustworthiness of syslems. However, this * interfaces between components,requires coniprehensible tools and methods for the * information exchange between components,assessments needed so that they provide trustworlhy results. * connected infrastructures,

* readily accessible information about securityV. SECURITY ASSESSMENT CAPABILITIES vulnerabilities and their exploit,

This section presents a structure of criteria describing the * malicious software,nature of relevant security assessment capabilities. The * the operation of systems,structure is based on the identified rationale for security * information flow,assessment and an additional analysis of the collected data to * dependencies in the security architecture, andobtain a richer understanding of the criteria for security * updatesofproductsandservices.assessment. The criteria deciding the nature of the capabilitiesof useful security assessment methods have been structured C. Ciharacteristics of'secuirity controlsinto the categories: (A) Security assessment domains, (B) The security controls used to increase the security ofSecurity-relevant factors, (C) Characteristics of security systems are central. Increased security can be accomplished bycontrols, and (D) Assessment results. the use of technology, methodology, reorganization, rules and

A. Secur-ity assessmient domiains regulations. There is a need to be able to model and assess

both introduced as well as planned controls regarding:To be useful security assessment capabilities have to * level of security,

address relevant domains (areas). The domains widely differ in * ability to limit the effects of psychological operationsphysical extent, from the IT infrastructure of a nation to a and deception on IT security,single component in an IT system. The physical extent of the * integrationwith other security controls,domain does not, however, necessarily correspond to the * inte nt otherdscomplexity of the assessment. Capabilities for security f

assessment for the following domains is of relevance: * reasonableness,. businesses, * completeness,* national IT infrastructure, sufficiency, and. systems of systems, ability to handle security aspects such as

X information systems, o access control,* servinformasedtion litionsofsystems , o multi-level security in networks and nodes,* service-based coalitionsofsystems o e-identification,* networks, o protection against attacks, and* system solutions, o insider problems.* services within information systems, D. Assessment results* e-businesses,* SCADA (supervisory control and data acquisition) What results an assessment of IT security should produce

systems, and how they are presented depends on various aspects:. web sites, * the purpose of the assessment, e.g. decision support,* system components, motivation, and approval (verification, validation, and

* external components, and accreditation),* filter frmnctions within networks. * the consumers of the assessment results, e.g. business

management, system developers, system administrators,B. Secur-ity-relevanttfictor-s and system users,Security assessment capabilities have to address various * the weighting of security characteristics, e.g.

security-relevant factors. In some cases, security assessment confidentiality, integrity, availability, and survivability,methods are restricted to a specific set of factors decided at the * the status of the assessed system, e.g. undertime of the design of the method. In other cases, the inclusion development, in operation, or under decommission, andof additional factors is an issue of extending the system models * the scope of the assessment considering included systemused for the assessments. Security assessment capabilities aspects (i.e. organizational, technical, human,considering the following factors are needed: contextual, or physical) and system size.

* standards,* human factors,* psychological operations and deception,

1-4244-1304-4/07/$25.00 ©C2007 IEEE 165

Proceedings of the 2007 IEEEWorkshop on Information Assurance

United States Military Academy, West Point, NY 20-22 June 2007

VI. DISCUSSIONS guide the development of tools and methods for the assessment

The quantification of IT security is necessary to render during the operational use of systems.methods and tools for efficient risk and security configuration This study has its limitations, being a case study performedmanagement possible. Still, there are few existing operational within the Swedish Armed Forces and with a limited numbermethods and tools for the assessment of IT security [1]-[5]. To of respondents and documents serving as input. It should,enhance the development of methods and tools in this area, thereby, be seen as explorative. There is need for furtherknowledge of what the users of those methods actually need is studies to establish and specify the detailed requirements ofessential. By understanding the rationale for and relevant tools and methods for assessment of IT security. Still, thiscapabilities of IT security assessment, the possibilities to study could serve as guidance for the development of methodsdevelop adequate and sufficient assessment tools and methods and tools for IT security assessment, complementing thewill increase. current focus on what is technically possible.

The objective of this study is to explore the rationale for ITsecurity assessment and based on that determine relevant VII. REFERENCEScapabilities for methods and tools within this area. For this [1] D. Gilliam, J. Kelly, J. Powell, anid M. Bishop, "'Development of a

purpose, acsestdywaprfrmd itinthSoftware Security Assessment Instrument to Reduce Software Securitypurpose, a case study was performed within the Networ Risk," Proc. of the 10th IEEE Ihternatonal Workshop on EnablingBased Defense (NBD) development effort of the Swedish Technologies: Infrastrutctutre for Collaborative Enterprises, pp. 144-Armed Forces. The rationale for IT security assessment 149. June 2001.implied that IT security assessment must be an integrated part [2] ACSA, "Proc. Workslhop on I nformiation Security System Scoring and

Ranking. Applied Computer Security Associates,"of activities. Further, the results of the study accentuate the fhtp://www.acsac.orgsmneasutrenmentlproceedings/wisssrl-need of IT security assessment tools and methods in all the proceedings.pdf, 2002.stages of the life-cycle of systems, possibly the systems [3] R. Vaughn, R. Henning. and A. Siraj, Information Assurance Measuresdecommission stage can be excluded. For instance, during the and Metrics - State of Practice and Proposed Taxonomy," Proc. of the

Hawaii International Conference on System Sciences (HICSS-36),development of systems, it is essential to assess security Waikoloa, Hawaii, January 6-9, 2003.requirements and security controls. [4] N. Seddigh, P. Pieda, A. Matrawy, B. Nandy, J. Laimbadaris, and A.

The dilemma of security requ-irements specification stems Hatfield,. "Current Trends and Advances in Information AssuranceMetrics," Second Annual Con,fkrence on Privacy. Security and Truist,

from the fact that general security requirements do not result in October 13-15, 2004. http://dev.hil.unb.ca Texts/PST/Idflseddigh.pdfany direct effects on the system, while the specification of [5] D. Geer, "Measurilng Security". Lecture Notes, Training program M3.

l *l l * * l * l- ~~~~~~~~~~~~~~~~~~~15t US8ENIX Sc, iritv Syinposi wns, Vancouver, Canada. J ulIy _3 I-AUgLS tdetailed security requirements, resulting in more direct effects 1t UScJon the system, demands a lot of effort. The results of the study [6] M. Greenwald, C Gunter, B Knutsson, A Scedrov, J Smith, and Sshows that assessment of security requirements should serve as Zdancewic, "Computer security is not a science," Large-Scale Networka way to motivate the extensive efforts needed, related to the Securit Workshop, Landsdowne, VA, 2003.

[7] Gacic, D. (2006). FSA - Framework for Security Assessment ofsecurity of the sy Distributed Information Systens. Masters's thesis, Royal Instittote qfdevelopment. Technology, Stockholmn, Sweden.

The complexity and multitude of different aspects of [8] Hallberg, J., Hunstad, A., Bond, A., Peterson, M., PThlsson, N., (2004),security implies that no single method will suffice to acquire a System IT Security Assessment, FOI-R-1468-SE, Defknee Research

Establishmient., Link6ping, Swevden.complete picture of the security status of systems. Instead, [9] D. S. Alberts, J. J. Garstka, and F. P. Steini, Nenvork centric warfare:there are needs to aggregate results from several different Developingngand leveraging information superiority, C41SR Cooperativemethodsareneedstools. Fugregather,osigle fomset l mferentaResearch Program Publications Series, Department of Defense, USA,methods anld tools. Further, no single method will meet all 1999.aspects of the rationale and capabilities presented in this paper.The stated rationale and capabilities should rather beconsidered as a palette from which parts of the rationale andcapabilities could be chosen as the foundation for thedevelopment of methods and tools.

The results of the study also show that several of the respon-dents have to defend and motivate their security work, both tomanagers and users. Therefore, the results of tools andmethods for the assessment of IT security should be useful forthe dialogue regarding security in general, and the impact onsecurity from security controls as well as from non-securitycontrols. Hence, the presentation of results must be pedagogicenough to be understandable to laymen. The results of thestudy also state the need for methods and tools for thecontinuous monitoring of security, when systems areoperational. Hence, security assessment is not an activityperformed at one instance, it is a process. This should also

1-4244-1304-4/07/$25.00 ©C2007 IEEE 166