Managing the development ofsecure electronic banking

MSc Thesis

GUC

Tom Nilsen

Problem• We have a security analysis for projects• The analysis should ensure compliance to our

security requirements• We think that the analysis contributes to secure

systems• We do not know the actual contribution• And security level varies across projects• We want to measure

– The the security status of system at delivery – The contribution of the Sec. analysis to security

Security requirements

The Bank'sSecurity Frameworkhas approx200 Securityrequirements

Security Analysis

ReccomendedSecurity solution or approach

Will the projectuse the solution?

If not, please describe…..

The relations that leads to secure systems

Sec. requirements Security analysis Security metrics

200 requirements 60 questions &answ

SecurityStatusatdelivery

ResearchQuestion

Research question

• How to define a security metric that measures the security status of a new system at delivery

• Security status:– Compliance to recommendations in the analysis– Compliance to the security framework

and requirements

The MSc work

• Review of previous work

• Choice of methods

• Defining a framework for the metric

• Defining the content of the metric

• Testing and improving the metric

• Analysing the project work and findings

Review of previous work

• Theory on security metrics– Security Metrics guide, 800-55 NIST

– A method for security metric design, Frost/Snekkenes

– Careful: "simplifying a complex security matter down to a number", McHugh/McCallam

• Measuring security in practice– Risk analyses and Checklists

– Security SLAs (I.e.the Banks own Security metric)

– ISF metrics

Choice of methods• The Authors role: The risk of being to involved and

proprietary:– Reference group and external reviews

• Defining a framework for the metric– Studying Previous work and apply principles– Qualitative Content analysis

• Security Analysis, Security Requirements• ISF metrics and ideas

• To define the topics to be measured– Analysing cross referencing and structuring findings from the

Content analyses– Semi structured interview of relevant managers

A method for security metric design (Frost/Snekkenes)

SecurityMetric

RISK MGMNT, SECURITY REQ, COMPLIENCE C I A TO SELECTED

RISK OF NON-COMPLIANCE

"A clear, known, agreed and understood definition…"

StakeholdersWho needs the metric?STAKEHOLDER REASON

Project manager / Business developer

To document security status and learn from their own performance.

System owner / Business owner

To know the security status when they take over the responsibility from development.

Top management To manage risk and to quality assure facts before they report externally to reduce their personal liability.

Management of System development

To collect statistics to see trends of systematic problems and flaws in development processes that lead to security problems.

Head of IT Security To assist top management in managing risk and to manage and improve the security process and deliverables (i.e. Security analyses).

"Clear line of sight"Accountability

• A clear connection betweenthe stakeholders actions and the measured result

– Scope of project

– Relevant security requirements• Selected requirements for Systems

Security program maturity:Ambitions versus reality NIST 800-55

ASSESSMENT OF THE BANK- Security program since 1994

-Policy- Baseline Sec requirement 1995- Security Analysis 1997

- > 400 conducted- Outsourcing and Security 2001

-Security program- Security testing, code review, IDS

CONCLUSION:- between level 3 and 4- data available at medium difficulties

=> focus on AUTOMATION of Data collection and testing

Maturity is is influenced by Mergers, Management +++

NIST 800-55 Metric detail form

IMPLEMENTATION EVIDENCE

Does the Audit trail provide a trace of user action ?

YES ?NO ?

How can we help in assessing?

ISF Metrics

ASPECTS AREAS SECTIONS

CONTROLS

Ctrl-DETAILS

SM SEC MGMNT

7 32 Ca 252 Ca 745

CB CRIT BUS

6 25 121 348

CI COMP INST

6 31 Ca 250 Ca 600

NW NETWORK

5 24 Ca 193 Ca 455

SD SYST DEV

6 23 Ca 143 Ca 399

TOT

30 135 Ca 1000 Ca 2500

Factors that increase # of incidents

ISF FIRM Special circumstances for a system or information recourse:         Subject to a high degree of change         Widely extended geographically         Large in scale         Complex         Immature         Accessible to external parties         Used to support call centres

The content of Security Analysis

Sec. requirementsSecurity analysis Security metrics

200 requirements60 questions &answ

SecurityStatusatdelivery

ResearchQuestion

COVERAGE-Authentication & Identity management-Access control & role based access mgmnt-Password process # 30-Access control for programs- Securing appl.data- Network & data exchange- Secure code & pentest- Audit, logs & incident mgt

- Resilience & contingency

- Tech. Platform & infra- System maint.& Operation- Agreements (SLA, 3.party)

- RISK SUMMARY# 80

Why 30 Q on Access control?• The Banks approach:

– Ease of use

• Reduced sign-on

– Ease of administration

• Single point of admin

– Advantage of scale

• Standardisation

• Re-use secure solutionsand processes

Sec. requirements Security analysis Security metrics

200 requirements 60 questions &answ

SecurityStatusatdelivery

ResearchQuestion

EXTERNAL VALIDITY:INTERNATIONALSECURITY STANDARDISF SOGP AND METRICS

HEALTH CHK

Security metric work sheet

Prototype metricCOVERAGE-Authentication & Identity management-Access control & role based access mgmnt-Password process # 50-Access control for programs- Securing appl.data- Network & data exchange- Secure code & pentest- Audit, logs & incident mgt

- Resilience & contingency# 90- Tech. Platform & infra- System maint.& Operation- Agreements (SLA, 3.party)# 116- RISK SUMMARY

SecurityMetric

Challenges

• Are questions asked– On the most important topics– With the right approach to

the mix of security and re-use of secure solutions– Right level of details

(Health check 170 <> Survey 2500)

• And in a way that we can– Collect answers and verify by testing?– Build on and compare to answers in Sec.Analysis– Use the measurement as a part of security

documentation?

SecurityMetric

Conclusions from the Bank

• Final metric versus MSc version– "minimal"MSc version in English

– Proposal and description for building:• new Security Analysis

• and Metric on WEB

• The bank needs– A metric that focus the weak security areas

– With relevant info for risk management

• The bank do not need a metric => 3.7 secure

Focus in the MSc report

• For the sake of the reader:

• We have chosen a few metric topics– Discuss them in more detail

• Instead of "scratching the surface" of 115 metrics

SecurityMetric

Testing and improving the metric

• The first test candidate:HR and Salary system run by an external SP

• Responsible roles had a need for a status

• Logistics– Gather data from Sec.Analysis– Arrange meeting with responsible roles– Establish/Confirm facts– Analyse non compliance areas

SecurityMetric

Challenges measuring

Implementation evidenceNIST 800-55

An evaluation of the metric

• Value to the bank • Validity: Internal , External • Reliability:

– improve by Implementation evidence– Trained ITS consultant to assist user

• The workload: => automation – Timing / project phase is important

SecurityMetric

Evaluation of the MSc work

• The authors role– Objectivity, the risk of being too involved:

• Reference group but difficult to control

– Proprietary setting - "home grown" orknowledge of general interest?

• Cross ref. to an open standard – SOGP

• Applied principles from science and ISF practice

=> "How to generalize the metric"

"How to generalize the metric"

The business must have defined which Security requirements/standards• Select the requirements to comply with• Define stakeholders and their needs• Evaluate the security program maturity• Define a framework for the metric• Define the actual content of the metric with cross ref. to the

requirements for validity • Use implementation evidence in forming questions• Test and improve the metric• Decide a process and IT-support for the metric• Remember that introducing a security metric is developing your

organisation

Further work

• Finish a Norwegian version of the metric– Update the Sec.Analysis and metric to

new version of ITS framework– Decide minimum version with final questions– Decide a process for the metric

• Stakeholders support

– IT-support/automation for the metric

Conclusion

• We have developed a prototype metricaccording to the assignment

• We have tested and improved it• We have described the steps for a final Norwegian

version of the metric• The process and the framework for designing the

metric is based on scientific principles and "industry practice"

• We hope to inspire others to build security metrics

New generation ISF metrics

• Web based in stead of Excel

• XML cross reference– SOGP, 17799, Cobit

• META Database– All controls from

important standards

New generation SA and S metrics

• Web based– On ISF metric

– With Bank extensions

• XML cross reference– SOGP, 17799, CobiT

– SA and metrics cr.ref

• META Database– Also BSR controls