1

Metrics and Maturity

Cartagena de Indias© ISM3 Consortium 2009

2

Managing is achieving results with the resources available for it.

There are specific activities for management that we will call

“Management Practices”.

Management

3

Testing: Assessment of whether process outputs are as expected when test data is put in.

Management Practices

4

Monitoring: Checking whether the outputs of the process and the

resources used are within normal range.

Management Practices

5

Improving: Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources.

Management Practices

6

Planning. Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process.

Management Practices

7

Assessment. How well the process matches the organization's needs and compliance goals.

Management Practices

8

Audit. Whether the process inputs, activities and results match their documentation.

Management Practices

9

Certify: Whether the process inputs, process documentation, activities and results comply with a pre-defined standard, law or regulation.

Management Practices

10

Benefits realization: Show how achieving security

objectives contributes to achieving business

objectives.

Management Practices

11

The more sophisticated your management practices, the higher your capability.

Management and Capability

12

Therefore, there is a strong link between the metrics used and capability.

Management

You can perform few management practices without metrics.

13

Types of Process Metrics

A quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements

It is possible to audit the capability of a process checking the metrics used to manage it.

14

Types of Process Metrics

Activity: Number of outputs produced and their mean age.

15

Types of Process Metrics

Scope: Percentage of all inputs producers covered by this process.

16

Types of Process Metrics

Unavailability: Number, frequency and duration of interruptions in the normal operation of the process.

17

Types of Process Metrics

Effectiveness: Number, mean time between inputs and percentage of Inputs that produce an Output.

18

Types of Process Metrics

Efficiency: Ratio between the number of outputs submitted and the available resources for this process in actual use.

19

Types of Process Metrics Load:

Percentage of resources reserved for the process in actual use.

20

Types of Process Metrics

Quality: Measure of the fitness for purpose of the outputs.

21

Description of what is measuredHow is the metric measuredHow often is the measurement takenHow are the thresholds calculatedCurrent range of values considered

normal for the metricBest possible value of the metricUnits of measurement

Metrics Specification

22

What are metrics good for?

Enable performing management practices. Determine whether security objectives are met (test

success); Show how security objectives contribute to business

objectives; Measure how changes in a process improve (or not) the

ISM system; Inform decisions to fix or improve the ISM processes.

23

What are metrics good for?

Detect significant anomalies (tell normal from abnormal, saving investigation efforts);

Diagnosis Business Decision

Fault in Plan-Do-Check-Act cycle leading to repetitive failures in a process

Fix the process

Weakness resulting from lack of transparency, partitioning, supervision, rotation or separation of responsibilities (TPSRSR)

Fix the assignment of responsibilities

Technology failure to perform as expected.

Change / adapt technology.

Inadequate resources . Increase resources or adjust security targets.

Security target too high. Revise the security target if the effect on the business would be acceptable.

Incompetence, dereliction of duty. Take disciplinary action.

Inadequate training. Institute immediate and/or long-term training of personnel

24

Security Investment, Maturity Level & Risk

Security Investment

Risk

Risk Reduction/Additional SecurityInvestment

ISM3 Maturity Levels

(Qualitative Graphic. Risk Reduction / Extra Security Investment, scaled x40 for readability)

25

ISM3 Maturity Levels (examples)

ISM3 Basic Level - Significant risk reduction from technical threats, for a minimum investment in essential ISM processes.

For organizations with low Information Security Targets in low risk environments.

ISM3 SMEs Level - Highest risk reduction from technical threats, for a significant investment in Information Security processes.

For organizations with high Information Security Targets in normal or high-risk environments.

ISM3 Military Level - Highest risk reduction from technical and internal threats, for a high and optimized investment in Information Security processes.

For organizations affected by specific requirements (such as utilities, and financial institutions) with high Information Security Targets in normal or high-risk environments.

26

3 – Definición Objetiva de Madurez

Inde

finid

o

Defin

ido

Ge

stio

nad

o

Con

trola

do

Op

timiz

ado

Prácticasde Gestión D

ocu

men

taci

ón

Act

ivid

ad

Alc

ance

Dis

po

nib

ilida

d

Efic

aci

a

Carg

a

Cob

ert

ura

Calid

ad

Efic

ienci

a

OptimizaciónEvaluaciónMejora de CalidadPlanificaciónRacionalizaciónMonitorizaciónPruebasCertificaciónAuditoria

27

ISM3 Capability Levels

CapabilityLevel

Metrics Requirements Enabled Managed Practices

Basic Documentation Audit and Certify.

Defined Basic, plus Activity, Scope, Unavailability and Effectiveness

Basic, plus Test

Managed Defined, plus Load Defined, plus Monitor, Benefits Realization, Planning and removing weaknesses before they produce incidents, and getting feedback on the result of changes.

Controlled Managed, plus Quality Managed, plus Assessment and removing bottlenecks that hamper performance.

Optimized Controlled, plus Efficiency Controlled, plus finding points of diminishing return and making trade-offs.

28

Learn to implement High Performance Security Management Processes http://cli.gs/ism3

Web www.inovement.esVideo Blog youtube.com/user/vaceitunoBlog ism3.comTwitter twitter.com/vaceitunoPresentationsslideshare.net/vaceituno/presentations

Articles slideshare.net/vaceituno/documents

29

THANK YOU