Systems Engineering Approach to MPS Risk Management

Kelly Mahoney

mahoney@jlab.org

Presented at the Workshop for Machine Protection in Linear Accelerators

June 8, 2012

Systems Approach (from Tuesday’s talk)

• Top-Down

• Encompasses all aspects of a technical project

• Focus on overall facility mission and goals

• Overall context for development of systems under specific standards, e.g. IEC 61508, 61511, 62062,…

• Accelerator is a system of systems• Similar lifecycle activities apply to all subsystems – rigor depends on risk under

consideration.• Assumptions under one analysis become requirements to another system

• Should be tracked

CERN MPS Workshop 6-8 June, 2012

System Engineering Processes

Slide 3

Agreement Process

Project Process

Organizational Process

Technical Process

Ref. IEC15288/12207/INCOSE Systems Safety Handbook

Stakeholder Requirements

Definition Process

VerificationProcess

RequirementsAnalysisProcess

Architectural DesignProcess

ImplementationProcess

TransitionProcess

OperationProcess

MaintenanceProcess

DisposalProcess

ValidationProcess

80/20 Rule Applied to Systems:

80% of system errors are introduced in the requirements, 20% in all remaining lifecycle stages.

80% of a project’s committed cost are determined during the 1st 20% of actual cost (Requirements + first stages of Architectural Design)

Cost to correct incorrect/incomplete requirements increase by an order of magnitude for each major project activity.

CERN MPS Workshop 6-8 June, 2012

Safety Risk Management

Slide 4

IdentifyHazards

Assess Risk

EstablishControls

ImplementControls

Maintain and Assess

IdentifyHazards

Assess Risk

EstablishControls

ImplementControls

Maintain and Assess

IdentifyHazards

Assess Risk

EstablishControls

ImplementControls

Maintain and Assess

Systems Assurance Software Assurance Cyber Security Assurance

CERN MPS Workshop 6-8 June, 2012

Integrated System Risk Management

Slide 5

IdentifyHazards

Assess Risk

EstablishSystem Level

Controls

ImplementSystem Level

Controls

Maintain and Assess

EstablishSoftwareControls

ImplementSoftwareControls

EstablishSecurityControls

ImplementSecurityControls

Systems Assurance• Central management of hazards and risks.

• Applies to all safety functions• Personnel Safety• Beam Containment• MPS

• Common high level requirements and assumptions; as well as assessments.

• Horizontal link of controls, assumptions, constraints

• Functional testing, Software QA, defensive programming, physical security, …

CERN MPS Workshop 6-8 June, 2012

Integrated System Risk Management

Slide 6

IdentifyHazards

Assess Risk

EstablishSystem Level

Controls

ImplementSystem Level

Controls

Maintain and Assess

EstablishSoftwareControls

ImplementSoftwareControls

EstablishSecurityControls

ImplementSecurityControls

Systems Assurance• Common Requirements Among Standards:

• Management Requirements• Competency in each

specialty area• Graded Approach to system

design, mitigations, and management based on risk

• Hazard and Risk Assessment• Configuration Management

CERN MPS Workshop 6-8 June, 2012

Cyber Security Risk• Not well defined in current safety management practices

• Large emphasis on control system cyber security

• US NIST Common Risk Evaluation Areas• Risk to Integrity• Risk to Availability• Risk to Confidentiality

• Latest version of IEC61508 attempts to address cyber security

Slide 7

CERN MPS Workshop 6-8 June, 2012

Cyber Security Risk• Risk is defined in terms of ‘vulnerability’• Consequences are same as identified in hazard analysis• Failure modes include malicious intent by internal or external party• Mitigations

• Staff training and security awareness• Physical security (limited access)• Least Privileges/Authentication• Segmentation• Passive monitoring• Defensive/Fault Tolerant programming• Forensic capability• Intrusion Response Plan

• Resources for control system cyber security• IEC 62443 Security for industrial process measurement and control • ISA S99.01 Security for Industrial Automation and Control Systems• US NIST “Special Publicaiton 800-53.” Recommended Security Controls for Federal Information Systems and

Organization • US ICW-CERT http://www.us-cert.gov/control_systems/ics-cert/• ENISA Protecting Industrial Control Systems Recommendations for Europe and Member States

Slide 8

CERN MPS Workshop 6-8 June, 2012

JLab Controls Cyber Security• Working to establish controls cyber security program

• Controls Cyber assurance program in process

• Covers all controls

• Risk Based Management

Slide 9

CERN MPS Workshop 6-8 June, 2012

JLab Global Risk Assessment Method

• Started as software risk assessment tool• Applicable to all aspects of risk management• Developed by team with representatives of all enclaves at JLab

• Safety Systems (facilitator)• Network and Infrastructure (Cyber Security)• Business Computing and Information Systems• Quality Assurance• Accelerator Controls and Networking• Experimental Physics• Physics Computing and Data Management• Chief Information Officer/Chief Information Security Officer

• Covers ALL software – from Experiment Data to FPGAs• Now used as basis for configuration management• Assurance process defines minimum activities for a given risk level. Does

not dictate how.

Slide 10

CERN MPS Workshop 6-8 June, 2012

JLab Global Risk Assessment Method

• Six Areas • Direct Risk of Financial Loss• Direct Risk of Loss of Tangible Property• Direct Risk of Harm to People• Direct Risk of Harm to the Environment• Direct Risk of Loss of Mission• Direct Risk of Regulatory Body Intervention

• Each subject evaluated in an FEMA type scenario• Each of the six areas are assigned a score 0-5, based on predefined

unmitigated consequences.

Slide 11

CERN MPS Workshop 6-8 June, 2012

JLab Global Risk Assessment Method

• Score is evaluated on BOTH max value of single category AND sum of all scores

• Some risks that were below the radar now pop up as more important• Because the system owner evaluates the risk, they are invested in the process

• Evaluator determines risk acceptance level of unmitigated and mitigated risk.• Intolerable• Unacceptable• Tolerable• Acceptable

• Amazing agreement between evaluation scores and risk acceptance levels among different enclaves.

Slide 12

CERN MPS Workshop 6-8 June, 2012

Functional Risk Assessment Methods Used for JLab MPS Safety Functions

• Event Tree• Risk Matrix• Risk Graph• Layer of Protection Analysis

All of the above can be used to assign a SIL level to a safety function.

Slide 13

CERN MPS Workshop 6-8 June, 2012

Conclusions

• Systems approach allows early identification and mitigation of operational risks

• Same approach can be used for all safety related systems• Correct Requirements are critical for correct and efficient

implementation of a protection system.• JLab Global Risk Assessment tool can uncover risks that fall

below radar in other assessments• SIL methods can be used to manage MPS safety functions’

Slide 14

CERN MPS Workshop 6-8 June, 2012

Additional Slides:

Slide 15

CERN MPS Workshop 6-8 June, 2012

MIL-STD-882E System Safety

Slide 16

Ref. MIL-STD-882E

CERN MPS Workshop 6-8 June, 2012

882 E Software Safety Criticality Matrix

Slide 17

Ref. MIL-STD-882E

CERN MPS Workshop 6-8 June, 2012

Software Assurance

Slide 18

CERN MPS Workshop 6-8 June, 2012

A Note on Safety Integrity Levels (SILs)

• A Safety Integrity Level applies to a mitigation function performed by a system.

• Individual SILs are determined by the difference between (unmitigated risk + risk reduction of other safety layers or functions) and acceptable risk goal.

• Examples:MPS Safety Requirement:

Prevent catastrophic loss of two or more superconducting dipole magnets due to a beam loss event.

Other Layers

SF1:

Slide 19

IEC61508Lifecycle Model

Concept

Overall scopedefinition

Hazard and riskanalysis

Overall safetyrequirements

Safety requirementsallocation

Back to appropriateoverall safety lifecycle

phase

Overall operation,maintenance and

repair

Overall modificationand retrofit

Overall safetyvalidation

Decommissioningor disposal 16

Safety-relatedsystems:E/E/PES

Realization(see E/E/PES

safetylifecycle)

Safety-relatedsystems:

othertechnologyRealization

Overall installationand commissioning

Overall planning

Overalloperation andmaintenance

planning

Overallinstallation andcommissioning

planning

Overallsafety

validationplanning

External riskreductionfacilities

Realization

Analysis Phase

Realization Phase

Operations Phase

© K Mahoney/S. Prior 2002-2004

USPASJune, 2004

IEC Safety Allocation