Critical systems engineering, 2013 Slide 1

Critical Systems Engineering

Prof Ian Sommerville

Critical systems engineering, 2013 Slide 2

Course aims

• When you have completed this course, you should:

– understand what is meant by a critical system and have learned about different types of critical systems.

– understand the fundamental concepts of system dependability and security and know about the key technical activities – specification, development and assurance - in critical systems engineering.

– understand that critical systems are usually not simply technical systems but are socio-technical systems that include people and processes and are profoundly affected by organisational politics and policies.

Critical systems engineering, 2013 Slide 3

Presentation

• 2.5 hour slot, one afternoon per week (normally Thursdays) from 13.30 to 1600.

• Benefits of this approach– Gives time for coverage of a topic so that you don’t

forget material between lectures

– Provides an opportunity to integrate work on case studies with the lecture material

– Allows time for class exercises where required

• Problems– More tiring for students (and lecturer) than separate

lecture slots

Critical systems engineering, 2013 Slide 4

Course topics

• Critical socio-technical systems, System failure

• System dependability, requirements engineering

• Dependability requirements

• Dependability engineering, fault tolerant system architectures

• Security engineering

• Dependability assurance

Critical systems engineering, 2013 Slide 5

Course topics

• Human and organisational factors

• Critical national infrastructure

• Resilience engineering

• Cybersecurity

There may be some changes in the order these topics are presented

Critical systems engineering, 2013 Slide 6

Assessment

• Examination (60%)– Covering all topics in the course

• Coursework (40%)– I piece of coursework which will involve reading

documents and applying a method for dependability requirements specification

Critical systems engineering, 2013 Slide 7

Web site

http://www.software-engin.com/teaching/critical-systems-engineering-2013

Copies of slides are on Slideshare (as well as studres) and will be linked from the course web site.

Twitter: @StACS5032CritSy

For news and changes

Critical systems engineering, 2013 Slide 8

Critical systems

Critical systems engineering, 2013 Slide 9

Critical system essentials

ReliabilityThe system must operate without serious failures

Availability The system must be available to deliver services when requested to do so

SafetyThe system should not harm people or the system’s environment

SecurityThe system must be able to protect itself and its data from malicious use

Critical systems engineering, 2013 Slide 10

Classes of critical system

• Safety-critical systems

– Failure results in loss of life, injury or damage to the environment e.g. chemical plant protection system;

• Mission-critical systems

– Failure results in failure of some goal-directed activity e.g. spacecraft navigation system;

• Business-critical systems

– Failure results in high economic losses e.g. customer accounting system in a bank;

• Infrastructure systems

– Failure results in a loss of infrastructure capability e.g. power distribution control system, broadband communications, etc.

Critical systems engineering, 2013 Slide 11

Critical systems stack

Physical infrastructure

Infrastructure systems

System hardware

Operating system and middleware

External systemsCritical system X

Critical systems engineering, 2013 Slide 12

System dependencies

• Independent critical systems– Infrastructure/hardware is part of the system

– System operation is not dependent on external systems

– Embedded control systems such as those in medical devices

• Critical software systems– Usually rely on commodity hardware/OS

– System operation is dependent on external infrastructure provision

– Hospital appointments system

Critical systems engineering, 2013 Slide 13

Systems of systems

• A critical system is rarely a single system but is a network of several software-intensive systems as well as infrastructure systems

• Systems that support organisational needs (e.g. an inter-bank payments system) have to be designed to be robust so that they can cope with failures and unavailability in the other systems on which they depend

Critical systems engineering, 2013 Slide 14

Systems of systems

• Systems of systems (SoS) are complex socio-technical systems with

– Different owners and management policies

– Distributed operation

– Heterogeneous hardware and software

• Individual systems may be part of several SoS so

– Conflicting requirements from different uses of the system

– Complex negotations may be required when system changes are to be made

Critical systems engineering, 2013 Slide 15

Socio-technical systems

• Socio-technical systems include IT systems and the social and organisational environment in which these systems are used

• Key influences are human behaviour, organisational processes and policies, regulations, culture

Critical systems engineering, 2013 Slide 16

Software-intensive system

Laws, regulations, custom & practice

Organisational policies and culture

Businessprocesses

System users

Socio-technical systems

Organisational strategies and goals

Social and political environment

Critical systems engineering, 2013 Slide 17

Regulation

• Regulators are government-appointed bodies whose job is to ensure that companies and other bodies conform to national and international laws.

• This normally involves interpreting the law and government policy and establishing standards and regulations that must be followed by industry.

• Examples of regulators– Data protection authority

– Civil Aviation authority

– Bank of England / Financial Services Authority

– Ofgen – electricity and gas regulator

Critical systems engineering, 2013 Slide 18

Regulators and critical systems

• Some critical systems may have to be certified by regulators before they are put into use. This is particularly true for safety-critical systems.

• This means that the regulators check that the system is conformant to current regulations and standards.

– This normally involves the system developers producing evidence (a safety case or a dependability case e.g.) that demonstrates that the system is dependable.

• Examples of certifiers– Civil Aviation Authority – aircraft systems

– Medical Devices Directorate – medical devices and instruments

Critical systems engineering, 2013 Slide 19

System criticality

• Primary critical systems– Systems where system failure leads directly to an

incident that has an associated loss of some kind

– Typically, these are control systems or systems that are closely associated with a control system

– Example – failure of engine management system in a car causes engine to cut out while driving

• Secondary critical systems – Systems whose failure may (but need not) lead to

failure in an associated system that then leads to loss of some kind

– Example – medical information system that maintains incorrect information about treatment

Critical systems engineering, 2013 Slide 20

Critical systems engineering

• Focus is on the use of techniques and methods to develop dependable and secure systems.

• The costs of critical system failure are so high that development methods may be used that are not cost-effective for other types of system.

• An important aim for many critical systems is certification and the development process has to be geared to achieving such certification.

• Certification costs can exceed development costs.

Critical systems engineering, 2013 Slide 21

Software engineering for critical systems

• Formal methods for systems specification and analysis.

• Use of specialized tools such as model checkers and static analyzers.

• Risk-driven approach to system specification and management.

• Argumentation systems to support the development of dependability cases.

• Disciplined configuration management of all software and hardware.

• Detailed process record keeping.

Critical systems engineering, 2013 Slide 22

Denver airport baggage system• System to control baggage

transfer at the (then new) Denver airport in the USA.

• Example system illustrating some of the issues and problems that arise with complex socio-technical critical systems.

• This is a business critical system – the effective functioning of the airport relies on its baggage handling system.

Critical systems engineering, 2013 Slide 23

System overview

• New baggage handling system, which was software controlled, based on individual baggage carts rather than conveyor belts.

• Intention was automated handling so that there was no manual handling of bags from plane to passenger.

• Very complex hardware/software system procured from several different companies.

• Encountered complex organisational, hardware and software problems.

Critical systems engineering, 2013 Slide 24

“Denver airport saw the future:

It didn’t work”– Baggage system did not recognise blockages and

simply continued to unload bags

– Bags fell off the carts due to timing problems

– System loaded bags onto carts that were already full

• At the time of the airport opening, only a very limited version of the system was available.

– This system had a 10% error rate (i.e. 10% of bags were delivered to the wrong place)

• Airport 18 months late opening

• System abandoned in 2005

Critical systems engineering, 2013 Slide 25

Key points

• Economic and human activities are increasingly dependent on software-intensive systems. These can be thought of as critical systems.

• For critical systems, the costs of failure are likely to significantly exceed the costs of system development and operation.

• Consequently, the dependability and security of the system are the most important development considerations.

• Critical systems are often subject to external regulation.