Formal Modelling and Analysis of Socio-technical Systems
Christian W ProbstTechnical University of Denmark
inria, Rennes, June 10, 2016
3
Motivation
• Attacks on systems and organisations exploit non-technical aspects.
• How do we formalize these human elements? – Social Engineering – Honest mistake – “Experimentation”
4
The TRESPASS Approach to Risk Assessment
• Information security threats to organisations have changed completely over the last decade.
• New attacks cleverly exploit multiple organisational vulnerabilities, involving physical security and human behaviour.
• Defenders need to make rapid decisions regarding which attacks to block, as both infrastructure and attacker knowledge change rapidly.
5
Key project goals
Predict complex attack scenarios spanning digital, physical and social engineering steps
Prioritise these scenarios via a planning tool that tells defenders where to expect the most serious issues
Prevent attacks by calculating and comparing cost-effectiveness of countermeasures
6
From organisational models to attacks
Attack Attack Attack Attack Attack Attack Attack Attack Attack
• Attack trees• Descriptive Method• Success based on experience and imagination of the
consultant/defende
• System Model• Analytic approach• Success based on experience and imagination of the
modeller
7
ModelOrganisation
Visualise
AnalysisModel
Choose most beneficial
countermeasures
Identify attacks and their impact
The TRESPASS Process
8
home
McScrooge card pin
pin
city Money BinMB
bank
$$$
$$$computer
C
settop boxSB
MB,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Beagle Boys
C: i
door
trust M: m
get cash
get cashat ATM
goto ATM get card[(pin,X)], (pin,X)& input cash at ATM
get Charlie’s credentialsand perform action
input cashat ATM
in cash at ATM
get Alice’s credentialsand perform action
get credentials
get card
get pin
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
in cardat Alice
SE Alicein Card
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
SE Alicein Pin
get card
perform inat Alice
in cardat Alice
SE Alicein Card
in pinat card
input cashat ATM
in cash at ATM
9
Organisation = Socio-technical Model
Organisation
• InfrastructureIT infrastructure
• Policies • Employees• Visitors
Socio-Technical System
• Interaction between people and technology in workplaces.
• Interaction between society’s complex infrastructures and human behaviour.
10
Our Contribution
• Explore the formal modelling and analysis of systems containing informal components
• For risk assessment in socio-technical systems.
14
The Attack Navigator
• Identifies and ranks attacks on an organisation.– Supports prediction, prioritisation, and prevention of
complex attack scenarios.
• Analytic risk assessment based on system model of organisation – infrastructure, policies, and employees.
• Identifies all possible attacks in the model!– Based on attacker profiles.
19
Attack Navigator Map
• Models real-world systems – physical, virtual, social layer.– Relevant properties/actions – Maps to an analysable formalism – Apply static analyses and model checking
• Directed graph. • Models all locations that can be accessed/store data. • Models all entities that can move in the system.
22
Socio-Technical Modelling Language
• Elements – Mobile components actors, programs – Data knowledge, artefacts – 3 basic actions • Input read, find something • Output print, loose something • Movement well, move around
24
home
McScrooge card pin
pin
city Money BinMB
bank
$$$
$$$computer
C
settop boxSB
MB,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Beagle Boys
C: i
door
trust M: m
27
What are “valuable” assets?
• Depends on the modelled organisation:– Virtual or physical assets,– Operational goals, or– Global policies.
• Assets can be accessed by actors or processes, and• Can move/be moved around the system!
Ø This blurred location results in additional challenges!
29
home
McScrooge card pin
pin
city Money BinMB
bank
$$$
$$$computer
C
settop boxSB
MB,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Beagle Boys
C: i
door
trust M: m
get cash
get cashat ATM
goto ATM get card[(pin,X)], (pin,X)& input cash at ATM
get Charlie’s credentialsand perform action
input cashat ATM
in cash at ATM
get Alice’s credentialsand perform action
get credentials
get card
get pin
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
in cardat Alice
SE Alicein Card
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
SE Alicein Pin
get card
perform inat Alice
in cardat Alice
SE Alicein Card
in pinat card
input cashat ATM
in cash at ATM
30
Systematic generation of attacks from TRESPASSmodels
• Defender specifies undesirable states of the organisation – the destinations of an attacker!
• Attack navigator identifies all possible ways of reaching that states – the routes of an attacker!
Goal: use M’s card to get $$$;
home
Margrethe card pin
pin
city ATMA1
bank
$$$
$$$computer
C
settop boxSB
A1,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Fred
C: i
door
trust M: m
31
How do I get the money?
home
Margrethe card pin
pin
city ATMA1
bank
$$$
$$$computer
C
settop boxSB
A1,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Fred
C: i
door
trust M: m
32
Ask the TRESPASS Attack Navigator!
home
Margrethe card pin
pin
city ATMA1
bank
$$$
$$$computer
C
settop boxSB
A1,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Fred
C: i
door
trust M: m
33
Outside Home Living Room
Remoteserver
Settopbox
Bank Account
M
Dongle
Malware
The Attack Navigator
34
• Goal: Actor Fred, Be at A1, perform action with card and pin
• Policy: have card and pin• Goal: Obtain card and pin
– Location: Magrethe– Location: Home
• Action: Go to home• Policy: be trusted by M
– Action: Impersonate trusted person or
– Action: Break in• Action: Steal• Action: Social engineer• Action: Go to ATM, Obtain money
home
Margrethe card pin
pin
city ATMA1
bank
$$$
$$$computer
C
settop boxSB
A1,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Fred
C: i
door
trust M: m
Goal: use M’s card to get $$$;
$$$
card
pin
• This is a navigator map and the identified routes!
• Specify what to protect – the attack navigator shows you how that will fail (in your model).
35
Attack Generation
• Identify Attackers based on policy to invalidate. • Identify target locations in system.• Generate attacks for reaching target location.
– This will identify and obtain required assets to perform any of these actions, and obtain all assets required to reach the target location.
• Move to Target Location and Perform Final Attack.
36
Policies in TREsPASS
• The TREsPASS policy-specification language is a combination of policies and processes.
• Captures local access-control policies and global organisational policies.
• The resulting policy language is designed to be easily extended.
38
Powerful Policy Resolution
• Policies and model are translated to Datalog– inference rules, specified as Horn clauses
• Datalog programs deduce required credentials against knowledgebase (model!)
• Basis for powerful extensions, eg, PEAL
40 TREsPASS Model of the Cloud Case Study
RoomInternal
DoorDataCenter
RoomDataCenter
server1
vm1
fileX,42
Grey
Finnpin,4
pwd,3
cardpin, 4
owner, Finn
server2
dataStore
DoorInternal
Hallway
DoorHallway
Outside
laptop switch1
WindowInternal
WindowDataCenter
Ethanpin,2
pwd,1
cardpin, 2
owner, Ethan
Bigpin,8
pwd,9
cardpin, 8
owner, Big
Sydneypin,6
pwd,7
cardpin, 6
owner, Sydney
Cleopin,5
cardpin, 5
owner, Cleo
Terrypin,10
pwd,11
cardpin, 10
owner, Terry
41
Generated Representation of Actors, Assets, and Policies
isActor(’Terry’).isItem(’id001’).hasName(’id001’,’card’).contains(’Terry’,’id001’).isData(’id002’).hasName(’id002’,’pin’).contains(’Terry’,’id002’).contains(’id001’,’id002’).
Terry• Terry is an actor, • who has a card (id001) • and a pin (id002) • that is also stored on the
card.
42
Generated Representation of Actors, Assets, and Policies
isPolicy(’id003’). hasName(’cred001’,’card’). requires(’id003’,’cred001’). enables(’id003’, ’move’). contains(’DoorInternal’,’id003’). isPolicy(’id004’). requires(’id004’,’cred001’). hasName(’cred002’,’pin’). requires(’id004’,’cred002’). enables(’id004’, ’move’). contains(’DoorDataCenter’,’id004’).
Policies• A policy (id003), • Requires a card (cred001) • And enables "move"• At DoorInternal.
• A policy (id004),• Requires a card (cred001)• And a pin (cred002)• And enables "move"• At DoorDataCenter.
45
Models vs Attack Trees
• Models and policies guide the creation of attacks– ”white box testing” of system models– Cluttered models result in cluttered attack trees
• Level of detail is important• The model contains only relevant artefacts
– Technical details should be ”hidden” from the model– Instead they are added in a library approach
46
Libraries
• Libraries support individualization of models, attack trees, attackers, defenders, etc– By providing patterns for elements.– Establish interfaces to central repositories, community
portals.
47
Attack Patterns
• Explain how attacks can be implemented.– Dependent of the type of the actual elements of the attack.– Credit card with a chip vs magnetic strip.
• Separate the what (attack) from the how (concrete attack steps) in attack trees.
IN A item:I actor:C
A steals I from CA social
engineers C to give I
MAKE A B IN B item:I actor:C
A threatens B to execute IN B I C A blackmails B
A collects intel about B A blackmails B to execute IN B I C
A bribes B to execute IN B I C A social engineers B
A impersonates authority
A orders B to execute IN B I C
48
Applying Patterns
• Patterns are applied to the generated attack trees.• Leaf nodes are matched against available patterns.
– Attack pattern library performs property lookups, and – Decorates the leaf nodes, e.g., with necessary resources, skill level,
or risk of detection.
label match { case IN attacker item container:
// get type attacker from attacker profile // get type item from knowledge base // get type container from knowledge base // insert APL attacks that extract item from container
case MAKE attacker actor action: // get type attacker from attacker profile // get type actor from attacker profile // insert APL attacks based on types and action //...
}
49
get key
get key from McScrooge
get key from shelf
IN BeBo key McScrooge
MAKE BeBo Donald IN Donald key McScrooge
get key
get key from McScrooge
get key from shelf
IN BeBo key McScrooge
MAKE BeBo Donald IN Donald key McScrooge
BeBo steals key from
McScrooge
BeBo social engineer McScrooge to give
them key
IN A item:I actor:C
A steals I from CA social
engineers C to give I
MAKE A B IN B item:I actor:C
A threatens B to execute IN B I C A blackmails B
A collects intel about B A blackmails B to execute IN B I C
A bribes B to execute IN B I C A social engineers B
A impersonates authority
A orders B to execute IN B I C
IN A item:I actor:C
A steals I from CA social
engineers C to give I
MAKE A B IN B item:I actor:C
A threatens B to execute IN B I C A blackmails B
A collects intel about B A blackmails B to execute IN B I C
A bribes B to execute IN B I C A social engineers B
A impersonates authority
A orders B to execute IN B I C
get key
get key from McScrooge
get key from shelf
IN BeBo key McScrooge
BeBo steals key from
McScrooge
BeBo social engineer McScrooge to give
them key
BeBo threaten Donald to execute
IN Donald key McScrooge
BeBo blackmails Donald
BeBo collects intel about Donald
BeBo blackmails Donald to execute
IN Donald key McScrooge
BeBo bribes Donald to execute
IN Donald key McScrooge
BeBo social engineers Donald
BeBo impersonates authority
BeBo orders Donald to execute
IN Donald key McScrooge
MAKE BeBo Donald IN Donald key McScrooge
50
home
McScrooge card pin
pin
city Money BinMB
bank
$$$
$$$computer
C
settop boxSB
MB,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Beagle Boys
C: i
door
trust M: m
get cash
get cashat ATM
goto ATM get card[(pin,X)], (pin,X)& input cash at ATM
get Charlie’s credentialsand perform action
input cashat ATM
in cash at ATM
get Alice’s credentialsand perform action
get credentials
get card
get pin
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
in cardat Alice
SE Alicein Card
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
SE Alicein Pin
get card
perform inat Alice
in cardat Alice
SE Alicein Card
in pinat card
input cashat ATM
in cash at ATM
52
Dealing with Uncertain Data
• Data available in practice often based on expert opinion – Can produce a wide range of estimates – Analysis methods must be able to cope
• Challenges – Data must be presented in an unambiguous form – Link data to the appropriate Basic Attack Steps – The uncertainty itself must be represented
• Can be addressed by sensitivity analysis or fuzzy reasoning
53
Analyses
• Valuate attack trees based on different factors– Impact on organisation, resources of the attacker,
probability of success– Attacker profiles guide resources and probability of
success• Resources are important
• Compute different quantitative properties
– Success/budget– Impact– Cost (time, resources, etc)
54
Data
Key metrics:probability, cost, time
Low-level model:CTMC, Interactive Markov Chain, Games
Quantitative & qualitative data
Quantitative results
Model
Stochastic analysis/
Model Checking
Model extraction
Dynamics Task 3.5
Attack model
Attack generation
Attacks,Countermeasure generation
ADVERTISEMENT SPACEHere could be your data
ADVERTISEMENT SPACEHere could be your analysis
55
home
McScrooge card pin
pin
city Money BinMB
bank
$$$
$$$computer
C
settop boxSB
MB,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Beagle Boys
C: i
door
trust M: m
get cash
get cashat ATM
goto ATM get card[(pin,X)], (pin,X)& input cash at ATM
get Charlie’s credentialsand perform action
input cashat ATM
in cash at ATM
get Alice’s credentialsand perform action
get credentials
get card
get pin
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
in cardat Alice
SE Alicein Card
goto Home
goto Door &get trust
SE Alice move Door move Door
move Home
perform inat Alice
SE Alicein Pin
get card
perform inat Alice
in cardat Alice
SE Alicein Card
in pinat card
input cashat ATM
in cash at ATM
59
What is missing?
• Analyses over-approximate – All performable actions are performed – Needed to guarantee correctness of the result
• Whenever an actor can loose something... – ...he will do so
60
Adding Behaviour
• “naïve” approach: whenever needed, assume behaviour
• Add likelihoods for actions to be performed – Annotate items with a probability of a certain action
being performed on them– This may depend on time of day, mood, item, location, ... – Compute probabilities for actions to be performed
• Add a behaviour component – Analysis “asks” each analysed process for next action – Based on system state and actor knowledge and state
61
quantitative data
InfrastructureBehaviour
Actors
Risk of detection
Assets
Policies
Cost
Time
Risk appetite
Goal
Social Engineering
Data
Locations
Locations
62
Security Dashboard forSocio-Technical Systems
• Based on identified risks.• Generate surveillance mechanism
based on possible attacks.• Tracks observable movements of
actors and data (logged manually or automatically).
JAN
HallLJanJANFRENT m:FRENT
Out-side
m:CLSRV
m:CLUSR
m:CLUSR
m:CLSRVm:CLUSRCLUSRUSRHALLLJanJAN
PC1 PC2e:PCe:PC1 e:PC2
m:CLSRV
m:FREXIT
CLSRVSRVHALLLJanJAN
home
Margrethe card pin
pin
city ATMA1
bank
$$$
$$$computer
C
settop boxSB
A1,card,pin: i
IPTV pwdip
IPTV remoteREM
LAN
card, pin, ip: e
REM: e(Ptransfer)TECH: e(Pfirmware)
SB: e
Fred
C: i
door
trust M: m
63
Conclusion
• Socio-technical security models provide tools to assess the risk faced by modern organisations.– Spanning the physical, virtual, and social domain.
• The TREsPASS project has developed the attack navigator to identify all possible attacks in socio-technical models.
• Analyses of attacks provide insight in– Risk level of the organisation and contribution of parts of
the organisation to attacks.– But we need a better understanding of actor motivation
and behaviour.