KTH ROYAL INSTITUTE
OF TECHNOLOGY
IncidentResponseSim: An Agent-Based
Simulation Tool for Risk Management of Online
Fraud
Dan Gorton
Center for Safety Research
Department of Transport Science
Outline
Background
Scenarios
Directions for future research
BackgroundThe incident response process of online banking, and the incident response tree (IRT) tool
Online Banking and Fraud 1 (2)
”Online banking (OLB) is an electronic payment system that
enables customers of a financial institution to conduct financial
transactions on a website operated by the institution” [Wikipedia]
An online bank may have several ”channels” providing different means for login, and a different set of online services depending on the level of security provided by the channel.
The size of online banking fraud is ”channel” specific and
depends on many parameters, including …
• The number of customers
• Countermeasures
• Transaction limit
• Distribution of wealth on customer accounts
Online Banking and Fraud 2 (2)
Prevention(e.g., authentication + IDS)
Detection(e.g., real time or batch fraud
detection)
Response(e.g., automatic or manual)
Front End Security Measures Back End Security Measures
Fig. Overview of electronic payment system [Julisch].
Threats
Impersonation
• Phishing, Man-in-the-middle, Man-in-the-browser, etc.
Deception
• Attacks where the customer performs the transaction on
behalf of the attacker
Server side attacks
• Attacks directed at the online bank servers
Ref: [Julisch]
Countermeasures
…
Updating prevention to include additional authentication methods, e.g., out-of-band authentication or adding control questions to customer support personnel
Updating detection using more aggressive intrusion and fraud detection
Blocking fraudulent transactions before clearing them
Closing down one or more channels
Closing down certain services (e.g., wire transfers) within specific online channels
Restricting functionality within services
Restricting the possibility to add new beneficiary accounts
Blacklisting fraudulent accounts (known money mules)
Grey listing potentially fraudulent accounts, to initiate manual review before allowing transactions to clear
Letting the fraud response team contact the customer for extra verification
…
The Incident Response Process of Online Banking
On a high level
• Event driven
• Risks are evaluated against the current production
environment
• Shortage of time
• Large scale incidents will typically activate crisis
management teams
On a low level
• The fraud response team works with each separate
incident
• Limited time for documentation
There is a need for a “quick” tool, which is “easy to grasp” for higher management
Existing Visual tools for Cyber Security
Attack Trees
• A methodical way or describing threats against, and
countermeasures protecting a system [Schneier]
Protection Trees
• An explicit protection tree that mitigates the attack steps
modeled in the corresponding attack tree [Edge]
Problem: “Fault tree” models fail to capture the chronological
ordering of events [Pat-Cornell]
Solution: Event trees have been used for cyber threats [Ezell]
Problem: Critique; huge problem with under-reporting [GAO]
Solution: Make sure under-reporting is a limited problem [Gorton]
Idea: Fraud is an area where under-reporting may be a minor problem,
because ”the customers want their money back”
Incident Response Tree (IRT)
Prevention Detection Response
Just Register the Frequencies…
Frequencies, C1 to C4
Ref: [Gorton]
Conditional Probabilities of Prevention Pp, Detection PD, and Response PR
The conditional probabilities change during the attack, up or down, depending
on the effectiveness of the countermeasures against the threat at hand
Ref: [Gorton]
Relative frequencies, RFC1 to RFC4
RFC1 = PIE (1 – PP) (1 – PD)
RFC2 = PIE (1 – PP) PD (1 – PR)
RFC3 = PIE (1 – PP) PD PR
RFC4 = PIE PP
RFFraud = RFC1 + RFC2 = PIE (1 – PP) (1 – PDPR)Ref: [Gorton]
Quality assurance
Use statistics for thresholds:
• Threshold for monthly reporting
• Threshold for weekly reporting
• Threshold for daily reporting
• Threshold for minor countermeasures
• Threshold for major countermeasures
Expected loss from fraud (EF)
Credit Risk Approach [BIS]: "𝐸𝐿 = 𝑃𝐷 ∙ 𝐸𝐴𝐷 ∙ 𝐿𝐺𝐷”
• Probability of default (PD)
• Exposure at default (EAD)
• Loss given default (LGD)
Expected Fraud: 𝐸𝐹 = 𝑃𝐹 ∙ 𝑖=1𝑁 (𝐸𝐴𝐹𝑖 ∙ 𝐿𝐺𝐹𝑖)
• Probability of fraud (PF)
– 𝑃𝐹 =# 𝑓𝑟𝑎𝑢𝑑
# 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠
• Exposure at fraud (EAF)
– 𝐸𝐴𝐹𝑖 = min(𝑇𝑟𝑎𝑛𝑠𝑎𝑐𝑡𝑖𝑜𝑛 𝐿𝑖𝑚𝑖𝑡, 𝐴𝑐𝑐𝑜𝑢𝑛𝑡 𝐵𝑎𝑙𝑎𝑛𝑐𝑒)
• Loss given fraud (LGF)
– 𝐿𝐺𝐹𝑖 =𝑆𝑡𝑜𝑙𝑒𝑛 𝐴𝑚𝑜𝑢𝑛𝑡
𝐸𝐴𝐹
Conditional fraud value at risk
Credit Risk Approach [BIS]
• VaR at 99.75th percentile
– Once every 400 years
• Unexpected Losses (UL)
• UL = VaR - EL
Online Fraud Approach
• VaR at 95th percentile
– Once every 20 years
• Simple Random Sampling of
Fraud Losses (FL)
– 𝐹𝐿𝑘 = 𝑖=1𝐼 (𝐸𝐴𝐹𝑖 ∙ 𝐿𝐺𝐹𝑖)
IncidentResponseSim – Simplified Model
IncidentResponseSim – GUI
IncidentResponseSim – Customer Inspector
IncidentResponseSim – Example output
SimulationsScenarios for IRT and the design of new methods for calculating the number of defrauded customers
Current Situation
In the following examples, we will use the following fictional
statistics to describe the current situation. We assume that:
• Probability of initiating event, PIE = 1
• Conditional probability of prevention, PP = 0.8
• Conditional probability of detection, PD = 0.9
• Conditional probability of response, PR = 0.9
Current Situation
We assume:
• 100,000 customers
• A maximum transaction limit of 30,000
• Fraud may not continue over several days
• Account balance drawn from an up-scaled Beta (below)
• Stolen amount drawn from a truncated Normal
IncidentResponseSim – SRS of Defrauded Customers (current situation)
Output from IncidentResponseSim (999 iterations):
Number of Defrauded Customers Bootstrap Mean: 38,10
Number of Defrauded Customers Bootstrap Std: 6,07
Number of Defrauded Customers Bootstrap 95%: 48,00
Number of Defrauded Customers Bootstrap Min: 22
Number of Defrauded Customers Bootstrap Max: 62
IncidentResponseSim – SRS of Direct Economic Consequences (current situation)
Output from IncidentResponseSim (999 iterations):
EF Mean: 941 425,53 SEK
EF Std: 62 547,99 SEK
EF SE Mean: 9 028,02 SEK
EF 95% (Fraud VaR): 1 042 430,61 SEK
EF Min: 765 797,88 SEK
EF Max: 1 110 622,08 SEK
Scenario 1 – Newly entered markets
Assume that we want to keep the number of fraud victims the
same, and that we use the probability of a customer being infected
as a proxy:
A: “reference risk of infection” vs B: e.g. 2.75 times as high risk of infection
Existing Online Bank New Online Bank
Threat Environment A Threat Environment B
PandaLabs
Results from IncidentResponseSim
SRS of Defrauded Customers:
DC Mean: 104,58
DC Std: 10.186893976135476
DC 95% (Fraud VaR): 121.0
DC Min: 76.0
DC Max: 143.0
SRS of Direct Economic Consequences:
EF Mean: 2 379 053,07 SEK
EF Std: 97 137,15 SEK
EF SE Mean: 8 830,65 SEK
EF 95% (Fraud VaR): 2 545 100,11 SEK
EF Min: 2 049 829,33 SEK
EF Max: 2 679 394,95 SEK
Scenario 2 – Single point of failure
Prevention(e.g., Authentication + IDS)
Detection(e.g., fraud detection)
Response(e.g., real time, batch, manual)
History:
RFFraud = 1(1-0.8)(1-0.9*0.9) = 0.038
Failed prevention:
RFFraud = 1(1-0)(1-0.9*0.9) = 0.19
Failed detection:
RFFraud = 1(1-0.8)(1-0*0.9) = 0.20
Failed response:
RFFraud = 1(1-0.8)(1-0.9*0) = 0.20
Scenario 3 – Emerging threats
Assume a new threat, highly contagious, 2 * infection rate, and very effective at overcoming current preventive measures, PP_B = 0.6.
SRS of Defrauded Customers:
Number of Defrauded Customers Bootstrap Mean: 152,05
Number of Defrauded Customers Bootstrap Std: 11,72
Number of Defrauded Customers Bootstrap 95%: 171,00
SRS of Direct Economic Consequences:
EF Mean: 3 352 588,36 SEK
EF Std: 114 012,55 SEK
EF 95% (Fraud VaR): 3 545 783,33 SEK
Existing Online Bank
Threat Environment A Threat Environment B
Max = min (Account Balance, Transaction Limit)
Random = rnd (0, min (Account Balance, Transaction Limit))
Mean Transaction = 500 + rnd (0, 10 000)
Trojan Strategies vs Transaction Limits
Return on Security Investment (ROSI)
MLR = Monetary Loss Reduction
COS = Cost of Solution
𝑅𝑂𝑆𝐼 =𝑀𝐿𝑅 − 𝐶𝑂𝑆
𝐶𝑂𝑆
Action COS # Frauds COST MLR ROSI
Do nothing 0 48 1,042,431 0 N/A
Add +0.1
prevention
400,000 26 581,281 461,150 0.15
Add +0.05
detection
300,000 38 826,431 215,999 -0.28
Add +0.05
response
200,000 38 826,431 215,999 0.08
Directions for future research
• The IRT, being a novel tool, needs to be investigated
further; preferably using real data from other financial
institutions to make sure it is general enough for wide
spread use
• Work in progress:
– More advanced multi-agent-based simulation (using
Mason [Luke])
• Interesting future possibilities are to include, for example:
– the use of prior information using Bayes
– dynamic models like game theory
– social network analysis for estimating the effects of
customer awareness.
References
[Wikipedia] Wikipedia, “Online Banking”, available at https://en.wikipedia.org/wiki/Online_banking (accessed on
August 15, 2015).
[Julisch] Julisch, K., “Risk-Based Payment Fraud Detection”, Research Report, IBM Research Zurich, available
at https://domino.research.ibm.com/library/cyberdig.nsf/papers/E4D71715CD00934A8525779800431D47/$File/
rz3787.pdf (accessed on August 15, 2015).
[Schneier] Schneier, B., “Secret & Lies: Digital Security in a Networked World”, New York, John Wiley & Sons,
pp.318-333, 2000.
[Edge] Edge, K. et al., “The Use of Protection Trees to Analyze Security for an Online Banking System” In the
proceedings of the 40th Hawaii International Conference on Systems Science (HICSS 07), 2007.
[Pat-Cornell] Pat-Cornell, M.E., “Fault trees vs. event trees in reliability analysis”, Journal of Risk Analysis,
Volume 4 No. 3, pp.177-186, 1984.
[Ezell] Ezell, BC. et al., “Probabilistic risk analysis and terrorism risk”, Journal of Risk Analysis, pp. 575-589,
2010.
[GAO] GAO, “Information Security: Computer Attacks at Department of Defense Pose Increasing Risk”, 1996.
[Gorton] Gorton, D., “Using Incident Response Trees as a Tool for Risk Management of Online Financial
Services”, Journal of Risk Analysis, Volume 34, No. 9, pp. 1763-1774, 2014.
[PandaLabs] PandaLabs, “PandaLabs Annual Report 2013 Summary”, available at
http://www.pandasecurity.com/mediacenter/src/uploads/2014/07/Annual-Report-PandaLabs-2013.pdf (accessed
on October 19, 2015).
[Franchot] Frachot, A., Moudoulaud, O., Roncalli, T., “Loss Distribution Approach in Practice”, Group de
Recherche Oprationnelle, Credit Lyonnais, France, 2003.
[BIS] Bank of International Settlements, “An Explanatory Note on the Basel II IRB Risk Weight Functions”,
https://www.bis.org/bcbs/irbriskweight.pdf (accessed on August 15, 2015).
[Luke] Luke, S. et al, “MASON: A Multi-agent Simulation Environment”, Simulation, July, 2005.
