Adversarial Machine Learning from an Adversarial Risk ... Conference - September 2017.pdf · –...

transcript

Adversarial Machine Learning

from an

Adversarial Risk Analysis

Perspective

David Ríos Insua

AXA-ICMAT Chair and Royal Academy

ICANN, Alghero, September 2017

with D. Banks, J. Rios, F. Ruggeri, R. Soyer, J. Ortega, R. Naveiro, A. Redondo and CYBECO

Outline

• (Almost) All things adversarial

• Adversarial risk analysis

• Adversarial statistical decision theory

• Adversarial point estimation

• Adversarial hypothesis testing

• Adversarial classification

• Discussion and challlenges

Adversarial problems

• Stats/ML: Standard problems – Point Estimation

– Prediction

– Learning

– Hypothesis testing

– Classification

– ..

• Many applications (security, mktg,…) entail adversaries – Spam detection

– Fraud detection

– Network monitoring

– ….

• Intelligent attackers adapting their behaviour to remain undetected and obtain a benefit

• Comparatively few attempts to deal with the problem….

• ...Mostly modelled through noncooperative game theory

Example: Adversarial

classification as a game • C, classifier. A, adversary

• Two classes: + malicious; - innocent.

• C and A maximise expected utility under common knowledge conditions

• Finding Nash equilibria extremely complex

• Dalvi et al (2005) propose a scheme

Utility sensitive Naive Bayes

Forward myopic approach under strong common knowledge

Adversarial problems

• Adversarial classification (Dalvi et al,…)

• Adversarial signal processing (Barni et

al,..)

• Adversarial learning (Lowd and Meek,..)

• Adversarial machine learning (Tygar,..)

• Adversarial SVMs (Zhou et al,…)

• …

• Current adversarial competition in Kaggle

Outline

• Adversarial statisticsl decision theory

• Adversaril hypothesis testing

From RA to ARA…

Motivation

• RA extended to include adversaries ready to increase our risks

• S-11, M-11,.. lead to large security investments globally, some of them criticised

• Many modelling efforts to efficiently allocate such resources

• Parnell et al (2008) NAS review

– Standard reliability/risk approaches not take into account intentionality

– Game theoretic approaches. Common knowledge assumptions…

– Decision analytic approaches. Forecasting the adversary action…

• Merrick, Parnell (2011) review approaches commenting favourably on ARA

ARA • A framework to manage risks from actions of intelligent adversaries (DRI, Rios, Banks,

JASA 2009)

• One-sided prescriptive support – Use a SEU model – Treat the adversary’s decision as uncertainties

– Bayesian games Kadane, Larkey (1982), Raiffa (1982, 2002) made operational

• Method(s) to predict adversary’s actions

– We assume the adversary is a expected utility maximizer • Model his decision problem • Assess his probabilities and utilities • Find his action of maximum expected utility

(But other descriptive models are possible) • Uncertainty in the Attacker’s decision stems from

– our uncertainty about his probabilities and utilities – but this leads to a hierarchy of nested decision problems

(random, noninformative, level-k, heuristic, mirroring argument,…) vs (common knowledge)

• Lippman, McCardle (2012) • Stahl and Wilson (1995) D. Wolpert (2012) • Rothkopf (2007) • MacLay, Rothschild, Guikema (2013,2014)

• Banks, Rios, DRI (2015)

Sequential Def-Att game

– Two intelligent players

• Defender and Attacker. D knows A’s judgements

– Sequential moves

• Def, then Attacker

( | , )Ap S d a

( , )Du d S ( , )Au a S

( | , )Dp S d a

Standard GT Analysis

Solution:

Nasheq. Subgame

perfect equilibrium

Expected utilities at node S

Best Attacker’s decision at node A

Assuming Defender knows Attacker’s analysis

Defender’s best decision at node D

Supporting the Defender

Defender problem Defender’s view of Attacker problem

Defender problem Defender’s solution

Modeling input: ??

Supporting the Defender:

The assessment problem

Defender’s view of

Attacker problem

Elicitation of

A is a EU maximizer

D’s beliefs about

MC simulation

Sequential D-A

Simultaneous DeffAtt game

• Decisions are made without knowing each other’s

decisions

Game Theory Analysis

• Common knowledge

– Each knows expected utility of every pair (d, a) for both of them

– Nash equilibrium: (d*, a*) satisfying

• When some information is not common knowledge

– Private information

• Type of Defender and Attacker

– Common prior over private information

– Model the game as one of incomplete information

Bayes Nash Equilibrium

– Strategy functions

• Defender

• Attacker

– Expected utility of (d,a)

• for Defender, given her type

• Similarly for Attacker, given his type

– Bayes-Nash Equlibrium (d*, a*) satisfying

• Defender’s decision analysis

How to

assess it ??

Assessing • Attacker's decision analysis as seen by the Defender

The assessment problem

• To predict Attacker’s decision

The Defender needs to solve Attacker’s decision problem

She needs to assess

• Her beliefs about are modeled through a probability distribution

• The assessment of requires deeper analysis – D’s analysis of A’s analysis of D’s problem

• It leads to an infinite regress thinking-about-what-the-other-is-thinking-about…

Hierarchy of nested models

Stop when the Defender has no more information about utilities and probabilities

at some level of the recursive analysis. level-k thinking

ARA templates

More general interactions

ARA: Examples/Cases Problem Defender Attacker Specificities Template

ATC protection Airport authority Terrorist Single site D-> A

Piracy Ship owner Pirates Single site D- >A - > D

Metro Operator Pickpock

Fare evasion

Multisite

Multiattack,

Cascade

Urban security Police Mob Multisite spatial D->A->D

Train DoT, DoD Terrorist Multisite network D->A->D

Reliability Manufacturer Customer -- D->A

SME IS.

CYBECO

Company Competitor Cyber, Integrated

with RA

Oil rig

cybercontrolled

Oil company Sponsored

hackers

Cyber, Multiattack D->A->D

CI Owner Terrorist Multistage General

Cybersec res

allocation+cybins

IT Owner Hacker(s) Several decisions

Random and

targeted attacks

D-A, D-A-D

Social robots Robot User Sequential D->A

Other themes

• Different opponent models, beyond SEU

• Concept uncertainty, Mixtures

• Robustness and ARA (GT, ARA, Robust ARA)

• Multiple attackers, Multiple defenders

• Differential games

• Competition and cooperation

• Efficient computational schemes

• Computational environment

• …

Outline

• Adversaril hypothesis testing

Statistical Decision Theory

• Point estimation under quadratic loss

Adversarial Statistical Decision

Theory

Outline

Adversarial point estimation

• Quadratic loss

Concept uncertainty

• A Bayesian adversary

• Mixture, e.g.

• Normal-normal model, for certain

parameter choices

Outline

Numerical example, Spam detection example

Outline

Adversarial classification through

ARA. ACRA

Dalvi et al’s pioneer AC model from ARA perspective

ACRA. Classifier problem

ACRA. Adversary problem

random version

ACRA. Spam detection. Approach

• Preprocessing 1.

For a given

training set,

we estimate

e.g. utility

sensitive

Naïve

Probability of

malicious

and innocent

Probability for

each email to be

malicious or

innocent

• Preprocessing 2.

For each email,

we compute

The probabilities of the

relevant attacks, given the

email and if is malicious or

innocent

• Operation.

Read a (possibly) modified email by

attacker

Compute all relevant

attacks

The Classifier maximize her expected utility to classify the email as spam

or not

Outline

Discussion

• Traditional statistical/ML problems perturbated by presence of

adversaries

• Traditionally treated from a game theoretic perspective (common

knowledge)

• An ARA approach to mitigate common knowledge

• Many challenges

– Multiple attackers vs Multiple defenders

– Efficient computation

– Generic approach: point estimation, interval estimation,…

• Classification: NB, NNs, SVMs,…

– Generative adversarial networks?

– Cybersecurity

Thanks!!!

Collabs welcome

david.rios@icmat.es

SPOR DataLab https://www.icmat.es/spor/

Aisoy Robotics https://www.aisoy.com

It’s a risky life @YouTube

CYBECO https://www.cybeco.eu/

Adversarial Machine Learning from an Adversarial Risk ... Conference - September 2017.pdf · –...

Documents