THECURSEOFDIMENSIONALITYANDIMAGERECOGNITION

BRANDONEDWARDS

OUTLINE

• Imageclassification• Worst-casetestimages(adversarialexamples)• Defenseagainstadversarialattacks

• ’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz,

Maithra Rughu,MartinWattenberg,IanGoodfellow(2018)ICLRPaper

• Relevancetoadversarialexamplesinimageclassification

IMAGECLASSIFICATION

IMAGECLASSIFICATION

ImageSource:“ImageNetClassificationwithDeepConvolutionalNeuralNetworks”,AlexKrizhevsky,IlyaSutskever,GeoffreyE.Hinton,2012

• ImageNet(ILSVRC):1000classes;training-1.2million,validation-50k,test-150k• ~83%successforgroundtruthinthetop5classes• Currenttop5performance>95%

MODELFUNCTION

ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=#classes

ADVERSARIALEXAMPLES‘CLOSE’IMAGESTHATCLASSIFY‘INCORRECTLY’

DIGITALATTACK

ImageSource:‘ExplainingandHarnessingAdversarialExamples’,ICLR2015,Goodfellow,Shlens,Szegedy

• AttackaboveisonGoogLeNet (ImageNet)(>94%top5accuracy).• Theperturbationisclearlysmallbyhumanstandards.

• Digitaladversarialattackscenario:Phishingdetection

PHYSICALATTACKImageSource:“AccessorizetoaCrime:RealandStealthyAttacksonState-of-the-ArtFaceRecognition”,MahmoodSharif,Sruti Bhagavatula,Lujo Bauer,MichaelK.Reiter;CCS2016

• Attackagainstpre-trainedfacialrecognitionmodel• 88%ofimageswithglassesclassifiedasMilla Jovovich• Meanconfidencewas78%.

• Thisperturbationislarger,butwoulditberaisesuspicion?

DEFENSEAPPROACHESLearnthe(distributional)differencebetweenadversarialexamplesand‘natural’data.

• Preprocessing(removingperturbation)– JPEG,neuralnetworkde-noiser

• Detectionofadversarialexamples

Imposeconstraintsonmodelfunctiontolimitlocalchanges

• RegularizationorLipschitzconstraints

Consideradversarialexamplesduringtraining

• Adversarialtraining

Improvemodelinotherways

• Capsulenetworks?

ADVERSARIALSPHERESPAPER

ADVERSARIALSPHERESPAPER

’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz, MaithraRughu,MartinWattenberg,IanGoodfellow (2018)ICLR

• Simpleclassificationtask

• Experimentalmodelresults

• Theoreticalresultsrelatingmodelaccuracyandproximityofadversarialexamples.

CLASSIFICATIONTASKANDEXPERIMENTALRESULTS

• Twospherescenteredattheorigininℝ)(R=1andR=1.3).• Trainanartificialneuralnetwork.• ModelInput:Apointinℝ)• ModelOutput:”probability”ofbeingclosertotheinnersphere

• ExperimentalFocusond=500• Trainonpointsuniformlysampledfrombothspheres• Testonpointsuniformlysampledfromtheinnersphere• Hightestaccuracy,butcloseadversarialexamplesremain

THEORETICALRESULT

• Non-zeroerrorimpliesarbitrarilycloseadversarialexamplesforlargeenoughdimensiond.

• Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).

Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).

SketchofProof

LetEbethesetofmisclassifiedpoints,so𝜇 𝐸 = 𝑞 > 0 [𝜇 𝑆8 = 1].

Let𝑑 𝐸 = 𝔼;~=>𝑑(𝑥, 𝐸) (averagedistancetoE).

Maximum𝑑 𝐸 occursfora”cap”(intersectionof𝑆8 withahalf-space)[Figiel et.al.1977].

Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).

SketchofProof(Continued)

(larged)anycoordinateon𝑆8 hasdistribution:𝑁(0,1/𝑑).– [Poincaré\Lévy]

Asmallbandaroundan“equator”containsthemajorityofspherevolume.Thecapboundaryisthusclosetotheequator– wherethemajorityofpointslie.

ADDITIONALTHOUGHTS

Adversarialproblemisworse

Theasymptoticresultsignoretheadversarialexamplesthatmaybefoundofftheinnersphere.

ModelErrorasafunctionofNumberofTrainingPoints

Learnmodelsthatusestheradiusofpoints- perfectmodelsforALLdimensions.I.E.Domainspecificfeaturesmayprovidethelowerrordesired.

LESSONSFORIMAGECLASSIFICATION?

• Insightintocurrentdefenseideas• Caution:Adversarialexamplescouldlieonthedatadistribution.

• Confirmation:Lipschitzconstraintsandadversarialtrainingwouldhelphere

• Imagesmaybedifferent:shapeofindividualclassdistributions

• Couldbebetterinsomeways,worseinothers

• Ex:𝐵C×𝐼)FC forsmallk.Betterforin-distributionexamples,butmoresurfaceareacouldallowmoreoff-surface?

SUMMARY

• Thespherestoyproblemprovidesinsightrelatedtocurrentadversarialimagedefensetechniques.

• LargedimensionsCANleadtoverystricterrorrequirementsinordertoavoidclose‘adversarialexamples’.

• Domainspecificlowdimensionalfeaturecreationorotherconstraintscouldprovidethelowerrorneededto‘pushoff’adversarialexamplesfortheaveragetestpoint.

THANKYOU

NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS

Proof:

• LetEbethesetofmiss-classifiedpointsontheinnersphere,witherrorrate𝜇(𝐸) =q.ForcalculatinganupperboundwereplaceEwitha“cap”𝐸′ with𝜇 𝐸′ = 𝜇(𝐸).Withoutlossofgenerality,weassume:

E′ = {𝑥 ∈ 𝑆8: 𝑥K > 𝛽/ 𝑑� } forsome𝛽 > 0.

• Thenq = 𝜇 𝐸′ ≅ ℙ 𝑁 0, K)> Q

)�= ℙ 𝑁 0,1 > 𝛽 = 1 − Φ(𝛽) ,where

Φ isthestandardnormalcdf.

NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS

Proof(Continued):

• Thus𝛽 = ΦFK(1 − 𝑞).

• Notethat𝑑 𝐸′ : = 𝔼;~=>𝑑 𝑥, 𝐸′ ≤ 𝔼 max 2� Q)�− 𝑁 0, K

), 0

= 𝑂(Z[\(KF])

)�).

• Finally,forfixedqwehave𝑑 𝐸 ≤ 𝑑 𝐸^ = 𝑂( K)�).

TWOEXPERIMENTALMODELSPiecewiseLinearModel

• TwolinearlayernetworkwithReLU activations.Mini-batchstochasticgradientdescentwasusedwithbatchsize50.Batchnormalizationwasperformedatthetwohiddenlayers.

QuadraticModel(Ellipsoidal)

• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobe

expressedas∑ 𝛼c)cdK 𝑥c`.

• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.

• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.

• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.

STRUCTUREDFEATUREDETECTION

• INPUT:Rawpixelvalues(2-Darray)structure.

• INTERMEDIATEFEATUREVARIABLES:

• LocalFeatures:Edges,Textures,…,Ears,Eyes,...

• GlobalFeatures:Face,Body,…

• Keyoperations:Convolutions, Down-Sampling,Up-Sampling,…

• Built-ininvariance:Shift,Scale,…

• FINALLAYERS:Usefeaturevaluestocomputeclassconfidencevalues.

ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=# classes

TWOTRAININGMODES

• [Online](makesurematchestheirstatements)Uniformlysamplefromtheinnerandoutsphereforeachnewtrainingpoint.

• [Batch]UniformlysamplefromtheinnerandoutsphereforNpointseach.Iterateoverthese2Npointsrepeatedlyduringtraining.

FORLARGEDIMENSIONS,CLOSEADVERSARIALEXAMPLESAREFOUND.

• Piecewiselinearmodel(Onlinetraining– 25millionpointspersphere)withd=500.Noerrorwasobservedin10milliontestpoints.

• Note:Volumeofthismisclassifiedspaceontheinnersphereissmall!!!

• Note:d=60wastheobservedpointwheretheexperimentabovestartedtohaveadversarialexamples.

ALINEARINCREASEINADVERSARIALDISTANCEREQUIRESANEXPONENTIALDECREASEINERRORRATE

Experimentalmodelerror

rateestimates,𝑞 v.s.𝑑(𝐸).

Theupperbound𝑑(𝐸^) is

thesolidblackplot.

ImageSource

`AdversarialSpheres’ICLR2018

MOREINFOONQUADRATICNETWORK

• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.– Thedecisionboundarywillbeanellipsoid.

• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)

cdK 𝑥c`.

• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.

• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.

• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.

QUADRATICMODELOBSERVATIONS

• A rotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)

cdK 𝑥c`.

• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.

Online:50milliontrainingpoints(samesetupfromReLU experiment)

All𝛼c wereinrangeforaperfectclassifier– ie perfectclassifier

QUADRATICMODELOBSERVATIONSBatch:batchsize1million

Noerrorsin20milliontestpoints

Adversarialexamplesarefound

394/500𝛼c areoutofrange.

Withhighprobability,the

effectsofthebad𝛼c cancel

eachotherout.

ImageSource:`AdversarialSpheres’ICLR2018