SECURITY TESTING USING MODELS AND TEST PATTERNS … · SECURITY TESTING USING MODELS AND TEST...

Budapest, 26-28 October 2016

SECURITYTESTINGUSINGMODELSANDTESTPATTERNS

Presentedby[BrunoLegeard,ElizabetaFourneret]

© All rights reserved

MODEL-BASEDSECURITYTESTINGPositionning withrespecttothestateoftheart


Model-BasedTesting

3

• Model-BasedTesting(MBT)isbasedorinvolvedonmodels,calledMBTmodels

• Itextendsandsupportsclassictestdesigntechniquesintegratingcloselywiththeexistinglifecycleinanenterprise.

©All rights reserved

MBTProcess

Functionaltests

Manualexecution

& scripts for automation

Test Repository(Excel, HP/ALM…)

Test design and implementation

Functional needsBusiness needsRequirements

Functionaltests

MBTProcess

Functionaltests

Manualexecution


Test Repository(Excel, HP/ALM…)Functional needs

Business needsRequirements

Functionaltests

Modeling for test generation

Automatic or manual testconception

MBTTaxonomyUtting etal.’12

6 © All rights reserved

WhatarethebeneficesofMBTfromindustrypointofview?


0%

20%

40%

60%

80%

100%

Our test design shall become more efficient

(“cheaper tests”).

Our tests shall become more

effective (“better tests”).

Models shall help us to manage the complexity of the

system with respect to testing.

We wish to improve the

communication between

stakeholders.

Models shall help us to start test design earlier.

What do you expect from a model-based approach to testing?

ExpectationsofMBTpractitioners(from2014MBTUserSurvey)

WhataretheMBTpitfallsanddrawbacks?

• MBTdoesnotsolvesallproblems• MBTisnotjustamatteroftooling• MBTmodelsarenotalwayscorrect• MBTgeneratesamyriadoftestcaseshowtodealwiththishighnumber?


Model-BasedSecurityTesting

• WhatarethechallengesinSecurityTesting?

• WhereMBTstandsforSecurityTesting?


SecurityTesting Approaches

Codereview

ManualPenetrationTesting

StaticApplicationSecurityTesting(SAST)

DynamicApplicationSecurityTesting(DAST)

ManualTechniques

AutomatedTechniques

StaticTechniques DynamicTechniques

Intrusiveproxies(Burpsuite,Webscarab,…)

VulnerabilityScanners,Fuzzingtools,…

SecurityTesting Approaches

Codereview

ManualPenetrationTesting

StaticApplicationSecurityTesting(SAST)

DynamicApplicationSecurityTesting(DAST)

ManualTechniques

AutomatedTechniques

StaticTechniques DynamicTechniques

Intrusiveproxies(Burpsuite,Webscarab,…)

VulnerabilityScanners,Fuzzingtools,…

DASTApproaches

• Rely on a variety of techniques to compute black-box test cases• Patterns, • Fuzzing, • Model-checking, etc.

• Promising results for pattern-based techniques• Better detection rates than scanners• Less time consuming than manual penetration testing• Test execution integrated within large-scale testbeds

7/53

Model-based Security Testing (MBST) Approaches

INPRACTICEPattern-drivenandModel-BasedSecurityTesting


Objectivesintheoryandinpractice

14

• Objective1:Improvethecoverageofsecurityrequirements,keepingoveralltraceability

• Objective2 :Increasethefaultdetectioncapabilityofthetestsuite

• Objective3:Cost-effectiveness


Pattern-basedprocesstoreachtheobjectives

Identificationofvulnerabilities&securityrequirements

Definitionofsecuritytestrequirements


Step 1

Step 2

Step 3Formalizationofsecuritytestrequirements

SecurityTestRepository

MBTProcessforsecuritytesting

Functionaltests

Manualexecution


Test Repository(Excel, TTCN-3, ALM…)

Security needs &requirements

Functionaltests


Automatic or manual testconception

RiskAnalysis

SecurityTest

Purposes

SecurityTest

Patterns

MBTProcessforsecuritytesting

Functionaltests

Manualexecution


Test Repository(Excel, TTCN-3, ALM…)

Security needs &requirements

Functionaltests


Automatic testgeneration

RiskAnalysis

SecurityTest

Purposes

SecurityTest

Patterns

MBTTool

BackgroundontheMBTApproach

MBTTool

BackgroundontheMBTApproach

MBTTool

Behavioral modeling notation (UML4MBT) based on UML metamodel:➢ Class diagrams specify the static structure ➢ Object diagrams specify concrete entities and initial state ➢ OCL (Object Constraint Language) to describe its behavioral characteristics

Backgroundonthe MBTApproach

Test selection depending on two test characteristics:➢ Functional behavioral testing → activate all behaviors➢ Security testing → formalization of test scenarios using temporal properties

(TOCL) and test patterns (TP)

Test generation relies on symbolic state exploration of the model:➢ Functional behavioral testing → A test target per behavior to activate➢ Security testing → Test targets derived by unfolding each TOCL and TP

MBTTool

MBTTool

Test cases may be published for test management and execution tools:➢ HP Quality Center, TestLink, etc.

Test cases must be concretized to be made executable:➢ Conformity table between abstract data and concrete data➢ Implementation of class operations

TOCLandTPtestselectioncriteria

• TOCL andTPmakepossibletogenerateteststhatexercisecornercases,relevantwhentestingsecuritypropertiesandvulnerabilities

• TOCL allowstoexpress temporalproperties,forinstanceofsuccessionorprecedence,contributingtotheMBTprocesswith:• Evaluationoftheexistingtestscoverage• Verificationofthemodel’sconformancetothesepropertiesØSimplifyingthemodeldebugging

• TP allowtoexpressintermsofproceduresoftestsbasedonaverboserepresentationandusingtheexpertsexperienceandknowledgeonthesystemvulnerabilities

• TOCL = Temporal OCL• overlay of OCL to express temporal properties• based on Dwyer et al. property patterns [DAC99]• does not require the use of a complex formalism (e.g. LTL,

CTL)

• TOCL Property = Pattern + Scope• Pattern: describes occurrences or orderings of events

(always, never, eventually k times, precedes, follows)• Scope: describes the observation window on which the

pattern is supposed to hold(globally, between, after, before)

DesignofTemporalPropertiesusingTOCL

[DAC99] M. Dwyer, G. Avrunin, and J. Corbett. Patterns in property specifications for finite-state verification. ICSE'99.

TestPatterns• TestPurposeLanguage

• reliesontheuseofkeywords torepresentatestscenarioexpressingacombinationsofteststepsandtestinputparameters

• powerfuland easytoreadbytestengineers• doesnotrequiretheuseofacomplexformalism(e.g.LTL,CTL)

• TestPurpose(TP)=Quantifiers+Blocks• Quantifiers:describesthecontextinwhichanactiondefinedby

theblockwillbeactivated• (for_each behavior $Xfrom {list})

• Blocks:describestheactionstobetakeninordertoactivateastateinthemodel• (use any_operation any_number_of_times to_active $X)

MBTProcess forSecurityTesting

Testrepository

MBT model

SecurityComponent

Execution

Publication

FunctionalRequirements

Coveragemonitoring

TOCLTPStructural test selection Dynamic test selection

Evaluation

Security Properties

SystemUnderTest

5’’5’

1 23

4

MBTTool

EXPERIENCEINSECURITYCOMPONENTSTESTING


Experienceinsecuritycomponentstesting

SecurityComponentshavetwocategoriesoftestrequirements:

- FunctionalRequirements- SecurityRequirements

Experienceinsecuritycomponentstesting

• PKCS#11 isanRSAstandardthatdefinesaninterfacecalledCryptoki topromoteinteroperabilityandsecurityofcryptographictokens.

• Scope:24 functions mostcommonlypresentinthetokens,suchassession, token, key and user management functions,aswellascryptographicfunctionsfor signing messages and verifying signatures.

• ToensuretherepeatabilityoftheMBTprocesswechoseSoftHSM -virtualcryptographicstorelargelyusedforexploringPKCS#11withoutthenecessitytopossesanHSM(createdbythegroupOPENDNSSEC).


PKCS#11:FunctionaldescriptionSpecification documents :

PKCS#11:Functionalrequirements

Class Diagram:à represents the business objects that can be used by the System Under Testà classes own operations that can be called on the system under test (control and observation points)

OCL:à represents the expected behavior of an operation on the System Under Test, regarding the system state and the operation parameters

The test model is a System Under Test abstraction, representing itsexpected behavior.

Instance Diagram:à represents the initial state of the System Under Test

PKCS#11:Testmodel





PKCS#11:Testmodel

PKCS#11:ClassDiagram





PKCS#11:Testmodel

PKCS#11:Testmodel





PKCS#11:Testmodel

PKCS#11:InitialState

ExerciseFunctionalvsSecurityFunctionalRequirements• FunctionalRequirement

• Cryptokisignsdataiftheuserlogged,otherwiseitrespondswithanerrorcodeUSER_NOT_LOGGED_IN

• SecurityFunctionalRequirement• Whenthelogoutsuccessfullyexecutes,anyoftheapplication’shandlestoprivateobjectsbecomeinvalid(evenifauserislaterloggedbackintothetoken,thosehandlesremaininvalid).Inaddition,allprivatesessionobjectsfromsessionsbelongingtotheapplicationaredestroyed.


Howwillyoutesttheserequirements?Howyouwillexpressthem,usingTPorTOCL?

ExerciseSolution(1/3)

• FunctionalRequirement• Thetoolcreatesatestcasebychoosingtheshortestpath



• SecurityFunctionalRequirement


for_each literal $KEY from KEY_ID1 or KEY_ID2 or KEY_ID4 or KEY_ID5,use any_operation any_number_of_times thenuse cryptoki.C_OpenSession(_)

to_activate behavior_with_tags {CKR:OK} thenuse cryptoki.C_Login(_)

to_activate behavior_with_tags {AIM:C_Login/CKU_USER_RW} thenuse cryptoki.nominal_generateKey(_,_,CK_TRUE,_)

to_activate behavior_with_tags {AIM:GENERATE_KEY/OK} thenuse cryptoki.C_Finalize(_) thenuse any_operation any_number_of_times thenuse cryptoki.C_Login(_)

to_activate behavior_with_tags {AIM:C_Login/CKU_USER_RW} thenuse cryptoki.C_SignInit(_,_,$KEY)


• SecurityFunctionalRequirement


PKCS#11innumbersPKCS#11 set up metrics

LOC: Lines of OCL constraints

PKCS#11innumbers

Fig. Distinct fault detection capabilities per coverage requirement

PKCS#11 test generation coverage and execution metrics

Tolearn moreaboutthis casestudy


Chapter 11 – PKCS #11 case study

Published in 2016 – Related to the ISTQB® Model-Based Tester Certification

EXPERIENCEINSECURITYTESTINGFORIOTSYSTEMS


IoTExistingSecurityFrameworks

• OWASPIoT definesaframeworkthatgathersinformationonsecurityissuesassociatedtotheIoTdevelopment,deploymentortechnologyassessment.However,itremainstoohighlevelandlacksaspecificmethodologythatcouldbeusedinasystemicway- forinstanceinsecurityaudits.

• GSMA providesasetofsecurityguidelinedocumentsthattargetallIoTinvolvedentities(serviceproviders,devicemanufacturers,developers,networkoperatorsetc.).However,theGSMASecurityFrameworkonlypointstocurrentlyavailablesolutions,standardsandbestpractices.

• oneM2M identifies4securitydomains(Application,IntraCommonServices,InterCommonServices,UnderlyingNetwork),and3layers(SecurityFunctions,SecurityEnvironmentAbstraction,SecureEnvironment).SelectedasastartingpointforARMOURriskanalysisandmitigationmethodology.


IoTSecurityFramework[H2020ARMOURProject]


SecurityFramework

TheARMOURsecurityframeworktakesasentrytheoneM2Mvulnerabilities,threatsandriskassessmentmethodology,andforeachexperiment:• performedananalysisoftherisks/vulnerabilitiestobeconsideredduringall

phasesoftheexperimentation.• defined asetofcountermeasuresinordertoaddressthevulnerabilitiesandreduce

associatedrisk.• describedthescenariosandmethodologythatwillbethebasisfortheexperiments

execution.VulnerabilityclassificationandpatternsinIoTsystems• Builtonexistingframeworksandadaptthemifnecessary,• Avulnerabilitypatternintendstodescribevulnerabilities,theirconditionsof

occurrenceandimpacts.• CVE(CommonVulnerabilitiesandExposure)framework:dictionaryofpublicly

knowninformationsecurityvulnerabilitiesandexposures


IoTSecurityFramework[H2020ARMOURProject]

MBTMethodologyandFrameworkforLarge-ScaleIoTTesting

49 Paris - 9 September

automated execution

Security MBT models

LocalorFIREtestbeds

Vulnerabilitypatterns

Standards

automatedMBTapproach

in-houseapproach

manualtestconception

Keepingoveralltraceability

Security tests TTCN-3

TPLan

MBTMethodologyin5stepsappliedtooneM2M

50

Vulnerability patterns identification

MBT model based on ARMOUR guidelines

Security test pattern formalization using the Smartesting Test Purpose Language

MBT test generation based on ARMOUR test strategies using CertifyIt

Publication in TPLan – test description and TTCN-3 test scripts&

testbed execution and results

①

②

③

④

⑤

ResultsfromtheoneM2Mexperience

1. Securitytestpatternformalizationfortestgeneration2. DefinitionofgenericMBTmodelsforTTCN-3andTPLanproduction

3. TTCN-3publisher4. TPLanpublisher5. DetectionofinconsistenciesintheIoTplatformundertestwiththespecificationduringtheoneM2MinteroperabilityeventinSouthKorea.


CONCLUSIONLessonslearntfromexperience


LESSONSLEARNT

53

BasedonMBTpitfallsanddrawbacks• MBTisnotjustamatteroftooling

• carefulevaluationoftheorganisationisnecessaryforasuccessfuladoption

• efficient&effectiveteststrategyonlongtermscale

• MBTmodelsarenotalwayscorrect• adequatetoolsarenecessarytomeasurethequalityofthemodels,asitleadstoqualityoftestcases

• MBTgeneratesamyriadoftestcases• todealwithitadequateformalismarenecessarytobenefitfromdomainexpertsexperience


LESSONSLEARNT

54

Benefits fromtheMBTapproachbasedonourexperience• itproducedcheaper tests(averagetimespentofmodelling- 2days)

• itproducesbetter tests(increasedfaultdetection,bettermodelquality)

• MBTmodelsareclear• early testdesign(detectionofinconsistenciesinspecification)


CONCLUSIONANDFUTUREWORK

55

• Model-BasedSecurityTestingpositionattheresearchandindustrystateoftheart

• MBTtoolingextendedtosecuritytesting• InitialMBTFrameworkforSecurityTestingofIoTsystemsatdifferentlevels

• OneM2Mexperienceà A.AhmadpresentationonFriday

• SecurityisnumberonechallengeintheIoTdomainà Lookingforwardfornewproofofconcepts


THANKYOUQUESTIONS?


Source- http://model-based-testing.info

Date post:	05-Nov-2018
Category:	Documents
Upload:	hanhi
View:	225 times
Download:	0 times

SECURITY TESTING USING MODELS AND TEST PATTERNS … · SECURITY TESTING USING MODELS AND TEST...

Documents