So#ware Test Challenges in IoT DevicesSo#ware Test Challenges in IoT Devices Jon D. Hagar,...

So#wareTestChallengesinIoTDevices

JonD.Hagar,Consultant,[email protected]

1

•  Challengesinbothhardwareandso4waredevelopment–  Canweproducequalitywithinscheduleandcostconstraints?–  Mergingofphysical,cyber,andnetworkedworlds

•  AlltheproblemsofITandMobileSo4ware

–  ItonlytakesafewminutesofusinganAppbeforeuserslikeordislikeit

•  Worsethanthat...–  IoTcanKill:Youmaybeonthenightlynews(badpressisnotgood)

–  CompanieswantapieceoftheIoTpie(4-10trillionUSDinnext10years)

Copyright2015JonD.Hagar-“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

2

TheIoTOpportunity

MyTopIoTChallenges(andwhichonesIcoverinred)

•  Complexso4wareandhardware(fortes8ng)–  Sensorsandthe“realworld”–  Howtoconductdevelopment

•  Numbersofdevicesandconfigura8ons(andhowtotest)–  Configura8onsandcompa8bility–  Reliabilityandfaulttolerance

•  Bigdataandanaly8cs•  PrivacyandSecurity•  Connec8vity(systemsandsystemsofsystems)

–  Integra8on

•  Safety•  Lifecycle–unifiedhardware-so4waredev-test-ops(adream)

–  Toolstosupportdevelopment,opsandtests–  Costandschedule–  Concurrentso4wareandhardwaredevelopment

•  IntegratedOpera8ons-giventheabove•  Interna8onalstandardsfordevicesandprotocols

3Copyright2016JonD.Hagar–"So4wareTestAXackstoBreakMobileandEmbeddedDevices"

BasicDefini5ons

•  Test–theactofconduc5ngexperimentsonsomethingtodeterminethequality(s)andprovideinforma5on–  Manymethods,techniques,approaches,levels,context–  Considera8ons:input,environment,output,instrumenta8on

•  Quality(ies)–Valuetosomeone(thattheywillpayfor)–  Func8onal–  Non-func8onal–  It“works”–  Doesnoharm

•  Arethere(cri8cal)bugs?

4Copyright2015JonD.Hagar-“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

TechnologySpace

PhysicalSystems(circa100,000BC)


CyberSystems(1950s)

Embedded

IoT

Mobile-Smart Personal Computers

Big Iron (Cloud)

Many Options Huge Numbers of Devices (billions)

Numbers of Devices (millions)

Cyber-PhysicalSystems(today)

WhereareIoTDevicesinComputerSpace?


Embedded

IoT


Big Iron (Cloud)

Many Options Numbers of Devices (billions)


•  Embedded–So4warecontainedin“specialized”hardware…–  Minimalnetworking-communica8ons

PLUS

•  Mobileandhandheldsmartdevices—small,heldinthehand,highlyconnected(web,cloud,servers,….)

•  IoT–InternetofThingsare“tradi8onal”andnewdeviceswithso4wareandcommunica8onadded

WhatisanIoTDevice


Test Brakes

IoT–2.5MainSegments(tonameafew)


Must be Interoperable Across segments

Industrial Value 2x Consumer

Industrial 4.0 Middle Consumer

Home - Security - Monitoring & control - Infotainment Human - Health - Fitness - Info

Vehicles - driverless - monitoring - Infotainment Office - Security - Energy - Worker info Medical - Health monitor/control - Records Retail - Ordering - Checkout - Advertize

Cities/States/Nation -  Health -  Safety -  Info -  Control and Monitor Transportation - Vehicles - Navigation - Logistics Worksite/Factories - Ops - Control - Info

Example:IoTConnectLandscape


IsIoTnew?

Wehavehadembedded,control,M-2-M,theinternetWhyisIoTsodifferent?

Itbringstogether:• Connec8vity• Bigdata• Resourcelimita8ons–size,baXeries,processing,memory,other• Numbersandtypesofdevices• Mixescloud,PC-IT,mobile,embedded,network,anduser• Securityandprivacy


Challenge1:ComplexSo4wareandHardware

forTes8ngInParallelandSuppor8ngDevelopment


IoTTes8ngOpportuni8es•  Requirementsverifica8onchecking

–  Necessarybutnotsufficient

•  Risk–basedtes8ng–  Historicbuttriedandtrue

•  PaXernoraXack-basedexploratorytests–  PaXern1:Model-basedtes8ng–  PaXern2(andChallenge2):Math-basedtes8ng–  PaXern3:Skill/experience-basedtes8ng–  PaXern4:Standards/process-basedtes8ng


PaXern1:Model-BasedTes-ng

•  Addresssystems,so4ware,andhardwaretest

•  DeveloperandIndependentModeling

•  ImprovedUnderstanding


•  Interestanduseofmodel-basedtes8ngisgrowinginindustrysegments•  Telecom,finance,automo8ve,aerospace•  EuropeanandUSAinterests

•  IoT“highintegrity”areaswillneedit

•  Model-basedtes8ngcansupport:•  Genera8onoftestcasesfrommodelsintotestautomatedexecu8onengines

directlyusingscriptsorthroughtheuseofkeywords•  Earlytes8ngwithimprovedunderstandingofthesystemandrisks•  Useofmodelstosupportsimula8onstodrivetestenvironments•  Verifica8onviacomparesbetweendevelopmentandtestmodels•  Genera8onoftestresultoraclesorjudges•  Supportofindependenttes8ngsuchasIndependentV&V(IV&V)•  Modelanalysisandformalverifica8on

Model-basedTes5nginIoT

Copyright2015JonD.Hagarexcerptedfrom“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

15

An Example Test Flow with Modeling for IoT

Copyright2016JonD.Hagar–"So4wareTestAXackstoBreakMobileandEmbeddedDevices"

•  UTP1.2(soon2.0)standardinplace

•  Toolsupportinplace•  Producetestautoma8on•  Graphicviewsaidunderstanding•  Serveasanoracle

•  Aidsinavoidanceand/oriden8fica8onofissuesearlyinlifecycle–  Beforecodeorhardwarecomplete

•  Considera8onsforgrowthandcon8nuingusage•  N-versionproblem•  Self-checkingproblemifonlyonemodeliscreated•  Skilledmodelersandtestersneeded•  Correctdevelopment/testenvironmentmustbeinplace

Model-basedTestAdvantagesandConsidera8ons


HowtoAddressPaXern2andChallenge2

withoneconceptHow do we handle

many configurations, options, and even test

data sampling?


Pa1ern2:Math-basedTes-ng

Tes8ngisasamplingproblem: Howcantes8nguseMathtohelp?

•  Testsystema8callythenumbersofdevices,configura8ons,

networks,etc.

•  Samplinginenvironmentsandqualitycontrol

•  Selec8onofdatafromtheinputdomainspace

•  BigDataanaly8csfedintotes8ng


•  Andriod

UseMathtoAddressChallenge2:Numbersofdevicesandconfigura8ons

19

•  Routers

•  IoTdevices

•  Data

•  CommChannels

•  IoTHomeProtocol

How many Tests? to address, data, configurations, devices Comms, resources, integration, resources 10 x 2 x 13 x 6 x 6 x 7 = 65,520 tests


UsingtheACTSCombinatorialTool:Example

20

Parameters:

AndriodAppPlasorm [Device1,Device2,Device3,Device4,Device6,Device7,Device8,Device9,Device10]

IoTProtocolHome [true,false]

IoTDevices

[Refrig,Stove,mircrowave,TV,frontdoor,Garagedoor,Homegaurd,Stereo,TempControl,Lights,Drapes,WaterHeater,windowopeners]

Routers [0,1,2,3,4,5]

Commproviders [Cell1,Broadband,cable,Cell2,Spacebased,Vendorgodzilla]

Data [1,0,-1,99999,-99999,100,-200]

TestCase# AndriodAppPlasorm IoTsHome IoTDevices Routers Commproviders Data

0 Device1 false Refrig 1 Broadband 0 1 Device2 true Refrig 2 cable -1 2 Device3 false Refrig 3 Cell2 99999

3 Device4 true Refrig 4 Spacebased -99999

4 Device6 false Refrig 5 Vendorgodzilla 100

5 Device7 true Refrig 0 Cell1 -200

119 Test Sample


Sta8s8calMathToolsGeneralTechniqueConcept ToolExamples

Examplesofwheretechniquecanbeused

Specificsub-techniqueexamples

CombinatorialTes-ng

ACT,HexawiserdExpertPICT

Medical,Automo8ve,Aerospace,Informa8onTech,avionics,controls,Userinterfaces

Pairwise,orthogonalarrays,3-way,andupto6waypairingarenowavailable

DesignofExperiments(DOE)

DOEProXLDOE++JMP

Hardware,systems,andso4waretes8ngwherethereare"unknowns"needingtobeevaluated

TaguchiDOE

RandomTes-ngandFuzztes-ngtools(security)

Randomnumbergeneratorfeatureusedfrommostsystemsorlanguages

Chipmakers,manufacturingqualitycontrolinhardwareselec8on

Tes8ngwithrandomlygeneratednumbersincludes:fuzzinganduseinmodel-basedsimula8ons

Sta-s-calSampling SAS

Mostsciences,engineeringexperiments,hardwaretes8ng,andmanufacturing

Numeroussta8s8calmethodsareincludedwithmoststa8s8caltools

SoMwareBlackboxDomainTes-ng

Mostlyusedinmanualtestdesign,thoughsometoolsarenowcomingavailable

Allenvironmentsandtypesofso4waretests.Theseare“classic”testtechniques,buts8llunderused

EquivalenceClass,BoundaryValueAnalysis,decisiontables


Pa1ern3:Skill/Experience-basedTes-ng


ExploratoryTes8ng-Defini8on

•  Quo8ngJamesBach:“Theplainestdefini8onofexploratorytes8ngistestdesignandtestexecu8onatthesame8me.Thisistheoppositeofscriptedtes8ng(predefinedtestprocedures,whethermanualorautomated).Exploratorytests,unlikescriptedtests,arenotdefinedinadvanceandcarriedoutpreciselyaccordingtoplan.”

hXp://www.sa8sfice.com/ar8cles/what_is_et.shtml

CredittoJeanAnnHarison2013Copyright2015JonD.Hagarexcerptedfrom“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

Pattern attack–based

Exploration:AnImportantSkillforTesters

•  Somepeoplethinkthatalltes8ngisexploratory

•  Scien8ficmethods

•  Usedatdifferent8mes–  Early–  Performance–  OnHardware–  Late

•  BasedinpaXernsofa1ack


PaGern-basedTes5ngWhatisana<ack?

•  ApaXern(fortes8ng)basedonacommonmodeoffailureseenoverandover–  Maybeseenasanega8ve,whenitisreallyaposi5ve–  Goesa4erthe“bugs”thatmaybeintheso4ware–  Mayincludeoruseclassictesttechniquesandtestconcepts

•  SeeLeeCopeland’sbookontestdesign•  Seemanyothergoodtestbooks

•  APaXern(morethanaprocess)whichmustbemodifiedforthecontextathandtodothetes8ng

•  TesterslearnmentalaXackpaXernswhenworkingovertheyearsinaspecificdomain


•  Verifica8onchecking(tests)ofrequirementsiscommon•  Expectednecessarytes8ng,butbyitselfisnotsufficient•  Singularfocusofmanytestteamsthatmisseserrorsandneeded

informa8on

•  IncorporatesriskandaXacktes8ngwithinexploratory,experience-basedtestplanning•  Allowsrapidtestexplora8onwithoutlimita8onsofhighlyscriptedtests•  Requires“skilled”testteams

•  Exploratorytes8ngmustbebalancedwithstrategies–  Verifica8onandValida8onusingstandards–  Math-based–  Model-based

HardwareTestPlanningwithExplora5onConcepts


ExploratoryTes5ngInIoT

•  Rapidfeedback•  Learning•  Upfrontrapidlearning

•  AXacking•  Addressrisk(s)•  Coverdata•  Reliability•  Performance

•  Independentassessment•  Targetadefect•  Prototyping•  Needinfofordevelopers•  Testbeyondtherequirements

•  Cloud•  FaultTolerance

CredittoJeanAnnHarison2013Copyright2015JonD.Hagarexcerptedfrom“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

MoreExamplesSo4wareAXacksforExploratoryTes8ngExcerptedfrom“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

28

SoMwareTestA1ackType A1ackFinds NotesontheA1ack

Developerlevela1acks Codeanddatastructureproblems

Almostaquarteroferrorsinmobileandembeddedcanbefoundbystructuraltes-ng

Controlsystema1acks HardwareandsoMwarecontrolsystemerrors

Manycri-calerrorsinmobileandembeddedarecenteredinthecontrollogic,forexampleanalog-to-digitalanddigital-to-analogcomputa-onproblems

Hardware-soMwarea1acks HardwareandsoMwareinterfaceissues ThesoMwareshouldbetestedtoworkwithanyuniquehardware

Communica-ona1acks Digitalcommunica-onsproblems

SoMwarecommunicateswithhardware,network,andothersoMwarewithcomplexinterfacesthatshouldbetested

Timea1acks Time,performance,sequence,andscenarioerrors

SystemsoMwarecanhavecri-cal-mingandperformancefactorsthattes-ngcanprovidevaluableinforma-onabout

Userinterfacea1acks Problemsbetweenmanandmachine TheusabilityofdevicesandsoMwarearecri-caltosuccess

Smart/Mobile/Hardwarea1acks Issuesspecifictosmartdeviceconfigura-onsincludingcloudissues

Cloud-hybridcompu-ngcomprisesamajorityofthenewsoMwaresystemsbeingdeployed

Securitytesthackinga1acks SoMwareerrorsthatcanexposedevicestosecuritythreats

Securityofdevicesorsystemsisincreasinginimportanceanda1acksinclude,forexample,GPSandiden-tyspoofing

Genericfunc-onalverifica-ona1acks

Requirementsandinteroperabilityerrors BasicchecksthattestersshouldconductonsystemsandsoMware

Sta-ccodeanalysisa1acks Hardtofinderrorsthatclassictes-ngoMenmisses

CanoMenbedonebythedevelopmentgroupbutsome-mesthetestgroupmustrunthisanalysis


Pa1ern4:Standards-basedTes-ngForProcesses,NotProducts


•  IEEE1012isastandardthatdefinesV&Vprocesses•  Specificac8vi8esandrelatedtasks•  AddressesV&Vatsystem,hardwareandso4warelevels•  Canbeappliedtoafullsystem,sub-system,orelement

•  Featuresinthestandardinclude:

•  Integritylevels•  MinimalV&Vtasksforeachintegritylevel•  Intensityandrigorconsidera8onappliedtoV&Vtasks•  DetailedcriteriaforV&Vtasks

IEEE1012-2012Verifica5onandValida5on(V&V)PlanningStandard

30 Copyright2016JonD.Hagar–"So4wareTestAXackstoBreakMobileandEmbeddedDevices"

ISO29119SoVwareTestStandard

31 Copyright2016JonD.Hagar–"So4wareTestAXackstoBreakMobileandEmbeddedDevices"

•  AddressedbasicsofModelingearlier

•  UTPisa“language”

•  ISOisconsideringamodel-basedprocessstandardusingUTP2.0

OMGUTP


Challenge3:IoTDataandAnaly8cs


TheEvolu8onofComputers=TheEvolu8onofDataUsage


Embedded

IoT


Big Iron (Cloud)

Many Options Huge Numbers of Devices (billions)


Data Used by The Few

Data Pulled (from whole web)

App that Customizes a Data Slice

Limited Data (if any)

We do not Know yet ? ? ?

IoTtoGenerateHugeAmountsofData(Petabyte,Exabyte,ZeXabyte,OraYoXabyte)

Currentanaly8csfocusisonmarke8ng/sales

Ifuserisatestergenera8ngdata…….

Testerswillneedtousedataanaly8cs

Butforwhat?


•  FromWikipedia:Taxonomyistheprac5ceandscienceofclassifica5on.Thewordfindsits

rootsintheGreekτάξις,taxis(meaning'order','arrangement')andνόμος,nomos('law'or'science').Taxonomyusestaxonomicunits,knownastaxa(singulartaxon).Inaddi5on,thewordisalsousedasacountnoun:ataxonomy,ortaxonomicscheme,isapar5cularclassifica5on("thetaxonomyof..."),arrangedinahierarchicalstructure.

–  Fielddatahelpingto“understandandknow”errorstoimproveIoTdevelopmentandtes8ng

36

AnExampleofUsingDataAnaly5csforTes5ng

SoVwareErrors: ABadSitua5ontoAvoid


37

ErrorDataTaxonomy(Earlyversionofresearch)SuperCategory

Aero-Space Medsys IoT/Mobile GeneralTime 3 2 3 Interrupted-Satura8on(over8me)

5.5 TimeBoundary–failureresul8ngfromincompa8blesystem8meformatsorvalues

0.5 1 Time-RaceCondi8ons

3 1 Time-Longrunusages 4 1 20Interrupt-8mingorpriorityinversions

0.7 3 Date(s)wrong/causeproblem

0.5 1 Clocks 4 2 Computa8on-Flow 6 23 19Computa8on-ondata 4 1 3 1


IoTDataAnaly5cs–OneFuture

SODA–SelfOrganizingDataAnaly8cs

• ThetoolsanddataareorganizedtosupportallaspectsofIoTwithAr8ficialIntelligenceandcustomizedselec8onbasedonnatureofuser

• Users=customer,middlemen,governments,developers,managers,etc.

• Researchtopic38Copyright2016JonD.Hagar–"So4wareTestAXackstoBreakMobileandEmbeddedDevices"

Challenge4:

IoTSecurityandPrivacy

(Manyexpertsthinkthesearetoppriority)


•  YourIoTAppgetsonthenightlynews

•  Yourteamseessecurityassomeoneelse’sproblem

•  YoulosepersonaldataoryourAppmakespersonaldataavailabletoanyone

ExampleofWhatWorriesMeatNight:SecurityandPrivacy


•  Mobile/IoT–IoTsystemsarehighlyintegratedhardware–so4ware–systemsolu8onswhich:–  Mustbehighlytrustworthysincetheyhandlesensi8vedata–  O4enperformcri8caltasks

•  Securityholesandproblemsabound–  CoverityScan2010OpenSourceIntegrityReport-Android

•  Sta8canalysistestaXackfound0.47defectsper1,000SLOC•  359defectsintotal,88ofwhichwereconsidered“highrisk”inthesecuritydomain

•  Carsandmedicaldeviceshacked

TheCurrentSecuritySitua5on


•  Fraud–Iden8ty•  Worms,virus,etc.

–  Faultinjec8on•  Processingontherun•  Hacksimpact

–  Power–  Memory–  CPUusage

SecurityErrors(refinementoftheso4wareerrordatataxonomy)

•  Eavesdropping–“yeseveryonecanhearyou”•  Hijacking•  Click-jacking•  Voice/Screen

•  PhysicalHacks•  Filesnooping•  Lostphone


Are you giving away someone else’s keys?

•  AXack28Penetra8onAXackTest •  AXack28.1Penetra8onSub–AXacks:Authen8ca8on—Password •  AXack28.2Sub–AXackFuzzTest•  AXack29:Informa8onThe4—StealingDeviceData •  AXack29.1SubAXack–Iden8tySocialEngineering •  AXack30:SpoofingAXacks •  AXack30.1Loca8onand/orUserProfileSpoofSub–AXack•  AXack30.2GPSSpoofSub–AXack

SecurityAXacks(from“So4wareTestAXackstoBreakMobileandEmbeddedDevices”)


Privacy–RestrictedData•  Differentfromsecurity

» Moreofanissueinsomecountries•  ExamplesImightnotwantexposed


•  CompaniesmustleveragethedatacomingfromIoT–  Sani8zedata

•  Bigdataanaly8cs

–  ImproveTestandDev-Ops

–  HowtomaintainPrivacy?

•  Likelywillneedoptin/outwith“mo8va8onfeature”

ALLTHISWILLNEEDTESTING

PrivacyImpactonData

45


Challenge5:Connectivity

WhatseparatesIoTfromEmbedded


Connec5vityOpportuni5es

•  Tes8ng

–  Testthedevice,thenetwork,thecloud,theapp,and????

–  Issuesinconnec8vity

–  Securityandprivacy(again)

–  Protocols–noclearwinnersyet

– Manyop8ons


SODA

Tes5ngOp5onsforConnec5vityTestEarly•  Model-basedtes8ng•  Math-basedtes8ng

TestO4en•  Testlabs

TestConsistently•  Risk-basedtes8ng•  Requirementsverifica8onchecking•  Automa8on


Connectivity

•  TotheUserSo4wareHardwareHuman

•  Tothesystem

•  Tothesystemofsystem

•  Tothedata


Connectivity Concerns

•  Timelag

•  Datacorrectness

•  Differentconfigura8ons

•  Datacompleteness

•  Privacyandsecurity(yetagain)


Connectivity Factors

•  Testenvironment

•  InterfaceProtocolop8ons

•  Deviceop8ons


IoTTes5ngSummary•  Todefeatanenemy,youmustknowthebug

•  TheIoTtestdataislimited,– Whatexistshasimplica8ons

•  TherearechallengesandpaXernsofopportunaty

•  So4warewillbeinverynearlyeverything–  Tes8ngmaybealimi8ngfactor

52Copyright2015JonD.Hagarexcerptedfrom“So4wareTestAXackstoBreakMobileandEmbeddedDevices”

References(myfavoritebooks)•  “SoVwareTestAGackstoBreakMobileandEmbeddedDevices”

–JonHagar–  IoTTestsBookin2016

•  “How to Break Software” James Whittaker, 2003

–  And his other “How To Break…” books •  “A Practitioner’s Guide to Software Test Design” Copeland, 2004 •  “A Practitioner’s Handbook for Real-Time Analysis” Klein et. al.,

1993 •  “Computer Related Risks”, Neumann, 1995 •  “Safeware: System Safety and Computers” Leveson, 1995 •  Honorable mentions:

–  “Systems Testing with an Attitude” Petschenik 2005 –  “Software System Testing and Quality Assurance” Beizer,

1987 –  “Testing Computer Software” Kaner et. al., 1988 –  “Systematic Software Testing” Craig & Jaskiel, 2001 –  “Managing the Testing Process” Black, 2002


MoreResources

•  www.stickyminds.com – Collection of test info •  www.embedded.com – info on attacks •  www.sqaforums.com - Mobile Devices, Mobile Apps - Embedded

Systems Testing forum •  Association of Software Testing

–  BBST Classes http://www.testingeducation.org/BBST/

•  Your favorite search engine

•  My web sites and blogs (see front page)


ReferencesforSta8s8calMathTools•  IEEE1012,StandardforSystemandSo4wareVerifica8onandValida8on-hXp://standards.ieee.org/findstds/standard/1012-2012.html,IEEEpress,2012•  ISO29119,So4wareTestStandard-hXp://www.so4waretes8ngstandard.org/•  Hagar,J.SoVwareTestAGackstoBreakMobileandEmbeddedDevices,CRCpress,2013•  Kuhn,Kacker,Lei,Introduc5ontoCombinatorialTes5ng,CRCpress,2013(includesthetoolACTS)•  Tool:Hexawise-app.hexawise.com/•  Tool:rdExpert–www.phadkeassociates.com/•  Tool:PICT–msdn.microso4.com/en-us/library/cc150619.aspx•  Reagan,Kiemele,Tool:DOEProXL-DesignforSixSigma,AirAcademyAssociates,selfpublish,2000•  DOE++-www.reliaso4.com/•  SAS-www.sas.com/•  Kaner,Hoffman,Padmanabhan,TheDomainTes5ngWorkbook,selfpublish,2013•  Bailey,DesignofCompara5veExperiments.CambridgeUniversityPress,2008•  Kacker,Kuhn,Hagar,Wissink,"IntroducingCombinatorialTes8ngtoaLargeSystem-So4wareOrganiza8on,”scheduled-2014,IEEESoVware•  WhiXaker,James2003,HowtoBreakSoVware,PearsonAddisonWesley•  WhiXaker,JamesandThompson,Herbert,HowtoBreakSoVwareSecurity,PearsonAddisonWesley,2004•  Andrews,WhiXaker,HowtoBreakWebSoVware,PearsonAddisonWesley,2006•  Levy,ToolsofCri5calThinking:MetathoughtsforPsychology,1996•  Bach,Bolton,“Tes8ngvs.Checking,”www.developsense.com/blog/2009/08/tes8ng-vs-checking/•  Hagar,“Whydidn’ttes8ngfindtheembeddedGMTruckfiresystemerror?”-www.breakingembeddedso4ware.wordpress.com/•  OMGUTP1.2,www.omg.org/spec/UTP/1.2/•  Baker,Dai,Grabowski,Schieferdecker,Williams,“Model-DrivenTes8ng:UsingtheUMLTes8ngProfile,”2008•  Green,Hagar,“Tes8ngCri8calSo4ware:Prac8calExperiences,”IFACConference1995•  Boden,Hagar,“HowtoBuilda20-YearSuccessfulIndependentVerifica8onandValida8on(IV&V)ProgramfortheNextMillennium,”QualityWeekConference1999

•  Port,Nakao,Katahira,Motes,ChallengesofCOTSIV&V,Springerpress,2005


Date post:	04-Jun-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

So#ware Test Challenges in IoT DevicesSo#ware Test Challenges in IoT Devices Jon D. Hagar,...

Documents