Parasoft Proprietary and Confidential 1
2016-03-11
End-to-End Testing for IoT integrity
Parasoft
Mark Lambert ([email protected])
Mirosław Zielinski ([email protected])
Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2
Agenda
IoT specifics
Challenges in quality assurance
Effectiveness of end-to-end testing
Doing it better
Demonstration
Parasoft Proprietary and Confidential 3Parasoft Proprietary and Confidential 3
IoT specifics
Multi-layered systems
Disparate technologies
Functionality
spanning across many
layers
Parasoft Proprietary and Confidential 4Parasoft Proprietary and Confidential 4
IoT specifics
Even simple functionality may require different teams to develop
Debugging requires solid understanding of many technolgies
Complex functionality may be hidden behind simple interface
Parasoft Proprietary and Confidential 5Parasoft Proprietary and Confidential 5
Why it is difficult to test IoT
Disparate development technologies hamper introducing consistent testing strategies
“Not enough code to justify testing” syndrom
Limited user interface make the solution to look deceptively simple
Inverted testing pyramid problem
Strong “temptation” to rely entirely on system testing
Parasoft Proprietary and Confidential 6Parasoft Proprietary and Confidential 6
Effectiveness of end-to-end testing
Labor intensive
Difficult to automate
Difficult to start early
Requires inter-teams
synchronization
Difficult to test error handlers
Failed tests do not provide precise information about failure points
Parasoft Proprietary and Confidential 7Parasoft Proprietary and Confidential 7
Effectiveness of end-to-end testing
End-to-End testing for diabetes health system example
Difficult to apply stimulation
Difficult to validate test result
Difficult to test error handlers
Parasoft Proprietary and Confidential 8Parasoft Proprietary and Confidential 8
Why is it difficult to test IoT?
Well known problems …
What makes it special?
So far unseen combination of technologies
Applies to high/low budget projects
Parasoft Proprietary and Confidential 9Parasoft Proprietary and Confidential 9
How to do it better?
Limit manual end-to-end testing
Invest more into automated tests at component
Follow testing pyramid rules
Measure: test results
test effectiveness
Parasoft Proprietary and Confidential 10Parasoft Proprietary and Confidential 10
How to do it better?
Give absolute priority to automated tests
Invest time into designing interfaces (API)
Use API (service) testing tools to cover interfaces
Measure the quality of the test
Parasoft Proprietary and Confidential 11Parasoft Proprietary and Confidential 11
How to do it better?
Use stubs and mocks to improve:
Facilitate test automation
Increase code penetration
Parasoft Proprietary and Confidential 12Parasoft Proprietary and Confidential 12
How to do it better?
Use service virtualization to improve testingautomation
Isolate at the message layer
Simulate functional scenarios and performance conditions
Parasoft Proprietary and Confidential 13Parasoft Proprietary and Confidential 13
How to do it better?
Report test results against requirements
Try to get holistic view of your testing process
Aggregate and review test data from different layers
Assess health of given feature across all layers
Parasoft Proprietary and Confidential 14Parasoft Proprietary and Confidential 14
Test scenario example
Parasoft Proprietary and Confidential 15Parasoft Proprietary and Confidential 15
Test Scenario: REQUEST_NEW_BOLUS_UPDATE_PUMP
Tested System
Tested System
Parasoft Proprietary and Confidential 16Parasoft Proprietary and Confidential 16
Summary: End-to-End Testing for IoT integrity
IoT combines previously isolated technologies Multiple Tiers = Complexity = Risk End-to-End Testing = Chain is only as strong as it’s
weakest link
Reliable quality process allows organizations to assess modules individually and in combination Continuously automate testing Tracability across tiers, technologies and testing
techniques
Parasoft Proprietary and Confidential 17Parasoft Proprietary and Confidential 17
Q & A