Functional Testing of RESTful Applications - Tools

and TechniquesOctober 2015

Nenad Bozic@NenadBozicNs

SmartCatwww.smartcat.io

@SmartCat_io

Why this presentation now

• Idea to do End-to-End tests is here for a long time (Selenium)

• Doing tests over presentation layer is hard

• Modern architecture separates client and server

• Servers with REST APIs ease up functional testing by skipping presentation

layer

Agenda

• Levels of tests

• Blackbox testing as monitoring tool

• Graybox testing as tool for external dependency control

• Bring functional tests closer to business with Cucumber

• Conclusion – tools and techniques overview

Levels of testing

• Unit testing

• Integration testing

• Component testing• Blackbox

• Whitebox

• Graybox

• System testing

“There are generally four recognized levels of tests: unit testing, integration testing, component interface testing, and system testing. Tests are frequently grouped by where they are added in the software development process, or by the level of specificity of the test.” - Wikipedia

Blackbox testing as monitoring tool

Blackbox testing as monitoring toolOverview of the system and goal of test

Blackbox testing as monitoring toolChallenge 1: Generate data

• Presentation from Gerard Meszaros was truly inspiring for our test data

• Hide test data in generators behind DSL

• Isolate data for easy cleanup

• Isolate data for testing in production

Blackbox testing as monitoring toolChallenge 2: Overcome latency in system

• Microservice architecture is bringing latency problems

• Spring Retry – setup number of attempts with pause in between

Blackbox testing as monitoring toolChallenge 3: Monitoring

• Save latest test result in memory

• Endpoint to fetch results of tests at any time

• Integration with Rollbar to log failed executions

• SMS/email notification strategy

Blackbox testing as monitoring toolResult: Tests as monitoring tool in production

• Scheduled tests with isolated data run each 10 minutes

• Health check of 5 applications in production (Rollbar integration and SMS/email notification)

• Test runs accessible via endpoint

• idea for future to have GUI and history of test runs

• Idea for future to save duration of tests so we can compare runs

• Admins sleep better at night

Graybox testing as tool for external dependency

control

Graybox testing as tool for external dependency controlOverview of the system and goal of test

Graybox testing as tool for external dependency controlChallenge 1: Mock external services

• Mock requests and provide controlled responses

• WireMock – library for stubbing and mocking web services

• Written in Java but provides a JSON API so you can use it with pretty much any language out there

• Application configuration challenges – use Spring Profiles to configure external URLs

• Local and functional environment use WireMock server

• Staging and production use real endpoints

Graybox testing as tool for external dependency controlChallenge 1: Mock external services

Graybox testing as tool for external dependency controlChallenge 2: Deployment flexibility

• WireMock has two ways of working:

• Standalone – configured via the Java API, JSON over HTTP or JSON files

• Library import – start server as part of the application

• WireMock is running and functional testing app sends rules and responses

Graybox testing as tool for external dependency controlResult: Fully tested flow with external dependencies before deployment

• Application is targeting WireMock instead of real external endpoints

• Functional testing application has controlled data which can be used for verification

• This approach can be used for internal dependencies as well

Bring functional tests closer to business with

Cucumber

Bring functional tests closer to business with CucumberWhy we decided to go with Cucumber

• Number of tests grew

• Code base of test application started to be hard to maintain

• Hard to group tests based on features

• Each person in the team had its own vocabulary and business perspective

• Our documentation got outdated faster and faster

Bring functional tests closer to business with CucumberOverview of Cucumber features

• You write scenarios and they are grouped in features

• Features can be tagged (@smoke-tests, @wire-mock)

• With @Before you can prepare environment for tests

• With @After you can do cleanup

• Everyone can write features which are translated to tests

• Dynamic documentation which is near code

Bring functional tests closer to business with CucumberFeature example

• Cucumber uses Gherkin notation (Given / When / Then)

• With Given you prepare application for test

• With When you issue command which will be tested

• With Then you state expected outcome

Given products CSV file for organization XWhen CSV file is uploaded to FTP server.Then the API call for organization X returns newly added products

Conclusion

Conclusion - tools and techniques overview

• Hint 1: try to organize data for testing directly in production

• Hint 2: use tools such as Spring Retry to overcome latency

• Hint 3: use tools such as WireMock to control dependencies

• Hint 4: use tools such as Cucumber to develop business language around

features

• Hint 5: schedule tests and leverage them as powerful monitoring tool

Links

• SmartCat Blog post - Blackbox testing microservices

• SmartCat Blog post - Graybox testing - Control your dependencies

• SmartCat Blog post - Bring functional tests closer to business with Cucumber

• Wikipedia testing levels in software

• Gerard Meszaros - Find the right abstraction for your tests

• Tools: Spring Retry, WireMock, Cucumber

Q&A

Thank you

Nenad Bozic@NenadBozicNs

SmartCatwww.smartcat.io

@SmartCat_io