Introduction to Agile TestingTristan McCarthy - Lead ConsultantOpenCredo

@trismccarthy

Agenda

9:00 - 10:30 Learning

10:30 - 10:50 Break

10:50 - 12:30 Learning

12:30 - 13:30 Lunch

13:30 - 15:00 Learning

15:00 - 15:20 Break

15:20 - 17:00 Learning

2

Focus areas

●Requirements

●Defining acceptance tests

●Implementing automation

●Integrating into the development team

3

Why test

4

●Does the system do what we expect?

● Have changes affected previous behaviour?

●How does the system work?

●Is this system ready to be used?

Common understanding

Regression Testing

Test Driven Developme

nt

Acceptance Testing

Test Automatio

nBehaviour

Driven Development

Functional Testing

Continuous

Integration

UI Testing

Agile Testing

5

Requirements

6

Gathering Requirements

7

●Good testing starts with good requirements

●Capture intent and not solution

●Think about non functional requirements

●Make sure that requirements are clear

8

Requirement

Secure expensive equipment

Acceptance Criteria

All equipment must have a kensington lock connected

Test Passes

Requirement

Secure expensive equipment

Acceptance Criteria

All equipment must be physically secured to desks

Test Fails

Example: Simple Requirement

Add a new LinkedIn contact

As a userIn order to keep in touch with a previous colleagueI want to be able to add a new contact

9

Add a new LinkedIn contact

Users who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.

Some assumptions….User account creation, authentication, updates and search functionality already

exists.Connecting with former colleagues is a unique journey with it’s own flow

compared with connecting with friends or strangers.

Practical: Create a requirement

●Choose a product owner

●Pick a well understood web based system as a basis

●Identify a feature (new or existing) to work against

●Describe the intent of the feature with a simple narrative

10

Example: Acceptance Criteria

11

Add a new LinkedIn contact

Users who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.

Acceptance Criteria

● User should be able to ask to connect with another user when viewing a profile

● Connection should be made only after the other user has agreed

● User should see the connection in their contact list if successful

Practical: Add acceptance criteria

●Capture high level statements describing expected outcome

●Add acceptance criteria around restrictions / limitations

12

Defining acceptance tests

13

Identifying scenarios

14

●Start by defining the use case

●Identify additional scenarios by considering different entry points

●Keep test scenarios small

●Don’t try to draw 1-1 relationships between acceptance tests and

criteria

●Focus on posing questions rather than answering them

Example: Scenario list

Scenario: Connection request is approved

Scenario: Connection request is rejected

Scenario: User attempts to request again after rejection

15

Add a new LinkedIn contactUsers who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.

Acceptance Criteria● User should be able to ask to connect with another user when viewing a profile● Connection should be made only after the other user has agreed● User should see the connection in their contact list if successful

Practical: Defining your scenarios

●Identify the “happy path” scenarios at a high level

●Extend the scenarios with negative test cases and “what if’s”

16

Test structure

Preconditions

Describe expected state at test start

Actions

Describe interactions with the system

Assertions

Describe expected outcomes

17

Introducing Behaviour Driven Development

Introduced by Dan North in 2006

Devised as a way of getting the benefits of TDD earlier in the process

Natural language layer on top of testing

Links directly into test automation

Helps when working with non technical stakeholders

Can produce “Living Documentation”

Maps standard test structure to “keywords”Preconditions = “Given”

Actions = “When”

Assertions = “Then”

18

Using Behaviour Driven Development

Preconditions, actions and assertions are all expressed as sentences with a keyword

Each sentence in a test is a “step”

Multiple steps come together to form a “scenario”

Multiple scenarios are grouped into a “feature”

Feature is not a story, multiple stories can go into a feature

19

Feature: Introducing BDD

Scenario: Explain BDDGiven a room full of peopleWhen I explain BDDThen attendees should be able to write their own BDD scenario

Example: Test StepsScenario: Connection request is approved

Given the following new users:|Bob||Jeff|And I am authenticated as user ‘Bob’When I request to connect with ‘Jeff’And ‘Jeff’ accepts my connection

requestThen I should see ‘Jeff’ in my contact

list

20

Scenario: User attempts to request after rejection

Given the following new users:|Bob||Jeff|And I am authenticated as user ‘Bob’And ‘Jeff’ has previously rejected a

connection request from meWhen I request to connect with ‘Jeff’Then I should be informed that I am

not permitted to send another request to ‘Jeff’

Practical: Capturing test steps

Choose a “happy path” scenario to expand

Describe the steps using BDD structure

Describe the preconditions as briefly as possible

Describe the actions a user will perform

Describe the assertions used to validate the functionality 21

Acceptance method

Sometimes useful to define how a story will be verified

Some functional outcomes may not be easily verifiable in the

standard testing approach

Gives clarity when team responsibilities are unclear

Enables a pragmatic approach to quality assurance 22

Test automation

23

Positives Negatives

vs

Faster feedback

Consistent results

Supports CI / CD

Integrates into Agile development

Can be easily reused and extended

Standardised reporting

Complexity

Skills gap

Lead time

Cost

Implementing automation

24

Levels of testing

●Low level unit tests

●Service / integration tests

●User interface tests

○ Note, not necessarily “Graphical User

Interface”

●Speed and value implications at each

layer

25

www.martinfowler.com

Building a test framework

26

●Collection of tools to support a number of needs○ Way of defining test cases

○ Tools for interacting with the system under test

○ Tools for making assertions

○ Mechanism for reporting of results

●Structured for re-use and maintenance

●Should be held to similar standards as development code

●Ideally uses the same development language as the system

under test

Example: An acceptance test framework

27

Growing over time

Start with just the basics

A test framework should evolve in line with the codebase

Capabilities are added to the framework only when required

Don’t be scared to refactor

28

Example: Growth over time

29

Authenticate as user

Authenticate as admin

Add a new contact

Log In User Dashboard

Admin Dashboard

User Search

Profile View

Request Connection

Managing state - why

30

All tests should be consistently repeatable

Tests should be able to run in parallel

It should not be possible for tests to be affected by other test runs

or manual testing

Tests should be focused on just the area relevant to the scenario

Managing state - how

31

Place the system in an expected state

Maintain test data per scenario

Destroy test data

Expose all capabilities in a programmatic way

Integrating into the team

32

Single team

●Testing should always be an aspect of the development team

●Automated tests should be written at the same time the code is

implemented

●Quality control must become part of the team definition of done

●All team members should be encouraged to engage with tests 33

Verifiable requirements

●Testing should be involved at every stage of the development

process

●Easier communication with demonstrable requirements

●Capturing test scenarios helps stakeholders streamline what they

want

●Scope clarity enables better estimation

34

Execution

35

Integrate automation into the continuous integration pipeline

Run against production like test environments

Use tests as deployment gates

Publish reports and make available

Example: CI /CD pipeline

36

Build Code& Run Unit Tests

Run Acceptance Tests

Deploy to QA Environment

Deploy to UAT Environment

Deploy to Production Environment

Deploy to Perf Test Environment

Commit code change

Run Performance Tests

Manual acceptance / look and feel testing

New Requirement

Developer

Tests fail

Change rejected

Tests FailTests pass

Change accepted

Defect management

●Review test reports and raise defects

●Ensure that defects make their way back into the workstream

●Manage tests based on expected outcome

●Use automation to speed up defect resolution

●Use defects to improve automation coverage 37

Further reading

Print ResourcesAgile Testing Lisa Crispin, Janet GregoryTest focused agile development, testing in a development team

The Cucumber BookMatt Wynne, Aslak HellesoyCucumber BDD tool and the ‘Gherkin’ DSL

38

Web Resourceshttps://dannorth.net/Dan North

Origins of BDD, team processhttps://gojko.net/ Gojko Adzik

Requirements gatheringhttp://sauceio.com/ Sauce LabsTesting process, automation tools

Questions

39

40

www.opencredo.com

@trismccarthy

https://opencredo.com/author/tristan/

https://github.com/tristanmccarthy