Test Design Made Easy (and Fun!)
Rik Marselis 5 November 2015
Let’s walk the testing talk
Question 1
Applying Test design techniques is
easy
Question 2
Applying Test design techniques is
fun
Question 3
How many Test Design Techniques exist?
Test design techniques & coverage typesTMap NEXT• Equivalence classes• Boundary value analysis• Decision points (CC, DC, CDC, MCDC,
MCC)• Orthogonal arrays• Pairwise testing• Data cycle test (CRUD)• Statistical usage (op.prof.• Right paths / Fault paths• Checklist• Decision table• Data Combination Test• Elementary comparison tst• Process Cycle test• Real-life test• Semantic test• Syntactic test• Use case test• Error Guessing• Exploratory testing• Reviews• Walkthroughs• Inspections
ISTQB• Equivalence partitioning• Boundary value analysis• Decision table testing• Cause Effect graphing• State transition testing• Use case testing• User story testing• Structure based (SC, CC, DC, CDC,
MCDC, MCC)• Error Guessing• Exploratory testing• Orthogonal arrays• Pairwise testing• Domain analysis• Defect based techniques• Checklist• Combinatorial / Class. tree• Informal review• Technical review• Walkthrough• Inspection• Static analysis
Torbjorn Ryber• Exploratory testing
(this is an approach, not a technique)• Inspection• Walkthrough• Technical review• Informal review• Modelling• Equivalence partitions• Boundary values• Domain tests• Business process testing• Use case testing• State based testing• Decision tables & trees• Elementary comparison• Combinatorial testing• Data cycle testing• Syntax testing• Time cycle testing• Program Logic (SC, BC, PC)• Data flow testing
Boris Beizer• Control-flow testing
• Loop testing
• Data-flow testing
• Transaction-flow testing
• Domain testing
• Syntax testing
• Finite-state testing
And more…
Question 4
How many Test Design Techniques do you apply in your day-to-day work??
Question 5:
Which test design techniques do you use in yourday-to-day work?
TMap HD test approaches
Why coverage?
What’s your testing challenge?Select from 4 groups of Coverage types
Better know & useone techniqueof each group
thanall techniques
of only one group
Why experience based?
So it needs a skilled tester !
Use results of previous tests to create new tests
Always use a mix
What’s your testing challenge?Select an experience based approach
Largely intuitive & ad hoc
Structured, chartered
Experience brought together
Checklist: Test Heuristics Cheat Sheet
• Abstract (remove details)• Never and Always• Interrupt• Reverse• Some, none, all• Starve• Too few, too many
Source: “Explore It!” by Elisabeth Hendrickson, appendix 2
• Useful approximations• Violate data format rules• Zero, one, many• Zoom• Back, forward, history• Bookmark it
More checklists from Explore it!
Numbers:
• Zero
• One
• Many
• Too many
• Too little
Selecting:
• None
• One
• Some
• Many
• All
Timing:
• Allways
• Sometimes
• Never
CRUD:
• Create
• Read
• Update
• Delete
Source: “Explore It!” by Elisabeth Hendrickson
What is the goal of testing?
• To report about quality and risks
• To build confidence
• So you need to log and report
Example of a test log
How to mix test approaches
• Create an Exploratory Testing charter(this can be done long before the test object is available)
• Use coverage based techniques as “test ideas”(thus your start-set of test cases can be made long before …)
• Execute your test based on prepared test cases complemented with exploratory testing
Charter
Explore <target>
With <resources>
To discover <information>
simple
Extensive:
Source: “Explore It!” by Elisabeth Hendrickson, page 13
Use a tool to get a start
Source: the test design tool “COVER” of Sogeti
Calculate Public Transport tariff
• Based onage, profession and discount card the tariff forpublic transport is calculated.
SPRINT 1: Calculate Transport tariff
• What kind of testing challenge is this?
Assignment:Test the public transport tariff decision
• Think of some test idea’s
• Use the DCT test cases as a starting point
• Execute the test
• Use the logsheetGroups of 2-3
20 minutes
Retrospective
• What is your conclusion about quality & risks?
• How much confidence do you have?
• Have you found defects?
Coffee break
What is the goal?
• With preparing test ideas and some test cases;You have a good start for exploring
– Most people get better coverage than with purely experiencebased test cases
– Proof of coverage at the spots where you need it
– Faster progress of testing compared to only exploring
– It fuels you imagination for the real exploration
Workshop: The Triangle exercise
In his 1979 book “The art of software testing” Glenford J. Myers starts with a simple test to see if you are suited as a tester.
The challenge is to test an application thatgets 3 integers as input which represent the lengths of the sides of a triangle. Then the program tells what type of triangle it is.
So we start thinking…
• What is/are our challenge(s)?
• What possible inputs?
• What possible outputs?
• What’s the process?
• How can we test this?
• How can we be efficient?
Process for triangle application
Input check
start
A = B ?
A = C ?
A = C ? B = C ?
A + B < C ORA + C < B ORB + C < A ?
Input numeric
?
Not a triangle
yes
yes
yesyesyes
yes
no
no
no nono
no
Inspired by Glenford J. Myers
IsoscelesEquilateral Scalene Not a triangle
Exercise 2a: Create a charter
• Include at least 3 test design techniques
Explore <target>With <resources>To discover <information>
Example of an extensive charter
Source: “Explore It!” by Elisabeth Hendrickson, page 13
Example of a simple charter
Groups of 2-35 minutes
Use MCDC for the complex decision
FALSE (0)(scalene triangle)
X Y Z
TRUE (1)(not a triangle)
X Y Z
(X) A + B < C
(Y) A + C < B
1 . .
. 1 .
. . 1
0 . .
. 0 .
. . 0 (Z) B + C < A
0 0
0 0
0 0
0 0
0 0
0 0
D2.1
D2.2
D2.3
D2.4
Result = Condition X OR Condition Y OR Condition Z
Neutral value (AND = 1, OR = 0 )0 0
MCDC =ModifiedConditionDecisionCoverage
Exercise 2b: Execute your charter
Test the triangle application using:
• Your charter
• Your test ideas and tools
• The logsheet
Groups of 2-320 minutes Note:
application and handouts will be provided
Retrospective
• What is your conclusion about quality & risks?
• How much confidence do you have?
• Have you found defects?
• How many of the tests that Glenford Myers has listed, did you do?
Tooling: Of course !But beware…
Source: the test design tool “COVER” of Sogeti
Conclusion
• Exploratory testing can be very well combined with test design techniques
• Exploratory tests can be prepared before test executionstarts
• Combining scripted and exploratory tests can be botheasy and fun!!
The end. Thank you !!
Happy testing
This whole presentation copyright © Sogeti 2015
The next slides contain handoutsand background information.
In this workshop Rik will make you see the beauty of test design by
making it easy and fun. Therefore this workshop is all about actively
testing software and learning while doing so.
The first step in making testing more easy is to determine the type of challenge you are facing and based on that select a proper approach. Early test design alone is not enough to really gain confidence, it always must be complemented with experience based testing. So in this workshop you will do both coverage based testing and exploratory testing and you will experience that proper testing has the following components:
Examine the type of challenge you are facing
Select the proper approach consisting of a mix of coverage based testing and experience based testing
Prepare both coverage based testing (by making test cases) and experience based testing (by making charters)
Execute your tests
Report the results (in a debriefing, supported by some sort of document)
Join Rik in this (inter)active workshop and be inspired to make better use of test design as a means to establish confidence in the system under test and to really enjoy the testing profession.
We’ll be actively testing, so bring your laptop or tablet along.
Logging testcasesNr. Age Soldier Discount
cardExpectedresult
Actualresult
Observations, remarks
1
2
3
4
5
6
7
8
9
10
Use the back of this page if needed
Logging testcasesNr. Side 1 Side 2 Side 3 Expected
resultActualresult
Observations, remarks
1
2
3
4
5
6
7
8
9
10
Use the back of this page if needed
Use MCDC for the complex decision
FALSE (0)(scalene triangle)
X Y Z
TRUE (1)(not a triangle)
X Y Z
(X) A + B < C
(Y) A + C < B
1 . .
. 1 .
. . 1
0 . .
. 0 .
. . 0 (Z) B + C < A
Result = Condition X OR Condition Y OR Condition Z
Neutral value (AND = 1, OR = 0 )0 0