How To Become a Testing Gladiator
Jules Barnes 27 July 2017
ANZTB Melbourne
Commercial in Confidence
Commercial in Confidence
The Arena is your IT system
The Crowd are your Customers
Your opponent is change
The battle is testing
The Game Editor is your Business
Commercial in Confidence
Commercial in Confidence
4 steps to become a testing gladiator
Step 1 Training
Commercial in Confidence
Your Weapons
Gladiators Testing Factors • Access Controls • Audit Trail • Authorization • Continuity of Processing • Correctness • Coupling • Ease of Operation • Ease of Use • Integrity • Maintainable • Methodology • Performance • Portable • Reliability • Service Levels • …
Commercial in Confidence
Types of Gladiators
Gladiator Types Testing Types • Accessibility Testing • Automated Testing • Compatibility Testing • Exploratory Testing • Functional Testing • Penetration Testing • Performance Testing • Production Verification Testing • Regression Testing • Security Testing • Static Testing • System Integration Testing • System Testing • Usability Testing • User Acceptance Testing • …
Commercial in Confidence
Commercial in Confidence
You need to continually train and practice your skills
Commercial in Confidence
Step 2 Know your Audience
Customer
Commercial in Confidence
Why know your Customer?
Commercial in Confidence
Combining your Testing Skills with an understanding of your customer will always result in better quality
Commercial in Confidence
• Company was rapidly growing, 100% year on year
• We were testing from the customers’ perspective, we just didn’t know it!
• "We find 95 per cent of defects before we go into production," Barnes says.^ • Agile development teams, testing was embedded from the start
• Story acceptance criteria was defined by the Testers & BAs
• Test cases were automated in continuous integration environments by developers
• New test environments could be deployed within hours
• Environments were refreshed regularly from production
• 48 hour performance stability tests were conducted for all major releases
• All testing was completed within 1 to 2 weeks of major releases from multiply sprints of work
• Test team could stop a release if it didn’t met testing standards
Wotif.com Test Team Story (2009)
^http://www.theaustralian.com.au/business/technology/no-room-for-website-defects/news-story/ce54a368799215aa328d83290b47e61b
Commercial in Confidence
Step 3 Talk the language of Technology
the Business
Commercial in Confidence
Your testing is too rigorous, we need to get changes out faster and are willing to accept higher risk of defects
going into production
Commercial in Confidence
Not talking the same language
Who here has been in a situation like this?
Commercial in Confidence
Commercial in Confidence
Test Cases
Low risk to the Business
Commercial in Confidence
Commercial in Confidence
Test Cases
Higher risk to the Business
Commercial in Confidence
Testing reduces the likelihood of risk
Commercial in Confidence
Commercial in Confidence
Test Cases
How much risk is the Business willing to accept?
Commercial in Confidence
A customers ability to find and book a room caused by a new search results interface may result in reduced bookings or reduce NPS.
Consequence Catastrophic / Major / Moderate / Minor / Insignificant
Likelihood
Very Likely / Likely / Possible / Unlikely / Rare
What testing would you do to reduce the Likelihood?
Example Risk
Commercial in Confidence
1. Identified a risk
[Event that has an effect on objectives] may result in [consequence/s]
caused by [cause/s]
2. We assessed Consequence based on the “may result in” statement
3. We agreed on a likelihood based on our understanding
4. We then conducted test design to work out what would be required to
reduce the likelihood of that risk from occurring
What did we just do?
Commercial in Confidence
Step 4 Bringing it all together
Commercial in Confidence
How do we combine our testing skills with our understanding of
customer expectations to communicate risk to the
business?
New problem
Commercial in Confidence
Commercial in Confidence
Low risk to the Business?
Commercial in Confidence
Commercial in Confidence
Higher risk to the Business?
Commercial in Confidence
Commercial in Confidence
Create a Customer Journey Map
Commercial in Confidence
Commercial in Confidence
Map the supporting business processes
Map the supporting IT systems
Commercial in Confidence
Example Use Case
Example Risk A customers ability to find and book a room caused by a new search results interface may result in reduced bookings or reduce NPS.
The Change a new search results interface
The Consequence 1. Reduced bookings 2. Reduce NPS
The Impact A customers ability to find and book a room
Commercial in Confidence
Commercial in Confidence
4 steps to become a testing gladiator
Step 1: Continually Train
Step 2: Know your Customers
Step 3: Talk the language of the Business
Step 4: Bring it all together
Repeat
Commercial in Confidence
Call to Action
Create a Customer/Business/Technology map
Commercial in Confidence
Q&A