@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting
Beyond the urban legend
–
Testing in an agile contextJochim Van Dorpe
@ Smals_ICT#DV14 # AgileContextTesting
In-house ICT shared services for e-government
• Focus on social security and e-health services
• For federal, regional, local & European institutions
• Based in Brussels
• Software development, ICT-operations & staffing
• >1700 people
Introduction: What is Smals?
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: Who Am I?
(QA) ( (lead) (technical) test(er)
analyst/coördinator/manager) (automator/engineer)
I am a tester
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: What do I have to tell?
@ Smals_ICT#DV14 # AgileContextTesting
What is testing?
It's not about proving that the software works, it's about finding bugs
@ Smals_ICT#DV14 # AgileContextTesting
Why is testing necessary?
To prevent:
• Death
• (Serious) injury
• Loss of business
• Loss of reputation
@ Smals_ICT#DV14 # AgileContextTesting
Why do we test?
@ Smals_ICT#DV14 # AgileContextTesting
The test project in waterfall
@ Smals_ICT#DV14 # AgileContextTesting
Agility
Agility is the continuous
delivery of prosperity (value) to
stakeholders of a system in a
sustainable and balanced
manner -Schalk Cronjé (@ysb33r)-
@ Smals_ICT#DV14 # AgileContextTesting
Agile fundamentalists are like …
Did you even read
our manifesto?
@ Smals_ICT#DV14 # AgileContextTesting
Agile manifestoOf course …
• Individuals & interactions over
processes and tools
• Working software over
comprehensive documentation
• Customer collaboration over
contract negotiation
• Responding to change over
following a plan
@ Smals_ICT#DV14 # AgileContextTesting
Urban legends
• Agile doesn’t need testers
• Agile = de facto good code, built-in quality
• Testing is dead
• Agile = TDD, ATDD, BDD
• Testers should be able to code
• Agile hasn’t time for testing
• 100% automation
• Faster, shorter, better!
• Developers and Testers are like oil and water
• You only need to unit test
• User Acceptance Testing is no longer necessary
@ Smals_ICT#DV14 # AgileContextTesting
So…
• The Agile manifesto shouldn’t stand between you &
good quality
• No documentation, few documentation or poor
documentation is no excuse for “not testing”
• Don’t put an Agile stamp on your team, just to avoid
the things you dislike, but find a way of doing them
so you like them …
@ Smals_ICT#DV14 # AgileContextTesting
No difference at all?
@ Smals_ICT#DV14 # AgileContextTesting
Many ways to be agile …
@ Smals_ICT#DV14 # AgileContextTesting
… but I’m no salesman
@ Smals_ICT#DV14 # AgileContextTesting
Principles of testing in an agile
context
-Elisabeth Hendrickson (@testobsessed)-
@ Smals_ICT#DV14 # AgileContextTesting
1. Testing moves the project forward
Shifting focus:
No longer the last line of defense
@ Smals_ICT#DV14 # AgileContextTesting
2. Testing is not a phase
Integrated & continuous:
develop & test together
@ Smals_ICT#DV14 # AgileContextTesting
3. QA = responsability of the team
@ Smals_ICT#DV14 # AgileContextTesting
4. Reduce feedback latency
No long gaps between development
& testing feedback
@ Smals_ICT#DV14 # AgileContextTesting
5. Don’t let bugs crawl around
Deal with bugs as soon as possible
@ Smals_ICT#DV14 # AgileContextTesting
6. Test should represent requirements
Don’t ASSUME, it makes an ASS of
U & ME
SCHOOLBUS
@ Smals_ICT#DV14 # AgileContextTesting
7. Reduce overhead
Think high level: don’t waste time on
too much detail
@ Smals_ICT#DV14 # AgileContextTesting
8. Don’t believe in fairy tales
If something
seems to good to be true, it probably is
@ Smals_ICT#DV14 # AgileContextTesting
9. Done means done
Done means implemented and tested
@ Smals_ICT#DV14 # AgileContextTesting
Concrete practices
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: This will drown your testers:
Manual tests
Acceptance tests
System &
integration tests
Comp.
tests
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: Make sure your testers can
cope with the pace
Component tests
System &
Integration tests
Acc.
Tests
Manual
testsManual tests
Acceptance tests
System &
integration tests
Comp.
tests
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate Testing, not only tests
Test
execution
ch
ec
kin
g
Load
Data-
sets
Reset
DB
Continuous integration
Logging
resultsGenerate
reports
Live
dashboards
Calculating
‘metrics’
Use the tools that fit your purpose
AND NOT THE OTHER WAY AROUND!
@ Smals_ICT#DV14 # AgileContextTesting
B. Explore
Leave some room for
exploratory testing
@ Smals_ICT#DV14 # AgileContextTesting
C. Diversify
• …in the test levels
• Unit-tests only won’t find
everything
• System test only, and you’ll
drown
• … in the way of testing
• Automation is good, automation
is fun, but in the end, an
automated test isn’t better than a
human based test (manual test)
@ Smals_ICT#DV14 # AgileContextTesting
D. Test the tests
• GREEN == GOOD ?
• DO: Review tests
• DONT: No test coverage fairy tales
• DONT: Mock lasagna
• DONT: I should adapt the test cases so it’s green again
@ Smals_ICT#DV14 # AgileContextTesting
E. Crowdsource …
• Crowdsource in your team to find
the good & passionate tester
• Accept that there are learning curves
• Testing is a craftmanship
• A passionate tester will tend to be more
effective than the chinese-volunteer-
tester
• Encourage collaborative testing
@ Smals_ICT#DV14 # AgileContextTesting
… or ask a testjumper
• Ask a ‘test jumper’ to join the
team
• May spend some hours as a consultant
• Or months as an ordinary tester
• Coaches or contributes himself
• Encourages testability
• Help the developers think
productively about testing
@ Smals_ICT#DV14 # AgileContextTesting
The test project in agile
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Metrics such as:
• # of bugs found
• Bugs per functionality
• Bugs per 1000 lines of code
• DDP (defect detection percentage)
• We can however show our QA-efforts
• QA-survey
• Test case progress
• Fixed test cases
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Qualification (good / bad) of the devs
• 100% green charts
• Upfront fixed number of test cases
• Automated self-generating auto-code-correcting
tests
@ Smals_ICT#DV14 # AgileContextTesting
conclusion
There are no agile testers,
but we need testers who
adapted their mindset to an
agile context
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 1/2• Books:
• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and
Agile Teams
• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the
Whole Team
• Blogs:
• Huibschoots.nl
• LisaCrispin.com
• http://testobsessed.com : Agile testing overview
• Http://www.satisfice.com : Test jumpers: one vision of Agile testing
• pascaldufour.wordpress.com
• agile-and-testing.chriss-baumann.de
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 2/2• Presentations:
• 40 agile methods in 40 minutes by Craig Smith
• Agile testing quadrants explained by @RubyTester
• Certification:
• CAT-training: Certified Agile Tester
• ISTQB foundation level add-on: Agile tester
• Conferences:
• Agile testing day(s)
• More references:
• Word clouds generated on wordle.net
• Images found with google image search
@ Smals_ICT#DV14 # AgileContextTesting
Questions