Date post: | 05-Dec-2014 |
Category: |
Technology |
Upload: | agileminds |
View: | 9,174 times |
Download: | 1 times |
1
Belgium, 2009
With Material from Lisa Crispin
Janet GregoryDragonFire Inc.
Testing Techniquesfor
Agile Testers
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
2
Topics
Test Planning - LevelsTesting and CollaborationThe Agile Testing QuadrantsAutomationPlanning Your Test StrategyReferences
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
3
Levels of Planning – Product, Release, IterationClic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
4
Release Planning ... Test Planning
Consider:Scope / FeaturesPrioritiesRisks
Budget time:For building test infrastructureFor finding test tools
What do you need to document?Consider its usefulness
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
5
Iteration Planning ... Test Planning
Consider:Doneness for each featurePriorities – which story to detail firstRisks
Budget time:For defect fixing & retesting
What do you need to document?At minimum, the acceptance test(s)
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
6
The Importance of CollaborationThe Star Experiment
Gerald Weinberg and Donald Gause,Exploring Requirements
What is the right answer?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
7
How Many Points?Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
8
Collaboration
Collaboration means …working together
Communication meanssharing ideas, information, decisions & solutions
Tools for collaborationFeedbackInteractionVisibility - allows people to talk about it.
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
9
Collaboration Discussion
1. Split into teams, and discuss2. Present your findings
TWO QUESTIONS….1. How does your teams communicate?2. What are the collaboration tools you use?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
10
What are the Agile Testing Quadrants?
Method for classifying testsLooks at the purpose of the tests
not the whenTests may cross boundariesTests that are created to support the team,
often turn into regression tests
…. Let’s look at the picture!
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
11
The Agile Testing QuadrantsIntroduced by Brian Marick
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
12
The Quadrants …..Can be used as a communication tool
To the project team
To management
To explain testing in a common language
Emphasize whole-team responsibility
Focus on collaboration
Whole team participation
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
13
Use to define ‘Doneness’
Defining “doneness” for release readinessNo story is done until testedCustomer needs captured as passingtestsAutomated regression testsDelivers value“Doneness” in all quadrants
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
14
Quadrant 1Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Benefits of TDD
Focus on internal code qualityBuilds testability into codeIncreases confidence in designProvides instant feedback to developers
Technology-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Other Benefits of Q1 Testing
Start of building quality into the productProvides refactoring supportForms the foundation of automation suite
Technology-facing Tests that Support the Team
Programmerunit tests are
the base
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
17
Q1 Toolkit
Source code managementIntegrated development environment
compile, debug, build GUI, refactorBuild tools
eg. CruiseControl, HudsonUnit test tools
xUnitMocking tools
All needed to enable collaboration
Technology-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
18
Discussion
What can testers do to help thedevelopers?
Technology-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Helping developers
Copyright 2009 Janet Gregory, DragonFire
19
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
20
Quadrant 2Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
21
Quadrant 2 Tests
Business-facing tests that support the team
Acceptance testsExamples
Used to elicit requirementsUser experience prototypes
wire frames, mock-upsPair testing – “Show me”
Business-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Drive Development
Example (or Acceptance Test) Driven DevUse examples to demonstrate differencesExamples can become testsExpress as executable testsCode until the tests pass
Identify hidden assumptions
Business-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Other Benefits for Q2 Testing
Help customers achieve advance clarity
Executable tests added to regression suite
Focus is external quality
Helps define when we're done a story
Customer – developer – tester collaboration
Business-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Given, When, Then - BehaviouralGiven ... Preconditions
When ... Triggers, actions
Then ... Consequences, results
Simple ex. (from Gojko Adzic, Bridging the Communication Gap):
Given the customer has a balance of $1.00 and deposits$5.00 to his account
When an authorization code A-123 is received
Then the transaction is completed, and the newbalance is $6.00
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
25
Q2 Collaboration ToolkitChecklistsPersonasMind maps
brainstormingwords, ideas, tasks
Mock-ups / paper prototypesUser-centered design
Flow diagramsWhiteboards (physical and virtual)Thin slice/steel thread
Business-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
26
Toolkit – Turning Examples into TestsFit/FitNesse
Collaboration in software developmentTakes place of regular UITake the place of UIUse to set up dataTest different inputs faster
xUnit frameworksWorks well for developers
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
27
More Tools to Turn Examples into Tests
Behaviour-driven development toolsAnother approach to TDDFocus on examples, “should” – common language
easyB, jBehave
GUI test toolsTest UI behaviour, system test, legacy systemsSome examples:
SeleniumWatir/WatijWebTest
Business-facing Tests that Support the Team
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
28
Fit Example – Failing TestClic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
29
Fit Example – Passing TestClic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Exercise StoryAs an online shopper, I want to be able to see thecost of different shipping options when I checkout, so I can choose which one I want.
Assumptions:- User has already entered shipping address.- API for cost calculator available, takes postal code and weight.- The options are Ground, 2 day and Overnight.- PO Boxes are ground only.- Items > 20 lbs are Ground only.- User will be able to choose different options for different items.
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
31
Exercise – Let’s Question
1. What kind of shipping options are we talking about?2. What costs do we have to consider?3. Can we break this story into smaller chunks?4. Are there multiple viewpoints? Who?5. What would the first slice be?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
32
Exercise - Let’s draw ….
The first “slice” is: …..
1. In your group, select a customer.2. Draw the process flow(s)3. Identify the steel thread or thin slice you can
start developing
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
33
Exercise – Let’s ATDD
1. Identify a high level acceptance test (s)for the first slice.
2. For the 2nd? 3rd
What did you learn?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Story Test Planning
Think of variations
Capture assumptions, thoughts....
Story test template
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
35
Exercise – Let’s AutomateTake your acceptance testsCreate Fit type tests or Given, When, Then
Example:
Business-facing Tests that Support the Team
Option Address PostalCode
Weight Cost
Ground Seattle 123456 5 lbs 3.52
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
36
Quadrant 3Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Quadrant 3 TestsBusiness-facing tests that critique the product
Exploratory Testing
Test for UsabilityCollaborative TestingDemosUser Acceptance Testing
Business-facing Tests that Critique the Product
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Exploratory Testing
Learn & designSession-based (time-box)Provides feedback into new storiesTest for usability
understand end users - personaswho will be using the system
Follow “smells”, your instinctsTouring (James Bach, James Whitaker)
Business-facing Tests that Critique the Product
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
39
Other Types of Testing
Scenario testingProcess flows
Realistic data
Soap opera testing (Hans Buwalda)
Usability testingPersonas
Navigation
Observing users
Don't forget documents, reports, help text
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Collaborative TestingProvide feedback ….
Turn learnings into tests that drive new featuresChange process as needed
Iteration reviewsBuilds confidenceQuick feedback loop
Informal demosPair exploratory testing with customerEven on unfinished code
Business-facing Tests that Critique the Product
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Benefits for Q3 Testing
Feedback to Quadrants One and TwoEvaluation of the actual productRecreate actual user experiencesRealistic use and data
Remember your contextWhat works for your situation“It depends”; a tool, not a rule
Business-facing Tests that Critique the Product
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
42
Tools to Critique the GUI
Record/playbackTraditional vendor toolsSome open source tools have recorders
Open source libraries for web browsersWatir is Ruby library for testing thru browser
Specifying testsCanoo WebTests specified in XML
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
43
Tools for Exploratory Testing
Test scenario setupeg. Watir/Watij scripts
Generate test dataeg. PerlClip, Ruby script, Fit
SimulatorsSimulate data, feed to app over time
MonitorsWatch log files
EmulatorsDuplicate system behaviour
eg. mobile devices
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Exploratory Test ExerciseStory: As an online shopper, I want to be able to deleteitems from my shopping cart.
Think of some personas or characters and deviseexploratory scenarios which that role might get into,for example:
Senior citizen who never shopped on the webHacker looking to cause troubleWorking mom in a rushInternet-savvy teen
44
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
45
Quadrant 4Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Quadrant 4 TestsBusiness-facing tests that critique the product
Non-functional tests“ility” testingPerformance, scalability, stress, loadMemory managementSecurity testing
Roles & permissions , system ‘hacking’Data migrationInfrastructure TestingRecovery
Technology-facing Tests that Critique the Product
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Benefits of Q4 Testing“Non-functional” requirements may be higherpriority than “functional”
PerformanceStabilitySecurity...other quality criteria
Helps your application deliver the ‘right’ valueMakes the ‘finished’ product
Technology-facing Tests that Critique the ProductCopyright 2009 Janet Gregory, DragonFire
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Any questions about the quadrants?Clic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
49
Why Automate?
Free up time for most important workRepeatableSafety netQuick feedbackHelp drive codingManual tests are error proneTests provide documentation
You know what your system does
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
50
Test Pyramid – OLD modelMost testing was done by the QA team
manually orPlay / record automationThrough the GUI
Based on the V-modelUnit tests were sporadic
Developers knew little about testing
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Automated Test Pyramid - Agile (Mike Cohn)
Unit tests are the base layerProvide the fastest feedbackBest ROI
Middle layerbecomes functional regressiontests
GUI layerMay be partially automatedMostly exploratory testing
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Test CoverageHow are you going to know when you are done?What is enough?Collaborate to decide what coverage you wantThink risk – include the customerFunctional coverageLines of code? … be carefulWhat tools can you use?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2008 Janet Gregory, DragonFire
53
Test Plan
Benefit is in the planningProject specificFocus on what is really neededRefer to a strategy or testing approach docfor static informationShould it be static or dynamic?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2008 Janet Gregory, DragonFire
54
Project Test Planning ToolsQuality Management Strategy
Outlines the consistent test approachAgile Testing QuadrantsAutomated Test PyramidTest Plan Alternatives
Test Plan (simplified)Test WorkbookTest Matrix
Test Coverage Checklist
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Light-weight Test Plan
Project specifichighlights project critical information
Risks, high level scope, assumptions,constraintsBenefit is in the planningFocus on what is really neededRefer to QM Strategy doc for static infoShould it be static or dynamic?
Sample test plan
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
56
Test MatrixMechanism to think about functionality andpossible test conditionsHigh level view for release planningValue is in the planningValue can also be in the visible progress report
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
57
Test Plan Matrix – one exampleClic
k to buy N
OW!PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2008 Janet Gregory, DragonFire
58
Testing Planning ToolsHeuristics / Mnemonics
Elisabeth HendicksonJonathan Kohl / James Bach (SFDPOT)
Decision tableLogical relationships
Mind mappingUse cases – flow mapping
Cause & effectFocus on userHappy pathOther paths
State transition diagrams
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
59
Planning Your Test Strategy
Consider scope, priorities, risksTools that solve the problemInvolve customersCollaborate with programmersDocument only what is usefulConsider all four quadrantsTest matrix - big picture
allows whole team to understand
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
60
Group Discussion
What quadrants does your team use ‘back home’Which quadrant could your team “back home”improve?What will you do to address this, when you goback?How will you approach your test planning
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
61
Session Retrospective
Use lessons learned to improveReview - are we doing all we could?What would you like to know more of?What was the most useful?
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
62
“The” Book
Agile Testing: A Practical Guide forTesters and Agile Teams
By Janet Gregory and Lisa Crispin
Available on• Amazon.com• Amazon.ca
www.agiletester.ca
www.janetgregory.ca
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com
Copyright 2009 Janet Gregory, DragonFire
63
ResourcesCollaboration Explained: Jean TabakaTesting Extreme Programming, By Lisa Crispinand Tip HouseFearless Change: Patterns for introducing newideas, Linda Rising and Mary Lynn [email protected] (Brian Marick’s site)www.testobsessed.comwww.satisfice.com
Click t
o buy NOW!
PDF-XChange
www.docu-track.com Clic
k to buy N
OW!PDF-XChange
www.docu-track.com