Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven Development
Naresh [email protected]
http://blogs.agilefaqs.com
1
Licensed Under Creative Commons by Naresh Jain
Warmup Scenarios
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
Going out for Movie (THX sound and Digital projection)
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
Going out for Movie (THX sound and Digital projection)
Going out for meal (one veg.)
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
Going out for Movie (THX sound and Digital projection)
Going out for meal (one veg.)
Going shopping ($50)
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
Going out for Movie (THX sound and Digital projection)
Going out for meal (one veg.)
Going shopping ($50)[10 Minutes]
2
Licensed Under Creative Commons by Naresh Jain
Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?
Going out for Movie (THX sound and Digital projection)
Going out for meal (one veg.)
Going shopping ($50)[10 Minutes]
Present back to the group your findings. [3 minutes per group]
2
Story is a smallest piece of functionality that add business value
What is a Story?
Ron Jeffries’ 3 Cs - Card, Conversation and Confirmation
As a .. <user who requires this feature>
I want .. <do something>
So that... <user goal/business justification>
Story Title - Actor Action Context
3
Title: Keen Reader subscribes to a blog
As a keen reader of your blog
I want to subscribe to your blog
So that I can stay up-to-date with the new posts
Story Example
4
Title: Social Networking Enthusiast uploads profile picture
As a Social Networking Enthusiast
I want to upload my profile picture
So my friends can see how I look and recognize me
Another Story Example
5
What makes a good Story?
6
Stories should follow the INVEST principle:
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
Negotiable
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
Negotiable
Valuable
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
Negotiable
Valuable
Estimate-able
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
Negotiable
Valuable
Estimate-able
Small
What makes a good Story?
6
Stories should follow the INVEST principle:
Independent
Negotiable
Valuable
Estimate-able
Small
Testable
What makes a good Story?
6
Stories are fundamental unit of activity
7
Business Goals
Stories are fundamental unit of activity
7
Business Goals
Inception
Stories are fundamental unit of activity
7
As a ____, I want to be able to ____ so
that ____
Product Backlog
Might have an initial estimate (perhaps for both analysis and development), and an expression of technical and
business confidence that this is real and achievable
Business Goals
Inception
Stories are fundamental unit of activity
7
As a ____, I want to be able to ____ so
that ____
Product Backlog
Might have an initial estimate (perhaps for both analysis and development), and an expression of technical and
business confidence that this is real and achievable
Release planning
Business Goals
Inception
Stories are fundamental unit of activity
7
As a ____, I want to be able to ____ so
that ____
Product Backlog
Might have an initial estimate (perhaps for both analysis and development), and an expression of technical and
business confidence that this is real and achievable
As a ____, I want to be able to ____ so
that ____
I will know this is done when _______
Release Backlog
More detailed estimate, and a specific acceptance test – low confidence stories might be
“spiked” or prototyped
Release planning
Business Goals
Inception
Stories are fundamental unit of activity
7
As a ____, I want to be able to ____ so
that ____
Product Backlog
Might have an initial estimate (perhaps for both analysis and development), and an expression of technical and
business confidence that this is real and achievable
As a ____, I want to be able to ____ so
that ____
I will know this is done when _______
Release Backlog
More detailed estimate, and a specific acceptance test – low confidence stories might be
“spiked” or prototyped
Release planning Sprint planning
Business Goals
Inception
Stories are fundamental unit of activity
7
As a ____, I want to be able to ____ so
that ____
Product Backlog
Might have an initial estimate (perhaps for both analysis and development), and an expression of technical and
business confidence that this is real and achievable
As a ____, I want to be able to ____ so
that ____
I will know this is done when _______
Release Backlog
More detailed estimate, and a specific acceptance test – low confidence stories might be
“spiked” or prototyped
As a ____, I want to be able to ____ so
that ____
I will know this is done when _______
To do this I must:1) _____2) _____
Sprint Backlog
Possible automation of the acceptance
test
Development team breaks out the detail of work needed to
pass test
Release planning Sprint planning
Business Goals
Inception
Stories are fundamental unit of activity
7
Is a set of conditions that the Story must meet for it to be accepted as complete
Is typically provided by the customer or product owner.
Is not a replacement for conversation.
Is the results of the conversation
Acceptance Criteria are NOT tests
Acceptance Criteria
8
Acceptance Criteria should contain:
ACTOR
VERB – DESCRIBING A BEHAVIOR
OBSERVABLE RESULT
To accommodate pre-conditions Acceptance Criteria can be expressed as
Given [Precondition]
When [Actor + Action]
Then [Observable Result]
Writing Acceptance Criteria
9
ExampleSocial Networking Enthusiast uploads profile picture
10
Example
Given the user has a valid facebook account and a digital picture on her computer,
When she uploads a picture in facebook,
Then her the picture should be visible to all her friends in her network.
Social Networking Enthusiast uploads profile picture
10
Example
Given the user has a valid facebook account and a digital picture on her computer,
When she uploads a picture in facebook,
Then her the picture should be visible to all her friends in her network.
Given an user is trying to find a friend on facebook,
When the user searches for a person using their name,
Then their profile picture should be displayed along with other details.
Social Networking Enthusiast uploads profile picture
10
Example
Given the user has a valid facebook account and a digital picture on her computer,
When she uploads a picture in facebook,
Then her the picture should be visible to all her friends in her network.
Given an user is trying to find a friend on facebook,
When the user searches for a person using their name,
Then their profile picture should be displayed along with other details.
As owner of facebook,
I want users to upload authentic, personal profile picture,
So facebook's reputation remains intact and facebook stays out of legal hassles.
Social Networking Enthusiast uploads profile picture
10
Licensed Under Creative Commons by Naresh Jain
Acceptance TestsAcceptance Criteria & Tests: Definition
11
Licensed Under Creative Commons by Naresh Jain
Acceptance TestsAcceptance Criteria & Tests: Definition
Acceptance Criteria
11
Licensed Under Creative Commons by Naresh Jain
Acceptance TestsAcceptance Criteria & Tests: Definition
Acceptance Criteria
+
11
Licensed Under Creative Commons by Naresh Jain
Acceptance TestsAcceptance Criteria & Tests: Definition
Acceptance Criteria
Examples (data + scenarios)+
11
Licensed Under Creative Commons by Naresh Jain
Acceptance TestsAcceptance Criteria & Tests: Definition
Acceptance Criteria
Examples (data + scenarios)
Acceptance Tests
+
11
TasksTeam members further break down each story into tasks that need
to be completed to meet the acceptance criteria for the story.
12
Tasks
To accomplish this story:
we start off with a simple upload and image display
Team members further break down each story into tasks that needto be completed to meet the acceptance criteria for the story.
12
Tasks
To accomplish this story:
we start off with a simple upload and image display
restrict user to only upload certain image types (gif, jpg and png)
Team members further break down each story into tasks that needto be completed to meet the acceptance criteria for the story.
12
Tasks
To accomplish this story:
we start off with a simple upload and image display
restrict user to only upload certain image types (gif, jpg and png)
figure out where to store the image. (performant and fault-tolarent)
Team members further break down each story into tasks that needto be completed to meet the acceptance criteria for the story.
12
Tasks
To accomplish this story:
we start off with a simple upload and image display
restrict user to only upload certain image types (gif, jpg and png)
figure out where to store the image. (performant and fault-tolarent)
scale down (size, resolution, etc.) of the image
Team members further break down each story into tasks that needto be completed to meet the acceptance criteria for the story.
12
Tasks
To accomplish this story:
we start off with a simple upload and image display
restrict user to only upload certain image types (gif, jpg and png)
figure out where to store the image. (performant and fault-tolarent)
scale down (size, resolution, etc.) of the image
and so on...
Team members further break down each story into tasks that needto be completed to meet the acceptance criteria for the story.
12
Licensed Under Creative Commons by Naresh Jain
DemoRoman Numerals to Decimal Conversion Example
13
Licensed Under Creative Commons by Naresh Jain
DemoReal World Domain Forwarding Server
14
Licensed Under Creative Commons by Naresh Jain
Thinking in Tables
15
Licensed Under Creative Commons by Naresh Jain
Only Tables ExecuteThinking in Tables
Ignored
Executed
16
Licensed Under Creative Commons by Naresh Jain
Foundational Table Structure
Name of Fixture
Interaction with Application
Table structure depends on type of Fixture
Thinking in Tables
17
Licensed Under Creative Commons by Naresh Jain
3 Foundation Fixtures
Column Fixture
Row Fixture
Action Fixture
Thinking in Tables
18
Licensed Under Creative Commons by Naresh Jain
Column FixtureThinking in Tables
19
Licensed Under Creative Commons by Naresh Jain
Row FixtureAnalogous to comparing against rows in a database table
Thinking in Tables
20
Licensed Under Creative Commons by Naresh Jain
Action Fixture
Think GUI window Counter WindowCounter:
Counter: 6 Count
public class CountFixture extends Fixture { private int counter = 0; public void count() { counter++; } public int counter() { return counter; }}
Thinking in Tables
21
Licensed Under Creative Commons by Naresh Jain
FitLibrary
Extension to FIT
Written by Rick Mugridge
Adds some handy Fixtures
22
Licensed Under Creative Commons by Naresh Jain
FitLibrary Fixtures
ArrayFixture for ordered lists
SetFixture for unordered lists
SetUpFixture
Supports
Graphics
Tree structures
Nested Tables
FitLibrary
23
Licensed Under Creative Commons by Naresh Jain
DoFixture
• Broken tables
• Highly readable
• Flexibility
FitLibrary
24
Licensed Under Creative Commons by Naresh Jain
FIT
Framework for Integrated Tests
Created by Ward Cunningham
Open Source
The most accepted solution for agile acceptance testing
Tools
25
Licensed Under Creative Commons by Naresh Jain
FitNesse
Environment build around FIT
Makes everything easier
Created by Object Mentor, Inc.
Open Source
Tools
26
Licensed Under Creative Commons by Naresh Jain
FIT FitNesse
- Tests written in HTML- Tests are executed on the command line- Tables are executed- Non-table markup is ignored- Tables map to Fixtures- Fixtures are code that is aware of the system- Supplies foundational Fixtures- Implementations ported to many languages
- Stand alone web server- Is a wiki- Tests written in wiki text- Tests are executed from within the wiki- Translates tests into HTML- Uses FIT to execute tests- Supports test suites- Supports variables in tests- Supports test refactoring- Written in Java- Supports FIT implementations in any language
27
Licensed Under Creative Commons by Naresh Jain
Acceptance Criteria and Tests:
A Critical Piece of Agile
28
Licensed Under Creative Commons by Naresh Jain
Analysis
Design
Implementation
DFDERD
DDST
1 May 1 Nov1 Jul 1 Sep
Traditional ApproachAcceptance Criteria & Tests: A Critical Piece of Agile
29
Licensed Under Creative Commons by Naresh Jain
Key Questions
Are we building the right product?
Are we building the product right?
Business Facing
Technology/Implementation Facing
30
Licensed Under Creative Commons by Naresh Jain
Brian Marick’s Test Categorization
Business Facing
Technology/Implementation Facing
Supp
orts
Pro
gram
min
gC
ritique product
31
Licensed Under Creative Commons by Naresh Jain
It Helps to Think of Tests this way...
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique product
32
Licensed Under Creative Commons by Naresh Jain
It Helps to Think of Tests this way...
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique productUnit Testing
32
Licensed Under Creative Commons by Naresh Jain
It Helps to Think of Tests this way...
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique productUnit Testing
Acceptance Testing Low-fi prototypes
32
Licensed Under Creative Commons by Naresh Jain
It Helps to Think of Tests this way...
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique productUnit Testing
Acceptance Testing Low-fi prototypes
Exploratory TestingUI and Usability Testing
32
Licensed Under Creative Commons by Naresh Jain
It Helps to Think of Tests this way...
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique productUnit Testing
Acceptance Testing Low-fi prototypes
Exploratory TestingUI and Usability Testing
Performance TestingSystem Tests
32
Licensed Under Creative Commons by Naresh Jain
Avatars of TDD
Business Facing
Technology/Implementation Facing
Dri
ves
Dev
elop
men
t Critique product
Outside In
Inside Out
33
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the TestPass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Fail
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Fail
Pass
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Refactor
Fail
Pass
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Refactor
Fail
Pass
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Refactor
Fail
Pass
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
TDD Rhythm - Test, Code, Refactor
Add a Test
Run the Test
Make a little change
Run the Test
Refactor
Fail
Pass
Fail
Pass
Test Driven Development
34
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven Development
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven Development
Story
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven Development
Story
Acceptance Criteria
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Story
Acceptance Criteria
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
Story
Acceptance Criteria
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
Story
Acceptance Criteria
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
AutomatedAcceptance
Tests
Story
Acceptance Criteria
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
AutomatedAcceptance
Tests
Story
Acceptance Criteria
ExploratoryTesting
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
AutomatedAcceptance
Tests
Story
Acceptance Criteria
Acceptance Criteria
ExploratoryTesting
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
AutomatedAcceptance
Tests
Story
Acceptance Criteria
Acceptance Criteria
ExploratoryTesting
Automated UI Tests
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Acceptance Test Driven DevelopmentIteration
Automated Acceptance
Tests
AutomatedAcceptance
Tests
Story
Acceptance Criteria
Acceptance Criteria
ExploratoryTesting
Automated UI Tests
TESTS
PERFORMENCE
Automated Unit Test
35
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
GUITests
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
GUITests
Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
AcceptanceTests
GUITests
Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
AcceptanceTests
GUITests
Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein
At least one per storyTools: Fit, FitNesse, RSpec, JBehave
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
Unit Tests
AcceptanceTests
GUITests
Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein
At least one per storyTools: Fit, FitNesse, RSpec, JBehave
36
Licensed Under Creative Commons by Naresh Jain
Mike Cohn’s Testing Pyramid
Unit Tests
AcceptanceTests
GUITests
Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein
At least one per storyTools: Fit, FitNesse, RSpec, JBehave
At least one per class or moduleTools: xUnit, TestNG
36
Licensed Under Creative Commons by Naresh Jain
Every story must have at least one Acceptance Test
A story is not DONE until it passes it’s Acceptance Tests
Criteria for DONEAcceptance Tests: A Critical Piece of Agile
37
Licensed Under Creative Commons by Naresh Jain
Manual Acceptance TestsAcceptance Tests: A Critical Piece of Agile
38
Licensed Under Creative Commons by Naresh Jain
Manual Acceptance Tests
Manual Acceptance Tests
Acceptance Tests: A Critical Piece of Agile
38
Licensed Under Creative Commons by Naresh Jain
Manual Acceptance Tests
Manual Acceptance Tests
Acceptance Tests: A Critical Piece of Agile
38
Licensed Under Creative Commons by Naresh Jain
Why Acceptance Tests?
Criteria for Completion
Great Collaboration tool
Source of Feedback
Real data to measure progress
39
Licensed Under Creative Commons by Naresh Jain
Data From Acceptance Tests
0
18
36
54
72
90
1 2 3 4 5 6 7 8 9 10
Total ATs Failing ATs Passing ATs
40
Licensed Under Creative Commons by Naresh Jain
Acceptance Tests Are Automated
41
Licensed Under Creative Commons by Naresh Jain
The Button
How often would you press it?
When would you press it?
Who would press it?
Testers, Developers, Managers, Customers, Spectators, etc.
Acceptance Tests: A Critical Piece of Agile
42
Licensed Under Creative Commons by Naresh Jain43
Licensed Under Creative Commons by Naresh Jain
Criteria for DONE
43
Licensed Under Creative Commons by Naresh Jain
Criteria for DONE
+
43
Licensed Under Creative Commons by Naresh Jain
Criteria for DONE
Automated+
43
Licensed Under Creative Commons by Naresh Jain
Criteria for DONE
Automated
Executable Specification
+
43
Licensed Under Creative Commons by Naresh Jain
Executable Specification
A new paradigm for testing
Puts quality first
Removes ambiguity from requirements
Acceptance Tests: A Critical Piece of Agile
44
Licensed Under Creative Commons by Naresh Jain
Who Writes Acceptance Tests?
45
Licensed Under Creative Commons by Naresh Jain
The Customer
The Customer Role
Stake holder
Business Analyst
Quality Assurance
Product Owner
Developer
Who Writes Acceptance Tests?
46
Licensed Under Creative Commons by Naresh Jain
Tests Get Technical
The “Customer” may need technical help to write tests
Developers and QAs are technical
Pair test authoring
Who Writes Acceptance Tests?
47
Licensed Under Creative Commons by Naresh Jain
Business Rules Get Fuzzy
Sometimes developers need help understanding tests
Customers know business rules
Pair test implementation
Who Writes Acceptance Tests?
48
Licensed Under Creative Commons by Naresh Jain
Exercise #1
49
Licensed Under Creative Commons by Naresh Jain
The Login Test
Write a test plan, in plain text, for the business rules of logging in.
Web application
User credentials are stored in relational database
Successful login redirects to “Welcome” page
Exercise #1
50
Licensed Under Creative Commons by Naresh Jain
Writing Good Acceptance Tests
51
Licensed Under Creative Commons by Naresh Jain
Login Test PossibilitiesWriting Good Acceptance Tests
1. Direct browser to URL for login page
52
Licensed Under Creative Commons by Naresh Jain
Login Test PossibilitiesWriting Good Acceptance Tests
1. Direct browser to URL for login page
52
Licensed Under Creative Commons by Naresh Jain
Login Test PossibilitiesWriting Good Acceptance Tests
1. Direct browser to URL for login page
1. Enter the username ‘wallace’
52
Licensed Under Creative Commons by Naresh Jain
Login Test PossibilitiesWriting Good Acceptance Tests
1. Direct browser to URL for login page
1. Enter the username ‘wallace’
52
Licensed Under Creative Commons by Naresh Jain
Login Test PossibilitiesWriting Good Acceptance Tests
1. Direct browser to URL for login page
1. Enter the username ‘wallace’
Build a Testable Environment First
52
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Add some users to the system
Writing Good Acceptance Tests
53
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Add some users to the system
Writing Good Acceptance Tests
53
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Add some users to the system
3. Enter a value into the username field
Writing Good Acceptance Tests
53
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Add some users to the system
3. Enter a value into the username field
Writing Good Acceptance Tests
53
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Add some users to the system
3. Enter a value into the username field
Be Specific
Writing Good Acceptance Tests
53
Licensed Under Creative Commons by Naresh Jain
Tests are Examples
Use concrete examples
Specify concrete behavior
No ambiguity allowed
Writing Good Acceptance Tests
54
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Insert into User table values (’wallace’, ‘ilikecheeze’)
Writing Good Acceptance Tests
55
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Insert into User table values (’wallace’, ‘ilikecheeze’)
Writing Good Acceptance Tests
55
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Insert into User table values (’wallace’, ‘ilikecheeze’)
2. Open a browser to the URL http://localhost/myapp
Writing Good Acceptance Tests
55
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Insert into User table values (’wallace’, ‘ilikecheeze’)
2. Open a browser to the URL http://localhost/myapp
Writing Good Acceptance Tests
55
Licensed Under Creative Commons by Naresh Jain
Login Test Possibilities
1. Insert into User table values (’wallace’, ‘ilikecheeze’)
2. Open a browser to the URL http://localhost/myapp
Avoid Implementation Details
Writing Good Acceptance Tests
55
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
56
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
S - SPECIFIC - Explicitly defined and definite
56
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
S - SPECIFIC - Explicitly defined and definite
M - MEASURABLE - Possible to observe and quantify
56
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
S - SPECIFIC - Explicitly defined and definite
M - MEASURABLE - Possible to observe and quantify
A - ACHIEVABLE - Capable of existing or taking place
56
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
S - SPECIFIC - Explicitly defined and definite
M - MEASURABLE - Possible to observe and quantify
A - ACHIEVABLE - Capable of existing or taking place
R - RELEVANT - Having a connection with the story
56
Licensed Under Creative Commons by Naresh Jain
Good Acceptable Criteria and Tests
S - SPECIFIC - Explicitly defined and definite
M - MEASURABLE - Possible to observe and quantify
A - ACHIEVABLE - Capable of existing or taking place
R - RELEVANT - Having a connection with the story
T - TIME-BOUND – When will the outcome be observed
56
Licensed Under Creative Commons by Naresh Jain
Avoid Implementation Details
Business Tier
View
Data Store
Acceptance Tests
DatabaseDatabase
UI
Writing Good Acceptance Tests
Model and Presenter
57
Licensed Under Creative Commons by Naresh Jain
Login Test: Possible Solution
Add user to system: (’wallace’, ‘ilikecheeze’)
Process login with username ‘wallace’ and password ‘blah’
Check login failed
Process login with username ‘wallace’ and password ‘ilikecheeze’
Check login succeeded
Writing Good Acceptance Tests
58
Licensed Under Creative Commons by Naresh Jain
Tools
59
Licensed Under Creative Commons by Naresh Jain
Commercial Tools
WinRunner
Silk
RFT
Tools
Are not suitable for Acceptance Testing in an Agile environment
TestPartner
QTP
Squish
EggPlant
TestComplete
WindowTester
60
Licensed Under Creative Commons by Naresh Jain
Open Source Options
FIT
FitNesse
Selenium
Among the few tools that support Test Driven Development
Sahi
Watir
Abbot
Frankenstein
Cucumber
RSpec/JBehave
Tools
61
Licensed Under Creative Commons by Naresh Jain
Wiki
62
Licensed Under Creative Commons by Naresh Jain
What is it?
A collaborative web site
Editable by any
Created by Ward Cunningham
Every project should have one
http://c2.com/wiki
http://en.wikipedia.com
Wiki
63
Licensed Under Creative Commons by Naresh Jain
Creating Tests
Use Wiki syntax to create a page with test tables Label the page as a Test Page
Use a page name of the form Test… Turn on the Test property
Make sure your Fixtures are in the classpath Use !path widget
Mechanics !path values are concatenated Java command to start FitServer is executed Testable HTML is passed to FitServer FitServer runs the tests Results are passed back to FitNesse
Wiki
64
Licensed Under Creative Commons by Naresh Jain
Creating Suites
Set the Suite property Create a page with the Suite property Created test pages inside this page When the suite is executed, all child test pages will
be included in the suite execution Use the !see widget
!see <name of test page> All “included” tests pages will be included in the
suite execution
There are 2 ways to make Suites
Run a Suite by clicking the Suite button
Wiki
65
Licensed Under Creative Commons by Naresh Jain
Hands-on Session
Conference Proposal Submission Portal
Some sample Stories
Should be able to submit new proposalShould be able to list all submitted proposalSubmitting proposal with same title should display appropriate error messageShould be able to delete submitted proposal based on the title Should be able to delete submitted proposal based on the titleShould be able to search proposals by titleShould be able to search proposals by IDShould be able to find all proposal by an author's name
66
Licensed Under Creative Commons by Naresh Jain
Break
67
Licensed Under Creative Commons by Naresh Jain
Patterns
68
Licensed Under Creative Commons by Naresh Jain
Organizing Tests
Allowing customers to add new tests without breaking the build
Patterns
69
Licensed Under Creative Commons by Naresh Jain
Version Control
Keeping the acceptance test in version control with the code.
Patterns
70
Licensed Under Creative Commons by Naresh Jain
Cross-Functional Pairing
Using FitNesse based acceptance tests for collaboration between cross-functional team members.
Patterns
71
Licensed Under Creative Commons by Naresh Jain
ATDD
Acceptance Test Driven Development
Patterns
72
Licensed Under Creative Commons by Naresh Jain
CSTT
Cleanup, Setup, Test, Teardown
Patterns
73
Licensed Under Creative Commons by Naresh Jain
Independent Tests
Tests shouldn’t depend on each other.
Tests leave the system in the same state it started in.
Patterns
74
Licensed Under Creative Commons by Naresh Jain
Dynamic Stubbing
Avoiding complications of external systems.
Patterns
75
Licensed Under Creative Commons by Naresh Jain
Non-Production Setup/Teardown
Using non-production light weigh code for setup and teardown.
Helps test only what you want to test.
Patterns
76
Licensed Under Creative Commons by Naresh Jain
Suite Levels
Creating different levels of suites depending on the depth/level of feedback desired.
Smoke, Current Iteration/Sprint, Regression
Patterns
77
Licensed Under Creative Commons by Naresh Jain
DRY
Using !include to avoid repeating yourself.
Patterns
78
Licensed Under Creative Commons by Naresh Jain
Make it Real
Write ATs as close as possible to the real environment.
Patterns
79
Licensed Under Creative Commons by Naresh Jain
Fixture Evolution
Allow Fixture implementation to evolve over time.
Treat fixtures as first class citizens.
Patterns
80
Licensed Under Creative Commons by Naresh Jain
At Least One Test/Story
Every story should have at least one acceptance test
Avoid long/multipurpose tests.
Patterns
81
Licensed Under Creative Commons by Naresh Jain
Anti-Patterns
82
Licensed Under Creative Commons by Naresh Jain
Developer ATs
Developers writing acceptance tests by themselves, for themselves.
Anti-Patterns
83
Licensed Under Creative Commons by Naresh Jain
Unit Testing
Don’t write ATs at the unit testing level
Unit tests are implementation specific
ATs are NOT implementation specific
Anti-Patterns
84
Licensed Under Creative Commons by Naresh Jain
QA Testing Tool
Hard to write tests up front.
Perhaps only on large projects.
Anti-Patterns
85
Licensed Under Creative Commons by Naresh Jain
Silver Bullet
Trying to use FitNesse for all types of Acceptance Tests
UI testing
XML testing
Anti-Patterns
86
Licensed Under Creative Commons by Naresh Jain
Test After
Writing tests after the code is already written.
Anti-Patterns
87
Licensed Under Creative Commons by Naresh Jain
Hidden Test Data
Hiding test data in the fixtures.
Anti-Patterns
88
Licensed Under Creative Commons by Naresh Jain
Implementation Dependent ATs
Making test pages (tables) dependent on implementation details and data structures.
Anti-Patterns
89
Licensed Under Creative Commons by Naresh Jain
Logging in Your Fixtures
Putting log statements or print statements in the fixture code.
Fixtures are probably too complicated.
Anti-Patterns
90
Licensed Under Creative Commons by Naresh Jain
Reference
Portions of this presentation is adopted from Micah Martin’s Introduction to Automated Acceptance Tests Presentation
Kent Beck, Test Driven Development By Example.
"Agile Testing Directions" - Brian Marick
http://www.opensourcetesting.org/
91
Licensed Under Creative Commons by Naresh Jain
The End
[email protected]://blogs.agilefaqs.com
Questions?
92