Date post: | 15-Jan-2015 |
Category: |
Technology |
Upload: | chassa |
View: | 4,757 times |
Download: | 14 times |
FROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION
Christian Hassa - [email protected] - @chr99ha .NET User Group Bern, 5th of December 2012
Specification-By-Example with Gherkin
TechTalk at a glance
• Agile consulting and delivery • Offices in: Vienna, Budapest, Zurich • Established in 1993
Vienna/Austria
3
How do we specify
requirements?
4
The purpose of user stories …
•Describe user needs •Describe system features •Unit of planning • Reminder for a discussion •Mechanism to defer a discussion
5
Impact Mapping
Story Mapping
Specification-By-Example
Establishing a shared understanding
Why?
How?
Code
Acceptance Criterion
Epic
Capability
Impact, Goal
Easier to define upfront Harder to define upfront
Reminder for a discussion
Bug report
Isolated, formalized example
User Activity
User Story
Example
6
Collecting Acceptance Criteria
“I would try to put a book into the shopping cart …”
“I would try to remove a book from the shopping cart…”
“I’d check whether the shopping cart is empty, when I enter the shop …”
“I would try to add the same book again to the shopping cart …”
Books can be placed into shopping cart.
Books can be removed from shopping cart.
Shopping cart should be empty when entering the shop.
Adding same book again to shopping cart should increase quantity.
As a potential customer I want to collect books in a shopping cart So that I can order several books at once.
“Imagine this story is already implemented:
how would you verify it?”
7
Why do we need
examples?
8
UI wire frames, existing UI
rules, key examples
existing artifacts, samples
Examples for user stories
10
Specification-by-Example
Examples … • make abstract descriptions
better understandable
However … • examples are usually not formally
exchanged or documented
Brian Marick
Examples Tests
Requirements
consist of
describe verify fulfillment of
11
Discussion of acceptance criteria
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order newOrder = new Order(newCustomer);
newOrder.AddBook(
Catalog.Find(“ISBN-0955683610”)
);
Assert.Equals(33.75,
newOrder.Subtotal);
}
Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75”
We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order.
Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
12
… illustrated with formalized examples
Given the user has not ordered yet
When the user adds a book with the price of EUR 37.5
into the shopping cart
Then the shopping cart sub-total is EUR 33.75.
Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
13
Discover hidden assumptions
Actually, this not quite right: Books on sale should be excluded.
Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
14
Collaboration: 3 amigos
“Happy Path”
Technical feasability
Exceptions, border cases
Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
15
Abstract acceptance criteria
As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once.
Books can be added to the shopping basket
Books can be removed from the shopping basket
Shopping basket is initially empty
The same book can be added multiple times to the shopping basket
16
Examples for acceptance criteria
As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once.
Books can be added to the shopping basket
Given my shopping basket is empty
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 1 copy of “Harry Potter”
17
As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once.
Books can be added to the shopping basket
Examples for acceptance criteria
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
The same book can be added multiple times to the shopping basket
Given my shopping basket contains 1 copy of “Harry Potter”
18
The same book can be added multiple times to the shopping basket
Structure of examples
Given my shopping basket contains 1 copy of “Harry Potter”
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
Title: Describes intention/abstract acceptance criterion
Arrange: Context, initial state of the system
Act: Execution of the feature
Assert: Assertion of observable behaviour
And I should see the warning: “Book already existed in basket”
Triple-A constraint “Checks”
Chaining up steps
20
How long do we
care about the
examples?
21
Purpose of the examples
• Shared understanding: acceptance criteria
• Documentation: Look-up detail aspects of the system
• Regression-tests: Understand what assumptions have been violated
22
Continuous validation with automation
Given my shopping basket contains 1 copy of “Harry Potter”
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
System
„Step Definitions“ are binding individual steps to an automatable interface of the application.
Automatable interface
UI Automation
Automation does not necessarily have to bind to the UI.
Automatability of system is supported/evolving with development.
23
Demo
http://www.specflow.org
Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) •NuGet Package
24
What does that mean for
testing?
25
UI automation becomes expensive when …
• trying to automate
manual tests
• making tests
unreadable when
automating them
• automating after
completing
development
structure
readability
point in time
26
Structure Manual tests
Asserts Multiple combined features
Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- …
Dependent on other features Long test path with high chance to break
Automated Check
Single aspect of a single feature
ARRANGE – ACT – ASSERT
Independent of other features Short test path with lower chance to break
27
Test automation pyramid
User journeys
Acceptance-criteria
Units
exploratory testing
Source: Mike Cohn
many
few
hard
easy
Au
tom
ata
bili
ty
28
// Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11));
Readability
29
A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146
30
When to test (point in time)
Acceptance criteria (ATDD, BDD)
Unit Tests (TDD)
business view
technical view
Exploratory tests Workflow tests
Performance, Scalability, Usability,Security, …
def
inin
g th
e p
rod
uct
criticizing th
e p
rod
uct
New dimension: defining the product Synergy: Specification of requirements and tests
Agile Testing Quadrants: Brian Marick
31
Cross-functional teams
Co-creation Fast feedback
32
Cross-functional work
Sprint 1 Sprint 2 Sprint 3
short iteration
US4
Plan
Implement & autom. test
US5
Plan
Implement & autom. test
US2
Plan
Implement & autom. test
US3
Plan
Implement & autom. test
US6
Plan
Implement & autom. test
US1
Plan
Implement & autom. test
US7
Plan
Implement & autom. test
US8
Plan
Implement & autom. test
US9
Plan
Implement & autom. test
Explo
ratory Tests
Specification and test
Collaboration for defining acceptance criteria
Collaboration for automation
Preventing bugs instead of finding them!
Extension of “Test Cases”
Limit WIP
Collaboration in manual
testing
33
Manual Testing is always necessary!
User journeys
Acceptance-criteria
Units
exploratory testing
Source: Mike Cohn
many
few
harder
easier
Au
tom
ata
bili
ty
Manual Check after Story Done
Main success pathes
Undiscovered acceptance criteria
No/(few) manual regression checks
Few pathes are enough
More time for exploration
34
What does that mean for
development?
35
Workflow - TDD
36
Extending TDD for business
37
Transparency for stakeholders
In Progress
38
Current sprint report: all sprint scenarios
39
Starting with first scenario (AC)
40
Finishing the first scenario (AC)
41
Progressing scenario after scenario
42
Progressing scenario after scenario
43
Progressing scenario after scenario
44
Implementing user stories in parallel
45
First user story ready for testing
46
Manual testing can start even earlier
47
Already done work can break again
48
Already done work can break again
49
See what is temporarily not working
51
Living documentation
52
Story Mapping
Optimizing and refining scope
Why?
How?
Code
Acceptance Criterion
Epic
Capability
Impact, Goal
Easier to define upfront Harder to define upfront
User Activity
User Story
Example
53
Story Maps
• Original concept by Jeff Patton • Supports iterative product design • Optimized for desired outcome or
deliverable the system should support
56
Building story maps
Get mainstream book fast and conveniently
delivered
Find book I want
Collect books
Complete order
Wait for book
Receive book
time
browse best
sellers
put into basket
enter address
receive delivery
slip
receive delivery notificat.
pay with credit card
search book by
title
create wish list
inquiry order status
desired outcome or deliverable
user activities
system features
ne
cess
ity
57
Walking skeleton
Enabling build – measure - learn
Find book I want
Collect books
Complete order
Wait for book
Receive book
time
browse best
sellers
enter address
receive delivery
slip
pay with credit card
search book by
title
create wish list
inquiry order status
put into basket
receive delivery notificat.
ne
cess
ity
manual workaround
omitted steps
Get mainstream book fast and conveniently
delivered
60
Story Maps in ALM with TFS
TFS work items in story maps
http://speclog.net
61
Story Map in SpecLog
62
Sprint 1
63
Sprint 2
64
Sprint 3
65
Sprint 4
66
Dropped Scope
67
Added Scope
69
User Stories vs. Features
Product/Sprint Backlog
User Story 1 AccCrit 1
AccCrit 2
User Story 2 AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n AccCrit 4
AccCrit m User Story n
AccCrit 5
AccCrit m
AccCrit 3 AccCrit 5
„Done“
• Future options of the system • Organized/refined according to
priority, value, effort, risk, ... • Next possible increments of
the product (units of work)
• Current state of the system • Organized/refined for
functional overview • Versioned and maintained
together with source code
71
Living documentation
• Link business readable automation from source code to story maps • Feed execution results into story maps
72
Additional
thoughts
73
Level of automation
Controller
Business Layer
Data Layer
Model
View
Browser automation
Trigger behaviour through controller
Assert behaviour on model, db, ..
Setup pre-conditions through service interfaces
Out-of-process
In-process
74
Performance
• Grouping tests
• Current WIP
• Completed work • Database
• In-memory
• Templates for setup • Parallel execution • Smart execution order
75
Internal vs. external DSL
Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java
76
Non-functional acceptance criteria
Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms
Given 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms
Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
77
Tools
78
Gherkin automation tools
www.cukes.info
www.behat.org
Ruby, Java, JavaScript, C++
www.specflow.org
.NET, Mono, Silverlight, WP7
PHP
79
80
Books
Gojko Adzic Bridiging the Communication Gap
Gojko Adzic Specification by Example
Elisabeth Hendrickson Explore It!
81 Christian Hassa: [email protected] - @chr99ha