Testable requirements

transcript

Requirements Delivery – Using Examples

About Us

Bharti Rupani@bhartir24brupani@enterprise-knowledge.com https://www.linkedin.com/in/bhartir

Wyn Van Devanter @wynvwyn.vandevanter@excella.comhttps://www.linkedin.com/in/wyntuitionhttp://www.slideshare.net/wynvandevanter

Terminology• Specification by Example• Executable Specifications• Automated Acceptance Tests (AAT)• Behavior Driven Development (BDD)• Acceptance Test Driven Development (ATDD)

• Testable Requirements

“Specifications by examples are acceptance tests”

- Gajko Adzic

Testable Requirements • User Journeys • Deliver business value• Clear examples • Executable

Why should you care? • Clearly communicate the specifications• No functional gaps • Verify business rules • Build the right thing

Building Quality

Plan Collaborate Deliver

Build the right thing

Build the thing right

Developing Examples

Collaboration

Prioritized Backlog

Business Goals User Journey

User Story 2

Acceptance Criteria 1

Example 1

User Story 1

Example 1

Collaborate How do I code this feature?What are

the details of this

feature?

How do I validate that this work is done?

Three Amigos • Business Analyst, Developers and Testers • Collaborate • Share knowledge • Think about the desired outcome • Shared understanding

User JourneyYou are a Business Analyst that is looking for a conference to attend.

Example of using an Example Attend BA

Conference

View Results

Region Date

View DCBADD

Details

View information

about IIBADC

View speaker profiles View Cost

Select DCBADD

Register via Eventbrite

Determine Discounts

User StoryAs a business analyst, I want to review the various IIBA discounts available so that I can pay a reduced ticket price for DCBADD.

What is Gherkin?• Business readable, domain specific language• Connects human concept of cause and effect to software

concept of input/process/output• Can be interpreted by automation tools to drive

acceptance tests

Gherkin Syntax

Source: Jeffrey Davidson

ScenariosScenario: Standard RateGiven I am purchasing a ticket to DCBADDAnd I am not an IIBA memberWhen I view my ticket priceThen I see the full rate of $200

Scenario: IIBA RateGiven I am purchasing a ticket to DCBADDAnd I am an IIBA member When I view my ticket price Then I see the discounted rate of $150

ExampleScenario Outline: Determine DCBADD ticket priceGiven I am purchasing a ticket to DCBADDAnd I am a “<membership type>”When I view my ticket priceThen I see the discounted “<ticket price>”

Examples:membership pricenonmember $200

IIBA national $150

IIBA DC $125

ExampleUsing tables as arguments as input to a Given or as expected output from a Then:

Scenario: Validate Order TotalGiven I am a business analyst And I am purchasing multiple tickets with varying “<membership type>” to DCBADD:

When I view my total Then I see my “<total>”

ticket number & type price2 nonmember $4001 IIBA national $150

Guidelines• Be precise• Make sure it’s testable • Focus on business functionality • Avoid specific details • Use active voice, not passive• Have a single action trigger the expected behavior

(WHEN condition)• Use parameters for concise expression of examples (i.e.

scenario outlines)

Automating the scenarios

Automated Acceptance Tests (AATs)

• Code automates them• They run constantly

Why Have AATs? (Pros)

Communication• Helps specify behavior of the system in plain text• Provides a medium for non-tech and devs to agree on

Are we talking about the right system?

Seams, unit test mistakes

Automation

“There’s no place for human beings to be doing regression testing manually.”

-Jez Humble

Speedier deployments

Save resources

More testing during development

Why NOT Have AATs? (Cons)

Why don’t more people use them?

Why NOT have AATs?• Maintenance • Speed• High false negatives, non-determinism

Impediments • Poor adoption • Bandwidth/Velocity• Learning Curve/Experience• Business users won’t write specifications• Developers won’t automate them • Brittle if recorded

Solutions

Acceptance Test Strategy

• Happy paths• Major unhappy paths• Legacy• Regression

Minimize # of end-to-end tests• AATs for journeys, not stories• Is your new story entirely new?• Balance high # of unit tests + selected end-to-end &

acceptance

GherkinSpecs shouldn’t have much setup code

Given my friend ‘Bob’ typed ‘I’m excited’ into the text boxAnd clicks the Post button on my timelineWhen I press the Like link on the postThen the screen should say ‘Like’And should include my name as liking it

Given my friend ‘Bob’ has posted ‘I’m excited’And I am on my timelineWhen I like on the postThen the post should be liked by me

General Tips• Create different levels of suites depending on depth/level

of feedback desired:• Smoke, Current iteration/sprint, Regression

• Run AATs as close to the real environment as possible

When Acceptances Tests catches a bug• See why bug got through unit/integration tests• Add unit, integration tests • Prune AATs?

UI tests• Sometimes tests will fail if the page doesn’t have enough

time to load. Use implicit waits, and explicit when really needed

• Capture screen shots when tests fail

• Selenium IDE

AAT Workflow

AAT workflow• Start with a scenario(s)

Given my friend ‘Bob’ has posted ‘I’m excited’And I am on my timelineWhen I like on the postThen the post should be liked by me

What can you write them in?Vim

Sublime Text

Tracking AATs

Conclusion• Worth the investment if done properly• Very expensive and wasteful if not• Requires collaboration between analysts, testers,

developers • Developers automate the gherkin• Focus on journey and scenarios as opposed to stories

Resources:Books:• Continuous Delivery, Jez Humble, David Farley• Growing Object-Oriented Software, Guided By Tests, Steve Freeman,

Nat Pryce• Specification by Example, Gajko Adzic

Articles:• Automated Acceptance Tests,

http://www.thoughtworks.com/insights/articles/automated-acceptance-tests

• Using SpecFlow with the Page Object, http://blogs.lessthandot.com/index.php/EnterpriseDev/application-lifecycle-management/using-specflow-to

• Maintaining Automated Acceptance Tests (ThoughtWorks), http://www.youtube.com/watch?v=uf0EVbH5hdA

• Creating Maintainable Automated Acceptance Test Suites, Jez Humble,

• http://www.youtube.com/watch?v=v-L_2y6g5DI

Thanks!

Bharti Rupani@bhartir24brupani@enterprise-knowledge.com https://www.linkedin.com/in/bhartir

Wyn Van Devanter @wynvwyn.vandevanter@excella.comhttps://www.linkedin.com/in/wyntuitionhttp://www.slideshare.net/wynvandevanter