Build the Right Regression Suite with Behavior-Driven Testing

T14

Test Techniques

5/8/2014 1:30:00 PM

Build the Right Regression Suite

with Behavior-Driven Testing

Presented by:

Anand Bagmar

ThoughtWorks

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Anand Bagmar

ThoughtWorks

Anand Bagmar is a hands-on and result-oriented software quality evangelist with seventeen years in the IT field. Passionate about shipping quality products, Anand specializes in building automated testing tools, infrastructure, and frameworks. He writes testing-related blogs and has built open-source tools related to software testing-WAAT (Web Analytics Automation Testing Framework), TaaS (for automating integration testing in disparate systems), and TTA (Test Trend Analyzer). Anand is the lead organizer for vodQA, the popular testing conference in India. Follow him on Twitter @BagmarAnand, email him at [email protected], or read his Essence of Testing blog.

Building the “right” regression suite using Behavior Driven Tes5ng

(BDT) Anand Bagmar

So#ware Quality Evangelist Principal Consultant, ThoughtWorks

•  “Succeeding with Agile” – Mike Cohn

•  MarCn Fowler – Test Pyramid –  hFp://marCnfowler.com/bliki/TestPyramid.html

The Test Pyramid

Func%onal Tests (GUI) •  Developers & / QA team

Web Service Tests •  Developers & / QA team

Integra%on tests • Developers Component tests •  Developers

Unit Tests •  Developers

Test AutomaCon Pyramid

Manual / Exploratory TesCng

Product under test

Ideal Test Pyramid

Business-‐facing Tests

Technology-‐facing Tests

•  Cost •  Effort •  Time

Test Pyramid reality

Development team Test Pyramid QA team

Test Pyramid






Test Ice-‐cream cone



Test Pyramid – AnC PaFern

Product under test






Product under test

Ideal Test Pyramid



IdenCfy the right type of tests that sit on top of the Test Pyramid

Case Study

•  MulCple, long running projects

•  Legacy applicaCons

•  Integrated

•  Limited AutomaCon •  Long regression cycle •  Huge cost of fixing defects

TesCng Challenges

•  No visibility into what is tested •  Outdated •  BriFle •  LiFle / less value •  Expensive •  Maintenance nightmare

As a result

Biggest problem

•  Trust deficit on the team –  In exisCng AutomaCon –  In manual tesCng

•  Finger poinCng / blame game

•  Remove the ambiguity •  Same visibility for all

EvoluCon of Test Frameworks Behavior Driven

Development (BDD)

OpCmized scripts

Record & Playback

Core Framework

Behavior Driven Development (BDD)

RED

GREEN REFACTOR

Behavior Driven TesCng (BDT)

The Big Picture

•  How is this funcConality going to be used by the end-‐user?

•  What is their thought process going to be when using this product?

•  What is the “core-‐business-‐value” I am delivering to the end-‐user?






Product under test

Ideal Test Pyramid



IdenCfy the right type of tests that sit on top of the Test Pyramid

Expected Func%onality:

The customer should be able to withdraw money from his account via an ATM machine

An example

Given the account has balance of INR 5000 When the customer requests INR 1000 Then the account is debited by INR 1000

BDD Example:

Given the account is in credit

And the dispenser contains cash

When the customer requests cash

Then ensure the account is debited

And ensure cash is dispensed

And ensure the card is returned

BDT Example:

New Expected Func%onality:

The customer should be able to withdraw money from his account only if his card is valid

Given the account is in credit And the card is valid And the dispenser contains cash When the customer requests cash Then ensure the account is debited And ensure cash is dispensed And ensure the card is returned

BDT Example -‐ updated:

Test specificaCon styles

•  ImperaCve •  DeclaraCve

•  Write user flows based on personas, and how they use the business funcConality, or,

•  Write business flows, and how different user personas can use that

Another example

For Indigo Airlines website –

•  Guest User is able to search for a flight for a single traveller

•  Ability to specify contact informaCon for person booking the flight

Search Search Results

Contact Informa%on

ImperaCve style Given I am a guest user on the Indigo home page And I select “round” trip opCon And I select “Pune” from the origin dropdown And I select “Bangalore” from the desCnaCon drop down And I select departure date as “5 December 2013” And I select returning date as “25 December 2013” When I click on Search Then I should see the search results page And I should see at least 1 opCon for my criteria ...

… When I select the “first” opCon Then I am on the Contact InformaCon page When I enter first name as “foo” And I enter last name as “bar” … … And I click the “Select and Con%nue” buFon Then I should be on the next page

Given I am a guest user When I search for flight opCons for a “one-‐way” trip for “1” “Adult” from “Delhi” to “Bangalore” And I select the “first” flight And I enter “valid” contact details for “traveller1” Then I am able to Save and ConCnue

DeclaraCve style

What’s next?

DO NOT AUTOMATE •  Low Value, High Cost

• Manual Regression!

2nd candidate for automa%on •  High Value, High Cost

Automate ONLY if you have %me •  Low Value, Low Cost

•  ? Manual Regression?

1st candidate for automa%on •  High Value, Low Cost

Cost

Value





BDT helps iden5fy the “right” type of regression tests!


Product under test

Ideal Test Pyramid

A real world Case Study

How did we accomplish this?

Who is doing the acCon?

What is the business funcConality?

AcCons and verificaCons happen implicitly Depicts a user flow

Given auc%oneer creates a sale And 5 items are added to the sale When auc%oneer starts the sale And auc%oneer sets a starCng bid of 5000 And buyer_1 bids And buyer_2 bids Then auc%oneer Sells the item And buyer_2 wins the item

Final state Tests running on Hudson / Jenkins Since 1st test was automated

Feature files 33

Scenarios 65

Smoke tests execuCon Cme 15 minutes

Regression tests execuCon Cme 45 minutes

Test case repository (manual + automated)

Cucumber .feature files with appropriate tagging

Project Management tool Mingle

?

Incrementally build User Flows

Regress required Product Business

Value

Tests in-‐sync

EffecCve Tests

Remove ambiguity

Understand new

funcConality

Living DocumentaCon

Onboarding

Tools to enable BDD / BDT

•  Cucumber •  JBehave •  SpecFlow •  Twist •  Concordion

[email protected]

@BagmarAnand

about.me/anand.bagmar

Thank you

Date post:	11-May-2015
Category:	Technology
Upload:	techwellpresentations
View:	458 times
Download:	3 times