Atd agile automation

1

Making Test Automation Work in

Agile Projects

Agile Testing Days 2012

Lisa Crispin With Material from Janet Gregory

2

Introductions: Experience, Goals § Form groups:

§ New to automation § One to two years experience w/ automation § More than two years experience

§ Talk with at least two other people § Tell each other your learning goals for today § Note the most interesting one you hear

Copyright 2012: Lisa Crispin 3

About me… Programmer, tester, agile team member

Ever-evolving views on automation


Introduction - You


Takeaways

Successful test automation is an innovation! •  Let go of traditional roles, reach across them •  Visualize how tests should look •  Be smart about when to automate •  Experiment to overcome obstacles •  Find/build the right tools We won’t do any hands-on automation, but will demo

some examples

6

Exercise: Your Learning Goals

§ Write one interesting learning goal you heard from another participant at the start of the class on a sticky note.

§ Write your number one learning goal for today on a sticky note.

§ Put the sticky notes on your table group “learning goals” sheet

§ Group similar ones together


Let’s start by defining “agile”

Agile teams: q  Deliver business value

frequently q  at a sustainable pace q  while adapting to the changing

needs of the business Source: Elisabeth Hendrickson


The key is “sustainable pace”

Technical debt slows us down

The Death Spiral

Copyright 2012: Lisa Crispin

High technical debt + insufficient automation = even less time


Barriers to Test Automation What’s holding you back?


Exercise: Your barriers Individually and silently – write one barrier

hindering your team per sticky note Put these on the “impediments” wall chart

for your table group Work with your group to identify patterns,

group similar items


Questions?


The Whole-Team Approach §  Automated tests are code §  Respecting the tests §  Collaborating §  Commitment to quality §  Return on investment


Automated tests are code public class CalculatorFixture extends

ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget; private Calculator calculator = new

Calculator(); private Double irr;

Source: Gojko Adzic, StarEast 2011 keynote

Source: Gojko Adzic, StarEast 2011 keynote


Testers are especially good at… § Eliciting examples § Turning them into tests § Ensuring the right testing gets done § Exploratory testing

19

Give testers time to do what we do best

How?

20 The Whole-Team Approach


Experiment: Iteration 1 §  Pair up: one will be tester, one programmer §  Sit back to back so you face away from each other §  Tester: You have a drawing which you need to

instruct your developer to draw §  Programmer: Remain silent while working, only do

what you’re instructed. §  No talking! §  No peeking!

When “done”: §  Programmer – pass your drawing to the tester §  Tester – note problems on separate card/paperWill

the customer be happy?


Experiment: Iteration 2 §  Now, turn to face each other. §  Tester:

§  Still don’t show your picture! §  Tell the programmer what to draw, but - §  Watch what the programmer draws, and point

out any mistakes immediately §  Programmer:

§  Still no peeking! §  Ask questions, ask for feedback, make

corrections immediately

Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game


Ways to collaborate

Pair: Tester-Coder Tester-Tester Any role – any role


Team responsibility

Automate all regression tests



Commitment to quality What’s your team’s commitment? The best possible software product?


Learn

Experiment

Meaningful Commitment


Under-commit

Plan less work than you think you can do Including all test automation


Learn to write maintainable tests §  Get over the “hump of pain”

From Gerard Meszaros’ XUnit Test Patterns

Whole-team software development

30

Create a user story

Write Customer (Q2) Tests

Start thinking how to code

Expand tests – Story Tests

Automate Q2 Tests

Exploratory testing

TDD

Customer User Acceptance

Product owner Product owner/ Tester Tester Tester/Programmer Programmer

Pair, “Show Me”


Exercise

• Think of an experiment to get your whole team engaged in automating tests • Share with your table group • Pick two to share with the class


Getting Over the Hump §  The test automation pyramid §  The agile testing quadrants §  What should be automated §  What shouldn't §  Difficult areas


Test Automation Pyramid


Test Automation Volcano

From Matt Barcomb


Agile Testing Quadrants


Shared Understanding § Start with tests § Collaborate


What Should We Automate? §  Quadrant 1 tests

§  Unit, component, TDD §  Quadrant 2 tests

§  API, service-level §  Quadrant 4 tests

§  Load, performance, stress §  Quadrant 3 tests?

§  Leverage automation where useful


What Shouldn’t We Automate?

§  Quadrant 2 tests §  Wizard of Oz, prototyping

§  Quadrant 3 tests §  Usability, UAT, ET

§  Tests that will never fail? §  Assess risk

§  ROI not enough §  One-off tests


Where Should We Be Careful?

§  GUI tests §  Watch ROI

§  End-to-End tests §  Push testing down to lowest level

§  Remember the Pyramid


Hard to Automate?

§  Legacy code §  Hard to automate, or just lack of skill? §  “Working Effectively with Legacy Code” –

Feathers §  “Strangling” – Fowler, Thomas


Exercise: Low-Hanging Fruit

Any “easy wins”?


Agile Automation Strategy

§  What hurts the most §  Layered approach §  Applying agile principles

§  Small chunks/thin slices §  Smart test design

§  Choosing the right tools


What Hurts the Most §  Use retrospectives §  Keep an impediment backlog


Multi-Layered Approach Example:

§  Learn TDD at unit level §  Automate GUI smoke tests


Simplicity

§  Address one or two needs at a time §  Understand the problem first §  Try simplest approach first §  Work in small chunks, thin slices §  Incremental & iterative


Automate a Slice at a Time

Example: 4-step UI to validate, upload profit sharing contribution data

•  Thread 1: All four pages with navigation •  Thread 2: Select year, enter description on page

1, display on page 2, browse and upload file on page 2

•  Thread 3: Validate data in file, display on page 3 •  Thread 4: Persist data, display ‘success’

message on page 4


Thin Slice Example Thin Slice Example


Exercise: Thin Slices

Here’s our user story (or theme): As an internet shopper, I want to create an account so that I do not have to enter my address and billing information each time I make a purchase Draw a mind map for this feature on a big sheet of paper Identify a basic end-to-end slice of functionality that can be coded, tested, and automated. If you have time, identify additional slices.

49

Test Design Patterns/Principles §  Code design patterns §  One clear purpose §  Don’t Repeat Yourself


Page Object Pattern A “page object” is a test object that holds the details of all the elements on a web page that might be involved in an automated test.


More on the Page Object Pattern http://www.beer30.org/?p=54


Demo: semi-automating exploratory tests


Iterative Feedback

§  Spike two different approaches §  Pick one to try for N # of iterations §  Use retrospectives to evaluate


Learn by Doing §  Courage – don’t be afraid to fail §  Production code practices for test code §  Incremental, thin slices §  Experiment


Questions About Automation Strategy?


Choosing Tools, Frameworks, Drivers

§  Team effort §  Time §  Requirements §  Focus on goals, problems, not tools. §  Experiment


Understand the Purpose

§  Who’s using the tests? What for? §  What’s being automated? §  Existing tools, environment §  Who’s doing what for automating?


What Fits Your Situation •  Existing skills •  Language of application under test •  Collaboration needs •  What’s being automated •  Life span, future use of tests


Test Drivers/Frameworks §  Divide into sub-teams and prototype

solutions §  Have a “bake-off” showing each prototype


Where To Find Tools

§  www.softwareqatest.com/qattls1.html §  http://bit.ly/AgileTestTools §  www.testingfaqs.org §  www.opensourcetesting.org §  groups.yahoo.com/group/agile-testing §  Grow your own!


Example: One Team’s Tool Choices

•  IntelliJ Idea •  Jenkins, ant, Maven •  JUnit •  FitNesse •  Canoo WebTest •  Watir •  Robot Framework •  JMeter •  Selenium 2.0 / WebDriver with Geb framework


Exercise: Test Frameworks

What format would work for your team? EG, BDD style? Tables? Assertions? Scenarios? Think of the test format that might work for your team. Write examples on sticky notes. Write down any particular types of test patterns, frameworks and driverss you want to learn more about. and place these sticky notes next to an impediment it might help overcome.. Discuss with group how that tool might help solve the problem.


Making Test Automation Work

§  Time to do it right §  Experiments, bake-offs §  Testable architecture §  Test data §  Managing tests


Time To Do It Right

§  Limit scope, don’t over-commit §  Write automation task cards §  Plan test code maintenance/ refactoring time §  Quality must be team goal §  Long-term, will let you go faster


Bake-offs

§  Have sub-teams/pairs experiment with prototypes

§  Budget lots of time for learning, experimenting

§  Look at pros & cons


Testable Architecture •  Layered architecture

•  eg. UI, business logic, data access •  Ports and Adapters pattern

•  App can work without UI or database •  Ports accept outside events •  Adapters convert for human or automated

users


Test Data

§  Avoid database access when possible §  Setup/Teardown

§  Independent, rerunnable tests §  Canonical data

§  Refresh before each test run §  Customizable data for ET §  Production-like data

§  Get customers to provide example data


Managing Automated Tests

§  Tests as Living Documentation §  Continuous Integration §  Reporting results §  Metrics


Tests as Living Documentation §  Understandable §  Who will really use them? §  Once passing, must always pass


Any Example Can Become a Test


Given/Then/When Example

Scenario: Valid name search returns results GIVEN that Kant is a supervisor with employees AND Kant has an employee named Smith WHEN Kant navigates to the employee name

search page AND enters the value “S” THEN Kant will see a search result that includes

Smith


Continuous integration/testing…

…short feedback loop


Keep tests passing

Stop the line to fix problems


Test Management Tools §  Manage tests, code together §  Some tools have own management §  What problem are you trying to solve?


Exercise: Tests as Documentation

76

Key Success Factors

Whole Team Approach�

Thin Slices�

Short Feedback Loops�

Experiments�

Specification by Example �


Exercise: Breaking Barriers


Remember §  It’s a team problem! §  Tackle automation problems with diversity §  Try small experiments §  Baby steps – start simple


Questions? “Aha” Moments?


Agile Testing: A Practical Guide for Testers and Agile Teams

By Lisa Crispin and Janet Gregory www.agiletester.ca


Experiences of Test Automation

Dorothy Graham and Mark Fewster


Beautiful Testing: Leading Professionals Reveal How They Improve Software

Edited by Tim Riley, Adam Goucher

Includes chapter by yours truly


Test Patterns

Xunit Test Patterns: Refactoring Test Code

By Gerard Meszaros

Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire

84

Specification by Example How successful teams deliver the right software

Gojko Adzic

Case studies from > 50 teams


Agile Test Automation Resources

§ dhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdf § http://bit.ly/AgileTestTools § lisacrispin.com § janetgregory.ca § gokjo.net § exampler.com § [email protected] § testobsessed.com § testingreflections.com § pairwith.us

Date post:	24-Apr-2015
Category:	Documents
Upload:	lisacrispin
View:	1,052 times
Download:	6 times

Atd agile automation

Documents