Agile Test Management with Open Source Testing

AGILE PERFORMANCE TESTING

Todd Bradfute, [email protected] Cohen, +01 408-871-0122, [email protected]

August 2, 2011

Managing Software Projects With Agile Methodology

Open Source Software (OST) Adoption

Open Source Test Automation

About PushToTest‣Our Mission: We Test, For A More Reliable World

‣Make It Easier To Move From Manual To Automated Testing

‣Support Advanced Testing, including Agile, Load, Integration

‣Open Source Testing (OST) For Everyone

2


Intellectual Property of PushToTest. (c) 2009 All rights reserved.

Your Team‣Frank Cohen, CEO and Founder, PushToTest

[email protected]

‣Todd Bradfute, Professional Services, [email protected]


Slides on Twitter‣http://twitter.com/fcohen

‣Please retweet

‣Please follow me

4

Next let’s talk about


Agile Open Source Testing

‣Test Driven Development

‣Roles and Fungability

‣Testing and Test Management

5


Agile Is Seldom Achieved‣Test-First development

‣Resource interchangeability (aka “fungibility”)

‣Commercial tools lock test tools to “testers”

‣OST lets everyone on the team be more involved.


Overview‣Test Driven Development

‣Resource Fungibility

‣OST as the Lingua Franca

‣Certification using OST

‣Demonstrable Results

‣Detailed Results

‣Provable Results


Test Driven Development‣Write the test before implementing the feature

‣To expand lookupZip(String addr) to return zip+4

• First: write test that expects #####-#### as output

• Second: Run it and watch it fail because output is #####

• Third: Change implementation

• Fourth: Run it and watch it pass

‣You’re finished adding the feature when this (and all other) tests pass

‣OST enables TDD


Resource Fungibility‣Any person can work any task

‣Most Agile teams don’t even try

‣The prize is extreme velocity and flexibility!


OST - The Lingua Franca‣Cross testing ... i.e. person A’s code is tested by person B

‣A and B are both members of the agile team

‣Both A and B may be developers, but not testing their own code

‣Using OST the team selects a standard tool set

‣On the current Best Buy project the primary OST is soapUI


Certification Using OST‣At Best Buy the organization has a “certification” process

‣Starts with the sub-teams own test artifacts

‣Reviews plans and execution output

‣Responsible for repurposing tests as concurrency/load/performance tests


Demonstrable Tests‣Reports are saved to the database

‣The Agile process is enabled by using Rally

‣Process flow:

‣User Story -> Test Case -> Test Card -> Test Results -> Test Output


Rally/TestMaker integration


Easy Control For EverybodyPushToTest TestCards - Best Buy

New TestCard

Assign To Rally

List TestCards

My TestCards

3rd Party Connector

TSH Mock Services

Address Lookup

Verizon Registration

fcohen signed-in, help, sign-out

TestMaker Enterprise 6.0.1

Licensed for 50 virtual users

and 10 service monitors

Welcome To TestMaker TestCards

3rd Party Connector View ResultsDeleteAssociate Rally User StoryCreate Copy


Detailed Results‣Seeing a pass is comforting, but how do you know it really did?

‣Use drill-down to see details on a passing case

‣Drill down from a fail to the actual error messages

‣Anyone on the team can review any pass/fail results


Transaction Report


Drilled Down Detail‣Input:

‣http://server1.bestbuy.com/TSH/v1/offers HTTP/1.1

‣…

‣{ "variable1": "value1", "variable2": "value2" …}

‣Output:

‣HTTP/1.1 500 Internal Server Error

‣...

‣<response>

‣ <error>Unrecognised resource: offers</error>

http://dng01apl.na.bestbuy.com:8080/TSHVendorService/retailers/v1/of

http://dng01apl.na.bestbuy.com:8080/TSHVendorService/retailers/v1/of


Provable Results‣Test results should be 100% reliable, but what’s the safety net?

‣Run it again, Sam.

‣Rally -> Test Case -> TestCard -> Execute

Next let’s talk about


Agile Performance Testing

‣Test Orchestration

‣Repurposing Methodology

‣Cloud and Grid Deployment

19


Why An OST Alternative?‣HP QuickTest Professional does a great job for testing a small part of

the Internet

‣Functional Testing (Smoke, Regression, Integration)

‣Windows using ActiveX

‣Flash, Flex

‣MS Internet Explorer

‣Web 2.0 with Coding Effort

‣Best Money Can Buy

20


OST For Everywhere Else‣Open Source Testing Tools

‣TestMaker Object Designer

‣Selenium

‣Sahi

‣Unit Tests (Java, Ruby, Python, PHP)

‣Record in WebKit based browsers (Chrome, Safari, Opera), IE, Firefox

‣Agile Testing, Agile Performance Testing and Production Monitoring

‣Rich Internet Applications (RIA,) Mobile, Pads

‣OST For Everyone (Developers, Testers, Business Managers)

21


OST Architecture

22

Application Under TestWeb, RIA, BPM, SOA

PTTMonitorCPU, Net, Memory

Test Injector

BundlesMySQL, HtmlUnit

ScriptRunnersSahi, Selenium, SoapUI, Java

Data Production Librariescsv files, rdbms queries

Test Operation Console

RepositoryMySQL RDBMS

Results AnalysisBusiness Intelligence Tool

DesignerRecord/Playback

Grid and CloudTest Deployment

Continuous Integration Test Management


Where To Go From Here‣Watch A Screencast and Tutorial

‣http://www.pushtotest.com/screencastcentral

‣Attend An Open Source Test Workshop

‣http://workshop.pushtotest.com

‣Ask For A Proposal on Licenses, Training, Support, Consulting

‣Call +01 408 871 0122, [email protected]

23

Date post:	11-Nov-2014
Category:	Technology
Upload:	appvance-inc
View:	2,016 times
Download:	3 times

Agile Test Management with Open Source Testing

Technology