Nokia Technology Institute

Software Test Automation Practices in Agile Development Environment: An Industry

Experience Report

Eliane Collins, Dr.Vicente F. De Lucena Jr.

AST 2012

Nokia Internal Use Only

Agenda

1. Introduction

2. Software Projects Characteristics

3. Study Cases

4. Lessons Learned

5. Conclusion

1Instituto Nokia de Tecnologia

Nokia Technlogy IntituteIntroduction

INdT – Nokia Technology Institute

Independent and Nonprofit institute

committed to conducting research and

development of technological solutions

through application development, new

technologies and concepts.

Independent and Nonprofit institute

committed to conducting research and

development of technological solutions

through application development, new

technologies and concepts.

Main Areas:

• Product Creation

• Apps and Services

• Hardware and Manufacturing

• Software Open Source and User

Interface

Main Areas:

• Product Creation

• Apps and Services

• Hardware and Manufacturing

• Software Open Source and User

Interface

Manaus

Recife

Brasília

São Paulo

Instituto Nokia de Tecnologia

2

Nokia Internal Use Only

Introduction - Scrum

• Agile development process focusing onteamwork. Large used in the industry.

• The functionalities are listed in theProduct Backlog.

• The team defines the features whichmay be accomplished within theiteration (Sprint).

• The Software Testing process must beagile for all projects.

3Instituto Nokia de Tecnologia

Nokia Internal Use Only

Introduction – Agile Testing

• Testing with a plan to learn about it,let the customer information guidethe testing in line with agile valuesworking software which responding tochange .

• Tests to prevent defects.

• Proactive Testers.

• Test Automation is the key ofsuccessfully agile development andthe core of agile testing.

• [Crispin, L.; Gregory, J. 2009]

4Instituto Nokia de Tecnologia

Nokia Internal Use Only

Software Projects Characteristics

� Project 1:

� Web interface to register campaigns, users and ads attachingimages, videos and texts with good performance.

� Team: Scrum Master, 3 full-time developers, 1 half time TestLeader and 2 testers.

� Project 2:

� Web project to attend Nokia factory. It automates proceduresto control materials and production costs.

� Team: 4 developers, Scrum Master one Test Leader half-timeand two Testers full time.

5Instituto Nokia de Tecnologia

Nokia Internal Use Only

Software Projects Characteristics

� The Testers used tools to automate test tasks:

� TestLink

� Mantis Bug Tracker

� Subversion

� Jmeter

� Selenium

� Fitnesse

� Java platform

6Instituto Nokia de Tecnologia

Nokia Internal Use Only

Study Cases

� Project 1 First Attempt:

� Development Team automating unit testing and coding;

� Testers automating all UI tests (rec-and-play) and non-functional tests;

� No support of developers and no continuous integration.

� Problems:

� Internally separation of Testers and Developers;

� No Knowledge transfer between scrum team;

� High learning curve in automation test tools.

� Automation difficult to maintain and wasn’t efficientbecause of changes in the interface;

7Instituto Nokia de Tecnologia

Nokia Internal Use Only

Study Cases

� Communication problems between scrum team;

� No time to execute tests to agregate value to the project;

� Waterfall effect;

� Load and Security tests were executed in the end of project

� Project Second Attempt:

� Development Team automating unit testing and supportingTesters with automated functional tests;

� Testers supporting unit tests and automating functional(rec-and-play) for tests in the sprint for stable interface.

� Use of Testlink and Selenium APIs to integrate tools andresults;

8Instituto Nokia de Tecnologia

Nokia Internal Use Only

Study Cases

� Improvements found:

� Knowledge transfer between scrum team;

� Automated regression tests works well for stable interface;

� Time to exploratory tests;

� Communication between team improved;

� Quick Feedback of tests and bugs to project Team

9Instituto Nokia de Tecnologia

Nokia Internal Use Only

Study Cases

� Project 2:

� Developers coding new features, unit testing, supporting UItests, using Continuous Integration (Hudson).

� Testers supporting Unit testing, supporting Hudsonenvironment, automating UI Tests, Load Tests and Securitytests in the beginning of project.

10Instituto Nokia de Tecnologia

Nokia Internal Use Only

Study Cases

� Project 2 Problems found:

� Accuracy data values. The tests for complex business roleswere missing for the layer behind the GUI.

� No Prioritization of the regression test execution.

� Project 2 Solution:

� Team together includes Integration and API Tests usingFitnesse Test Tool and prioritize regression tests in order ofrisk.

11Instituto Nokia de Tecnologia

Nokia Internal Use Only

Lessons Learned

� Collaboration is an Essential Factor for the Success of TestAutomation in Agile Projects;

� Fit Testing Tools with Test Strategy and Agile Method;

� Automate Each Layer of Software When It Is Possible and Just forAcceptance Tests in the Sprint;

� Test Automation in the Agile Project Should Be Simple;

� Prioritize the Regression Test Execution;

� Automate and Run Security Tests and Stress Tests Early ReducesRisks and Rework;

� Use Test Automation for Documentation and InformationFeedback ;

12Instituto Nokia de Tecnologia

Nokia Internal Use Only

Conclusion

� It is feasible to adapt agile practices and values to test automation.

� The automation was a resource to document software, reducecost, and allocate tasks in smaller parts.

� Next steps:

� Evaluate the effectiveness of the test automation agile modelin other development platforms;

� Evaluate the impact of team collaboration levels in the testautomation.

� To extract the testing effort, number of detected failures preand post-delivery, and schedule adherence.

13Instituto Nokia de Tecnologia

Nokia Technology Institute

INFO + CONTACT

www.indt.org

ext-eliane.collins@nokia.com

Thank you.

Questions?