How to develop a successful Selenium

Automation Program

Dave Rader - CTO

Sreekanth Singaraju – VP of Testing Services

DATE: 6/28/2012

2

About Alliance

Alliance Global Services is a software development firm that partners with software, technology and information-intensive businesses on their mission-critical work. We plan, design, architect, build, test and implement software applications, platforms and products that become a primary driver of innovation and revenue growth for our clients’ businesses. We are recognized for driving quality and speed-to-market when business success depends on the software inside.

We operate agile development centers in the United States and Hyderabad, India and have over 650 employees

We have partnered with more than 400 innovative clients in the Information Services, Software and Technology, Insurance and Healthcare industries

3

Agenda

• Definition of success

• Impediments in Achieving Best in Class

Automation

• Challenges & Solutions for Selenium Automation

• Key Takeaways

• Titanium xAFT Overview

4

Definition of Success for an Automation Program

Effective

Maintainable

Robust

Re-Usable

Test Independ

ence

ROI

Identifies defects of significance

and adopted by Developers &

Testers

The framework should be of low

maintenance and should not take

significant effort to handle routinue

changes to AUT

Can be leveraged on different

releases, configurations &

environments.

Automated test execution produces

consistent and provable results.

Ability to run Automated test scripts

cases in any order.

Delivers measurable value in

terms of decreased cycles of

execution, decreased effort and

improved quality

5

Impediments to Achieving Best in Class Automation

Objectives Non-existent or Unrealistic automation objectives

Focused around technical goals rather than business goals

Automation Strategy

Highest automation ROI areas not identified

Subscale initiatives or insufficient commitment

Start-Up Costs

Tool licenses and framework creation costs

Up front Labor cost for implementing test cases

Expertise & Assets

Manual test teams lack development expertise

Knowledge of test case lifecycle mgmt. missing

Measuring Effectiveness

Lack of a metrics program to measure alignment with overall objectives

Evaluate effectiveness of testing programs and fine-tune strategy

6

Challenges with Selenium Automation

Object References

Ability to identify Objects and manage them to limit the expense and effort to maintain updates to UI

Coding Standards

Develop scripts in a consistent and maintainable format for easy maintenance and effective execution

Reporting Provide commercial quality reporting that enables testers to rapidly identify risks inherent in software and provide information to developers towards remediation

Framework Implement a framework and process that will streamline management of common elements and resources in automation development

Expertise & Skillset

The ability to develop and maintain effectively automation and related assets to deliver ROI on automation investments

7

UI Object Reference

//html/body/div/div/div[2]/ui/div/div[2]/div/h2/a

• Object reference created using

XPATH, ID, Name, Class or CSS.

Common mistakes in constructing

Object references

• Improper selection of Object

selection property type

• Duplicate references to same objects

• Full length XPATH from root used for

object identification

• Leading to brittle references that

break when minor changes to UI are

implemented

• Significant maintenance effort when

UI is redesigned

Most of the Commercial automation tools have the provision to create and manage UI objects in a centralized repository. Selenium has limited tools to address this leading to high-maintenance scripts

8

UI Object Reference – Recommendations

• Utilize common stored location

for objects using logical names

• Use Key-Value-Pair combination

for Logical name and Object

identification

• Logical hierarchies to group

objects for same features at the

same place

• Recommend UI developers use

frameworks that provide unique

identifiers for controls

• Use the smart identification

mechanism and use the

optimized XPATH

//div[@id=’icon-edit’/h2/a

9

Coding Practices

selenium.type("name=j_username",

"user1");

selenium.type("name=j_password",

"user123);

selenium.click("\\div\\div[1]\span\login

");

Thread.sleep(30000);

selenium.click("class=Compose_Email");

• Not using a design pattern or

development strategies to

develop framework and scripts

• No exception catching block in

the coding

• Use of Objects reference such as

path/xPATH in the code

• Use of hard coded values for the

data for inputs and validation

Test engineers that perform script development on Selenium may not have formal development training leading to common coding errors. These lead to expensive maintenance when the scripts need to be updated or unexplained behavior for execution

10

Coding Practices – Recommendations

• Using a design pattern and

development strategies like Page

Object Model OR Domain Driven

Design

• Utilize code style/quality checking

tools to address:

• Exception handling

• Commenting

• Hard coding

• Use reusable coding practices for

handling input data, validations,

object references and common

library functions

• Utilize reference based object

synchronization for script to

synchronize with the app

try {

//Login to the system and Verify Email

selenium.type(userNameLocator,

userNamevalue);

selenium.type(passwordLocator,passwordvalu

e);

selenium.click(loginLocator);

selenium.isTextPresent(welcomeText);

} catch (Exception e) {

Logger.Debug(("Element :["+ welcomeText+

"not found" +

"continuing text execution"));

selenium.click("Compose_Email");

}

11

Reporting & Metrics

• Poring over logs to identify

exceptions is time consuming

and error prone

• Execution metrics such as

coverage, script effectiveness etc

help fine-tune framework

• Test Execution meta data such

as environments, browsers,

pass/fail etc need to be extracted

from the basic logs

• Mapping of pass/fail data to

validation points needs to be

created

• Fatures such as screenshots are

completely missing

Most of the Commercial automation tools provide extensive reporting functionality is not available in Selenium and needs to be developed

12

Reporting & Metrics

• Multiple levels of logging for

easy debugging and analysis

• Drill down Test Result

reports

• Automatic Emailing the Test

Result Reports

• Test Result Report

Integration with Test Case

Management Tools

• Screenshot capture on every

step or on exception

• Defect identification

effectiveness, designed

coverage and traceability to

features

13

Automation Framework

Automation Framework: Target

Browsers, Test Data, Object References,

Reusable Functions, Exception Handling,

Reporting

An example of a poorly designed

framework:

• Poor architecture design limiting

flexibility of choosing target

browsers, target environments

and test data suites

• Lack of standards and guidelines

for reusable functions,

components and object references

• Reporting features limited to logs

from Selenium

• Weak governance leading to

diluted framework effectiveness

Poorly designed automation frameworks only transfer inefficiencies from scripts to the framework while providing limited benefits

14

Automation Framework

Application Specific Business

Functions

Configuration Files

Test Data

Object Repository

Generic Functions

Logging and Reporting

API Support

• Framework should be application-independent.

• Easy to expand, maintain, and perpetuate as the underlying application changes

• Generic functionality like navigation, link checking, login/logout and reports should be developed as reusable functions

• Business functionality will be addressed in test scripts/suites

• Test Data treated as a separate input into scripts

• Implement support for Windows File and Authentication dialogs

• Scripts should have minimal external dependencies from:

– Browser versions

– Target environments

15

Expertise & Skillset

Developers

Automation Engineer

• Facilitate the creation of

automation testers by

creating a cross

functional/technical

training

• Support them with

robust automation

frameworks and

reusable assets

• Ultimately Expensive to

hire, maintain and retain

resources

Currently there are a lot of good developers and good testers – but it is very hard to find good automation Engineers that can develop effective automation

Tester

16

Key Take Aways

Existing Automation Program

Run scripts on more than one browser

Run Sonar and identify code quality issues

Metrics and reports from framework

Validate scripts can run multiple data conditions with no code changes

Change IDs for a few UI controls and evaluate impact on scripts

Governance and process on managing framework

New Automation Program

Review off-the-shelf automation frameworks first before building from scratch

17

Titanium xAFT Overview

Titanium xAFT (eXcel Automated Functional Tester) is a robust enterprise automation framework that enables technical and non-technical users to manage and leverage automation suites

Across browsers and web services

Flexible & Maintainable

framework

Based on Open and proven

technologies

xAFT is a Selenium based Automation tool to test web based applications from an

Excel worksheet using English like language.

30% productivity improvements over Selenium automation

Execute scripts from Excel

Web Mobile WebServices

allianceglobalservices.com/innovation

18

Thank You