Download - Leveraging Open Source Automation: A Selenium WebDriver Example

W9

Test Automation

5/7/2014 1:45:00 PM

Leveraging Open Source

Automation: A Selenium

WebDriver Example

Presented by:

David Dang

Zenergy Technologies

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

David Dang

Zenergy Technologies

David Dang is an industry recognized automation expert who has provided strategy, design, and implementation assistance to more than one hundred organizations. By designing advanced frameworks that reduce maintenance and increase scalability, David ensures clients get the return on investment they expect from automation efforts. As VP of automation solutions for the software QA and testing firm Zenergy Technologies, David is in high demand for his significant knowledge in HP UFT and ALM, SAP automation, Selenium open source frameworks, and mobile testing.

4/26/2014

1

Slide 1 Software Peace of Mind in Action

Presented By: David Dang

Leveraging Open Source Test Automation: A

Selenium WebDriver Case Study

StarEast 2014


???

� Introduction

� Case Study Background

� Decision to select open source vs. packaged tool

� Assess and plan for test automation

� Design an automation framework for Selenium WebDriver

� Framework Implementation

� Outcome

� Conclusion

4/26/2014

2


???

Most companies grasp the importance of leveraging test automation to increase test coverage, reduce execution time, and provide more frequent testing. But test automation can be expensive. More than ever, companies are seeking open source tools like Selenium to reduce the cost. This case study demonstrates one company’s experience using Selenium for test automation.

Introduction


???

� Software company that provides online instructional tools for teachers and students

� Agile software development methodologies used

� Continuous integration deployed

� Company open-minded toward open source toolsets

� Good collaboration between QA and development

� Automated unit testing paired with Continuous Integration tool

Case Study Background

4/26/2014

3


???

� Needed a robust automation suite to perform “UI”testing for integration/regression

� Needed a subset of the automation suite (smoke test) to run as part of Continuous Integration

� Needed method for product owner, developers, and QA to create automated tests

� Needed to ensure maintainability of automated suite

� Needed the automated suite to handle advanced and dynamic web content

Case Study Background


???

� Selenium - Positives

� FREE!

� Supports multiple browsers

� Scripts can be created with multiple programming languages (Java, Ruby, Python, etc…)

� Runs on Windows, Linux, and Mac

� Not confined to the features of the tool

� Integrates with other tools

Selenium vs. HP UFT

4/26/2014

4


???

� Selenium - Downsides

� Only supports web-based natively

� No formal support (the user community provides good feedback but questions are not always answered)

� Higher technical skillsets necessary to optimize the Selenium framework

� Reporting features are not as robust

� Harder to find resources with both QA/testing and Selenium experience

Selenium vs. HP UFT


???

� UFT - Positives

� Established tool with wide usage

� Supports multiple technologies (Web, Java, .Net, SAP, Siebel CRM, PeopleSoft, Terminal Emulator, etc.)

� Has many built-in features (embedded data table, smart identification, object configuration, object repository, etc.)

� Ease of use

� Professional support, although room for improvement

Selenium vs. HP UFT

4/26/2014

5


???

� UFT - Downsides

� Cost! Purchase cost and yearly maintenance fees

� Only supports IE and Firefox

� Cannot run multiple instances on one machine

Selenium vs. HP UFT


???

� Cost

� Alignment with the organization direction (most of the development toolsets are open source)

� Alignment with Agile methodologies

� Ability to test on Mac

� Ability to test on multiple browsers

� Perception by both development and QA that Selenium was easier to adapt

Factors in choosing Selenium

4/26/2014

6


???

� QA roles and responsibilities

� Type of testing within a sprint (unit, functional, integration, etc.)

� Test environments

� Test data strategies

� Level of testing (UI, backend, database, etc.)

� Application technologies

� Defect tracking

� QA resource skillsets

Assess QA Process


???

� Develop a test automation framework/approach based on findings and recomendations from the assessment

Plan and Design Test Automation

4/26/2014

7


???

� Determine the number of resources necessary to build the framework

� 1 Lead and 3 automation developers

� Determine the timeline to completely build out the framework

� 5 months

� Identify Selenium components, development language, and IDE

� Selenium WebDriver, Ruby, Eclipse



???

� Identify execution environment (physical machines, VMs, or cloud services)

� Cloud service

� Identify long-term maintenance and execution resources

� The automation engine is handled by a dedicated automation specialist

� The automated tests are supported by QA or development

� Determine automation development methodologies

� Scrum with 3 weeks sprint

� Determine training and mentoring needs


4/26/2014

8


???

� Build the components:

� Data file

� Object map

� Engine components

� Logging file

� Reporting engine

Test Automation Implementation


???

� Data file: Store test case information


4/26/2014

9


???

� Object map: Store object information



???

� Engine components: Interpret test case information to dynamically create “automation script” during execution


4/26/2014

10


???

� Logging file: The logging file will capture all the events that take place in the Interpret Engine. This will help in debugging the Interpret Engine

� Reporting engine: The reporting will be created in html format. The reporting can be set in two modes; info and debug. In info mode, the html report will only report on Verify Action. In debug mode, the html report will contain all steps in the test case



???

� Automated 830 test cases

� Integration level test cases with average of 38 steps/test case

� 20 test cases are used for smoke testing by pairing with continuous integration tool

� Test cases are divided into three priorities (high, medium, low)

� Development and QA determine tests to run

Project Outcome

4/26/2014

11


???

� Execution of full automation suite takes around 3 hours using Sauce Lab

� Manual testing would have taken 4 resources 40 hours to execute the same suite

� Maintenance of the suite, on average, is less than 8 hours per execution

� New enhancements for the automation framework

� Data file will be replaced by database tables

� A web UI will be created to make test case creation faster

� A test execution scheduler will be created to manage off-hour execution

Outcome of the project


???

There are many benefits to leverage Selenium for test automation. However, it is important to create a detailed plan and roadmap to ensure success.

Conclusion