A Structured Approach to Enterprise Test Automation Venkatesh Kovvuri Head of Innovation &...

A Structured Approach to Enterprise Test Automation

Venkatesh KovvuriHead of Innovation & Automation Practice

© 2012 Computer Sciences Corporation - Proprietary 2 2

Overview

» With increase in the complexity of application software and enterprise infrastructure, the

quality guidelines & standards have also mounted. Effective testing of such apps can’t be

assured by manual testing alone. By reducing human intervention during test processes

and automating them can result in an effective and efficient methodology to meet those

standards.

» A Structured Approach will provide opportunity to deal with complex application software

and enterprise infrastructure.

• Extends the usage of automation to entire software testing lifecycle in addition to conventional automated test

execution (design, data, tasks, infra etc)

• Addresses challenges around selection, prioritization and execution of automation projects in large enterprises

• Encompasses all key components essential for executing complex automation projects

• Provides an asset ‘toolkit’ for Test Automation - Process, Technology, Integration and Quality

• Supports integration with industry leading open source and commercial software testing tools

• Provides visibility into the aspects of Quality Governance on the application portfolio

© 2012 Computer Sciences Corporation - Proprietary 3

Enterprise Application Landscape

Legacy

systems

Packaged

software

External

interfacesCustom

applications

•Integration APIs (partner-specific)

•EDI•Web Services (SOAP/REST)

• Handheld interfaces

• Thick/thin custom clients

•Custom web applications

•Custom reports•OLTP Databases•Data warehouses

•Transaction processing (Mainframes and AS/400s)

•Old databases •Obsolete technologies

•ERP•Sales•CRM•HRMS•Supply chain•BI•CMS•Accounting

Maintenance Migration

New development Integration

Quality Assurance &Testing

Functionality, P

erformance and S

ecurity


Tool support for Enterprise Testing

Test Management Tools Test Automation Tools

Test Design Tools

Data Preparation Tools

Review/Inspection Tools

Proprietary Test Runners

Custom Test Harnesses (and Drivers)

Dynamic Analysis ToolsStatic Analysis Tools

Comparators

Performance Testing ToolsSecurity Testing Tools

Lack of Unification into a Singular Platform


Enterprise Test Automation Business ChallengesChallenge Description

Candidate Selection/Prioritization

Enterprise IT stack contains a large number (dozens to hundreds to thousands) of applications and often pose the difficulty to select and prioritize from automation perspective.

Tool/technology Proliferation Automation tools operate as vendor-locked and tightly coupled solutions with the technologies without giving the business a way to control them in a unified and standardized fashion.

Change Management Alignment

Change is the continuous mantra (maintenance, migration, integration, retirement etc) in the IT stack and often times the strategy to align automation remains out of control.

Success Criteria Definition and Measurement

While ROI remains a known metric, continuous measurement of the value delivered by automation requires a rich definition of the success criteria and continuous measurement (with corrective actions)

Future-proofing automation assets

The percentage of assets that can be used beyond the scope of the planned cycles remains an unknown and often times events like tool migration etc make some of the old assets vendor-locked and void.


Enterprise Test Automation Technical Challenges

1. Multiple testing tools for heterogeneous technology stacks2. Multiple interfaces(GUI, Web, API, EDI, Mobile) with varied automation needs 3. Test coverage involving emerging versus legacy applications4. Test scenarios involving complexly integrated partner systems5. Test suite adequacy for controlled and emergency releases6. Capability of automated regression test automation to find new bugs in old

functionality7. Changing test priorities

1. Compliance vs. Business functionality2. Manual bug hunts vs. Automation efforts3. Functionality vs. Performance vs. Security

8. Lack of rich test data (variety &volume) that simulates the live enterprise9. Impact of Migration on Test Automation assets10. Multi-environment deployments (repetitive testing)

1. Synchronization2. Data population


TestPlanningToolkitExecutive

QualityControl

Dashboard

TestPlanningToolkit

Reusable Automation

Assets

TestDesignToolkit

Test Data

Toolkit

Test Execution

Toolkit

Test Management

Toolkit

Packaged Application

Accelerators

TestInfrastructure

Toolkit

ExecutiveDashboard

Enterprise Test

AutomationApproach

A Structured Approach

Market Test

Automation Tools

Internal Test Automation Tools

Automation Asset Library

Automation Methods

Automation

Approach

Enterprise Applications

Test Automation


Test Planning Toolkit

ROI Calculator

» Web enabled application to forecast returns and thereby prioritize automation projects

» Ease decision making process based upon qualitative and quantitative ROI information

Feasibility Analyzer

» Logical, quick and efficient way to identify automation candidates from the application landscape

Select, Prioritize and Plan for test automation

ROI Calculator Feasibility Analyzer


Reusable Automation Assets

Reusable assets and Process assets to reduce time to market » Over a period of time CSC has developed lots of reusable assets and Process assets» CSC ACOE made those assets generic and can be used for any projects.» Category wise reusable assets repository has set of assets that enables to start and execute an

automation project with minimum expendable effort » Process assets has Checklists, Guidelines, Templates and process documents

Category Number of functions

Test Logging & Reporting 133

Notification Utility 25

Test Management Tools Integration 26

Ready-made Plug-In for Global Utility, Component/Reusable Functions,Events,Business Processes. 66

Exception handling & Diagnosis Information 14

Folder Management 5

File Management 38

Web Application's Control 263

Technology specific (Linux+Sieb_GlobalFuncs+Siebel_DP_Functions) 54

Total number of reusable functions 624

050

100150200250300

Series1

Check lists and Guidelines

Hand Book-ATM - Test Automation

Automated Test Script Review Checklist

Test Automation Requirements Elicitation Checklist

Test Automation Framework Guidelines

Test Automation Guidelines

Test Automation Metrics Operational Definition

Test Automation Tool Evaluation Guidelines

Test Automation Engineering Process

Test Automation Framework Selection Guidelines

Templates

Test Automation Cost Calculator Template

Test Automation Execution Report Template

Test Automation Maintenance User Guide Template

Test Automation Plan Template

Test Automation ROI Calculator Template

Test Automation Scope and Traceability Template

Test Automation Suite Release Notes

Automation Test Tool Feasibility Report Template

Test Automation Tool Evaluation Template

Reusable assets Process assets


Test Design Toolkit

Test Case Optimizer» Proprietary implementation of Pair

wise/Orthogonal and other scientific techniques that reduce the exploded test combinations into an optimized/smaller number

» Capability to deal with large Test Input Models

» Detects up to 80% of the total defects with optimized test cases

Web Services Test Case Generator

» Provides automated generation of test cases specifically for SOA testing

» Covers both positive and negative test cases ensuring 100% test coverage

» Reduces test case authoring effort up to 90%

Optimal and efficient test case generation

Test Input Model Test Coverage Configuration

Optimized Test Cases

Test

In

pu

t Mo

del Journey mode: Return trip, One way

Preferred time: Morning, Afternoon, Evening, Late night

Leaving from: Type1_City,Type2_City,Type3_City

Going to: Type1_City,Type2_City,Type3_City

Booking type: Economy, First class

Customer type: Registered, New

Adults: 1, 2, 3, 4, 5, 6

Children: 0, 1, 2, 3, 4

Test

Cov

erag

e C

onfig

urat

ion

2-way coverage strength O

ptim

ized

Tes

t C

ases #Total Test

Configurations for 100% coverage : 8640

#Total (reduced) Test Configurations : 31


Test Data Toolkit

Test Data Generator» Large volumes of real-looking test data is

generated.» Support for 50+ data types for US & UK (like

real names, addresses, SSNs, custom-syntax and so on)

» Support for multiple groups of fields with dependencies like referential integrity

» No restriction on volume of data generated (speed: 1 million records per minute)

Field Data Validator

» Automates negative and security test data generation for use by developers to test code under development

» Provides both command line and GUI interface options

Reduce cycle time by quickly generating large volumes of test data

Field set Metadata

Volume configuration

CSV file with real-looking data

Fie

ld s

et

Met

adat

a <?xml version="1.0"?>

<TestFields>

<Groups groupName = "sample" groupNumber ="1">

<TestField fieldName ="person name" id = "42">• <PersonName>

• <Country>US</Country>• <Gender>Female</Gender>• <Format>FirstName</Format>

• </PersonName>

</TestField>

…

Vol

ume

Con

figur

atio

n

1000000 records

Rea

l-loo

king

da

ta person name,PhoneNumber,SSN,CompanyNameTASHA,+91(0)52522-86434 ext.6072,117-93-2658,Diamond of CaliforniaJULEE,+91(0)532-3213402 ext.5136,628-62-5959,MerriamWebster IncPENNY,+91(0)8343-878361 ext.5427,622-23-8243,Komag IncBRYNN,+91(0)2462-078412 ext.7113,174-80-4445,Sweetheart Cup Company Inc


Test Execution Toolkit

Unified Interface Automation Framework» CSC keyword driven framework for test case development

that abstracts the process enabling test case development to be performed by SMEs & BAs

» CSC Hybrid framework is the combination of Functional Decomposition & Data Driver approach.

» Tool and Technology agnostic » Integrated with QC and TEMA, Customized report

generation

Abstract test execution and simulate third party integration

Other Automation Frameworks» GSTAR – Transaction based framework for Insurance

domain» FLAME – Accelerated framework for Finance Domain» Hybrid (Functional and Data Driven) Framework

» QTP ,Selenium and Silk Test» Wiki Based framework

» Finesse + Selenium


Test Management Toolkit

TEMA (Test Management)

» Integrated QA suite with an intuitive web-based interface for effectively managing and monitoring the Quality Assurance processes

» Offers full product life cycle support and can be used to manage multiple test cycles

» Integrated with CSC customized version of the popular BugZilla bug tracking application which is further enhanced for mandatory authorization, limited privilege access and custom reporting features

Manage test lifecycle intuitively


Packaged Application AcceleratorsAccelerate test automation of complex systems

» Integrates market, third party and homegrown solutions to accelerate key testing activities on packaged software» Provides accelerators to readily automate plain vanilla installations like

» Key SAP processes» Key Siebel processes» Key PeopleSoft processes» Key Oracle E-business processes

Module Manual Test Cases

Automation Test Scripts

Oracle-GL 127 112 Oracle-AP 138 122 Oracle-AR 80 74 Oracle-FA 67 60 Oracle-CM 30 29 Oracle-Iexpense 13 13 Oracle- Purchasing 90 90 Oracle-I procurement 27 27 Oracle-OM 159 155 Oracle-Inv 124 118

Total 855 800 SAP-GL 42 40 SAP-AP 32 30 SAP-AR 17 17 SAP-AA 33 33 SAP-Co 22 22 SAP-BA 5 5 SAP-MM 92 92 SAP-SD 152 152 SAP-IS Utilities 78 39

Total 473 430 PeopleSoft-ESS 21 14 PeopleSoft-MSS 23 20 PeopleSoft- Payroll 17 17 PeopleSoft-WFA 66 55

Total 127 106

0

50

100

150

© 2012 Computer Sciences Corporation - Proprietary 15 15

Test Infrastructure Toolkit

AIM (CSC Infrastructure Management)» Multi-tenant inventory and infrastructure management web application» Interfaces with Test Automation tools, providing an environment for

management of un-attended test execution and results monitoring

Manage test infrastructure remotely and efficiently


Executive Dashboard

Facilitates Go/No-Go decisions for production releases

A centralized dashboard for management teams to view quality & productivity data and reports:» Enables benchmark comparison across enterprise testing projects» Provides just in time statistics» Relieves QA team from routine capture and computation processes» Eliminates errors caused by manual processes and wipes out data manipulation possibilities» Helps take informed decisions on project performance


Contact:

Venkatesh KovvuriMobile: 09885579911Email: [email protected]

18© 2011 Computer Sciences Corporation

Date post:	17-Dec-2015
Category:	Documents
Upload:	damian-jenkins
View:	216 times
Download:	0 times

A Structured Approach to Enterprise Test Automation Venkatesh Kovvuri Head of Innovation &...

Documents