Faster SAP Testing Means Business Agility[1]

8/2/2019 Faster SAP Testing Means Business Agility[1]

http://slidepdf.com/reader/full/faster-sap-testing-means-business-agility1 1/12

FASTER SAP TESTINGMEANS BUSINESS

AGILITY

02.04.2012

AN EASY-TO-USE, CLOUD-BASED, SECURE SOLUTIONFOR TESTING SAP SYSTEMS



- 1 -

FASTER SAP TESTING MEANS

BUSINESS AGILITYIf you have an SAP ERP installation at your company, or run one for a client, one of the biggest

challenges is avoiding negative side effects to the production system when changing or adding

business processes, customizations, patches, or upgrades. This is the main reason SAP requires

businesses to run a separate instance of its ERP system in order to conduct testing. Regression

testing helps ensure that changes do not impact production systems.

Thorough testing slows the pace of needed changes and reduces agility. If you could speed up

the testing process significantly, you could move faster without increasing risk.

Most organizations test their SAP applications manually before they move packages of changes,called transports, to production systems. Manual testing is labor-intensive, requiring hours of

tedious work. There are a few automated testing solutions on the market, but for the most part

they are expensive, difficult to implement, and time-consuming to use and maintain. Many

administrators would like a more cost effective and simplified solution to test SAP applications.

There is now a third, emerging generation of testing solutions that focuses specifically on

packaged applications and reduces the effort of capturing and running tests scenarios by a

factor of 10. This new breed of testing solutions also offers increased peace of mind, knowing

that test coverage is complete and that everything will run smoothly.In this paper, we track the development of SAP testing techniques through these three

generations and explain how the latest generation of testing tools represents a giant step

forward in addressing a challenge that all SAP customers face.

THE THREE GENERATIONS OF SAP

TESTINGSoftware testing methodologies and tools have evolved to support the demands of the business

world. In this section we discuss the pros and cons of three generations of testing solutions:

manual, automated, and intelligent.



- 2 -

THREE GENERATIONS OF SAP TESTING TOOLS

GENERATION OF TESTING

TOOLS

CREATING SCRIPTS RUNNING SCRIPTS MAINTAINING SCRIPTS

FIRST GENERATION:

MANUAL TESTING

Manual Manual Manual

SECOND GENERATION:

AUTOMATED TESTING

Semiautomated Automated Manual

THIRD-GENERATION:INTELLIGENT TESTING

Automated Automated Automated

FIRST GENERATION: MANUAL TESTINGThe first generation of testing, manual testing, is still the method most companies use for the

bulk of their testing. Because business processes trigger functions in SAP, manual tests are

often created and run by power users or SAP functional experts, who view testing as an extra

step in the release process. They see it as time-consuming and tedious, with very little payoff.

Manual testing causes friction between end users, power users, functional experts, and IT

managers. While IT managers worry about system stability, users in all categories see testing as

a burden that is added on top of their regular responsibilities.

More than half of the organizations surveyed don‟t use a testing solution at all. Of those who

do test, 74% don‟t use an automated solution, and 53% use general-purpose business software

such as Microsoft Word and Excel. (Source: 2011 SAP Testing Survey)

Despite the investment level of most companies in SAP ERP and its importance to their business,

testing still gets short shrift.

Manual testing remains the most common method of SAP testing. On the one hand, it is

relatively economical, at least from the perspective of purchasing software and staffing.

Because no special skills—besides understanding the SAP application and having a great deal of

patience—are required, and no special software needs to be purchased, many organizations

continue to test this way, creating test plans and executing them.



- 3 -

On the other hand, manual testing is a costly and ineffective way to limit risk to an expensive

investment. Manual testing is time-consuming, inefficient, and highly prone to errors. It‟s

impossible to achieve 100% coverage for all possible usage scenarios. The result is that

frequently little quality testing gets done when manual testing is the only option.

In many ways, this slows down a business‟s ability to innovate and to keep up to date on the

latest software releases. Concerned about the cost and delays involved with upgrades and

testing, many organizations are slow to make changes to their SAP installations. This limits

opportunities provided by SAP‟s „best practices‟ enhancements to innovate and become more

efficient on a business level; on the technical system front it actually heightens the risk of a

catastrophic system failure or non-supported technical problems.

PROS AND CONS OF MANUAL REGRESSION TESTINGPROS CONS

Can handle any script and playback

permutation scenario using “human

intelligence”

Economical in terms of software purchases—

most manual testers use standard business

software that is already in-house

Simple—the only requirement is to accurately

record the results of changes made to the SAP

software

Economical on a per-resource basis—little skill

required

Not comprehensive—limits the ability of an

organization to roll out changes to SAP

Labor-intensive—high labor cost relative to level

of risk control provided

Risky and error prone because of human

limitations related to extended focus, accuracy,and repetition

Slow—Since all testing is done by humans,

thorough testing takes time. The tradeoff is

whether to sacrifice quality or delay the release

SECOND GENERATION: AUTOMATED TESTINGThe second generation of SAP testing delivered a significant leap forward. Quality assurance

(QA) platforms were built to test software, offering several advantages. Business users orfunctional experts no longer needed to devote significant time to running the tests, and the

tests could be run continuously with minimal disruption. The ability to automate a far greater

number of tests offered new potential to diagnose issues with SAP installations and accelerate

rollout schedules.

There were still significant disadvantages, however.



- 4 -

COMPLEXITY AND EFFORT

Test automation is a resource-intensive undertaking, requiring programmers for home-baked or

vendor solutions. The typical test automation project includes setting up dedicated, on-premise

hardware and software and hiring a full-time test automation programmer and administrator tomanage the project. The deployment of these systems offers substantial configurability but it is

quite complex.

EXPENSE

Organizations could spend close to $300,000 before a single test is run. Often, the tests run

aground before any useful information is collected.

PROS AND CONS OF AUTOMATED REGRESSION TESTING

PROS CONS

Business users not as heavily consumed in

tedious testing execution

Increased test execution efficiency as

greater volume of test repetitions can be run

in a given time period

Testing depth and accuracy improves—for the

constructed scripts

Once tests are created, they can be run

again and again (subject to maintenance

work)

Offshore testing resources can help develop

tests

Complexity—Takes up to a year to implement. Coverage is

limited because of time and effort to create scripts for

each business process

Effort—full-time test automation developers are needed

Expense—as much as $300,000 before the first test is run

Tools are general-purpose; not explicitly designed for

testing SAP systems—requires tedious specification and

programming work by both functional experts and test

engineers

Disconnect between functional experts and testing pros

(external and offshore) can lead to coordination issues and

even logical errors in the created test scripts

Deployment costs—an expensive solution with regard to

effort to deploy and system landscape requirements

Brittle scripts—Once created, scripts are hard to maintain

because even a small change breaks them

Developing scripts using test automation tools requires

scarce and expensive engineering talent



- 5 -

THE STATE OF TESTING TODAYAccording to the 2011 SAP Testing Survey, only about 26% of organizations have implemented

automated testing of their SAP systems. Even at organizations that have adopted such tools,

automated tests still do a fraction of the work. Those organizations using automated testing

tools automate only about 20% of their testing processes, meaning that the bulk of testing is

still done manually—organizations write test scripts and keep track of results using Word

documents and Excel spreadsheets.

Even more telling with regard to testing is the apparent lack of satisfaction with the solutions.

Nearly half of the respondents to the survey are unhappy with their current situation; of the

26% who have tried automated solutions, 15% are not satisfied with what they've achieved using

them. Of the 74% who have yet to automate testing, about 34% plan to do so.

With nearly half unhappy with their current situation, there is a need for a new take on the

very old problem of testing SAP landscapes.

THE STATE OF TESTING AUTOMATION (SOURCE: 2011 SAP TESTING SURVEY)



- 6 -

THIRD GENERATION: INTELLIGENT TESTING FOR SAP SYSTEMSThe third generation of SAP testing is a software-as-a-service (SaaS) testing solution designed

specifically for SAP. First, this solution does an impact analysis to show you what areas of the

system are impacted by the changes and where to focus testing. Then, business users or

functional experts generate testing scripts on the fly by running the business process once on

their actual system. From then on, anyone can automatically run the test scenarios whenever

needed. The generated test scripts can be executed automatically and they self-adjust, based

on test results, with no need to re-record. Alongside the automated script, a natural-language

step-by-step document is created, which is useful for management, reviews, and manual

execution if needed. As a result, customers can reduce the effort associated with running SAP

ERP tests by up to 80%.

PROS AND CONS OF INTELLIGENT TESTINGPROS CONS

Reduced time for testing staff—up to 80% less time

spent testing

Reduced time-to-market for SAP changes of all

types (days instead of months)

Software-as-a-Service—no hardware or software to

install—set-up in 20 minutes

Cloud-based library stores 70,000+ test scripts, and

adds more every day

Regenerative learning—learns from and logs errors;

skips them on the next run

Simple and intuitive—business users can record

their daily activities, then automate test scripts

Organizations with an investment in automated

testing might find it challenging to justify a second

investment



- 7 -

THE PANAYA PROPOSITIONPanaya is the leading third-generation SAP testing solution.

Panaya‟s cloud approach to SAP testing is different. Instead of weeks or months to begin

testing, Panaya can be up and running in 20 minutes.

Panaya is an intelligent testing system that combines the human intelligence from manual

testing with the efficiency and adaptability of computers. Panaya‟s customers perform their

manual tests only once. Panaya records their keystrokes as they work through business

processes and creates scripts. The next time that they want to test the process, the scripts run

automatically.

Panaya creates both manual and automated test scenarios in minutes. You don‟t need testautomation engineers on staff to replicate and test your business processes. The money you

save will come from spending less on dedicated testing staff. The time you save will reduce the

time-to-market for the next iteration of your SAP ERP application. And finally, the increased

test coverage improves the quality of your SAP ERP application and provides additional peace of

mind.

Here are a few of the value propositions of Panaya.

BUSINESS CLASSThe level of effort required to set up most testing scripts is beyond the reach of the business

user and requires dedicated programmers. Panaya is different. With Panaya, business users—the

people who use SAP every day—can start recording and testing within minutes. The key here is

that the business user doesn‟t feel like they‟re preparing tests because they are simply using

the ERP application, just like they always do.

SAAS = SIMPLICITY, SECURITY, AND STABILITYPanaya is delivered as a SaaS solution. That means no machines to host, no dedicated

administrators, and no installation routines. Screens are simple and intuitive. Setup takes about

20 minutes.

Creation and execution of tests is always initiated and controlled by the customer; Panaya

doesn‟t have any access to the customer landscape or to customer data. Panaya runs on

Amazon.com‟s EC2 cloud, which has the strongest security certification of any cloud offering

available. Even FDA-regulated companies and financial institutions with strict security

requirements use Panaya.



- 8 -

THE WISDOM OF THE CROWDThere are more than 20,000 companies using SAP. Before rolling out changes, they need to

generate regression-testing scripts and perform regression testing. Because there are many

shared processes across SAP instances, many of the scripts generated at individual installations

are redundant. So why spend time creating them?

At Panaya, we have analyzed more than 2,500 SAP instances and have created, stored, and

catalogued more than 70,000 test scripts. Our algorithm uses the wisdom of the crowd, more

thoroughly automating your scripts than any automation test engineer who could possibly

work for you.

Think of Panaya as the Google Trends of ERP testing. The entire SAP community benefits from aknowledge base that is more powerful than any one company could create.

REGENERATIVE LEARNING AND SELF-HEALINGNo test run is likely to be perfect, but every glitch is a learning opportunity. Panaya can learn

from errors, and avoid repeating them the next time. Here are three common scenarios where

Panaya excels and other solutions fall short.

SMART PARAMETERIZATION

Suppose you are creating a purchase order and then use this purchase order for goods receipt.SAP always generates unique purchase order numbers, so every time you run this test, you will

have to use the order number you received while running the test. Manual testers simply take

notes of the order number they received and enter it manually. Test automation engineers must

have the business knowledge and do some fair amount of programming to get such processes

working, parameterizing the purchase order number. Panaya automatically knows to pass

document numbers exactly as you would expect it to work without additional effort on anyone's

part.

CROWD-BASED WORKAROUNDS

Test recordings naturally cover only one specific case. When running the test, the system maybehave slightly differently. This is where the cloud can work for you. Panaya‟s software already

knows how other customers reacted in the very same situations. So when unexpected behavior

occurs, Panaya automatically looks up a way to handle it via the community knowledge base

and keeps running. If there is no workaround available, it switches to manual mode to let the

user deal with the unexpected situation.



- 9 -

For example, if the user name field content is different from the recording, Panaya knows

that‟s not a problem. However, if you suddenly have a different item category, it will alert you

of a potential error.

TESTING AT THE PUSH OF A BUTTON

There may be no better example of Panaya‟s advantage than the following diagram. This

diagram represents the difference between Panaya and a generic testing solution. We‟ve

collapsed some of the sub-tasks for clarity, but think you‟ll get the picture.

To capture a business process and run a test script in a general-purpose test-automation tool, a

test-automation expert must negotiate his way through 32 steps and the business user must be

involved in 11 of those steps. With Panaya‟s SAP-specific testing solution, the same process canbe automated, by business user alone, in just two steps: record and play.

PANAYA STREAMLINES TESTING

“AN INCH WIDE AND A MILE DEEP” Generic testing solutions are made to work on every Windows application in the enterprise;

they are a “mile wide and an inch deep.” Panaya was built using domain-specific knowledge,

explicitly for SAP ERP systems—we‟re “an inch wide and a mile deep.” Our solution brings a new

depth to testing software—the way people actually work on SAP.



- 10 -

DOMAIN AWARE: INHERENTLY UNDERSTANDS SAP PROCESSES

Let‟s go back to that purchase order for a minute. Suppose you are attempting to test the

integrity of a purchase order process in SAP. With a generic testing solution, you would write along script that would run the same process, with allowances for every expected amount

($10,000 increments, for example). Now, suppose there are certain business rules that must

occur in order to accept certain numerical increments: If an order is more than $10,000, it

should be approved by the site manager; if an order is more than $20,000, it needs the regional

manager‟s approval. In a generic testing solution, you would need to write a highly complex

branch script to cover these business cases. If the approval protocol changes, you have to

change the script. That‟s a lot of “if‟s.”

With Panaya, you quickly record a short script for each approval condition—so changing the

protocol for any one increment won‟t break the entire order processing routine.

CONCLUSION: FROM QUANDARY TOQUALITY Panaya is a high-quality, low-impact, easy-to-use ERP testing solution. Manual testing is time

consuming and an ineffective use of resources. Homegrown and generic commercial automated

testing is expensive and difficult to set up. With Panaya, the intelligent testing solution for SAP,

you don‟t have to choose between time-to-market, cost reduction, and risk reduction. You can:

Reduce the effort and duration of regression tests by a factor of 10Increase the number of useful scripts by a factor of 10Commit internal IT staff to higher-value projectsLet SAP users test their own functionsIntroduce changes more frequently, while maintaining or reducing risk levelsTest 10 times a year, rather than two

By using Panaya to test your SAP ERP system, you can be more innovative, agile, and meetchanging business conditions head-on. By helping you move faster, Panaya speeds up business

and moves testing from a roadblock to a competitive advantage.



- 11 -

DISCLAIMER AND TRADEMARK NOTICESThis report is provided by Panaya Ltd. It is completely independent of and not affiliated with

Oracle or SAP AG. Oracle is a registered trademark of Oracle. Oracle and other Oracle products

and services mentioned herein as well as their respective logos are trademarks or registered

trademarks of Oracle. SAP is a registered trademark of SAP AG. SAP and other SAP products and

services mentioned herein as well as their respective logos are trademarks or registered

trademarks of SAP AG in Germany and in several other countries. All other product and service

names mentioned are the trademarks of their respective companies.

DISCLAIMER OF WARRANTY

Panaya Ltd. makes no representation or warranties, either express or implied by or with respect

to anything in this document, and shall not be liable for any implied warranties of

merchantability or fitness for a particular purpose or for any indirect special or consequential

damages.

COPYRIGHT NOTICE

No part of this publication may be reproduced, stored in a retrieval system or transmitted, in

any form or by any means, photocopying, recording or otherwise, without prior written consent

of Panaya Inc. No patent liability is assumed with respect to the use of the information

contained herein. While every precaution has been taken in the preparation of this publication,

Panaya Inc. assumes no responsibility for errors or omissions. This publication is subject to

change without notice.

© 2012 Panaya Ltd. All rights reserved.

ABOUT PANAYAPanaya‟s software-as-a-service helps companies that use SAP or Oracle reduce 80% of their

upgrade and testing risk and effort. Utilizing a cloud-based supercomputer, Panaya simulates

the upcoming upgrade, automatically pinpointing which custom programs will break as a result

of the upgrade and automatically fixing most of these problems.

Panaya‟s testing solutions dramatically expedite ERP testing and eliminate the need for manual

test script maintenance. Seamlessly capturing business knowledge in the background, as users

work with the ERP applications, Panaya automatically generates plain-English test scripts that

are rapidly executed and continually self-adjust based on test results.

Date post:	05-Apr-2018
Category:	Documents
Upload:	amit-behera
View:	221 times
Download:	0 times

Faster SAP Testing Means Business Agility[1]

Documents