| ©2011, Cognizant ©2011, Cognizant Image Area 6 th Sept. 2011 AMEX Pega Testing – Moving...

| ©2011, Cognizant ©2011, Cognizant

Image Area6th Sept. 2011

AMEX Pega Testing – Moving towards TCoE

| ©2011, Cognizant ©2011, Cognizant

2

Cognizant’s Pega Testing footprint in AMEX

Why move to a TCoE?

Core-Flex model of Resourcing

Appendix

Case studies

Agenda

| ©2011, Cognizant | ©2011, Cognizant 3

Cognizant’s Pega Testing Footprint in Amex

| ©2011, Cognizant | ©2011, Cognizant

Cognizant’s footprint of PEGA engagements with AMEX

Delivered Engagements• ECM EMEA Release 1 & Release 2• ECM EMEA R1 & R2 Reporting• B2B RCubed• Global Merchant Services• Online Merchant Services• iCruse• ECM Disputes – CRs• GCM Acquirer - CRs

Ongoing Engagements• GCM GDN• System Assurance - UAT• iCruse – BAU• OMS Support• ECM Disputes – CRs• GCM Acquirer – CRs

Services provided• Functional Testing• Integration Testing• Report Testing• Test Data identification • Regression Testing• Automation Testing• Performance Testing• User Acceptance Testing• Business Acceptance Testing• E2E Test Management

What we noticed?

Different SDLC models followed : Iterative model followed in ECM, while B2B adopted SmartBPM then Iterative.

Differences in testing approach: Risk based testing followed in ECM

Automation testing : Adopted in ECM release 1, Benefits accrued in Release 2.

Comprehensive status reporting followed in ECM R1

Common Challenges: Requirements Management Change Management Test Data identification & preparation


TCoE Evolution: Where are we today?

Business Unit

Business Unit

Business Unit

People People People

Process

Tools

Process Process

Tools Tools

Development / BA led Testing

Business Unit

Business Unit

Business Unit


Project led Independent Testing

Process

Tools

Business Unit

Business Unit

Business Unit


Automation

Shared Services Model

Performance

Tools

Process

QA/ Testing is part of Development

Independent QA for each project separately

QA/ Testing as a service

QA

cost

s not

track

ed

Cost

Tr

ack

ed &

M

anaged

Cost

of

Qualit

y

dri

ven T

CoE

No separate function for QA

QA is a separate function, but each project is treated

separately

QA is a separate function providing shared services


Why move to a TCoE Model? Where do we want to be? Key Elements of a TCoE? What is different for Pega Testing?

Benefits of moving to a TCoE


Why move to a TCoE model?

7

Software Quality Organization

• Varied methodologies, processes, tools, infrastructure and metrics across groups

• Little or no sharing of resources and knowledgebase; Resource management (on-boarding, ramp-up and release) processes have to be managed by each project

• Number of software licenses that can be used is typically lesser that what is needed, due to cost considerations; extensive use of Microsoft Excel

• Consistent methodologies, processes, tools, infrastructure and metrics across groups

• Centralised work management & resource planning in Core-Flex model

• Shared Infrastructure and tools. Maintenance and upgrade costs shared across projects

Decentralized Testing Organization

App 1

Development Team

Testers Testers Testers

Development Team

Development Team

Testing Tools Testing Tools Testing Tools

App 2 App N

ResourcesKnowledgeRepository

Common Tools &Infrastructure

StandardProcesses

StructuredSoftware Testing

Testing Center of Excellence

Centralized Testing Organization

The need for QA Centralization


TCoE Evolution: Where do we want to be?

Business Unit

Business Unit

Business Unit


Process

Tools

Process Process

Tools Tools

Development / BA led Testing

Business Unit

Business Unit

Business Unit


Project led Independent Testing

Process

Tools

Business Unit

Business Unit

Business Unit


Automation

Shared Services Model

Performance

Tools

Process

QA/ Testing is part of Development

Independent QA for each project separately

QA/ Testing as a service

QA

cost

s not

track

ed

Cost

Tr

ack

ed &

M

anaged

Cost

of

Qualit

y

dri

ven T

CoE

No separate function for QA

QA is a separate function, but each project is treated

separately

QA is a separate function providing shared services


Key Elements of a TCoE

9

Ce

ntr

aliz

ed

QA

Single test organization with defined governance model

Benchmarks for productivity and SLAs based on metrics gathered over time

Centralized resource management (Core-Flex team) for efficient demand management

Common processes for On-boarding, training & competency development

Standardized templates, guidelines and checklists across STLC

Framework based test automation that is used across projects

Centralized license management for testing tools

Test environment management

Centralized test data management

In-house product and technology specialists who serve as “go-to” gurus for all projects


TCoE Ecosystem (Future State)

10

Test Planning & Estimation

Test Design &Build

Test Execution &Management

Test Reporting

SLA / Metrics Management

Test ProgramManagement

TestingCentre of Excellence

TCoE

Business

Development

Infrastructure Services

Business process definition

Provide/validate/signoff requirements

Recommendations to Change Control board

Environment Management

Test Data Management

Release & Configuration Management

Project Management Office

Strategic Planning & Governance

Budget Allocation

Overseeing TCoE SLA adherence

Inter group relationship of TCOE with external groups

Process Team*

Process Definition/ Maintenance

Metrics Institution

Test Process Training

Continuous Process Improvement

Best Practice Implementation

Tools standardization

Tools administration

Tools support

Automation/NFT*

Proof of concept

Standards & Guidelines

Feasibility Analysis

Technical Consultancy

Reusable Frameworks

Core-Flexresourcing

Standard process& Frameworks

Common Tools,Infrastructure & Automation

SolutionAccelerations/Best practices

Governancestructure

Application Development

Maintenance/ Enhancements

Defect Fixes

Cognizant

Test Tools COE*

Functional Testing Regression TestingPerformance /Load

Testing Automation

Testing UAT Support

Service Spectrum

AMEX

* Refer to Appendix for details about each team


Pega TCoE – what is different for Pega, that makes it necessary to have a dedicated TCoE?

11

Ce

ntr

aliz

ed

TC

oE

for

Pe

ga

Available models include Waterfall, Iterative, Agile and Pega’s own Smart BPM approach. By standardizing the model, it is possible to optimize processes, tools and templates and derive benchmarks for reference

SDLC Model

Team composition In addition to Business Analysts, the team needs to include Pega specialists who understand Pega PRPC product and have prior experience in Pega Testing

Test Strategy Pega Testing is much more than UI based functionall testing. It is important to know how to test Rules and Workflows, how to test web-services, how to focus on specific flow paths for test execution during different stages of application development, how to rules that are data intensive, and how to use tools / utilities with Pega ie AUT, TMF, PAL etc

Tools & Automation For some needs, Pega’s own tools i.e AUT, TMF, PAL, PLA are recommended, whereas other tools like QTP and Cognizant’s proprietary tools like ADPART for Pega , CRAFT, TCGEN work better in other cases. Knowledge how each of these tools work is therefore critical

Change Management

It is common to find requirements evolving frequently in Pega projects. With changing requirements, it is necessary to identify changes in test scenarios immediately, and continuously maintain regression test scripts. Tools like ADPART for Pega can be used to deal with this challenge very effectively


Benefits of moving to a TCoE model?

12

Testing is delivered as a Shared Service, i.e, Functional Testing, Automation Testing, Performance Testing across projects, thus reducing the cost of testing to each project

Common pool of experts is leveraged by all projects

Enables enterprise wide adoption of frameworks for Automation Testing – thus reducing cost of script maintenance due to product upgrades (ie Pega 5.5 to 6.2)

Having a common knowledge repository ensures each project team does not go through the same learning curve separately.

OptimisedResourcing

CentralRepository

Common Tools& Infrastructure

StandardProcesses

• Higher system quality

• Better planning & estimation

• Rigorous metrics collection

• Continuous process improvement

• Lower labor costs through optimised utilisation of resources

• Dedicated team builds expertise over time

• Reduced effort through reuse of common frameworks, templates, and data repositories

• Maximised test automation

• Optimized tool licensing requirements

• Institutionalise knowledge

• Better test coverage• Efficient knowledge

transfer


The Core-flex model of Resourcing

Core-flex model Governance Structure


Core Flex Resourcing Model

1. Staffing is done based on demand projections and average productivity observed on a quarterly basis

2. In case of ramp-up in Core team, flexi team resources at offshore would be moved to core, and new associates inducted in flexi team

| ©2011, Cognizant

Capacity Planning in the Core-Flex model

The capacity model is built on Fixed Requirements – i.e. core team of fixed number of resources having identified

skills

Flex team to support short term requirements for scaling up at short notice (typically up to 10% of core team size, provided core team size > 25)

Timely Demand Forecasting

Factoring a minimum lead time for ramp-ups (Ramp-up of Core team is done by moving resources from flex team, and replenishing the flex team within 6 to 8 weeks typically)

Floor and Ceiling Limits

15

Fixed Capacity1,680 hours per month

Forecasted demand based on 3 month rolling

forecast

Ceiling Limit(Forecasted demand

+ 10% of fixed capacity)

Floor Limit(90% of forecasted

demand)


Core Flex Resourcing Model

16

TiersCapacity

Slab (Person hrs/ Month)

Equivalent FTE

slab

Resource Mix (Person hrs / Month)

Onsite/Offshore Ratio

Flex Team at offshore

ON OFF ON OFF OFF

1 2496 15 480 2016 19% 81% -

2 4168 25 640 3528 15% 85% 2

3 6680 40 800 5880 12% 88% 4

4 10032 60 960 9072 10% 90% 6


Steady State TCoE Governance Structure

17


TCoE – Cognizant/AMEX Roles & Responsibilities

18

Cognizant will assume delivery ownership and strategy/ planning/ execution/reporting for all testing activities done as part of the TCoE

AMEX will have ownership of SME support activities and Supplier Coordination

* Cognizant to partner with AMEX for formalizing strategies and help implement Vision, Policies and procedures and Budget allocation, as well as Release Planning, and Business Prioritization

** Cognizant to help AMEX with Environment Management and Configuration Management, by leveraging existing/proposed Cognizant presence in those areas

Program management, Risk management and Communication Management will be shared responsibilities

* Vision, Goals & Objectives

* Policies & Procedures

** Environment Management

* Release Planning

* Budget Management

Supplier Coordination

Business Analysis & SME

** Configuration Management

* Business Prioritization

Program Management Risk Management

Communication Management

Manage & Deploy Resources

Estimation

PMO Reporting KPI Tracking Performance Management

Test Strategy/Planning

Test Design/Execution Defect Management

AMEX

Cogn

izan

tSh

ared

Tool Administration


Thank you

19


Transition Approach: from current state to TCoE from incumbent team


Typical TCoE Implementation Timeline*

21

Metric s

benchmarke

d

Pega Testing

as a service offered to BUs

Decentralized state

Define Basic Processes

Identify initial set of applications to be brought

under Pega TCoE

Knowledge Transition

Environment Set-up & access creation

Configuration of reusable assets

Execution and base-lining

Quarterly Demand forecasting

M0 M1 M2 M3 M4 M5 M6 M7 M36* Indicative sample only. Actual timeline will be formalized after more details are gathered

Consolidation at Enterprise Level

Transition from

incumbent team

Define guidelines and frameworks


Vendor Transition - Overview

22

Cognizant

Incumbent

Entry criteria • SoW signed• High level KT Plan shared• KT team identified

• Draft KT document reviewed

• Tester logins created• Separate instance for

Cognizant testers

• All existing test cases executed at least once

• Test Management processes defined

Scope • Business Processes• Application & Interfaces• QA Environment• Test Mgnt processes

• Identified set of test scenarios (business critical)

• Testing of batch processes

• Full scope of application & Interfaces that are available in QA environment

Exit criteria • Draft KT document prepared by Cognizant

• Gaps (between application and test scripts) identified

• Metrics for test cases executed by Cognizant team

• Updated gap analysis document

• Ongoing assessment through agreed metrics and delivery review

How is it measured?

• No of topics covered, hours of KT

• Productivity• Defect Leakage• Any other metric

specifically agreed

Cognizant understands existing processes and application under the supervision of existing team

Cognizant will undertake execution of majority of test cases in steady-state

Test Execution by Cognizant team on trial to validate knowledge gained

Shadow Share Lead


Appendix Case-studies

23


Experience in setting TCoE for leading Industry Players

24

Healthcare

Customer ProfilePeak Team

Size

One of the largest U.S. health plan 450+

3rd largest health plan in the U.S. 350+

One of the largest Blues 180+

One of the largest clearing houses 100+

TCoEs

Life Sciences


Size

One of the largest Pharma co. 250+

One of the oldest EU Pharma co. 50+

Insurance


Size

A Fortune 100 insurance company 150+

The largest U.S insurance company 250+

U.S based Intl. fin. Servicers firm 100+

Leading fin. services product co. 150+

Top 3 fin services co. of U.S 200+

Banking & Financial Services


Size

Leading UK based financial group 800+

One of the oldest fin. services firms in the world

350+

One of the largest banking and insurance group in UK

350+

Switzerland based fin. services Org 250+

Large U.S saving bank holding co. 100+

Communication, Media and Entertainment


Size

Leading Telecom Equipment vendor 70+

Leading Broadband Service provider

30+

Large legal solutions and risk analytics company

250+

One of the world’s largest information co.

150+

Rich experience establishing large scale TCoEs across more than 30+ clients globally

Technology

Customer ProfilePeak

Team SizeOnline stock brokerage fin. service co.

120+

Largest Software Products co. 100+

Largest Engg Design Software Co.

70+

Retail, Travel & Manufacturing


Size U.S based Internet travel company

150+

World's largest office supply retail store chain

100+

One of the largest Manufacturing conglomerates

100+

??


Project Summary• Business Objective:

Implementation of a Credit Cards Dispute Management Application at the Acquirer end

Eliminate existing manual processes like case creation, case processing etc.

Automate processes by implementation of the Pega PRPC Enterprise Case Management system

• Testing carried out in various business centers and Markets.

• 2 cycles of Testing in 6 months to perform confirmation and regression testing to ensure that the product meets requirements

• Used Quality Center 9.0 for Test management tool, QTP 9.2 for Automation and Load Runner 9.5 for Performance Testing

• Technology Stack:

• PEGA Rules based Testing• Functional Testing • Integration Testing • Regression testing • E2E Testing• UAT Support • Performance Testing

Key ModulesPRPC• SSO Agent Login• Get Work & Search Case• Retrieval Requests• Charge backs• Financial adjustments

Scope

I wanted to share with you all the very positive feedback we have received from the UAT testers. Big wins on two fronts: Testing Training – really engaging, useful, enjoyable; ECM System: really useable, easy to navigate, professional, barely need training its so easy. We had a bunch of seasoned users who are not easily impressed by things - to get this kind of feedback is a resounding success! WOW!!!!!!Manager, Strategic Project Implementation

Just wanted to say thank you to all the Team for their tremendous effort and getting this back on trackDirector, World Services Technologies

Automation Coverage

• ~40% of System Test Cases were automated - More than 50% reduction in Test Execution time

• Reusable automation framework resulting in ~30% reduction in script creation effort

Client Benefits

Multi Market Testing

• Airlines• British Airways• Lufthansa• Pay pal• Highways

Team Composition

• 1 Onsite & 6 Offshore Test Analysts• Specialized PEGA Testing team

comprising of Manual, Automation and Performance Test Analysts

• Automation of 90% of the manual Credit Card dispute processes using PRPC

• Significant reduction in case processing time• 76% of defects were identified before UAT, ensuring

stability of the application• 0% production defects• 18% defects were raised in the Rules Testing phase

PRPC V5.5 SP1 IBM Web-sphere Portal V6.x.x, JDK 1.4.2, IBM DB2 database

Applause

Integration with• GC&S (Case creation)• OpsNet & Towerscan (Image storage)• Unitech

(Case validation)

PRPC Testing – Global Financial Services Company

25


No

of T

est C

ases

SSO Login

Get Work

Search Case

Search Resu

lts

0

5

10

15

6.7971.98400000000

0022.094

11.50

10

20

30

40

50

60

High Medium Low Info

23

5342

12

0 200 400 600 800 1000 1200 1400

PRPC functional

GC&S

OpsNet / Towerscan

Unitech

Financial Adj.

1382107

59

3

3

Enh

ance

d Te

st

Cove

rage

Increased

Productivity

Performance Testing

Defe

ct D

etec

tion

Efficie

ncy-

Def

ects

by P

riorit

y

Automation coverage

Cycle 1 Cycle 20

500

1000

1500

AutomationManual

Manual versus Automation Coverage

Optimized Delivery

1/11/2

010

2/11/2

010

3/11/2

010

4/11/2

010

5/11/2

0100

400

800

1200

1600

Time in secs

Test Execution Productivity

Defect Detection

Efficiency- Defects by

Severity

Defects by SeverityDefects by Priority

23

57

50

High Medium Low

1554 test cases designed and executed across various modules

76% defects identified before UAT

26

PRPC Testing – Global Financial Services Co. …contd.

| ©2011, Cognizant

Testing Centre of Excellence - Large Financial Services Provider

27

Cognizant Solution

Cognizant performed a

strategic assessment of the

client’s testing organization

and established a dedicated

Testing Center of Excellence

(TCoE) which encompasses

resources from both the

client and Cognizant jointly

addressing the testing

needs of the client.

The client identified the need for a dedicated Testing Center of Excellence

(TCoE) for catering to the testing needs of all its IT systems and

applications as well as to centralize its testing processes and inculcate

best quality practices across the organization.

Background

Scope of the TCoE

Project Highlights

Effort Details

• 2 Years elapsed time &

Ongoing

• Peak team size: 147

Onsite – 31

Offshore – 116

Key LoB

• Customer Ops

• Employee Benefits

Applications Tested

• Microsoft CRM

• Policy Admin System

• Mainframe and Web

Applications

| ©2011, Cognizant

Testing Centre of Excellence - Large Financial Services Provider

28

Productivity: Automation resulted in over 90% of effort

and over 80% of cost savings

Resourcing: Established core + Flex staffing model which

enabled staff ramp up to 63% in 2 weeks

Quality: High test case coverage and continuous process

improvements ensured a very high quality of deliverables

Balance Scorecard: Establishment of a Balance

Scorecard with Key Performance Indicators (KPIs)

covering Budget, timely delivery, Quality and CSAT.

Risk Based Testing (RBT) : Cognizant has also

developed a Risk based testing model for the client to

reduce the overall testing cost by optimizing the number of

test cases and reducing the testing cycle time.

Client Benefits Cost Savings through Automation

Risk Based Testing Approach

| ©2011, Cognizant

Independent Branch

New feature on independent branch

Release VersionsTargeted Release

Unit Test -> QA approval for merge

Enhancement to existing feature cut from branch

Uprev’s , synch up’s btw branches

Final Merge, ownership transfer to Main

Merge with Main Trunk, transfer of ownership to client

Merge

• A Branch is cut from main trunk (code which is in prod)

• New feature is developed on independent branch

• QA signoff/ release before an enhancement/ feature can be merged with the Branch

• After promotion to trunk, regression test done on trunk

• Minimum Acceptance Tests after regression test

Branch & Merge Strategy

Test Planning in Agile model for an e-trading leader

29

Date post:	25-Dec-2015
Category:	Documents
Upload:	mervyn-hutchinson
View:	225 times
Download:	6 times

| ©2011, Cognizant ©2011, Cognizant Image Area 6 th Sept. 2011 AMEX Pega Testing – Moving...

Documents