Post on 25-Dec-2015
transcript
| ©2011, Cognizant ©2011, Cognizant
Image Area6th Sept. 2011
AMEX Pega Testing – Moving towards TCoE
| ©2011, Cognizant ©2011, Cognizant
2
Cognizant’s Pega Testing footprint in AMEX
Why move to a TCoE?
Core-Flex model of Resourcing
Appendix
Case studies
Agenda
| ©2011, Cognizant | ©2011, Cognizant 3
Cognizant’s Pega Testing Footprint in Amex
| ©2011, Cognizant | ©2011, Cognizant
Cognizant’s footprint of PEGA engagements with AMEX
Delivered Engagements• ECM EMEA Release 1 & Release 2• ECM EMEA R1 & R2 Reporting• B2B RCubed• Global Merchant Services• Online Merchant Services• iCruse• ECM Disputes – CRs• GCM Acquirer - CRs
Ongoing Engagements• GCM GDN• System Assurance - UAT• iCruse – BAU• OMS Support• ECM Disputes – CRs• GCM Acquirer – CRs
Services provided• Functional Testing• Integration Testing• Report Testing• Test Data identification • Regression Testing• Automation Testing• Performance Testing• User Acceptance Testing• Business Acceptance Testing• E2E Test Management
What we noticed?
Different SDLC models followed : Iterative model followed in ECM, while B2B adopted SmartBPM then Iterative.
Differences in testing approach: Risk based testing followed in ECM
Automation testing : Adopted in ECM release 1, Benefits accrued in Release 2.
Comprehensive status reporting followed in ECM R1
Common Challenges: Requirements Management Change Management Test Data identification & preparation
| ©2011, Cognizant | ©2011, Cognizant 5
TCoE Evolution: Where are we today?
Business Unit
Business Unit
Business Unit
People People People
Process
Tools
Process Process
Tools Tools
Development / BA led Testing
Business Unit
Business Unit
Business Unit
People People People
Project led Independent Testing
Process
Tools
Business Unit
Business Unit
Business Unit
People People People
Automation
Shared Services Model
Performance
Tools
Process
QA/ Testing is part of Development
Independent QA for each project separately
QA/ Testing as a service
QA
cost
s not
track
ed
Cost
Tr
ack
ed &
M
anaged
Cost
of
Qualit
y
dri
ven T
CoE
No separate function for QA
QA is a separate function, but each project is treated
separately
QA is a separate function providing shared services
| ©2011, Cognizant | ©2011, Cognizant 6
Why move to a TCoE Model? Where do we want to be? Key Elements of a TCoE? What is different for Pega Testing?
Benefits of moving to a TCoE
| ©2011, Cognizant | ©2011, Cognizant
Why move to a TCoE model?
7
Software Quality Organization
• Varied methodologies, processes, tools, infrastructure and metrics across groups
• Little or no sharing of resources and knowledgebase; Resource management (on-boarding, ramp-up and release) processes have to be managed by each project
• Number of software licenses that can be used is typically lesser that what is needed, due to cost considerations; extensive use of Microsoft Excel
• Consistent methodologies, processes, tools, infrastructure and metrics across groups
• Centralised work management & resource planning in Core-Flex model
• Shared Infrastructure and tools. Maintenance and upgrade costs shared across projects
Decentralized Testing Organization
App 1
Development Team
Testers Testers Testers
Development Team
Development Team
Testing Tools Testing Tools Testing Tools
App 2 App N
ResourcesKnowledgeRepository
Common Tools &Infrastructure
StandardProcesses
StructuredSoftware Testing
Testing Center of Excellence
Centralized Testing Organization
The need for QA Centralization
| ©2011, Cognizant | ©2011, Cognizant 8
TCoE Evolution: Where do we want to be?
Business Unit
Business Unit
Business Unit
People People People
Process
Tools
Process Process
Tools Tools
Development / BA led Testing
Business Unit
Business Unit
Business Unit
People People People
Project led Independent Testing
Process
Tools
Business Unit
Business Unit
Business Unit
People People People
Automation
Shared Services Model
Performance
Tools
Process
QA/ Testing is part of Development
Independent QA for each project separately
QA/ Testing as a service
QA
cost
s not
track
ed
Cost
Tr
ack
ed &
M
anaged
Cost
of
Qualit
y
dri
ven T
CoE
No separate function for QA
QA is a separate function, but each project is treated
separately
QA is a separate function providing shared services
| ©2011, Cognizant | ©2011, Cognizant
Key Elements of a TCoE
9
Ce
ntr
aliz
ed
QA
Single test organization with defined governance model
Benchmarks for productivity and SLAs based on metrics gathered over time
Centralized resource management (Core-Flex team) for efficient demand management
Common processes for On-boarding, training & competency development
Standardized templates, guidelines and checklists across STLC
Framework based test automation that is used across projects
Centralized license management for testing tools
Test environment management
Centralized test data management
In-house product and technology specialists who serve as “go-to” gurus for all projects
| ©2011, Cognizant | ©2011, Cognizant
TCoE Ecosystem (Future State)
10
Test Planning & Estimation
Test Design &Build
Test Execution &Management
Test Reporting
SLA / Metrics Management
Test ProgramManagement
TestingCentre of Excellence
TCoE
Business
Development
Infrastructure Services
Business process definition
Provide/validate/signoff requirements
Recommendations to Change Control board
Environment Management
Test Data Management
Release & Configuration Management
Project Management Office
Strategic Planning & Governance
Budget Allocation
Overseeing TCoE SLA adherence
Inter group relationship of TCOE with external groups
Process Team*
Process Definition/ Maintenance
Metrics Institution
Test Process Training
Continuous Process Improvement
Best Practice Implementation
Tools standardization
Tools administration
Tools support
Automation/NFT*
Proof of concept
Standards & Guidelines
Feasibility Analysis
Technical Consultancy
Reusable Frameworks
Core-Flexresourcing
Standard process& Frameworks
Common Tools,Infrastructure & Automation
SolutionAccelerations/Best practices
Governancestructure
Application Development
Maintenance/ Enhancements
Defect Fixes
Cognizant
Test Tools COE*
Functional Testing Regression TestingPerformance /Load
Testing Automation
Testing UAT Support
Service Spectrum
AMEX
* Refer to Appendix for details about each team
| ©2011, Cognizant | ©2011, Cognizant
Pega TCoE – what is different for Pega, that makes it necessary to have a dedicated TCoE?
11
Ce
ntr
aliz
ed
TC
oE
for
Pe
ga
Available models include Waterfall, Iterative, Agile and Pega’s own Smart BPM approach. By standardizing the model, it is possible to optimize processes, tools and templates and derive benchmarks for reference
SDLC Model
Team composition In addition to Business Analysts, the team needs to include Pega specialists who understand Pega PRPC product and have prior experience in Pega Testing
Test Strategy Pega Testing is much more than UI based functionall testing. It is important to know how to test Rules and Workflows, how to test web-services, how to focus on specific flow paths for test execution during different stages of application development, how to rules that are data intensive, and how to use tools / utilities with Pega ie AUT, TMF, PAL etc
Tools & Automation For some needs, Pega’s own tools i.e AUT, TMF, PAL, PLA are recommended, whereas other tools like QTP and Cognizant’s proprietary tools like ADPART for Pega , CRAFT, TCGEN work better in other cases. Knowledge how each of these tools work is therefore critical
Change Management
It is common to find requirements evolving frequently in Pega projects. With changing requirements, it is necessary to identify changes in test scenarios immediately, and continuously maintain regression test scripts. Tools like ADPART for Pega can be used to deal with this challenge very effectively
| ©2011, Cognizant | ©2011, Cognizant
Benefits of moving to a TCoE model?
12
Testing is delivered as a Shared Service, i.e, Functional Testing, Automation Testing, Performance Testing across projects, thus reducing the cost of testing to each project
Common pool of experts is leveraged by all projects
Enables enterprise wide adoption of frameworks for Automation Testing – thus reducing cost of script maintenance due to product upgrades (ie Pega 5.5 to 6.2)
Having a common knowledge repository ensures each project team does not go through the same learning curve separately.
OptimisedResourcing
CentralRepository
Common Tools& Infrastructure
StandardProcesses
• Higher system quality
• Better planning & estimation
• Rigorous metrics collection
• Continuous process improvement
• Lower labor costs through optimised utilisation of resources
• Dedicated team builds expertise over time
• Reduced effort through reuse of common frameworks, templates, and data repositories
• Maximised test automation
• Optimized tool licensing requirements
• Institutionalise knowledge
• Better test coverage• Efficient knowledge
transfer
| ©2011, Cognizant | ©2011, Cognizant 13
The Core-flex model of Resourcing
Core-flex model Governance Structure
| ©2011, Cognizant | ©2011, Cognizant 14
Core Flex Resourcing Model
1. Staffing is done based on demand projections and average productivity observed on a quarterly basis
2. In case of ramp-up in Core team, flexi team resources at offshore would be moved to core, and new associates inducted in flexi team
| ©2011, Cognizant
Capacity Planning in the Core-Flex model
The capacity model is built on Fixed Requirements – i.e. core team of fixed number of resources having identified
skills
Flex team to support short term requirements for scaling up at short notice (typically up to 10% of core team size, provided core team size > 25)
Timely Demand Forecasting
Factoring a minimum lead time for ramp-ups (Ramp-up of Core team is done by moving resources from flex team, and replenishing the flex team within 6 to 8 weeks typically)
Floor and Ceiling Limits
15
Fixed Capacity1,680 hours per month
Forecasted demand based on 3 month rolling
forecast
Ceiling Limit(Forecasted demand
+ 10% of fixed capacity)
Floor Limit(90% of forecasted
demand)
| ©2011, Cognizant | ©2011, Cognizant
Core Flex Resourcing Model
16
TiersCapacity
Slab (Person hrs/ Month)
Equivalent FTE
slab
Resource Mix (Person hrs / Month)
Onsite/Offshore Ratio
Flex Team at offshore
ON OFF ON OFF OFF
1 2496 15 480 2016 19% 81% -
2 4168 25 640 3528 15% 85% 2
3 6680 40 800 5880 12% 88% 4
4 10032 60 960 9072 10% 90% 6
| ©2011, Cognizant | ©2011, Cognizant
Steady State TCoE Governance Structure
17
| ©2011, Cognizant | ©2011, Cognizant
TCoE – Cognizant/AMEX Roles & Responsibilities
18
Cognizant will assume delivery ownership and strategy/ planning/ execution/reporting for all testing activities done as part of the TCoE
AMEX will have ownership of SME support activities and Supplier Coordination
* Cognizant to partner with AMEX for formalizing strategies and help implement Vision, Policies and procedures and Budget allocation, as well as Release Planning, and Business Prioritization
** Cognizant to help AMEX with Environment Management and Configuration Management, by leveraging existing/proposed Cognizant presence in those areas
Program management, Risk management and Communication Management will be shared responsibilities
* Vision, Goals & Objectives
* Policies & Procedures
** Environment Management
* Release Planning
* Budget Management
Supplier Coordination
Business Analysis & SME
** Configuration Management
* Business Prioritization
Program Management Risk Management
Communication Management
Manage & Deploy Resources
Estimation
PMO Reporting KPI Tracking Performance Management
Test Strategy/Planning
Test Design/Execution Defect Management
AMEX
Cogn
izan
tSh
ared
Tool Administration
| ©2011, Cognizant | ©2011, Cognizant
Thank you
19
| ©2011, Cognizant | ©2011, Cognizant 20
Transition Approach: from current state to TCoE from incumbent team
| ©2011, Cognizant | ©2011, Cognizant
Typical TCoE Implementation Timeline*
21
Metric s
benchmarke
d
Pega Testing
as a service offered to BUs
Decentralized state
Define Basic Processes
Identify initial set of applications to be brought
under Pega TCoE
Knowledge Transition
Environment Set-up & access creation
Configuration of reusable assets
Execution and base-lining
Quarterly Demand forecasting
M0 M1 M2 M3 M4 M5 M6 M7 M36* Indicative sample only. Actual timeline will be formalized after more details are gathered
Consolidation at Enterprise Level
Transition from
incumbent team
Define guidelines and frameworks
| ©2011, Cognizant | ©2011, Cognizant
Vendor Transition - Overview
22
Cognizant
Incumbent
Entry criteria • SoW signed• High level KT Plan shared• KT team identified
• Draft KT document reviewed
• Tester logins created• Separate instance for
Cognizant testers
• All existing test cases executed at least once
• Test Management processes defined
Scope • Business Processes• Application & Interfaces• QA Environment• Test Mgnt processes
• Identified set of test scenarios (business critical)
• Testing of batch processes
• Full scope of application & Interfaces that are available in QA environment
Exit criteria • Draft KT document prepared by Cognizant
• Gaps (between application and test scripts) identified
• Metrics for test cases executed by Cognizant team
• Updated gap analysis document
• Ongoing assessment through agreed metrics and delivery review
How is it measured?
• No of topics covered, hours of KT
• Productivity• Defect Leakage• Any other metric
specifically agreed
Cognizant understands existing processes and application under the supervision of existing team
Cognizant will undertake execution of majority of test cases in steady-state
Test Execution by Cognizant team on trial to validate knowledge gained
Shadow Share Lead
| ©2011, Cognizant | ©2011, Cognizant
Appendix Case-studies
23
| ©2011, Cognizant | ©2011, Cognizant
Experience in setting TCoE for leading Industry Players
24
Healthcare
Customer ProfilePeak Team
Size
One of the largest U.S. health plan 450+
3rd largest health plan in the U.S. 350+
One of the largest Blues 180+
One of the largest clearing houses 100+
TCoEs
Life Sciences
Customer ProfilePeak Team
Size
One of the largest Pharma co. 250+
One of the oldest EU Pharma co. 50+
Insurance
Customer ProfilePeak Team
Size
A Fortune 100 insurance company 150+
The largest U.S insurance company 250+
U.S based Intl. fin. Servicers firm 100+
Leading fin. services product co. 150+
Top 3 fin services co. of U.S 200+
Banking & Financial Services
Customer ProfilePeak Team
Size
Leading UK based financial group 800+
One of the oldest fin. services firms in the world
350+
One of the largest banking and insurance group in UK
350+
Switzerland based fin. services Org 250+
Large U.S saving bank holding co. 100+
Communication, Media and Entertainment
Customer ProfilePeak Team
Size
Leading Telecom Equipment vendor 70+
Leading Broadband Service provider
30+
Large legal solutions and risk analytics company
250+
One of the world’s largest information co.
150+
Rich experience establishing large scale TCoEs across more than 30+ clients globally
Technology
Customer ProfilePeak
Team SizeOnline stock brokerage fin. service co.
120+
Largest Software Products co. 100+
Largest Engg Design Software Co.
70+
Retail, Travel & Manufacturing
Customer ProfilePeak Team
Size U.S based Internet travel company
150+
World's largest office supply retail store chain
100+
One of the largest Manufacturing conglomerates
100+
??
| ©2011, Cognizant | ©2011, Cognizant
Project Summary• Business Objective:
Implementation of a Credit Cards Dispute Management Application at the Acquirer end
Eliminate existing manual processes like case creation, case processing etc.
Automate processes by implementation of the Pega PRPC Enterprise Case Management system
• Testing carried out in various business centers and Markets.
• 2 cycles of Testing in 6 months to perform confirmation and regression testing to ensure that the product meets requirements
• Used Quality Center 9.0 for Test management tool, QTP 9.2 for Automation and Load Runner 9.5 for Performance Testing
• Technology Stack:
• PEGA Rules based Testing• Functional Testing • Integration Testing • Regression testing • E2E Testing• UAT Support • Performance Testing
Key ModulesPRPC• SSO Agent Login• Get Work & Search Case• Retrieval Requests• Charge backs• Financial adjustments
Scope
I wanted to share with you all the very positive feedback we have received from the UAT testers. Big wins on two fronts: Testing Training – really engaging, useful, enjoyable; ECM System: really useable, easy to navigate, professional, barely need training its so easy. We had a bunch of seasoned users who are not easily impressed by things - to get this kind of feedback is a resounding success! WOW!!!!!!Manager, Strategic Project Implementation
Just wanted to say thank you to all the Team for their tremendous effort and getting this back on trackDirector, World Services Technologies
Automation Coverage
• ~40% of System Test Cases were automated - More than 50% reduction in Test Execution time
• Reusable automation framework resulting in ~30% reduction in script creation effort
Client Benefits
Multi Market Testing
• Airlines• British Airways• Lufthansa• Pay pal• Highways
Team Composition
• 1 Onsite & 6 Offshore Test Analysts• Specialized PEGA Testing team
comprising of Manual, Automation and Performance Test Analysts
• Automation of 90% of the manual Credit Card dispute processes using PRPC
• Significant reduction in case processing time• 76% of defects were identified before UAT, ensuring
stability of the application• 0% production defects• 18% defects were raised in the Rules Testing phase
PRPC V5.5 SP1 IBM Web-sphere Portal V6.x.x, JDK 1.4.2, IBM DB2 database
Applause
Integration with• GC&S (Case creation)• OpsNet & Towerscan (Image storage)• Unitech
(Case validation)
PRPC Testing – Global Financial Services Company
25
| ©2011, Cognizant | ©2011, Cognizant
No
of T
est C
ases
SSO Login
Get Work
Search Case
Search Resu
lts
0
5
10
15
6.7971.98400000000
0022.094
11.50
10
20
30
40
50
60
High Medium Low Info
23
5342
12
0 200 400 600 800 1000 1200 1400
PRPC functional
GC&S
OpsNet / Towerscan
Unitech
Financial Adj.
1382107
59
3
3
Enh
ance
d Te
st
Cove
rage
Increased
Productivity
Performance Testing
Defe
ct D
etec
tion
Efficie
ncy-
Def
ects
by P
riorit
y
Automation coverage
Cycle 1 Cycle 20
500
1000
1500
AutomationManual
Manual versus Automation Coverage
Optimized Delivery
1/11/2
010
2/11/2
010
3/11/2
010
4/11/2
010
5/11/2
0100
400
800
1200
1600
Time in secs
Test Execution Productivity
Defect Detection
Efficiency- Defects by
Severity
Defects by SeverityDefects by Priority
23
57
50
High Medium Low
1554 test cases designed and executed across various modules
76% defects identified before UAT
26
PRPC Testing – Global Financial Services Co. …contd.
| ©2011, Cognizant
Testing Centre of Excellence - Large Financial Services Provider
27
Cognizant Solution
Cognizant performed a
strategic assessment of the
client’s testing organization
and established a dedicated
Testing Center of Excellence
(TCoE) which encompasses
resources from both the
client and Cognizant jointly
addressing the testing
needs of the client.
The client identified the need for a dedicated Testing Center of Excellence
(TCoE) for catering to the testing needs of all its IT systems and
applications as well as to centralize its testing processes and inculcate
best quality practices across the organization.
Background
Scope of the TCoE
Project Highlights
Effort Details
• 2 Years elapsed time &
Ongoing
• Peak team size: 147
Onsite – 31
Offshore – 116
Key LoB
• Customer Ops
• Employee Benefits
Applications Tested
• Microsoft CRM
• Policy Admin System
• Mainframe and Web
Applications
| ©2011, Cognizant
Testing Centre of Excellence - Large Financial Services Provider
28
Productivity: Automation resulted in over 90% of effort
and over 80% of cost savings
Resourcing: Established core + Flex staffing model which
enabled staff ramp up to 63% in 2 weeks
Quality: High test case coverage and continuous process
improvements ensured a very high quality of deliverables
Balance Scorecard: Establishment of a Balance
Scorecard with Key Performance Indicators (KPIs)
covering Budget, timely delivery, Quality and CSAT.
Risk Based Testing (RBT) : Cognizant has also
developed a Risk based testing model for the client to
reduce the overall testing cost by optimizing the number of
test cases and reducing the testing cycle time.
Client Benefits Cost Savings through Automation
Risk Based Testing Approach
| ©2011, Cognizant
Independent Branch
New feature on independent branch
Release VersionsTargeted Release
Unit Test -> QA approval for merge
Enhancement to existing feature cut from branch
Uprev’s , synch up’s btw branches
Final Merge, ownership transfer to Main
Merge with Main Trunk, transfer of ownership to client
Merge
• A Branch is cut from main trunk (code which is in prod)
• New feature is developed on independent branch
• QA signoff/ release before an enhancement/ feature can be merged with the Branch
• After promotion to trunk, regression test done on trunk
• Minimum Acceptance Tests after regression test
Branch & Merge Strategy
Test Planning in Agile model for an e-trading leader
29