Date post: | 09-Jul-2015 |
Category: |
Software |
Upload: | erika-barron |
View: | 212 times |
Download: | 1 times |
Parasoft Proprietary and Confidential 1
2014-11-12
Service VirtualizationDelivering Complex Test Environments on Demand
Frank Jennings; Director, Performance Test & Engineering, TQM, Comcast
Mark Lambert; VP Products and Support, Parasoft Corporation
Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2
Drivers of the SDLC
SDLC Speed is the
difference between a
first mover and a
follower
Damages associated
with software failure
are increasing and
very real
Brand equity is critical
and Quality drives
brand loyalty
Parasoft Proprietary and Confidential 3Parasoft Proprietary and Confidential 3
Current Pressures of the SDLC
Adoption of Agile to meet Acceleration
Requires Continuous Infrastructure
Need to assess Business Risk of rapidly changing code
Needs Continuous Testing with automated Quality Gates
Unit Test is not enough
Testing complexity grows further into the SDLC
DevOps requires infrastructure to “shift left” On-demand Environment access
Parasoft Proprietary and Confidential 4Parasoft Proprietary and Confidential 4
Complexity is a Barrier to Innovation
3rd Party System
Evolving ComponentMainframe
Scheduled Access
Parasoft Proprietary and Confidential 5Parasoft Proprietary and Confidential 5
An application is not just “one application” …
Parasoft Proprietary and Confidential 6Parasoft Proprietary and Confidential 6
… it’s made up of „several components” …
Parasoft Proprietary and Confidential 7Parasoft Proprietary and Confidential 7
… and they are not all ready at the same time.
Parasoft Proprietary and Confidential 8Parasoft Proprietary and Confidential 8
How do we ensure quality of the components?
Static Analysis, Unit Testing, Code Review
Parasoft Proprietary and Confidential 9Parasoft Proprietary and Confidential 9
How to test without dependant components?
API testing
Reusable Virtual Assets
Parasoft Proprietary and Confidential 10Parasoft Proprietary and Confidential 10
Test as soon as the components are ready
Parasoft Proprietary and Confidential 11Parasoft Proprietary and Confidential 11
Service Virtualization delivers simulated application behavior
in dev / test environmentsallowing an organization to test
on-demand
Service Virtualization
Parasoft Proprietary and Confidential 12Parasoft Proprietary and Confidential 12
How to apply Service Virtualization
1. Identify the assets;
Which types of asset are good to simulate?
2. Model behaviour;
How much is enough?
3. Deploy and reuse;
Provide easy ad-hock access and sharing
4. Automate Provisioning;
Integrated into a Continuous Test Environment
Parasoft Proprietary and Confidential 13Parasoft Proprietary and Confidential 13
When to use SV? Access vs Configuration
Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications
Configuration Complex to configure No control Limited variability Consumes test time
Test Access
Simple Difficult
Configura
tion
Complex
Internal
Service
Internal
Application 3rd Party
Service
ERP
Mainframe
External
Application
External
Database
Internal
Database
Cloud
App
Message
Queue
ESB
Web
Server
Parasoft Proprietary and Confidential 14Parasoft Proprietary and Confidential 14
When to use SV? Access vs Configuration
Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications
Configuration Complex to configure No control Limited variability Consumes test time
Test Access
Simple Difficult
Configura
tion
Complex
Internal
Service
Internal
Application 3rd Party
Service
ERP
Mainframe
External
Application
External
Database
Internal
Database
Cloud
App
Message
Queue
ESB
Web
Server
Service Virtualization
Virtual Lab / Server
Virtualization
(i.e. VMWare)
Parasoft Proprietary and Confidential 15Parasoft Proprietary and Confidential 15
The Total Cost of System Access
Service Virtualization dramatically reduces the cost of
delivering a dev / test environment
Hardware Savings
Access EfficiencyStaged w/ Virtual Test Lab
Staged Application Instance
System Behavior
Cost
Only manage the functionality you need
Only manage data you need Low-Risk “disposable instance”
that can be accessed at anytime Dramatically reduce test
environment setup and management
Service Virtualization
Parasoft Proprietary and Confidential 16Parasoft Proprietary and Confidential 16
Setting up Service Virtualization
Define &
Capture
Initiated from the system under test,
the user has the ability to capture
detail from a live monitor that
analyzes system traffic, from
analyzing transaction logs or by
modeling virtual behavior within the
Parasoft Virtualized interface.
Model &
Deploy
After the virtualized artifact has been
captured, users can now instruct the
details of the virtualized asset
behavior. This includes: performance,
data sources and conditional response
criteria. The virtualized asset is then
provisioned for simplified uniform
access across teams and business
partners.
Provision &
Consume
The virtualized asset can now be called
for unit, functional and performance tests.
The virtualized asset can be leveraged by
any test suite – including Parasoft Test.
Parasoft Proprietary and Confidential 17Parasoft Proprietary and Confidential 17
Example: Capturing current behavior
ApplicationUnder Test
2 Capture
4 Deploy
3 Create
Application
Database
Service
QA and Test
Performance TestEngineer
Mainframe
Define Monitors1
Development
Service
Test
LoadRunner
Parasoft Proprietary and Confidential 18Parasoft Proprietary and Confidential 18
Provision the Environment on Demand
ApplicationUnder Test
Application
Database
Service
Mainframe
EnvironmentManagement
5 Manage
6 Consume
QA and Test
Performance TestEngineer
Development
QC/ALM
Rational
Continuous Integration
Parasoft Proprietary and Confidential 19Parasoft Proprietary and Confidential 19
Rapid Environment Access
Parasoft Proprietary and Confidential 20Parasoft Proprietary and Confidential 20
Reduced wait time for test team by 60% for a major media conglomerate
The Challenge
Large agile development effort to adopt Service Oriented Architecture (SOA)
High risk project but the Test team “stuck waiting for the first build”
Development of functionality was not easy to coordinate as different teams had different schedules; not all finished at the same time
Agile/Parallel development limited by system dependencies
Iteration
Iteration
Iteration
Team A
Team B
Team C
Iteration Iteration
Iteration Iteration Iteration
Iteration Iteration
Current
Development/Testing
dependencies
Parasoft Proprietary and Confidential 21Parasoft Proprietary and Confidential 21
Reduced wait time for test team by 60% for a major media conglomerate
The Solution
Use descriptions of the new services (WSDL, XSD, example JSON payloads) to build Virtual assets prototyping the new functionality.
Test team builds tests with against the prototypes with SOAtest and the independent development tests use the prototypes to perform early stage Integration Testing
Agile/Parallel development limited by system dependencies
Iteration
Iteration
Iteration
Team A
Team B
Team C
Iteration Iteration
Iteration Virtual Asset Iteration
Virtual Asset Iteration
Current
Development/Testing
dependencies
Parasoft Proprietary and Confidential 22Parasoft Proprietary and Confidential 22
Reduced wait time for test team by 60% for a major media conglomerate
The Business Benefits
Met business goals and timelines, were able to test functionality “as soon as” it was available. Practiced TDD against prototypes to get a head-start on ‘full system testing’
Agile/Parallel development limited by system dependencies
Iteration
Iteration
Iteration
Team A
Team B
Team C
Iteration Iteration
Iteration Iteration Iteration
Iteration Iteration
✔✖✔
✖✔✖
Parasoft Proprietary and Confidential 23Parasoft Proprietary and Confidential 23
Provisioning of test environments went from weeks to minutes from large government organization
The Challenge Test results did not reflect actual performance in production
Inability to test multiple projects at once
Testing environments were unstable and unreliable
Limited access to mainframes, ERPs, or 3rd party systems
Parasoft Proprietary and Confidential 24Parasoft Proprietary and Confidential 24
Provisioning of test environments went from weeks to minutes from large government organization
The Solution Combining Service Virtualization and VMWare technology to build a
testing environment that properly replicates the production environment while allowing multiple virtual copies of that environment to be provisioned on-demand.
Limited access to mainframes, ERPs, or 3rd party systems
PV
PV
PVPV
PV
PV
PV
PV
Parasoft Proprietary and Confidential 25Parasoft Proprietary and Confidential 25
Provisioning of test environments went from weeks to minutes from large government organization
The Business Benefit Developers empowered to provision their own test environments.
Provisioning delays reduced from weeks to minutes.
Project reliability enhanced.
Limited access to mainframes, ERPs, or 3rd party systems
&
Parasoft Proprietary and Confidential 26Parasoft Proprietary and Confidential 26
Reduced cost of creation and management of Test Environments by 85% for a major international bank
The Challenge
Complex test environments including; ATM, website and mobile front-end systems with WebSphere, Mainframe, MQ, SAP and DataPower back-end systems
Each test system costs >$2m, therefore only two environments available
Active development at every level of the system resulting in full end-to-end testing requiring careful coordination and sub-system teams often offline or unavailable due to upgrades
Database
SAP
Mainframe
Application
Mobile Application development and extension of current systems
Parasoft Proprietary and Confidential 27Parasoft Proprietary and Confidential 27
The Solution
Create a library of virtual assets for teams to share and use Environment Manager to replicate and manage Virtual test labs using different performance charactistics for different load scenarios.
Use automated test solution to test and validate at each layer independantly
EnvironmentManagement
Reduced cost of creation and management of Test Environments by 85% for a major international bank
Mobile Application development and extension of current systems
Database
SAP
Mainframe
Application
Virtual AssetVirtual
AssetVirtual Asset
Virtual AssetVirtual
Asset
Virtual Asset
Virtual AssetVirtual
AssetVirtual AssetVirtual
Asset
Virtual Asset Virtual
Asset
Virtual Asset
✔✖
Parasoft Proprietary and Confidential 28Parasoft Proprietary and Confidential 28
The Business Benefit
Improved efficiency, enabling Agile development, with the teams able to test in isolation without effecting full end-to-end system tests.
Reduced cost of creation and management of Test Environments by 85% for a major international bank
EnvironmentManagement
Mobile Application development and extension of current systems
✔✖
✔✖
✔✖
✔✖
Parasoft Proprietary and Confidential 29Parasoft Proprietary and Confidential 29
Comcast Case Study: Applying Service Virtualization to
Performance Testing
Parasoft Proprietary and Confidential 30Parasoft Proprietary and Confidential 30
Network Provising
Inventory Billing
Enterprise Services Portal
Location DB
Commercial Residential
Fron
tend
OSS / B
SSM
idd
leware
Ne
two
rk
Order Entry
CustomerCare
SelfService
Order EntryCustomer
CareSelf Service eCommerce
Voice Response
System
Commercial Services Platform
Ticketing
Inventory
Authentication
Offer MgmtCustomer &
AccountProvisioning Location CDV Services
Event Mgmt
Customer & Account
Activation
Order Fullfillment
Order Mgmt
Business Process Mgmt
Order Management
Billing
Self Install Process
UI Widget Warehouse
Knowledge Management
3rd PartyNetworkServices
eMTASTB
ModemsWifi
IMS
CMTSNetwork ElementStatus
Networks Services
TN Inventory Managment
E911 Services
Customer DB Offer DB
Performance Team Test CoverageTest TargetVirtualizedTest with Virtualize
Commercial DBs
Parasoft Proprietary and Confidential 31Parasoft Proprietary and Confidential 31
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 32Parasoft Proprietary and Confidential 32
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 33Parasoft Proprietary and Confidential 33
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 34Parasoft Proprietary and Confidential 34
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 35Parasoft Proprietary and Confidential 35
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 36Parasoft Proprietary and Confidential 36
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 37Parasoft Proprietary and Confidential 37
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 38Parasoft Proprietary and Confidential 38
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 39Parasoft Proprietary and Confidential 39
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 40Parasoft Proprietary and Confidential 40
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 41Parasoft Proprietary and Confidential 41
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 42Parasoft Proprietary and Confidential 42
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 43Parasoft Proprietary and Confidential 43
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 44Parasoft Proprietary and Confidential 44
System Under Test
Test Automation
Services & DBs
Virtualized Services
Real End Points
Development
Deployment Services Regression Performance Manual Test
QA Stage Production
Unit
Intelligent Proxy
DeploymentAutomation
Continuous Deployment Engine
Code
DeployDev
DeploymentValidation
Code Quality
Regression
BaselinePerformance
DeployQA
DeployStage
DeployProd
DeploymentValidation
Regression
FunctionalDeploymentValidation
PerformanceDeploymentValidation
Parasoft Proprietary and Confidential 45Parasoft Proprietary and Confidential 45
Service Virtualization
Emulates dependencies for the Test Environment
Reduces the complexity for early stage testing
Increases predictability
On-Demand “Test Anytime, Anywhere, Anyway”
Automated Provisioning for different use-cases
Automated Test Data Management/Simulation
Remember: Does not eliminate the need for System/Integration Testing
Parasoft Proprietary and Confidential 46
2014-11-12
Thank you
Questions?