From technology to solutions: Customer Centric Testing

From technology to solutions: Customer Centric Testing

@ Silicon India Software Testing Conference Hyderabad

March 26 2011

Ramesh LoganathanVP (Products) & Center head, progress software- Hyderabad

Familiar..?

In next 30 minutes..

Software consumption patterns are shifting. Enterprises are now increasingly looking for solutions to their biz problems, And not

just technology to address their software needs.

The biz problems are also undergoing paradigm shifts with problems spanning departments within and outside the

organizations.

The talk will deliberate on some of these challenges and approaches possible to address them.

© 2011 Progress Software Corporation. All rights reserved.4

Enterprise scenarios today

Continuously anticipate problematic events and re-prioritize to keep planes in the air and passengers happy

Investigate potentially fraudulent activity while it’s happening and prevent it from spreading

Dynamically up-sell based on customer activity, context and location

Business Strives for Operational Responsiveness…


Constantly Increasing Pressures for Responsiveness

Typical Business SLAs

Algorithmic trading100 ms 20 ms

Call center inquiries 8 hr 10 sec

Supply chain updates1 day 15 min

Phone activation1 min

0.5 hour Refresh data warehouse

Track financial position5 min1 day

Airline operations20 min 30 sec

Trade settlement2 hrs.

Build-to-order PC 4 weeks 1 day

weeks days hours minutes seconds

Responsiveness is the key to surviving the pace of competition

3 days

1 week

5 days

Source:

mail express fax e-mail Document transfer


At this speeds, one cant operate looking at the rear-view


Cannot afford not being prepared or quick

Visibility

Response

What is needed:

Operational

Responsiveness


Enterprises need to be Operationally Responsiveness

Continuously improve your business’ Operational

Responsiveness

Continuous Business Process

Improvement

Immediate Sense-and-Respond

Real-time Business Visibility


Challenge: Can’t Rip/Replace Critical Business Applications

Business improvement in mission-critical systems

Changing the engines on

an airliner while in flight=


Enterprises massively distributed today:Example- The Transportation & Logistics Value Chain

Sales & Outbound Processing

RoutingIntermodal Services

Billing

Collections

Planning &Dispatching

Customer Service

Claims Reconciliation

Carrier Determination

Planned Tenders

Invoicing Engines

Settlement

Customer

Domestic

Trade Services

International

Warehousing & Distribution

MRO

Carrier Optimization

Crew Mgt

GoodsMgt

HRMSContractServices

Asset Mgt

Warehouse Mgt

Project Mgt

Document Mgt

Event & Information Network


Processes & Integration everywhere

Non-intrusive monitoring of existing processesDoes not require re-automation of process

Order Entered

Credit Checked

OrderApproved

OrderDenied

ExistingProcess

ShadowProcess

TypicalBPM

ShadowProcess


CRM ERP

Logistics

Product catalog

SAPOracleSiebel

Warehouse status

SAP shipping

Automate Exception Management Workstep

Need to Continuously track & optimize


Event driven enterprise solutions

Examples:Fraud detectionCross-promotion


Aggravated by the cloud



Billing

Collections



Planned Tenders


Invoicing Engines

Settlement

Customer

Customer Service

Domestic

Trade Services

International


MRO


Crew Mgt

GoodsMgt


Warehouse Mgt

Asset Mgt

Project Mgt

Document Mgt

Business Process

Business processes that span people, systems and departments

www


www

The SOA enabled cloud- services everywhere!

virtualizationplatform

5

appservicesUI

enterpriseintranet

1

3

2

4

appservicesUI

appservicesUI

Users

access on

www

!Web based

Admin

www

AmazonEC2 S3s3

5

appservicesUI

SaaS

5

appservicesUI

hosted

virtualization SaaS

cloud

Google App engine

5

appservicesUI

PaaS

events

5

Event EngineBAM

…

Managed; Clear SLAs; SOA; Scaleable;

Virtualized; Dynamic provisioning; Location transparency

SOA. Web2.0. Dist Management

Serious challenge testing in this environment…

What does this mean for Testing & Quality Assurance?


Must be very reliable.. Out of the doorElse, with poor reliability can never ensure Operational Responsiveness

Not this kind of reliability though…


Customer centric testing

Test, Keeping customer more in focus Testing biz solutions as opposed to technology

New challenges to the software quality assurance process. Software QA organizations must be much more aware of these usecases and not just the

functionality defined in the functional specifications or the technology standards. Devise approaches that bring customer environments inhouse

into the automated testing suites & processes. Must stress and regress functionality. in more complex endues scenarios.


Beyond solution stacks.. Distributed solutions’ testing

Challenges: Multiple layers &

distributed Data Driven Testing (No

GUI) Difficulty in isolating bugs Lack of effective reporting

and debugging capabilities Integration &

interoperability Involvement of 3rd party

services Different authorization /

authentication approach Performance testing

problems


Stress the new features in an integrated enterprise

• Software Systems have to be very robust• Reliable and work as designed• Under stress and varied use conditions


Testing in & for the cloud

Testing a solution in the context of a cloud • Integration to apps in the

cloud Testing solutions running in the

cloud Acceptance testing of solutions

procured OFF the cloud (SaaS)

Testing solutions in the context of massive integration• Boundary conditions become

extreme


Release cycles now more frequentTesting for rapid releases (Web2.0 & SaaS style)

Continuous integration is a trend that is rapidly being adopted in testing• Include build automation, auto-deployment of the build into a production-like environment, and

ensuring a self-test mechanism such that at the very least, a minimal set of tests are run. Leveraging virtualised test environments

• As soon as code is checked in, a build workflow kicks in that compiles the code - deploys it on to a virtualised test environment, triggers a set of unit and functional tests on the test environment, and reports on the results.

VSTS takes the build workflow one step further, even before checkins• Performs the build before the check-in is finalised, allowing the check-in to be aborted if it

would cause a break, or if it fails the tests. • And given historical code coverage data from test runs, the tool can identify which one of the

several thousand test cases needs to be run when a new build comes out - significantly reducing the build validation time.

Failed builds and tests are found Immediately • The developer who checked in the offending code is probably still nearby and can quickly fix

or roll back the change. Go Agile!

• With Agile getting more mainstream, what are the QA changes needed in integrating into Agile methodologies. In particular, with scrum, how would QA function? If QA is part of a scrum team, how will integration QA work


On-demand (virtualized) test infrastructure

Have access to lot of customer scenarios- real or assumed

Easy setup of environment• Accelerate set-up/tear down and restoration of complex

virtual environments to a clean state, improving machine utilization

Developers can access• Eliminate no repro bugs by allowing developers to recreate

complex environments easily Better automation of customer environment testing

• Improve quality by automating virtual machine provisioning, building deployment, and building verification testing in an integrated manner (details later)


Can we be pro-active?

Top-down, dev driven Avert bugs, rather than test-

find-fix

From definition & design thru implementation

Interesting research work in ensuring Dev Tools help

We can do better than this..


About Progress Software

A publicly traded (PRGS) $500M enterprise software company with a solid track record of growth with more than 25 years of profitability

More than 140,000 customers and 2,500 partners in more than 175 countries

More than $5 billion of Progress technology, partner applications and services are sold each year to thousands of companies

Invented both the Enterprise Service Bus and commercial Complex Event Processing


Progress Responsive Process Management Suite

Financial Services Accelerators

Travel & Logistics Accelerators

Communications Accelerators

Real-timeMarket Surveillance

Smart Airlines

Revenue Assurance

Enterprise Integration Data Services


Why Progress?Progress Delivers Value for the Banking Industry

Enable Business Strategy

Trusted Partner

Lowest Cost of Ownership

Industry Expertise


Driving Your Business with Responsive Process Management (RPM)


Progress Success in Banking

Progress Internal use only


Operational Responsiveness

1. Customer Request for Outbound Service (via phone, email, EDI, fax)2. Appointment for Pickup – Service Provider schedules pickup 3. Routing Assignment Scheduling (Equipment, crew, intermodal options, delivery windows…)4. Intermodal Notification (Anticipate inbound schedule crew, equipment,… )5. Outbound Origination (Shipments in transit)6. Location Updates with current location several times en route7. Alert Intermodal Carrier - Delayed Inbound 8. Outbound Delivery to Intermodal Service- Reschedule Equipment and crews9. Intermodal Outbound – Shipment in Transit10. Location Updates with current location several times en route11. Late Arrival- Shipment arrives late at customer. 12. Drop Off. Proof of Delivery (POD) received.13. Reconciliation. Customer reports damaged goods and SLA violation.

1

2

3 4 5 6

57

8

10

11

136 98 12

Monitor and share continuous real time information to the

appropriate service providers.

Enable invoicing engines to

simultaneously manage tariffs, claims, discounts,

and equalization.

Continuous Process Improvement – automate, adapt and create new business processes


??

For products automation is easy (as it is a necessity to reduce repetitive work). But how would we incentivize automation for services- especially turnkey projects (that may or may not have maintenance phase following dev)

In multi project/SI environments, what has your experience been building QA/Automation frameworks that could be used across projects?

How do you factor in frequent handle UI changes when automating tests Challenges in testing multi-channel UI (web/mobile/et al) Do you think it will be practical to integrate testing & code-coverage on a nightly basis

and help accelerate fixing the bugs (by identifying regressive checkins, that caused test failures)?

Cloud challenges? What shifts in QA thinking/test approaches would we need to test in the cloud? If there are multiple solutions in the cloud, how would integration testing work? Your experiences. (all)

80s manual. 90s rich UI. 2000 web. 2010s- what?


The Transportation & Logistics Value Chain



Billing

Collections


Customer Service



Planned Tenders

Invoicing Engines

Settlement

Customer

Domestic

Trade Services

International


MRO


Crew Mgt

GoodsMgt


Asset Mgt

Warehouse Mgt

Project Mgt

Document Mgt

Event & Information Network

Real Time Business Visibility


Solution Stacks to be tested

Solutions often present as a stack or suite in a customer environments.

The possible number and complexity of usecases in these combined environments are much higher

The testing complexity thus significantly amplified.

Date post:	23-Feb-2016
Category:	Documents
Upload:	shing
View:	37 times
Download:	0 times

From technology to solutions: Customer Centric Testing

Documents