Edwin van Loon, Valid

What’s in the cloud for testing?

The sky is the limit

www.eurostarconferences.com

@esconfs #esconfs

Introduction Edwin van Loon

• ISEB Practitioner

• Lean Six sigma green belt

• Allmost 15 years of experience

within different testing roles

• Living in Belgium

• Working in the Netherlands

edwinvanloon

Edloon

Edwin.van.loon@valid.nl

Cloud

Millennium change

Agile manifesto

ISO SQuaRE (25010)

Web based development

Introduction of the Euro coin

Model based testing

Risk based testing

Regulatory compliance

Test Certification

Agenda

•The Cloud in general

•The Cloud and Testing

•The cloud and it’s possibilities

• Testing the Cloud

What’s in the cloud for testing

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Applications

Runtime

Security

Databases

Servers

Network

Storage

Hardware

Virtualization

Cloud layers

Applications

Runtime

Security

Databases

Servers

Network

Storage

Hardware

Virtualization

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Cloud layers

Applications

Runtime

Security

Databases

Servers

Network

Storage

Hardware

Virtualization

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Cloud layers

History of the Cloud

1997 2000 2002 2004 2008 2010 2012 -2015

Early SaaS

adaptors

16.3% compound

annual growth rate

through

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Announcement Cloud in 1997 “In the last 7 years, do you know how many times I have lost any personal

data? Zero.

Do you know how many times I have backed up my computer? Zero…

I have computers at Apple, at NeXT, at Pixar, and at home. I walk up at any

of them and log in as myself, it goes over the network, finds my home

directory on a server, and I got my stuff where ever I am…

Carrying around these non-connected computers or computers with tons of

data and state in them is Byzantine by comparison.”

Steve jobs at WWDC 1997 (Apples Worldwide Developers Conference) (http://www.youtube.com/watch?v=3LEXae1j6EY)

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Cloud computing - Advantages

1. Time to market

2. Scalability

3. Reduction of IT-costs

Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General

Cloud testing

• a form of software testing in which

web applications use cloud computing

environments to simulate real-world user

traffic.

• allows you to use simulated web traffic to ‘load

test’ a website or web application with a

flexible, scalable and affordable testing

environment.

Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing

Test tools:

- Functional and Performance testing

- Cross browser testing

- Test management

Runtime

Security

Databases

Servers

Network

Storage

Hardware

Virtualization

Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing

Cloud test tools

History of the Cloud

1997 2000 2002 2004 2008 2010 2012 -2015

Early SaaS

adaptors

16.3% compound

annual growth rate

through

Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing

Advantages cloud test tools

Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing

• Accessible at Any place, Any time, Any device

• Scalability

• No infrastructural constraints

• No geographical constraints

• Low time to market

• Temporary contract / low costs

• No installation

• Availability

• Usability

• Possibilites for endurance testing

• ………

Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing

Demo

Changed quality expectations

Cloud General Cloud and Testing Testing the Cloud Cloud Possibilities

Do you know how many times I have lost any personal data?

Do you know how many times I have backed up my computer? Zero!

Users of cloud solutions expect

• Being able to use the solution at any time, any

place using any device

• Being able to run their processes

without failures

• No loose of data

Cloud General Cloud and Testing

Test tools

Test portal

Templates

Standards

KPI’s

Virtual testmachines

Test servers

Network types (latency)

Testing the Cloud Cloud Possibilities

Test layers Testing as a

Service

Portal

Tactical

Tactical

TaaS (or Quality as a Service)

Operational

Software under Test

Define

Goals

Define

KPI’s Improve

Managed: In Control

Operational

Test

tools Standards

Test

results Metrics Test

machines

Define

Approach

Measure

Improve (continuous)

Consolidate

Advice

Infrastructure Test

machines

Test employees / test activities

Processes Standards Tools

Templates

TaaS Service Level agreement

• Agreements on:

• Number of test cycles per period (including peak and off-peak)

• Test coverage (quality level) or effort

• Through put time (test execution, test preparation, etc.)

• Defect detection percentage (DDP)

• Test or QA activities in general

• Reports (operational and tactical)

• But also ……

• Quality level of documents used for testing and the system under

test

Cloud General Cloud and Testing Testing the Cloud Cloud Possibilities

Testing the cloud

• Security:

• Compliancy statement

• Hack testing

• Continue monitoring

• Test of cost models

• Be aware of possible fixed release dates

of SaaS solutions

• Juridical aspects of the cloud

• Availability of cloud solutions: • Test of vendors (Cloudsleuth)

• Two internet providers

• Two instances / multi-region

Cloud General Cloud and Testing Cloud Possibilities Testing the Cloud

Resume

• The cloud is a TaaS enabler

• The cloud will change the expectation of users

regarding quality

• Cloud services requires different test approach

• Cloud test tools allow test teams to:

• avoid high costs on investments

• execute test global tests

• The cloud allows the test community to cooperate

and connect

The cloud has a already a high impact on

Software Testing

…….. and the impact will become huge

Stay Ahead