Challenges and Solutions in Testing Modern Software Paradigms like

Mobile and Cloud Computing Satya Kaliki

Chief Technology Officer

satya.kaliki@zanec.com

Who am I?

CTO and Co-founder of ZANEC Technologies

2

Interim CTO at an Insurance Company owned by Harvard Medical Institutions

Passionate about Mobile and Cloud Computing, Semantic Web, SOA

13 Years of industry experience

Agenda

Mobile Computing

Challenges

Factors to consider in your test strategy

Case Study – Search App for iPhone, Android, Windows Phone 7

Cloud Computing

Challenges

Factors to consider in your test strategy

Case study - Air Cargo Supply Chain

Q&A

3

CHALLENGES

Features

5

Locations D

ev

ice

s

Carriers

Test case complexity

Mobile Apps vs. Traditional Apps

6

vs.

Native Apps vs. Mobile Browser App

7

Courtesy: Global Intelligence Alliance

Location and Localization

8

Field Testing – Signal strength variations

9

THINGS TO CONSIDER IN YOUR TEST STRATEGY

11

Use Emulators in the Lab

Use Real Devices in the Lab

Use Real Devices in the Field

Adopt a crowdsourcing strategy

App Maturity and Testing Progress

Lowest Common Denominator

and choose based on market share

12

Usa

bil

ity

13

Navigation

14

Varied end user expectations

15

Field Testing

16

Ba

ckg

rou

nd

no

ise

Field Testing - Multitasking

17

Use Emulators to simulate network variations

18

Android Emulator Commands

emulator – netdelay gprs [Emulates a GPRS, means min of 150 max of 550 ms]

emulator – netspeed gprs [Transfer rates of a gprs network]

emulator – netspeed 14.4 14.4 [Exact transfer rate for up and down]

CASE STUDY Search App for iPhone, Android,

Windows 7 Mobile

App Overview

The Search App provides secure, real-time access to documents, structured data and messages - all from a single click Can connect to wide variety of sources including PC’s local hard

drives, Popular social Networking sites, Internet Search engines, Internet Email Providers, MS SharePoint, MS Exchange Server

Can distinguish variety of file types including MS Office

Currently the App is targeting two broad categories of Users Personal Use - Social Networking, SMS, Contacts, Media, Photos Business Users - Email, Exchange, Contacts etc

20

Test Strategy

iPhone as first Platform and iOS Versions 4.1 (3G, 3GS and 4)

Extended to Android in Phase 2

Windows Phone 7 is Phase 3

21

Test Approach

Emulators in the Lab

Real Devices in the Lab

Real Devices in the Field

Customer’s priority focus is UK

Test Strategy

Scope of Testing

Usability

Navigation

Intuitiveness

User Personas

Varied demographics

Mobile Literacy not considered for this App

Performance

Multi-tasking

Varying signal and network parameters

22

Challenges in Cloud Computing

Testing

Introduction

Cloud Computing presents significant benefits to organizations that adopt this paradigm:

Reduced Cost

Improved Flexibility

Significant Mobility

Increased Service Focus

Testing in a Cloud Computing environment must address key risks associated with migrating to Cloud

In the absence of a comprehensive Test Strategy realization of the promise remains a dream

24

Cloud Computing

Risks

Security

Reliability

Manageability

SLAs

Multi-tenancy

Resilience

&

Self-healing

Governance

Location Independence

25

Test teams must understand these models

Consumers

Networks & Firewalls

Data Centers and Power

Physical Servers

Virutalisation

Application Platforms

Operating Systems

Software

Infr

astr

uctu

re a

s a

Serv

ice (

IaaS

)

Pla

tform

as a

Serv

ice (

PaaS

)

Soft

ware

as a

Serv

ice (

SaaS

)

THINGS TO CONSIDER IN YOUR TEST STRATEGY

Choose a Test Environment

On-Premise vs. On-Cloud

28

vs.

Must include tests for each of the Cloud Computing risks

29

Don’t forget to include the App Functional Testing!

Assess Data Security

30

Evaluate App specific security best practices V

uln

era

bility

A

na

lysis

Au

dit

, Co

mp

lia

nce

a

nd

Go

ve

rna

nce

31

Performance Testing

32

Multi-tenancy and

data security

33

Test for Reliability

34

Test for Maintainability and

Manageability

35

Estimate optimal Service Levels and test whether you are able to

meet them

36

Checklist for assessing Go Live

Readiness

CASE STUDY Air Cargo Supply Chain

App Overview

The App provides secure, real-time access to shipment documents and role based view of consignment location One-Time Data Entry End to end, role based visibility to all stakeholders

(Ground handlers, General Sales Agents, Airlines, Customs and Freight Forwarders)

38

Offers great benefits to stakeholders Increased reliability on consignment’s current status Reduced costs Improved Accuracy and Quality Enormous Scalability

Test Strategy

Infrastructure as a Service model

On Cloud Test Environment

Security model best practices

Multi-tenant

Multi-tiered (Same document, different view based on organisation, role and level within)

Need to perform well at 1 million consignments being shipped concurrently

39

Test Strategy

Scope of Testing

Functional

Usability

Various levels of Security

Performance

Manageability

Recovery

Operational readiness

40

Questions?