1

Presents:

Webinar

2

Expert Perspectives on Mobile Performance Testing

TODAY’S SOASTA PRESENTERS

Mark Tomlinson: West Evergreen Consulting, former LoadRunner PMDan Bartow: SOASTA VP Product ManagementModerator: Brad Johnson

Agenda: • Poll question• Embracing mobile performance testing – what's new and what's

not• Building a performance test plan to address what really matters• Testing and measuring for "Fast" and "Strong" all at once for

mobile and web• Implementing a solution that does it all with agility to match the

new normal

Questions: Submit in the question box during event

3

o Sheer Number of Devices (953M Smartphones)

o Different Operating Systems

o Scale of Global Customers (6B)

o Dynamic Content (Video, Animation, …)

o Emerging People to Machine

Interfaces

Mobile performance at scale can not be tested manually.

4

CPU

Battery

Memory

Native Apps

Mobile Web AppsApp

Server

Database

App Server

Web Server

CacheLoad

Balancer Web

ServerMobile

browserWeb

trafficHTTP(S)

Shared web & mobile infrastructure

Web browser users

Mobile browser users

Hybrid Mobile AppsApp

Server

Database

App Server

Web Server

CacheLoad

Balancer Web

ServerNative App Web

traffic

Dedicated or shared mobile infrastructure

Real devices & large scale mobile traffic

mUX is most affected by back

end load and network

performance

Mobile User Experience (mUX)

is affected by app & device

performance only

mUX is affected by native app, device,

back-end and network

performance Fast AND Strong

5

o Development teams test at a unit and component level (as always)

o Test teams load test with web-based traffic (if load testing at all)

o Ops teams monitor with services built pre-Smartphone

o No one communicates, yet agile dev is all about communication!

6

7

o 20 years in testing

o Performance Guy

o Small tests & Big tests

o Small & Big companies

o …speed freak.

8

- Old applications migrated to new mobile formats – “transformation”

- New Applications added-on to existing systems – “bolt-on”

- Entire new business units created for mobile consumer experience

- Device manufacturers are investing and expanding – alarmingly!!

- “The number of Smartphone users worldwide is predicted to exceed 1 billion by 2014” (Parks Associates, 2010)

- “Tablets to hit 100M shipments in 2012” (Source: ABI Research, 2012)

“Every customer I’ve worked with in the last 18 months is prioritizing mobile performance testing.”

-Mark Tomlinson

9

WHAT’S NEW?

oDevices – lot’s of ‘em

oBattery Life & Heat

oGeographic Diversity

oLatencies – insanely high

oVariable Processor Frequency

WHAT’S NOT!

oThe risks of poor performance

oResponse Time & Volumes

oRoot-cause Analysis

oCapacity Estimation

oTest Planning

oBugs…lot’s of ‘em

10

A real story about a customer situation:

1.1,000 requests per second into the services infrastructure

2.Systems were fine-tuned, low-latency (< 250 ms)

3.Memory (in .NET CLR) was also optimized with perfection

4.They introduced a new mobile experience to the end-user

5.10% of the company started using the mobile app

6.What happened next…???

YOU MAKE THE CALL…

11

Enhance existing plans for mobile:

• Specific response-time goals for mobile

• Connected/disconnected test scenarios

• Mobile test lab setup & configuration

• Matrix of devices to be tested/supported

• Define end-user location and conditions

• Monitors for device’s physical resources

• Monitor client-side performance

Let’s review an example test strategy…

12

Ensure that new mobile plans include:

• Mobile Performance Objectives (time)

• Defined scope for device types

• Detailed explanation of end-user behavior

• Inter-app test conditions

• Carrier-specific test conditions

• Failure and recovery conditions

• Back-end system dependency and impacts

Let’s review a new mobile test plan…

13

Dan Bartow

14

Any mix of traffic at any scale

Hybrid Mobile AppsApp

Server

Database

App Server

Web Server

CacheLoad

Balancer Web

ServerNative App Web

traffic

Dedicated or shared mobile infrastructureReal

devices & large scale mobile traffic

Global Web browser users

Global Mobile browser users

Measure what Matters

o Dev teams test early and continuously for FREE with CTLite

o Test teams respond quickly with complete mobile coverage

o Ops team validate and tune production infrastructure

15

Thanks & Q&A

Contact SOASTA:www.soasta.com/cloudtest/info@soasta.com866.344.8766Follow us:

twitter.com/cloudtesttwitter.com/cloudtest

facebook.com/cloudtestfacebook.com/cloudtest

White Papers, Webinar Recordings, Case Studieswww.soasta.com - Knowledge Center

Next Webinar: Sept. 27, 10 a.m. PST“Automated Testing & Continuous Deployment

for Mobile Apps in the Cloud”(SOASTA & CloudBees)

Register at www.soasta.com/knowledge-center/webinars

Contact Markmtomlins@westevergreen.com@mtomlins

Contact Dandbartow@soasta.com@PerfDan