CapturingMobileExperienceintheWild: …pages.cs.wisc.edu/~patro/papers/Insight_Slides.pdf0.2...

Capturing Mobile Experience in the Wild: A Tale of Two Apps

Ashish Patro*

Shravan Rayanchu, Michael Griepentrog

Yadi Ma, Suman Banerjee University of Wisconsin Madison

*[email protected]

Deploying a mobile applicaDon…

Ashish Patro / Insight / CoNext 2013 2

Mobile app stores Developers App Users

Internet

ApplicaDon Server

What is the “ApplicaDon Experience”?


What factors are impacDng the users’ experience?

What factors are impacDng my applicaDon revenues?

What is the baWery drain of my applicaDon across different devices?

..........

Developers

Challenge for developers…


Developers App Users

Example 1: Diverse Devices



ApplicaDon Server

Type (tablet, phone) and pla_orm

Screen type/area, OS, features (keyboard), baWery

capacity, device age

Example 2: User diversity



ApplicaDon Server

LocaDon, Dme of day, New vs. old users

Network quality (signal, congesDon), session duraDons, revenues

Example 3: Diverse Networks



ApplicaDon Server

802.11, HSDPA, EVDO, LTE Latency, throughput

Goal: Capture “applicaDon experience”

Ashish Patro / Insight / CoNext 2013

Networks diversity

8

Device diversity

User diversity

How can we enable developers to capture the “applicaDon experience”?

Understanding “applicaDon experience”


Device + UI Design (e.g., screen size)

Network performance (e.g., latency)

ApplicaDon Overhead (e.g., BaWery, CPU

etc.)

ApplicaDon Experience

User Engagement (e.g., Session length,

interacDvity, retenDon)

Developer Revenues (e.g., virtual currency

usage)

9

User types (e.g., old vs.

new)

Developers

Understanding “applicaDon experience”



Network performance (e.g., latency)

ApplicaDon Overhead (e.g., BaWery, CPU

etc.)

ApplicaDon Experience

User Engagement (e.g., Session length,

interacDvity, retenDon)

Developer Revenues (e.g., virtual currency

usage)

10


new)

How can we capture these metrics across all users?

Developers

SoluDon: Embedded measurements •  Developed a measurement toolkit: “Insight” •  Using the applicaDon as a vantage point


Insight

In this talk…

•  Study with 2 popular applicaDons –  MMORPG game: > 1 million users (over 3 years)

– Study applicaDon: > 160,000 users (over 1 year)


•  Insight: Our mobile applicaDon analyDcs toolkit

Outline

•  Insight overview and deployment

•  Understanding applicaDon usage

•  Impact of network performance

•  Related work and summary


Insight measurement toolkit

Mobile App

App Code

Insight threads

App Servers


Insight servers

Insight toolkit advantages

Mobile App

App Code

Insight threads


Light-‐weight library code with easy to use API

Contextual measurements: When desired app is running

No addiDonal overhead for deployment of framework

Insight measurements


Mobile App + Insight

16




In-‐app acDviDes

User acDvity

17

Session duraDons

ApplicaDon specific events (e.g. in-‐app purchases, fighDng monsters)



Mobile App + Insight Device

Device info

18

Vendor, screen size, Pla_orm, OS version




19

Status Info (LocaDon, BaWery, CPU, Network)

Status updates

LocaDon: Country, State

ApplicaDon Overhead: BaWery + CPU + Memory usage

Network status: Signal, type (WiFi, HSDPA,

EVDO etc.)



Mobile App + Insight Insight server

20

AcDve network Measurements

ApplicaDon level latency: Round Trip Time (RTT) measurements to our server



Mobile App + Insight Insight server

In-‐app acDviDes (e.g., in app-‐purchases)

User acDvity

Device InformaDon

Device Info

21

Status Info (LocaDon, BaWery, CPU, Network)

AcDve network measurements

Status updates

In-‐situ applicaDon vantage point captures a rich set of metrics

Deployed applicaDons (1)

MMORPG Game: Parallel Kingdom (PK) •  Analyzing both iOS and Android users •  AcDons: Spend food, trade items, aWack players/monsters

•  Deployed more than 3 years •  > 1 million players •  > 61 million sessions


Deployed applicaDons (2)


Study Tool: StudyBlue (SB) •  Analyzed Android users only •  AcDons: Study, Create flashcards,

quizzes •  Deployed more than 1 year •  > 160,000 users tracked •  > 1.1 million sessions

23

0

30000

60000

90000

120000

150000

0 200 400 600 800 1000 1200 1400

Tota

l sess

ions

per

day

Days Elapsed (PK)

Age 2Age 3

Age 4

0

3000

6000

9000

12000

0 50 100 150 200 250

Tota

l sess

ions

per

day

Days elapsed (SB)

SchoolStart Break

Unique sessions per day

ApplicaDon usage for PK spikes with new updates while it is highly correlated with Dme of week and year for SB


Outline


•  Understanding applicaKon usage




Understanding “ApplicaDon Experience”



Resource ConsumpDon (e.g., BaWery, CPU etc.)

26


new)

0

0.2

0.4

0.6

0.8

1

iPad iPhone DroidXtreme

MyTouch3G

Droid2

HTCVision

HTCEVO

SGHCaptivate

HTCHero

SGHMoment

No

rm.

Act

ion

s/se

c

PK: Device Model

Screen9.7" 3.5" 4.1" 3.4" 3.7" 3.7" 4.3" 4" 3.2" 3.2"

Slide out keyboard

size (in.):

Impact of device on user interacDvity

48% lower


0

0.2

0.4

0.6

0.8

1

KindleFire

Nexus7

GalaxyS2

HTCEVO

GalaxyNexus

GalaxyS

HTCThunderbolt

DesireHD

No

rm.

Act

ion

s/se

c

SB: Device Model

Tablets

56% lower

27

User interacDvity = User acDons / Dme User interacDvity varied by 2x based on device types and

hardware features (screen size and slide out keyboards).





28


new)

Device form factor and pla_orm impacted interacDvity (upto 2x).

BaWery consumpDon in the wild (PK)

3x variaDon


High variaDon in applicaDon footprint (baWery usage) due to diversity of devices

BaWery drain = % usage/ Dme

BaWery drain vs. session duraDon (Evo 4G)


Sessions with high baWery drain exhibited upto 2x Dmes lower duraDons

Increase in baWery drain

50% lower

Conserving baWery consumpDon…


0

0.02

0.04

0.06

0.08

0.1

0.12

0 0.2 0.4 0.6 0.8 1

Pro

babili

ty D

istr

ibutio

n

HTC Evo 4G - Normalized battery drain rate (levels/min)

High (255)Moderate (102)

0

0.1

0.2

0.3

0.4

0.5

0 0.2 0.4 0.6 0.8 1

Pro

babili

ty D

istr

ibutio

nKindle Fire - Normalized

battery drain rate (levels/min)

High (255)Moderate (175)

Mode 36% lower

31

Device specific opDmizaDons (e.g., screen brightness, GPS frequency) can help, but variable gains per device

Low Impact!





32


new)

Device form factor and pla_orm impacted interacDvity (upto 2x).

High variability in baWery overhead (3x) Gains from device opDmizaDons can vary

Impact of retaining users on revenue (PK)

4x variaDon


Over a period of 7 months (Age 2 of

the game)

Old users tend to spend more money daily (4x more than new users)

New users Old users





34


new)

Device form factor and pla_orm impacted interacDvity (upto 2x)

High variability in baWery overhead (3x) Gains from device opDmizaDons can vary

User retenDon is important: Upto (4x) more daily revenues per user

Outline






Impact of network performance



36

Network latency measurements

User InteracDvity

Insight server

Network type usage

Developer revenues




37


User InteracDvity

Insight server

Network type usage

Developer revenues

Impact of latencies on user interacDvity


0

0.2

0.4

0.6

<=0.3 0.4 0.6 0.8 >=1

Cel

lula

r usa

ge ra

tio(m

ax. o

f 1)

Avg. cellular RTT (in seconds)

PKSB

0 0.2 0.4 0.6 0.8

1

<= 0.3 0.6 0.9 1.2 >= 1.4

Nor

m. a

ctio

ns/ti

me

Avg. RTT of session (in seconds)

PKSB

40% lower

38

40% drop in user interacDvity for MMORPG at high latencies (900ms). Lower impact on study applicaDon.

Higher network latencies




39


User InteracDvity

Insight server

Higher drop in interacDvity for MMORPG (40%) at poor latencies

Network type usage

Developer revenues

Impact of latencies on cellular usage


0

0.2

0.4

0.6

<=0.3 0.4 0.6 0.8 >=1

Cel

lula

r usa

ge ra

tio(m

ax. o

f 1)

Avg. cellular RTT (in seconds)

PKSB

0 0.2 0.4 0.6 0.8

1

<= 0.3 0.6 0.9 1.2 >= 1.4

Nor

m. a

ctio

ns/ti

me

Avg. RTT of session (in seconds)

PKSB

42% lower cellular usage

40

High cellular latencies correlated with higher preference for WiFi networks for both apps

Higher average cellular latencies




41


User InteracDvity

Insight server


Greater user preference for WiFi networks at high cellular latencies across both apps

Network type usage

Developer revenues

PK: Impact of latencies on revenues


52% lower

revenue

42

High network latencies caused upto a 50% drop in developer revenues

Latency vs. Revenue / Dme




43


User InteracDvity

Insight server


Greater user preference for WiFi networks at high cellular latencies.

Upto 50% drop in developer revenues (MMORPG) due to poor network performance

Network type usage

Developer revenues

Outline






Related work

•  Commercial Tools: Android/iOS, Flurry •  Understanding network performance (MobiSys’10) – 3G Test: Standalone mobile app. to measure cellular and WiFi performance

•  Smartphone usage studies (MobiSys’11) – Falaki et al. : ApplicaDon, network and device usage study across 255 users

•  Mobile applicaDon usage characterisDcs (IMC’11)


Summary •  Presented “Insight”, a measurement framework for mobile applicaDons –  >3 year deployment on 2 popular apps

•  High variance (3x) in resource usage across devices –  Device specific opDmizaDons can help (gains vary)

•  User retenDon is criDcal for applicaDons revenue generaDon and engagement: 4x variability

•  High latencies lowered both revenues (52%) and user-‐engagement (40%) (more for MMORPG)



Thanks!

Date post:	28-Jun-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Capturing*Mobile*Experience*in*the*Wild:* …pages.cs.wisc.edu/~patro/papers/Insight_Slides.pdf0.2...

Documents

CapturingMobileExperienceintheWild: …pages.cs.wisc.edu/~patro/papers/Insight_Slides.pdf0.2...