Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing

#mceconf #mce-addingknobs 11 Jan 2014 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Adding Knobs to your App – Modifying & Measuring Behavior in the Wild (without Republishing) Cheng Luo, Amazon Appstore 11th January, 2014 @chengluo

#mceconf #mce-addingknobs 11 Jan 2014

Agenda •  Why Amazon Mobile SDK •  How to use A|B testing •  Add your A|B testing in 10 minutes •  Demo


30% 25%

45%

<$50 $50-‐$200 $200+

Monthly Unique Visitors (US):

97M

6.6 Active Global Customer

Accounts:

200M Avg. Daily Unique Visitors

(US):

12.9M

Note: Active Customer Accounts are unique email addresses that have placed an order in the last 12 months Sources: 1. Amazon Investor Relations; 2. ComScore PlanMetrix January, 2013.

10%

38%

52%

<$25K $25K-‐$75K $75K +

1 2 2 2

8%

32%

18%

42%

0-‐18 18-‐34 35-‐44 45+

Household Income of Amazon Unique Visitors2

Age Composition of Amazon Unique Visitors2

% of Amazon Shoppers Who have Made an Online Purchase2

Amount Spent

Avg. Visits per Visitor per

Month (US):

Active Users


•  Amazon Appstore runs Android apps

•  75% of Android apps work without any need for customisaTon •  Simple APIs to publish free-‐to-‐user, freemium and premium apps

A/B TesTng Service API

GameCircle API

Maps API Device Messaging API

In-‐App Purchasing API

Simplicity


The latest Kindle Fire HDX

Kindle Fire HDX 7" Tablet $229, with 24x7 May Day support


What A|B Testing can do for you

Precisely control who is affected

Change in the wild; No new APK needed

Safely test back-‐end load


Case Study - Air Patriots Used A/B Testing to decide: •  Include in-game advertising •  Provide purchasing short-cuts •  Notification frequency •  Level difficulty •  Promotional imagery


Case 1 – Image


Case 2 – Ads placing


Case 3 - Notification Notification appears •  x1 day •  x 3 days → Sweet spot •  x 7 days


Case 4 - User Retention

Level: 1 Difficulty: Easy

Level: 5 Difficulty: Medium


Air Patriots Sample Results


Add A/B Testing to Your App






Initialize A/B Testing in Your App private static final String APP_KEY = "876c809ba7084b9fbe2c08a22e5acda1"; private static final String PRIVATE_KEY = "9nxxk4//qL89ymXl/P5aL7w8uhs3DVDrLcyYViG6yn0="; AmazonInsights insightsInstance; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); . . . InsightsCredentials credentials = AmazonInsights.newCredentials(APP_KEY, PRIVATE_KEY); insightsInstance = AmazonInsights.newInstance(credentials, getApplicationContext()); }


Create an A/B Testing Project






Add a New Launch


Add a New Launch


Add a New Launch


Multiple Projects Are Allowed

private static final String AB_PRJ_ACC = "Acceleration"; //Project name private static final String AB_VAR_MAXACC = "maxAcceleration"; //Variable name private static final String AB_PRJ_VEL = "Velocity"; //Project name private static final String AB_VAR_MAXVEL = "maxVelocity"; //variable name


Load Variation Variables private static final int DEF_MAXACC = 5;

private static final int DEF_MAXVEL = 50;

private int maxAcc;

private int maxVel; private void loadTreatment() { ABTestClient abClient = insightsInstance.getABTestClient(); abClient.getVariations(AB_PRJ_VEL, AB_PRJ_ACC).setCallback(new InsightsCallback<VariationSet>() { @Override public void onComplete(VariationSet variations) { Variation variation = variations.getVariation(AB_PRJ_ACC); maxAcc = variation.getVariableAsInt(AB_VAR_MAXACC, DEF_MAXACC); variation = variations.getVariation(AB_PRJ_VEL); maxVel = variation.getVariableAsInt(AB_VAR_MAXVEL, DEF_MAXVEL); } }); }


Changing Values on the Fly








Add a Segment


Add a Segment


Add a Segment


Use the Segment

//get user age from the app settings SharedPreferences myPrefs; myPrefs = getSharedPreferences("com.example.testingapp", Context.MODE_PRIVATE); int userAge = myPrefs.getInt("com.example.testingapp.USER_AGE", ""); //Initial the segments here

UserProfile userProfile = insightsInstance.getUserProfile(); userProfile.addDimensionAsNumber("age", userAge);

Use App Segngs to idenTfy your user segments


Demo


Wrap up •  A/B Testing

– Free cross platform services – Precise measuring behaviours using custom

segments – Modifying app behaviors without

republishing


Thank you@AmazonAppDev

Date post:	09-May-2015
Category:	Technology
Upload:	cheng-luo
View:	423 times
Download:	1 times

Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing

Technology