Analytical Optimization Technologies for Games & AppsAnalytics, A/B Testing, Segmentation &

Dynamic Best-Fit

Alan Avidan, Exec. Director & Chief BeezzzDev

Points We’ll Cover

What is optimizationWhat can be measured and optimized

Optimization technologies for games and apps AnalyticsA/B TestingUser SegmentationDynamic Best-Fit

Let’s get started!

Optimization Family Tree

What is Optimization?

Data-driven efforts formulated and designedto maximize Key Performance Indicators (KPI)by enhancing in-game/app conversions

Max Z {f(x)} ≡ f(Engagement, Retention, Monetization,

Virality) X

s.t. g(x)=0, h(x)<0

Which Key Performance Indicatorsshould you target for optimization?

Monetization Engagement Retention Virality

Analytics and Optimization Companies

Optimization Results

88.9% improvement on landing page

Which Game Elements Can Be Optimized?

New Features

Arts (Creative)

Message Wordings

Game Mechanics

Game Flow Landing Pages

Promotions

Optimization Technologies We Use

Analytics A/B Testing (Split Testing)

User Segmentation Dynamic Best-Fit

 The process of developing optimal or realistic decision recommendations based on insights derived throughthe application of statistical models and analysisagainst existing and/or simulated future data - Wikipedia

Typical uses of Analytics

Engagement Tracking Funnel Analysis

Measure, Display, Analyze, Change, Repeat

Analytics

Analytics - Bottom Line

Upside• Monitor, record, & display Key Performance

Indicators (KPI) • Measure effectiveness of game mechanics and

monetization Efforts • Access and display data to understand how

users interact with game/app; decide where improvements are needed

Downside The capture and storage of data, followed by

analytics and visualization is tedious, provides retroactive information about the “Average User.”

A/B Testing

Credit: Steve Collins, Swrve

A/B Testing Uses

New features are introduced to a selection of users, and their reactions measured. Features remain only if users engage with them - Wooga

Photo: Spencer Higgins; Illustration: Si Scott

Q: A/B Testing: What are the most unexpected things people have learned from A/B tests?

Answer Wiki

1.Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count2.I have learned how dramatically, and ridiculously wrong my most basic assumptions were3.It's empirically proven that you should let the data tell you what works or not and you should constantly be testing4.That the devil is in the detail - a minor change can generate a significant result

A/B Testing – Bottom line

Upside

Simple; understandable; can achieve very good results

Downside:

• One size fit all

User-Base Segmentation

A Priori Segmentation:• Geographic - states, regions, countries• Demographic - age, gender, education• Psychographic - lifestyle, personality, values• Positive - similar wants or needs

Clustering Segmentation:• Behavioral - similarities of behavioral patterns and like-

properties

Segmentation - Uses

Cohort Analysis – Track over time users with common reference feature

Targeting - Serve different treatments for each segment to maximize KPIs

Segmentation – the bottom line

Upside

Can be effective especially reaching out to groups identifiable by known attributes

Downside:

– Clusters are predefined and thus remain unchanged during the analysis

– Requires storage of terabytes of data– privacy issues

Dynamic Best-FitReal-Time Automated Action Optimization

A predictive algorithmic technology used to serve each user the page option they are most likely to convert on at any feature point

DNA Signature Attributes

Geo-Demographic attributes: age, gender, education, countryGeo-Demographic attributes: age, gender, education, country

Facebook attributes: Friends, Likes, Interests, Posts, Events

Behavioral attributes: level, spending, score, progress, custom

Session attributes: time of day, day, durationSession attributes: time of day, day, duration

Proprietary attributes: novice, high-bidder, risk-averseProprietary attributes: novice, high-bidder, risk-averse

3rd Party attributes: income level, education3rd Party attributes: income level, education

How Dynamic Best-Fit Works

Advanced statistical algorithms find strong correlations between user DNA data

and past conversions

Best-Fit Wording

Best-Fit Options

Payment Pages: Different Ranges

Best-Fit Options

Payment Pages: Different Incentives

Best-Fit: Game Flows

Option 1 Option 2Open page

Full tutorial

Stage 1

Open page

Short tutorial

Stage 1

Option 3Open page

No tutorial

Stage 2

Best-Fit: Payouts

Only large and less frequent

winnings*

*The sum of all winnings are the same

Mostly small but more

Frequent winnings*

Best-Fit:

Invite Friends - Different Layouts

Best-Fit: Promotions

Go VIP

Buy VIP card for 5 EURand enjoy 30% more coins

for all future buys

Buy 1 Get 1 FREE

Receive twice theamount of goldfor regular price

Triple your money

Buy 100 GoldGet 200 Free

Best-Fit Arts

Dynamic Best-Fit: Results

“Total Friends” attribute as conversion indicator in payment pageInsight: users with less than 100 friends more readily reach the payment page, and moreover convert better

“Like” attribute as conversion indicator in payment pageInsight: users with more than 25% of Likes associated with apps monetize much better, and moreover clearly prefer Layout 2

Increases conversions and KPIs

Gain Valuable new insights to improve app design and user targeting

Review

• Optimization is vital to your game/app’s success • Retrofit existing games and plan for future

games• Match objectives with technologies: Different

technologies have different uses; Require a different level of involvement; and produce different Uplift results

• Future? -- Lots and lots moredata. Those that will learn toharness it will succeed

Alan@BeesAndPollen.com

Q & A