A/B Test

A/B TEST

IN GOD WE TRUSTOTHERS MUST BRING DATA

GLOBO.COM

BOOKING.COM

LOOPINFINITO.COM.BR

COLETÂNEA FRONT-END

BASICS

AT GOOGLE

USED TO SET OPTIMAL NUMBER ON SEARCH RESULTS

AROUND 2000

ORIGIN

DEFINITIONA WAY TO TEST YOUR PAGE AGAINST THE CURRENT DESIGN AND DETERMINE WHICH ONES PRODUCES POSITIVE RESULTS

BASICS

BFLIP COIN

ABASICS

HiPPOHIGHEST PAID PERSON OPINION

I THINK

I KNOW

TEST CASUAL RELATIONSHIP

INSULATE EXTERNAL FACTORS

STANDARD FDA DRUG TEST

BASICS

CORRELATION

CAUSALITY*

AS İCE CREAM SALES İNCREASE, THE RATE OF DROWN İNG DEATHS İNCREASES SHARPLY. THEREFORE, İCE CREAM CONSUMPT İON CAUSES DROWN İNG.

CORRELATION / CAUSALITY

EXAMPLE 1

HTTP://EN.W İK İPED İA.ORG/W İK İ/CORRELAT İON_DOES_NOT_ İMPLY_CAUSAT İON


CAUSALITY

CAUSALITY

CO

RR

ELA

TIO

N

PALM S İZE CORRELATES W İTH YOUR L İFE EXPECTANCY. THE LARGER YOUR PALM, THE LESS YOU W İLL L İVE , ON AVERAGE.


EXAMPLE 2

WOMEN HAVE SMALLER PALMS AND L İVE 6 YEARS LONGER ON AVERAGE


CORRELATION

CONVERSION

DEFINITIONTHE POINT AT WHICH A USER TAKES THE DESIRED ACTION

CONVERSION

FOR E-COMMERCE•COMPLETED PURCHASES •PRODUCT PAGE VIEW

CONVERSION

FOR MEDIA•PAGE VIEW •ARTICLES READ

CONVERSION

FOR DONATION•FORM COMPLETION

CONVERSION

CASES

YOU ARE THE HiPPO

1

2

3

4

5

6

MSN REAL STATE

HTTP://A İ .STANFORD.EDU/~RONNYK/EXPTH İNKWEEK2009PUBL İC.PDF

6

MSN REAL STATE

↑9.7%

HTTP://A İ .STANFORD.EDU/~RONNYK/EXPTH İNKWEEK2009PUBL İC.PDFHTTP://A İ .STANFORD.EDU/~RONNYK/EXPTH İNKWEEK2009PUBL İC.PDF

MSN HOME PAGE SEARCH BOX

1

2

HTTP://DL.ACM.ORG/C İTAT İON.CFM? İD=1281295&CFID=431848911&CFTOKEN=49176768

MSN HOME PAGE SEARCH BOX

1

2

NEUTRAL RESULT

HTTP://DL.ACM.ORG/C İTAT İON.CFM? İD=1281295&CFID=431848911&CFTOKEN=49176768

MICROSOFT OFFICE ONLINE STORE

1

HTTP://WWW.EXP-PLATFORM.COM/DOCUMENTS/EXP_DMCASESTUD İES.PDF


2



↓64%

2


SAFESOFT SOLUTIONS

1

HTTPS://VWO.COM/BLOG/PROM İNENT-PR İCE-D İSPLAY- İNCREASES-LEADS-GENERATED-BY-100/

2

2

SAFESOFT SOLUTIONS

↑100%

HTTPS://VWO.COM/BLOG/PROM İNENT-PR İCE-D İSPLAY- İNCREASES-LEADS-GENERATED-BY-100/

HIGHRISE

1 2

HTTPS://S İGNALVNO İSE.COM/POSTS/2991-BEH İND-THE-SCENES-AB-TEST İNG-PART-3-F İNAL

HIGHRISE

2

↑37.5%HTTPS://S İGNALVNO İSE.COM/POSTS/2991-BEH İND-THE-SCENES-AB-TEST İNG-PART-3-F İNAL

HIGHRISE

1 2


HIGHRISE

2

↑102.5%HTTPS://S İGNALVNO İSE.COM/POSTS/2991-BEH İND-THE-SCENES-AB-TEST İNG-PART-3-F İNAL

HIGHRISE

3


HIGHRISE

↓22.72%HTTPS://S İGNALVNO İSE.COM/POSTS/2991-BEH İND-THE-SCENES-AB-TEST İNG-PART-3-F İNAL

3

HIGHRISEHTTPS://S İGNALVNO İSE.COM/POSTS/2991-BEH İND-THE-SCENES-AB-TEST İNG-PART-3-F İNAL

HIGHRISE

↑4.78%

↑3.49% ↓3.38% ↓0.8%

↑2.2%


REFINEMENT / EXPLORATION

HTTP:// İNS İDE İNTERCOM. İO/W İREFRAM İNG-FOR-WEB-APPS/

NETFLIX

BASEHTTP://BLOG.OPT İM İZELY.COM/2014/08/14/2-ALEXA-500-S İTE-REDES İGNS-THAT-SHOULD- İNSP İRE-YOU-TO-AB-TEST/

NETFLIX

VARIANTHTTP://BLOG.OPT İM İZELY.COM/2014/08/14/2-ALEXA-500-S İTE-REDES İGNS-THAT-SHOULD- İNSP İRE-YOU-TO-AB-TEST/

JUNE 2011NEW DESIGN WAS ROLLED OUT FOR 100% OF USERS

NETFLIXHTTP://BLOG.OPT İM İZELY.COM/2014/08/14/2-ALEXA-500-S İTE-REDES İGNS-THAT-SHOULD- İNSP İRE-YOU-TO-AB-TEST/

PPL WENT MAD


DATA DON’T LIE↑ RETENTION ↑ ENGAGEMENT


“WHAT PEOPLE SAY AND WHAT THEY DO ARE RARELY THE SAME.”— BRYAN GUMM


LIVECHAT

1

2

DAN S İROKER & PETE KOOMEN. “A/B TEST İNG.”

LIVECHAT

2

↑14.6%

DAN S İROKER & PETE KOOMEN. “A/B TEST İNG.”

FRAMING EFFECTS*

DEFINITIONAN EXAMPLE OF COGN İT İVE B İAS, İN WH İCH PEOPLE REACT TO A PART İCULAR CHO İCE İN D İFFERENT WAYS DEPEND İNG ON WHETHER İT İS PRESENTED AS A LOSS OR AS A GA İN

FRAMING EFFECTHTTP://EN.W İK İPED İA.ORG/W İK İ/FRAM İNG_EFFECT_(PSYCHOLOGY)

PART İC İPANTS WERE ASKED TO CHOOSE BETWEEN TWO TREATMENTS FOR 600 PEOPLE AFFECTED BY A DEADLY D İSEASE

EXAMPLE 1


RESEARCHTREATMENT A WAS PRED İCTED TO RESULT İN 400 DEATHS, WHEREAS TREATMENT B HAD A 33% CHANCE THAT NO ONE WOULD D İE BUT A 66% CHANCE THAT EVERYONE WOULD D İE


POSITIVE FRAMING

TRATEMENT A SAVES 200 L İVES

TRATEMENT B A 33% CHANCE OF SAV İNG ALL 600 PEOPLE, 66% POSS İB İL İTY OF SAV İNG NO ONE


NEGATIVE FRAMING

TRATEMENT A 400 PEOPLE W İLL D İE

TRATEMENT B A 33% CHANCE THAT NO PEOPLE W İLL D İE, 66% PROBAB İL İTY THAT ALL 600 W İLL D İE


TRATMENT ACHOSEN BY 72% WHEN PRESENTED W İTH POS İT İVE FRAM İNG


CHOSEN BY 22% WHEN PRESENTED W İTH NEGATIVE FRAM İNG

BOOKING.COMHTTP://BLOG.BOOK İNG.COM/HAMBURGER-MENU.HTML

1

2

BOOKING.COM

1

2

HTTP://BLOG.BOOK İNG.COM/HAMBURGER-MENU.HTML

NEUTRAL RESULT

SUMMARY

DON’T BELIEVE THE GURUSWHAT WORKS FOR “EVERYBODY” MAY NOT WORK FOR YOU

SUMMARY

DON’T BELIEVE THE HiPPOHiPPO DON’T THINK. THEY USED TO. YOU ARE BEING PAID TO DO IT

SUMMARY

DON’T BELIEVE IN YOUYOU DON’T KNOW WHAT CONSUMER WANTS. YOU JUST HAVE HYPOTESHIS

SUMMARY

BELIEVE IN DATAIT NEVER LIES

SUMMARY

BE OPEN TO CHANGEEXPERIMENT OFTEN , FAIL FAST , BELIEVE IN DATA

SUMMARY

TESTEVERYTHING

THE MOST IMPORTANT SLIDE EVER

@ [email protected]

CAIOGONDIM .COM

@CAIO_GONDIM

CONTACT

Date post:	26-Jun-2015
Category:	Software
Upload:	caio-gondim
View:	244 times
Download:	0 times

A/B Test

Software