A/B TESTINGIn data we trust!

Pedro Marques CI&T Digital Day

PEDRO MARQUESUX Designer @ booking.com @pmarquees

700kProperties

135Offices

207Countries

42Languages

74Currencies

8.5kEmployees

8kServers

perl + html templatesCore

3º MAIORe-commerce do mundo

A/B TestingWTF?

Emojiana

Emojiana Hippo

50%

50%

11%

2%

Born at GoogleEarly 00’s

PharmaceuticalTo test new drugs

But why A/B Test?

No Regression

But bear in mind:

A/B Test will prove you wrong

Data never lies

Define your KPI’s

Correlation does NOT imply causation

Conver$ion

User performs a given action

Define your KPI's

What should I test?

Test all your hypothesis

Assumption is the mother of all fuck ups

Framing Effect

A/B Testing Kills creativity

Junior designers X Sr. Designes

https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e

Would like fries with that?http://blog.booking.com/hamburger-menu.html

No significant impact.At all.

NICE!But how do I do that?

Coin flip

https://www.optimizely.com

Now what?Data analysis is the key

Some advicesThese are mine, test and see if it works for you =)

Shit X ShitNope, a big nope.

Ask questionsEven before starting the experiment

Dont trust the gurusThis is science, not religion

Flip the coin properlyCalculate the amount of time and sample that you need

https://vwo.com/ab-split-test-duration/

A/B Testing is a big subject

Link

Thank you folks! <3hey@pedromarques.io@pmarquees