Data Driven Product Development

Ishan AgrawalFounder, Customator

@ishanagr

About Me

• Founder, Customator

• Data Engineer @ Viki, a Rakuten Company

• Research Intern @ Paypal

• Teaching Assistant @ SoC

• Software Engineer @ Rocketful Inc.

Fun stuff I didProducts I built

• Tuition Agency for Indian education boards

• FML NUS

• CorruptionTrak

• Devops as a Service

Overview• Early Products and Lessons

Learnt

• Data Driven Product Development

• What?

• Why?

• How?

• Data @ Viki

• Why?

• How?

• Impact?

• Customator

• Activity

Data Driven Product Development

• Lets say we are building ‘NUS TimeTable Builder’• Problem?

• Solution?

• Is it working?

• Can we make it better?

Is it working?• What should we measure?

Can we make it better?

• What should we measure?

Can we make it better?

• What do we do to make it better?

Data @ Viki

Data Products @ Viki• Dashboard - easy reporting

• AB testing

• Personalisation

• Real time charting and alerting

• Data Science

Data Products @ Viki

AB Testing

AB Testing types• Split url testing

• Landing page testing

• Multi page testing

• Multi variate testing

AB Testing pitfalls• Dilution

• Statistical Significance

• AB tests don’t always win

• Novelty effect

• External factors

• Choosing your goals wisely

Subscription Service Info page

Homepage Personalisation

User engagement - Favorite button

Personalisation

Experimentation and Personalisation made

easy• Cross Platform

• Experimentation in algorithms, not just UI

• Easy to use for non-technical users

• Auto personalisation

• Data Driven experimentation

Activity

• Lets pick a popular product of your choice

• Facebook? Quora?

• Brainstorm through the goals of the product

• What should we measure?

• How can we improve the product?

Thanks@ishanagr

agrawal.ish@gmail.com