15
Data Driven Product Development Lessons learnt in a startup environment by @will_ogden, CTO at TrustedHousesitters.com
DataDrivenProductDevelopment
Lessonslearntinastartupenvironmentby@will_ogden,CTOatTrustedHousesitters.com
AboutTrustedHousesitters
• Enablespetownerstofindasittertocarefortheirhome&petsforFREE,inexchangeforaplacetostay• Largestglobalhouse&petsittingbusiness• Membersinover140countries• Facilitatedover1mnightsofhouseandpetsittingtodate• Pioneeringanewindustrymeanslessestablishedbenchmarks;thereforearealfocusondatadriveninsightsandproductdevelopment
Keytousingdatainproductdevelopmentistounderstandwhatsuccesslookslike…• Areyoutryingtodriveengagement?Areyouwantingtoincreaseconversion(probably)?Areyouwantingtoreducechurn?• Ensurethateveryoneisusingthesamemetricstomeasurethesesuccessfactors.• Alwayslookatthemetricsholistically.
A/Btestingandthefollyofmarginalgains
• A/Btestingsoftwarehasbecomeincreasinglyeasytouse.Veryattractivetoastartupduetoeaseofactivationandlowcost.Youcanreacheasilyasegmentable,globaluserbase.• Assistsinshiftingproductdevelopmentconversationsfrom“wethink”to“weknow”• PosterchildofA/Btestingsoftware…the“calltoaction...”
Focusofourattention
Testinground1...
Testinground2...
• Ourtestingcenteredaround“microconversions”.Improvingindividualsectionsandinteractionsofthesiteiteratively.• Mantraofmarginalgainsmakealotofthingsalittlebitbetter...however...
• There’sabigassumptionthatthese“microconversions”areisolated...inmostuserjourneyshoweverthisisalmostneverthecase...changingtrafficpatternsanduserjourneyshigherupthefunnelcanimpactengagementfurtherdown...potentiallywipingoutpreviousgains.• Imperativetohaveastructuredplanaroundtesting.Startatthetopofthefunnelandworkdown.Seewheretheopportunitiesare.• Candefinitelyimproveexperienceandoptimizeconversion...butgenerallymoreevolutionratherthanrevolution.• Forlarger“stepchange”or“revolution”functionality,A/Btestingisstillagoodfittotrialnewfunctionalitywithoutrisk.
Usertestingandcustomerfeedback
• Remainingcustomercentricthroughrapidgrowthisachallenge;butit’sparamounttostayingfocusedondeliveringwhatthecustomerwants…especiallyasheadcountincreases.• Usertestingandresearchisgreatingettingdataabouthowreallifehumansareengagingwithyourproduct.Variouswaysofcapturingthisdatae.g.remoteobservations,surveys,interviews,etc.• Importanttopickthecorrectaudience– askingprofessionalusabilitytestersmaygivesomegoodinsights…howevertargetaudiencemaybequitefarremovedfromthise.g.notastechnicallysavvy.• Remembertheobservereffect.Someofthebestinsightswegleamed,usedremoteobservationsoftware.Unmoderatedrealworldinteractionsfromactualmembersusingtheirowndevice.
Bakinginfutureinsightrequirementsandmakingdatasustainable• Whentryingtounderstandcustomerneeds,typicallywe’reanalyzingeverythingweknowaboutthecustomersintentionsandbehaviors.Thisdataisofteninseparatesiloswhichneedexploring• There’sa(slightlydubious)analogythatdataisthenewoil…
Data
Typicaldatascientists
• Foragivenproductecosystem,therewillbeafiniteamountofreportabledata.Asthelowhangingfruitispicked,findingpatternsandbehaviorswithinthedatabecomesmoreofachallenge…
• Torealizemoresourcesofdata,sometimesthelowtechsolutionisbest…youdon’talwaysneedtoreachforthelatestDeepLearningsoftware...
• Whenattheideagenerationanddesignstage,factorinwhatdatacouldbecapturedtoprovidefutureinsightsthatcouldfuelfutureproductdevelopment.
• Putincreasedfocusontheacquisitionofdataaswellasuserrequirements…giveyourDataScienceteamsmoredatatoanalyze.
• Augmentexistingdatawithrelateduserbehaviourdata.Manyofftheshelfsolutions,butwe’vehadsuccessinbuildingourowninhouse...providesusbothreportinginsightsanddrivescustomizedcontentwithinthesite
Thevalueofdataisintheinsights
• Ifyou’retakingadatacentricapproachtoproductdevelopment,andaremakingcommercialdecisionsbasedondatait’simperativethatthereisarobustQAprocessaroundthevalidityofthedata.• Importanttochallengefindingsofqualitativedatawithsupportingquantitativedata.Usequalitativedatatocreatehypothesisandquantitativetotestthem.Theyworkbestintandem.
Thankyouandhappyhousesitting!
• Usepromotioncode“EYEFORTRAVEL”for1monthfreehome&petownermembership!
