Date post: | 06-Apr-2018 |
Category: |
Documents |
Upload: | henry-bendik |
View: | 217 times |
Download: | 0 times |
of 12
8/3/2019 Conversion Architecture Whitepaper
1/12
IImmppoo
rroovv
hhii
vv YYooleeRRrrssii
uurrdduu
onnnnllii
iinngg
W
Pr
Co
AArreePPYYoo
IWhitePparedby:version
chhiirrffoo
rrSS
per
DougSchrchitectu
tteerrmma
eenn
usteExpert,
ttuunnccee
WSI
ee::
8/3/2019 Conversion Architecture Whitepaper
2/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page2of
IInnttrroodduuccttiioonn
Researchhasshownamajordifferenceinconversionandreturnoninvestmentbetweencompanies
that
test
their
online
communications
and
those
who
dont.
Sowhydoesnteveryonlinebusinesshavearegulartestingprogram?
Thefactisthatmanybusinessesandmarketersareconfusedaboutwheretobegin,andhow.Somethat
dotestdontdoitoftenenough,orarenotsurehowtointerprettheresults.
Withsomanypotentialareastotest,includinghomepage,landingpages,payperclickads,emailsand
nowtheinfluenceofsocialmedia,marketersneedtoknow:
Whichtestsprovidethebiggestpotentialreturnoninvestment Howtostructuretestsforconsistencyandaccuracy Howtocontinueimprovingtestresultsandduplicatesuccess
Firstofallyouneedasolidbenchmarkingframeworkforbasiconlinetestingthatenablesyoutorealize
significantgainsinconversionandrevenue.
Wehopethisoverviewhelpsyourbusinessandifyouhavenotyetimplementedaformalprogramas
wellasthoselookingforhigherconversions,werecommendaWSIBenchmarkingProgramtoguideand
improveyourefforts.PleasecontactaWSIMarketingConsultanttoday!
Thereis
only
one
key
thought
process
you
need
to
act
on
to
increase
your
online
conversion
rate
and
thatissimplyTest,TestandTest.
Withinthattestingthereare4keyareasyouneedtounderstandtoincreaseyouronlineconversionand
reduceyouroverallcostperlead.
AskTheRightQuestions MeasureTheRightItems PerformTheRightTests Analyze&MakeAdjustmentsBasedOnResults
Thesetests
will
give
you
abetter
understanding
of:
your
customer
profiles,
their
online
behavior
patternsandwhatdrivesthemtotakeaction.
8/3/2019 Conversion Architecture Whitepaper
3/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page3of
IIttAAllllBBeeggiinnssWWiitthhTTeessttiinnggTTooEEssttaabblliisshhBBeenncchhmmaarrkkss::
AtWSI,weapplytheconceptofbenchmarkingtoourtestingprocess.
Whatdowemeanbybenchmarking?Atestingprocessthatisuseful,practical,anddeliverspredicable
resultsthatyoucancarryforwardintoasuccessfulmarketingprogram.
Thebenchmarkresultsofouronlinetestingprocessaredirectlyrelatedtofivekeyelements.
ThisformulawascreatedbyMarketingExperimentsandisthebasisforonlineconversiontoday.
Wherein:
u=Utility
q=ResearchQuestion
t=Treatment
m=MetricSystem
v=ValidityFactor
i=Interpretation
Inotherwords,asktherightquestion,measuretherightthing,performtherighttest,andlearnfrom
theresults.
WerecommendthatbusinessesinterestedinestablishingastrongonlinepresenceusetheWSI
BenchmarkingProgramandincorporatetheseelementsintoyouronlinestrategies.
Thereare3mainareasthatneedtobetested,homepage,landingpage,emailandtheimpactofsocial
media.Usingthesetestingtechniquesyoucanseewhatinfluencesaclientorpotentialclientwhenthey
connectwithyouonline.
8/3/2019 Conversion Architecture Whitepaper
4/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page4of
TTHHEEBBEENNCCHHMMAARRKKIINNGGPPRROOGGRRAAMM
11..SSttaarrttwwiitthhtthheerriigghhttqquueessttiioonn..InarecentWebcliniconthissubject,morethan60%oftheaudiencesaidthattheveryfirst
stepinsuccessfulonlinetestingaskingtherightresearchquestionistheirbiggest
challenge.
AllsinglefactorA/Bsplittestsbeginwithwhich?
Marketersmaystartwiththeinterrogativewhat?orwhy?whentheyarebrainstormingon
aninformalbasis,buttheirresearchquestionmustprogresstowhich?inorderforittobe
specificenough
to
be
used
in
aformal
test.
Forexample,Whatisthebestheadline?becomesWhichheadlinewillconvertbest?and
Whatisthebestprice?becomesWhichofthreepricepointsisbestforthisproduct?
22..DDeevveelloopptthheeTTrreeaattmmeenntt((ss))..Avariableisthegeneralelementyouintendtotest.Headline,price,andcopyareallexampleof
variables,andtherearedozensmore.Again,onekeytosuccessfulA/Btestingisknowingwhich
oneyouwanttosingleout.
Avalue
is
the
specific
option
(content,
size/amount,
color,
position)
of
the
variable
you
are
goingtotest.
ThetestControlAwillbeyourcurrentpage,ad,oremail.
ThetestTreatmentorTreatmentsBorCarethepages,ads,oremailscontainingthe
newvaluesofyourvariable.
ValuesbecomeTreatments.WhenanewTreatmentperformsbetter(betterthantheControl,
betterthananotherTreatmentifthereismorethanone),itbecomesthenewControlin
subsequenttesting.
33..DDeetteerrmmiinneetthheerriigghhttmmeettrriiccss..Yourmainobjectivewillbetodeterminethemetricsthatwillbestansweryourprimary
researchquestion(thewhich?),thoughsecondarymetricsmayalsoprovideinsight.
First,establishwhatyouaretryingtoaccomplish.
8/3/2019 Conversion Architecture Whitepaper
5/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page5of
Areyoutryingtoattractmoresubscribers?
Capturemorequalifiedleads?
Obtainahigher
conversion
rate?
ProduceabetterROIonapayperclick(PPC)campaign?
Afterestablishingwhatyouwanttoachieve,choosetheelementyouwishtotestinorderto
selecttheappropriatemetric.
Thereareessentiallyfourelementsyoucanmeasure:
Theamountofactivityonyoursite Thesourceofthatactivity Thenatureofthatactivity Theresultsofthatactivity
Useorganizingquestionstohelpyouchoosetheappropriatemetric;forexample:
Question Metric
Who
visited
my
page?
Number
of
unique
visitors
(UV)
Wheredidtheycomefrom? ReferringURL,organicsearch,PPCad&socialmediainfluence
Whatdidtheydo/view? Pageviewspervisitor;clicks
Whatdidtheybuy/signupfor
ordidtheycall?Numberoforders,averageamount,totalrevenue
44..CChheecckkvvaalliiddiittyy,,ssttaarrttiinnggwwiitthhssaammpplleessiizzeessuuffffiicciieennccyy..Forconfidenceinyourfindings,testresultsmustbestatisticallymeaningful.Inotherwords,
therehastohavebeenenoughdifferenceinresultstomakethemeaningfulrightchoice.
Typically,researcherswilltoleratenomorethana5%chancethatthetestresultsaredueto
randomvariation.Thisisreferredtoas5%significancelevelor95%confidencelevel.For
example,youwouldwanttobeatleast95%confidentthattheresultsofyourtestwilldetecta
differenceinclickthroughrateof0.5%ormore.
8/3/2019 Conversion Architecture Whitepaper
6/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page6of
Keypoint:assamplesize(n)increases,MarginofError(E)declines.
Checkforfouradditional,majorvaliditythreats.
Thereare
anumber
of
threats
to
the
validity
of
testing,
but
these
four
threats
are
common
in
onlinetesting:
Historyeffects Instrumentationeffects Selectioneffects Samplingdistortioneffects
Ensurethatyourtestresultshavenotbeenskewedorinvalidatedbythesethreats.
Historyeffects.Theeffectonatestvariablebyanextraneousvariableassociatedwiththe
passageoftime.Industryevents,newsevents,andholidaysareallexamplesofhistoryeffects
thatcouldskewyourtestresults.
Instrumentationeffects.Theeffectonatestvariablebyanextraneousvariableassociatedwith
achangeinthemeasuringinstrument.Havingaservergodowninthemiddleofyourtestisone
exampleofaninstrumentationeffect.
Whenreviewingyourtestprotocoltoexcludethisthreat,ask:Didanythinghappentothe
technicalenvironment,measurementtools,orinstrumentationthatcouldhavesignificantly
influencedthe
results?
Selectioneffects.Theeffectonatestvariablebyanextraneousvariableassociatedwith
differenttypesofsubjectsnotbeingevenlydistributedbetweenexperimentaltreatments.
Mixingtrafficsourcescanskewresults.Agreatlandingpagecreatedforonespecificchannel
forexample,aPPCadwithgreatcopyrelatedtoaspecificorganicsearchtermcandopoorly
whenpresentedtovisitorscomingfromadifferentPPCadforacompletelydifferentproductor
service.
Sampling
distortion
effects.
The
effect
on
the
test
outcome
caused
by
failing
to
collect
a
sufficientnumberofobservations.
Dependingontheamountoftrafficyourlandingpagenormallygets,thenumberofemail
addressesyouhavetosendyournewemailto,ortheclicksyounormallygetonyourexisting
PPCad,itmaytakedays,weeksorevenmonthstoreachthenumberofobservationsnecessary
toachievetestingvalidity.
8/3/2019 Conversion Architecture Whitepaper
7/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page7of
55..CCoorrrreeccttllyyiinntteerrpprreetttthheerreessuullttss..Ifresultsarevalid,determinetheimpactanddecidewhatyournexttestshouldbe.Remember,
thewinningTreatmentbecomestheControlforyournexttest.
Ifresultsareinvalidorinconclusive,considerrerunningthetest,butkeepinmindthateven
invalidorinconclusivetestscanprovidevaluewhenproperlyanalyzed.Learnfromyourresults:
Expertiseiscumulativeknowledge.Keepcarefulrecordsoftestsandresults.Whenresultsaresurprising,lookforhiddensourcesoferrororinsight.
Lackofdifferenceismeaningful.Thatchangingthevariable(s)inthewaythatyoudidhadlittleeffectonperformanceisvaluableknowledge.Thismaybethesinglemostvaluableinsightyou
gainfromaninconclusivetest.
Try
stratification.
Try
filtering
the
results,
looking
for
patterns
(days
of
the
week,
hours
of
the
day,etc.).Youmayfindaclearstatisticalwinner.
Notealternativeorsecondaryconclusions.Lookforpatternsofperformance,eventhosethatidentifythingsthatyoudefinitelyDONTwanttodo.
BBrreeaakkiinnggDDoowwnnYYoouurrCCaammppaaiiggnnssaannddMMaarrkkeettiinnggCChhaannnneellss
HHeellppssttooDDeetteerrmmiinneetthheeFFoolllloowwiinngg::
Whoyourclientsare,whattheirneedsandwantsareandwhatdrivesthemtotakeaction.Theseare
thekeytoanyonlinemarketingprogram.Thediagrambelowillustratesastandardconversionfunnel.
8/3/2019 Conversion Architecture Whitepaper
8/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page8of
Testingallowsyoutoidentifyyourclientsorpotentialclientsprofileswhichultimatelyleadstohigher
conversions. Understandingeachpotentialaudienceallowsyoutocommunicatetherightmessageat
therighttime.
Understandinghowyouraudiencemakestheirdecisionisveryimportant.Thegraphicbelowillustrates
astandardbuyingprocessinaB2Benvironmentbuyingprocess.
8/3/2019 Conversion Architecture Whitepaper
9/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page9of
TTeessttPPrroottooccoollffoorrAA//BBSSpplliittTTeessttiinnggffoorrLLaannddiinnggPPaaggeess::
11..DDeevveellooppYYoouurrCCaappaabbiilliittiieessaannddSSeelleecctttthheeRRiigghhttTToooollssA/BsplittestingtoolsvaryfromsimpleCGIscriptstosophisticatedsoftwareapplications.You
willfindalistofservicesintheResourcessectionattheendofthisreport.
EvenwithoutsophisticatedA/Btestingcapability,sequentialtestingoffersyouanopportunity
tolearnmanyinsightsaboutyourpages.Formoreonsequentialtesting,seetheConcluding
Comments,below.
22..IIddeennttiiffyyYYoouurrEEssttaabblliisshheeddCCoonnttrroollPPaaggeeYourcontrolpagewillbethepageagainstwhichyoutestallsubsequentoptimizationefforts.If
youare
just
getting
started
with
A/B
testing,
your
control
page
will
be
your
current
landing
page
beforeanyoptimization.Whenanewpageperformsbetterthantheexistingcontrolpage,it
thenbecomesyourcontrolpageinsubsequenttesting.
33..EEssttaabblliisshhYYoouurrTTeessttiinnggGGooaallssaannddPPaarraammeetteerrssWhatareyoutryingtoaccomplishwithA/Bsplittesting?Areyouaftermoresubscribers,a
higherconversionrate,oragreaterreturnoninvestmentonyourPPCcampaigns?Yourgoals
willdetermineyourtestingparameters,whichwilldeterminethepotentialsuccessofyour
testingefforts.
44..DDeetteerrmmiinneeYYoouurrSSuuffffiicciieennttTTeessttIInntteerrvvaallThistimeperiodshouldallowyouenoughtimetogathersufficientdatatogaugerealinsight
aboutyourA/Btests.Identifythenumberofuniquevisitorsand/orconversionsneededto
establishgooddataandthendeterminehowlongitwilltakeyoutogeneratethistraffic.This
numberwillvaryfrombusinesstobusiness,butshouldgiveyouenoughdatatoconfidently
declarea"winner."
55..CCrreeaattee1133RRaaddiiccaallRReeddeessiiggnnssKEYPOINT:Thesepagesarenotsubtleoptimizationschangingonlyoneortwoelementsonthe
page,butarewhollydifferentpagesrepresentingaradicallydifferentapproach.
66..EEvvaalluuaatteeTThheesseeRReeddeessiiggnnssiinnAA//BBSSpplliittTTeessttssTestthesealternatelandingpagesagainstthecontrolpage.Ideally,eachpagewillbetested
againsteveryotherpage,butifthatisimpractical,testtwopagesatatimeandkeepthebestas
8/3/2019 Conversion Architecture Whitepaper
10/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page10of
yourcontrolforsubsequenttesting.Weliketouseheatmapstohelptracktheimpactofallthe
keyelementsanditspotentialimpact.
77..BBaasseeddoonnRReessuullttss,,DDeetteerrmmiinneeYYoouurrTTrruueeCCoonnttrroollPPaaggeeTheradicalredesignmethodwillbemorelikelytogenerateaquantumleapinimproved
conversionrate
than
optimizing
amediocre
page
with
little
potential.
Once
you
have
identified
thebestgeneralapproach,youarenowreadytooptimizeindividualelementsonthepage.
88..OOppttiimmiizzeewwiitthhTTrraaddiittiioonnaallVVaarriiaabblleeSSppeecciiffiiccAA//BBTTeessttiinnggVVaarriiaabblleessttooTTeesstt::
Headline CalltoAction PageCopy Graphics Color ConfigurationofPageElements Etc.
8/3/2019 Conversion Architecture Whitepaper
11/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page11of
CCoonncclluuddiinnggCCoommmmeennttss::
ThereareafewimportantpointstorememberaboutA/Bsplittesting:
1. Evenifyoucan'tsetupatrueA/Bsplittest(wheretwoversionsofapagearebeingdisplayedoneafteranothertoalternatevisitorstoyoursite),itisveryeasytocreateasequentialA/Bsplit
test.
KEYPOINT:Asequentialtestiswhenyoushowoneversionofapageforacertainperiod,like
twodaysoraweek,andthenshowanotherversionforthefollowingtwodaysoraweek.The
resultsmaybealittlelessreliable,butcanstillyieldvaluableinformationandtrends.
2. Testinggivesyoutheopportunitytomaximizeconversionrates,solveproblems,andchallengeassumptions.Andkeepinmindthatyouhaveopportunitiesbeyondtestingsmallchangestoa
page.
Youcanalsochallengeanexistingpagebydesigningandwritingaradicallydifferentversion,
wherealmosteverythingisdifferent.Infact,itisthroughthesedramaticallychanged
approachesthatyouaremostlikelytoachievebreakthroughimprovements.
3. Testingprovidescompanieswithaninvaluablemeanstodemonstratetocompanyheadsandmanagementthehardfiguresbehindanysuggestedchangesorimprovements.
4. Useofconsistenttestingwillincreasetheknowledgebaseofyourwebgrouporcompanysignificantly.Youwilllearnmore,andsoonbeabletodetermineasetofoptimizedpractices
thatworkbestforyourparticularbusiness.
Theabsenceofrigoroustestingleavesyouinthedark,dependingonguessworkalonewhen
creatingyourpages.
CCoonncclluussiioonn::
Mostbusinessestodaydonottakeadvantageofhowimportantitistotestregularly.Thekeyforany
businesswantingtoachieveonlinesuccessisknowinghowtocommunicatetoyourtargetaudienceand
knowingthatallpartiesdonotactonthesameprinciples.Thisalsoallowsyourtofigureoutyouronline
adspendinaneffectivemanner.Theaverageonlineconversiontodayis2%,whichmeansthat98%of
youradbudgetisnotbeingusedeffectively.
Thekeyisreallytokeepthetestingsimpleanduncomplicatedandonceyouhaveestablishedyour
benchmarksthenyoucanstarttodivertfromyourControlPage.Thiscanbeasimpleprocessandora
morecomplicatedonebutthekeyissimplytesting,testingandmoretestingoftherightthingsandyou
willincreaseyouronlineconversion.
8/3/2019 Conversion Architecture Whitepaper
12/12
CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd
Copyright2009byResearchandManagement.Allrightsreserved. Page12of
AAbboouutttthheeAAuutthhoorr
DougSchustsbackgroundconsistsofbothonlineandtraditionalmarketing.Hismain
focusisonlinemarketingandhelpingclientsfocusnotontrafficbuttheresultsfrom
thattraffic.MostrecentlyDoughasspokenatseveralGoogleeventsonconversion
andcontinuestosharehisknowledgeandexperiencewithotherconsultantsand
clients.
RReessoouurrcceess::
MarketingExperiments MarketingSherpa Omniture Google