TESTING“MARKETBEATING”SCHEMESANDSTRATEGIES

2

TestingMarketEfficiency

• Testsofmarketefficiencylookatthewhetherspecificinvestmentstrategiesearnexcessreturns,afteradjustingforrisk.Sometestsalsoaccountfortransactionscostsandexecutionfeasibility.

• Atestofmarketefficiencyisajointtestofmarketefficiencyandtheefficacyofthemodelusedforexpectedreturns.Whenthereisevidenceofexcessreturnsinatestofmarketefficiency,itcanindicatethatmarketsareinefficientorthatthemodelusedtocomputeexpectedreturnsiswrongorboth.

3

Benchmarkstoassessperformance

¨ Comparisontoindices:Comparetoreturnsyouwouldhavemadebyinvestinginanindex,withoutadjustingforrisk.

¨ RiskandReturnModels:Youcanadjustforrisk,whenmakingyourcomparison:¤ MeanVarianceMeasures

n SharpeRatio:AverageReturn/StandarddeviationofReturnsfromStrategyn InformationRatio:(ReturnonStrategy– ReturnonIndex)/TrackingErrorversus

theIndex¤ CAPMbasedmeasures

n Jensen’salpha=Actualreturn– ExpectedReturn(fromCAPM)n TreynorIndex=(ReturnonStrategy– RiskfreeRate)/Beta

¤ ArbitragePricingandMulti-factorModels¤ ProxyandCompositeModels

4

Reviewingthechoices…

5

TestingEfficiency:Threeforms

¨ Inaneventstudy,weexaminewhethermarketsefficientlyincorporatethe”news”inanewsstory(earningsannouncement,acquisitionorevenaFederalReserveinterestratechange).

¨ Inaportfoliostudy,weexaminewhetherinvestinginangroupofassets/companiesthatsharethesamecharacteristic(lowPEratio,highdividendyieldsetc.)generateshigherreturnsthanitshould,giventheriskintheinvestments.

¨ Inaninvestorgroupstudy,weevaluatewhetheragroupofinvestorswhosharethesamecharacteristic(valueinvestors,hedgefunds,growth-orientedmutualfunds)beatthemarket,afteradjustingforriskandtransactionscosts.

6

1.EventStudy

• Aneventstudy isdesignedtoexaminemarketreactionsto,andexcessreturnsaroundspecificinformationevents.

• Theinformationeventscanbemarket-wide,suchasmacro-economicannouncements,orfirm-specific,suchasearningsordividendannouncements.

• Theobjectiveistoexaminewhethertheeventcausesstockpricestomoveabnormally(upordown).

7

Stepsinconductinganeventstudy

1. Specifythe“event”thatyouaretestinganddentifythe“time”theeventoccurred

2. Returnsarecollectedaroundthesedatesforeachofthefirmsinthesample.

a. Decideontimeintervals(hourly,daily,weekly)b. Determinehowmanyintervalsbeforeandafterevent.

3. Adjustthereturnsformarketperformanceandrisk,i.e.,estimateexcessorabnormalreturns.

4. Estimatetheaverageandstandarderrorinthesereturns.¤ Checkforstatisticalsignificance(Tstatistics,forexample)¤ Checkforeconomicsignificance(Areexcessreturnslarge

enoughtocoverexecutiondifficultiesandcosts?)

8

AnExample:TheEffectsofOptionListingonStockPrices

Step1:ThedateonwhichtheannouncementthatoptionswouldbelistedontheCBOEonaparticularstockwascollected.Step2:Thereturnsoftheunderlyingstock(j)werecomputedforthetendaysprior,thedayof,andeachofthetendaysafterannouncementStep3a:Thebetaforthestock(bj)wasestimatedusing100tradingdaysfrombeforetheeventand100tradingdaysaftertheevent.Thereturnsonthemarketindex(Rmt)werecomputedforeachofthe21tradingdays.Step3b:Excessreturnswerecomputedforeachofthetradingdays:

Erj,t =Rj,t - bj Rm,t .......... t=-10,-9,-8,....,+8,+9,+10Step4:Theaverageandstandarderrorofexcessreturnsacrossallstockswithoptionlistingswerecomputedforeach

9

TheResultsoftheStudy

10

2.PortfolioStudy

¨ Insomeinvestmentstrategies,firmswithspecificcharacteristicsareviewedasmorelikelytobeundervaluedorovervalued,andthereforehaveexcessreturns,thanfirmswithoutthesecharacteristics.¤ Inthesecases,thestrategiescanbetestedbycreatingportfoliosoffirmspossessingthesecharacteristicsatthebeginningofatimeperiod,andexaminingreturnsoverthetimeperiod.

¤ Toensurethattheseresultsarenotcoloredbytheidiosyncraticbehaviorofanyonetimeperiod,thisisrepeatedforanumberofperiods.

¨ Finally,tomakesurethatyourresultsarenottheresultofdatamining,youtryyourstrategyonaholdoutperiod.

11

StepsinDoingaPortfolioStudy

1. Thevariableonwhichfirmswillbeclassifiedisdefined,usingtheinvestmentstrategyasaguide.Thedataonthevariableiscollectedforeveryfirminthedefineduniverseatthestartofthetestingperiod,andfirmsareclassifiedintoportfoliosbaseduponthevariable.

2. Thereturnsarecollectedforeachfirmineachportfolioforthetestingperiod,andthereturnsforeachportfolioarecomputed.

3. The“risk”ofeachportfolioisestimated,usingoneoftheriskandreturnmodels.

4. Theexcessreturnsandstandarderrorsearnedbyeachportfolioarecomputed.

5. Usestatisticalteststoseeiftheexcessreturnsaredifferentfromzero.Theextremeportfolioscanbematchedagainsteachothertoseewhethertheyarestatisticallydifferent.

12

TestingalowPEstrategy

1. UsingdataonPEratiosfromtheendof1987,firmsontheNewYorkStockExchangewereclassifiedintofivegroups,thefirstgroupconsistingofstockswiththelowestPEratiosandthefifthgroupconsistingofstockswiththehighestPEratios.Firmswithnegativeprice-earningsratioswereignored.

2. Returnsoneachportfoliowerecomputedannually from1988to1992.Stocksthatwentbankruptorweredelistedwereassignedareturnof-100%.

3. Thebetasforeachstockineachportfoliowerecomputedusingmonthlyreturnsfrom1983to1987,andtheaveragebetaforeachportfoliowasestimated.Theportfolioswereassumedtobeequallyweighted.The returnsonthemarketindexwascomputedfrom1988to1992.

4. Theexcessreturnsoneachportfoliowerecomputedusingthebetasfromstep3andmarketreturnsfromstep4.

13

LowPEStrategy:ExcessReturns

P/E Class 1988 1989 1990 1991 1992 1988-1992Lowest 3.84% -0.83% 2.10% 6.68% 0.64% 2.61%2 1.75% 2.26% 0.19% 1.09% 1.13% 1.56%3 0.20% -3.15% -0.20% 0.17% 0.12% -0.59%4 -1.25% -0.94% -0.65% -1.99% -0.48% -1.15%Highest -1.74% -0.63% -1.44% -4.06% -1.25% -1.95%

Extreme portfolio testThe lowest PE portfolio earned 4.56% more than the highest PE portfolio. (2.61% - (-1.95%)) = 4.56%

Don’t forget your statistics. Every one of these numbers has a standard error attached to them and you can compute statistics (t, F) that will tell you whether these numbers are statistically significant.

14

Regressions

¨ Oneofthelimitationsofportfoliostudiesisthattheybecomeincreasingunwieldy,asthenumberofvariablesthatyouuseinyourstrategyincreases.

¨ Theotherproblemwithportfoliostudiesisthatyougroupfirmsintoclassesandignoredifferencesacrossfirmswithineachclass.Thus,thestocksinthelowestPEratioclassmayhavePEratiosthatrangefromthe4to12.

¨ Ifyoubelievethatthesedifferencesmayaffecttheexpectedreturnsonyourstrategy,youcouldgetabettermeasureoftherelationshipbyrunningamultipleregression.Yourdependentvariablewouldbethereturnsonstocksandtheindependentvariableswouldincludethevariablesthatformyourstrategy.

15

Runningaregression

1. Independentvariable:Thisisthevariablethatyouaretryingtoexplain.Inmostinvestmentschemes,itwillbeameasureofthereturnyouwouldmakeontheinvestmentbutyouhavetodecidehowyouaregoingtomeasurereturns(totalorexcess)andhowoften(daily,weekly,quarterly).

2. Dependentvariables:Thesearethevariablesthatyouthinkwillhelpyoufind“better”investments.Iftheyarequantitative(PEratios),youareset.Iftheyarequalitative(goodmanagement),youhavetocomeupwithaquantitativemeasureofthevariableatthebeginningofeachperiod thatyouarecomputingreturns.

3. Linearitycheck:Runscatterplotsforeachvariableagainstindependentvariabletoseeifrelationshipislinearornot.

4. Runtheregression:Youcaneitherruncrosssectionalregressions(acrossfirms)ortimeseriesregressions(acrosstime)

5. Checkforstatisticalsignificance:ChecktheR-squaredfortheregressionandthetstatisticsforthecoefficients.

16

3.InvestorGroupStudies

¨ Inthesestudies,youfocusonaninvestorgroupthatsharesacommoncharacteristicandexaminewhetherthereturnsdeliveredbythatgrouparehigherthanwhattheyshouldhaveearnedonarisk-adjustedbasistobreakeven.

¨ Sincethereturnsreflectexecutiondifficultiesandtransactionscosts,thisisamoredifficulttestformarketstopasstobedeemed“efficient”.

¨ Ineffect,youareexaminingwhetherstatisticalproofofmarketinefficiencytranslatesintoeconomicinefficiency.

17

Stepsininvestorgroupstudies

1. Identifythegroup:Specifythecommoncharacteristicsofthegroup.Insomecases,thiswillbeeasierthanothers.

2. Avoidsurvivorbias:Sincetheworstoftheinvestorsinyourspecifiedgroupmayfail,youshouldlookatallinvestorsinthegroupduringtheperiodofyourassessment,notjustsurvivors.

3. Measureactualreturns:Iftheinvestmentsareinmarket-tradedassets,thiswillbeeasier.Ifnot,bewaryofself-reportedreturns.

4. Evaluaterisk:Lookatboththeriskoftheunderlyinginvestments,aswellasthevariabilityintotalreturnsreportedforportfolios.

5. Calculateexcessreturns:Convertriskmeasureintoexpectedreturns,andcomputeexcessreturnasactualreturnminusexpectedreturn.

18

Example:Hedgefunds

1. Identifythegroup:Hedgefunds.2. Avoidsurvivorbias:Lookatallhedgefundsthatwereopenfor

investmentonJanuary1,20143. Measureactualreturns:Computereturnsonthesefundsevery

yearfrom2014-2018.Ifafundfails,computethereturnstoinvestorsinthatfundduringthefailedyear.

4. Evaluaterisk:Measurethevariabilityinhedgefundreturnsovertime.IfyouareusingtheCAPMoramarket-basedmodel,computethecorrelation(andbeta)ofthesereturns,relativetoamarketindex.

5. Calculateexcessreturns:Convertriskmeasureintoexpectedreturnseachyearandcomputeexcessreturnsforeachfund,eachyear.Computestatisticstoseeiftheaverageexcessreturnacrossfundsisdifferentfromzero.

19

TheCardinalSinsinTestingStrategies

1. Using'anecdotalevidence’:Anecdotescanbetailoredtocometoanyconclusion.

2. Noholdoutperiods:Aninvestmentschemeshouldalwaysbetestedoutonatimeperioddifferentfromtheoneitisextractedfromoronauniversedifferentfromtheoneusedtoderivethescheme.

3. SamplingBiases:Ifyoursamplingisbiased,itcanprovideresultsthatarenottrueinthelargeruniverse.

4. Failuretocontrolformarketperformance:Whentheoverallmarketisdoingwell(badly),allstrategieslookgood(bad).

5. Failuretocontrolforrisk:Afailuretocontrolforriskleadstoabiastowardsacceptinghigh-riskinvestmentschemesandrejectinglow-riskinvestmentschemes.

6. Mistakingcorrelationforcausation:Statisticaltestsoftenpresentevidenceofcorrelation,ratherthancausation.

20

OtherSins

1.DataMining:Theeasyaccesstohugeamountsofdataisadouble-edgedsword.Whenyourelatestockreturnstohundredsofvariables,youareboundtofindsomethatseemtopredictreturns,simplybychance.2.SurvivororSurvivalBias:Ifyoustartwithaexistinguniverseofpubliclytradedcompaniesandworkbackthroughtime,youcreateabiassinceyoueliminatefirmsthatfailedduringtheperiod.Ifthetestedstrategyissusceptibletopickingfirmswithhighbankruptcyrisk,thismayleadtoanoverstatementofreturnsonthescheme.3.NotallowingforTransactionsCosts:Someinvestmentschemesaremoreexpensivethanothersbecauseoftransactionscosts- executionfees,bid-askspreadsandpriceimpact.4.Notallowingfordifficultiesinexecution:Somestrategieslookgoodonpaperbutaredifficulttoexecuteinpractice,eitherbecauseofimpedimentstotradingorbecausetradingcreatesapriceimpact.

21

Askeptic’sguidetoinvestmentstrategies

1. Cantheinvestmentstrategybetested?Canitbeimplemented?

2. Ifthestrategycanbetested,isthetestthathasbeendevisedafaironeofthestrategy?

3. Doesitpasstheeconomicsignificancetests?4. Hasitbeentriedbefore?Thereistruthtothesayingthatalmosteverythingthatismarketedasnewanddifferentininvestinghasbeentriedbefore,sometimessuccessfully,andsometimesnot.