Journal of Monetary Economics 24 (1989) 455-470. North-Holland

MARKET RATIONALITY TESTS BASED ON CROSS-EQUATION RESTRICTIONS

Benjamin RUSSO, John M. GANDAR, and Richard A. ZUBER*

Ut~r~rsrtr of North Curolw~ at Charlotte, Charlotte. NC 28223. USA

Received November 1988, final version received July 1989

The information used by researchers in cross-equation tests of rational expectations may be a small subset of the mformation available to the market. Such tests produce consistent estimates, but have low power We develop a mechanism for dealing with this problem and use the procedure to perform cross-equation tests on predictions from the football bettmg market, The tests fail to reJect rationahty. We provide evidence that market predictions from betting markets are not formed rationally. It appears, therefore, that cross-equation tests have low power even if the information sets used by the researcher and the market are approximately the same.

1. Introduction

The propositions that market expectations are rational forecasts, that capital markets are efficient, and that monetary policy is neutral in the short run, share the axiom that information use by economic agents is optimal. But the empirical literature on the three propositions forms three strands that are surprisingly diverse. Abel and Mishkin (1983) draw these strands together to provide an integrated view of empirical tests of the axiom that information is rationally used by market participants. The element common to these tests is the cross-equation restriction.

In the context of tests of rationality of market forecasts, the cross-equation restriction requires that market forecasts depend upon available information in the same way that ex post market outcomes do. If cross-equation restric- tions fail to hold, the market forecast is not a rational forecast. This test provides consistent estimation even if the information set used by the re- searcher is only a subset of the information set contained in the market’s forecast. The test may have low power. It will fail to detect inefficient use of information contained in the market’s forecast that is not used by the researcher. In this situation the probability of type II errors is high.

*We wish to thank George Ignatin for suggesting the use of the filter employed in this paper, an anonymous referee for suggesting the out-of-sample tests of section 6, Tom O’Brien for discussion of these issues, and Chris Piros and Hal Stem for helpful criticism and suggestions. David Cooper and Craig Brown provided timely and accurate assistance with data collection and processing.

0304-3932/89/$3.50@1989. Elsevier Science Publishers B.V. (North-Holland)

456 B. Russo et al., Market ratronality tests

In this paper we show how the severity of the problem can be reduced, in principle, when an unbiased nonmarket forecast and a large sample are available. We then apply the technique to data from the football betting market. Neither the original version of the cross-equation test nor our modi- fied version allow rejection of the hypothesis of rationality of expectations in the betting market. Betting simulations, however, indicate existence of unex- ploited profit opportunities in betting markets. These results indicate that the cross-equation tests have less power to reject false null hypotheses than Abel and Mishkin intimate. The tests apparently fail to detect departures from rational use of information in the betting market when the researcher’s information set approximates that available to the market.

The results presented here support similar conclusions drawn in the context of asset markets. Perron and Shiller (1985) examine the power of tests of the random walk hypothesis against the alternative of mean reversion in asset prices. They use the results of Monte Carlo studies to argue that the power of t-tests and normalized beta tests is low; in the case of runs tests, power falls when the number of observations rises while the sampling interval is held fixed. Summers (1986) constructs a simple asset model incorporating autocor- related returns, as a plausible alternative to the hypothesis of efficiency. He shows that both excess returns built into the model and the theoretical autocorrelations in excess returns are too small to be detected given reasonable assumptions on parameters. Summers concludes that standard statistical tests cannot distinguish between efficient asset pricing and at least one plausible alternative pricing mechanism. Poterba and Summers (1988) find evidence in stock prices of short-run deviations from fundamental values, but mean reversion over longer periods. Although variance ratio tests of the joint hypothesis of market efficiency and constant required returns are most power- ful, Monte Carlo results indicate that the variance ratio tests have low power.

A common theme of these authors is that the failure of standard statistical tests to reject the null hypothesis of rationality does not provide evidence for that hypothesis unless it is tested against specific alternatives. They show that standard statistical measures are not powerful enough to reject rationality against some specific alternatives. We suggest the specific alternative hypothe- sis that profits can be earned in the NFL betting market and find evidence for profits. As a result, we treat the failure of the cross-equation test to detect betting market irrationality as evidence, from an actual market, of the low power of these tests.

In the next section we review the cross-equation test of rationality of market forecasts. In section 3, we present results of cross-equation tests on data from the professional football betting market. In the fourth section we describe a forecast model and a filter that are used to modify the sample of data from that market. We show that the filter may be used to reduce the probability of type II errors, and re-perform the cross-equation test on the altered sample.

B. Russo et al., Market ratronalrty tests 451

The fifth section describes betting simulations that uncover irrationality in the

betting market. We use Brown and Maital’s (1981) distinction between com- pleteness and rationality to aid interpretation of the results. Section 6 presents out-of-sample cross-equation tests and betting simulations. These tests confirm the results of earlier sections. Conclusions are presented in section 7.

2. Cross-equation tests

If market forecasts are rational expectations, they will be based on all available information useful in predicting the values of economic variables.’ Rational market forecasts will depend on information available to the market in the same way that values observed ex post depend on that information. Cross-equation tests of market rationality are constructed to test this restric- tion.

Abel and Mishkin describe the structure of cross-equation tests of market rationality in the following way. Let A, represent the value of an economic variable observed at time t. Let +,_i represent the set of information available to the market at t - 1 that is useful in predicting A,. Then, if Xi, I_1 and X 2, I~ 1 are the two mutually exclusive and exhaustive subsets of +z, i,

where E(u,I+,_,) = 0. Xi differs from X, in that only the information in Xi is available for use in econometric tests of market rationality.

Let F,!!‘“, represent a market forecast of A, made at t - 1. Relating F,!!‘“, to the information in $r_i produces

F m = q*x, ,_I + a;x* r_l + u,_1. r-1 (4

The cross-equation restrictions implied by the rational expectations hypothesis require: (i) (pi = (Y:, (ii) (Ye = $. and (iii) E( u,_il$+ i) = 0.2 Subtracting eq. (2) from eq. (1) yields the market forecast error

Rationality implies (pi - a; = 0, (Ye - (Y; = 0, and E(u, - u,_i~$~_i) = 0. Abel and Mishkin point out that in the linear least squares projection

‘This statement IS subject to the implicit, but obvious, quahfication that information whose marginal cost exceeds margmal benefits will be ignored by the market.

*Regarding (ui), E( tl,+ i I+,,- i ) = 0, this is a slightly weaker restriction than might be imposed on the random error a, _ 1. Interpreted in a strict fashion, the hypothesis requires u,_ 1 to be identically equal to zero We follow Abel and Mishkin by imposing the weaker condition.

458 B. Russo et al., Market ratronality tests

ci is an unbiased and consistent estimator of ((or - CY: ) even if X, is nonempty. Abel and Mishkin also point out, however, that hypothesis tests based on eq. (3) have low power if, as we assume, information in X2 cannot be used by the econometrician. We return to the issue of low power in the fourth section.

3. An example from the point-spread betting market

The National Football League (NFL) point-spread betting market provides a convenient context for a test of market rationality based on eqs. (3) and (4). We use the betting line from this market to measure the market’s forecast of the outcomes of professional football games. In our tests we use an ‘opening’ and ‘closing’ betting line. Opening point-spread handicapping is done in Las Vegas by the consensus of a handful of ‘experts’. Following the posting of an opening betting line on a game, betting activity can cause the line to move. The closing line is set as betting comes to an end shortly before games begin.3

Public betting in the point-spread betting market has many characteristics of arbitrage in asset markets.4 The line can be viewed as the betting public’s expectation of the outcome of a football game. If the current line differs from the public’s consensus, it will change until the two become approximately equal. Suppose, for example, that the line has team A favored by three points over team B, but the consensus is that A will probably win by more than three. ‘Investors’ can expect to earn a profit by betting on team A at this line. With the level of betting on A exceeding that on B, A becomes more heavily favored as linemakers adjust the line. Thus, the line comes to reflect the betting market’s expectation. This expectation is conditional on the teams’ past performances and information that becomes available after the posting of the opening line, such as injuries, trades, weather, team morale, and the like.

In two important respects, testing rationality in the football betting market provides a ‘cleaner’ test than tests in standard asset markets. First, in the betting market, whether one wins or loses a bet depends on the line at the time the bet is placed, not on subsequent line changes. Thus, the probability that the actual return equals the expected return is not affected by the public’s evaluation of new information, or by unexpected changes in microeconomic or macroeconomic variables. The football betting data do not suffer from many of the specification problems that arise in financial markets due to the difficulty of measuring risk and changes in the public’s perception of risk.5 A second, and related, advantage of this betting market is that the simple

3For a detailed descrlptlon of the mechanics of the football bettmg market see Gandar, Zuber. O’Brien, and Russo (1988).

4Though it IS certamly not riskless arbitrage. But see Mishkm (1983, p 10).

‘For example. Mishkm (1983) assumes that liquidity nsk and mflation risk are constant during the sample period 1954:1-1976:4 See Fama (1976a) on inflation risk.

B. Russo et ul., Market rutlonalgv tests 459

structure of this market allows us to test market rationality without presuppos- ing an equilibrium model of this market. Hence, using betting market data avoids statistical problems associated with joint tests of rational expectations and a model of market equilibrium.6

Let PS, represent the vector of actual game point spreads observed at time t; let K!,-i be a vector of the betting line (either opening or closing) on each of the football games to be played at t. Replacing A, with PS, and VL,_, with F,?“, in eq. (4) yields

PS*- VL,-, = ;x,.,_,, (5)

where PS, - VL,_ , is a vector of point-spread prediction errors and Xi 1_1 is a matrix of variables useful in explaining the observed point spreads in is,. In our tests, Xi I_ 1 contains yards rushed, yards passed, fumbles, interceptions, number of penalties, the ratio of passing plays to total plays, number of rookies, and wins prior to week t.7 Data used in the tests of this section are from the 1983-85 seasons. We collected data on the variables in Xi from Sporting News, NFL Media Information Guides, individual team media guides, and the Charlotte Observer. Opening betting lines, closing betting lines, and actual game point spreads were taken from College and Pro Football Newsweek&.

If the betting line is a rational forecast of actual game point spreads. ci in eq. (5) will not differ significantly from zero. Panel A of table 1 reports F

statistics for the test of the null hypothesis that the estimated coefficients in 2 are jointly zero for each year separately and for the three years combined. For seven of the eight regressions the null hypothesis cannot be rejected at any conventional level of significance. The exception is 1985, where rationality of the opening line is rejected at a marginal significance level of 4.6 percent. Taken together, these test results provide evidence that the betting line is an efficient predictor of actual game point spreads.

4. The power of cross-equation tests

Recall that the set of information +,,i consists of the mutually exclusive subsets Xi and X2. These differ in that X2 contains a wider information set

than Xi, since X, contains information available to the market that is not available to the econometrician. As Abel and Mishkin point out, rationality

‘On the nature of stattstical problems associated with such tests see Fama (1970,1975). Nelson and Schwert (1977). and Mishkin (1983).

‘We measured each of the vanables, for each team, as an average for all weeks m the season through week t - 1. These variables have been shown to have a great deal of explanatory power m Zuber, Gandar. and Bowers (1985). Below we use the model to produce nonmarket forecasts of point spreads

460 B. Russo et al., Market ratronolity tests

Table 1

F statistics for cross-equation tests of rationality of the football betting market, 1983-85

1983 1984 1985 1983-85

Dependent variable PS-VLO” PS-VLCa PS-VLO PS-VLC PS-VLO PS-VLC PS-VLO PS-VLC

(A) All observationsb F’

nc

(B) X > 5 F

n

(C) x > 4 F

n

(D) X>3 F

n

(E) X z 2 F

n

0.191 0.147 1.040 1.064 2.066d 1.615 112 112 112 112 112 112

0.837 0.700 336 336

0 332 0.375 0.996 1.001 1.800 1330 1.24 0.922 94 92 71 73 107 89 263 254

0.445 0.378 0.672 0.536 1.991 1.252 1.06 0 923 78 87 62 58 86 79 226 224

0 379 0.491 0.572 0.585 1238 1.466 0 997 0.897 60 65 44 44 65 59 169 168

0.864 0.878 1851 2.267 1.713 1.290 1.508 1.956 43 46 29 28 40 60 112 114

aPS-VLO and PS-VLC are the market forecast errors for the openmg and closing lures, retpectively.

Panel A reports statistics on tests that include all observations; panels B through E omit observations in accord with the filter X = IF;?, - I;!!‘r]. [See eqs. (8) and (9) and the surrounding discussion.]

‘F is the F statistic for the null hypothesis that the forecast error is unrelated to any variable in the information set described in the text; n is the number of observations.

dSignificant at the 5 percent level.

tests based on regressions of the market forecast error on Xi may have low power; these tests cannot detect inefficient use of the information contained in X2. In addition, since X2 is omitted from the projection in eq. (4) the standard error of the regression will tend to be high making it less likely that the test results will reject a zero value for the vector of coefficients o. Since X2 may not be empty, the tests reported in the previous section provide weak evidence for rationality in the point-spread betting market.

Low power is not a difficulty in tests of markets where the information available to the market is also available to the econometrician; that is, where X2 is small. However, markets where all qualitative and quantitative informa- tion used by market participants is conveniently available to the econometri- cian do not exist. Hence, a method is required to limit observations used in econometric tests to those where X2 is likely to be small.

This is possible if an unbiased nonmarket forecast is available. Define Fe to be a nonmarket forecast of the variable A in eq. (1). Fe may be a time-series forecast, a survey forecast, or a forecast from a structural or reduced-form econometric model. Assume Fe is unbiased in the statistical sense. In this case

B. Russo et al., Murket rutionnlrg tests 461

it must be related to information in X, in the same way the market forecast (F “) is related to information in X, if F m is a rational forecast. Thus, in the least-squares projection formed at t - 1,

F,‘-, = p^x,.,_,. (6)

p^ should differ from a: of eq. (2) by, at most, a zero mean random variable, 8. That is, p^ = a; + 8. E(b) = E(cy: + e) = a:, and

F’ r-1 = b: + ml.,-1.

To find the difference between the market forecast and the econometric forecast, subtract eq. (7) from eq. (2):

F” - F,‘, = (Y;X~,~-~ - OX,,,_, + u,_~. f-l

The expected value of this difference between forecasts is

E[F,:“, - F,‘,I+,-11 = a:%,,-,.

Eq. (9) indicates that the market forecast and the nonmarket forecast are most likely to differ when the former is based on information not available to the latter. The two forecasts are least likely to differ when the information sets conditioning the two forecasts are similar. Thus, one can hope to reduce the probability of type II errors in cross-equation tests of market rationality by restricting the test sample to those observations for which Fe and F”’ are relatively close. Of course, this requires a large sample. If such tests fail to reject market rationality, one has stronger evidence for market rationality than if the entire sample is used.

We employed this procedure on our data sample from the NFL point-spread betting market. Again, the betting line serves as the market forecast of actual game outcomes. The nonmarket forecasts are based on a regression model developed by Zuber, Gandar, and Bowers (1985; ZGB hereafter). ZGB argue that the final outcome of each football game (the actual point spread) can be viewed as the net result of the efforts of the two teams involved. These net efforts, in turn, can be represented by the game statistics and quantifiable team characteristics. Algebraically,

PS, = pz, + 8,. (10)

where PS, is a vector of actual game point spreads of the games played in week t, p is a vector of coefficients to be estimated, Z, is a matrix of team performance and characteristics variables, and 8, is an error term. The

462 B. Russo et al., Market rationahty tests

variables in Z, are yards rushed, yards passed, number of wins previous to week t, fumbles, interceptions, number of penalties, ratio of passing plays attempted to total offensive plays, and number of rookies.

We develop out-of-sample, linear least-squares forecasts of actual point spreads by estimating p through week t - 1, and forming the projection

For each season, the first forecast was produced after the eighth week of play. To form out of sample projections for the ninth week, we used actual game data from the first eight weeks of the season to estimate p, and team-specific averages over the same period to estimate the performance variables in Z. For

each subsequent week’s projections we updated the variables in Z and the estimate of /3.

Recall, from eq. (9) that the expected difference between the market’s forecast and the projection in eq. (11) will vary with the subset of information that is available to the market but not to the researcher. That is. F,!!!, and Fp_, are likely to differ most when information sets used by the market and the researcher differ and are likely to be similar when information sets are similar. Since our sample is large, we can omit observations for which X = IF,‘- I - &!!!“,I is large, with little sacrifice of degrees of freedom. Using the restricted sample, we then perform the cross-equation test by estimating w of eq. (4).

The test was re-run four times. In the first re-run, the sample was reduced by omitting point spreads for which X > 5. Panel B of table 1 reports F

statistics for tests of the hypothesis that the betting lines’ forecast errors are unrelated to information in Z. In each subsequent re-run the value of X was decreased in unit steps. Panels C through E report F statistics for these tests.

Large values of F would indicate that the information in Z is not efficiently used by the market. The F statistics reported in panel B through E of table 1 do not allow rejection of the hypothesis of rationality. As h becomes smaller the conditioning information excluded from X2 diminishes, and the test is less likely to fail to reject a false null hypothesis. Thus, the results in panel E provide stronger support for rationality of betting lines than the results in

panel B.

5. Betting simulations

The tests described above use statistical criteria to evaluate market rational- ity. The null hypothesis tested was that the market forecast error was unre- lated to a particular set of regressors; the alternative hypothesis did not posit a specific type of market failure. There are more direct tests of rationality. If betting market forecasts are rational, it will not be possible to earn profits by betting on the basis of widely available information. Hence, a more direct test

B. Russo et al., Market ratronahty tests 463

of rationality is a test of the specific alternative hypothesis that unexploited

profit opportunities exist. The tests reported in this section rely on a filter similar to that used in the

cross-equation tests of section 4. Earlier we showed that the econometric forecast is likely to differ most from the market forecast when the latter is conditioned on information not available for use by the former. The market may use qualitative information, such as changes in personnel, injuries, ‘classic’ team rivalries, a team’s ‘momentum’, reports on morale, etc. The market may also use quantitative information precluded from use by the econometrician because of timing, frequency, or the specification of the prediction equation. Some examples of this type of information are the past record of a particular team against other teams, the past average margin of victory or loss of a particular team, and statistics for a particular game uncovered by sports analysts (the Los Angeles Raiders’ unusually good record on Monday nights. the Dallas Cowboys’ history of winning opening games, etc.).

Under the null hypothesis of rationality, it should prove costly to bet against the market (bet with the econometric forecast) when the market has the benefit of added useful qualitative and quantitative information. We used the filter to differentiate observations based on disparate information sets.

Let X again represent the difference between an econometric forecast and the market’s forecast. X will be positive if the econometric forecast exhibits more confidence in the home team (more of a favorite or less of an underdog) than the market; X will be negative in the opposite case. We used the ZGB model to forecast the results of football games and simulated betting by ‘placing bets’ on the home team when X > 0 and against the home team when X < 0. In other words, we placed a bet on the home team (visitor) when the ZGB model exhibited more (less) confidence in the home team than did the betting market.

Table 2 shows the results of the betting simulations. The table shows summary statistics for betting simulations based on five different values of X. The statistics reported in table 2 are the win-bet ratio and two Z statistics and their corresponding levels of significance. Z, tests the hypothesis that the observed win-bet ratio is random. Z, tests the hypothesis that these simula- tions are unprofitable.’ Panel A of table 2 shows the results of betting simulations on all games for which 0.5 =< 1x1. In panel B, the total number of bets is reduced by imposing an upper bound on X of five points. That is,

0.5 s 1x1~ 5. In panels C through E, the upper bound on X is successively reduced to two points. Note that, by arguments made earlier, the smaller is the

‘2, is the normal approximation to the bmomial dlstributlon. Z2 is found in Tryfos et al. (1984). See notes to table 2 for additional information.

464 B. Russo et al., Market ratmnnlrty tests

Table 2

Summary results of bettmg s~mnlations for last half 1983-1985 NFL seasons. _-__ --~ _..... _“. .I”_.

Filter Wins Bets W/B (X = IFe - PI) (W) (B) Pi) Zld s 8 1 Z,h sz ___” ._““.. . ---“- ---.~

Opening betting he

(A) 0.5sk 146 298 48.99 -035 0.637 - 1.17 0.879 (B) 0.5sJXxS 122 230 53.04 0.92 0.179 020 0.421 (C) 0.5s;hs4 107 193 55.44 1.51 MMc 0.86 0.195 (D) O.S$XI3 79 141 56.03 1.43 0.076” 0.87 0.192 (E) 0.5 s X 4 2 51 87 65.52 2.89 0.002e 2.58 0.005e

Clusrng betmg he

(A) 05s:x 143 296 48 31 -0.58 0 719 -1.40 0.919 (B) 0.5 $ h 5 5 113 221 5113 0.34 0 367 -0.37 0.644 tet 0.5SAhr4 105 194 54.12 1.15 0.125 0.49 0.312 (D) 0.5 s x $3 77 138 55.80 1.36 0.087c 0.81 0.209 (E) 0.5 $ h 5 2 49 79 62.03 2.14 0.016d 1.77 0.038d

aThese are the 2 values (Z, ) and slgmficance levels (q) for the null hypothesis that the win to bet record is random (the long-run win-to-bet proportion is 50 percent) against the alternative h~atbesjs that It is not random. Using the normal appro~matio~ to the binomial distribution this Z value is calculated as Z, = [ W-p(B)][B( p)(l -p)]-‘/‘, where lV and B are, respec- tlvely, the number of wins and total bets, and p = 0.5

bThese are the 2 values (Z,) and significance levels (s2) for the Tryfos et al. test for evaluating the profitability of bettmg strategies. The null hypothesis for this test is that a given rule IS unprofitable against the alternative that It is profitable The Z value is calculated as 2, = [(W/B) - l.l( L/R)]/[(l/B){(( w/R) + 1.21( L/B)) -.. ((W/B) - 1.1( &/B))2}]‘/z, where W, L, and B are, respectively. the number of wms. losses, and total bets for a given strategy To reduce the nsk of a type I error, the prubabxlity of erroneously including that the strategy is profitable, Tryfos et al. advocated the use of a ‘low’ Q level. For further details see Tryfos et al. (1984)

~Sigm~ica~t at the 10 p&cent level. ~S~8~~cant at the 5 percent level. ‘Significant at the 1 percent level.

upper bound on h, the smaller the dXerence between information sets ~nco~orated in the market and noumarket forecasts.

Panel A of table 2 indicates that betting against the market is a losing proposition when all observations are used in the simulation. The win-bet ratios are less than the 52.4 percent breakeven level.’ This changes, however, as the filter and the difference between info~ation sets narrows. In panel C, win-bet ratios exceed the 52.4 percent level, but the Z statistic is not large enough to reject r~do~ess in three of four cases. In panels I3 and FL the win-bet ratios rise steadily and the reported statistics indicate that these ratios are both nonrandom and profitable.

‘~arnbI~n8 on NFL games IS based on the ‘ll-for-lo’ rule. For example if a gambler wishes to bet the mimmum, he or she must lay out $11. If the bet is won. the $11 bet is rctu~~d to the gambler along with $10 in winnings: if the bet loses, the $11 bet is Forfeited In order to break even, the gambler must wm 52.4% of all bets.

B. Russo et ul., Murket rutronali[v t&s 465

Interpretation of these results is assisted by Brown and Maital’s (1981) distinction between ‘completeness’ and rationality. These authors point out the ways that market rationality may fail. The market may ignore usable information or may incorrectly process the information it does use. In the first case, the information set is incomplete and market predictions fall short of rationality even if the market employs the correct model. In the second case, the market incorrectly processes a complete information set. A third possibil- ity, suggested by Black (1986), is that market participants trade on data that is noise rather than information. In this case, the information set may be complete, but the noise, of course, is processed incorrectly.

Our test results appear to indicate that the football betting market incor- rectly processes an information set that tends toward completeness. Recall that the cross-equation tests fail to detect irrationality in the NFL betting market, while betting simulations reveal unexploited profit opportunities. But profits are possible only when the information sets available to the market and the econometrician are similar. When the market has the advantage of a wider information set. profits cannot be earned by our betting simulations. This suggests that the betting market tends toward completeness. Further, the previous observation, together with evidence that the closing line is not a better predictor than the opening line [see Gandar et al. (1988)], suggests that noise trading does not have a significant impact on this betting market. This is not surprising since fundamental values (game outcomes) are observed at close intervals in this market. Finally, since betting simulations are profitable when information sets are similar, it appears that the betting market does not correctly process all of the information it does use.

Although we hope the reader will find the conjectures of the previous paragraph to be of interest, our main concern is not irrationality in sports betting markets. Instead, the important result is that the cross-equation test failed to detect a relation between the market’s forecast error and a set of variables, X,, shown by standard t and F tests to be significantly related to game outcomes. If the market does not efficiently process information we include in X, then, a fortiori, we should expect the test to detect a relation between the forecast error and X,. We conclude that the cross-equation tests of market rationality have even less power than Abel and Mishkin suggest.

6. Out-of-sample tests

Stock market filters are notorious for indicating profit opportunities within- sample but failing out-of-sample. This is also true in the NFL betting market.” Out-of-sample betting simulations are needed to guard against this

“See, for example. Tryfos et al (1~64) and Gandar et al. (1988)

466 B. Russo et al., Murket rationakty tests

Table 3

F statisttcs for cross-equatron tests of rationality, 1988.

Dependent vanable PS-VLO” PS-VLCd

(A) All observatronsb F’

n’

(B) h > 5 F

(C) ;: z 4 F

(D) :>3 F

(E) ;: > 2 F

n

0.173 1.412 112 112

1.098 0.172 92 95

1.515 0.180 83 86

1.779 68

1.550 49

0.209 66

0237 52

“PS-VLO and PS-VLC arc the market forecast errors for the opening and closmg lines, respectively.

‘Panel A reports statistics on tests that include all observations: panels B through E omtt observations in accord with the filter X = ]F;e , - F;!!‘,I. [See eqs. (8) and (9) and the surrounding discussion.]

‘F is the F statrstlc for the null hypothesis that the forecast error is unrelated to any vanable m the information set described m the text; n 1s the number of observatrons

possibility. l1 For our pu p r oses, there is a second reason to conduct out-of- sample tests. Abel and Mishkin show that cross-equation tests provide consis- tent estimation even if the set of relevant information is not precisely specified. The argument works for the exclusion of relevant variables, not the inclusion of irrelevant ones. If our specification includes irrelevant variables the estima- tors are inefficient, standard errors are inflated and tests may fail to reject the null hypothesis.

We conduct out-of-sample tests using data from the 1988 regular season.12 Tables 3 and 4 indicate that the main results of the earlier cross-equation tests and betting simulations also hold for 1988. Table 3 reports F statistics for the null hypotheses that the coefficient vector, 0 of eq. (4) is zero. As in earlier tests, the test is confined to the second half of the season. None of the F

statistics are significant at conventional levels: market rationality is not rejected in this sample. Table 4 reports the results of simulated betting for the second half of the 1988 season. In each of the ten simulations reported the win-bet ratio exceeds the 52.4 percent breakeven level. As in earlier years,

t1 We thank an anonymous referee for pointmg thts out and suggesting the out-of-sample tests.

“We used 1988 data at the suggestion of the referee.

B. Russo et al., Murkei rationaIr@ tests 461

Table 4

Summary results of betting simulations for last half 1988 NFL season

Filter (h = IF ~ PI)

Wins

(W)

Bets

(B) W/B @) Z,” sia Z*b

h $2

Opemng hettlng he

(A) 0.5 5 X 57 99 57.58 1.51 0.067’ 0.13 0.448 (B) 0.5 2 X s 5 47 81 58.02 1.44 0.075’ O.i4 0.444 (C) 0.5 $ h s 4 42 72 58 33 1.42 0.078 0.15 0.440 (D) 0.5jX$3 33 58 56 90 1.05 0.147 0.11 0.456 (E) 052X52 22 39 56.41 0.80 0.212 0.97 0.460

Closing heltrng he

(A) 0.5 6 X 55 100 55.00 1.00 0.159 0.06 0 476 (B) 0.5 <X $5 45 83 54.22 0.77 0 221 0.04 0 484 (C) 0.5 s h s 4 39 14 52.70 0.47 0.319 0.01 0.496 (D) 0.5 2 h s 3 32 55 58 18 1.22 0 113 0.15 0444 (E) 0.5 s X 5 2 24 41 58.54 1.09 0 131 0.15 0.440

“These are the Z values (Z,) and significance levels (si) for the null hypothesis that the win to bet record is random (the long-run win-to-bet proportion ts 50 percent) against the alternative hypothesis that it IS not random. Using the normal approximation to the binomial distribution this Z value is calculated as Z, = [W -p( B)][ B( p)(l -P)]~‘/~, where W and B are, respec- tively, the number of wins and total bets, and p = 0.5.

‘These are the Z values ( Zz ) and significance levels ( sz) for the Tryfos et al. test for evaluating the profitability of betting strategies. The null hypothesis for this test is that a given rule is unprofitable agamst the alternative that it is profitable. The Z value is calculated as Z2 = [( W/B) - l.l( L/B)]/[(l/B){(( W/B) + 1.21( L/B)) ~ ((W/B) - l.l( L/B))2}]1/‘, where W. L. and B are, respectively, the number of wms, losses, and total bets for a given strategy. To reduce the nsk of a type I error, the probability of erroneously concludmg that the strategy is profitable, Tryfos et al advocated the use of a ‘low’ a level. For further detatls see Tryfos et al. (1984).

‘Significant at the 10 percent level

simulated betting indicates profit opportunities undetected by the cross-equa- tion tests.

There are two noticeable changes in results of these simulations. First, the filter mechanism no longer produces a discernible pattern in the win-bet ratios: second. the Z statistics are no longer significant. Neither of these changes is surprising in so small a sample.

As we stated at the outset, a large sample size is necessary in order for the filter to provide a noticeable impact. For the 1983-85 period, implementation of the filter reduced the number of observations by an average of 53.6 per step; the largest one-step decline was 75 and the smallest was 27. For the 1988 season, however, the filter reduced the number of observations by an average 14.9 per step; the largest one-step decline was 19 observations and the smallest was 9. For the 1988 season, the total decline over the four steps was not much more than the average decline per step in the earlier sample.

The Z statistics in the betting simulations are also reduced by the small number of observations in the 1988 sample. It is clear from the expressions for

468 B. Russo et (II., Marker rutronahty tests

Z, and Z, (see notes to table 2 or 4) that even if the win-bet ratio is constant, both statistics decrease at increasing rates as the number of bets declines.13 Hence, in this case Z, is less likely to indicate statistically significant devia- tions from randomness, and Z, is less likely to indicate significant profits, even though profits are consistently positive. To see the results of this consider that in the largest sample of the early period (298 observations) the win-bet ratio must be 56.68 percent in order for Z, to be significant at the 5 percent level.14 In 1988 this win-bet ratio is exceeded in seven of ten cases, but none of these are statistically significant. On the other hand, in 1988 the smallest win-bet ratio that would produce a significant Z, is 59.85 percent, a ratio which is attained only twice in the earlier period. Note that the probability of attaining ten winning win-bet ratios merely by chance is practically zero. We believe, therefore, that the small size of the 1988 sample masks the significance of the consistently profitable win-bet ratios.

7. Conclusion

Abel and Mishkin point out conditions by which cross-equation tests of rational expectations will have low power. This will occur when the informa- tion available to the researcher is a subset of information useful in making predictions. In such cases, market forecasts may be inefficient, but the test will not detect it. In addition, standard errors of coefficient estimates on informa- tion included in the test will tend to be large, biasing the test away from rejection of null hypotheses.

This paper develops a simple ad hoc algorithm for reducing the probability of type II errors in cross-equation tests of rational expectations. A large sample and a nonmarket forecast must be available. We provide an example of the technique in the context of the football betting market. A filter that suggests a method of omitting troublesome observations is developed and used to modify the tests. The statistical tests that employ filtered data do not reject rationality.

13Substituting a constant, denoted c, for the win-bet ratio (W/B) in Z,,

aZ,/aB=Bm"5((~-0.5)/21>0,

a2z,/as'= - B-‘~((P 05)/41<0,

whenever c > 0 5. as in table 4. Substitutmg the constant c for the win-bet ratro m Zz.

~ZJ~B = B ’ $/26 z 0.

c32Zz/aB’ = -B-’ 5~/4~ < 0,

whenever c > 0.5238, as m table 4. Here @= [c- l.l(l -c)] and 0 = {c+ 1.21(1 -c) - [c - l.l(l - c)]” )‘15, Thus, Z, and Z, decrease at increasing rates if the wu-bet ratto is constant and the number of bets falls.

t4To find this win-bet ratio, set B m the expression for Zt equal to 298, set Z, equal to 1.96, and solve for the win-bet ratio. W/B

B. Russo et al., Market rationality tests 469

However, evidence of irrationality in football betting markets exists. Betting simulations indicate that it is possible to make profits by betting against the market forecast when the market and the econometrician use similar informa- tion sets, but not when the market has superior information. This suggests the market uses available information, but processes some of this information in a less than optimal way.

These results are consistent with the theme of a growing literature on the power of statistical tests of asset market efficiency. Failure to reject efficiency may not provide evidence in favor of efficiency, since the test employed may have low power against some plausible alternatives. Summers (1986, p. 594) for example, states: ‘The usefulness of any test of a hypothesis depends on its ability to discriminate between it and’ other plausible alternatives.’ We have juxtaposed the results of cross-equation tests of the null hypothesis of betting market rationality against results of a test of the specific alternative hypothesis that profits can be earned in that market. This comparison provides an example, based on actual market data, of a case where a statistical test has low power and shows that the results of the test are misleading.

References

Abel, A.B. and F.S. Mishkin, 1983, An integrated view of tests of rationality, market efficiency, and the short-run neutrality of monetary policy, Journal of Monetary Economics 11. 3-24

Black, F.. 1986, Noise, Journal of Finance 41, 529-543 Brown, B.W. and S. Maital, 1981, What do economists know? An empirical study of experts’

expectations, Econometrica 49, 491-504. Canes, M.E., 1976, The market for pro football betting, in: W.R. Eadington, ed., Gambling and

society: Interdisciplinary studies on the subJect of gambling (Charles C. Thomas, Springfield. IL) 108-137

Fama. E.F , 1970, Efficient capital markets: A review of theory and empirical work, Journal of Fmance 25, 383-417.

Fama, E.F., 1975. Short-term interest rates as predictors of inflation, American Economic Review 65, 269-282.

Fama, E F., 1976a, Inflation uncertainty and expected returns on Treasury bills, Journal of Political Economy 84, 427-448.

Fama, E.F., 1976b. Foundations of finance (Basic Books, New York, NY). Figlewski, S , 1978, Market ‘efficiency’ in a market with heterogeneous information, Journal of

Political Economy 86. 581-597. Gandar, J., R. Zuber, T. O’Brien, and B. Russo, 1988, Testing rationality in the point spread

betting market, Journal of Finance 43, 995-1008. Granger, C.W.J. and P. Newbold, 1977, Forecasting economic time series (Academic Press, New

York, NY). JaKe, J.F. and R Winkler, 1976, Optimal speculation against an efficient market, Journal of

Finance 31, 49-61. Kahneman, D., P Slavic, and A Tversky, eds., 1982, Judgement under uncertainty: Heuristics

and biases (Cambridge University Press, Cambridge). Lovell, M.C., 1986. Tests of the rational expectations hypotheses, American Economic Review 76,

110-124. M&kin, F.S., 1983, A rational expectations approach to macroeconometrics: Testing policy

ineffectiveness and efficient markets models (University of Chicago for National Bureau of Economic Research, Chicago, IL)

470 B. Russo et al., Marker ratlonulrty tests

Muth, J., 1961, Rational expectations and the theory of price movements, Econometrica 29, 315-335

Nelson, C.R. and G.W. Schwert, 1977, Short-term interest rates as predictors of inflatron. On testing the hypothesis that the real rate is constant, American Economx Review 67, 478-486.

Perron. P. and R J. Shiller, 1985. Testing the random-walk hypothesis: Power vs. frequency of observations, Economics Letters 18, 381-386

Poterba, J.M. and L H. Summers. 1988, Mean reversion m stock prices: Evrdence and implica- trons, Journal of Financial Economics 22, 27-59.

Quandt, R.E.. 1986. Betting and equilibrium, Quarterly Journal of Economics 101,201-207 Sargent, T J., 1987, Macroeconomic theory (Academrc Press, New York, NY). Summers, L.H., 1986, Does the stock market rationally reflect fundamental values?, Journal of

Finance 41, 591-601. Tryfos, P., S. Casey, S. Cook, G. Leger, and B. Pylypiak, 1984, The profitablhty of wagering on

NFL games, Management Science 30, 123-132 Vergin, R.C. and M. Scriabin, 1978, Winning strategies for wagenng on national football league

games, Management Science 24, 809-818. Zuber. R.A.. J.M. Gandar. and B.D. Bowers, 1985, Beating the spread: Testing the efficrency of

the gambling market for NFL games, Journal of Political Economy 93. 800-806