A Framework for Dynamic Oligopoly in Concentrated Industries∗

Bar Ifrach† Gabriel Y. Weintraub‡

August 2016

Abstract

In this paper we introduce a new computationally tractable framework for Ericson and Pakes (1995)-style dynamic oligopoly models that overcomes the computational complexity involved in computingMarkov perfect equilibrium (MPE). First, we define a new equilibrium concept that we call moment-based Markov equilibrium (MME), in which firms keep track of their own state, the detailed state ofdominant firms, and few moments of the distribution of fringe firms’ states. Second, we provide guide-lines to use MME in applied work and illustrate with an application how it can endogenize the marketstructure in a dynamic industry model even with hundreds of firms. Third, we develop a series of resultsthat provide support for using MME as an approximation. We present numerical experiments showingthat MME approximates MPE for important classes of models. Then, we introduce novel unilateral de-viation error bounds that can be used to test the accuracy of MME as an approximation in large-scalesettings. Overall, our new framework opens the door to study new issues in industry dynamics.

1 Introduction

Ericson and Pakes (1995)-style dynamic oligopoly models (hereafter, EP) offer a framework for modelingdynamic industries with heterogeneous firms. The main goal of the research agenda put forward by EPwas to conduct empirical research and evaluate the effects of policy and environmental changes on marketoutcomes in different industries. The importance of evaluating policy outcomes in a dynamic setting andthe broad flexibility and adaptability of the EP framework has generated many applications in industrialorganization and related fields (see Doraszelski and Pakes (2007) for an excellent survey).

Despite the broad interest in dynamic oligopoly models, there remain significant hurdles in applyingthem to problems of interest. Dynamic oligopoly models are typically analytically intractable, hence numer-ical methods are necessary to solve for their Markov perfect equilibrium (MPE). With estimation methods,∗We would like to thank Vivek Farias for numerous fruitful conversations, valuable insights, important technical input, and

careful reading of the paper. We have had helpful conversations with Lanier Benkard, Jeff Campbell, Allan Collard-Wexler, DeanCorbae, Pablo D’Erasmo, Uli Doraszelski, Przemek Jeziorski, Ramesh Johari, Boyan Jovanovic, Sean Meyn, Ariel Pakes, JohnRust, Bernard Salanie, Minjae Song, Ben Van Roy, Daniel Xu, as well as with seminar participants at various conferences andinstitutions. We thank Yair Shenfeld for exemplary research assistance. We are grateful to the editor Stephane Bonhomme andthree anonymous referees for constructive feedback that greatly improved the paper.†Airbnb, Barifrach@gmail.com‡Graduate School of Business, Stanford University, gweintra@stanford.edu

1

such as Bajari et al. (2007), it is no longer necessary to solve for the equilibrium in order to structurally es-timate a model. However, in the EP framework solving for MPE is still essential to perform counterfactualsand evaluate environmental and policy changes. The complexity of this computation has severely limited thepractical application of EP-style models to industries with just a handful of firms, far less than the real-worldindustries the analysis is directed at. Notably, such limitations have made it difficult to construct realisticempirical models.

Thus motivated, in this paper we propose a new computationally tractable model to study industrydynamics in settings with a few dominant firms and many fringe firms; in applications, dominant firmswould typically be the market share leaders. A market structure in which a few firms holding significantmarket share co-exist with many small firms is prevalent in many industries, such as supermarkets, banking,beer, cereals, and heavy duty trucks, to name a few (see, e.g., concentration ratios from the U.S. EconomicCensus). These industries are intractable in the standard EP framework due to the large number of fringefirms. Although individual fringe firms have negligible market power, they may have significant cumulativemarket share and may collectively discipline dominant firms’ behavior. Our model and methods capturethese types of interactions and therefore significantly expand the set of industries that can be analyzedcomputationally.

In an EP-style model, each firm is distinguished by an individual state at every point in time. Theindustry state is a vector (or ‘distribution’) encoding the number of firms in each possible value of theindividual state variable. Assuming its competitors follow a prescribed strategy, a given firm selects, ateach point in time, an action (e.g., an investment level) to maximize its expected discounted profits. Theselected action will depend in general on the firm’s individual state and the industry state. Even if firmswere restricted to anonymous and symmetric strategies, the computation entailed in selecting such an actionquickly becomes infeasible as the number of firms and individual states grows. This renders commonly useddynamic programming algorithms to compute MPE infeasible for many problems of practical interest.

The first main contribution of this work is the introduction of a new framework and equilibrium conceptthat overcomes the computational complexity involved in computing MPE for EP-style models.1 In anindustry with many competitors it may be reasonable to assume that firms are more sensitive to changes inthe dominant firms’ states. Further, it may be unrealistic to expect that firms have resources to keep track ofthe evolution of all rivals. Thus motivated, we postulate a plausible model of firms’ behavior: firms closelymonitor dominant firms, but do not monitor fringe firms as closely. Specifically, we assume that firms’strategies depend on: (i) their own individual state; (ii) the detailed state of dominant firms; and (iii) a smallnumber of aggregate statistics (such as a few moments) of the distribution of fringe firms. We refer to thefringe firms’ distribution or state interchangeably. We call these strategies moment-based strategies, wherewe use the term ‘moments’ generically, as firms could keep track of other statistics of the distribution offringe firms, such as un-normalized moments or quantiles.

Based on these strategies, we introduce an equilibrium concept that we call moment-based Markov equi-librium (MME). In MME, firms’ beliefs about the evolution of the industry satisfy a consistency condition,and firms employ an optimal moment-based strategy given these beliefs. One challenge in defining thisconcept, however, is that fringe moments may not be sufficient statistics to predict the future evolution of

1We focus on the important class of ‘capital accumulation’ games with out investment spillovers that EP originally introduced.

2

the industry. For example, suppose firms keep track of the first moment of the fringe state. For a given valueof the first moment, there could be many different fringe distributions consistent with that value, from whichthe future evolution of the industry could vary greatly. Technically, the issue that arises is that the stochas-tic process of moments may not be a Markov process even if the underlying dynamics are. Therefore, todefine MME we introduce a ‘Markov perceived transition kernel’ to approximate the dynamics of the (non-Markov) moment-based state; we provide natural and concrete examples for such a kernel. Notably, thisconstruction allows for fringe firms to become dominant as they grow large and vice-versa, endogenizingthe market structure in equilibrium.

We introduce MME as an approximation in EP-style games of symmetric information. However, be-cause MME limits firms’ information sets (by allowing dependence on the fringe state only through fewmoments), it could also be cast as an equilibrium concept in a game of asymmetric information even ifthe underlying economic model is not. Hence, while their model and motivation is fundamentally differentfrom ours, MME is related to the notion of experienced-based equilibria of Fershtman and Pakes (2012)that weakens the restrictions imposed by perfect Bayesian equilibria for dynamic games of asymmetricinformation. We discuss this relation in more detail in Sections 5.3 and 8.3.

After defining MME, our second main contribution is a series of guidelines to use MME in appliedwork. We introduce an algorithm that efficiently computes MME even in industries with hundreds of firms,provided that firms keep track of a few moments of the fringe state and there are a few dominant firms. Inaddition, we discuss important implementation issues for the use of MME in practice, such as how to selectmoments and the number of dominant firms allowed.

To further illustrate the applicability of our approach, we show how it can be used to endogenize themarket structure in a fully dynamic model with hundreds of firms for which MPE would be impossible tocompute. In particular, we perform numerical experiments motivated by the long trend towards concentra-tion in the beer industry in the US during the years 1960-1990. Over the course of those years, the numberof active firms dropped dramatically, and three industry leaders emerged. One common explanation of thistrend was the emergence of national TV advertising as an ‘endogenous sunk cost’ (Sutton, 1991). We buildand calibrate a dynamic advertising model of the beer industry and use MME to determine how a singleparameter related to the returns to advertising expenditures critically affects the resulting market structureand the level of concentration in the industry. We also show how MME generates rich strategic interactionsbetween dominant and fringe firms, in which, for example, dominant firms make investment decisions todeter the entry and growth of fringe firms.2

While MME may be appealing intuitively, moment-based strategies may not be close to a best response,and therefore, MME may not be close to a subgame perfect equilibrium in general. The reason is that be-cause moments may not be sufficient statistics to predict the future evolution of the industry, they may notinduce a sufficient partition of histories and may not summarize all payoff-relevant history in the sense ofMaskin and Tirole (2001). Indeed, observing the full distribution of fringe firms may provide valuable infor-mation for decision making. Our third main contribution is a series of results that addresses this difficulty,

2As further evidence of the usefulness of our approach, Corbae and D’Erasmo (2014) has already used our framework to studythe impact of capital requirements in market structure in a calibrated model of banking industry dynamics with dominant and fringebanks.

3

providing support for using MME as an approximation.First, we explore the performance of MME as a low-dimensional approximation to MPE. We carry out

a large set of numerical experiments comparing MME and MPE outcomes in industries with a few firmsand a few individual states per firm, for which MPE can be computed. We consider two important dynamicoligopoly models: a quality ladder model and a capacity competition model. We show that MME is ableto provide accurate approximations to MPE across a wide range of parameter values even when firms keeptrack of a single and natural fringe moment. For example, in the capacity competition model this moment isthe total installed capacity among all fringe firms.

Alternatively, MME can be thought of as an appealing heuristic and behavioral model on its own. Toformalize this interpretation, we define the ‘value of the full information deviation’ that measures the extentof sub-optimality of MME strategies in terms of a unilateral deviation to a strategy that keeps track of thefull industry state, similar to the notion of ε-equilibrium. If this value is small, MME may be an appealingbehavioral model, as unilaterally deviating to more complex strategies does not significantly increase profits,and doing so may involve costs associated with gathering and processing more information.

Measuring the value of the full information deviation is intractable in large-scale applications with manyfirms; in these instances, computing a best response is almost as computationally challenging as solving fora MPE. In these cases we could not compare MME with MPE either. To address this limitation, using ideasfrom robust dynamic programming, we propose a novel computationally tractable upper bound to the valueof the full information deviation. This bound is useful because it allows one to evaluate whether the stateaggregation is appropriate or whether a finer state aggregation is necessary, for example, by adding moremoments. We provide numerical experiments showing how the error bound works in practice.

Our moment-based strategies are similar to and follow the same spirit of the seminal paper by Kruselland Smith (1998), which replaces the distribution of wealth across agents in the economy with its moments,when computing stationary stochastic equilibrium in a stochastic growth model. While our main focushas been on dynamic oligopoly models, our methods can also be used to find better approximations instochastic growth models, as well as in macroeconomic dynamic industry models with an infinite number ofheterogeneous firms and an aggregate shock (see, e.g., Khan and Thomas (2008) and Clementi and Palazzo(2016)). In particular, in Section 8.3 we discuss how our error bounds can be applied to this class of modelsas well as to Fershtman and Pakes (2012)-style dynamic games of asymmetric information.

In summary, our approach offers a computationally tractable model for industries with a dominant/fringemarket structure, opening the door to studying novel issues in industry dynamics. As such, our frameworkgreatly increases the applicability of dynamic oligopoly models. The rest of the paper is organized asfollows. We discuss connections to other related literature in Section 2. Section 3 describes our dynamicoligopoly model. Section 4 introduces important elements to define MME. Then, we define MME and analgorithm to compute it in Section 5. Section 6 provides a numerical comparison between MME and MPEand discusses important implementation issues. Section 7 describes the application to the beer industry andSection 8 introduces our approximation error bounds. Finally, Section 9 provides conclusions.

4

2 Related Literature

Previous work has already recognized the challenges involved in applying dynamic oligopoly models inpractice. For example, researchers have used different approaches to overcome the burden involved incomputing equilibria in empirical applications. Some papers empirically study industries that contain onlya few firms in which exact MPE computation is feasible if the number of individual states assumed ismoderate (e.g., Benkard (2004), Ryan (2012), Collard-Wexler (2013), and Collard-Wexler (2011)). Otherresearchers structurally estimate models in industries with many firms using approaches that do not requireMPE computation, and do not perform counterfactuals that require computing equilibria (e.g., Benkard et al.(2010)). Our work provides a method to perform counterfactuals in concentrated industries with many firms.

Another set of papers perform counterfactuals computing MPE but in reduced-size models compared tothe actual industry. These models include a few dominant firms and ignore the rest of the (fringe) firms (e.g.,see Ryan (2012) and Gallant et al. (2016)). Other papers coarsely discretize the individual states (e.g., seeCollard-Wexler (2014) and Corbae and D’Erasmo (2013) that assume homogenous firms). Finally, somepapers allow for richer heterogeneity but assume a simplified model of dynamics in which a certain processof ‘moments’ that summarize the industry state information is Markov (e.g., see Kalouptsidi (2014), Jia andPathak (2015), Santos (2010), and Tomlin (2010); Lee (2013) use a similar approach in a dynamic modelof demand with forward-looking consumers). Our methods can help researchers determine the validity ofthese simplifications.

Our work is related to recent methodological work that tries to overcome the complexity involved incomputing MPE. For example, Pakes and McGuire (2001) proposes a stochastic approximation algorithmthat solves the model only on a recurrent class of states, and reduces the computation time at each statethrough simulation. Doraszelski and Judd (2011) suggests casting the game in continuous time, whichgreatly reduces the computational cost at each state by reducing the number of future states reachable fromeach current state (see also Jeziorski (2014) for an application of this technique). Other papers proposedifferent computational approaches to approximate MPE. Farias et al. (2012) develops a method based onapproximate dynamic programming with value function approximation (see also Sweeting (2013) for anempirical application using value function approximation). Santos (2012) introduces a state aggregationtechnique based on quantiles of the industry state distribution.

A stream of empirical literature related to our work uses simplified notions of equilibrium for estimationand counterfactuals. In particular, Xu (2008), Iacovone et al. (2015), and Qi (2013), among others usethe notion of oblivious equilibrium (OE) introduced by Weintraub et al. (2008), in which firms assume theaverage industry state holds at any time. OE can be shown to approximate MPE in industries with manyfirms by a law of large numbers, provided that the industry is not too concentrated. Our approach extendsthis type of approach to industries that are more concentrated.

Our work is particularly related to Benkard et al. (2015), which extends the notion of oblivious equi-librium to include dominant firms. In that paper, there is a fixed and pre-determined set of dominant firms,which all firms in the industry monitor. Firms assume that at every point in time the fringe state is equal tothe expected state conditional on the state of dominant firms. Our paper builds on and generalizes Benkard

5

et al. (2015)’s idea of considering a dominant/fringe market structure.3 In particular, our approach is moregeneral in at least two important dimensions: (i) in that firms not only keep track of the dominant firms’states but also of statistics that summarize the fringe firms’ states; and (ii) that in our model the set of domi-nant firms arises endogenously in equilibrium. That is, we allow for firms to transition from the fringe to thedominant tier and vice-versa, and therefore, we endogenize the market structure and explicitly model howfirms grow to become dominant. We provide more details about this connection in Section 5.3.

3 Dynamic Oligopoly Model and Solution Concept

In this section we formulate a model of industry dynamics in the spirit of Ericson and Pakes (1995). Similarmodels have been applied to numerous settings in industrial organization such as advertising, auctions,R&D, collusion, consumer learning, learning-by-doing, and network effects (see Doraszelski and Pakes(2007) for a survey). As in the original EP paper, our model is a ‘capital accumulation’ game with outinvestment spillovers, for which (i) decisions that determine static profits (e.g., prices) do not affect a firm’sstate; and (ii) a firm’s individual state is only affected by its own dynamic action.

3.1 Model

We introduce the main elements of our dynamic oligopoly model.Time Horizon. The industry evolves over discrete time periods and along an infinite horizon. We indextime periods with nonnegative integers t ∈ N (N = 0, 1, 2, . . .). All random variables are defined on aprobability space (Ω,F ,P) equipped with a filtration Ft : t ≥ 0. We adopt a convention of indexing byt variables that are Ft-measurable.Firms. Each firm that enters the industry is assigned a unique positive integer-valued index. The set ofindices of incumbent firms at time t is denoted by St.State Space. Firm heterogeneity is reflected through firm states that represent the quality level, productivity,capacity, the size of its consumer network, or any other aspect of the firm that affects its profits. At time tthe individual state of firm i is denoted by xit ∈ X , where X is a finite subset of Nq, q ≥ 1. We define theindustry state st to be a vector over individual states that specifies, for each firm state x ∈ X , the number ofincumbent firms at x in period t. Note that because we will focus on symmetric and anonymous equilibriumstrategies in the sense of Doraszelski and Pakes (2007), we can restrict attention to industry states for whichthe identities of firms do not matter. The integer number N represents the maximum number of incumbentfirms that the industry can accommodate at every point in time and we let nt be the number of incumbentfirms at time period t.Exit process. In each period, each incumbent firm i observes a nonnegative real-valued sell-off value φit thatis private information to the firm. If the sell-off value exceeds the value of continuing in the industry, thenthe firm may choose to exit, in which case it earns the sell-off value and then ceases operations permanently.

3In their analysis of a stylized model of dynamic mergers, Gowrisankaran and Holmes (2004) also provide an earlier exampleof the dominant/fringe dichotomy.

6

We assume the random variables φit|t ≥ 0, i ≥ 1 are i.i.d. and have a well-defined density function withfinite moments.Entry process. We consider an entry process similar to the one in Doraszelski and Pakes (2007). At timeperiod t, there areN −nt potential entrants, ensuring that the maximum number of incumbent firms that theindustry can accommodate is N (we assume n0 < N ). Each potential entrant is assigned a unique index.In each time period each potential entrant i observes a positive real-valued entry cost κit that is privateinformation to the firm. We assume the random variables κit|t ≥ 0 i ≥ 1 are i.i.d., independent of allpreviously defined random quantities, and have a well-defined density function with finite moments. If theentry cost is below the expected value of entering the industry then the firm will choose to enter. Potentialentrants make entry decisions simultaneously. Entrants appear in the following period at state xe ∈ X andearn profits thereafter.4 As is common in this literature and to simplify the analysis, we assume potentialentrants are short-lived and do not consider the option value of delaying entry. Potential entrants that do notenter the industry disappear and a new generation of potential entrants is created in the next period.Transition dynamics. If an incumbent firm decides to remain in the industry, it can take an action toimprove its individual state. Let I ⊆ <k+ (k ≥ 1) be a convex and compact action space; for concreteness,we refer to this action as an investment. Given a firm’s investment ι ∈ I and state at time t, the firm’stransition to a state at time t+ 1 is described by the following Markov kernel Q:

Q[x′|x, ι] = P[xi,t+1 = x′∣∣∣xit = x, ιit = ι].

Uncertainty in state transitions may arise, for example, due to the risk associated with a research and de-velopment endeavor or a marketing campaign. The cost of investment is given by a nonnegative functionc(ιit, xit) that depends on the firm’s individual state xit and investment level ιit. Both the kernel and theinvestment cost are assumed to be continuous functions of investment. We assume that transitions areindependent across firms conditional on the industry state and investment levels. We also assume thesetransitions are independent of all previously defined random quantities.Aggregate shock. There is an aggregate profitability shock zt that is common to all firms and that mayrepresent a common demand shock, a common shock to input prices, or a common technology shock. Weassume that zt ∈ Z : t ≥ 0 is a finite, ergodic Markov chain, and independent of all previously definedrandom quantities.Single-Period Profit Function. Each incumbent firm earns profits on a spot market. We denote byπ(xit, st, zt) as the single-period expected profits garnered by firm i at time period t, which depends onits individual state xit, the industry state st, and the value of the aggregate shock zt.Timing of Events. In each period, events occur in the following order: (1) Each incumbent firm observes itssell-off value and then makes exit and investment decisions; (2) Each potential entrant observes its entry costand makes entry decisions; (3) Incumbent firms compete in the spot market and receive profits; (4) Exitingfirms exit and receive their sell-off values; (5) Investment shock outcomes are determined, new entrantsenter, and the industry takes on a new state st+1.Firms’ objective. Firms aim to maximize expected discounted profits. The interest rate is assumed to

4It is straightforward to generalize the model by assuming that entrants can also invest to improve their initial state.

7

be positive and constant over time, resulting in a constant discount factor of β ∈ (0, 1) per time period.Finally, we assume that for all competitors’ decisions and all continuation values, a firm’s one time-stepahead optimization problem to determine its optimal investment has a unique solution.5

3.2 Markov Perfect Equilibrium

The most commonly used equilibrium concept in such dynamic oligopoly models is that of symmetricpure strategy Markov perfect equilibrium (MPE) in the sense of Maskin and Tirole (1988). To simplifynotation we re-define the industry state to incorporate the aggregate shock, that is st = (st, zt). Hence,there is a function ι such that at each time t, each incumbent firm i invests an amount ιit = ι(xit, st),as a function of its own state xit and the industry state st. Similarly, each firm follows an exit strategythat takes the form of a cutoff rule: there is a real-valued function ρ such that an incumbent firm i exitsat time t if and only if φit > ρ(xit, st). Weintraub et al. (2008) show that there always exists an optimalexit strategy of this form even among very general classes of exit strategies. We define the state spaceS =

(s, z) ∈ N|X | ×Z

∣∣∣∑x∈X s(x) ≤ N

, that is, S is the set of industry states including the aggregateshock. LetM denote the set of exit/investment strategies such that an element µ ∈M is a pair of functionsµ = (ι, ρ), where ι : X × S → I is an investment strategy and ρ : X × S → R is an exit strategy.6

Similarly, each potential entrant follows an entry strategy that takes the form of a cutoff rule: there is areal-valued function λ such that a potential entrant i enters at time t if and only if κit < λ(st). It is simple toshow that there always exists an optimal entry strategy of this form even among very general classes of entrystrategies (see Doraszelski and Satterthwaite (2010)). We denote the set of entry strategies by Λ, where anelement of Λ is a function λ : S → R.

We define the value function V (x, s|µ′, µ, λ) to be the expected discounted value of profits for a firm atstate x when the industry state is s, given that its competitors each follow a common strategy µ ∈ M, theentry strategy is λ ∈ Λ, and the firm itself follows strategy µ′ ∈M. Formally,

V (x, s|µ′, µ, λ) = E µ′,µ,λ

[τi∑k=t

βk−t (π(xik, sk)− c(ιik, xik)) + βτi−tφi,τi

∣∣∣xit = x, st = s

],

where i is taken to be the index of a firm at individual state x at time t, τi is a random variable representingthe time at which firm i exits the industry, and the subscripts of the expectation indicate the strategy followedby firm i, the strategy followed by its competitors, and the entry strategy. In an abuse of notation, we will usethe shorthand, V (x, s|µ, λ) ≡ V (x, s|µ, µ, λ), to refer to the expected discounted value of profits when firmi follows the same strategy µ as its competitors. An equilibrium in our model comprises an investment/exitstrategy µ = (ι, ρ) ∈M and an entry strategy λ ∈ Λ that satisfy the following conditions:

5This assumption is similar to the unique investment choice admissibility assumption in Doraszelski and Satterthwaite (2010)that is used to guarantee the existence of an equilibrium to the model in pure strategies. It is satisfied by many of the commonlyused specifications in the literature.

6We restrict attention to states (x, s) ∈ X × S, such that s(x) > 0 (recall that s(x) is the the x-th component of s).

8

1. Incumbent firms’ strategies represent a MPE:

supµ′∈M

V (x, s|µ′, µ, λ) = V (x, s|µ, λ) , ∀(x, s) ∈ X × S.

2. For all states, the cut-off entry value is equal to the expected discounted value of profits of enteringthe industry:7

λ(s) = β E µ,λ [V (xe, st+1|µ, λ)|st = s] , ∀s ∈ S.

Doraszelski and Satterthwaite (2010) establish the existence of an equilibrium in pure strategies for asimilar model. With respect to uniqueness, in general we presume that our model may have multiple equi-libria. Doraszelski and Satterthwaite (2010) and Iskhakov et al. (2015) also provide examples of multipleequilibria in related models. A limitation of MPE is that the set of relevant industry states grows quicklywith the number of firms in the industry and individual states, making its computation intractable when thereare more than a few firms. This motivates our alternative approach of defining a new equilibrium conceptthat overcomes the computational complexity involved in solving for MPE.

4 Main Elements of Moment-Based Markov Equilibrium

In this section we introduce important objects that will serve as foundations to define moment-based Markovequilibrium (MME) in Section 5.

4.1 Dominant and Fringe Firms

We focus on industries that exhibit the following market structure: there are a few dominant firms and manyfringe firms. Let Dt ⊂ St and Ft ⊂ St be the set of incumbent dominant and fringe firms at time period t,respectively. The sets Dt and Ft are common knowledge among firms at every period of time. These setscan change over time and our model incorporates a mechanism that endogenizes the process through whichfringe firms become dominant and vice-versa.

We let Xf ⊆ X and Xd ⊆ X be the set of feasible individual states for fringe and dominant firms,respectively. We define the state or distribution of fringe firms ft to be a vector over individual states thatspecifies, for each fringe firm state x ∈ Xf , the number of incumbent fringe firms at x in period t. We defineSf =

f ∈ N|Xf |

∣∣∣∑x∈Xf f(x) ≤ N

to be the set of all possible states of fringe firms.Let dt be the state of dominant firms, specifying the individual state of each dominant firm at time period

t. The set of all possible dominant firms’ states is defined by Sd = XDd , where D is the maximum numberof dominant firms the industry can accommodate. Hence, the dominant firms’ state is represented by a listof individual states.8 Note that by defining an inactive state, the number of active dominant firms could be

7Hence, potential entrants enter if the expected discounted profits of doing so are positive. Throughout the paper it is implicitthat the industry state at time period t+ 1, st+1, includes the entering firm in state xe whenever we write (xe, st+1).

8Because we focus on equilibrium strategies that are anonymous, we can restrict attention to a set Sd for which the identity ofdominant firms does not matter. For example, if the state is one dimensional, we can define Sd = d ∈ XDd

∣∣∣ d(1) ≤ d(2), ...,≤d(D). We choose this state representation for dominant firms as opposed to a distribution over individual states, because we will

9

anywhere between 0 and D in a given time period. With some abuse of notation, we define the state spaceS = Sf × Sd × Z , where a state s = (f, d, z) ∈ S is given by a distribution of fringe firms, a state fordominant firms, and a value for the aggregate shock.

The division between dominant and fringe firms will depend on the specific application at hand. Typ-ically, however, dominant firms will be market share leaders or, more generally, firms that most affectcompetitors’ profits. For concreteness, suppose that the state of the firm represents its ability to compete inthe market and firms in higher individual states have larger market shares, as in the two examples we intro-duce below in Section 6.1. It may then be natural to define dominant firms as those that have a sufficientlylarge individual state. In our numerical experiments, we separate dominant firms from fringe firms by anexogenous threshold state x, such that i ∈ Dt if and only if xit ≥ x. We note that in this case, Xf ∩Xd = ∅,which we assume throughout the rest of the paper to simplify notation.9

More generally, one could consider a relative threshold that depends on some statistics of the currentfringe state; for example, a fringe firm becomes dominant if it is certain number of standard deviationsabove the current average size of fringe firms. The threshold could also depend on the current state ofdominant firms. Note that under all these specifications the transitions among the fringe and dominanttiers are naturally embedded in the firms’ transitions. Our approach could also accommodate alternativespecifications on how firms become dominant depending on the context.

In the applications we have in mind, dominant firms are a few and have significant market power. Incontrast, fringe firms are many and individually hold little market power, although their aggregate marketshare may be significant. This market structure suggests that firms’ decisions should be more sensitive tothe state of dominant firms than to the state of fringe firms. Moreover, given that the fringe firms’ state isa highly dimensional object, gathering information on the state of each individual small firm is likely to bemore expensive than doing so for larger firms that not only are few, but also usually more visible and oftenpublicly traded. Consequently, as the number of fringe firms increases it seems implausible that firms keeptrack of the individual state of each one. Instead, we postulate that firms only keep track of the state ofdominant firms and of a few summary statistics of the fringe state.

4.2 Moment-Based Strategies

Our approach requires that firms compute best responses in strategies that depend only on a few summarystatistics of the fringe state. A set of such summary statistics is a multi-variate function θ : Sf → Θ,where Θ is a finite subset of <n. For example, when the fringe firm state is one-dimensional, θ(f) =∑

x∈Xf xαf(x) is the α−th un-normalized moment with respect to the distribution f . If α = 1, the statistic

θ(ft) corresponds to the first un-normalized moment, which is equivalent to the sum of the individual statesof all incumbent fringe firms,

∑i∈Ft xit. Note that with a finite number of fringe firms (N ) and a finite

number of individual states per firm (Xf ), this and all other fringe statistics that we use in the paper take afinite number of values.

typically consider applications with a small number of dominant firms.9Note that more generally we can always make the assumption that Xf ∩Xd = ∅ by appending one dimension to the individual

state in order to indicate whether the firm is fringe or dominant. This construction is useful as it allows, for example, for fringe anddominant firms to have different model primitives and strategies.

10

For brevity and concreteness, we call such summary statistics fringe moments, with the understandingthat they could include normalized or un-normalized moments, but also other functions of the fringe distri-bution such as quantiles or the number of fringe firms that are about to become dominant. Accordingly, wedefine a set of moments of the distribution of fringe firms:

θt = θ(ft). (1)

We introduce firm strategies that depend on their own individual state xit, the state of dominant firms dt,the aggregate shock zt, and fringe moments θt as defined in (1). We call such strategies moment-basedstrategies. We also define Sθ as the set of admissible moments defined by (1); that is, Sθ = θ|∃f ∈Sf s.t. θ = θ(f). In light of this, we define the moment-based industry state by s = (θ, d, z), and themoment-based state space by S = Sθ × Sd ×Z .

A moment-based investment strategy is a function ι such that at each time t, each incumbent firm i ∈ Stinvests an amount ιit = ι(xit, st), where st is the moment-based industry state at time t. Similarly, eachfirm follows an exit strategy that takes the form of a cutoff rule: there is a real-valued function ρ such that anincumbent firm i ∈ St exits at time t if and only if φit ≥ ρ(xit, st). Let M denote the set of moment-basedexit/investment strategies such that an element µ ∈ M is a pair of functions µ = (ι, ρ), where ι : X×S → Iis an investment strategy and ρ : X × S → < is an exit strategy.10

Each potential entrant follows an entry strategy that takes the form of a cutoff rule: there is a real-valuedfunction λ such that a potential entrant i enters at time t if and only if κit ≤ λ(st). We denote the set ofentry functions by Λ, where an element of Λ is a function λ : S → <. We assume that all new entrantsbecome part of the fringe. Note that moment-based strategies and the moment-based state space are definedwith respect to a specific function of moments (1). Relative to MPE strategies, moment-based strategies donot keep track of the full fringe state; they only keep track of a few fringe moments.

4.3 Transition Kernel

An important construct to define MME is a perceived transition kernel as introduced in this subsection.With Markov strategies (µ, λ) the underlying state, st = (ft, dt, zt) : t ≥ 0, is a Markov process with atransition kernel denoted by Pµ,λ. In addition, we denote by Pµ′,µ,λ the transition kernel of (xit, st) whenfirm i uses strategy µ′, and its competitors use strategy (µ, λ). Note that given strategies, both these kernelscan be derived from the primitives of the model, namely, the distributions of φ and κ, the kernel of theaggregate shock, the kernel Q, and the sets Xf ,Xd. We emphasize that the underlying industry state stshould be distinguished from the moment-based industry state st = (θt, dt, zt).

Defining our notion of equilibrium in moment-based strategies will require the construction of whatcan be viewed as a ‘Markov’ approximation to the dynamics of the moment-based state process (xit, st) :

t ≥ 0, where i is a generic firm. Note that this process is, in general, not Markov even if the dynamicsof the underlying state (xit, st) : t ≥ 0 are. To see this, consider for example that the individual staterepresents a firm’s size and that firms keep track of the first un-normalized moment of the fringe state,

10We restrict attention to states (x, s), such that if the firm in state x is a dominant firm, then its individual state needs to coincidewith one of the components of d.

11

θt = θ(ft) =∑

x∈Xf xft(x). Suppose the current value of that moment is θt = 10; this value is consistentwith one fringe firm in individual state 10, but also with 10 fringe firms in individual state 1. These twodifferent states do not necessarily yield the same probabilistic distribution for the first moment in the nextperiod. Therefore, θt may not be a sufficient statistic to predict the future evolution of the industry, becausethere are many fringe distributions that are consistent with the same value of θt. In the process of aggregatinginformation via moments the resulting process is no longer Markovian.

Assuming that firm i follows the moment-based strategy µ′, and that all other firms use moment-basedstrategies (µ, λ), we will describe a kernel Pµ′,µ,λ[·|·] with the hope that the Markov process described bythis kernel is a good approximation to the (non-Markov) process (xit, st) : t ≥ 0. The Markov processdescribed by Pµ′,µ,λ are firm i’s perception of the evolution of its own state in tandem with the moment-based industry state. We will describe this joint kernel by decomposing it into few sub-kernels. In particular,in Section 4.3.1 we introduce the kernel Pµ,λ[θ′|s] that describes the evolution of a hypothetical Markovprocess over moments, that is, the kernel that specifies perceived transition probabilities from moment-basedstates to next period moments. Then, in Section 4.3.2 we introduce transition probabilities from fringe intodominance. Finally, in Section 4.3.3 we collect them together to define the joint perceived transition kernel.

4.3.1 Examples of Perceived Transition Kernels Over Moments

There are many possible specifications for the perceived transition kernel over moments Pµ,λ[θ′|s]; however,some yield better approximations. We provide some useful examples next.

To construct this kernel we let the industry evolve for a long time horizon T under strategies (µ, λ) andwe record the moment-based states visited, st = (θt, dt, zt) : t = 0, ..., T.

Example 4.1 (Observed transitions). One possible definition for Pµ,λ[θ′|s] is the kernel that coincides withthe long-run average observed transitions from the moment-based state in the current time period to themoment in the next time period under strategies (µ, λ). More specifically, for each moment-based states that is visited, we observe a transition to a new moment θ′. We count the observed frequency of thesetransitions and set the kernel Pµ,λ[θ′|s] to be the observed distribution corresponding to these frequencies.A similar construction is used by Fershtman and Pakes (2012) in a setting with asymmetric information.

We now formalize the definition of this kernel. The evolution of the underlying industry state is describedby the kernel Pµ,λ. We let Rµ,λ be the recurrent class of moment-based states the industry will eventuallyreach. For all θ′ ∈ Sθ and s ∈ Rµ,λ, we define:

Pµ,λ[θ′|s] = lim supT→∞

∑Tt=1 1st = s, θt+1 = θ′∑T

t=1 1st = s, (2)

and Pµ,λ[θ′|s] is arbitrarily defined for states s /∈ Rµ,λ. In other words, perceived transitions for fringemoments coincide with their observed transitions in the recurrent class of states, and are arbitrary outsidethis class. To define our equilibrium concept firms need to have perceived transitions in the entire state space.In Section 6.3 we provide numerical experiments showing how different specifications of the transitionsoutside the recurrent class affect equilibrium outcomes.

12

To make the example more concrete, suppose that θt = θ(ft) =∑

x∈Xf xft(x), the first un-normalizedmoment of the fringe state. To simplify the exposition, we ignore dominant firms and the aggregate shock.We let the industry evolve for a long time under given strategies (µ, λ). Suppose that in the course of thissimulation, we observe that one half of the time periods in which θt = 10, the moment the next time periodis given by θt+1 = 10, one fourth is given by θt+1 = 12, and the other fourth is given by θt+1 = 8. Then,we set Pµ,λ[θ′ = 10|θ = 10] = 1/2, Pµ,λ[θ′ = 12|θ = 10] = 1/4, Pµ,λ[θ′ = 8|θ = 10] = 1/4. We wouldset Pµ,λ[θ′|θ] similarly for all moment states in the recurrent class.

We note that for all model instances that we consider in our numerical experiments, the industry stateprocess admits a single recurrent class for given strategies. In this case, one can show that the right-hand sideof equation (2) converges almost surely (see equation (14) in the appendix). For the ‘capital accumulation’games without investment spillovers that we study there are simple sufficient conditions that ensure a singlerecurrent class.11 Alternatively, in other settings that yield more than one recurrent class, to construct theperceived transition kernel we would need to assume that all firms agree on which recurrent class the industrywill eventually transition into.12

Another possible construction of the perceived transition kernel has been successfully used in stochasticgrowth models in macroeconomics (Krusell and Smith, 1998) and subsequent literature.

Example 4.2 (Parametric linear transitions). This specification for Pµ,λ[θ′|s] assumes a parameterized anddeterministic evolution for moments given strategies (µ, λ). That is, starting from industry state st =

(θt, dt, zt), the next moment value is assumed to be

θt+1 = G(θt; ξ(dt, zt)),

where ξ(dt, zt) are parameters. For example, this could represent a linear relationship with one moment,θt+1 = ξ0(dt, zt) + ξ1(dt, zt)θt. In this case the goal would be to choose functions ξ0 and ξ1 that approxi-mate the actual transitions best, for instance by employing linear regressions with the data of the simulatedtrajectory st : t = 0, ..., T. Then, for all θ′ ∈ Sθ and s ∈ Rµ,λ, we define:

Pµ,λ[θ′|s] =

1, if θ′ = G(θ; ξ(d, z)) ,

0, if θ′ 6= G(θ; ξ(d, z)) .(3)

In addition, Pµ,λ[θ′|s] is arbitrarily defined for states s /∈ Rµ,λ.

Relative to the kernel in Example 4.1, this perceived transition kernel has the disadvantage of impos-ing strong parametric restrictions and assuming deterministic transitions. On the other hand, these samerestrictions significantly reduce the computational burden of solving for the equilibrium.

Both of these kernels are meant to capture the long-run evolution of the industry. We could also introducea perceived kernel meant to capture the short-term dynamics of the industry starting from a given initial state,

11For example, a sufficient condition is that the sell-off value has support on R+.12Fershtman and Pakes (2012) face this issue in the models they consider and assume that the recurrent class firms agree on is

the one selected by a reinforcement learning real-time dynamic programming algorithm that computes their equilibrium concept.

13

for example, after a policy or an environmental change. For this, we could consider the average observedtransitions from the current moment to the next, over many finite (and short) trajectories that start from thesame state; this state describes the initial condition of the industry (see Appendix A for more details).

4.3.2 Fringe to Dominance Transitions

In general, moments in θ(·) may not contain sufficiently detailed information to precisely calculate the tran-sition probabilities of a competitor into dominance. For example, consider the setting used in our numericalexperiments in which firms are dominant if and only if they are above a pre-determined state x. Supposeθ(f) is the first un-normalized moment of the fringe state. Then, the probability that a fringe competi-tor becomes dominant depends on the fringe state beyond that moment. Therefore, in this case we needto specify firms’ perceived probabilities that a fringe competitor will transition into dominance given themoment-based industry state. We provide one such specification next.13

To simplify the dynamics we impose the constraint that at most one firm becomes dominant in a timeperiod; this constraint was typically never binding in our numerical experiments. Let the event (f → d)

correspond to a fringe competitor becoming a dominant firm in the next period. With a similar motivationto Example 4.1, we assume that for a given moment-based state, firms’ perceived probability of this transi-tioning event occurring is equal to its long-run average observed frequency. Therefore, with some abuse ofnotation, we define for all s ∈ Rµ,λ:

Pµ,λ[(f → d)|s] = lim supT→∞

∑Tt=1 1st = s, (f → d)t∑T

t=1 1st = s, (4)

where (f → d)t corresponds to the event of a fringe competitor transitioning into dominance in time periodt. In addition, Pµ,λ[(f → d)|s] is arbitrarily defined for states s /∈ Rµ,λ. Again, if the industry state processadmits a single recurrent class, the right-hand side of the equation above converges almost surely.

4.3.3 Joint Perceived Transition Kernel

Having defined the kernel for fringe moments, Pµ,λ[θ′|s], and for transitions into dominance, Pµ,λ[(f →d)|s], we now define the perceived kernel Pµ′,µ,λ[·|·] over states (xit, st), given strategies (µ′, µ, λ), accord-ing to:

Pµ′,µ,λ[x′, s′|x, s] =

Pµ′,µ,λ[x′, d′, z′|x, s]Pµ,λ[θ′|s]Pµ,λ[(f → d)|s], if s′ includes a new dominant competitor

associated to the event (f → d)

Pµ′,µ,λ[x′, d′, z′|x, s]Pµ,λ[θ′|s](

1− Pµ,λ[(f → d)|s]), if not .

(5)We note that if the number of dominant firms in s is equal to the maximum allowed, D, then the transitionprobabilities associated to the event (f → d) are omitted. With some abuse of notation Pµ′,µ,λ[x′, d′, z′|x, s]above denotes the marginal distribution of the next state of firm i at state x (including its own chances of

13In Section 6.3 we also study how MME performs when including an additional moment that encodes the probability a fringefirm becomes dominant the next time period.

14

becoming dominant in the next time period), the next state of current dominant firms, and the next value forthe aggregate shock, conditional on the current moment-based state, according to the kernel of the underlyingstate Pµ′,µ,λ. The definition above makes the following facts about this perceived process transparent:

1. Were firm i a fringe firm, the above definition asserts that this fringe firm ignores its own impact on theevolution of industry moments. This is evident in that x′ is distributed independently of θ′ conditionalon x and s. This seems reasonable in applications with many fringe firms, for which changes in afringe firm’s individual state have little impact on the value of the moments. However, in the small-scale experiments we run below to compare MME with MPE, we also tried a specification in whicha fringe firm that becomes dominant removed itself from the moment. With a small number of fringefirms this had an important effect in improving the approximation to MPE, specifically in the capacitycompetition model described below.14

2. Given information about the current moment-based state, the firm correctly assesses the distributionof its next state, the next state of current dominant firms, and the next aggregate shock. Note thatbecause firms use moment-based strategies, the moment-based state (x, s) is sufficient to determinethe transition probabilities of (x, d, z) according to the transition kernel of the underlying state Pµ′,µ,λ.

3. However, it should be clear that the Markov process given by the above definition remains an ap-proximation since it posits that the evolution of moments θ and dominance transitions (f → d) areMarkov with respect to s as described by Pµ,λ[θ′|s] and Pµ,λ[(f → d)|s], respectively. In fact, thedistribution of moments and the transition into dominance at the next time period potentially dependon the full distribution of the fringe firms and not only its moments.

4. To construct the perceived transition kernel we assume that the events that a fringe competitor be-comes dominant and vice versa are both statistically independent of the moment transitions. Thismay be particularly reasonable if generally the value of the individual state from which fringe firmstransition in and out of dominance is small relative to the current value of the moment. Our numericalresults in Section 6.3 suggest that MME provides accurate approximations even under this indepen-dence assumption. Moreover, this simplification is computationally useful, because without it wewould need to store the joint transition kernel of dominant firms and moment transitions, requiringsignificantly more memory; this requirement could be relevant in large-scale applications like the onepresented in Section 7.15

14Since we assumed that at most one firm becomes dominant in a time period, in principle, one should explicitly consider andrule out the event that firm i and a competitor become dominant in the same time period. Even though we explicitly consider thisevent in our numerical experiments, it turned out not to be important in practice, because the constraint of having at most one fringefirm becoming dominant in a time period was typically never binding. Hence, for simplification, we ignored this effect in the kerneldescribed above.

15Under our simplification, to build the transition kernel we need to store the following elements in memory: the perceivedtransition kernel for moments, the tier transitions’ probabilities, and the strategy of dominant firms. This roughly requires memoryspace proportional to |S||Sθ| + 2|S| (assuming no aggregate shock). Instead, the joint transition kernel of dominant firms andmoment transitions would require memory space roughly proportional to S2. Recall that |S| = |Sθ||Sd|. For example, if Sθ =Sd = 100, then the first expression is equal to 1,020,000 and the second to 100,000,000.

15

4.4 Single-Period Profit Function

The single-period profit function can be written as π(xit, st): a function of the individual state xit and theindustry state st = (ft, dt, zt). If firms use moment-based strategies and only keep track of fringe moments,they do not have the ability to evaluate a single-period profit function that depends on the entire fringe state.Hence, to define our equilibrium concept in these cases, similar to the perceived transition kernel, we need todefine a perceived single-period profit function over moment-based states, πµ,λ(xit, st), for given strategies(µ, λ).

One possible specification for the perceived single-period profit function is:

πµ,λ(x, s) = lim supT→∞

∑Tt=1 π(x, st)1st = s∑T

t=1 1st = s, (6)

for all s in the recurrent class Rµ,λ. Note that πµ,λ(x, s) corresponds to the long-run average single-periodprofit experienced whenever the moment-based state is given by (x, s), because equation (6) takes an av-erage of the primitive profit function π(x, st) over all realizations of the industry state st for which thecorresponding moment-based state is st = s. This specification is similar to the one used in Fershtman andPakes (2012). As before, if the industry state process admits a single recurrent class, the right-hand side ofthe equation above converges almost surely.

Despite this definition, we envision that our approach will be most useful in settings for which thesingle-period profit function only depends on the fringe state through a few moments (or that this providesan accurate approximation), as the next assumption makes explicit and that we keep throughout the paperunless otherwise noted.

Assumption 4.1. The single-period expected profit of firm i at time t, π(xit, θt, dt, zt), depends on itsindividual state xit, a vector θt ∈ Θ ⊆ Θ of fringe moments, the state of dominant firms dt, and thevalue of the aggregate shock zt.

The single-period profit functions we will consider either satisfy the previous assumption exactly orcan be well approximated by functions of fringe moments. Given this, in our approach, firms will alwayskeep track of the moments that determine (or approximate) the profit function, θt; that is θt will always beincluded in θt in equation (1). The state space spanned by θt is much smaller than the original one if Θ is lowdimensional and significantly smaller than |Xf |. In fact, many single profit functions of interest depend on asmall number of functions of the distribution of firms’ states. For example, commonly used profit functionsthat arise from monopolistic competition models depend on a particular moment of that distribution (Dixitand Stiglitz, 1977; Besanko et al., 1990). Further, in Section 6.1 we describe two important profit functionsfor applied work that we use in our numerical experiments that can be well approximated by a function of asingle fringe moment.

We note that firms may keep track of additional contemporaneous moments beyond the moments θtto improve their predictions of the future evolution of the industry. In our applications, however, we willtypically start our analysis by assuming that firms only keep track of the fringe moments associated with thesingle-period profit function, and therefore, θt = θt.

16

5 Moment-Based Markov Equilibrium

We use the elements introduced in the previous section to define MME. We also provide an algorithm tocompute MME. Then, we discuss the relation between MME and previously defined equilibrium concepts.Finally, we provide an initial discussion, more thoroughly developed in the following sections of the paper,on how to interpret MME as an approximation.

5.1 Definition of Moment-Based Markov Equilibrium

A moment-based Markov equilibrium (MME) is an equilibrium in moment-based strategies. Having defineda Markov process approximating the process (xit, st) : t ≥ 0, we define the perceived value function by adeviating firm i that uses moment-based strategy µ′ in response to strategies (µ, λ). Importantly, this valuefunction assumes that firm i’s perception of the evolution of its own state and the moment-based industrystate are described by the perceived kernel Pµ′,µ,λ defined above. Formally,

V (x, s|µ′, µ, λ) = E µ′,µ,λ

[τi∑k=t

βk−t[π(xik, sk)− c(ιik, xik)

]+ βτi−tφi,τi

∣∣∣xit = x, st = s

],

where the expectation is taken with respect to the perceived transition kernel Pµ′,µ,λ. We will use theshorthand notation V (x, s|µ, λ) ≡ V (x, s|µ, µ, λ).

A moment-based Markov equilibrium (MME) is defined with respect to: (i) a specific function of mo-ments θ in equation (1); and (ii) a specification for the perceived kernel Pµ′,µ,λ, like the one defined inequation (5). Hence, the function of moments and the specification for the perceived kernel are inputs to de-fine MME. We could derive different MME for the same model primitives by changing these inputs. In ournumerical experiments, we use the kernel defined in equation (5), with the specifications for Pµ,λ[θ′|·] andPµ,λ[(f → d)|·] provided in equations (2) and (4), unless otherwise noted. Recall that the perceived ker-nel is a function of (µ′, µ, λ); hence, different strategies yield different instances of the specified perceivedkernel. We now define MME.

Definition 5.1. A MME of our model is comprised of an investment/exit strategy µ = (ι, ρ) ∈ M and anentry cutoff function λ ∈ Λ that satisfy the following conditions:

C1: Incumbent firms’ strategies optimization:

supµ′∈M

V (x, s|µ′, µ, λ) = V (x, s|µ, λ) ∀(x, s) ∈ X × S.

C2: At each state, the cut-off entry value is equal to the expected discounted value of entering the industry:

λ(s) = β E µ,λ

[V (xe, st+1|µ, λ)

∣∣∣st = s]

∀s ∈ S.

Note that the two conditions are similar to the conditions imposed in MPE, but incorporate the moment-based state space and strategies. Computationally, MME is appealing if agents keep track of a few moments

17

of the fringe state and there are a small number of dominant firms, because agents optimize over low dimen-sional strategies. We note that if the function θ is the identity, so that θ(f) = f , and the perceived transitionkernel coincides with the actual transition kernel for all states, then MME is equivalent to MPE.

Using a similar argument to Doraszelski and Satterthwaite (2010) and imposing some additional tech-nical conditions we can show that an MME always exists. The additional conditions are required to handlepossible discontinuities of the perceived transition kernel in states that are outside of the recurrent class.We provide a precise statement and existence result in Appendix B. In the numerical experiments that wepresent later in the paper we are always able to computationally find an MME over a large set of primitivesthat satisfy the standard assumptions required for the existence of MPE in EP-style models. With respect touniqueness, in general we presume that our model may have multiple equilibria.

5.2 Algorithm to Compute MME

In this section we introduce an algorithm that computes MME based on iterating best responses (Algorithm1). The solver starts with a strategy profile, checks for the equilibrium conditions, and updates the strategiesby best responding to the current strategy until an equilibrium is found. In each iteration the algorithmconstructs the perceived transition kernel given firms’ strategies. For the kernels introduced in Section 4.3this is done using simulated sample paths. The following remarks are important:

1. The algorithm terminates when the norm of the distance between a strategy and its best response issmall. We consider the following norm ‖µ − µ′‖h = maxx∈X

∑s∈S |µ(x, s) − µ′(x, s)|h(s)

,

where h is a probability vector. We take h to be the frequency in which each industry state is visited,hence states are weighted according to their relevance. This is useful because simulation errors in theestimated transition kernel are higher for states that are visited infrequently. We also try a variation ofh in which we initially add positive weight to all states.

2. For different perceived transition kernels, line 6 in Algorithm 1 would be executed differently. Further,even if in theory all states are recurrent, in a finite length simulation it is possible that some stateswill not be visited, and for these states the perceived transition kernels defined in Section 4.3 are notspecified. For these states we set the transitions in P to be some predetermined value. We try differentvalues and discuss their impact on equilibrium outcomes in Section 6.3.

3. The value functions in lines 7 and 8 of the algorithm involve a single-agent best response, and there-fore, can be computed via a number of standard dynamic programming algorithms; we specificallyuse the value iteration algorithm (Bertsekas, 2001).

If the algorithm terminates with ε = 0 and h(·) assigns strictly positive weight to all industry states wehave found an MME. A positive value of ε allows for numerical error. We also note that in the algorithm,0 < σ < 1 is chosen to speedup convergence by smoothing out the update of strategies from one iterationto the next.16

16In practice, we use σ = 2/3.

18

Algorithm 1 Equilibrium solver

1: Initialize with (µ, λ) and industry state s0 = (f0, d0, z0) with corresponding s0;2: n := 1, ∆ := ε+ 1;3: while ∆ > ε do4: Simulate a T period sample path (ft, dt, zt)Tt=1 with corresponding stTt=1 for large T ;5: Calculate the observed frequencies of industry states h(s) := 1

T

∑Tt=1 1st = s, for all s ∈ S;

6: Use the same sample path to compute the perceived transition kernel Pµ′,µ,λ[·|x, s], for all x, s;7: Solve µ′ := argmax

µ∈MV (x, s|µ, µ, λ), for all (x, s) ∈ X × S;

8: Let λ′(s) := β Eµ,λ[V (xe, st+1|µ′, µ, λ)|st = s], for all s ∈ S ;9: ∆ := max(‖µ− µ′‖h, ‖λ− λ′‖h);

10: µ := µ+ (µ′ − µ)/(1 + nσ);11: λ := λ+ (λ′ − λ)/(1 + nσ);12: n := n+ 1;13: end while;

We found that for the perceived transition kernel defined by equations (2), (4), and (5), a real-timestochastic algorithm similar to Pakes and McGuire (2001) and Fershtman and Pakes (2012) is much fasterthan Algorithm 1. In this variant the kernel is not simulated in every iteration, but instead continuationvalue functions that allow to solve for firms’ optimal strategies are kept in memory and updated throughsimulation draws. In this sense, this modified real-time algorithm performs the simulation and optimizationsteps simultaneously. We then use one step of Algorithm 1 to check that the MME conditions hold and thatconvergence has been achieved. The details of this variant of the algorithm are presented in Appendix F.

5.3 Relation with Other Equilibrium Concepts

We finish this section by discussing the relationship between MME and related equilibrium concepts previ-ously defined in the literature.

First, MME is related to the equilibrium concepts defined in Fershtman and Pakes (2012) that considersdynamic oligopoly models with asymmetric information. While their model and motivation is fundamen-tally different from ours, they are related in that MME limits firms’ information sets, and therefore could becast as an equilibrium concept in a game of asymmetric information even if the underlying economic modelis not. To make the connection apparent, assume that fringe firms privately observe their own states, and thepublicly observed information are the fringe moments, the state of dominant firms, and the aggregate shock.

A difference between MME and the restricted experienced based equilibrium (REBE) defined in Fer-shtman and Pakes (2012), however, is the treatment of states outside of the recurrent class. REBE imposesweaker optimality conditions relative to MME, because it does not require optimality of strategies nor cor-rect evaluation of continuation values outside of the recurrent class.17 In contrast, in MME the perceivedtransition kernel can be arbitrarily defined in states outside of the recurrent class. However, given theseperceptions, MME requires optimality at all states. For these reasons, if the set of states in the recurrent

17The continuation values outside of the recurrent class only need to guarantee that the strategies chosen given these values keepthe industry process within the recurrent class.

19

class is a strict subset of the state space, which is typically the case, then REBE is different from MME. Ifall industry states belong to the recurrent class, MME with the transition kernel of Example 4.1 coincideswith REBE.

This equivalence holds when REBE strategies only keep track of the current moments, state of dominantfirms, and aggregate shock. However, REBE strategies typically keep track of past observations as well. Toextend the equivalence to these cases, we could extend the definition of MME so that firms keep trackof past values of the moment (θt−1, θt−2, ...). Nevertheless, to provide a more direct connection with thecommonly used concept of MPE, in which firms keep track only of the current industry state, and to simplifyexposition, our definition of MME considers strategies that only depend on current moments of the fringestate. In addition, in Section 8.3 we provide further connections between our work and Fershtman and Pakes(2012), in particular in relation to our error bounds.

Finally, if we assume (i) the set of dominant firms is fixed and pre-determined; (ii) firms only keep trackof the dominant firms’ state and do not keep track of fringe moments; and (iii) that the perceived single-period profit function is specified as in equation (6), then MME coincides with the simulated version ofpartially oblivious equilibrium (POE) of Benkard et al. (2015).18

5.4 MME as Approximation

Suppose firms play moment-based strategies with moments θ(·) and use a perceived transition kernel Pµ′,µ,λ.As previously noted, generally st : t ≥ 0 is not Markov, so it may not be a sufficient statistic to predictthe future evolution of the industry as described by the primitive transition kernel Pµ,λ. Hence, θ(·) doesnot necessarily summarize all payoff relevant history in the sense of Maskin and Tirole (2001), and ob-serving the underlying full fringe state may provide valuable information for decision making. Generally,MME strategies may not be close to a best response, and therefore, may not be close to a subgame perfectequilibrium.

It is illustrative to study whether there are models for which equilibrium strategies yield moments thatare indeed sufficient statistics, and therefore for which MME strategies are optimal. In Appendix C weintroduce one such class of models. Suppose that the single-period profit function for fringe firms exhibitsconstant returns to scale, that the dynamics of a fringe firm’s evolution are linear in its own state, and thatthere are no transitions between the dominant and fringe tiers. Then, we show in the appendix that MMEstrategies yield moments that form a Markov process and hence summarize all payoff relevant informationas the number of fringe firms grows. As a consequence, it is also possible to show that for this model MMEstrategies become nearly-optimal when the number of fringe firms is large.

The model just described imposes strong assumptions on the model primitives for fringe firms and maybe too restrictive for many applications. A natural question is whether we can obtain similar aggregationresults for a broader class of models. Unfortunately, the answer is usually no. Specifically, in Appendix Cwe also show that when firms keep track of a single moment, moments are sufficient statistics only if fringefirms’ equilibrium transitions are, in an appropriate sense, linear in their own state. To our knowledge,

18Instead, define πµ,λ(xit, dt, zt) = π(xit, fµ,λ(dt, zt), dt, zt), where fµ,λ(d, z) = limT→∞

∑Tt=1 ft1dt=d, zt=z∑Tt=1 1dt=d,zt=z

. In thelatter case, we recover the regular notion of POE.

20

such equilibrium transitions arise only in the constant returns to scale model mentioned above or in closevariations.

The previously mentioned results suggest that it seems implausible to prove a general result showingthat MME becomes an exact approximation for broad classes of models that are typically used in appliedwork. Instead, a more practical approach, that we take in our numerical experiments, is to use MME in thesemodels assuming that firms approximate the evolution of the moment-based state with a perceived Markovprocess, and then try to measure the extent of the approximation error.

There are at least two alternative ways of interpreting MME as an approximation and, therefore, tomeasure the approximation error. First, we can interpret MME as a low-dimensional approximation toMPE. We explore this interpretation in Section 6. Alternatively, MME can be thought of as an appealingheuristic and behavioral model on its own; this is related to the optimality of moment-based strategies andthe extent of a unilateral deviation. In turn, we discuss this interpretation in Section 8.

6 MME as Approximation to MPE: Numerical Experiments

As mentioned in Section 5.4, MME can be interpreted as a low-dimensional approximation to MPE and inthis section we numerically explore this interpretation. In Section 6.1 we introduce the two profit functionswe use in our numerical experiments and in Section 6.2 we provide other model specifications. Then, inSection 6.3 we provide our numerical results showing that MME approximates MPE well. We also discussimportant implementation issues to consider when using MME in applied work.

6.1 Single-Period Profit Functions

In this section we provide two important profit functions that we use in our numerical experiments and thatcan be well approximated by a function of a single fringe moment. Our first model is similar to standardindustrial organization models commonly used in empirical work.

Example 6.1 (Quality-Ladder). Similarly to Pakes and McGuire (1994), we consider an industry withdifferentiated products, where each firm’s state variable xit is a natural number that represents the quality ofits product. For clarity, we do not consider aggregate shocks in this example. There are m consumers in themarket. In period t, consumer j receives utility uijt from consuming the good produced by firm i given by:

uijt = α1 ln(xit) + α2 ln(Y − pit) + νijt , i ∈ St, j = 1, . . . ,m,

where Y is the consumer’s income, and pit is the price of the good produced by firm i. The random variablesνijt are i.i.d. Gumbel that represent unobserved characteristics for each consumer-good pair. There is also anoutside good that provides consumers zero expected utility. We assume consumers buy at most one producteach period and that they choose the product that maximizes utility. Under these assumptions our demandsystem is a standard logit model.

Considering a constant marginal cost of production d, there exists a unique Nash equilibrium in purestrategies that can be computed by solving the first-order conditions of the pricing game (see Caplin and

21

Nalebuff (1991)). Let p∗(x) denote the price charged by a firm in state x in such an equilibrium. LetK(x, p) = exp(α1 ln(x) + α2 ln(Y − p)). Then, expected profits are given by:

π(xit, st) = m(p∗(xit)− d)K(xit, p

∗(xit))

1 +∑

x∈X st(x)K(x, p∗(x)), ∀i ∈ St. (7)

Now, if, similar to the logit model of monopolistic competition of Besanko et al. (1990), we assume fringefirms set prices assuming they have no market power, so they all set the price p∗ = (Y + cα2)/(1 +α2) (butdominant firms continue to compete Nash in prices), expected profits are given by:

π(xit, st) = m(p∗(xit)− d)K(xit, p

∗(xit))

1 + (Y − p∗)α2∑

y∈Xf yα1ft(y) +

∑j∈Dt K(xjt, p∗(xjt))

(8)

Therefore, under this assumption, π(xit, st) can be written as π(xit, θt, dt), that is, as a function of the singlemoment θt =

∑y∈Xf y

α1ft(y), the α1−th un-normalized moment of ft. In Section 6.3 we numericallyshow that (8) approximates (7) well in the type of applications we have in mind in which fringe firms areindeed smaller in size compared to dominant firms.

Our second model is structurally different and we chose it because it has been shown in previous workto yield rougher MPE value functions relative to our first model (see Farias et al. (2012)). In this sense,approximating MPE for this model imposes a more demanding test for MME.

Example 6.2 (Capacity Competition). This model is based on the quantity competition model of Besankoand Doraszelski (2004). We consider an industry with homogeneous products, where each firm’s statevariable determines its production capacity q(xit). We assume that capacity grows linearly in state, that isq(x) = qi + qsx, for some constants qi, qs > 0. Investment increases this capacity. We also ignore anaggregate shock. At each period, firms compete in a capacity-constrained quantity setting game. There isa linear demand function Q(p) = m(e − fp) and an inverse demand function P (Q) = e/f − Q/(mf),where P represents the common price charged by all firms in the industry, Q represents the total industryoutput, and e, m and f are positive constants. To simplify the analysis, we assume the marginal costs of allfirms are equal to zero. Given the total quantity produced by its competitors Q−i,t, the profit maximizationproblem for firm i at time t is given by max0≤qit≤q(xit) P (qit + Q−i,t)qit, where q(xit) is the productioncapacity at individual state xit. It is possible to show that a simple iterative algorithm yields the unique Nashequilibrium of this game q∗(xit).19 Profits for firm i are then given by:

π(xit, st) = P

(∑x∈X

st(x)q∗(x)

)q∗(xit). (9)

Suppose that all fringe firms are relatively small and they produce at full capacity in the Nash equilibrium.19The equilibrium quantities are characterized by the following set of equations: q∗(xit) =

max

0,minq(xit),

12

(me+ q∗(xit)−

∑x∈X st(x)q∗(x)

), ∀ i ∈ St.

22

Then,

π(xit, st) = P

∑y∈Xf

ft(y)q(y) +∑j∈Dt

q∗(xjt)

q∗(xit). (10)

Hence, under this assumption π(xit, st) can again be written as a function of a single moment, π(xit, θt, dt),where θt =

∑x∈Xf ft(x)q(x), the total current installed capacity of fringe firms. In Section 6.3 we numeri-

cally show that (10) approximates (9) well in the type of applications we have in mind in which fringe firmsare indeed smaller than dominant firms.

In our numerical experiments firms keep track of the moments described above that approximate theprofit function. In the logit model, the fringe moment summarizes the impact of the fringe on market shares;this is a natural statistic to keep track of. In the capacity competition model, firms monitor the aggregatecapacity among fringe firms. This again is natural, because (i) it may be a statistic that is easy to gather,for example, from trade associations; and (ii) fringe firms may indeed produce at full capacity. Ryan (2012)uses a profit function similar to our capacity competition model and argues that these two characteristics arepresent in his study of the cement industry.

6.2 Other Model Specifications

In our numerical experiments in this section we consider the single-period profit functions from Examples6.1 and 6.2. The rest of the model specifications are common across these two profit functions.

Firms can invest ι ≥ 0 in order to improve their individual state over time (either product quality orcapacity, depending on the model) at a constant marginal investment cost of c. A firm’s investment issuccessful with probability aι

1+aι , in which case the individual state increases by one level. The firm’s productmay also depreciate by one state with probability δ, independently each period. Our model differs fromPakes and McGuire (1994) here because the depreciation shocks in our model are idiosyncratic. Combiningthe investment and depreciation processes, it follows that the transition probabilities for a firm in state x thatdoes not exit and invests ι are given by:

P[xi,t+1

∣∣∣xit = x, ι]

=

(1−δ)aι

1+aι if xi,t+1 = x+ 1

(1−δ)+δaι1+aι if xi,t+1 = x

δ1+aι if xi,t+1 = x− 1 .

While it would be straightforward to implement entry and exit as we do in Section 7, in order to simplifythe model we omit them from our computations in this section.

In practice, parameters would either be estimated using data from a particular industry or chosen to re-flect an industry under study. We use a set of parameter values to reflect reasonable economic fundamentals,summarized in Table 1. We keep these parameters fixed for all experiments, unless otherwise noted.

23

Parameters quality ladder model m d α1 α2 Y c

Value 60 0.5 1 0.5 1 0.65Parameters capacity model m qi qs e f c

Value 2 0 2 50 10 4Parameters shared both models β δ a X

Value 0.925 0.7 1 1,...,9

Table 1: Default parameters for quality ladder model and capacity model experiments.

6.3 Comparison to MPE

In this section we numerically compare MME with MPE for the two models described above. Becauseexact computation of MPE is only feasible for small state spaces, we consider |X | = 9 individual statesand N = 5, 6, 7 firms. We also consider several parameter regimes. First, for the quality ladder model, thedefault parameters in the tables above represent a regime in which there is a small market size and a smallinvestment cost. We also consider a set of parameters in which these two quantities are relatively larger.20

We also consider similar regimes for the capacity competition model with a similar interpretation for thedefault parameters.21

Both for MME and MPE there are potentially multiple equilibria. We have not made an attempt toexhaustively search for all of them; however, our algorithm always found the same equilibria for a given in-stance, even when starting from different initial conditions (see Besanko et al. (2010) for an explanation ofthis phenomenon). All of the results below apply to this particular equilibria. Instead of comparing equilib-rium strategies directly, we compare economic indicators induced by these strategies. These indicators arelong-run averages of various functions of the industry state under the strategy in question.22 The indicatorswe examine are those that are typically of interest in applied work; we consider average total investment,average producer surplus, average consumer surplus, and average social welfare.

For the comparison results presented in Table 2, we consider MME withD = 3 dominant firm slots. Weassume that firms become dominant if they are large enough, specifically if they grow above (and including)a pre-determined individual state x. For all the instances, we fixed x = 6 (recall there are 9 individualstates). This threshold state seems like a reasonable compromise between defining dominant firms oncethey indeed grow large enough, but at the same time allowing enough richness in dominant firms’ dynamics.

We assume firms always keep track of a single moment, namely, the one suggested in Section 6.1, foreach profit function. Computing MME typically takes a matter of minutes, while computing MPE could takemany hours, in particular for the case N = 7.23 We discuss below how the results change when considering

20For the latter, we consider m = 90 and c = 1. The default market size parameters are used for N = 6. For N = 5 and N = 7we subtract and add 5 units to these numbers, respectively.

21For the regime with a larger investment cost, we consider e = 65 and c = 6.5. Again, the default market size parameter isused for N = 6. For N = 5 and N = 7 we use m = 1.66 and m = 2.33, respectively.

22As discussed in Section 4.3.1, the specification of the perceived transition kernel that we use to compute these MME is meantto capture the long-run evolution of the industry; accordingly, we compare long-run indicators of interest.

23For example, for N = 7 in the capacity competition model, the number of industry states for MME and MPE are given by1120 and 11440, respectively. All runs were performed on a shared cluster of 27 dual Xeon processors. The code was programmedin Java.

24

different specifications for dominant firms and moments.

Long-Run StatisticsInstance Number Total Prod. Cons. Soc.

of firms Inv. Surp. Surp. Welf.

Quality ladder model

High inv. costN = 5 7.9 5.0 1.7 2.1N = 6 3.1 4.4 2.5 2.7N = 7 1.7 4.0 3.3 3.4

Low inv. costN = 5 7.8 5.0 1.7 2.1N = 6 3.4 4.2 2.4 2.6N = 7 1.4 3.7 3.1 3.1

Capacity competition model

High inv. costN = 5 0.1 2.3 5.1 2.8N = 6 2.8 2.8 7.8 3.7N = 7 5.0 3.5 11.6 5.1

Low inv. costN = 5 4.6 1.5 5.7 2.7N = 6 1.7 1.5 7.6 3.3N = 7 1.0 1.7 11.9 4.7

Table 2: Comparison of MPE and MME indicators. Long-run statistics computed simulating industryevolution over 106 periods. Numbers presented are percentage differences of the economic indicators underMPE and MME.

In Table 2, we observe that MME provides accurate approximations to MPE for all economic indicatorsof interest in all instances. As expected, MME does a better job approximating MPE for the quality laddermodel, for which most indicators are approximated within 3-5 %. For the capacity competition model, theapproximation errors are similar, except for consumer surplus, for which the approximation error is around11-12% for two instances (but significantly smaller for the rest).

Note that there are several sources of approximation error for MME relative to MPE. First, differentlyto MPE, even though fringe firms in MME consider the possibility of becoming dominant when investing,they do not have immediate incentives to invest to deter the growth of other firms, because their state is notmonitored by competitors unless the fringe firm becomes dominant. As a consequence, in some states, fringefirms in MME underinvest relative to MPE and this is an important driver in the differences in consumersurplus between MME and MPE, which in all instances is larger in the latter. However, we expect thatMME may provide an even better approximation to MPE as the number of firms grows, because, in thiscase, the deterrence incentives of small firms in MPE would also get reduced. Unfortunately, we cannot testthis hypothesis because we cannot compute MPE for larger industries.24

A second source of approximation error for MME relative to MPE could be that in MME we use approx-imations to the single-period profit function that only depend on a single moment. However, when numer-ically comparing the long-run average difference between the ‘approximate’ single-period profit functionwith the exact profit function, we observe that this does not seem to play an important role. For the capacitycompetition model, there is no difference; fringe firms are small relative to the size of the market and they

24In the range of industries we compute, fringe firms are still reasonably sized relative to dominant firms even for N = 7. Forthis reason and because there are several moving parts in our numerical experiments (e.g., typically the average number of dominantfirms increases as N increases) the approximation errors in Table 2 do not necessarily always decrease as N increases.

25

always produce at full capacity in all instances. For the quality ladder model, prices in both profit functionsare slightly different but the average differences in profits are always less than 1% over all instances.

A third source of approximation error for MME relative to MPE is that moments may not be sufficientstatistics to predict the evolution of the industry. As previously mentioned, in our numerical experiments wealways keep track of a single moment, namely, the one suggested in Section 6.1, for each profit function.25

As we argued there, these are natural moments to keep track of and it is encouraging that we obtain goodapproximations to MPE in the two models. If in certain applications using a single moment is not enough toachieve an accurate approximation, one could add an additional moment. One informative statistic that couldpotentially improve the approximation is the number of fringe firms that have a high chance of becomingdominant soon. For example, in our model this could be the number of fringe firms that are close to thethreshold state x. We tried adding this statistic in our experiments and in general it did not improve theapproximation significantly. Instead, in our experiments adding dominant firm slots significantly improvedthe approximation, as we discuss next.

A fourth source of approximation error for MME relative to MPE is that in the former there is a po-tentially small number of dominant firm slots. In fact, in our numerical experiments we find that havingenough slots to accommodate fringe firms transitioning to become dominant is important for providing ac-curate approximations. For the results presented in Table 2, the number of slots is typically large enoughso there is at least one slot empty in 90% or more of the industry states in the recurrent class. Further, with3 dominant slots, the average number of dominant firms in the experiments is between 1 and 2. This isimportant to avoid distorting investment incentives of fringe firms that are about to become dominant. Forexample, in the low investment cost instance of the capacity competition model for N = 6, decreasing thenumber of dominant slots from 3 to 2 and then to 1, increases the error in approximating consumer surplusfrom 7.6% to 18.1% and then to 37.5%, respectively. On the other hand, increasing the number of dominantslots to 5 improves the approximation of consumer surplus to only 4.3.% (without significantly improvingthe approximation of producer surplus and total investment), with almost a four-fold increase in the size ofthe state space. The relatively small improvement in approximation errors does not seem to compensate forthe large increase in computational cost.

Finally, we also study the impact of firms’ perceived transition probabilities outside the recurrent classon MME. Recall that the perceived transition kernel used in our experiments (defined in Section 4.3) onlyspecifies transitions in the recurrent class for given strategies; transitions outside the recurrent class arearbitrary. In the results presented in Table 2, we assume that firms’ perceptions in states outside the recurrentclass are such that the current value of the moment remains the same in the next time period, which we call‘status-quo’ perceptions. In our numerical experiments, we show that small perturbations around status-quo perceptions (e.g., instead of assuming that the moment stays the same in the next time period, firmsassume that the moment transitions to a near-by value) have a minor impact on MME outcomes. We alsorun experiments with extreme ‘optimistic perceptions’; in this case, firms’ perceptions in states outside the

25If the set of feasible values the moment can take is large due to the presence of a potentially large number of fringe firms, wecould reduce the state space of moments using a manageable grid and interpolate the value function in between points of the grid.We experimented with this type of setting using linear interpolation. Further, in the experiments in the beer industry in Section 7we use a bicubic spline to interpolate between grid points when computing firms’ optimal strategies (see, e.g., Judd (1998)). Wenote that changing the fineness of the grid within reasonable ranges did not change the results significantly.

26

recurrent class are that the moment transitions to its lowest possible value regardless of its current value.We observe that the differences in MME outcomes between these two modes of perceptions outside therecurrent class could be large in general; however, these differences tend to decrease as the number of firmsin the industry increases.26 For robustness, users should experiment with different specifications for theperceptions outside the recurrent class in their applications.

7 Large-Scale Application: The Beer Industry

In the previous section we provided results for small-scale numerical experiments to be able to compareMME with MPE. In this section, to show how our approach increases the applicability of dynamic oligopolymodels, we provide an application of MME in a large industry for which MPE is infeasible to compute.

Some questions that have puzzled economists for decades are: What are the determinants of marketstructure? Why do some industries become dominated by a handful of firms while still holding many smallfirms? How does the resulting market structure affect market outcomes? We believe that our approach canbe used to shed light on these questions, and more broadly, to develop counterfactuals in different empiricalsettings in which the market structure is endogenously determined in a fully dynamic model.

As an example, we perform numerical experiments that are motivated by the long trend towards concen-tration in the beer industry in the US during the years 1960-1990. In the course of those years, the number ofactive firms dropped from about 150 to 30, and three industry leaders emerged: Anheuser-Busch (owner ofBudweiser brand among others), Miller, and Coors. Two competing explanations for this trend are commonin the literature (see Tremblay et al. (2005)): an increase in the minimum efficient scale (MES), and anincrease in the importance of advertising that with the emergence of national television benefited big firms.The role of advertising as an ‘endogenous sunk cost’ in determining market structure is discussed in detailin Sutton (1991) (see Chapter 13 for a discussion of the beer industry). In this section we calibrate a modelof the beer industry, similar to the dynamic advertising model by Doraszelski and Markovich (2007), toexamine the role advertising may have on market structure. We emphasize that it is not our goal to developa complete and exhaustive empirical model of the beer industry, but rather to illustrate our approach in asetting of potential empirical relevance.

The model is similar to Example 6.1, where xit is the goodwill of firm i in period t with associatedmarket share

σ(xit, st) ∝ (xit)α1(Y − pit)α2 .

Firms invest in advertising to increase their goodwill stock over time and compete in prices every period.The number of dominant firms is determined endogenously in the equilibrium, with a maximum numberof three. The transitions between goodwill states are similar to those introduced in Section 6.2, but we usea multiplicative growth model (as opposed to an additive one) that has been previously used in empiricalapplications of advertising (Roberts and Samuelson, 1988). Firms become dominant when their individualgoodwill level becomes larger than a predetermined value.

26For example, in the low investment cost instance of the quality ladder model, the difference in long-run expected consumersurplus between MME with status-quo and optimistic perceptions is 15.5%, 4.4%, and 1.1% for, N = 6, N = 10, and N = 20,respectively.

27

The numerical experiments examine the effect of different specifications of the contribution of goodwillon firms’ profits as captured by the parameter α1. Firms keep track of the α1-th un-normalized moment andMME is computed for each parameter specification. We say that the profit function exhibits: decreasingreturns to advertising (DRA) if α1 < 1, constant returns to advertising (CRA) if α1 = 1, and increasingreturns to advertising (IRA) if α1 > 1. We consider three specifications of returns to advertising with αf1and αd1 controlling the returns to advertising for fringe and dominant firms, respectively. These take valuesin (αf1 , α

d1) ∈ αD, αC × αD, αC , αI, where (αD, αC , αI) = (.85, 1, 1.1). The three cases under

consideration are: (1) DRA-DRA with (αD, αD), (2) DRA-CRA with (αD, αC), and (3) CRA-IRA with(αC , αI).

We calibrate the model parameters from a variety of empirical research that studies the beer industryor related advertising settings.27 For example, α2 and Y are chosen to match the price elasticity in theaverage price, and the average sell-off value is taken from the sales’ price of used manufacturing plants.See Appendix G for a list of the parameters and their sources. We emphasize that each of our experimentsincludes 200 firms and 29 different individual states, which makes it much larger than any problem that canbe solved if MPE was used as an equilibrium concept.

Figure 1 plots (on a log-scale) the average goodwill distribution of firms for the three cases, and Table 3reports some average industry statistics. The experiments suggest that higher returns to advertising indeedgive rise to more right-skewed size distributions, as is expected. Indeed, it is clear that in the DRA-DRA casethere is on average a vacancy in the dominant tier, and dominant firms are not much bigger than the biggestfringe firms. In both DRA-CRA and CRA-IRA the industry is much more concentrated and dominant firmsare much larger than fringe firms. Also, dominant firms remain dominant for a much longer period oftime. In addition, as the industry becomes more concentrated, there are fewer incumbent fringe firms, theyare smaller, and they spend less time in the industry on average. In that sense, large dominant firms deterthe growth of fringe firms. Overall, our results show that increasing returns to advertising could yield aconcentrated market structure with a few dominant firms like in the beer industry.

Industry averages CRA-IRA DRA-CRA DRA-DRAConcentration ratio C2 0.42 0.36 0.13Concentration ratio C3 0.53 0.44 0.17First moment (normalized by # fringe) 1.09 1.22 1.74Active fringe firms (#) 147 166 180Active dominant firms (#) 3 3 2.3Size incumbent fringe (goodwill) 1.1 1.4 2Size dominant (goodwill) 83.4 64.9 24.1Entrants per period (#) 16.5 12.3 7.6Time in industry fringe 12.9 16 26.2Time as dominant 1536 360 21

Table 3: Average industry statistics

27We thank Carol and Victor Tremblay for providing supplementary data.

28

!

"!

#!

$!

!%& " # ' ( ") $# )' "#(

!"#$%&'

())*+$,,

*+,

-+*

-+-

Figure 1: Long-run average size distribution (un-normalized) of firms in log-scale. The curve lines representfringe firms for the different levels of returns to advertising. The squares represent dominant firms for D-D,the triangles for D-C, and the circles for C-I.

Finally, we examine the effect fringe firms have on dominant firms by computing MPE with dominantfirms only. In fact, ignoring fringe firms is a common practice in the applied literature to simplify computa-tion. In order to make the comparison fair we normalize the profit function to compute MPE by fixing thefringe firms’ moment to its MME long-run average. We obtain that, for example, in the DRA-CRA case theaverage size of dominant firms drops by 17% and the average size of the smallest dominant firm falls around50% in the MPE relative to the MME values. This suggests that deterring entry and pushing down invest-ment from fringe firms are important determinants in dominant firms’ investment incentives. The collectivepresence of fringe firms, in spite of their weak individual market power, disciplines dominant firms andforces them to invest more than in an oligopoly in which there are only dominant firms. We conclude thatexplicitly modeling fringe firms may have important effects on conclusions derived from counterfactuals.

8 MME as Behavioral Model: Approximation Error Bounds

In this section we introduce approximation error bounds that can be used to justify the use of MME as anappealing behavioral model on its own. We also provide numerical experiments to test our bounds. Finally,we discuss the applicability of our bounds to different economic models.

MME strategies would provide an appealing behavioral model if they are close to optimality, that is,assuming that competitors play MME strategies themselves, unilaterally deviating to a strategy that keepstrack of the full underlying fringe state would not improve expected discounted profits by much. In thiscase, it may be reasonable to assume firms use MME strategies, as unilaterally deviating to more complexstrategies does not significantly increase payoffs, and doing so may involve costs associated with gatheringand processing more information.

29

To formalize this notion, for MME strategies (µ, λ), we define the value of the full information deviation:

∆µ,λ(x, s) = supµ′∈M

V (x, s|µ′, µ, λ)− V (x, s|µ, λ). (11)

Note that V (x, s|µ, λ) provides the actual expected discounted profits a firm would get if all firms play MMEstrategies. Similarly, supµ′∈M V (x, s|µ′, µ, λ) provides the expected discounted profits if the firm wouldunilaterally deviate to the best response Markov strategy that keeps track of the full industry state. We usethe value of the full information deviation to measure the extent of sub-optimality of MME strategies. If thisvalue is small, the MME strategy achieves essentially the same profits compared to the best response Markovstrategy. In this sense, the value of the full information deviation is similar to the notion of ε−equilibrium.Note that we expect this value to be small when moments in MME are close to being sufficient statistics ofthe future evolution of the industry; that is, when Pµ,λ[θ′|s] ≈ Pµ,λ[θ′|s], and the industry state s does notprovide additional information to predict the future industry evolution beyond the moment-based industrystate s.28

While in small-scale experiments like those presented in Section 6.3 we can compute the value of thefull information deviation exactly, this is not possible in large-scale applications like the one presented inSection 7. In these instances, computing a best response is almost as computationally challenging as solvingfor a MPE. Of course, neither could we compare MME with MPE in these cases.

Thus motivated, we introduce a novel computationally tractable error bound that upper bounds the valueof the full information deviation. This error bound is derived by observing a connection between the unilat-eral deviation problem of a firm in MME and problems in robust dynamic programming (RDP); see Iyengar(2005) for a reference to the RDP literature.29 We note that to the best of our knowledge ideas from RDPhave not been previously used to derive sub-optimality error bounds for dynamic programs with partialinformation as we do here.

This bound is useful because it allows one to evaluate whether the state aggregation is appropriate orwhether a finer state aggregation is necessary, for example, by adding more moments. In fact, if the valueof the full information deviation is small, it is plausible that firms would use the relatively simpler MMEstrategies as opposed to more complex Markov strategies that do not yield significant additional benefits.

Before deriving the bound, we observe that while generally ε−equilibria may not be close to equilibriaeven for small ε (see, e.g., Fudenberg and Levine (1986)), in Appendix D we are able to show that if thevalue of the full information deviation becomes small for all initial states (e.g., by including more momentsin MME), then MME strategies approach MPE strategies. This result provides a connection between theinterpretation of MME as an appealing behavioral model on its own provided in this section with that of

28An alternative to computing the value of the full information deviation is determining whether additional contemporaneousmoments or higher order lags would be significant and explain a significant fraction of the variation of the next value of the MMEmoments in a regression. This would indicate that the moments used in MME are not Markov and that perhaps additional momentsshould be considered. The advantage of this method is that it may be simpler to implement than measuring the value of the fullinformation deviation. The disadvantage is that it does not directly consider payoffs.

29RDP considers Markov decision processes with unknown transition kernels in which the decision maker chooses a ‘robust’strategy to mitigate this ambiguity. In our case, observing the moment, but not the underlying industry state, is equivalent to havingambiguity about the underlying transition probabilities. Therefore, upper bounding the extent of a unilateral deviation of an agentin MME can be reformulated as a RDP problem.

30

MME as an approximation to MPE provided in Section 6.

8.1 Derivation of Error Bound

Let (µ, λ) be a fixed MME strategy for the remainder of this section. Recall that V (x, s|µ, λ) is the actualvalue of playing MME strategies starting from (x, s). Denote by V ∗(x, s|µ, λ) = supµ′∈M V (x, s|µ′, µ, λ).The value of the full information deviation is ∆µ,λ(x, s) = V ∗(x, s|µ, λ) − V (x, s|µ, λ) (see (11)). Notethat given MME strategies (µ, λ), V (x, s|µ, λ) can be computed using forward simulation. However, theproblem of finding the optimal strategy that achieves V ∗ is highly dimensional. Instead we find an upperbound to V ∗ with which we can upper bound ∆µ,λ. To do so we construct a robust Bellman operator asfollows. For every s = (θ, d, z) ∈ S define the consistency set Sf (s) = f ∈ Sf

∣∣θ(f) = θ, that is, theset of all fringe distributions that are consistent with the value of the moments in state s. Note that momentsare not sufficient statistics for the evolution of the industry, because typically Sf (s) is not a singleton anddifferent fringe distributions in the consistency set may have different future evolutions. Now, define

(TRV )(x, s) = supι∈Iρ≥0

supf∈Sf (s)

π(x, s) + E

[φ1φ ≥ ρ

+ 1φ < ρ[− c(x, ι) + β E µ,λ[V (xi,t+1, st+1)

∣∣xit = x, st = (f, d, z), ι]]]

, (12)

where V ∈ V is a bounded vector, V : X × S → < and 1· is the indicator function. Recall that weassume all competitors use MME strategies (µ, λ). The robust Bellman operator is defined on X × S and itis identical to the Bellman operator associated with the best response in C1 in the definition of MME, exceptthat the firm can also choose any underlying fringe state consistent with observed moment θ.

In Lemma E.1 in the Appendix, we show that the robust Bellman operator TR has a unique fixed pointV ∗ that we call the robust value function. The next result, also proved in the appendix, relates it to theoptimal value function V ∗.

Theorem 8.1. For all (x, s) ∈ X × S

V ∗(x, s) ≤ V ∗(x, s),

where s is the moment-based industry state that is consistent with s, that is, θ = θ(f)

In essence, the robust value function resolves the indeterminacy of moment transitions by choosing thebest fringe state from the associated consistency set. Intuitively, this should provide an upper bound for V ∗.Therefore, we can bound the value of the full informational deviation with ∆(x, s) = V ∗(x, s) − V (x, s),where s is the moment-based industry state associated to s. The advantage of computing V ∗ over V ∗ isthat TR operates on the moment-based state space S, which is much smaller than S . Nevertheless, thecomputation of V ∗ is still demanding, as we explain next.

Finding the fixed point of the operator TR is generally a hard computational problem, in fact it is NP-complete (Iyengar, 2005, §3). This is not surprising, since the optimization over consistency sets may bevery complex. However, iterating the operator TR becomes much simpler if there is a large number of

31

fringe firms. In this case, the one-step transition of the fringe state is close to being deterministic, because,conditional on the current state, the transitions of individual fringe firms average out at the aggregate levelby a law of large numbers. When one-step transitions are deterministic, finding the optimal consistent f in(12) is equivalent to choosing the next moment from a set of moments that are accessible from the currentindustry state. This considerably simplifies the computation of the inner maximization in (12), since themoment accessibility sets are low dimensional. Moreover, characterizing these accessibility sets can bedone efficiently. This procedure, described in detail in Appendix H, is tractable for problems for whichcomputing MME is tractable. We note that this method, based on deterministic transitions for the fringestate, only provides an approximation to the robust upper bound for models with a large but finite numberof fringe firms.30

It is simple to observe that if moments are sufficient statistics to predict the future evolution of the in-dustry, then the bound is tight. If this is not the case, the gap between the actual value of the full informationdeviation and the quantity computed with the robust bound depends on the distance V ∗(x, s) − V ∗(x, s).To lower this gap and make the bound tighter we can compute the robust bound over a refinement of themoment-based state space used in the equilibrium computation by including moments in the robust operatornot included in MME. For example, we could add another contemporaneous moment like a quantile, or thenumber of fringe firms that are close to becoming dominant, or we could add a lagged moment. This willreduce the size of the consistency sets, making the bound tighter, albeit increasing the computational cost.We illustrate this with numerical experiments in the next subsection.

8.2 Numerical Experiments

We have done extensive numerical experiments using the robust bound. We present some results on twosimplified models similar to the beer industry experiments of Section 7 with a single dominant firm thatoperates under constant returns to advertising. In the first model N = 200 and fringe firms can enter andexit the market (model E), in the second N = 100 and firms cannot enter or exit the industry (model NE).For each model we vary αf1 , the parameter controlling fringe firms’ returns to advertising, from .45 to .65.The lower the returns to advertising, the smaller the reward from investment and the more homogenousthe fringe firms are in equilibrium. Since the moment is a sufficient statistic of the future evolution of theindustry when fringe firms are homogenous, we expect the error bound to be lower for low values of αf1 .

In both models E and NE, to simplify computation, we use the parametric approach described inExample 4.2 to construct the perceived transition kernel. In particular, we assume a linear relationshipθt+1 = ξ0(dt) + ξ1(dt)θt, and the parameters (ξ0(d), ξ1(d))d∈Sd are taken to fit the simulated transitionsusing ordinary least squares. To simplify, we assume there is no aggregate shock. We report in Figures2a and 2b the expected error bound for both dominant and fringe firms as a percentage of the actual value

30It is possible to formally derive a probabilistic version of the robust bound that corrects for the fact that the number of fringefirms N is finite using standard probability bounds. Loosely speaking, by a central limit theorem the correction term is of the order√N . However, given our numerical experience, we believe that in many settings of interest in which N is large, the square root N

correction will not have a significant impact for any practical purposes.

32

(a) No Entry/Exit (b) Entry/Exit

Figure 2: Error Bound

function. Namely, we plot

E µ,λ

[ ∆µ,λ(x, s)

V (x, s|µ, λ)

],

where the expectation is taken with respect to the stationary distribution of the underlying industry states.31 In addition to the standard robust bound, we consider a variation where a lagged fringe state enters thecomputation of the consistency sets. This reduces the size of these sets and so decreases the gap between theactual value of the full information and the error bound. Indeed, we see that the lagged state decreases theerror bound, in particular in model E. Adding further contemporaneous or lagged moments should generallymake the error bound even tighter.

The experiments agree with our conjecture that a more homogenous fringe tier (low αf1 ) will result ina lower error bound. In addition, we see that the error bounds are generally lower for model NE. This isexplained by the better fit of the linear moment transitions without entry and exit. Entry and exit decisions areinherently nonlinear, and therefore, the assumed linear moment transitions approximate the actual momenttransitions in NE better than in E, resulting in a lower value of the bound.

These experiments suggest that the robust bound can be useful to test the extent of sub-optimality ofMME strategies in terms of a unilateral deviation. Based on our numerical results, we find that depending onthe model, the bound can be sometimes tight indicating that the extent of a unilateral deviation is small, but itcan also be loose. In the latter cases, refining the state space by adding additional moments when computingthe robust bound can be helpful to make the bound tighter and more useful. Also, adding additional momentsto firms’ MME strategies should generally improve the accuracy of the perceived transition kernel, and it isplausible to expect that this would decrease the bound as well. There may still be settings in which even afteradding several moments, the bound will be loose. This is not entirely surprising as the error bound derivedin this paper is very generic and does not use problem-specific information. We leave potential refinements

31For dominant firms, the expectation is also taken with respect to the stationary distribution of their individual state evolution.For fringe firms, x is taken to be the most-visited fringe state.

33

of our bound that use problem-specific information as matter for further research.

8.3 Extensions to Other Economic Models

In this section we discuss how our bounds could potentially be used in other economic models, in particular,in dynamic games of asymmetric information and in macroeconomic model with heterogeneous agents.

8.3.1 Dynamic Games of Asymmetric Information

In the dynamic games of asymmetric information studied by Fershtman and Pakes (2012) a firm keeps trackof the history it has observed up to that period. In principle, optimal strategies could depend on the infinitehistory of observations, because past history may provide useful information about current competitors’private states. To make the model tractable, the authors assume that histories are finite and discuss environ-ments in which this assumption may be plausible, for example, when there is public periodic revelation ofall information. Our error bounds and related ideas can be used to upper bound the extent of a unilateraldeviation from the finite history REBE of Fershtman and Pakes (2012) to a strategy that keeps track of theinfinite history.

In the spirit of approximating equilibrium in a game of complete information, our error bounds assumethat the deviant firm has access to the full current industry state. However, in a game of asymmetric in-formation it is more appropriate to consider a unilateral deviation to a strategy that instead keeps track ofthe observed infinite history. Note that typically, fixing competitors’ strategies, a unilateral deviation to astrategy of the former type provides larger expected discounted profits than one of the latter type, becausehaving access to the full current state typically provides more information than having access to the observedinfinite history. For example, while in the former the firm knows the exact current state of competitors, thismay not necessarily be true in the latter. As a consequence, our error bounds are valid for the model ofasymmetric information; however, tighter bounds could potentially be derived.

With the purpose of deriving tighter bounds we could formulate the best response problem of a firm inthe dynamic game of asymmetric information as a partially observable Markov decision process (POMDP);see, for example Lovejoy (1991) for a reference on POMDPs. Then, we could use the results from White IIIand Scherer (1994) that derive upper bounds to the optimal value function of the POMDP that keeps track ofthe entire infinite history, based on the POMDP that only keeps track of a finite history of length M . WhiteIII and Scherer (1994) also shows that the bounds become weakly tighter as M grows.32

In summary, our approach and that based on ideas from POMDPs allow to derive bounds on the extent ofa unilateral deviation for fixed competitors’ strategies from REBE with a given memory length to a strategythat keeps track of the full infinite history.

32They also provide sufficient conditions under which the extent of the unilateral deviation becomes zero for finite M . Thesufficient conditions essentially establish that distant history becomes irrelevant and are related to the literature on contraction ratesand forecast horizons in fully observed MDPs (see references in Lovejoy (1991)). However, checking such sufficient conditionsover model primitives depends on the underlying stochastic process and is typically quite complex. For example, Bernhardt andTaub (2012) provide a dynamic duopoly model with cost and demand private shocks for which far away history does not becomeirrelevant for decision making.

34

8.3.2 Macroeconomic Models with Heterogeneous Agents

Our approach is related to the important literature in macroeconomics that studies models with heteroge-neous agents in the presence of aggregate shocks. Notably, Krusell and Smith (1998) studies a stochasticgrowth model with heterogeneity in consumer income and wealth. Other papers focus on firm-level hetero-geneity in capital and productivity (see, e.g., Khan and Thomas (2008) and Clementi and Palazzo (2016)).All of these models assume a continuum of agents and therefore agents’ dynamic programming problemsare infinite dimensional; strategies depend on the distribution over individual states. Motivated by the sem-inal idea in Krusell and Smith (1998), in these papers agents are assumed to keep track of only a smallnumber of statistics of this distribution to simplify the problem. Note that these macroeconomic modelsdo not incorporate dominant agents, so our framework accommodates them by considering moment-basedstates that include moments and the value of the aggregate shock only.

While our approach is partially inspired by this previous literature, we believe that in turn our results canalso be useful in these macroeconomic models. For concreteness, we focus on the seminal paper of Kruselland Smith (1998) that show that in their model the first moment of the wealth distribution is essentially asufficient statistic for the evolution of the economy, a property they call ‘approximate aggregation’. Thereason is that agents’ equilibrium decisions turn out to be almost linear in their state. In fact, if decisionswere exactly linear, the first moment would be an exact sufficient statistic.

While the approximate aggregation property has been shown to be quite robust in this class of models,Krusell and Smith (2006) and Algan et al. (2013) acknowledge that it does not hold for all models and maynot hold for important models considered in the future. For this reason, it is important to understand theboundaries of approximate aggregation by testing its accuracy. Den Haan (2010) describes the limitationsof commonly used accuracy tests, such as the so called ‘R2 test’ and provides other alternatives. These testsoften try to measure the accuracy of the perceived transition kernel proposed. We believe that our compu-tationally tractable error bounds provide a more direct test of the validity of a state aggregation technique,because it directly measures improvements in payoffs, in particular, on how much an agent can improve itsexpected discounted utility (in monetary terms) by unilaterally deviating from the approximate equilibriumstrategy to a strategy that keeps track of the full distribution of wealth. The bound is useful because it canhelp determine whether approximate aggregation holds or a finer approximation is required.

9 Conclusions and Extensions

In this paper we introduced a novel framework to study dynamic oligopolies in concentrated industries thatopens the door to study new issues in the empirical analysis of industry dynamics. In particular, we firstintroduced MME and an algorithm to compute it. We also discussed important implementation issues forapplied work and illustrated our approach in a large-scale application. Then, we provided results to justifyMME as an approximation by comparing it to MPE and by introducing error bounds.

We believe that our work suggests several other future directions for research in dynamic oligopolies.First, our error bound is very general and we envision that tighter bounds can be derived using problem-specific information. Second, in our definition of MME, firms keep track of current moments of the fringe

35

state. Our equilibrium concept and error bound can be modified to allow firms to keep track of past values ofmoments. Third, MME with the perceived transition kernels used in our experiments are intended to studythe long-run equilibrium market structure. We believe that an appropriate modification of our approachwould allow the study of short-term dynamics and how the market structure would evolve in few years aftera policy or an environmental change. Finally, in this paper we focused on ‘capital accumulation’ games without investment spillovers. Extending our framework to a broader set of EP-style models, for example, toinclude learning-by-doing, network effects, or dynamic auctions, would further increase its applicability.

In Section 8.3 we discussed how our methods, specifically our error bounds, can be extended to otherimportant classes of economic models. In addition, we envision that some of our ideas could potentially alsobe useful to study dynamic models with forward looking consumers. For example, in a dynamic oligopolymodel with durable goods, firms may need to make pricing and investment decisions, keeping track ofthe distribution of consumers’ ownership, which is a highly dimensional object (e.g., Goettler and Gordon(2011)). Our ideas may be useful in this context as well, where one could replace this distribution by someof its moments. We leave all these extensions for future research.

References

Algan, Y., O. Allais, W. J. Den Haan, P. Rendahl. 2013. Solving and simulating models with heterogeneousagents and aggregate uncertainty. Handbook of Computational Economics Vol. 3. North-Holland, Elsevier.

Altug, S., P. Labadie. 1994. Dynamic Choice and Asset Markets. Academic Press.

Bajari, P., C. L. Benkard, J. Levin. 2007. Estimating dynamic models of imperfect competition. Economet-rica 75(5) 1331 – 1370.

Benkard, C. L. 2004. A dynamic analysis of the market for wide-bodied commercial aircraft. Review ofEconomic Studies 71(3) 581 – 611.

Benkard, C. L., A. Bodoh-Creed, J. Lazarev. 2010. Simulating the dynamic effects of horizontal mergers:U.S. airlines. Working Paper, Stanford.

Benkard, C. L., P. Jeziorski, G. Y. Weintraub. 2015. Oblivious equilibrium for concentrated industries. TheRAND Journal of Economic 46(4) 671–708.

Bernhardt, D., B. Taub. 2012. Oligopoly learning dynamics. Working Paper, UIUC.

Bertsekas, D. P. 2001. Dynamic Programming and Optimal Control. 2nd ed. Athena Scientific.

Bertsekas, D. P., S. Shreve. 1978. Stochastic Optimal Control: The Discrete-Time Case. Academic PressInc.

Besanko, D., U. Doraszelski. 2004. Capacity dynamics and endogenous asymmetries in firm size. RANDJournal of Economics 35(1) 23 – 49.

36

Besanko, D., U. Doraszelski, Y. Kryukov, M. Satterthwaite. 2010. Learning-by-doing, organizational for-getting, and industry dynamics. Econometrica 78(2).

Besanko, D., M. K. Perry, R. H. Spady. 1990. The logit model of monopolistic competition: Brand diversity.The Journal of Industrial Economics 38(4) 397 – 415.

Blundell, R., T.M. Stoker. 2005. Heterogeneity and aggregation. Journal of Economic Literature XLIII 347– 391.

Caplin, A., B. Nalebuff. 1991. Aggregation and imperfect competition - on the existence of equilibrium.Econometrica 59(1) 25 – 59.

Cho, G.E., C.D. Meyer. 2001. Comparison of perturbation bounds for the stationary distribution of a Markovchain. Linear Algebra and its Applications 335(1-3) 137 – 150.

Clementi, G.L., D. Palazzo. 2016. Entry, exit, firm dynamics, and aggregate fluctuations. American Eco-nomic Journal: Macroeconomics 8(3) 1 – 41.

Collard-Wexler, A. 2011. Productivity dispersion and plant selection. Working Paper, NYU.

Collard-Wexler, A. 2013. Demand fluctuations in the ready-mix concrete industry. Econometrica 81(3)1003–1037.

Collard-Wexler, A. 2014. Mergers and sunk costs: An application to the ready-mix concrete industry.American Economic Journal: Micro 6(4) 407–447.

Corbae, D., P. D’Erasmo. 2013. A quantitative model of banking industry dynamics. Working paper, UTAustin.

Corbae, D., P. D’Erasmo. 2014. Capital requirements in a quantitative model of banking industry dynamics.Working Paper, Maryland University.

Den Haan, W. J. 2010. Assessing the accuracy of the aggregate law of motion in models with heterogeneousagents. Journal of Economic Dynamics and Control 34 79 – 99.

Dixit, A. K., J. E. Stiglitz. 1977. Monopolistic competition and optimum product diversity. AmericanEconomic Review 67(3) 297 – 308.

Donoghue, W. F. 1969. Distributions and Fourier Transforms. Academic Press.

Doraszelski, U., K. Judd. 2011. Avoiding the curse of dimensionality in dynamic stochastic games. Quan-titative Economics 3(1) 53–93.

Doraszelski, U., S. Markovich. 2007. Advertising dynamics and competitive advantage. RAND Journal ofEconomics 38(3) 557–592.

Doraszelski, U., A. Pakes. 2007. A framework for applied dynamic analysis in IO. Handbook of IndustrialOrganization, Volume 3. North-Holland, Amsterdam.

37

Doraszelski, U., M. Satterthwaite. 2010. Computable markov-perfect industry dynamics. RAND Journal ofEconomics 41 215–243.

Ericson, R., A. Pakes. 1995. Markov-perfect industry dynamics: A framework for empirical work. Reviewof Economic Studies 62(1) 53 – 82.

Farias, V., D. Saure, G.Y. Weintraub. 2012. An approximate dynamic programming approach to solvingdynamic oligopoly models. The RAND Journal of Economics 43(2) 253–282.

Fershtman, C., A. Pakes. 2012. Dynamic games with asymmetric information: A framework for empiricalwork. Quarterly Journal of Economics 127(4) 1611 – 1661.

Fudenberg, D., D. K. Levine. 1986. Limit games and limit equilibria. Journal of Economic Theory 38(2)261 – 279.

Gallant, A. R., H. Hong, A. Khwaja. 2016. The dynamic spillovers of entry: An application to the genericdrug industry. Forthcoming, Management Science.

Goettler, R. L., B. Gordon. 2011. Does AMD spur Intel to innovate more? Journal of Political Economy119(6) 1141–1200.

Gowrisankaran, G., T. Holmes. 2004. Mergers and the evolution of industry concentration: results from thedominant-firm model. RAND Journal of Economics 35(3) 1–22.

Iacovone, L., B. Javorcik, W. Keller, J. Tybout. 2015. Supplier responses to Walmart’s invasion in mexico.Journal of International Economics 95(1) 1–15.

Iskhakov, F., J. Rust, B. Schjening. 2015. Recursive lexicographical search: Finding all markov perfectequilibria of finite state directional dynamic games. Forthcoming, Review of Economic Studies.

Iyengar, G. 2005. Robust dynamic programming. Mathematics of Operations Research 30(2) 257–280.

Jeziorski, P. 2014. Estimation of cost synergies from mergers: Application to us radio. RAND Journal ofEconomics 45(4) 816–846.

Jia, P., P. Pathak. 2015. The costs of free entry: An empirical study of real estate agents in greater Boston.The RAND Journal of Economic 46(1) 103–145.

Judd, K. 1998. Numerical Methods in Economics. MIT Press.

Kalouptsidi, M. 2014. Time to build and fluctuations in bulk shipping. American Economic Review 104(2)564–608.

Khan, A., J.K Thomas. 2008. Idiosyncratic shocks and the role of nonconvexities in plant and aggregateinvestment dynamics. Econometrica 76(2) 395 – 436.

Krusell, P., A. A. Smith, Jr. 1998. Income and wealth heterogeneity in the macroeconomy. Journal ofPolitical Economy 106(5) 867–896.

38

Krusell, P., A. A. Smith, Jr. 2006. Quantitative macroeconomic models with heterogeneous agents. Advancesin Economics and Econometrics: Theory and Applications, Ninth World Congress.

Lee, R. S. 2013. Vertical integration and exclusivity in platform and two-sided markets. American EconomicReview 103(7) 2960–3000.

Lovejoy, W. S. 1991. A survey of algorithmic methods for partially observed markov decision processes.Annals of Operations Research 18 47 – 66.

Lucas, R. E., E. C. Prescott. 1971. Investment under uncertainty. Econometrica 39(5) 659 – 681.

Maskin, E., J. Tirole. 1988. A theory of dynamic oligopoly, I and II. Econometrica 56(3) 549 – 570.

Maskin, E., J. Tirole. 2001. Markov perfect equilibrium I. observable actions. Journal of Economic Theory100 191 – 219.

Pakes, A., P. McGuire. 1994. Computing Markov-perfect Nash equilibria: Numerical implications of adynamic differentiated product model. RAND Journal of Economics 25(4) 555 – 589.

Pakes, A., P. McGuire. 2001. Stochastic algorithms, symmetric Markov perfect equilibrium, and the ‘curse’of dimensionality. Econometrica 69(5) 1261 – 1281.

Pakes, A., M. Ostrovsky, S. Berry. 2007. Simple estimators for the parameters of discrete dynamic games,with entry/exit examples. RAND Journal of Economics 38(2) 373 – 399.

Qi, S. 2013. The impact of advertising regulation on industry: The cigarette advertising ban of 1971. RANDJournal of Economics 44(2) 215–248.

Resnick, S.I. 1998. A Probability Path. Birkhauser.

Roberts, M. J., L. Samuelson. 1988. An empirical analysis of dynamic, nonprice competition in anoligopolistic industry. The RAND Journal of Economics 19(2) pp. 200–220.

Rojas, C. 2008. Price competition in U.S. brewing. The Journal of Industrial Economics 56(1) 1–31.

Ryan, S. 2012. The costs of environmental regulation in a concentrated industry. Econometrica 80(3) 1019– 1062.

Santos, C. D. 2010. Sunk costs of R&D, trade and productivity: the moulds industry case. Working Paper,U. of Alicante.

Santos, C. D. 2012. An aggregation method to solve dynamic games. Working paper .

Sutton, J. 1991. Sunk Costs and Market Structure. 1st ed. MIT Press.

Sutton, J. 1997. Gibrat’s legacy. Journal of Economic Literature 35(1) 40 – 59.

Sweeting, A. 2013. Dynamic product positioning in differentiated product markets: The effect of fees formusical performance rights on the commercial radio industry. Econometrica 81(5) 1763–1803.

39

Tomlin, B. 2010. Exchange rate volatility, plant turnover and productivity. Working Paper, Boston Univer-sity.

Tremblay, V. J., N. Iwasaki, C. H. Tremblay. 2005. The dynamics of industry concentration for u.s. microand macro brewers. Review of Industrial Organization 26 307–324.

Tremblay, V. J., C. H. Tremblay. 2005. The US Brewing Industry: Data and Economic Analysis, MIT PressBooks, vol. 1. The MIT Press.

Weintraub, G. Y., C. L. Benkard, B. Van Roy. 2008. Markov perfect industry dynamics with many firms.Econometrica 76(6) 1375–1411.

White III, C. C., W. T. Scherer. 1994. Finite-memory suboptimal design for partially observed markovdecision processes. Operations Research 42(3) 439 – 455.

Whitt, W. 1980. Representation and approximation of noncooperative sequential games. SIAM J. on Controland Optimization 18(1) 33–48.

Xu, Y. 2008. A structural empirical model of R&D, firm heterogeneity, and industry evolution. Workingpaper, NYU University.

40

Appendices

A Perceived Transition Kernel for Short-Term Dynamics

Let the industry evolve for the time horizon T of interest under strategies (µ, λ) starting from a given initialstate s0 that describes the initial condition of the industry. Repeat this simulationL times for largeL. Recordthe moment-based states visited, slt = (θlt, d

lt, z

lt) : t = 0, ..., T, l = 1, ..., L. Now, define the following

perceived transition kernel:

Pµ,λ(θ′|s) = (1/L)

L∑l=1

∑Tt=0 1slt = s, θlt+1 = θ′∑T

t=0 1slt = s,

for a set of states s visited at least a certain number of pre-determined times over the simulation runs. Thekernel is defined arbitrarily outside this set.

One issue with the previous definition is that it does not allow for different perceived transition proba-bilities when the same moment-based state s is visited at different time periods over the finite simulation.For example, at t = 0, firms know the initial state f0, while if they observe θ(f0) later in the horizon theydo not. We may want to allow for different transition probabilities in these two situations. To incorporatethis flexibility, however, we would require a non-stationary transition kernel, which in turn would requiregeneralizing MME to a non-stationary equilibrium concept.

B Existence of MME

In this section we provide conditions under which MME exist for the perceived transition kernel defined byequations (2), (4), and (5). In this section we also assume that fringe firms become dominant when growingabove a pre-determined state. Analogous existence results can be proved for transition kernels with similarcontinuity properties. First, we provide the main ideas behind the result that we then formalize in a theorem.

We use Brouwer’s fixed point theorem to show existence of MME. First, it is simple to show that underthe basic assumptions of our model, the strategy space is convex and compact. Second, we need to show thatBR(µ, λ), the MME best response strategy function when competitors play strategy µ and enter accordingto λ, is continuos. One issue though is that in principle the perceived transition kernel may be discontinuousas a function of the strategies in states outside of the recurrent class.

To see this, suppose that for strategies (µ, λ), a given state s is not in the recurrent class Rµ,λ, andtherefore, Pµ,λ is arbitrary for that state. For example, let x be an individual state that is unreachable understrategy µ and let s be a state that contains firms in x. Now, consider a small perturbation of strategy µ (e.g.,a small additional investment in some states) for which the state s becomes recurrent. Now, the perceivedtransition kernel at that state is determined by the expressions (2), (4), and (5), introducing a discontinuity.To preclude this possibility we make the following assumption.

41

Assumption B.1. For any strategies (µ, λ), the Markov process that describes the evolution of the industrystate st ∈ S : t ≥ 0 is irreducible and aperiodic.

The assumption is useful because together with the basic assumptions of our model it implies that allstates s ∈ S are recurrent, and that further, the industry state process admits a unique stationary distribu-tion. Under this assumption one can show that the perceived transition kernel is continuous, and therefore,BR(µ, λ) is continuos.

Assumption B.1 is somewhat strong, but it is satisfied in many empirical models. For example, indiscrete games such as Pakes et al. (2007) the i.i.d. error term provides full support on the set of availablechoices, making the industry state process irreducible. In other models, such as Ericson and Pakes (1995),closer to the ones presented in this paper, additional conditions are required to satisfy Assumption B.1.For example, in our model a set of sufficient conditions that imply Assumption B.1 are the presence ofdepreciation and appreciation shocks in all states and that entry costs and sell-off values have unboundedsupport.

While these conditions may appear strong, we note that Assumption B.1 is satisfied even if the proba-bilities of depreciation and appreciation shocks, and of realizations of entry costs and sell-off values outsidebounded intervals are infinitesimal (but strictly positive). More importantly for practical purposes, in ournumerical experiments we were always able to computationally find an MME over a large set of primitivesthat satisfy the standard assumptions required for the existence of MPE in EP-style models and with out theadditional conditions required to satisfy Assumption B.1, in particular, with out imposing an appreciationshock.33 We think this is the case because in the applications we study strategies that arise as best responsesshare similar and relatively large recurrent classes. We believe this may also be the case in many applicationsof interest.

Now, we present the main result of this section. All proofs are presented in Appendix E.

Theorem B.1. Consider our model with the basic assumptions introduced in Sections 3.1 and 4. Assume theperceived transition kernel is given by equations (2), (4), and (5). Further, suppose Assumption B.1 holds.Then, a MME always exists.

C Moments Become Sufficient Statistics

In this section we present a class of industry dynamic models for which a succinct set of moments essentiallysummarize all payoff relevant information and is close to being sufficient statistics. In these models, thevalue of the full information deviation is zero, or becomes zero asymptotically as the number of fringe firmsgrows large.

A simple class of models for which this holds is when fringe firms are homogeneous, in the sense thatXf is a singleton. In this case, a single moment, namely the number of incumbent fringe firms, is a sufficientstatistic of the future evolution of the industry. Next, we describe a model with firm fringe heterogeneity

33Further, the MME found were always the same when moving from a model with no appreciation shock to one with a smallappreciation shock.

42

that does not allow for entry and exit, in which moments also become sufficient statistics as the number offringe firms grows large.

Constant Returns to Scale Model for Fringe Firms. We take Xf = <+ (with another component thatwe suppress for clarity indicating that the firm is fringe). We assume that there is no entry and exit (thereforeµ = ι), that there are N −D fringe firms, and that there are no transitions between the fringe and dominanttier. The analysis below can be applied to single-period profit functions and transition kernels that dependon any integer moments of the fringe state. However, to simplify the exposition we assume that they bothdepend only on the first moment. A model like Example 6.1 with α1 = 1 would give rise to this type ofprofit function. Accordingly, we assume that firms keep track only of the first un-normalized moment of thefringe state, that is, θt =

∑x∈Xf xft(x). We make the following assumptions on the primitives of fringe

firms only; no restrictions are placed on the primitives of dominant firms.

1. The single-period profit is linear in the fringe firm’s own state, π(x, s) = xπ1(s) + π0(s) withsups∈Sπ1(s), π0(s) <∞. The assumption imposes constant returns to scale.

2. For a fixed state x, the cost function increases linearly with the investment level ι. In addition, themarginal investment cost increases linearly with the state. Formally, c(x, ι) = (cx)ιwith ι ∈ I = <+.

3. The dynamics of a fringe firm’s evolution are linear in its own state: xi,t+1 = xitζ1(ι, st, w1it) +

ζ0(st, w0it), where ι is the amount invested. Each of the sequences of random variables w0it|t ≥0, i ≥ 1 and w1it|t ≥ 0, i ≥ 1 is i.i.d. and independent of all previously defined random quantityand of each other. In addition, we assume the functions ζ0 and ζ1 are uniformly bounded over allrealizations, investment levels, and industry states.34

We begin by showing that for any perceived kernel Pµ the corresponding best response investmentstrategy for a fringe firm does not depend on its own individual state.35 All proofs are relegated to AppendixE.

Lemma C.1. Consider the constant returns to scale model described above. For any MME strategy µ andfor every x, x′ ∈ Xf and s ∈ S we have µ(x, s) = µ(x′, s) = µ(s).

We use this result to characterize the evolution of the moment under MME strategies as the number offringe firms grows. To obtain a meaningful asymptotic regime, we scale the market size together with thenumber of fringe firms. Formally, we consider a sequence of industries with growing market size m; marketsize would typically enter the profit function, through the underlying demand system like in Example 6.1.Therefore, we consider a sequence of markets indexed by market sizes m ∈ N with profit functions denotedby πm. We assume the number of firms increases proportionally to the market size, that is Nm = Nm−D,for some constant N > 0. We assume that all other model primitives, including the maximum numberof dominant firms, are independent of m. Quantities associated to market size m are indexed with thesuperscript m. We state the following assumption.

34The assumption about linear transitions is similar to assuming Gibrat’s law in firm’s transitions (Sutton, 1997).35This result is similar to Lucas and Prescott (1971) that studies a dynamic competitive industry model with similar assumptions

to ours, but with deterministic transitions and stochastic demand.

43

Assumption C.1. Let µm : m ≥ 1 be a sequence of MME strategies followed by all firms in market m.Then, there exists a compact set X f ⊂ Xf such that for all m ≥ 0, t ≥ 0, and i ∈ Fmt , P[xmit ∈ X f ] = 1.

While the previous assumption imposes conditions on equilibrium outcomes, it is quite natural in thiscontext; MME is a sensible equilibrium concept only if fringe firms do not grow unboundedly large. Wehave the following result.

Proposition C.1. Consider the constant returns to scale model under Assumption C.1, and suppose that foreverym ≥ 1 all firms use MME strategy µm. For a given t, conditional on the realizations of xit ∈ X f |i ∈Fmt and smt , we have

(1/Nm)θmt+1 −[ζm1 (smt ) (1/Nm) θmt + ζ0(smt )

]→ 0, a.s.,

as m → ∞, where θmt =∑

i∈Fmtxit =

∑x∈Xf xf

mt (x), ζm1 (smt ) = E µm [ζ1it(µ

m(smt ), smt , w1it)], andζ0(smt ) = E µm [ζ0it(s

mt , w0it)].

The result shows that the first moment becomes a sufficient statistic for the evolution of the next moment.In particular, the next moment becomes a linear function of the current moment. Heuristically, this suggestsdefining the following perceived transition kernel:36

Pµm [ζm1 (smt )θmt +Nmζ0(smt )|smt ] = 1, (13)

which will become a good approximation as m grows large. In fact, with this perceived transition kerneland under suitable continuity conditions, one can show that the value of the full information deviation(appropriately normalized) converges to zero as the market size m approaches infinity.37

We finish this section showing that it seems implausible to obtain a similar aggregation result to the oneobtained for the constant returns model introduced above for a broader class of models. Suppose that thesingle-period profit function depends on the α un-normalized moment of the fringe state (like in Example6.1). We have the following result.

Proposition C.2. AssumeXf is a closed interval in<,N−D ≥ 3 , there is no entry and exit in the industry,the set of dominant firms is fixed, and the set of moments contains the α un-normalized moment only, thatis, θ(f) =

∑x∈Xf x

αf(x). Suppose that under MME strategy µ, the moment is a sufficient statistic for theevolution of the industry, that is:

Pµ,λ[θ′|s] = Pµ,λ[θ′|s],

for all θ′ ∈ Sθ, s= (f, d, z) ∈ S , and s= (θ, d, z) ∈ S, with θ(f) = θ. Then, fringe firms’ transitions inMME must be linear in their own state, in the sense that E µ[xαi,t+1|xit = x, st] = xαζ1(st|µ) + ζ0(st), forgiven functions ζ1(·|µ) and ζ0(·).

36Note that a derivation similar to that in Proposition C.1 will show that any k-th moment of fringe firms’ states for an integer kwould depend on moments k, k − 1, . . . , 1 only, as m grows large. Therefore, if higher integer moments are payoff relevant theycould be accounted for as well.

37This result requires appropriate continuity conditions on the model primitives, and also that the equilibrium investment strategyis continuous in the moment-based fringe state.

44

The result shows that when firms keep track of a single moment, moments are sufficient statistics only iffringe firms’ equilibrium transitions are, in an appropriate sense, linear in their own state. To our knowledge,such equilibrium transitions arise only in the constant returns to scale model mentioned above or in closevariations. There, equilibrium transitions are linear, because the primitive transitions are linear and, as weshowed, the equilibrium investment strategy does not depend on the fringe firms’ individual state.38

D Convergence to MPE

In this section we show that if the value of the full information deviation becomes small for all initial states(e.g., by including more moments in MME), then MME strategies approach MPE strategies. Let Γ ⊆M×Λ

be the set of MPE strategies. For all (µ, λ) ∈ M× Λ, let us define D(Γ, (µ, λ)) = inf(µ′,λ′)∈Γ ‖(µ′, λ′) −(µ, λ)‖∞.39 That is,D(Γ, (µ, λ)) measures the distance between (µ, λ) to the closest strategy (µ′, λ′) withinthe set of MPE (which may not be a singleton).

Theorem D.1. Consider a sequence of MME strategies (µn, λn)|n ∈ N, such that (i) the value ofthe full information deviation converges to zero for all industry states, that is limn→∞∆µn,λn(x, s) =

0, ∀(x, s) ∈ X ×S; and (ii) the cut-off entry condition is satisfied asymptotically, that is limn→∞ |λn(s)−β E µn,λn [V (xe, st+1|µn, λn)|st = s] | = 0, ∀s ∈ S.40 Then, limn→∞D(Γ, (µn, λn)) = 0.

The proof closely follows the proof of a similar result in Farias et al. (2012); we provide it in AppendixE for completeness. Note that the convergence is to a particular MPE. We note that the assumption that thereis no overlap of individual states between dominant and fringe firms is important to obtain the result; if not,MME would not admit symmetric Markov strategies. In addition, the assumption that the value of the fullinformation deviation converges to zero for all industry states is key to prove the result. However, in practice,when using a transition kernel like the one in Example 4.1 we may only expect for this to be the case instates in the recurrent class, because transitions outside this class are arbitrary. This issue would be avoidedif the entire state space is recurrent under strategies that could potentially be best responses.41 Alternatively,if we only assume that the value of the full information deviation converges to zero in states in the recurrentclass, we can show that we approximate a weaker equilibrium concept that does not impose optimalityrequirements for states that are not reachable in equilibrium, similar to those proposed in Fershtman andPakes (2012).

38These results are closely related to the literature in macroeconomics that deals with the aggregation of macroeconomic quanti-ties from the decisions of a single ‘representative agent’ (Blundell and Stoker, 2005). With heterogeneous agents, similarly to ourmodel, this type of result is only obtained under strong assumptions about consumers’ preferences that are also akin to linearity(see, for example, Altug and Labadie (1994)).

39‖ · ‖∞ denotes the supremum norm40When the value of the full information deviation becomes small, it is also reasonable to expect that the error in the cut-off entry

rule also becomes small.41For example, if the model has depreciation and appreciation shocks in all states.

45

E Proofs

Proof of Theorem B.1. The proof is similar to Doraszelski and Satterthwaite (2010) with appropriate mod-ifications to consider our perceived transition kernel. For brevity, we omit details that replicate their argu-ment. The proof is divided in 4 steps: (1) showing compactness and convexity of the strategy space; (2)showing continuity of the perceived transition kernel; (3) showing continuity of the best response function;and (4) concluding with the application of Brouwer’s fixed point theorem.

Step 1. First, recall that I is convex and compact. Further, we can restrict attention to exit and entry cut-off strategies such that sup(x,s)∈X×S max(|ρ(x, s)|, |λ(s)|) ≤ supx,s π(x,s)

1−β +supι∈I,x c(ι,x)

1−β + φ, where φ isthe expected discounted profit of entering the market, investing zero and earning zero profits each period,and then exiting at an optimal stopping time (which is finite because φit has finite moments). These factstogether with |X ×S| <∞ imply that the space of moment-based strategies (M, Λ) is convex and compact.

Step 2. Now, consider the transition kernel defined in equation (5). We make the following observations:

1. It is simple to show that our assumptions on model primitives guarantee that the actual kernel Pµ′,µ,λ[·|x, s]is continuous in (µ′, µ, λ), for all (x, s).

2. Let s = (θ, d, z) ∈ S and let S(s) = s ∈ S∣∣s = (f, d, z), θ(f) = θ. In words, S(s) is the set of

underlying industry states that are consistent with the moment-based state s. By standard results inMarkov processes the right-hand side that defines Pµ,λ[θ′|s] in equation (2) converges almost surelyand is equal to (Resnick, 1998): ∑

s∈S(s) qµ,λ(s)Pµ,λ[θ′|s]∑s∈S(s) qµ,λ(s)

, (14)

where qµ,λ(s) is the unique stationary distribution of the industry state process st : t ≥ 0 understrategies (µ, λ), which is well defined by Assumption B.1 and because the state space is finite. Withsome abuse of notation, we denote Pµ,λ[θ′|s] as the transition probability from industry state s to anyindustry state with moment θ′ according to the actual underlying kernel Pµ,λ.

3. Similarly, the right-hand side that defines Pµ,λ[(f → d)|s] in equation (4) converges almost surelyexists and is equal to: ∑

s∈S(s) qµ,λ(s)Pµ,λ[(f → d)|s]∑s∈S(s) qµ,λ(s)

.

4. Our assumptions on model primitives guarantee that Pµ,λ[θ′|s] and Pµ,λ[(f → d)|s] are continuous in(µ, λ), for all (s, θ′). Further, our assumptions on model primitives and continuity results of stationarydistributions of ergodic Markov processes (Cho and Meyer, 2001), imply that qµ,λ(s) is continuousin (µ, λ), for all s. Therefore, under Assumption B.1, the perceived transition kernel Pµ′,µ,λ[·|x, s]defined by (2), (4), and (5), is continuos in (µ′, µ, λ), for all states (x, s) ∈ X × S.

46

Step 3. We define the best response operator BR(µ, λ) = (µ∗(µ, λ), λ∗(µ, λ)), where:

V (x, s|µ∗, µ, λ) = supµ′∈M

V (x, s|µ′, µ, λ) ∀(x, s) ∈ X × S

λ∗(s) = β E µ,λ

[V (xe, st+1|µ, λ)

∣∣∣st = s]

∀s ∈ S.

Note that a fixed point of BR constitutes a MME. Step 2 together with our assumptions on model primitivesguarantee that V (x, s|µ′, µ, λ) is continuous in (µ′, µ, λ) for all (x, s). Moreover, the assumption that forall competitors’ decisions and all continuation values, a firm’s one time-step ahead optimization problemto determine its optimal investment has a unique solution, yields in addition that BR(·) is a continuousfunction on M × Λ. For a proof of this fact, we can use a similar argument to the proof of Proposition 2in Doraszelski and Satterthwaite (2010) which in turn employs the maximum theorem and Lemmas 3.1 and3.2 of Whitt (1980).

Step 4. The result follows because by Steps 1 and 3 and the application of Brouwer’s fixed point theorem,BR admits a fixed point.

Proof of Lemma C.1. Under the assumptions of model (N) of Chapter 9 in Bertsekas and Shreve (1978) wehave from Proposition 9.8, that the optimal value function satisfies:

(TV )(x, s|µ) := maxι∈I

xπ1(s0) + π0(s0)− cxι+ β E µ

[V (x1, s1)

∣∣∣ι, s0 = s, x0 = x]

= V (x, s|µ)

Moreover, we have that TnV→V if V = 0 by Proposition 9.14. We first show that

supµ′∈M

V (x, s|µ′, µ) = xV 1(s) + V 0(s)

for appropriate functions V 1(·) and V 0(·) by demonstrating that the posited form of the perceived valuefunction is stable under an application of the Bellman operator. We have:

(T V )(x, s|µ) = maxι∈I

xπ1(s) + π0(s)− cxι+ β E µ

[(xζ1(ι, s, w1) + ζ0(s, w0))V1(st+1)

+ V0(st+1)∣∣∣ι, st = s

]

= xmaxι∈I

− cι+ β E µ

[ζ1(ι, s, w1)V1(st+1)

∣∣∣ι, st = s]

+ xπ1(s) + V0(s) (15)

= xV1(s) + V0(s),

where we define V0(s) = π0(s) + β E µ

[ζ0(s, w0)V1(st+1) + V0(st+1)

∣∣∣st = s]. Now, let us denote by V n

47

the iterates obtained by applying the Bellman operator T . Then, we have concluded that

xV n1 (s) + V n

0 (s)→V (x, s), ∀x, s,

as n→∞, where V is the optimal value function. But since the above holds for at least two distinct valuesof x for any given s, this suffices to conclude that V n

1 (s)→V∞1 (s) and V n0 (s)→V∞0 (s).

Now, under the additional Assumption C of Chapter 4 in Bertsekas and Shreve (1978), and furtherassuming that the supremum implicit in the dynamic programming operator applied to V is attained for every(x, s) , the second claim follows immediately from equation (15) and Proposition 4.3 of the reference.

Proof of Proposition C.1. Note that

θmt+1 =

Nm∑i=1

xi,t+1 =

Nm∑i=1

[xitζ1(µm(smt ), smt , w1it) + ζ0(smt , w0it)],

where (w0it, w1it) are i.i.d. We need to show that conditional on smt ,

(1/Nm)∣∣θmt+1 − E µm [θmt+1|smt ]

∣∣→ 0, a.s.,

where E µm [θt+1|smt ] = ζm1 (smt )θmt +Nmζ0(smt ). The result follows by Assumption C.1 and a law of largenumbers (see, for example, Corollary 7.4.1 in Resnick (1998)).

Proof of Proposition C.2. If α 6= 1 redefine the state of fringe firms to be y = xα. Without loss of generalityassume Xf = [0, x]. Let st, s′t be two consistent underlying industry states, that is, dt = d′t, zt = z′t, andθ(ft) = θ(f ′t) = θt. For moments to be sufficient statistics it must be that for any such consistent underlyingstates the expected next moment are the same E µ[θ(ft+1)|st] = E µ[θ(ft+1)|s′t].

Let us fix a moment-based industry state s and let us define the function g(x; s) = E µ[xi,t+1|xit =

x, st = s]. A function h is midpoint convex if for all x1, x2 ∈ (x, x) ⊂ < the following holds h((x1 +

x2)/2) ≤ (h(x1) + h(x2))/2. Midpoint concavity is defined by reversing the inequality. A function that ismidpoint convex (concave) and bounded on an interval is convex (concave) (see Donoghue (1969)). We willshow that g(·; s) is both convex and concave and so linear by showing that midpoint convexity and midpointconcavity hold. Note that we need to show this only for x ≤ θ as a single fringe firm cannot be larger thanthe moment.

For any x0 in the interior of Xf with 2x0 ≤ θt we can find (we assumed N − D ≥ 3) a consistentfringe state f with f(x0) ≥ 2 and δ > 0 such that x0 ± δ ∈ Xf . Construct f ′ with f ′(x0) = f(x0) − 2,f ′(x0 − δ) = f(x0 − δ) + 1, f ′(x0 + δ) = f(x0 + δ) + 1 and f ′(x) = f(x) for all x 6∈ x0, x0 ± δ. Lets = (f, d, z) and s′ = (f ′, d, z). Clearly s = s′, because θ(f) = θ(f ′). By assumption E µ[θ(ft+1)|st] −E µ[θ(ft+1)|s′t] = 2g(x0; s)−g(x0− δ; s)−g(x0 + δ; s) = 0 or g(x0; s) = (g(x0− δ; s) +g(x0 + δ; s))/2.That is g(x; s) is midpoint convex and midpoint concave in the variable x, and so linear.

For x0 ≥ θ/2 we use the following construction. From the result above we have that g(θt/2 − δ; s) islinear for 0 ≤ δ ≤ θt/2. By assumption, g(θt/2 + δ; st) + g(θt/2 − δ; st) = 2g(θt/2; st), or g(θt/2 +

δ; st) = 2g(θt/2; st) − g(θt/2 − δ; st). Substituting for the right hand side we get g(θt/2 + δ; st) =

48

(θt/2+δ)ζ1(st|µ)+ ζ0(st|µ) for appropriately defined ζ0, ζ1 and for all 0 ≤ δ ≤ min(x−θt/2, θt/2). Thiscompletes the proof.

Proof of Theorem D.1. Assume the claim to be false. It must be that there exists an ε > 0 such that for all n,there exists an n′ > n for which d(Γ, (µn′ , λn′)) > ε. We may thus construct a subsequence (µn, λn) forwhich infn d(Γ, (µn, λn)) > ε.Now, because the space of strategies is compact, we have that (µn, λn) hasa convergent subsequence; call this subsequence (µ′n, λ′n) and its limit (µ∗, λ∗). We have thus establishedthe existence of a sequence of strategies and entry rate functions (µ′n, λ′n), satisfying:

V (x, s|BR(µ′n, λ′n), µ′n, λ

′n)− V (x, s|µ′n, λ′n)→ 0, ∀(x, s) ∈ X × S (16)

λ′n(s)− βEµ′n,λ′n[V (xe, st+1|µ′n, λ′n)|st = s

]→ 0, ∀s ∈ S (17)

(µ′n, λ′n)→ (µ∗, λ∗). (18)

infnd(Γ, (µ′n, λ

′n)) > ε, (19)

where BR(µ, λ) denotes the best response strategy when competitors play strategy µ and enter according toλ.

Now, it is simple to show that our assumptions on model primitives guarantee that V (x, s|µ′, µ, λ) iscontinuous in (µ′, µ, λ) for all (x, s). Moreover, the assumption that for all competitors’ decisions and allterminal values, a firm’s one time-step ahead optimization problem to determine its optimal investment hasa unique solution, yields in addition that BR(·) is a continuous function on M× Λ. For a proof of thisfact, see the proof of Proposition 2 in Doraszelski and Satterthwaite (2010) which in turn employs Lemmas3.1 and 3.2 of Whitt (1980). Thus, we have from (16), (17), and (18) that V (x, s|BR(µ∗, λ∗), µ∗, λ∗) −V (x, s|µ∗, λ∗) = 0, ∀(x, s) ∈ X × S , and λ∗(s) − βEµ∗,λ∗ [V (xe, st+1|µ∗, λ∗)|st = s] = 0, ∀s ∈ S.Hence, (µ∗, λ∗) ∈ Γ. But by (18), (19), and the triangle inequality d(Γ, (µ∗, λ∗)) > ε, a contradiction. Theresult follows.

Lemma E.1. The operator TR satisfies the following properties:

1. TR is a contraction mapping modulo β. That is, for V , V ′ ∈ V , ‖TRV − TRV ′‖∞ ≤ β‖V − V ′‖∞.

2. The equation TRV = V has a unique solution V ∗.

3. V ∗ = limk→∞ TkRV for all V ∈ V .

Proof. Our result is based on a special case of Iyengar (2005) (see Theorem 3.2). However, to allow forgreater generality in the state space and action space we use a different proof based on classic dynamicprogramming results. Define

(TRV )(x, s) = supf∈Sf (s)ι∈Iρ≥0

π(x, s) + E

[φ1φ ≥ ρ

+ 1φ < ρ[− c(x, ι) + β E µ,λ[V (xi,t+1, st+1)

∣∣xit = x, st = (f, d, z), ι]]]

. (20)

49

The statement of the lemma holds for TR from the results for model (D) in Bertsekas and Shreve (1978),using our assumption of bounded profits. The statement follows for operator TR since TRV = TRV for allbounded V .

Proof of Theorem 8.1. Take vectors V ∈ V and V , where V : X × S → <, such that V (x, s) = V (x, s),for all moment-based industry state s that is consistent with industry state s. Let T ∗ be the Bellman operatorassociated with the full Markov best response that keeps track of the entire industry state. It is simple toobserve that T ∗V (x, s) ≤ TRV (x, s), for all x, and all s and s consistent. Using this, together with themonotonicity of the operator TR we inductively conclude that (T ∗)kV (x, s) ≤ T kRV (x, s), for all k ≥1. Taking k to infinity we get T kRV → V ∗ from Lemma E.1, and (T ∗)kV → V ∗ by standard dynamicprogramming arguments. Therefore, V ∗(x, s) ≤ V ∗(x, s) for s and s consistent.

F Real-Time Algorithm for MME

Given strategies (µ, λ) and their associated value functions, it is useful to define

W (x, s|µ, λ) = E µ,λ

[V (x, st+1|µ, λ)

∣∣st = s], (21)

where the expectation is taken with respect to the perceived transition kernel. The functionW is the expectedcontinuation value starting from industry state s and landing in state x in the next period. Note that we onlyintegrate over the possible transitions of s, excluding the firm’s own transition to x. It is worth emphasizingthat if x ∈ Xd in (21) then st+1 depends on x, whereas if x ∈ Xf the next industry state, st+1, is independentof x. Namely, dominant firm i that transitions to x will integrate over (θt+1, d−i,t+1, zt+1) with dt+1 =

(xi,t+1, d−i,t+1), that is, over its competing dominant firms’ states. Now, we can write the Bellman equationassociated with C1 in the equilibrium definition as follows:

V (x, s;W ) = supι∈Iρ≥0

π(x, s) + E

[φ1φ ≥ ρ+ 1φ < ρ

[− c(x, ι) + β E[W (xi,t+1, s)

∣∣xit = x, ι]]]

,

where the first expectation is taken with respect to the sell-off random value φ, and the second with respectto the firm’s transition under investment level ι. Note that when evaluated at the optimal value function, thefunction W is sufficient to compute a best response strategy. We have omitted the dependence of V and Won (µ, λ) to simplify notation. Based on this formulation we introduce our real-time dynamic programmingalgorithm to compute MME; see Algorithm 2.

The steps of the algorithm are as follows. We begin with W functions with which firms’ optimal de-cisions can be computed (investment, exit, and entry). Once these are determined, we can simulate thenext industry state. The continuation value in the simulated state is used to update the W functions. Theupdate step depends on the number of times the state was visited e(s) and on the number of rounds (we takeσ(n) = min(n, n) for some integer n > 0 to allow for quick updating in the early rounds). At the end ofthe simulation/optimization phase we check whether the W functions have converged, and if so we check

50

Algorithm 2 Equilibrium solver with real-time dynamic programming

1: Initiate W (x, s) := 0, for all (x, s) ∈ X × S;2: e(s) = 0 for all s ∈ S;3: Initiate industry state (f0, d0, z0) and s0 := (θ(f0), d0, z0));4: ∆w := εw + 1; n := 15: while ∆w > εw do6: W ′(x, s) := W (x, s) for all (x, s) ∈ X × S;7: t := 1;8: while t ≤ T do9: for all x with ft(x) > 0 or x ∈ dt, do

10: Compute optimal strategies using V (x, st;W ) and store them;11: end for;12: Compute optimal entry cutoff from V (xe, st;W ) and store it;13: Simulate (ft+1, dt+1, zt+1) and st+1 from these strategies;14: Let γ := 1

σ(n)+e(st);

15: for all x′ ∈ Xf do16: Compute V (x′, st+1;W );17: Update W (x′, st) := γV (x′, st+1;W ) + (1− γ)W (x′, st);18: end for;19: for all Dominant firm i and x′ ∈ Xd that is accessible in one step from xit do20: Define s′t+1 to be the industry state st+1 when firm i transitions to state x′;21: Compute V (x′, s′t+1;W );22: Update W (x′, st) := γV (x′, s′t+1;W ) + (1− γ)W (x′, st);23: end for;24: e(st) := e(st) + 1, t := t+ 1;25: end while;26: ∆w := ‖W ′ −W‖∞;27: e(s) := 0 for all s ∈ S;28: (f0, d0, z0) := (fT+1, dT+1, zT+1), and s0 := (θ(f0), d0, z0);29: n := n+ 1;30: end while;31: Compute µ(x, s) and λ(s) from V (x, s;W ) for all (x, s) ∈ X × S;32: Initialize with industry state s0 = (f0, d0, z0) with corresponding s0;33: Simulate a T period sample path (ft, dt, zt)Tt=1 with corresponding stTt=1 for large T ;34: Calculate the observed frequencies of industry states h(s) := 1

T

∑Tt=1 1st = s, for all s ∈ S;

35: Use the same sample path to compute the observed transition kernel Pµ′,µ,λ[·|x, s], for all x, s;36: Solve µ′ := argmax

µ∈MV (x, s|µ, µ, λ), for all (x, s) ∈ X × S;

37: Let λ′(s) := Eµ,λ[V (xe, st+1|µ′, µ, λ)|st = s], for all s ∈ S ;38: ∆ := max(‖µ− µ′‖h, ‖λ− λ′‖h);39: if ∆ < ε then40: STOP;41: else42: Go back to line 3;43: end if;

51

for convergence of strategies using one step of Algorithm 1 (lines 31–40). If strategies have not convergedwe continue iterating on the simulation/optimization phase.

G Beer Industry Numerical Experiments

Denote by xit the goodwill of firm i at time t. The evolution of goodwill is similar to Pakes and McGuire(1994), but with a multiplicative growth model following Roberts and Samuelson (1988):

xit+1 =

min(xn, xit(1 + ρ)) w.p. δψ(x)ιit

1+ψ(x)ιit

xit w.p. 1−δ′+(1−δ)ψ(x)ιit1+ψ(x)ιit

max(x1, xit/(1 + ρ)) w.p. δ′

1+ψ(x)ιit.

This is equivalent to a depreciation factor 1/(1 + ρ) as is common in the literature on goodwill. With thisin mind we define a grid of states x1, . . . , xn for the possible values of goodwill firms can take, wherexk = x1(1 + ρ)k−1 for some x1 > 0. To maintain the relationship between goodwill and advertising costs,we choose the parameter ψ(x) such that E[xit+1|xit = x, ιit = x] = x, that is, a firm with goodwill x has toinvest x dollars in advertising to maintain goodwill level x on average. It follows that ψ(x) = δ′

δ1−(1+δ)−1

δx .Under this condition, the average goodwill (state) of a firm that invests xit in every period is approximatelyxit.

The moment space is discretized linearly and we use a bicubic spline to interpolate between grid pointswhen computing firms’ optimal strategies.

We use the profit function of Example 6.1. We take β = .925 and (δ′, δ) = (1, .55) as the transitionparameters. After some experimentation we choose the market sizem = 30. The other parameters are listedin the next table with their relevant sources.

52

Description Value Source

Number of firms (N ) 200 This number is chosen to be greater than the maximal num-ber of active firms in this period.

Maximal number of dominantfirms (D)

3 This is the actual number of dominant firms in the industry.

Depreciation of goodwill (ρ) .25 Roberts and Samuelson (1988) estimate this by .2 for thecigarette market.

Production cost per barrel (c) $120 Rojas (2008) estimated the markup to be about a third ofthe price, and the average price is $165 per barrel in theperiod studied.

Average entry cost (exponentialdist.)

35× 106 Based on costs of new plants.

Sell-off value (exponential dist.) 7× 106 Based on sales’ price of used plants.

Fixed cost per period fringe(double for dominant)

106 We introduce this fixed cost in the single-period profitfunction.

Profit function parameters (Yand α2)

200 and 1, resp. Chosen to match the price elasticity (-.5) for the averageprice, see (Tremblay and Tremblay, 2005, p. 23).

H Computation of Robust Bound

The remainder of the appendix provides details on the computation of the robust bound under the assumptionthat there are large number of fringe firms and therefore the one-step transition of the fringe state is assumedto be deterministic. When transitions are deterministic, finding the optimal consistent f in (12) is equivalentto choosing the next moment from an accessibility set of moments that can be reached from the currentindustry state. As we will explain in detail now, this considerably simplifies the computation of the innermaximization in (12), since the moment accessibility sets are low dimensional. Moreover, characterizatingthese accessibility sets can be done efficiently.

We assume throughtout that the set of individual fringe firm states is discrete and univariate, Xf =

x1, . . . xn, where xn ∈ <, for all n = 0, 1, . . . , n <∞.42 In addition, for simplicity, we assume that thereare no transitions between the dominant and fringe tiers.

Finding the set of accessible moments amounts to solving an integer feasibility problem. To simplify,let us assume that θ consist of only one moment of the form θ =

∑x∈Xf f(x)(x)α for α ≥ 0, where

Xf ⊆ Xf . This representation is general enough to include moments and other statistics. The extension tomore moments is direct.

Assuming that the next fringe state is deterministic and equals to its expected value, we say that momentθ′ is accessible from moment θ in industry state s if there exists a fringe state f ∈ Sf (s) that solves the

42The extension to multivariate xn is straighforward.

53

following system of linear equations,∑x∈Xf

f(x) (x)α = θ

∑x∈Xf

f(x) E µ[(xi,t+1)α1xi,t+1 ∈ Xf|(xit, st) = (x, s)] + 1xe ∈ Xf(xe)αP(κit ≤ λ(s))N e(f) = θ′

(22)∑x∈Xf

f(x) ≤ N, f ∈ Nn,

where the first equation states that the current moment is consistent with the fringe state, and the second thatthe expected next moment is θ′. We denote by N e(f) as the number of potential entrants at fringe state f .We say that moment θ′ is accessible from s if this system of linear equations has a solution. This motivatesthe definition of the accessibility set A(s), where θ′ ∈ A(s) if and only if it is accessible from s, that is, ifthere is a fringe state consistent with s such that the expected next moment is θ′. Note that the accessibilitysets depend on the investment and entry MME strategies that control the expected next moment in (22).Due to the integrability constraint f(x) ∈ Nn, the computation of A(s) is demanding. However, we canrelax this integrability constraint by replacing it with f ≥ 0. With that, the accessibility problem amounts tosolving a feasibility problem of a system of linear equations that can be solved easily. We denote the relaxedaccessibility set by A(s); this set contains A(s).

Define the operator

(T V )(x, s) = supι∈Iρ≥0

supθ′∈A(s)

π(x, s) + E

[φ1φ ≥ ρ

+ 1φ < ρ[− c(x, ι) + β E µ,λ[V (xi,t+1, (θ

′, dt+1, zt+1))∣∣xit = x, st = s, ι]

]],

where V ∈ V . The next proposition states that we can search over accessibility sets instead of the muchlarger consistency sets.

Proposition H.1. Assume that there are no transitions between tiers and that the fringe state follows deter-ministic transitions given by the expected next state (equation (22)). Then

(TRV ) ≤ (T V ),

for every V ∈ V . Moreover, the operator T satisfies the same properties than the operator TR given inLemma E.1.

Proof. Without tier transitions the evolution of the moments is independent of the evolution of dominantfirms. If fringe firms’ transitions are also deterministic each f ∈ Sf (s) maps to an expected next momentand so we can optimize over the set of these moments instead. The inequality follows because we considerthe relaxed accessibility sets, so we optimize over a larger set in T .

Based on this, we propose the following computationally tractable algorithm to find the robust error

54

bound: (i) construct the relaxed accessibility sets by solving the relaxed feasibility problem for all s ∈ Sand θ′ ∈ Sθ and store them; and (ii) iterate the operator T over the relaxed accessibility sets until a fixed pointis found. Since the relaxed accessibility sets contain the accessibility sets, this provides an upper bound toV ∗, assuming deterministic fringe transitions (by Theorem 8.1 together with the previous proposition). Forproblems for which MME is solvable this procedure is generally computationally manageable. In addition,this procedure can be easily extended to accommodate tier transitions between dominant and fringe firms.In this case, however, the robust bound typically becomes looser.

55