Stationary equilibria in stochastic games: structure, selection, and computation

This paper introduces an algorithm to compute stationary equilibria in stochastic games that is guaranteed to converge for almost all such games. Since in general the number of stationary equilibria is overwhelming, we pay attention to the issue of equilibrium selection. We do this by extending the linear tracing procedure to the class of stochastic games, called the stochastic tracing procedure. As a by-product of our results, we extend a recent result on the generic finiteness of stationary equilibria in stochastic games to oddness of equilibria.     

Ergodic chaos, learning and sunspot equilibrium

Summary. In this paper we construct sunspot equilibria that arise from chaotic deterministic dynamics. These equilibria are stationary and have absolutely continuous stationary measures. We prove that they can be learned by a simple rule based on the histograms of past state variables. This work gives a theoretical justification for complex deterministic models that might compete with stochastic models to explain real data. Also we prove the stochastic stability of the indeterminate equilibrium.     

Indeterminacy of equilibrium in stochastic OLG models

Summary. This paper studies the equilibria of a stochastic OLG exchange economies consisting of identical agents living for two periods, and having the opportunity to trade a single infinitely-lived asset in constant supply. The agents have uncertain endowments and the stochastic process determining the endowments is Markovian. For such economies, the literature has focused on studying strongly stationary equilibria in which quantities and prices are functions of the exogenous states of nature which describe the uncertainty: such equilibria are generalizations of deterministic steady states, and this paper investigates if they have the same special status as asymptotic limits of other equilibrium paths. The difficulty in extending the analysis of equilibria beyond the class of strongly stationary equilibria comes from the presence of indeterminacy: we propose a procedure for overcoming this difficulty which can be decomposed into two steps. First backward induction arguments are used to restrict the domain of possible prices; then if some indeterminacy is left, expectation functions are introduced to make the forward equilibrium equations determinate. The properties of the resulting trajectories, in particular their asymptotic properties, can then be studied. For the class of models that we study this procedure provides a justification for focusing on strongly stationary equilibria. For the model with positive dividends (equity or land) the justification is complete, since we show that the strongly stationary equilibrium is the unique equilibrium. For the model with zero dividends (money) there is a continuum of self-fulfilling expectation functions resulting in a continuum of equilibrium paths starting from any admissible initial condition: under conditions given in the paper, these equilibrium paths converge almost surely to one of the strongly stationary equilibria-either autarchy or the stochastic analogue of the Golden Rule. Received: November 19, 2001; revised version: March 22, 2002 RID="*" ID="*" We are grateful for the stimulating environment and research support provided by the Cowles Foundation at Yale University during the Fall 2000 when this paper was first conceived. We are also grateful to the participants of the SITE Workshop at Stanford University and the Incomplete Markets Workshop at SUNY Stony Brook during the summer 2001 for helpful discussions. Correspondence to: M. Magill     

Growth in economies with non convexities: sunspots and lottery equilibria

Summary. We investigate the relation between lotteries and sunspot allocations in a dynamic economy where the utility functions are not concave. In an intertemporal competitive economy, the household consumption set is identified with the set of lotteries, while in the intertemporal sunspot economy it is the set of measurable allocations in the given probability space of sunspots. Sunspot intertemporal equilibria whenever they exist are efficient, independently of the sunspot space specification. If feasibility is, at each point in time, a restriction over the average value of the lotteries, competitive equilibrium prices are linear in basic commodities and intertemporal sunspot and competitive equilibria are equivalent. Two models have this feature: Large economies and economies with semi-linear technologies. We provide examples showing that in general, intertemporal competitive equilibrium prices are non-linear in basic commodities and, hence, intertemporal sunspot equilibria do not exist. The competitive static equilibrium allocations are stationary, intertemporal equilibrium allocations, but the static sunspot equilibria need not to be stationary, intertemporal sunspot equilibria. We construct examples of non-convex economies with indeterminate and Pareto ranked static sunspot equilibrium allocations associated to distinct specifications of the sunspot probability space.Received: 25 August 2003, Revised: 16 March 2004, JEL Classification Numbers: D84, D90.Correspondence to: Paolo SiconolfiWe thank Herakles Polemarchakis for helpful conversations on the topic. The research of Aldo Rustichini was supported by the NSF grant NSF/SES-0136556.     

Optimal stationary consumption with stochastic production and resources

Uncertainty about technology and resources is represented in terms of uncertainty about an (exogenous) environment whose successive states form a stationary stochastic process, with probabilities that are unaffected by economic decisions. The successive states of the economy depend both on the environment and on the decisions taken with regard to production and consumption. It is shown that, under conditions that are natural extensions of “neoclassical” conditions in the case of certainty, (1) Capital saturation is possible, i.e., an optimal stationary stochastic program exists, and (2) An optimal program can be sustained by a price system that takes the form of a stationary stochastic process of price vectors. In other words, an optimal stationary program can be sustained by a stochastic “equilibrium,” in which at each date the optimal production decisions maximize expected intertemporal profit, and the optimal aggregate consumption vector has minimum cost among all aggregate consumption vectors yielding no less (social) utility.     

Rational expectations in the overlapping generations model

This paper reports results on the character of the rational expectations equilibria of a stochastic overlapping generations model with heterogenous markets. The model considered is a stationary overlapping generations model in which the endowments of young agents are subject to i.i.d. random shocks. The main result shown is that if there are l > 1 commodities traded in every period, then for most preferences, the rational expectations equilibrium stochastic process of prices and allocations necessarily exhibits serial correlation. This is in marked contrast to the one commodity model in which there always exists an equilibrium which is measure isomorphic to the endowment process.     

Consistent planning under quasi-geometric discounting

We study properties of stationary Markov-perfect equilibria in a general model of intertemporal choice under quasi-geometric discounting. The dynamics generated by stationary Markov-perfect equilibria can be very complicated, even if the model satisfies strict convexity and smoothness properties and the decision maker is arbitrarily patient. If there exist multiple stationary Markov-perfect equilibria, then it is in general possible to construct infinitely many non-degenerate stationary sunspot equilibria as well.     

Non-parametric counterfactual analysis in dynamic general equilibrium

In this paper, we examine non-parametric restrictions on counterfactual analysis in a dynamic stochastic general equilibrium model. Under the assumption of time-separable expected utility and complete markets all equilibria in this model are stationary. The Arrow-Debreu prices uniquely reveal the probabilities and discount factor. The equilibrium correspondence, defined as the map from endowments to stationary (probability-free) state prices, is identical to the equilibrium correspondence in a standard Arrow-Debreu exchange economy with additively separable utility. We examine possible restriction on this correspondence and give necessary as well as sufficient conditions on profiles of individual endowments that ensure that associated equilibrium prices cannot be arbitrary. Although restrictions on possible price changes often exist, we show that results from a representative-agent economy usually do not carry over to a setting with heterogeneous agents.     

Real Indeterminacy of Stationary Monetary Equilibria in Centralized Economies

We show that real indeterminacy of stationary equilibria, by which the set of stationary equilibria is a continuum and the real allocation varies among equilibria, may arise in some general equilibrium models with fiat money. The conditions under which such equilibria arise are: (i) each household optimally saves a constant amount of money; and (ii) at least two households face different budget constraints. We present various models, including a decentralized money search model and a centralized model with a monopoly firm, to explain how these conditions lead to real indeterminacy. Finally, we present a policy that uniquely implements any desirable outcome.     

Expectations equilibrium and informational efficiency for stochastic environments

This paper studies the existence of expectations equilibria for message processes on stochastic exchange environments. A message process which permits the general existence of expectations equilibria is termed admissible. The main results are: (1) Every admissible process satisfies a weakened version of the Hurwicz “privacy” condition. (2) The competitive process is admissible, and has a message space of locally minimal size among admissible nonwasteful processes. (3) The only admissible condensations of the competitive process are constant functions.     

Quasi-fundamental exchange rate variation

Summary. This paper uses a general equilibrium model to study the determination of the exchange rate in an economy with fundamental uncertainty. The model has steady state equilibria in which the exchange rate is constant. These equilibria may coexist with “quasi-fundamental” equilibria – nonstationary equilibria in which the exchange rate displays stochastic fluctuations that are correlated with the fluctuations in fundamental random variables. The quasi-fundamental equilibria are Pareto dominated by the corresponding constant-exchange-rate steady states. They also converge to these steady states, inevitably or with positive probability. Received: October 2, 1999; revised version: March 26, 2002 RID="*" ID="*" This paper began as a joint project with Alex Mourmouras, who has made many helpful comments and suggestions but is not responsible for any errors or deficiencies. In addition, I thank an anonymous referee for helpful comments.     

Dynamic legislative policy making

We prove existence of stationary Markov perfect equilibria in an infinite-horizon model of legislative policy making in which the policy outcome in one period determines the status quo for the next. We allow for a multidimensional policy space and arbitrary smooth stage utilities, and we assume preferences and the status quo are subject to arbitrarily small shocks. We prove that equilibrium continuation values are differentiable and that proposal strategies are continuous almost everywhere. We establish upper hemicontinuity of the equilibrium correspondence, and we provide weak conditions under which each equilibrium of our model determines an aperiodic transition probability over policies. We establish a convergence theorem giving conditions under which the invariant distributions generated by stationary equilibria must be close to the core in a canonical spatial model. Finally, we extend the analysis to sequential move stochastic games and to a version of the model in which the proposer and voting rule are determined by play of a finite, perfect information game.     

The Structure of Sunspot Equilibria: The Role of Multiplicity

This paper examines the structure of sunspot equilibria in a standard two period exchange economy with real assets. We show that for a generic choice of utility functions and endowments, there exists an open set of real asset structures whose payoffs are independent of sunspots such that the economy with this asset structure has a regular sunspot equilibrium. An important implication of our result is that the multiplicity of non-sunspot equilibria is not necessary for the existence of sunspot equilibria. Our technique is general and can be applied to show the existence of sunspot equilibria in other frameworks.     

Pareto optimal improvements for sunspots: The golden rule as a target for stabilization

Summary The stationary sunspot equilibria of a simple one good OLG economy are considered. These equilibria are known to be suboptimal. We show that, for any such equilibrium allocation, there always exists a Pareto optimal improvement which has the additional property of reaching the Golden Rule in finite time, i.e., the monetary steady state acts as a target. We also show that, in general, periodic allocations cannot be used as targets. The result is interpreted as a welfare theoretical justification for stabilization policy.This paper is based on my dissertation at SUNY Stony Brook. I would like to thank my supervisor, T. J. Muench, for his advice and encouragement. O. Galor's comments on a previous draft, and discussions with M. Kurz, J. Peck, H. M. Polemarchakis, B. Smith, and I. Zilcha, are gratefully acknowledged. Comments from F. Marhuenda, two referees and, especially, the Co-Editor, M. Woodford, did much to improve the exposition. Thanks are due to the Lady Davis Foundation for supporting my stay at the Hebrew University, and the IVIE and DGICYT PB92-0342 at Alicante, where successive drafts were prepared.     

Learning and Equilibrium Selection in a Monetary Overlapping Generations Model with Sticky Prices

We study adaptive learning in a monetary overlapping generations model with sticky prices and monopolistic competition for the case where learning agents observe current endogenous variables. Observability of current variables is essential for informational consistency of the learning setup with the model setup but generates multiple temporary equilibria when prices are flexible and prevents a straightforward construction of the learning dynamics. Sticky prices overcome this problem by avoiding simultaneity between prices and price expectations. Adaptive learning then robustly selects the determinate (monetary) steady state independent from the degree of imperfect competition. The indeterminate (non-monetary) steady state and non-stationary equilibria are never stable. Stability in a deterministic version of the model may differ because perfect foresight equilibria can be the limit of restricted perceptions equilibria of the stochastic economy with vanishing noise and thereby inherit different stability properties. This discontinuity at the zero variance of shocks suggests one should analyse learning in stochastic models.     

Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models

Summary We study a strategic version of the neoclassical growth model under possible production uncertainty. For a general specification of the problem, we establish (i) the existence of stationary Markov equilibria in pure strategies for the discounted game, and (ii) the convergence, under a boundedness condition, of discounted equilibrium strategies to a pure strategy stationary Markovian equilibrium of the undiscounted game as the discount factor tends to unity. The same techniques can be used to prove that such convergence also obtains in all finitestate, finite-action stochastic games satisfying a certain full communicability condition. These results are of special interest since there are well known examples in the literature in which the limit of discounted equilibria fails to be an equilibrium of the undiscounted game.We are grateful to Marcus Berliant, M. Ali Khan, Mukul Majumdar, and an anonymous referee for helpful suggestions, and to Bonnie Huck for technical assistance. The first author acknowledges research support from the Columbia University Council for the Social Sciences.     

On persistent disequilibrium

Consider a standard model of a Walrasian economy where the time derivative of price change is a sign preserving function of excess demand. The only steady states of such a system are the Walrasian equilibria. For if the system is not at a Walrasian equilibrium then there must be excess demands or supplies in some market and thus those prices must be changing. Walras' law implies that all prices cannot change in the same direction, and thus relative prices must change.However, it seems that in real world economies there have been states of persistent disequilibrium. How can this be? How can there be stationary states of an economy that are not Walrasian equilibria?The answer presented in this paper goes something like this: the demands actually presented to the market, i.e., the demands that affect price movements, are not the Walrasian demands. Rather, these demands, the effective demands, are a function of both price and quantity signals. There is no reason why there cannot be equilibria of the effective demand system that are not equilibria of the Walrasian demand system.In this paper, I present a simple abstract model of effective demand systems and given a condition for this system to have non-Walrasian equilibria. I also present a simple example of this phenomena. Finally I discuss the real world implications of this model. Related concepts of effective demand are discussed in [A1H], [2], [3], [L5], and [L6].     

Capital Accumulation in a Stochastic Overlapping Generations Model with Social Security

This paper analyzes the impact of pay-as-you-go financed social security on the stochastic process for the capital stock in a stochastic overlapping generations model. It is shown that the probability distribution of the capital stock in absence of social security dominates that in a pay-as-you-go system in the sense of stochastic dominance. Furthermore, the study demonstrates that the sufficient conditions ensuring the existence and uniqueness of stationary equilibria in a pay-as-you-go system are more restrictive than in the model without social security. Journal of Economic Literature Classification Numbers: C62, H55.     

Markov-perfect Nash equilibria in a class of resource games

Summary. A general model of non-cooperating agents exploiting a renewable resource is considered. Assuming that the resource is sufficiently productive we prove that there exists a continuum of Markov-perfect Nash equilibria (MPNE). Although these equilibria lead to over-exploitation one can approximate the efficient solution by MPNE both in the state space and the payoff space. Furthermore, we derive a necessary and sufficient condition for maximal exploitation of the resource to qualify as a MPNE. This condition is satisfied if there are sufficiently many players, or if the players are sufficiently impatient, or if the capacity of each player is sufficiently high.Received: November 1, 1996This revised version was published online in February 2005 with corrections to the cover date.