Correlated Equilibrium in Stochastic Games

We study the existence of uniform correlated equilibrium payoffs in stochastic games. The correlation devices that we use are either autonomous (they base their choice of signal on previous signals, but not on previous states or actions) or stationary (their choice is independent of any data and is drawn according to the same probability distribution at every stage). We prove that any n-player stochastic game admits an autonomous correlated equilibrium payoff. When the game is positive and recursive, a stationary correlated equilibrium payoff exists. Journal of Economic Literature Classification Numbers: C72, C73.     

Markov Equilibria of Stochastic Games with Complementarities

The existence of Markov equilibria for stochastic games with a continuum of states is a complex issue for which no general result holds as yet. In this article, the problem is solved for a class of stochastic games that satisfy assumptions of complementarity and monotonicity. The proof of existence relies on results from lattice programming. In the Markov equilibria singled out by the Theorem of Existence, the policies and continuation values are increasing and Lipschitz continuous functions of the state variable.Journal of Economic LiteratureClassification Numbers: C62, C73.     

Binary Games with State Dependent Stochastic Choice

This paper studies a stochastic equilibrium selection model for binary coordination games. Players switch strategies stochastically so that the mistake probabilities are fully dependent on the population states. A probabilistic behavior is said to be aspiration (imitation, resp.) oriented if strategy switches are mainly driven by the aspiration (imitation, resp.) effect. In general, a strategy switch by one player generates externalities on others. Strategies in a coordination game can be classified according to the relative magnitude of their externality effects. It is shown that the selection outcome for a linear coordination game is determined in a specific way by the balance of the risk dominance, the aspiration/imitation, and the externality effects. It is also shown that an aspiration (imitation, resp.) oriented behavior tends to select payoff dominant (maxmin, resp.) equilibrium and that risk dominant equilibrium is always selected if and only if the aspiration and the imitation effects exactly cancel each other out, which in turn makes the selection process insensitive to externality effects. Journal of Economic Literature Classification Number: C72.     

Stochastic Replicator Dynamics

This article studies the replicator dynamics in the presence of shocks. I show that under these dynamics, strategies that do not survive the iterated deletion of strictly dominated strategies are eliminated in the long run, even in the presence of nonvanishing perturbations. I also give an example that shows that the stochastic dynamics in this article have equilibrium selection properties that differ from other dynamics in the literature.     

Continuous Stochastic Games of Capital Accumulation with Convex Transitions

We consider a discounted stochastic game of common-property capital accumulation with nonsymmetric players, bounded one-period extraction capacities, and a transition law satisfying a general strong convexity condition. We show that the infinite-horizon problem has a Markov-stationary (subgame-perfect) equilibrium and that every finite-horizon truncation has auniqueMarkovian equilibrium, both in consumption functions which arecontinuous and nondecreasingand have all slopes bounded above by 1. Unlike previous results in strategic dynamic models, these properties are reminiscent of the corresponding optimal growth model.Journal of Economic LiteratureClassification Codes: C73, O41, Q20.     

Stochastic Common Learning

We present a general model in which agents learn by observing a stochastic process. We show that agents not only learn in such a model but also learn commonly. By this we mean that in the long run agents will form common beliefs concerning important facts and in particular they will commonly believe that their views regarding the future are similar. Journal of Economic Literature Classification Numbers: C72, C73, D82.     

A Behavioral Learning Process in Games

This paper studies the cumulative proportional reinforcement (CPR) rule, according to which an agent plays, at each period, an action with a probability proportional to the cumulative utility that the agent has obtained with that action. The asymptotic properties of this learning process are examined for a decision-maker under risk, where it converges almost surely toward the expected utility maximizing action(s). The process is further considered in a two-player game; it converges with positive probability toward any strict pure Nash equilibrium and converges with zero probability toward some mixed equilibria (which are characterized). The CPR rule is compared in its principles with other reinforcement rules and with replicator dynamics. Journal of Economic Literature Classification Number: C72.     

Learning Dynamics with Private and Public Signals

This paper studies the dynamics of learning in a model of technology adoption. Firms learn about an unknown technology by observing both private and public signals. Because of the externality associated with the public signal, the social planner has firms experiment more in the initial period of the model, relative to the market equilibrium. In certain cases, this more rapid generation of information results in the planner experimenting less in later periods of the model. In contrast, typical models with public signals result in the planner inducing more experimentation in all periods of the model relative to the market equilibrium. I would like to thank Matt Mitchell and Tom Holmes for their advice and encouragement. I would also like to thank Thor Koeppl, Cyril Monnet, John Stevens, and Jason Cummins, and two anonymous referees for their helpful comments. This paper is the second chapter of my dissertation. The views expressed herein are my own and not necessarily those of the Bureau of Economic Analysis or the US Department of Commerce.     

Uncertainty and Learning in Stochastic Macro Models

Limits on information have deep economic impact and affect the conduct of economic policy. In the present paper we explore the effect of substantive uncertainty in a macro model, from both an analytical and methodological point of view. Agents are boundedly rational and make their forecasts according to different techniques and try to learn the values of the various parameters. In this context, a Markov regime switching rule, a VAR system, and recursive least square are considered and compared. As a result, we obtain a model which is mostly keynesian in nature that can be compared with the new neoclassical synthesis models. Simulations are carried out and show the possible appearence of endogenous and persistent fluctuations.     

Comparative Learning Dynamics*

We study economic environments in which agents make choices on the basis of relative performance criteria and call the associated class of dynamic adjustment rules comparative dynamics. We distinguish two classes of learning behavior: learning from the population experience (imitative dynamics) and learning only from one's own experience (introspective dynamics). Paradoxically, for a broad class of models, comparing stochastically stable states across dynamics, agent payoffs are lower for imitative than introspective dynamics—mimicking best practice in the population is counterproductive.     

Congestion Pricing and Learning in Traffic Network Games

A stochastic model describing the learning process and adaptive behavior of finitely many users in a congested traffic network with parallel links is used to prove convergence almost surely toward an efficient equilibrium for a related game. To prove this result, we assume that the social planner charges on every route the marginal cost pricing without knowing what is the efficient equilibrium. The result is a dynamic version of Pigou’s solution, where the implementation is made in a decentralized way and the information about players gathered by the social planner is minimal. Our result and setting may be extended to the general case of negative externalities.     

A General Approach to Rational Learning in Games

This paper provides a general framework for analysing rational learning in strategic situations in which the players have private priors and private information. The author analyses the behaviour of Bayesian rational players both in a repeated game and in a recurrent game when they are uncertain about opponents' behaviour and the game they are playing. The aim of the paper is to explain how Bayesian rational agents learn by playing and to characterize the outcome of this learning process. By studying the concept of 'conjectural equilibrium' and analysing the process of convergence of players' behaviour, the roles played by the notions of merging and of consistency are demonstrated.     

Scaling Up Learning Models in Public Good Games

We study three learning rules (reinforcement learning (RL), experience weighted attraction learning (EWA), and individual evolutionary learning (IEL)) and how they perform in three different Groves–Ledyard mechanisms. We are interested in how well these learning rules duplicate human behavior in repeated games with a continuum of strategies. We find that RL does not do well, IEL does significantly better, as does EWA, but only if given a small discretized strategy space. We identify four main features a learning rule should have in order to stack up against humans in a minimal competency test: (1) the use of hypotheticals to create history, (2) the ability to focus only on what is important, (3) the ability to forget history when it is no longer important, and (4) the ability to try new things.     

Stochastic Learning and the Evolution of Conventions

This paper shows that aspiration driven, stochastic learning processes can enable the members of a population to reach more efficient conventions, which are modeled as equilibria of a coordination game. As opposed to adaptation processes based on either myopic optimization or imitating behaviour, which generally select the risk dominant equilibrium, the present paper provides conditions under which the efficient equilibrium is selected, even if it is risk dominated by another equilibrium. Further, the impact of players' aspiration levels on the existence and local stability of a convention is analyzed. In general, higher aspirations induce more efficient outcomes. In addition, in the case of a local interaction structure there exist an inverse relation between the size of the individual neighbourhoods and the aspiration level required in order to sustain efficient outcomes.     

基于双因素随机变动的石油投资期权博弈分析

将期权博弈分析引入到石油勘探开发项目（E＆P）投资决策中,从实际问题出发,建立了基于需求和经营成本均符合随机变动的双头垄断市场下的E＆P项目投资的期权博弈模型,并对模型进行了分析,为石油企业投资决策提供依据。     

Stochastic Cooperative Games: Superadditivity, Convexity, and Certainty Equivalents

This paper extends the notions of superadditivity and convexity to stochastic cooperative games. It is shown that convex games are superadditive and have nonempty cores, and that these results also hold in the context of NTU games. Furthermore, a subclass of stochastic cooperative games to which one can associate a deterministic cooperative game is considered. It is shown that such a stochastic cooperative game satisfies properties like nonemptiness of the core, superadditivity, and convexity if and only if the corresponding deterministic game satisfies these properties.Journal of Economic LiteratureClassification Number: C71.     

Saving Rates, Trade, Technology, and Stochastic Dynamics

This paper develops a framework for analyzing the stochastic dynamics of small growing trading economies with CES sector technologies. The open neoclassical two-sector growth model with a diffusion process (uncertainty) for the aggregate saving/investment ratio is demonstrated with sample paths and long-run probability distributions of the overall factor endowment ratio. Stochastic endogenous growth and cycles require a combination of fundamental growth parameter values: saving rates, terms of trade, and sectorial substitution elasticities.     

Renegotiation in Repeated Games with Side-Payments

We consider repeated games with side-payments: players have an endowment of wealth in each period in which transfers can be made. We show that if endowments are large enough and the common discount factor high enough, then a strongly renegotiation–proof equilibrium (SRP) in the sense of Farrell and Maskin exists. As the discount factor goes to 1, the set of SRP payoffs converges to the set of efficient, individually rational payoffs. These results provide a justification for the efficiency principle when agreements are not enforceable. Journal of Economic Literature Classification Numbers: C73, D23, L14.     

Stochastic stability in networks with decay

This paper considers a stylized model of network formation in which relations among agents are subject to frictions, as described in Bala and Goyal [A non cooperative model of network formation, Econometrica 68 (2000) 1181-1231]. We analyze the process of network formation in a dynamic framework where self-interested individuals can form or delete links and, occasionally, make mistakes. Then, using stochastic stability, we identify the network structures to which the formation process will converge.     

Communication in Repeated Games with Private Monitoring

The paper considers repeated games where each player can be observed by only a subset of the other players, and where players can make public announcements about the behavior of the players they observed. We address the following question: What is the minimal level of observability that is required to obtain efficient outcomes? The main result is that the limit set of sequential equilibrium payoffs, when the discount factor tends to one, contains the set of individual rational payoffs whenever each player is observed by at leasttwoother players.Journal of Economic LiteratureClassification Number: C73.