Markov Perfect Equilibrium: I. Observable Actions

We define Markov strategy and Markov perfect equilibrium (MPE) for games with observable actions. Informally, a Markov strategy depends only on payoff-relevant past events. More precisely, it is measurable with respect to the coarsest partition of histories for which, if all other players use measurable strategies, each player's decision-problem is also measurable. For many games, this definition is equivalent to a simple affine invariance condition. We also show that an MPE is generically robust: if payoffs of a generic game are perturbed, there exists an almost Markovian equilibrium in the perturbed game near the initial MPE. Journal of Economic Literature Classification Numbers: C72, C73.     

Equilibria in large games with continuous procedures

We study existence of equilibria in large games where players use boundedly rational procedures. The equilibria are different from Nash equilibria; the difference persists even when players use procedures for which the observations gathered in any period are used to evaluate the payoff from different actions.     

Global dynamics in repeated games with additively separable payoffs

This paper studies the global dynamics of a class of infinitely repeated two-player games in which the action space of each player is an interval, and the one-shot payoff of each player is additively separable in actions. We define an immediately reactive equilibrium (IRE) as a pure-strategy subgame perfect equilibrium such that each player's action is a stationary function of the opponent's last action. We completely characterize IREs and their dynamics in terms of certain indifference curves. Our results are used to show that in a prisoners' dilemma game with mixed strategies, gradual cooperation occurs when the players are sufficiently patient, and that in a certain duopoly game, kinked demand curves emerge naturally.     

A folk theorem for repeated games played on a network

I consider repeated games on a network where players interact and communicate with their neighbors. At each stage, players choose actions and exchange private messages with their neighbors. The payoff of a player depends only on his own action and on the actions of his neighbors. At the end of each stage, a player is only informed of his payoff and of the messages he received from his neighbors. Payoffs are assumed to be sensitive to unilateral deviations. The main result is to establish a necessary and sufficient condition on the network for a Nash folk theorem to hold, for any such payoff function.     

Belief-Based Equilibria in the Repeated Prisoners' Dilemma with Private Monitoring

We analyze repeated prisoners' dilemma games with imperfect private monitoring and construct mixed trigger strategy equilibria. Such strategies have a simple representation, where a player's action only depends upon her belief that her opponent(s) are continuing to cooperate. When monitoring is almost perfect, the symmetric efficient outcome can be approximated in any prisoners' dilemma game, while every individually rational feasible payoff can be approximated in a class of such games. The efficiency result extends when there are more than two players. It requires that monitoring be sufficiently accurate but does not require very low discounting when a public randomization device is available. Journal of Economic Literature Classification Numbers: C73, D82.     

Monotone equilibria in Bayesian games of strategic complementarities

For Bayesian games of strategic complementarities, we provide a constructive proof of the existence of a greatest and a least Bayesian Nash equilibrium, each one in strategies that are monotone in type. Our main assumptions, besides strategic complementarities, are that each player's payoff displays increasing differences in own action and the profile of types and that each player's interim beliefs are increasing in type with respect to first-order stochastic dominance (e.g., types are affiliated). The result holds for general action and type spaces (single-, multi-, or infinite-dimensional; continuous or discrete) and no prior is assumed. We also provide the following comparative statics result: the greatest and least equilibria are higher if there is a first-order stochastic dominant shift in the interim beliefs. We apply this result to strategic information revelation in games of voluntary disclosure.     

Learning to play games in extensive form by valuation

Game theoretic models of learning which are based on the strategic form of the game cannot explain learning in games with large extensive form. We study learning in such games by using valuation of moves. A valuation for a player is a numeric assessment of her moves that purports to reflect their desirability. We consider a myopic player, who chooses moves with the highest valuation. Each time the game is played, the player revises her valuation by assigning the payoff obtained in the play to each of the moves she has made. We show for a repeated win-lose game that if the player has a winning strategy in the stage game, there is almost surely a time after which she always wins. When a player has more than two payoffs, a more elaborate learning procedure is required. We consider one that associates with each move the average payoff in the rounds in which this move was made. When all players adopt this learning procedure, with some perturbations, then, with probability 1 there is a time after which strategies that are close to subgame perfect equilibrium are played. A single player who adopts this procedure can guarantee only her individually rational payoff.     

Repeated signaling games

I analyze a class of repeated signaling games in which the informed player's type is persistent and the history of actions is perfectly observable. In this context, a large class of possibly complex sequences of signals can be supported as the separating equilibrium actions of the “strong type” of the informed player. I characterize the set of such sequences. I also characterize the sequences of signals in least cost separating equilibria (LCSE) of these games. In doing this, I introduce a state variable that can be interpreted as a measure of reputation. This gives the optimization problem characterizing the LCSE a recursive structure. I show that, in general, the equilibrium path sequences of signals have a simple structure. The shapes of the optimal sequences depend critically on the relative concavities of the payoff functions of different types, which measure the relative preferences towards payoff smoothing.     

Nash equilibria in ∞-dimensional spaces: an approximation theorem

Summary. We show, by employing a density result for probability measures, that in games with a finite number of players and ∞-dimensional pure strategy spaces Nash equilibria can be approximated by finite mixed strategies. Given ε>0, each player receives an expected utility payoff ε/2 close to his Nash payoff and no player could change his strategy unilaterally and do better than ε. Received: July 15, 1997; revised version: February 6, 1998     

On Sustaining Cooperation without Public Observations

This paper examines a dynamic game in which each player only observes a private and imperfect signal on the actions played. Our main result is that in a repeated prisoner's dilemma where defections are irreversible (at least for a long enough period of time), patient enough players may achieve almost efficient outcomes. Dealing with models of imperfect private monitoring is difficult because (i) continuation games are games of incomplete information, hence they do not have the same structure as the original game. In particular, continuation equilibria are correlated equilibria. (ii) Players are typically uncertain about their opponents' past observations and actions, and they use their entire own private history to learn about these actions. As a result equilibrium strategies are in general nontrivial and increasingly complex functions of past observations. We bypass these difficulties by looking at correlated equilibria of the original game and find correlated equilibria in which the decision problem faced by each player remains the same over time. Journal of Economic Literature Classification Numbers: C72.     

Bounding the inefficiency of equilibria in nonatomic congestion games

Equilibria in noncooperative games are typically inefficient, as illustrated by the Prisoner's Dilemma. In this paper, we quantify this inefficiency by comparing the payoffs of equilibria to the payoffs of a “best possible” outcome. We study a nonatomic version of the congestion games defined by Rosenthal [Int. J. Game Theory 2 (1973) 65], and identify games in which equilibria are approximately optimal in the sense that no other outcome achieves a significantly larger total payoff to the players—games in which optimization by individuals approximately optimizes the social good, in spite of the lack of coordination between players. Our results extend previous work on traffic routing games.     

Communication in repeated network games with imperfect monitoring

I consider repeated games with private monitoring played on a network. Each player has a set of neighbors with whom he interacts: a player's payoff depends on his own and his neighbors' actions only. Monitoring is private and imperfect: each player observes his stage payoff but not the actions of his neighbors. Players can communicate costlessly at each stage: communication can be public, private or a mixture of both. Payoffs are assumed to be sensitive to unilateral deviations. First, for any network, a folk theorem holds if some Joint Pairwise Identifiability condition regarding payoff functions is satisfied. Second, a necessary and sufficient condition on the network topology for a folk theorem to hold for all payoff functions is that no two players have the same set of neighbors not counting each other.     

A general structure theorem for the Nash equilibrium correspondence

I consider n-person normal form games where the strategy set of each player is a non-empty compact convex subset of an Euclidean space, and the payoff function of player i is continuous in joint strategies and continuously differentiable and concave in the player i's strategy. No further restrictions (such as multilinearity of the payoff functions or the requirement that the strategy sets be polyhedral) are imposed. I demonstrate that the graph of the Nash equilibrium correspondence on this domain is homeomorphic to the space of games. This result generalizes a well-known structure theorem in [Kohlberg, E., Mertens, J.-F., 1986. On the strategic stability of equilibria. Econometrica 54, 1003–1037]. It is supplemented by an extension analogous to the unknottedness theorems in [Demichelis S., Germano, F., 2000. Some consequences of the unknottedness of the Walras correspondence. J. Math. Econ. 34, 537–545; Demichelis S., Germano, F., 2002. On (un)knots and dynamics in games. Games Econ. Behav. 41, 46–60]: the graph of the Nash equilibrium correspondence is ambient isotopic to a trivial copy of the space of games.     

Directed Networks with Spillovers

We study noncooperative network formation in two types of directed networks. In the first type, called the model with global spillovers, the payoff of a player depends on the number of links she forms as well as the total number of links formed by all other players. In the second type, called the model with local spillovers, the payoff of a player depends on the number of links she forms and the total number of links formed by her immediate neighbors, as well as the number of links formed by players outside her neighborhood. For both classes of games we investigate the existence of pure strategy Nash equilibria and characterize the Nash networks under a number of different second order conditions on the payoff function.     

Cycling in a stochastic learning algorithm for normal form games

In this paper we study a stochastic learning model for 2×2 normal form games that are played repeatedly. The main emphasis is put on the emergence of cycles. We assume that the players have neither information about the payoff matrix of their opponent nor about their own. At every round each player can only observe his or her action and the payoff he or she receives. We prove that the learning algorithm, which is modeled by an urn scheme proposed by Arthur (1993), leads with positive probability to a cycling of strategy profiles if the game has a mixed Nash equilibrium. In case there are strict Nash equilibria, the learning process converges a.s. to the set of Nash equilibria.     

Interim Bayesian Nash equilibrium on universal type spaces for supermodular games

We prove the existence of a greatest and a least interim Bayesian Nash equilibrium for supermodular games of incomplete information. There are two main differences from the earlier proofs and from general existence results for non-supermodular Bayesian games: (a) we use the interim formulation of a Bayesian game, in which each player's beliefs are part of his or her type rather than being derived from a prior; (b) we use the interim formulation of a Bayesian Nash equilibrium, in which each player and every type (rather than almost every type) chooses a best response to the strategy profile of the other players. There are no restrictions on type spaces and action sets may be any compact metric lattices.     

Aggregate comparative statics

In aggregative games, each playerʼs payoff depends on her own actions and an aggregate of the actions of all the players. Many common games in industrial organization, political economy, public economics, and macroeconomics can be cast as aggregative games. This paper provides a general and tractable framework for comparative static results in aggregative games. We focus on two classes of games: (1) aggregative games with strategic substitutes and (2) nice aggregative games, where payoff functions are continuous and concave in own strategies. We provide simple sufficient conditions under which positive shocks to individual players increase their own actions and have monotone effects on the aggregate. The results are illustrated with applications to public good provision, contests, Cournot competition and technology choices in oligopoly.     

Infinite horizon common interest games with perfect information

We consider infinite horizon common interest games with perfect information. A game is a K-coordination game if each player can decrease other players' payoffs by at most K times his own cost of punishment. The number K represents the degree of commonality of payoffs among the players. The smaller K is, the more interest the players share. A K-coordination game tapers off if the greatest payoff variation conditional on the first t periods of an efficient history converges to 0 at a rate faster than K^−t as t→∞. We show that every subgame perfect equilibrium outcome is efficient in any tapering-off game with perfect information. Applications include asynchronously repeated games, repeated games of extensive form games, asymptotically finite horizon games, and asymptotically pure coordination games.     

Equivalence of strong and coalition-proof Nash equilibria in games without spillovers

Summary This paper examines the conditions which guarantee that the set of coalition-proof Nash equilibria coincides with the set of strong Nash equilibria in the normal form games withoutspillovers. We find thatpopulation monotonicity properties of the payoff functions, when the payoff of a player changes monotonically when the size of the group of players choosing the same strategy increases, are crucial to obtain the equivalence of these two solution concepts. We identify the classes of games, satisfying population monotonicity properties, which yield the equivalence of the set of coalition-proof Nash equilibria and the set of strong Nash equilibria. We also provide sufficient conditions for the equivalence result even when the population monotonicity assumptions are relaxed.We wish to thank Mamoru Kaneko, Akihiko Matsui, Tomoichi Shinotsuka, Benyamin Shitoviz, Tayfun Sonmez, William Thomson, the participants of the Southeastern Economic Theory Meeting in Charlottesville and the seminars at CORE and University of Tsukuba for useful discussions and comments. Our special thanks due anonymous referee for the suggestion to add a section addressing the issue of existence of a strong Nash equilibrium.     

Random belief equilibrium in normal form games

In defining random belief equilibrium (RBE) in finite, normal form games we assume a player's beliefs about others' strategy choices are randomly drawn from a belief distribution that is dispersed around a central strategy profile, the focus. At an RBE: (1) Each chooses a best response relative to her beliefs. (2) Each player's expected choice coincides with the focus of the other players' belief distributions. RBE provides a statistical framework for estimation which we apply to data from three experimental games. We also characterize the limit-RBE as players' beliefs converge to certainty. When atoms in the belief distributions vanish in the limit, not all limit-RBE (called robust equilibria) are trembling hand perfect Nash equilibria and not all perfect equilibria are robust.