Cycling in a stochastic learning algorithm for normal form games

In this paper we study a stochastic learning model for 2×2 normal form games that are played repeatedly. The main emphasis is put on the emergence of cycles. We assume that the players have neither information about the payoff matrix of their opponent nor about their own. At every round each player can only observe his or her action and the payoff he or she receives. We prove that the learning algorithm, which is modeled by an urn scheme proposed by Arthur (1993), leads with positive probability to a cycling of strategy profiles if the game has a mixed Nash equilibrium. In case there are strict Nash equilibria, the learning process converges a.s. to the set of Nash equilibria.     

Characterization of the extreme core allocations of the assignment game

A characterization of the extreme core allocations of the assignment game is given in terms of the reduced marginal worth vectors. For each ordering in the player set, a payoff vector is defined where each player receives his or her marginal contribution to a certain reduced game played by his or her predecessors. This set of reduced marginal worth vectors, which for convex games coincide with the usual marginal worth vectors, is proved to be the set of extreme points of the core of the assignment game. Therefore, although assignment games are hardly ever convex, the same characterization of extreme core allocations is valid for convex games.     

Learning with perfect information

For extensive form games with perfect information, consider a learning process in which, at any iteration, each player unilaterally deviates to a best response to his current conjectures of others' strategies; and then updates his conjectures in accordance with the induced play of the game. We show that, for generic payoffs, the outcome of the game becomes stationary, and is consistent with Nash equilibrium. In general, if payoffs have ties or if players observe more of each others' strategies than is revealed by plays of the game, the same result holds provided a rationality constraint is imposed on unilateral deviations: no player changes his moves in subgames that he deems unreachable, unless he stands to improve his payoff there. Moreover, with this constraint, the sequence of strategies and conjectures also becomes stationary, and yields a self-confirming equilibrium.     

Dismissals and quits in repeated games

This paper studies repeated games, where player 1 can decide to let the opponent continue or replace her by a new player. We also allow for the possibility of player 2 quitting the game. When only layoffs can occur, a folk theorem for finite horizons obtains due to the threat that termination of the relationship imposes on player 2. However, quits limit this result to those cases in which the outside option for player 2 is small (lower than some Nash equilibrium payoff of the stage game).     

Directed Networks with Spillovers

We study noncooperative network formation in two types of directed networks. In the first type, called the model with global spillovers, the payoff of a player depends on the number of links she forms as well as the total number of links formed by all other players. In the second type, called the model with local spillovers, the payoff of a player depends on the number of links she forms and the total number of links formed by her immediate neighbors, as well as the number of links formed by players outside her neighborhood. For both classes of games we investigate the existence of pure strategy Nash equilibria and characterize the Nash networks under a number of different second order conditions on the payoff function.     

TIME CONSISTENCY IN ALTERNATING‐MOVE POLICY GAMES*

We examine alternating‐move policy games where the government and the private sector alternate their moves. In contrast with the standard policy game, the set of equilibrium payoffs of the present model is bounded away from the payoff under the one‐shot Nash equilibrium, called the Kydland–Prescott outcome, and the upper bound is close to the payoff under the optimal policy (called the Ramsey policy) if the government is sufficiently patient. In other words, the Kydland–Prescott outcome is not time consistent, while the Ramsey outcome could be approximated by a time‐consistent policy of the same game.     

One-sided patience with one-sided communication does not justify stackelberg equilibrium

The theory of games (with complete information) in which a single patient longrun player faces a succession of short-run opponents cannot plausibly be used to justify the Stackelberg solution concept, because if that player can select which subgame-perfect equilibrium is to be played then she can presumably also change her selection. Consequently, while she can choose among one-shot Nash outcomes, she cannot achieve the Stackelberg outcome.     

Reputation and Perfection in Repeated Common Interest Games

We consider a wide class of repeated common interest games perturbed with one-sided incomplete information: one player (the informed player) might be a commitment type playing the Pareto dominant action. As discounting, which is assumed to be symmetric, and the prior probability of the commitment type go to zero, it is shown that the informed player can be held close to her minmax payoff even when perfection is imposed on the equilibrium.Journal of Economic LiteratureClassification Numbers: C73, D83.     

A Folk Theorem for Repeated Sequential Games

We study repeated sequential games where players may not move simultaneously in stage games. We introduce the concept of effective minimax for sequential games and establish a Folk theorem for repeated sequential games. The Folk theorem asserts that any feasible payoff vector where every player receives more than his effective minimax value in a sequential stage game can be supported by a subgame perfect equilibrium in the corresponding repeated sequential game when players are sufficiently patient. The results of this paper generalize those of Wen (1994), and of Fudenberg and Maskin (1986). The model of repeated sequential games and the concept of effective minimax provide an alternative view to the Anti–Folk theorem of Lagunoff and Matsui (1997) for asynchronously repeated pure coordination games.     

UNIQUENESS OF EQUILIBRIUM PAYOFFS IN FINITELY REPEATED GAMES WITH IMPERFECT MONITORING*

This paper examines when a finitely repeated game with imperfect monitoring has a unique equilibrium payoff vector. This problem is nontrivial under imperfect monitoring, because uniqueness of equilibrium (outcome) in the stage game does not extend to finitely repeated games. A (correlated) equilibrium is equilibrium minimaxing if any player's equilibrium payoff is her minimax value when the other players choose a correlated action profile from the actions played in the equilibrium. The uniqueness result holds if all stage game correlated equilibria are equilibrium minimaxing and have the same payoffs. The uniqueness result does not hold under weaker conditions.     

An experimental study of information and mixed-strategy play in the three-person matching-pennies game

Summary. Recent experiments on mixed-strategy play in experimental games reject the hypothesis that subjects play a mixed strategy even when that strategy is the unique Nash equilibrium prediction. However, in a three-person matching-pennies game played with perfect monitoring and complete payoff information, we cannot reject the hypothesis that subjects play the mixed-strategy Nash equilibrium. Given this support for mixed-strategy play, we then consider two qualitatively different learning theories (sophisticated Bayesian and naive Bayesian) which predict that the amount of information given to subjects will determine whether they can learn to play the predicted mixed strategies. We reject the hypothesis that subjects play the symmetric mixed-strategy Nash equilibrium when they do not have complete payoff information. This finding suggests that players did not use sophisticated Bayesian learning to reach the mixed-strategy Nash equilibrium. Received: August 9, 1996; revised version: October 21, 1998     

Evolving Aspirations and Cooperation

A 2×2 game is played repeatedly by two satisficing players. The game considered includes the Prisoner's Dilemma, as well as games of coordination and common interest. Each player has anaspirationat each date, and takes an action. The action is switched at the subsequent period only if the achieved payoff falls below aspirations; the switching probability depends on the shortfall. Aspirations are periodically updated according to payoff experience, but are occasionally subject to trembles. For sufficiently slow updating of aspirations and small tremble probability, it is shown that both players must ultimately cooperate most of the time.Journal of Economic LiteratureClassification Numbers C72, D83.     

Communication in repeated network games with imperfect monitoring

I consider repeated games with private monitoring played on a network. Each player has a set of neighbors with whom he interacts: a player's payoff depends on his own and his neighbors' actions only. Monitoring is private and imperfect: each player observes his stage payoff but not the actions of his neighbors. Players can communicate costlessly at each stage: communication can be public, private or a mixture of both. Payoffs are assumed to be sensitive to unilateral deviations. First, for any network, a folk theorem holds if some Joint Pairwise Identifiability condition regarding payoff functions is satisfied. Second, a necessary and sufficient condition on the network topology for a folk theorem to hold for all payoff functions is that no two players have the same set of neighbors not counting each other.     

Combinatorial Games under Auction Play

ARichman gameis a combinatorial game in which, rather than alternating moves, the two players bid for the privilege of making the next move. The theory of such games is a hybrid between the classical theory of games (von Neumann, Morgenstern, …) and the combinatorial theory of games (Berlekamp, Conway, Guy, …). We expand upon our previous work by considering games with infinitely many positions as well as several variants including thePoorman variantin which the high bidder pays a third party (rather than the other player).Journal of Economic LiteratureClassification Number: C7.     

Representing equilibrium aggregates in aggregate games with applications to common agency

An aggregate game is a normal-form game with the property that each playerʼs payoff is a function of only his own strategy and an aggregate of the strategy profile of all players. Such games possess properties that can often yield simple characterizations of equilibrium aggregates without requiring that one solves for the equilibrium strategy profile. When payoffs have a quasi-linear structure and a degree of symmetry, we construct a self-generating maximization program over the space of aggregates with the property that the solution set corresponds to the set of equilibrium aggregates of the original n-player game. We illustrate the value of this approach in common-agency games where the playersʼ strategy space is an infinite-dimensional space of nonlinear contracts. We derive equilibrium existence and characterization theorems for both the adverse selection and moral hazard versions of these games.     

Reputation with equal discounting in repeated games with strictly conflicting interests

We analyze reputation effects in two-player repeated games of strictly conflicting interests. In such games, player 1 has a commitment action such that a best reply to it gives player 1 the highest individually rational payoff and player 2 the minmax payoff. Players have equal discount factors. With positive probability player 1 is a type who chooses the commitment action after every history. We show that player 1's payoff converges to the maximally feasible payoff when the discount factor converges to one. This contrasts with failures of reputation effects for equal discount factors that have been demonstrated in the literature.     

Sophisticated Experience-Weighted Attraction Learning and Strategic Teaching in Repeated Games

Most learning models assume players are adaptive (i.e., they respond only to their own previous experience and ignore others' payoff information) and behavior is not sensitive to the way in which players are matched. Empirical evidence suggests otherwise. In this paper, we extend our adaptive experience-weighted attraction (EWA) learning model to capture sophisticated learning and strategic teaching in repeated games. The generalized model assumes there is a mixture of adaptive learners and sophisticated players. An adaptive learner adjusts his behavior the EWA way. A sophisticated player rationally best-responds to her forecasts of all other behaviors. A sophisticated player can be either myopic or farsighted. A farsighted player develops multiple-period rather than single-period forecasts of others' behaviors and chooses to “teach” the other players by choosing a strategy scenario that gives her the highest discounted net present value. We estimate the model using data from p-beauty contests and repeated trust games with incomplete information. The generalized model is better than the adaptive EWA model in describing and predicting behavior. Including teaching also allows an empirical learning-based approach to reputation formation which predicts better than a quantal-response extension of the standard type-based approach. Journal of Economic Literature Classification Numbers: C72, C91.     

The single deviation property in games with discontinuous payoffs

We study equilibrium existence in normal form games in which it is possible to associate with each nonequilibrium point an open neighborhood, a set of players, and a collection of deviation strategies, such that at any nonequilibrium point of the neighborhood, a player from the set can increase her payoff by switching to the deviation strategy designated for her. An equilibrium existence theorem for compact, quasiconcave games with two players is established as an application of a general equilibrium existence result for qualitative games. A new form of the better-reply security condition, called the strong single deviation property, is proposed.     

Decompositions and potentials for normal form games

A basic model of commitment is to convert a two-player game in strategic form to a “leadership game” with the same payoffs, where one player, the leader, commits to a strategy, to which the second player always chooses a best reply. This paper studies such leadership games for games with convex strategy sets. We apply them to mixed extensions of finite games, which we analyze completely, including nongeneric games. The main result is that leadership is advantageous in the sense that, as a set, the leader's payoffs in equilibrium are at least as high as his Nash and correlated equilibrium payoffs in the simultaneous game. We also consider leadership games with three or more players, where most conclusions no longer hold.     

Bargaining with Imperfect Commitment

We study the effect of imperfect commitment in noncooperative two-person bargaining games. By establishing the reputation for being stubborn, a player sometimes commits to her initial demand, becoming unable to change her demands or to accept an inferior offer from her opponent. When the probability of being stubborn is small, the set of equilibria is shown to be small and agreement may be reached immediately despite the possibility of stubbornness. A player has greater bargaining power when she is more patient and/or is more likely to be stubborn. Journal of Economic Literature Classification Number: C78.