Cycling in a stochastic learning algorithm for normal form games

In this paper we study a stochastic learning model for 2×2 normal form games that are played repeatedly. The main emphasis is put on the emergence of cycles. We assume that the players have neither information about the payoff matrix of their opponent nor about their own. At every round each player can only observe his or her action and the payoff he or she receives. We prove that the learning algorithm, which is modeled by an urn scheme proposed by Arthur (1993), leads with positive probability to a cycling of strategy profiles if the game has a mixed Nash equilibrium. In case there are strict Nash equilibria, the learning process converges a.s. to the set of Nash equilibria.     

CORRELATED EQUILIBRIA,GOOD AND BAD: AN EXPERIMENTAL STUDY*

We report results from an experiment that explores the empirical validity of correlated equilibrium, an important generalization of Nash equilibrium. Specifically, we examine the conditions under which subjects playing the game of Chicken will condition their behavior on private third‐party recommendations drawn from publicly announced distributions. We find that when recommendations are given, behavior differs from both a mixed‐strategy Nash equilibrium and behavior without recommendations. In particular, subjects typically follow recommendations if and only if (1) those recommendations derive from a correlated equilibrium and (2) that correlated equilibrium is payoff‐enhancing relative to the available Nash equilibria.     

Testing the TASP: An experimental investigation of learning in games with unstable equilibria

We report experiments studying mixed strategy Nash equilibria that are theoretically stable or unstable under learning. The Time Average Shapley Polygon (TASP) predicts behavior in the unstable case. We study two versions of Rock-Paper-Scissors that include a fourth strategy, Dumb. The unique Nash equilibrium is identical in the two games, but the predicted frequency of Dumb is much higher in the game where the NE is stable. Consistent with TASP, the observed frequency of Dumb is lower and play is further from Nash in the high payoff unstable treatment. However, Dumb is played too frequently in all treatments.     

The power of ESS: An experimental study

Abstract. Our experimental design mimics a traditional evolutionary game framework where players are matched pairwise to play a symmetric 33 bimatrix game that has two Nash equilibria. One equilibrium is an evolutionary stable state, or ESS; the other is an equilibrium in dominated strategies. Our primary experimental result is the observation that the ESS becomes extremely attractive when subjects have minimal information about the payoff functions, although the dominated equilibrium assures the highest equilibrium payoff. The attractiveness of the ESS is only moderate when players are completely informed about the 33 payoff matrix. Correspondence to: S.K. Berninghaus     

Learning behavior and mixed-strategy Nash equilibria

The paper considers whether an adaptive justification, like those commonly available for non-interactive optimization models, can be found for the mixed-strategy Nash equilibrium. Although it is known that such a justification is frequently available for pure-strategy equilibria, it is shown that all members of a wide class of behaviorally plausible learning mechanisms must fail to converge in ‘almost all’ games in which the equilibrium involves mixed strategies. An exact formal analogy is developed, which allows these learning mechanisms to be viewed as discrete ta?tonnement processes in properly chosen competitive exchange economies. In the analogy, the instability derives from the independence of excess demand functions (except at the boundaries) from ‘own’ prices. This independence arises because of the linearity in probabilities of von Neumann-Morgenstern risk preferences, and therefore does not extend to pure-strategy equilibria in general. Thus, assuming that agents will play equilibrium strategies implicitly assumes more sophistication when the equilibrium involves mixed strategies than when it involves only pure strategies.     

Existence and Uniqueness of Nash Equilibrium in Electoral Competition Games: The Hybrid Case

This paper analyzes the traditional unidimensional, two‐party electoral competition game when parties have mixed motivations, in the sense that they are interested in winning the election, but also in the policy implemented after the contest. In spite of having discontinuous payoffs, this game, referred to as the hybrid election game, is shown to be payoff secure and reciprocally upper semi‐continuous. Conditional payoffs, however, are not quasi‐concave. Hence, the existence of a pure strategy Nash equilibrium (psne ) is ensured only if parties have homogenous interests in power. In that case, an equilibrium not only exists, but it is also unique. Instead, if parties have heterogeneous motivations, depending upon the relationship between the electoral uncertainty, the aggregate opportunism, and its distribution across parties, a psne may or may not exist. The mixed extension, however, is always better reply secure. Therefore, a mixed strategy Nash equilibrium does indeed exist.     

An Experiment to Evaluate Bayesian Learning of Nash Equilibrium Play

We report on an experiment designed to evaluate the empirical implications of Jordan's model of Bayesian learning in games of incomplete information. A finite example is constructed in which the model generates unique predictions of subjects' choices in nearly all periods. When the “true” game defined by players' private information was one with a unique equilibrium in pure strategies, the experimental subjects' play converged to the equilibrium, as Jordan's theory predicts, even when the subjects had not attained complete information about one another. But when there were two pure strategy equilibria, the theory's predictions were not consistent with observed behavior. Journal of Economic Literature Classification numbers: D83, C72, C92.     

Unobserved heterogeneity and equilibrium: an experimental study of Bayesian and adaptive learning in normal form games

We describe an experiment based on a simple two-person game designed so that different learning models make different predictions. Econometric analysis of the experimental data reveals clear heterogeneity in the subjects’ learning behavior. But the subjects follow only a few decision rules for basing their play on their information, and these rules have simple cognitive interpretations. There is a unique equilibrium in pure strategies, and many equilibria in mixed strategies. We find that the only equilibrium consistent with the data is one of the mixed strategy equilibria. This equilibrium is shown, surprisingly, to be consistent with Jordan's Bayesian model.     

When less information is good enough: experiments with global stag hunt games

There is mixed evidence on whether subjects coordinate on the efficient equilibrium in experimental stag hunt games under complete information. A design that generates an anomalously high level of coordination, Rankin et al. (Games Econo Behav 32(2):315–337, 2000), varies payoffs each period in repeated play rather than holding them constant. These payoff “perturbations” are eerily similar to those used to motivate the theory of global games, except the theory operates under incomplete information. Interestingly, that equilibrium selection concept is known to coincide with risk dominance, rather than payoff dominance. Thus, in theory, a small change in experimental design should produce a different equilibrium outcome. We examine this prediction in two treatments. In one, we use public signals to match Rankin et al. (2000)’s design; in the other, we use private signals to match the canonical example of global games theory. We find little difference between treatments, in both cases, subject play approaches payoff dominance. Our literature review reveals this result may have more to do with the idiosyncrasies of our complete information framework than the superiority of payoff dominance as an equilibrium selection principle.     

Evolution of Beliefs and the Nash Equilibrium of Normal Form Games

The paper formulates a simple two-person model of learning with pattern recognition and discusses its implications. In particular, it focuses on the asymptotic behavior of players' beliefs when the game has a mixed-strategy Nash equilibrium.Journal of Economics LiteratureClassification Numbers: C72, D83.     

Purification in the infinitely-repeated prisoners' dilemma

This paper investigates the Harsanyi [Harsanyi, J.C., 1973. Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points. International Journal of Game Theory 2 (1), 1–23]-purifiability of mixed strategies in the repeated prisoners' dilemma with perfect monitoring. We perturb the game so that in each period, a player receives a private payoff shock which is independently and identically distributed across players and periods. We focus on the purifiability of one-period memory mixed strategy equilibria used by Ely and Välimäki [Ely, J.C., Välimäki, J., 2002. A robust folk theorem for the prisoner's dilemma. Journal of Economic Theory 102 (1), 84–105] in their study of the repeated prisoners' dilemma with private monitoring. We find that any such strategy profile is not the limit of one-period memory equilibrium strategy profiles of the perturbed game, for almost all noise distributions. However, if we allow infinite memory strategies in the perturbed game, then any completely-mixed equilibrium is purifiable.     

Equilibrium selection through incomplete information in coordination games: an experimental study

We perform an experiment on a pure coordination game with uncertainty about the payoffs. Our game is closely related to models that have been used in many macroeconomic and financial applications to solve problems of equilibrium indeterminacy. In our experiment, each subject receives a noisy signal about the true payoffs. This game (inspired by the “global” games of Carlsson and van Damme, Econometrica, 61, 989–1018, 1993) has a unique strategy profile that survives the iterative deletion of strictly dominated strategies (thus a unique Nash equilibrium). The equilibrium outcome coincides, on average, with the risk-dominant equilibrium outcome of the underlying coordination game. In the baseline game, the behavior of the subjects converges to the theoretical prediction after enough experience has been gained. The data (and the comments) suggest that this behavior can be explained by learning. To test this hypothesis, we use a different game with incomplete information, related to a complete information game where learning and prior experiments suggest a different behavior. Indeed, in the second treatment, the behavior did not converge to equilibrium within 50 periods in some of the sessions. We also run both games under complete information. The results are sufficiently similar between complete and incomplete information to suggest that risk-dominance is also an important part of the explanation.      

How sensitive are bargaining outcomes to changes in disagreement payoffs?

We use a human-subjects experiment to investigate how bargaining outcomes are affected by changes in bargainers’ disagreement payoffs. Subjects bargain against changing opponents, with randomly drawn asymmetric disagreement outcomes that vary over plays of the game, and with complete information about disagreement payoffs and the cake size. We find that subjects only respond about half as much as theoretically predicted to changes in their own disagreement payoff and to changes in their opponent’s disagreement payoff. This effect is observed in a standard Nash demand game and a related unstructured bargaining game, in both early and late rounds, and is robust to moderate changes in stake sizes. We show theoretically that standard models of expected utility maximisation are unable to account for this under-responsiveness, even when generalised to allow for risk aversion. We also show that quantal-response equilibrium has, at best, mixed success in characterising our results. However, a simple model of other-regarding preferences can explain our main results.     

Robustness to strategic uncertainty

We introduce a criterion for robustness to strategic uncertainty in games with continuum strategy sets. We model a player's uncertainty about another player's strategy as an atomless probability distribution over that player's strategy set. We call a strategy profile robust to strategic uncertainty if it is the limit, as uncertainty vanishes, of some sequence of strategy profiles in which every player's strategy is optimal under his or her uncertainty about the others. When payoff functions are continuous we show that our criterion is a refinement of Nash equilibrium and we also give sufficient conditions for existence of a robust strategy profile. In addition, we apply the criterion to Bertrand games with convex costs, a class of games with discontinuous payoff functions and a continuum of Nash equilibria. We show that it then selects a unique Nash equilibrium, in agreement with some recent experimental findings.     

Batch queues with choice of arrivals: Equilibrium analysis and experimental study

We study the decisions agents make in two queueing games with endogenously determined arrivals and batch service. In both games, agents are asked to independently decide when to join a queue, or they may simply choose not to join it at all. The symmetric mixed-strategy equilibrium of two games in discrete time where balking is prohibited and where it is allowed are tested experimentally in a study that varies the game type (balking vs. no balking) and information structure (private vs. public information). With repeated iterations of the stage game, all four experimental conditions result in aggregate, but not individual, behavior approaching mixed-strategy equilibrium play. Individual behavior can be accounted for by relatively simple heuristics.     

Continuity of the payoff functions

Summary. In his Nash equilibrium paper, Glicksberg states that the payoff functions are continuous. Such a function is defined on the product of mixed strategies, which are the Borel probability measures on a compactum, endowed with the product of the weak topologies. The continuity property is used in proving the existence of Nash equilibria. This note proves that the payoff functions are continuous, which is not immediate to establish. Received: June 10, 1999; revised version: June 24, 1999     

On the Coalition-Proofness of the Pareto Frontier of the Set of Nash Equilibria

This article shows that the Pareto efficient frontier of the Nash equilibrium set of games with strategic substitutes is coalition-proof under the following conditions: (1) the game has three players, or, alternatively, a player's payoff depends on her own strategy and on the sum (but not on the composition) of other players' strategies; (2) an increase in a player's strategy either raises all other players' payoffs monotonically or reduces them monotonically; and (3) each player's payoff is strictly concave in her own strategy. Under these conditions, the Pareto dominance refinement is equivalent to the coalition-proof Nash equilibrium refinement.Journal of Economic LiteratureClassification Number: C72.     

An Experimental Study of Communication and Coordination in Noncooperative Games

This paper reports the results of an experiment designed to test the usefulness of alternative solution concepts to explain players' behavior in noncooperative games with preplay communication. In the experiment subjects communicate byplain conversationprior to playing a simple game. In this setting, we find that the presumption ofindividualisticandindependentbehavior underlying the concept of Nash equilibrium is inappropriate. Instead, we observe behavior to becoordinatedandcorrelated. Statistical tests reject Nash equilibrium as an explanation of observed play. The coalition proof correlated equilibrium of the game, however, explains the data when the possibility of errors by players is introduced.Journal of Economic LiteratureClassification Numbers: C72, C92.     

Nash equilibrium and party polarization in an electoral model with mixed motivations

We study the existence problem of Nash equilibrium as well as the patterns of equilibrium policy outcomes in an electoral competition model with mixed motivations. Each party maximizes a sum of party members’ expected utility and office rent. The inclusion of office rent renders the payoff of each party discontinuous. This makes it difficult to apply usually fixed point arguments to prove the existence of Nash equilibria. By using a recently developed concept, multiple restrictional security (MR‐security) we provide conditions under which a pure‐strategy Nash equilibrium exists within fairly general settings, and further the analysis by presenting conditions under which various patterns of policy choices, including polarization, arise in equilibrium.     

Interim Bayesian Nash equilibrium on universal type spaces for supermodular games

We prove the existence of a greatest and a least interim Bayesian Nash equilibrium for supermodular games of incomplete information. There are two main differences from the earlier proofs and from general existence results for non-supermodular Bayesian games: (a) we use the interim formulation of a Bayesian game, in which each player's beliefs are part of his or her type rather than being derived from a prior; (b) we use the interim formulation of a Bayesian Nash equilibrium, in which each player and every type (rather than almost every type) chooses a best response to the strategy profile of the other players. There are no restrictions on type spaces and action sets may be any compact metric lattices.