Collusion as public monitoring becomes noisy: Experimental evidence

This paper uses laboratory experiments to test the implications of the theory of repeated games on equilibrium payoffs and estimate strategies in an infinitely repeated prisoners' dilemma game with imperfect public monitoring. We find that subjects' payoffs (i) decrease as noise increases, and (ii) are lower than the theoretical maximum for low noise, but exceed it for high noise. Under the assumption that the subjects' strategy uses thresholds on the public signal for transition between cooperation and punishment states, we find that the best fitting strategy simply compares the most recent public signal against a single threshold.     

Decompositions and potentials for normal form games

A basic model of commitment is to convert a two-player game in strategic form to a “leadership game” with the same payoffs, where one player, the leader, commits to a strategy, to which the second player always chooses a best reply. This paper studies such leadership games for games with convex strategy sets. We apply them to mixed extensions of finite games, which we analyze completely, including nongeneric games. The main result is that leadership is advantageous in the sense that, as a set, the leader's payoffs in equilibrium are at least as high as his Nash and correlated equilibrium payoffs in the simultaneous game. We also consider leadership games with three or more players, where most conclusions no longer hold.     

Stable games and their dynamics

We study a class of population games called stable games. These games are characterized by self-defeating externalities: when agents revise their strategies, the improvements in the payoffs of strategies to which revising agents are switching are always exceeded by the improvements in the payoffs of strategies which revising agents are abandoning. We prove that the set of Nash equilibria of a stable game is globally asymptotically stable under a wide range of evolutionary dynamics. Convergence results for stable games are not as general as those for potential games: in addition to monotonicity of the dynamics, integrability of the agents' revision protocols plays a key role.     

Higher order game dynamics

Continuous-time game dynamics are typically first order systems where payoffs determine the growth rate of the players? strategy shares. In this paper, we investigate what happens beyond first order by viewing payoffs as higher order forces of change, specifying e.g. the acceleration of the players? evolution instead of its velocity (a viewpoint which emerges naturally when it comes to aggregating empirical data of past instances of play). To that end, we derive a wide class of higher order game dynamics, generalizing first order imitative dynamics, and, in particular, the replicator dynamics. We show that strictly dominated strategies become extinct in n-th order payoff-monotonic dynamics n   orders as fast as in the corresponding first order dynamics; furthermore, in stark contrast to first order, weakly dominated strategies also become extinct for n?2$n?2$. All in all, higher order payoff-monotonic dynamics lead to the elimination of weakly dominated strategies, followed by the iterated deletion of strictly dominated strategies, thus providing a dynamic justification of the well-known epistemic rationalizability process of Dekel and Fudenberg [7]. Finally, we also establish a higher order analogue of the folk theorem of evolutionary game theory, and we show that convergence to strict equilibria in n-th order dynamics is n orders as fast as in first order.     

Stochastic evolutionary selection in finite populations

We primarily focus on a wide range of stochastic evolutionary game dynamics between two strategies which are characterized by a condition we call monotonicity: the sign of the difference between the probabilities of increasing and decreasing an A-individual completely depends on the difference of payoffs based on different strategies. When mutations are excluded, we provide sufficient conditions for selection to favor one strategy over the other and necessary conditions for selection to favor or oppose change, respectively. Moreover, we discuss which strategy will be favored in case of rare mutations and give a simple rule to determine evolutionary selection of strategies for large populations under some specific stochastic mutation–selection dynamics.     

Search theory,competitive equilibrium,and the Nash bargaining solution

We investigate a canonical search-theoretic model without entry. Two agents are randomly matched with a long side being rationed. The matched agents face a pair of randomly drawn non-transferable payoffs, and then choose whether or not to form a partnership subject to a small probability of exogenous break down. As this probability and friction vanish, the Nash bargaining solution emerges as the unique undominated strategy equilibrium outcome if the mass of each party is the same. If the size of one party is larger than the other, the short side extracts the entire surplus, a sharp contrast to Rubinstein and Wolinsky (1985) [16].     

Repeated games with one-memory

We study the extent to which equilibrium payoffs of discounted repeated games can be obtained by 1-memory strategies. We establish the following in games with perfect (rich) action spaces: First, when the players are sufficiently patient, the subgame perfect Folk Theorem holds with 1-memory. Second, for arbitrary level of discounting, all strictly enforceable subgame perfect equilibrium payoffs can be approximately supported with 1-memory if the number of players exceeds two. Furthermore, in this case all subgame perfect equilibrium payoffs can be approximately supported by an ε-equilibrium with 1-memory. In two-player games, the same set of results hold if an additional restriction is assumed: Players must have common punishments. Finally, to illustrate the role of our assumptions, we present robust examples of games in which there is a subgame perfect equilibrium payoff profile that cannot be obtained with 1-memory. Thus, our results are the best that can be hoped for.     

Individual Differences in EWA Learning with Partial Payoff Information

We extend experience-weighted attraction (EWA) learning to games in which only the set of possible foregone payoffs from unchosen strategies are known, and estimate parameters separately for each player to study heterogeneity. We assume players estimate unknown foregone payoffs from a strategy, by substituting the last payoff actually received from that strategy, by clairvoyantly guessing the actual foregone payoff, or by averaging the set of possible foregone payoffs conditional on the actual outcomes. All three assumptions improve predictive accuracy of EWA. Individual parameter estimates suggest that players cluster into two separate subgroups (which differ from traditional reinforcement and belief learning).     

On uniquely implementing cooperation in the prisoners' dilemma

Summary We consider the problem of a principle who wishes to induce two agents playing a one shot prisoner's dilemma to behave cooperatively. We assume that the principal cannot observe the actions of the agents, and is not able to change the strategy sets or payoff functions in the underlying game. The only power the principle has is to randomly delay the arrival of payoffs. Specifically, agents choose their one shot strategies, and then the principle randomly determines whether these are cheap talk, or if payoffs should be distributed. If the round is cheap talk, then each agent observes the strategy choice of the other and play moves to a new round. This continues until payoffs are distributed. We establish conditions under which the probability of cheap talk can be chosen at the beginning of the induced game in such a way that full cooperation is the only equilibrium outcome. The sufficiency condition is met by a wide class of economic interpretations of the prisoners' dilemma, including those involving strategic complementarities among players.The authors wish to thank Dilip Abreu, Robert Aumann, Michael Baye, James Friedman, Richard McLean, Herve Moulin, Ariel Rubinstein, Rajiv Vohra and Simon Wilkie for their comments. Also, participants in seminars and conferences at Arizona, Bellcore, Brown, Illinois, Northwestern, Princeton, Rochester, Vanderbilt and West Virginia have provided stimulating comments. We also thank the referee for many detailed and useful suggestions. The third author's research was supported in part by NSF grant SES-9213145.     

On the convergence of reinforcement learning

This paper examines the convergence of payoffs and strategies in Erev and Roth's model of reinforcement learning. When all players use this rule it eliminates iteratively dominated strategies and in two-person constant-sum games average payoffs converge to the value of the game. Strategies converge in constant-sum games with unique equilibria if they are pure or if they are mixed and the game is 2×2. The long-run behaviour of the learning rule is governed by equations related to Maynard Smith's version of the replicator dynamic. Properties of the learning rule against general opponents are also studied.     

Evolutionary selection of ‘chivalrous’ conventions in coordination games without common expectations

We study an evolutionary game-theoretic model where players have to choose within a predetermined set of mixed strategies in a coordination game. Players are of two different kinds, male and female. No common expectations assumption is made; players tend therefore to adopt the strategy that yields larger than average expected payoffs for their kind. In this framework, every stable stationary point of the population dynamics can be interpreted as the emergence of a particular convention. A classification of the possible conventions is provided; conditions for their emergence are determined.     

Excess payoff dynamics and other well-behaved evolutionary dynamics

We consider a model of evolution in games in which agents occasionally receive opportunities to switch strategies, choosing between them using a probabilistic rule. Both the rate at which revision opportunities arrive and the probabilities with which each strategy is chosen are functions of current normalized payoffs. We call the aggregate dynamics induced by this model excess payoff dynamics. We show that every excess payoff dynamic is well-behaved: regardless of the underlying game, each excess payoff dynamic admits unique solution trajectories that vary continuously with the initial state, identifies rest points with Nash equilibria, and respects a basic payoff monotonicity property. We show how excess payoff dynamics can be used to construct well-behaved modifications of imitative dynamics, and relate them to two other well-behaved dynamics based on projections.     

Uniform payoff security and Nash equilibrium in compact games

We introduce a condition, uniform payoff security, for games with compact Hausdorff strategy spaces and payoffs bounded and measurable in players’ strategies. We show that if any such compact game G is uniformly payoff secure, then its mixed extension is payoff secure. We also establish that if a uniformly payoff secure compact game G has a mixed extension with reciprocally upper semicontinuous payoffs, then G has a Nash equilibrium in mixed strategies. We provide several economic examples of compact games satisfying uniform payoff security.     

The target projection dynamic

We study the target projection dynamic, a model of learning in normal form games. The dynamic is given a microeconomic foundation in terms of myopic optimization under control costs due to a certain status-quo bias. We establish a number of desirable properties of the dynamic: existence, uniqueness and continuity of solution trajectories, Nash stationarity, positive correlation with payoffs, and innovation. Sufficient conditions are provided under which strictly dominated strategies are wiped out. Finally, some stability results are provided for special classes of games.     

Equilibrium selection and the dynamic evolution of preferences

A population of fully rational agents plays a symmetric 2-player game in biological fitnesses, but each agent?s play is determined by his payoffs, which are free to evolve according to “survival of the fittest” pressures. An equilibrium-selection mechanism is assumed to exist, and deliver a unique outcome for any given profile of payoffs; this allows the evolution of payoffs to be modeled as a well-defined replicator dynamics. The existing static stability results that “efficient strict Nash implies stability” and “stability implies efficiency” are translated to this dynamic context, although the latter gives way to indeterminacy in the absence of a specific equilibrium-selection mechanism. A strong form of stability is established for the efficient outcome of games with common interests, whilst a weaker stability result is provided for efficient mixed-strategy equilibria of doubly symmetric games. The results are illustrated using the equilibrium-selection mechanism provided by global games.     

An existence result for discontinuous games

We introduce a notion of upper semicontinuity, weak upper semicontinuity, and show that it, together with a weak form of payoff security, is enough to guarantee the existence of Nash equilibria in compact, quasiconcave normal form games. We show that our result generalizes the pure strategy existence theorem of Dasgupta and Maskin [P. Dasgupta, E. Maskin, The existence of equilibrium in discontinuous economic games, I: Theory, Rev. Econ. Stud. 53 (1986) 1-26] and that it is neither implied nor does it imply the existence theorems of Baye, Tian, and Zhou [M. Baye, G. Tian, J. Zhou, Characterizations of the existence of equilibria in games with discontinuous and non-quasiconcave payoffs, Rev. Econ. Stud. 60 (1993) 935-948] and Reny [P. Reny, On the existence of pure and mixed strategy equilibria in discontinuous games, Econometrica 67 (1999) 1029-1056]. Furthermore, we show that an equilibrium may fail to exist when, while maintaining weak payoff security, weak upper semicontinuity is weakened to reciprocal upper semicontinuity.     

Between-game rule learning in dissimilar symmetric normal-form games

Rule learning posits that decision makers, rather than choosing over actions, choose over behavioral rules with different levels of sophistication. Rules are reinforced over time based on their historically observed payoffs in a given game. Past works on rule learning have shown that when playing a single game over a number of rounds, players can learn to form sophisticated beliefs about others. Here we are interested in learning that occurs between games where the set of actions is not directly comparable from one game to the next. We study a sequence of ten thrice-played dissimilar games. Using experimental data, we find that our rule learning model captures the ability of players to learn to reason across games. However, this learning appears different from within-game rule learning as previously documented. The main adjustment in sophistication occurs by switching from non-belief-based strategies to belief-based strategies. The sophistication of the beliefs themselves increases only slightly over time.     

Program equilibrium

In a computerized setting, players' strategies can be implemented by computer programs, to be executed on a shared computational devise. This situation becomes typical to new Internet economies, where agent technologies play a major role. This allows the definition of a program equilibrium. Following the fundamental ideas introduced by von Neumann in the 1940s (in parallel to his seminal contribution to game theory), a computer program can be used both as a set of instructions, as well as a file that can be read and compared with other files. We show that this idea implies that in a program equilibrium of the one-shot prisoners dilemma mutual cooperation is obtained. More generally, we show that the set of program equilibrium payoffs of a game coincides with the set of feasible and individually rational payoffs of it.     

Branching bandits: a sequential search process with correlated pay-offs

This paper presents a model of a sequential search process for the best outcome of many multi-stage projects. The branching structure of the search environment is such that the payoffs to various actions are correlated; nevertheless, it is shown that the optimal strategy is given by a simple reservation price rule.     

Efficiency and stability of sampling equilibrium in public goods games

Most models of social preferences and bounded rationality that are effective in explaining efficiency‐increasing departures from equilibrium behavior cannot easily account for similar deviations when they are efficiency‐reducing. We show that the notion of sampling equilibrium, subject to a suitable stability refinement, can account for behavior in both efficiency‐enhancing and efficiency‐reducing conditions. In particular, in public goods games with dominant strategy equilibria, stable sampling equilibrium can involve the play of dominated strategies with positive probability both when such behavior increases aggregate payoffs (relative to the standard prediction) and when it reduces aggregate payoffs. The dominant strategy equilibrium prediction changes abruptly from zero contribution to full contribution as a parameter crosses a threshold, whereas the stable sampling equilibrium remains fully mixed throughout. This is consistent with the available experimental evidence.