Fictitious play in 2×n games

It is known that every discrete-time fictitious play process approaches equilibrium in nondegenerate 2×2 games, and that every continuous-time fictitious play process approaches equilibrium in nondegenerate 2×2 and 2×3 games. It has also been conjectured that convergence to the set of equilibria holds generally for nondegenerate 2×n games. We give a simple geometric proof of this for the continuous-time process, and also extend the result to discrete-time fictitious play.     

The One-Shot-Deviation Principle for Sequential Rationality

This article analyzes the fictitious play process originally proposed for strategic form games by Brown (1951) and Robinson (1951). We interpret the process as a model of preplay thinking performed by players before acting in a one-shot game. This model is one of bounded rationality. We discuss how fictitious play should then be defined for extensive form games and conclude that this is somewhat problematic. We therefore study two alternative definitions. For either of these, under a weak condition of initial uncertainty, a convergence point of a fictitious play sequence is a sequential equilibrium. For generic games of perfect information initial uncertainty also implies convergence of fictitious play.Journal of Economic LiteratureClassification Number: C72.     

Fictitious Play in Extensive Form Games

This article analyzes the fictitious play process originally proposed for strategic form games by Brown (1951) and Robinson (1951). We interpret the process as a model of preplay thinking performed by players before acting in a one-shot game. This model is one of bounded rationality. We discuss how fictitious play should then be defined for extensive form games and conclude that this is somewhat problematic. We therefore study two alternative definitions. For either of these, under a weak condition of initial uncertainty, a convergence point of a fictitious play sequence is a sequential equilibrium. For generic games of perfect information initial uncertainty also implies convergence of fictitious play.Journal of Economic LiteratureClassification Number: C72.     

Generalised weakened fictitious play

A general class of adaptive processes in games is developed, which significantly generalises weakened fictitious play [Van der Genugten, B., 2000. A weakened form of fictitious play in two-person zero-sum games. Int. Game Theory Rev. 2, 307–328] and includes several interesting fictitious-play-like processes as special cases. The general model is rigorously analysed using the best response differential inclusion, and shown to converge in games with the fictitious play property. Furthermore, a new actor–critic process is introduced, in which the only information given to a player is the reward received as a result of selecting an action—a player need not even know they are playing a game. It is shown that this results in a generalised weakened fictitious play process, and can therefore be considered as a first step towards explaining how players might learn to play Nash equilibrium strategies without having any knowledge of the game, or even that they are playing a game.     

The rate of convergence of continuous fictitious play★

Summary The rate of convergence to Nash equilibrium of continuous fictitious play is determined for a generic set of utilities and initial beliefs in 2 x 2 games. In addition, an example is provided comparing the rate of convergence of discrete fictitious play to the rate for continuous fictitious play. Finally, the convergent dynamic of fictitious play is related to the nonconvergent gradient process dynamic in 2 x 2 games.I would like to thank Jim Jordan for many helpful discussions and for detailed comments on this paper. I also thank an anonymous referee for several helpful suggestions.     

Learning in games with strategic complementarities revisited

Fictitious play is a classical learning process for games, and games with strategic complementarities are an important class including many economic applications. Knowledge about convergence properties of fictitious play in this class of games is scarce, however. Beyond games with a unique equilibrium, global convergence has only been claimed for games with diminishing returns [V. Krishna, Learning in games with strategic complementarities, HBS Working Paper 92-073, Harvard University, 1992]. This result remained unpublished, and it relies on a specific tie-breaking rule. Here we prove an extension of it by showing that the ordinal version of strategic complementarities suffices. The proof does not rely on tie-breaking rules and provides some intuition for the result.     

Evolution in games with randomly disturbed payoffs

We consider a simple model of stochastic evolution in population games. In our model, each agent occasionally receives opportunities to update his choice of strategy. When such an opportunity arises, the agent selects a strategy that is currently optimal, but only after his payoffs have been randomly perturbed. We prove that the resulting evolutionary process converges to approximate Nash equilibrium in both the medium run and the long run in three general classes of population games: stable games, potential games, and supermodular games. We conclude by contrasting the evolutionary process studied here with stochastic fictitious play.     

Fictitious play in ‘one-against-all’ multi-player games

Summary. A compound game is an (n + 1) player game based on n two-person subgames. In each of these subgames player 0 plays against one of the other players. Player 0 is regulated, so that he must choose the same strategy in all n subgames. We show that every fictitious play process approaches the set of equilibria in compound games for which all subgames are either zero-sum games, potential games, or games. Received: July 18, 1997; revised version: December 4, 1998     

The geometry of inductive reasoning in games

Summary.  This paper contributes to the recent focus on dynamics in noncooperative games when players use inductive learning. The most well-known inductive learning rule, Brown’s fictitious play, is known to converge for games, yet many examples exist where fictitious play reasoning fails to converge to a Nash equilibrium. Building on ideas from chaotic dynamics, this paper develops a geometric conceptualization of instability in games, allowing for a reinterpretation of existing results and suggesting avenues for new results. Received: October 27, 1995 revised version May 2, 1996     

Learning in games with unstable equilibria

We propose a new concept for the analysis of games, the TASP, which gives a precise prediction about non-equilibrium play in games whose Nash equilibria are mixed and are unstable under fictitious play-like learning. We show that, when players learn using weighted stochastic fictitious play and so place greater weight on recent experience, the time average of play often converges in these “unstable” games, even while mixed strategies and beliefs continue to cycle. This time average, the TASP, is related to the cycle identified by Shapley [L.S. Shapley, Some topics in two person games, in: M. Dresher, et al. (Eds.), Advances in Game Theory, Princeton University Press, Princeton, 1964]. The TASP can be close to or quite distinct from Nash equilibrium.     

Self-tuning experience weighted attraction learning in games

Self-tuning experience weighted attraction (EWA) is a one-parameter theory of learning in games. It addresses a criticism that an earlier model (EWA) has too many parameters, by fixing some parameters at plausible values and replacing others with functions of experience so that they no longer need to be estimated. Consequently, it is econometrically simpler than the popular weighted fictitious play and reinforcement learning models. The functions of experience which replace free parameters “self-tune” over time, adjusting in a way that selects a sensible learning rule to capture subjects’ choice dynamics. For instance, the self-tuning EWA model can turn from a weighted fictitious play into an averaging reinforcement learning as subjects equilibrate and learn to ignore inferior foregone payoffs. The theory was tested on seven different games, and compared to the earlier parametric EWA model and a one-parameter stochastic equilibrium theory (QRE). Self-tuning EWA does as well as EWA in predicting behavior in new games, even though it has fewer parameters, and fits reliably better than the QRE equilibrium benchmark.     

Learning from Personal Experience: One Rational Guy and the Justification of Myopia

The paper examines a large population analog of fictitious play in which players learn from personal experience, focusing on what happens when a single rational player is added to the population. Because the learning process naturally generates contagion dynamics, the rational player at times has an incentive to act nonmyopically. In 2 × 2 games the dynamics are asymmetric and favor risk dominant equilibria. A variety of other examples are presented.Journal of Economic LiteratureClassification Number: C7.     

On the Rate of Convergence of Continuous-Time Fictitious Play

This paper shows, first, that continuous-time fictitious play converges (in both payoff and strategy terms) uniformly at ratet ^{− 1}in any finite two-person zero-sum game. The proof is, in essence, a simple Lyapunov-function argument. The convergence of discrete-time fictitious play is a straightforward corollary of this result. The paper also shows that continuous-time fictitious play converges in all finite weighted-potential games. In this case, the convergence is not uniform. It is conjectured, however, that any given continuous-time fictitious play of a finite weighted-potential game converges (in both payoff and strategy terms) at ratet ^{− 1}.Journal of Economic LiteratureClassification Numbers: C6, C7.     

Fictitious Play, Shapley Polygons, and the Replicator Equation

For many normal form games, the limiting behavior of fictitious play and the time-averaged replicator dynamics coincide. In particular, we show this for three examples, where this limit is not a Nash equilibrium, but a Shapley polygon. Journal of Economic Literature Classification Numbers: C72, C73.     

Coordination between a sophisticated and fictitious player

Successful coordination is a common and important social problem. Achieving it relies on the players’ ability to accurately anticipate future choices from known information. Individuals may not only lack this cognitive ability, but differ in it. Fictitious Play is an adaptive behavior where a myopic best response to the historical play of an opponent is selected. I consider the interaction between a fictitious player and a sophisticated player in 2 × 2 coordination games. The existence of coordination, the selection of equilibria, and the payoffs generated are analyzed.     

Learning Dynamics in Games with Stochastic Perturbations

Consider a generalization of fictitious play in which agents′ choices are perturbed by incomplete information about what the other side has done, variability in their payoffs, and unexplained trembles. These perturbed best reply dynamics define a nonstationary Markov process on an infinite state space. It is shown, using results from stochastic approximation theory, that for 2 × 2 games it converges almost surely to a point that lies close to a stable Nash equilibrium, whether pure or mixed. This generalizes a result of Fudenherg and Kreps, who demonstrate convergence when the game has a unique mixed equilibrium. Journal of Economic Literature Classification Numbers: 000, 000, 000.     

Dynamics in near-potential games

We consider discrete-time learning dynamics in finite strategic form games, and show that games that are close to a potential game inherit many of the dynamical properties of potential games. We first study the evolution of the sequence of pure strategy profiles under better/best response dynamics. We show that this sequence converges to a (pure) approximate equilibrium set whose size is a function of the “distance” to a given nearby potential game. We then focus on logit response dynamics, and provide a characterization of the limiting outcome in terms of the distance of the game to a given potential game and the corresponding potential function. Finally, we turn attention to fictitious play, and establish that in near-potential games the sequence of empirical frequencies of player actions converges to a neighborhood of (mixed) equilibria, where the size of the neighborhood increases according to the distance to the set of potential games.     

Basins of attraction and equilibrium selection under different learning rules

A deterministic learning model applied to a game with multiple equilibria produces distinct basins of attraction for those equilibria. In symmetric two-by-two games, basins of attraction are invariant to a wide range of learning rules including best response dynamics, replicator dynamics, and fictitious play. In this paper, we construct a class of three-by-three symmetric games for which the overlap in the basins of attraction under best response learning and replicator dynamics is arbitrarily small. We then derive necessary and sufficient conditions on payoffs for these two learning rules to create basins of attraction with vanishing overlap. The necessary condition requires that with probability one the initial best response is not an equilibrium to the game. The existence of parasitic or misleading actions allows subtle differences in the learning rules to accumulate.     

Fictitious play in 2×2 games: A geometric proof of convergence

Summary This paper provides a new proof of Miyasawa's (1961) result showing the convergence of fictitious play in 2×2 games. The novelty of the approach used here is that it rests entirely on the geometric properties of the best-response correspondence. The geometric approach greatly shortens the exposition, and it suggests some possible extensions to more difficult convergence conjectures.We thank Glenn Ellison, Vijay Krishna, Eric Maskin and an anonymous referee for helpful comments. This paper was written while Metrick was supported by an NSF Graduate Fellowship and Polak was supported by a Junior Fellowship in the Society of Fellows at Harvard University.     

Regret-based continuous-time dynamics

Regret-based dynamics have been introduced and studied in the context of discrete-time repeated play. Here we carry out the corresponding analysis in continuous time. We observe that, in contrast to (smooth) fictitious play or to evolutionary models, the appropriate state space for this analysis is the space of distributions on the product of the players' pure action spaces (rather than the product of their mixed action spaces). We obtain relatively simple proofs for some results known in the discrete case (related to ‘no-regret’ and correlated equilibria), and also a new result on two-person potential games (for this result we also provide a discrete-time proof).