Learning in games with strategic complementarities revisited

Fictitious play is a classical learning process for games, and games with strategic complementarities are an important class including many economic applications. Knowledge about convergence properties of fictitious play in this class of games is scarce, however. Beyond games with a unique equilibrium, global convergence has only been claimed for games with diminishing returns [V. Krishna, Learning in games with strategic complementarities, HBS Working Paper 92-073, Harvard University, 1992]. This result remained unpublished, and it relies on a specific tie-breaking rule. Here we prove an extension of it by showing that the ordinal version of strategic complementarities suffices. The proof does not rely on tie-breaking rules and provides some intuition for the result.     

On the Rate of Convergence of Continuous-Time Fictitious Play

This paper shows, first, that continuous-time fictitious play converges (in both payoff and strategy terms) uniformly at ratet ^{− 1}in any finite two-person zero-sum game. The proof is, in essence, a simple Lyapunov-function argument. The convergence of discrete-time fictitious play is a straightforward corollary of this result. The paper also shows that continuous-time fictitious play converges in all finite weighted-potential games. In this case, the convergence is not uniform. It is conjectured, however, that any given continuous-time fictitious play of a finite weighted-potential game converges (in both payoff and strategy terms) at ratet ^{− 1}.Journal of Economic LiteratureClassification Numbers: C6, C7.     

Dynamics in near-potential games

We consider discrete-time learning dynamics in finite strategic form games, and show that games that are close to a potential game inherit many of the dynamical properties of potential games. We first study the evolution of the sequence of pure strategy profiles under better/best response dynamics. We show that this sequence converges to a (pure) approximate equilibrium set whose size is a function of the “distance” to a given nearby potential game. We then focus on logit response dynamics, and provide a characterization of the limiting outcome in terms of the distance of the game to a given potential game and the corresponding potential function. Finally, we turn attention to fictitious play, and establish that in near-potential games the sequence of empirical frequencies of player actions converges to a neighborhood of (mixed) equilibria, where the size of the neighborhood increases according to the distance to the set of potential games.     

A STATISTICAL THEORY OF EQUILIBRIUM IN GAMES*

This paper describes a statistical model of equiliobrium behaviour in games, which we call Quantal Response Equilibrium (QRE). The key feature of the equilibrium is that individuals do not always play responses to the strategies of their opponents, but play better strategies with higher probability than worse strategies. we illustrate several different applications of this approach, and establish a number of theoretical properties of this equilibrium concept. We also demonstrate an equililance between this equilibrium notion and Bayesian games derived from games of complete information with perturbed payoffs     

The rate of convergence of continuous fictitious play★

Summary The rate of convergence to Nash equilibrium of continuous fictitious play is determined for a generic set of utilities and initial beliefs in 2 x 2 games. In addition, an example is provided comparing the rate of convergence of discrete fictitious play to the rate for continuous fictitious play. Finally, the convergent dynamic of fictitious play is related to the nonconvergent gradient process dynamic in 2 x 2 games.I would like to thank Jim Jordan for many helpful discussions and for detailed comments on this paper. I also thank an anonymous referee for several helpful suggestions.     

The One-Shot-Deviation Principle for Sequential Rationality

This article analyzes the fictitious play process originally proposed for strategic form games by Brown (1951) and Robinson (1951). We interpret the process as a model of preplay thinking performed by players before acting in a one-shot game. This model is one of bounded rationality. We discuss how fictitious play should then be defined for extensive form games and conclude that this is somewhat problematic. We therefore study two alternative definitions. For either of these, under a weak condition of initial uncertainty, a convergence point of a fictitious play sequence is a sequential equilibrium. For generic games of perfect information initial uncertainty also implies convergence of fictitious play.Journal of Economic LiteratureClassification Number: C72.     

Fictitious Play in Extensive Form Games

This article analyzes the fictitious play process originally proposed for strategic form games by Brown (1951) and Robinson (1951). We interpret the process as a model of preplay thinking performed by players before acting in a one-shot game. This model is one of bounded rationality. We discuss how fictitious play should then be defined for extensive form games and conclude that this is somewhat problematic. We therefore study two alternative definitions. For either of these, under a weak condition of initial uncertainty, a convergence point of a fictitious play sequence is a sequential equilibrium. For generic games of perfect information initial uncertainty also implies convergence of fictitious play.Journal of Economic LiteratureClassification Number: C72.     

Two more classes of games with the continuous-time fictitious play property

Fictitious Play is the oldest and most studied learning process for games. Since the already classical result for zero-sum games, convergence of beliefs to the set of Nash equilibria has been established for several classes of games, including weighted potential games, supermodular games with diminishing returns, and 3×3 supermodular games. Extending these results, we establish convergence of Continuous-time Fictitious Play for ordinal potential games and quasi-supermodular games with diminishing returns. As a by-product we obtain convergence for 3×m and 4×4 quasi-supermodular games.     

Fictitious play in 2×n games

It is known that every discrete-time fictitious play process approaches equilibrium in nondegenerate 2×2 games, and that every continuous-time fictitious play process approaches equilibrium in nondegenerate 2×2 and 2×3 games. It has also been conjectured that convergence to the set of equilibria holds generally for nondegenerate 2×n games. We give a simple geometric proof of this for the continuous-time process, and also extend the result to discrete-time fictitious play.     

Brown's original fictitious play

What modern game theorists describe as “fictitious play” is not the learning process George W. Brown defined in his 1951 paper. Brown's original version differs in a subtle detail, namely the order of belief updating. In this note we revive Brown's original fictitious play process and demonstrate that this seemingly innocent detail allows for an extremely simple and intuitive proof of convergence in an interesting and large class of games: nondegenerate ordinal potential games.     

Stochastic uncoupled dynamics and Nash equilibrium

In this paper we consider dynamic processes, in repeated games, that are subject to the natural informational restriction of uncoupledness. We study the almost sure convergence of play (the period-by-period behavior as well as the long-run frequency) to Nash equilibria of the one-shot stage game, and present a number of possibility and impossibility results. Basically, we show that if in addition to random experimentation some recall, or memory, is introduced, then successful search procedures that are uncoupled can be devised. In particular, to get almost sure convergence to pure Nash equilibria when these exist, it suffices to recall the last two periods of play.     

Stochastic uncoupled dynamics and Nash equilibrium

In this paper we consider dynamic processes, in repeated games, that are subject to the natural informational restriction of uncoupledness. We study the almost sure convergence of play (the period-by-period behavior as well as the long-run frequency) to Nash equilibria of the one-shot stage game, and present a number of possibility and impossibility results. Basically, we show that if in addition to random experimentation some recall, or memory, is introduced, then successful search procedures that are uncoupled can be devised. In particular, to get almost sure convergence to pure Nash equilibria when these exist, it suffices to recall the last two periods of play.     

Dynamic legislative policy making

We prove existence of stationary Markov perfect equilibria in an infinite-horizon model of legislative policy making in which the policy outcome in one period determines the status quo for the next. We allow for a multidimensional policy space and arbitrary smooth stage utilities, and we assume preferences and the status quo are subject to arbitrarily small shocks. We prove that equilibrium continuation values are differentiable and that proposal strategies are continuous almost everywhere. We establish upper hemicontinuity of the equilibrium correspondence, and we provide weak conditions under which each equilibrium of our model determines an aperiodic transition probability over policies. We establish a convergence theorem giving conditions under which the invariant distributions generated by stationary equilibria must be close to the core in a canonical spatial model. Finally, we extend the analysis to sequential move stochastic games and to a version of the model in which the proposer and voting rule are determined by play of a finite, perfect information game.     

On Stackelberg competition in strategic multilateral exchange

In this paper, we consider a strategic equilibrium concept which extends Stackelberg competition to cover a general equilibrium framework. From the benchmark of strategic market games proposed by Sahi and Yao (1989), we define the notion of Stackelberg equilibrium. This concept captures strategic interactions in interrelated markets on which a finite number of leaders and followers compete on quantities. Within the framework of an example, convergence and welfare are studied. More specifically, we analyze convergence toward the competitive equilibrium and make welfare comparisons with other strategic equilibria.     

A Model of Behavior in Coordination Game Experiments

This paper constructs a structural model for behavior in expeiments where subjects play a simple coordination game repeatedly under a rotating partner scheme. The model assumes subjects' actions are stochastic best responses to beliefs about opponents' choices, and these beliefs update as subjects observe actual choices during the experiment. The model accounts for heterogeneity across subjects by regarding prior beliefs as random effects and estimating their distribution. Maximum likelihood estimates from experimental data suggest that distributions of initial beliefs vary across games, but in all games studied imply a convergence dynamic toward risk-dominant equilibrium.     

Variational convergence: Approximation and existence of equilibria in discontinuous games

We introduce a notion of variational convergence for sequences of games and we show that the Nash equilibrium map is upper semi-continuous with respect to variationally converging sequences. We then show that for a game G with discontinuous payoff, some of the most important existence results of Dasgupta and Maskin, Simon, and Reny are based on constructing approximating sequences of games that variationally converge to G. In fact, this notion of convergence will help simplify these results and make their proofs more transparent. Finally, we use our notion of convergence to establish the existence of a Nash equilibrium for Bertrand-Edgeworth games with very general forms of tie-breaking and residual demand rules.     

On the impossibility of achieving no regrets in repeated games

Regret-minimizing strategies for repeated games have been receiving increasing attention in the literature. These are simple adaptive behavior rules that lead to no regrets and, if followed by all players, exhibit nice convergence properties: the average play converges to correlated equilibrium, or even to Nash equilibrium in certain classes of games. However, the no-regret property relies on a strong assumption that each player treats her opponents as unresponsive and fully ignores the opponents’ possible reactions to her actions. We show that if at least one player is slightly responsive, it is impossible to achieve no regrets, and convergence results for regret minimization with responsive opponents are unknown.     

Did you really take a hit? Understanding how video games playing affects individuals

This study addresses the important and recurring question of whether playing video games is detrimental to the socio-economic development of a person. It does this by using novel data from the Taking Part Survey in England to establish whether games playing is associated with particular socio-economic characteristics and/or other forms of cultural participation. The results do not indicate any obviously negative effects of video games playing: rather, those who play are typically better educated and no less wealthier, and games players are also more likely than non-games players to participate in other forms of culture, particularly active forms of participation. These findings are reinforced when comparing the characteristics of individuals who did and did not play video games when younger.     

Group play in games and the role of consent in network formation

We study games played between groups of players, where a given group decides which strategy it will play through a vote by its members. When groups consist of two voting players, our games can also be interpreted as network-formation games. In experiments on Stag Hunt games, we find a stark contrast between how groups and individuals play, with payoffs playing a primary role in equilibrium selection when individuals play, but the structure of the voting rule playing the primary role when groups play. We develop a new solution concept, robust-belief equilibrium, which explains the data that we observe. We provide results showing that this solution concept has application beyond the particular games in our experiments.     

The supercore for normal-form games

This paper analyzes the supercore of a system derived from a normal-form game. For the case of a finite game with pure strategies, we define a sequence of games and show that the supercore coincides with the set of Nash equilibria of the last game in that sequence. This result is illustrated with the characterization of the supercore for the n-person prisoner's dilemma. With regard to the mixed extension of a normal-form game, we show that the set of Nash equilibrium profiles coincides with the supercore for games with a finite number of Nash equilibria.