Learning in games with strategic complementarities revisited

Fictitious play is a classical learning process for games, and games with strategic complementarities are an important class including many economic applications. Knowledge about convergence properties of fictitious play in this class of games is scarce, however. Beyond games with a unique equilibrium, global convergence has only been claimed for games with diminishing returns [V. Krishna, Learning in games with strategic complementarities, HBS Working Paper 92-073, Harvard University, 1992]. This result remained unpublished, and it relies on a specific tie-breaking rule. Here we prove an extension of it by showing that the ordinal version of strategic complementarities suffices. The proof does not rely on tie-breaking rules and provides some intuition for the result.     

Fictitious play in 2×n games

It is known that every discrete-time fictitious play process approaches equilibrium in nondegenerate 2×2 games, and that every continuous-time fictitious play process approaches equilibrium in nondegenerate 2×2 and 2×3 games. It has also been conjectured that convergence to the set of equilibria holds generally for nondegenerate 2×n games. We give a simple geometric proof of this for the continuous-time process, and also extend the result to discrete-time fictitious play.     

Perturbed communication games with honest senders and naive receivers

This paper studies communication games in which the sender is possibly honest (tells the truth) and the receiver is possibly naive (follows messages as if truthful). The characterization of message-monotone equilibria in the perturbed games explain several important aspects of strategic communication including sender exaggeration, receiver skepticism and message clustering. Surprisingly, the strategic receiver may respond to more aggressive claims with more moderate actions. In the limit as the probabilities of the non-strategic players approach zero, (i) the limit equilibrium corresponds to a most-informative equilibrium of the limit (Crawford-Sobel) game; (ii) only the top messages are sent.     

Learning in games with unstable equilibria

We propose a new concept for the analysis of games, the TASP, which gives a precise prediction about non-equilibrium play in games whose Nash equilibria are mixed and are unstable under fictitious play-like learning. We show that, when players learn using weighted stochastic fictitious play and so place greater weight on recent experience, the time average of play often converges in these “unstable” games, even while mixed strategies and beliefs continue to cycle. This time average, the TASP, is related to the cycle identified by Shapley [L.S. Shapley, Some topics in two person games, in: M. Dresher, et al. (Eds.), Advances in Game Theory, Princeton University Press, Princeton, 1964]. The TASP can be close to or quite distinct from Nash equilibrium.     

Value of public information in sender-receiver games

I find in two classes of sender-receiver games that the receiver’s equilibrium payoff is not increasing in the informativeness of a public signal because the sender may transmit less information when the public signal is more informative.     

Hope springs eternal: learning and the stability of cooperation in short horizon repeated games

This paper considers learning rates in finitely repeated prisoners’ dilemmas. If players think their opponents might be relatively cooperative (e.g., tit-for-tat or grim types), they will cooperate in finitely repeated prisoners’ dilemmas (see Kreps et al., J. Econom. Theory 27 (1982) 245). However, if there are actually no cooperative types, players will eventually learn this and cooperation will break down. This paper shows that this learning is extremely slow, so it will take an extremely long time for cooperation to break down.Thus, suppose the world is either “good” or “bad.” The probability of a grim type is δ>0 if the world is good, and zero if the world is bad. Successive generations pair up to play finitely repeated prisoners’ dilemmas. Players observe play in previous generations and use Bayes’ rule to update their prior, π, that the world is good. We show that, if the world is really bad, then π falls per generation on average. Thus, if δ is small, there is less cooperation if the world is good, but cooperation may become more stable. For a representative 19 period repeated prisoners’ dilemma, beliefs fall one percentage point on average after a thousand generations.To derive these learning rates, we must refine existing results on the sensitivity of repeated games to Kreps et al. (1982) type perturbations. Specifically, we show cooperation is possible in perturbed prisoners’ dilemmas repeated O(log(1/δ)) times. This improves significantly on the O(1/δ) results in previous work. The paper thus provides two new reasons why cooperation tends to be stable, even in short horizon repeated games.     

Search and Knightian uncertainty

Suppose that “uncertainty” about labor market conditions has increased. Does this change induce an unemployed worker to search longer, or shorter? This paper shows that the answer is drastically different depending on whether an increase in “uncertainty” is an increase in risk or that in true uncertainty in the sense of Frank Knight. We show in a general framework that, while an increase in risk (the mean-preserving spread of the wage distribution that the worker thinks she faces) increases the reservation wage, an increase in Knightian uncertainty (a decrease in her confidence about the wage distribution) reduces it.     

Coordination failure in repeated games with private monitoring

Players coordinate continuation play in repeated games with public monitoring. We investigate the robustness of such equilibrium behavior with respect to ex-ante small private-monitoring perturbations. We show that with full support of public signals, no perfect public equilibrium is robust if it induces a “regular” 2×2$2\times 2$ coordination game in the continuation play. This regularity condition is violated in all belief-free equilibria. Indeed, with an individual full rank condition, every interior belief-free equilibrium is robust. We also analyze block belief-free equilibria and point out that the notion of robustness is sensitive to whether we allow for uninterpretable signals.     

Between-game rule learning in dissimilar symmetric normal-form games

Rule learning posits that decision makers, rather than choosing over actions, choose over behavioral rules with different levels of sophistication. Rules are reinforced over time based on their historically observed payoffs in a given game. Past works on rule learning have shown that when playing a single game over a number of rounds, players can learn to form sophisticated beliefs about others. Here we are interested in learning that occurs between games where the set of actions is not directly comparable from one game to the next. We study a sequence of ten thrice-played dissimilar games. Using experimental data, we find that our rule learning model captures the ability of players to learn to reason across games. However, this learning appears different from within-game rule learning as previously documented. The main adjustment in sophistication occurs by switching from non-belief-based strategies to belief-based strategies. The sophistication of the beliefs themselves increases only slightly over time.     

Adaptive learning in extensive form games and sequential equilibrium

Summary. This paper studies adaptive learning in extensive form games and provides conditions for convergence points of adaptive learning to be sequential equilibria. Precisely, we present a set of conditions on learning sequences such that an assessment is a sequential equilibrium if and only if there is a learning sequence fulfilling the conditions, which leads to the assessment. Received: November 5, 1996; revised version: May 28, 1997     

The geometry of inductive reasoning in games

Summary.  This paper contributes to the recent focus on dynamics in noncooperative games when players use inductive learning. The most well-known inductive learning rule, Brown’s fictitious play, is known to converge for games, yet many examples exist where fictitious play reasoning fails to converge to a Nash equilibrium. Building on ideas from chaotic dynamics, this paper develops a geometric conceptualization of instability in games, allowing for a reinterpretation of existing results and suggesting avenues for new results. Received: October 27, 1995 revised version May 2, 1996     

Testing the TASP: An experimental investigation of learning in games with unstable equilibria

We report experiments studying mixed strategy Nash equilibria that are theoretically stable or unstable under learning. The Time Average Shapley Polygon (TASP) predicts behavior in the unstable case. We study two versions of Rock-Paper-Scissors that include a fourth strategy, Dumb. The unique Nash equilibrium is identical in the two games, but the predicted frequency of Dumb is much higher in the game where the NE is stable. Consistent with TASP, the observed frequency of Dumb is lower and play is further from Nash in the high payoff unstable treatment. However, Dumb is played too frequently in all treatments.     

Contagion and efficiency

We consider a population of agents, either finite or countably infinite, located on an arbitrary network. Agents interact directly only with their immediate neighbors, but are able to observe the behavior of (some) other agents beyond their interaction neighborhood, and learn from that behavior by imitating successful actions. If interactions are not “too global” but information is fluid enough, we show that the efficient action is the only one which can spread contagiously to the whole population from an initially small, finite subgroup. This result holds even in the presence of an alternative, -dominant action.     

When mandatory disclosure hurts: Expert advice and conflicting interests

We study the quality of advice that an informed and biased expert gives to an uninformed decision maker. We compare two scenarios: mandatory disclosure of the bias and nondisclosure, where information about the bias can only be revealed through cheap-talk. We find that in many scenarios nondisclosure allows for higher welfare for both parties. Hiding the bias allows for more precise communication for the more biased type and, if different types are biased in different directions, may allow for the same for the less biased type. We identify contexts where equilibrium revelation allows but mandatory disclosure prevents meaningful communication.     

Robustness and ambiguity in continuous time

We use statistical detection theory in a continuous-time environment to provide a new perspective on calibrating a concern about robustness or an aversion to ambiguity. A decision maker repeatedly confronts uncertainty about state transition dynamics and a prior distribution over unobserved states or parameters. Two continuous-time formulations are counterparts of two discrete-time recursive specifications of Hansen and Sargent (2007) [16]. One formulation shares features of the smooth ambiguity model of Klibanoff et al. (2005) and (2009) [24] and [25]. Here our statistical detection calculations guide how to adjust contributions to entropy coming from hidden states as we take a continuous-time limit.     

Long persuasion games

This paper characterizes geometrically the sets of all Nash and perfect Bayesian equilibrium payoffs achievable with unmediated communication in persuasion games, i.e., games with an informed expert and an uninformed decisionmaker in which the expert's information is certifiable. The first equilibrium characterization is provided for unilateral persuasion games, and the second for multistage, bilateral persuasion games. As in Aumann and Hart [R.J. Aumann, S. Hart, Long cheap talk, Econometrica 71 (6) (2003) 1619-1660], we use the concepts of diconvexification and dimartingale. A leading example illustrates both geometric characterizations and shows how the expert, whatever his type, can increase his equilibrium payoff compared to all equilibria of the unilateral persuasion game by delaying information certification.     

Time-consistent policies

In many cases the optimal open-loop policy to influence agents who solve dynamic problems is time inconsistent. We show how to construct a time-consistent open-loop policy rule. We also consider an additional restriction under which the time-consistent open-loop policy is stationary. We use examples to illustrate the properties of these tax rules.     

On the convergence of reinforcement learning

This paper examines the convergence of payoffs and strategies in Erev and Roth's model of reinforcement learning. When all players use this rule it eliminates iteratively dominated strategies and in two-person constant-sum games average payoffs converge to the value of the game. Strategies converge in constant-sum games with unique equilibria if they are pure or if they are mixed and the game is 2×2. The long-run behaviour of the learning rule is governed by equations related to Maynard Smith's version of the replicator dynamic. Properties of the learning rule against general opponents are also studied.     

Iterated potential and robustness of equilibria

For any given set-valued solution concept, it is possible to consider iterative elimination of actions outside the solution set. This paper applies such a procedure to define the concept of iterated monotone potential maximizer (iterated MP-maximizer). It is shown that under some monotonicity conditions, an iterated MP-maximizer is robust to incomplete information [A. Kajii, S. Morris, The robustness of equilibria to incomplete information, Econometrica 65 (1997) 1283-1309] and absorbing and globally accessible under perfect foresight dynamics for a small friction [A. Matsui, K. Matsuyama, An approach to equilibrium selection, J. Econ. Theory 65 (1995) 415-434]. Several simple sufficient conditions under which a game has an iterated MP-maximizer are also provided.