Rage against the machines: how subjects play against learning algorithms

We use a large-scale internet experiment to explore how subjects learn to play against computers that are programmed to follow one of a number of standard learning algorithms. The learning theories are (unbeknown to subjects) a best response process, fictitious play, imitation, reinforcement learning, and a trial & error process. We explore how subjects’ performances depend on their opponents’ learning algorithm. Furthermore, we test whether subjects try to influence those algorithms to their advantage in a forward-looking way (strategic teaching). We find that strategic teaching occurs frequently and that all learning algorithms are subject to exploitation with the notable exception of imitation.     

An Asymptotic Solution for Sealed Bid Common-Value Auctions with Bidders Having Asymmetric Information

We investigate a first-price common-value auction where bidders have asymmetric information about an item of unknown value. We compute the unique Nash equilibrium when the bidders are constrained to translation-invariant bid functions. Further, this profile of bid functions is also an asymptotic Nash equilibrium (without the constraint on the bidders' strategies) as the a priori distribution of the true value becomes increasingly diffuse. All bidders have positive expected profits at equilibrium. In the second-price analogue with two bidders there is a continuum of Nash equilibria in which both bidders have positive expected profits. Journal of Economic Literature Classification Number: C7.     

Monotone imitation

We analyze the social learning process of a group of individuals who have limited information about the payoff distributions of each action. We say that a behavioral rule is first-order monotone (FOM) if the number of individuals who play actions with first-order stochastic dominant payoff distributions is expected to increase in any environment. We provide a characterization of FOM rules. Both Imitate if Better and Schlag’s (J Econ Theory 78:130–156, 1998) Proportional Imitation rule are FOM. No FOM rule is dominant in the sense of having the best performance in every environment.     

Coordination, “Magic,” and Reinforcement Learning in a Market Entry Game

Previous experimental studies have documented quick convergence to equilibrium play in market entry games with a large number of agents. The present study examines the effect of the available information in a 12-player game in an attempt to account for these findings. In line with the prediction of a simple reinforcement learning model (Roth and Erev, 1995,Games Econ. Behav.8, 164–212), quick convergence to equilibrium is observed even given minimal information (unknown payoff rule). However, in violation of the basic model, information concerning other players' payoff increases the number of entrants. The information effect can be described by a variant of the basic reinforcement learning model assuming that the additional information changes the player's reference point.Journal of Economic LiteratureClassification Number: C7, C92.     

A new unit root test based on F-statistic in ESTAR framework

This article proposes a new F-type unit test in the exponential smooth transition autoregressive framework. We derive the asymptotic nonstandard distribution of the proposed test and explore its finite sample properties; simulation results show our test has greater power than the t_kss test proposed by Kapetanios et al.(2003). Finally, an application on the real exchange rates further underpins its superiority.     

An experimental test of observational learning under imperfect information

Summary. Nearly all observational learning models assume that individuals can observe all the decisions that have previously been made. In reality, such perfect information is rarely available. To explore the difference between observational learning under perfect and imperfect information, this paper takes an experimental look at a situation in which individuals learn by observing the behavior of their immediate predecessors. Our experimental design uses the procedures of Çelen and Kariv [9] and is based on the theory of Çelen and Kariv [10]. We find that imitation is much less frequent when subjects have imperfect information, even less frequent than the theory predicts. Further, while we find strong evidence that under perfect information a form of generalized Bayesian behavior adequately explains behavior in the laboratory, under imperfect information behavior is not consistent even with this generalization of Bayesian behavior.Received: 29 January 2002, Revised: 12 May 2004, JEL Classification Numbers: C92, D8. Correspondence to: Boaçhan ÇelenWe completed most of this paper when we were both graduate students at New York University. This research was supported by the Center for Experimental Social Sciences (C.E.S.S.) and the C.V. Starr Center for Applied Economics at New York University. We are grateful to Andrew Schotter for his guidance and to an anonymous referee for his comments. We benefited from the expositional suggestions of William Baumol. We also acknowledge helpful discussions of Colin Camerer, Liran Einav, Xavier Gabaix, Douglas Gale, Charles Holt, David Laibson, and Matthew Rabin. We also benefited from suggestions by the participants of the 2002 International ESA Meeting and seminars at several universities.     

A Bayesian Model of Voting in Juries

We model voting in juries as a game of incomplete information, allowing jurors to receive a continuum of signals. We characterize the unique symmetric equilibrium of the game, and give a condition under which no asymmetric equilibria exist under unanimity rule. We offer a condition under which unanimity rule exhibits a bias toward convicting the innocent, regardless of the size of the jury, and give an example showing that this bias can be reversed. We prove a “jury theorem” for our general model: As the size of the jury increases, the probability of a mistaken judgment goes to zero for every voting rule except unanimity rule. For unanimity rule, the probability of making a mistake is bounded strictly above zero if and only if there do not exist arbitrarily strong signals of innocence. Our results explain the asymptotic inefficiency of unanimity rule in finite models and establishes the possibility of asymptotic efficiency, a property that could emerge only in a continuous model. Journal of Economic Literature Classification Numbers: C72, D72.     

Conditional Universal Consistency

Players choose an action before learning an outcome chosen according to an unknown and history-dependent stochastic rule. Procedures that categorize outcomes, and use a randomized variation on fictitious play within each category are studied. These procedures are “conditionally consistent:” they yield almost as high a time-average payoff as if the player knew the conditional distributions of actions given categories. Moreover, given any alternative procedure, there is a conditionally consistent procedure whose performance is no more than epsilon worse regardless of the discount factor. We also discuss cycles, and argue that the time-average of play should resemble a correlated equilibrium. Journal of Economic Literature Classification Numbers: C72, C73, C63, D83.     

Evolution of Beliefs and the Nash Equilibrium of Normal Form Games

The paper formulates a simple two-person model of learning with pattern recognition and discusses its implications. In particular, it focuses on the asymptotic behavior of players' beliefs when the game has a mixed-strategy Nash equilibrium.Journal of Economics LiteratureClassification Numbers: C72, D83.     

A Behavioral Learning Process in Games

This paper studies the cumulative proportional reinforcement (CPR) rule, according to which an agent plays, at each period, an action with a probability proportional to the cumulative utility that the agent has obtained with that action. The asymptotic properties of this learning process are examined for a decision-maker under risk, where it converges almost surely toward the expected utility maximizing action(s). The process is further considered in a two-player game; it converges with positive probability toward any strict pure Nash equilibrium and converges with zero probability toward some mixed equilibria (which are characterized). The CPR rule is compared in its principles with other reinforcement rules and with replicator dynamics. Journal of Economic Literature Classification Number: C72.     

Evolutionary Learning in Signalling Games

We study equilibrium selection by evolutionary learning in monotone two-type signalling games. The learning process we study extends that introduced by Young (1993, Econometrica61, 57–84) to deal with incomplete information and sequential moves; it thus involves stochastic trembles. For vanishing trembles the process gives rise to strong selection among sequential equilibria: if the game has separating equilibria, then in the long run only play according to the so-called Riley equilibrium will be observed frequently. Journal of Economic Literature Classification Number: C72.     

PAIRWISE DIFFERENCE ESTIMATION WITH NONPARAMETRIC CONTROL VARIABLES*

This article extends the pairwise difference estimators for various semilinear limited dependent variable models proposed by Honoré and Powell (Identification and Inference in Econometric Models. Essays in Honor of Thomas Rothenberg Cambridge: Cambridge University Press, 2005) to permit the regressor appearing in the nonparametric component to itself depend upon a conditional expectation that is nonparametrically estimated. This permits the estimation approach to be applied to nonlinear models with sample selectivity and/or endogeneity, in which a “control variable” for selectivity or endogeneity is nonparametrically estimated. We develop the relevant asymptotic theory for the proposed estimators and we illustrate the theory to derive the asymptotic distribution of the estimator for the partially linear logit model.     

Learning to Play Limited Forecast Equilibria

This paper provides a learning justification for limited forecast equilibria, i.e., strategy profiles such that (1) players choose their actions in order to maximize the discounted average payoff over their horizon of foresight as given by their forecasts and (2) forecasts are correct on and off the equilibrium path. The limited forecast equilibria appear to be the stochastically stable outcomes of a simple learning process involving (vanishing) trembles.Journal of Economic LiteratureClassification Numbers: C72, D83.     

TESTING FOR STATIONARITY IN HETEROGENEOUS PANEL DATA IN THE CASE OF MODEL MISSPECIFICATION

This paper investigates the performance of the tests proposed by Hadri and by Hadri and Larsson for testing for stationarity in heterogeneous panel data under model misspecification. The panel tests are based on the well known KPSS test (cf. Kwiatkowski et al.) which considers two models: stationarity around a deterministic level and stationarity around a deterministic trend. There is no study, as far as we know, on the statistical properties of the test when the wrong model is used. We also consider the case of the simultaneous presence of the two types of models in a panel. We employ two asymptotics: joint asymptotic, T, N →∞ simultaneously, and T fixed and N allowed to grow indefinitely. We use Monte Carlo experiments to investigate the effects of misspecification in sample sizes usually used in practice. The results indicate that the assumption that T is fixed rather than asymptotic leads to tests that have less size distortions, particularly for relatively small T with large N panels (micro‐panels) than the tests derived under the joint asymptotics. We also find that choosing a deterministic trend when a deterministic level is true does not significantly affect the properties of the test. But, choosing a deterministic level when a deterministic trend is true leads to extreme over‐rejections. Therefore, when unsure about which model has generated the data, it is suggested to use the model with a trend. We also propose a new statistic for testing for stationarity in mixed panel data where the mixture is known. The performance of this new test is very good for both cases of T asymptotic and T fixed. The statistic for T asymptotic is slightly undersized when T is very small (≤10).     

Learning in asymmetric duopoly markets: competition in information and market correlation

This paper explores experimentation and learning in asymmetric duopoly markets with product differentiation and demand uncertainty. We define the concepts of strategic substitutability and strategic complementarity in information and we show how both the mode of information competition and the transmission of information across markets affect duopoly experimentation. We relate information competition with market competition and we find that, when goods are substitutes and the correlation between market shocks is negative, firms will have a higher incentive to experiment in asymmetric markets than in symmetric ones. The opposite result follows when such correlation is positive. Also, when goods are complements the above findings are reversed.JEL Classification: D83, C72The authors thank partial financial support from the Spanish Ministry of Science and Technology under project B2000-1429, from the Spanish Ministry of Education and Science under project SEJ2004-07554 and from the “Generalitat Valénciana” under project GRUPOS04/13.     

Where are we in the political economy of reform?

We review the experiences of developing countries with market-oriented reforms, using the tools of modern political economy. We impose intellectual discipline by requiring that actors behave rationally using available information and that basic economic relationships such as budget constraints be accounted for. We attempt to integrate two approaches, one based on dynamic games played by interest groups, with one that focus on limited information and the dynamics of learning.

We describe the “starting point” as the set of “old” policies and we attempt to explain the dynamics (political, economic and informational) that lead to reform (section II). We analyze strategies for reformers subject to political constraints (section Ш). We evaluate the aggregate and distributional costs of reforms, emphasizing the importance of looking at the right counterfactuals (section IV).

We conclude by pointing to the challenges ahead: the second-stage institutional reforms necessary to take off from underdevelopment.     

How are preferences revealed?

Revealed preferences are tastes that rationalize an economic agent's observed actions. Normative preferences represent the agent's actual interests. It sometimes makes sense to assume that revealed preferences are identical to normative preferences. But there are many cases where this assumption is violated. We identify five factors that increase the likelihood of a disparity between revealed preferences and normative preferences: passive choice, complexity, limited personal experience, third-party marketing, and intertemporal choice. We then discuss six approaches that jointly contribute to the identification of normative preferences: structural estimation, active decisions, asymptotic choice, aggregated revealed preferences, reported preferences, and informed preferences. Each of these approaches uses consumer behavior to infer some property of normative preferences without equating revealed and normative preferences. We illustrate these issues with evidence from savings and investment outcomes.     

A Model of Efficient Discovery

We construct a model of costly pretrial discovery (represented as learning the drift rate of a diffusion process by observing its path) and settlement negotiations (analyzed as a mechanism design problem) in which the gains from settling are the avoided costs of a trial whose outcome is correlated with their private information. We show that for this particular model, the parties' expected gains from a joint plan of discovery preceding the settlement negotiations are unaffected by their privately known parameters; hence there is no intrinsic impediment to initial agreement on an efficient plan of discovery.Journal of Economic LiteratureClassification Nos.: C78, D74, D82, K41.     

An Experiment to Evaluate Bayesian Learning of Nash Equilibrium Play

We report on an experiment designed to evaluate the empirical implications of Jordan's model of Bayesian learning in games of incomplete information. A finite example is constructed in which the model generates unique predictions of subjects' choices in nearly all periods. When the “true” game defined by players' private information was one with a unique equilibrium in pure strategies, the experimental subjects' play converged to the equilibrium, as Jordan's theory predicts, even when the subjects had not attained complete information about one another. But when there were two pure strategy equilibria, the theory's predictions were not consistent with observed behavior. Journal of Economic Literature Classification numbers: D83, C72, C92.     

On the Irrelevance of Risk Attitudes in Repeated Two-Outcome Games

We study equilibrium and maximin play in supergames consisting of the sequential play of a finite collection of stage games, where each stage game has two outcomes for each player. We show that for two-player supergames in which each stage game is strictly competitive, in any Nash equilibrium of the supergame, play at each stage is a Nash equilibrium of the stage game provided preferences over certain supergame outcomes satisfy a natural monotonicity condition. In particular, equilibrium play does not depend on risk attitudes. We establish an invariance result for games with more than two players when the solution concept is subgame perfection. Journal of Economic Literature Classification Numbers: C72, C9.