Comparative Learning Dynamics*

We study economic environments in which agents make choices on the basis of relative performance criteria and call the associated class of dynamic adjustment rules comparative dynamics. We distinguish two classes of learning behavior: learning from the population experience (imitative dynamics) and learning only from one's own experience (introspective dynamics). Paradoxically, for a broad class of models, comparing stochastically stable states across dynamics, agent payoffs are lower for imitative than introspective dynamics—mimicking best practice in the population is counterproductive.     

Optimal search,learning and implementation

We characterize the incentive compatible, constrained efficient policy (“second-best”) in a dynamic matching environment, where impatient, privately informed agents arrive over time, and where the designer gradually learns about the distribution of agents? values. We also derive conditions on the learning process ensuring that the complete-information, dynamically efficient allocation of resources (“first-best”) is incentive compatible. Our analysis reveals and exploits close, formal relations between the problem of ensuring implementable allocation rules in our dynamic allocation problems with incomplete information and learning, and between the classical problem, posed by Rothschild (1974) [20], of finding optimal stopping policies for search that are characterized by a reservation price property.     

A Note on Best Response Dynamics

We investigate the relationship between the continuous time best response dynamic, its perturbed version, and evolutionary dynamics in relation to mixed strategy equilibria. We find that as the level of noise approaches zero, the perturbed best response dynamic has the same qualitative properties as a broad class of evolutionary dynamics. That is, stability properties of equilibria are robust across learning dynamics of quite different origins and motivations. Journal of Economic Literature Classification Numbers: C72, D83.     

What do you want to know? Information acquisition and learning in experimental Cournot games

This paper presents an experiment designed to study firms’ behavior and market dynamics, when information about the market structure and opponents’ actions is difficult to acquire and process. Our experimental software controls the information acquisition process of the subjects. Paying attention not only to what players do, but also to what they know, it is possible to better understand the cognitive processes guiding their choices and, consequently, the impact of the informational structure on their behavior. According to our results, Best-Response Dynamics is the main component of subjects’ learning process. We also find that, when subjects look at the strategies individually adopted by their competitors, they tend to imitate the most successful behavior, which drives the market towards a more competitive outcome.     

Melioration learning in games with constant and frequency-dependent pay-offs

The paper explores the implications of melioration learning—an empirically significant variant of reinforcement learning—for game theory. We show that in games with invariable pay-offs melioration learning converges to Nash equilibria in a way similar to the replicator dynamics. Since melioration learning is known to deviate from optimizing behavior when an action’s rewards decrease with increasing relative frequency of that action, we also investigate an example of a game with frequency-dependent pay-offs. Interactive melioration learning is then still appropriately described by the replicator dynamics, but it indeed deviates from rational choice behavior in such a game.     

Excess payoff dynamics and other well-behaved evolutionary dynamics

We consider a model of evolution in games in which agents occasionally receive opportunities to switch strategies, choosing between them using a probabilistic rule. Both the rate at which revision opportunities arrive and the probabilities with which each strategy is chosen are functions of current normalized payoffs. We call the aggregate dynamics induced by this model excess payoff dynamics. We show that every excess payoff dynamic is well-behaved: regardless of the underlying game, each excess payoff dynamic admits unique solution trajectories that vary continuously with the initial state, identifies rest points with Nash equilibria, and respects a basic payoff monotonicity property. We show how excess payoff dynamics can be used to construct well-behaved modifications of imitative dynamics, and relate them to two other well-behaved dynamics based on projections.     

Uncertainty, co-ordination and path dependence

Our objective is to assess whether dynamics hinder or assist co-ordination in a game with strategic complementarities. We study two dynamic aspects: different agents make decisions at different points in time; and extra information about a payoff-relevant state of nature becomes available over time. We find that the dynamic resolution of information matters most for uniqueness of equilibrium. This is demonstrated by showing that the condition for uniqueness is weaker when learning occurs. We also analyse how successfully agents co-ordinate when there is a unique equilibrium. Finally, we show that path dependence occurs: the order in which signals arrive matters, as well as the total amount of information received.     

Transient and asymptotic dynamics of reinforcement learning in games

Reinforcement learners tend to repeat actions that led to satisfactory outcomes in the past, and avoid choices that resulted in unsatisfactory experiences. This behavior is one of the most widespread adaptation mechanisms in nature. In this paper we fully characterize the dynamics of one of the best known stochastic models of reinforcement learning [Bush, R., Mosteller, F., 1955. Stochastic Models of Learning. Wiley & Sons, New York] for 2-player 2-strategy games. We also provide some extensions for more general games and for a wider class of learning algorithms. Specifically, it is shown that the transient dynamics of Bush and Mosteller's model can be substantially different from its asymptotic behavior. It is also demonstrated that in general—and in sharp contrast to other reinforcement learning models in the literature—the asymptotic dynamics of Bush and Mosteller's model cannot be approximated using the continuous time limit version of its expected motion.     

Adaptive behavior,demand and preferences

This paper extends the work on endogenous change of tastes of Von Weizsäcker to the n-commodity framework and for a general adaptive behavior process. The paper examines the relation between the effect of taste changes to income and price changes. It provides sufficient conditions for stability of the underlying dynamic process, establishes uniqueness of the equilibrium demand vector and some useful relations between the long-run demand functions and the equilibrium short-run demand functions. It is also shown that the long-run demand functions can be rationalized by a utility function if and only if the short-run utility function is such that any good that experiences learning or taste change is separable from all other goods.     

Learning by replicator and best-response: the importance of being indifferent

This paper compares two learning processes, namely those generated by replicator and best-response dynamics, from the point of view of the asymptotics of play. We base our study on the intersection of the basins of attraction of locally stable pure Nash equilibria for replicator and best-response dynamics. Local stability implies that the basin of attraction has positive measure but there are examples where the intersection of the basin of attraction for replicator and best-response dynamics is arbitrarily small. We provide conditions, involving the existence of an unstable interior Nash equilibrium, for the basins of attraction of any locally stable pure Nash equilibrium under replicator and best-response dynamics to intersect in a set of positive measure. Hence, for any choice of initial conditions in sets of positive measure, if a pure Nash equilibrium is locally stable, the outcome of learning under either procedure coincides. We provide examples illustrating the above, including some for which the basins of attraction exactly coincide for both learning dynamics. We explore the role that indifference sets play in the coincidence of the basins of attraction of the stable Nash equilibria.     

结构嵌入、动态能力与组织绩效:一个混合模型

供应链企业对外部关系网络的结构性嵌入，有利于其获取更多异质性资源，这为组织绩效提升奠定了基础。在构建结构嵌入、动态能力与组织绩效关系混合模型的基础上，实证检验了结构嵌入对组织绩效的影响。研究结果表明：结构嵌入与组织绩效间存在显著的倒“U”型关系，即结构嵌入具有“拐点”效应，过高水平的结构嵌入不利于组织绩效提升；环境动态性在结构嵌入对组织绩效的影响中发挥有中介的调节作用；学习能力和创新能力在结构嵌入对组织绩效的影响中存在有调节的中介效应。     

Transient and asymptotic dynamics of reinforcement learning in games

Reinforcement learners tend to repeat actions that led to satisfactory outcomes in the past, and avoid choices that resulted in unsatisfactory experiences. This behavior is one of the most widespread adaptation mechanisms in nature. In this paper we fully characterize the dynamics of one of the best known stochastic models of reinforcement learning [Bush, R., Mosteller, F., 1955. Stochastic Models of Learning. Wiley & Sons, New York] for 2-player 2-strategy games. We also provide some extensions for more general games and for a wider class of learning algorithms. Specifically, it is shown that the transient dynamics of Bush and Mosteller's model can be substantially different from its asymptotic behavior. It is also demonstrated that in general—and in sharp contrast to other reinforcement learning models in the literature—the asymptotic dynamics of Bush and Mosteller's model cannot be approximated using the continuous time limit version of its expected motion.     

Preference Evolution,Two-Speed Dynamics,and Rapid Social Change

We present a dynamic analysis of the evolution of preferences in a strategic environment. In our model, each player's behavior depends on both the game's payoffs and his idiosyncratic biases, but only the game's payoffs determine his evolutionary success. Dynamics run at two speeds at once; while natural selection slowly reshapes the distribution of preferences, players quickly learn to behave as their preferences dictate. We establish the existence and uniqueness of the paired trajectories of society's preferences and behavior. While aggregate behavior adjusts smoothly in equilibration games, in coordination games aggregate behavior can jump discretely in an instant of evolutionary time. Journal of Economic Literature Classification Numbers: C72, C73.     

On the Dynamic Foundation of Evolutionary Stability in Continuous Models

We show in this paper that none of the existing static evolutionary stability concepts (ESS, CSS, uninvadability, NIS) is sufficient to guarantee dynamic stability in the weak topology with respect to standard evolutionary dynamics if the strategy space is continuous. We propose a new concept, evolutionary robustness, which is stronger than the previous concepts. Evolutionary robustness ensures dynamic stability for replicator dynamics in doubly symmetric games. Journal of Economic Literature Classification Numbers: C70, 72.     

Technological learning,technological substitution,and technological change

From a simple dynamic model of competition between product lines it is shown that the shape of learning curves has a powerful influence on the dynamics of technological substitution. Learning of both production efficiency and marketing efficiency is considered. It is asserted that both types of learning are important and that the two are complementary. It is further speculated that production learning is probably more important for commodities and in situations of low per capita income, whereas market learning gains ascendancy in cases of high income and specialized and diversified product lines. In closing, it is noted that simple competitive models are misleading, first because complementarities and coevolutionary processes are probably as important in the overall development of technology as are competitive processes, and second because optimization of the technological system's parts does not guarantee improvement of the performance of the system as a whole.     

What to maximize if you must

The assumption that decision makers choose actions to maximize their preferences is a central tenet in economics, often justified formally or informally by appealing to evolutionary arguments. In contrast, we show that in almost every game and for almost every family of distortions of a player's actual payoffs, some degree of this distortion is beneficial to the player, and will not be driven out by any evolutionary process involving payoff-monotonic selection dynamics. Consequently, under any such selection dynamics the population will not converge to payoff-maximizing behavior. We also show that payoff-maximizing behavior need not prevail when preferences are imperfectly observed.     

The projection dynamic and the replicator dynamic

We investigate a variety of connections between the projection dynamic and the replicator dynamic. At interior population states, the standard microfoundations for the replicator dynamic can be converted into foundations for the projection dynamic by replacing imitation of opponents with “revision driven by insecurity” and direct choice of alternative strategies. Both dynamics satisfy a condition called inflow–outflow symmetry, which causes them to select against strictly dominated strategies at interior states; still, because it is discontinuous at the boundary of the state space, the projection dynamic allows strictly dominated strategies to survive in perpetuity. The two dynamics exhibit qualitatively similar behavior in strictly stable and null stable games. Finally, the projection and replicator dynamics both can be viewed as gradient systems in potential games, the latter after an appropriate transformation of the state space.     

Infrastructures, incentives, and institutions: Fostering distributed knowledge bases for the learning society

While much attention has been devoted to information and communication technologies, a more fundamental change at the start of the new millennium is the increasing importance of innovation for economic prosperity and the emergence of a learning society. The analysis in this paper shows that innovation should be understood as a broad social and economic activity: it should transcend any specific technology, even if revolutionary, and should be tied to attitudes and behaviors oriented towards the exploitation of change by adding value.We build on the idea of inclusive learning, which entails a process of shared prosperity across the globe following local specific conditions, and argue that it is crucial to understand the features of knowledge-induced growth in rich countries, as well as the challenges and opportunities for late-industrialized and less-developed countries. To achieve these objectives, we emphasize the relative importance of infrastructures and incentives, but considering the increasingly important role of institutions towards the development of social capital. This is because learning societies will increasingly rely on “distributed knowledge bases” as a systematically coherent set of knowledge maintained across an economically and/or socially integrated set of agents and institutions.This broad concept has motivated the work behind the present paper, which builds on material presented at the 5th International Conference on Technology Policy and Innovation (ICTPI), which was held in Delft, The Netherlands, in June of 2001. Under the broad designation of “critical infrastructures,” the Conference brought together a range of experts to discuss technology, policy and management in a context much influenced by the dynamics of the process of knowledge accumulation, which drives learning societies. Thus, this special issue includes a set of extended contributions to the Delft conference, and the aim of this introductory paper is to set the stage for these contributions, with an original contribution on possible views on the role critical infrastructures play to foster innovation in the learning society.     

Pollution externalities,endogenous health and the speed of convergence in an endogenous growth model

Based on a dynamic general equilibrium model, we study the important link between pollution, health and growth during transition which has not received much attention in the existing theoretical literature on (semi-) endogenous growth. The focus of this paper is on the transitional dynamics behavior, with an emphasis both, on the asymptotic as well as on the average speed of convergence. We calibrate the model for a typical OECD country. On the basis of dynamic welfare calculations, which consider transitional dynamics effects, we show that welfare differences are most pronounced for an economy, which values health relatively highly and where the environmental tax change is more than just marginal. Hence, if an economy moves towards a more sustainable society, the policy maker should take into consideration transitional dynamic effects.     

Nonparametric learning rules from bandit experiments: The eyes have it!

How do people learn? We assess, in a model-free manner, subjectsʼ belief dynamics in a two-armed bandit learning experiment. A novel feature of our approach is to supplement the choice and reward data with subjectsʼ eye movements during the experiment to pin down estimates of subjectsʼ beliefs. Estimates show that subjects are more reluctant to “update down” following unsuccessful choices, than “update up” following successful choices. The profits from following the estimated learning and decision rules are smaller (by about 25% of average earnings by subjects in this experiment) than what would be obtained from a fully-rational Bayesian learning model, but comparable to the profits from alternative non-Bayesian learning models, including reinforcement learning and a simple “win-stay” choice heuristic.