GENERALIZED STOCHASTIC GRADIENT LEARNING*

We study the properties of the generalized stochastic gradient (GSG) learning in forward‐looking models. GSG algorithms are a natural and convenient way to model learning when agents allow for parameter drift or robustness to parameter uncertainty in their beliefs. The conditions for convergence of GSG learning to a rational expectations equilibrium are distinct from but related to the well‐known stability conditions for least squares learning.     

Heterogeneous learning

There is by now a large literature characterising conditions under which learning schemes converge to rational expectations equilibria (REEs). It has been claimed that these results depend on the assumption of homogeneous agents and homogeneous learning. This paper analyses the stability of REEs under heterogeneous adaptive learning, for the class of self-referential linear stochastic models. Agents may differ in their initial perceptions about the evolution of the economy, the degrees of inertia in revising their expectations, or the learning rules they use. General conditions are provided for local stability of an REE. In general, it is not possible to show that stability under homogeneous learning implies stability under heterogeneous learning. To illustrate how to apply the results, several examples are provided.     

On capital accumulation paths in a neoclassical stochastic growth model

Summary. Boldrin and Montrucchio [2] showed that any twice continuously differentiable function could be obtained as the optimal policy function for some value of the discount parameter in a deterministic neoclassical growth model. I extend their result to the stochastic growth model with non-degenerate shocks to preferences or technology. This indicates that one can obtain complex dynamics endogenously in a wide variety of economic models, both under certainty and uncertainty. Further, this result motivates the analysis of convergence of adaptive learning mechanisms to rational expectations in economic models with (potentially) complicated dynamics. Received: June 21, 1996; revised version: October 31, 1996     

Learning and Equilibrium Selection in a Monetary Overlapping Generations Model with Sticky Prices

We study adaptive learning in a monetary overlapping generations model with sticky prices and monopolistic competition for the case where learning agents observe current endogenous variables. Observability of current variables is essential for informational consistency of the learning setup with the model setup but generates multiple temporary equilibria when prices are flexible and prevents a straightforward construction of the learning dynamics. Sticky prices overcome this problem by avoiding simultaneity between prices and price expectations. Adaptive learning then robustly selects the determinate (monetary) steady state independent from the degree of imperfect competition. The indeterminate (non-monetary) steady state and non-stationary equilibria are never stable. Stability in a deterministic version of the model may differ because perfect foresight equilibria can be the limit of restricted perceptions equilibria of the stochastic economy with vanishing noise and thereby inherit different stability properties. This discontinuity at the zero variance of shocks suggests one should analyse learning in stochastic models.     

Initial beliefs and the global stability of least squares learning

This article provides an operational framework of global stability analysis when Ljung's (1977) ordinary differential equation (ODE) approach is applied to the recursive stochastic system. We first establish the notion of stable set under which sufficient conditions for the equivalence between them can be translated to restrictions on initial beliefs in agents' forecasts. The maximum ODE-stable set is simply the largest range of initial beliefs. We then demonstrate how to implement this operational framework for the analysis of stability beyond the local sense in some well-known models in the learning literature.     

The convergence of least squares learning in stochastic temporary equilibrium models

Summary. This paper provides conditions for the almost sure convergence of the least squares learning rule in a stochastic temporary equilibrium model, where regressions are performed on the past values of the endogenous state variable. In contrast to earlier studies, (Evans and Honkapohja, 1998; Marcent and Sargent, 1989), which were local analyses, the dynamics are studied from a global viewpoint, which allows one to obtain an almost sure convergence result without employing projection facilities. Received: April 7, 2001; revised version: September 5, 2001     

Stability under learning: The endogenous growth problem

A local dynamic analysis, in the neighborhood of the steady state, is developed for one and two-sector endogenous growth models. The problem differs from the conventionally assumed growth setups because one considers that expectations concerning the next period value of the control variable (consumption) are formed through adaptive learning. In such scenario, the found stability conditions reveal that convergence to the unique steady state point is feasible if a minimum requirement regarding the quality of learning in the long run equilibrium is fulfilled. Therefore, stability of growth under learning is dependent on the efficiency with which expectations are generated.     

Recontracting and stochastic stability in cooperative games

An evolutionary style model of recontracting is given which guarantees convergence to core allocations of an underlying cooperative game. Unlike its predecessors in the evolution/learning literature, this is achieved without assumptions of convexity of the characteristic function or a reliance on random errors. The stochastic stability properties of the model are then examined and it is shown that stochastically stable states solve a simple and intuitive minimization problem which reduces to maximizing a Rawlsian SWF for a common class of utility functions. In contrast to previous analyses, the stochastically stable state is unique for a broad class of utility functions.     

Learning and behavioral stability An economic interpretation of genetic algorithms

This article tries to connect two separate strands of literature concerning genetic algorithms. On the one hand, extensive research took place in mathematics and closely related sciences in order to find out more about the properties of genetic algorithms as stochastic processes. On the other hand, recent economic literature uses genetic algorithms as a metaphor for social learning. This paper will face the question of what an economist can learn from the mathematical branch of research, especially concerning the convergence and stability properties of the genetic algorithm. It is shown that genetic algorithm learning is a compound of three different learning schemes. First, each particular scheme is analyzed. Then it is shown that it is the combination of the three schemes that gives genetic algorithm learning its special flair: A kind of stability somewhere in between asymptotic convergence and explosion.     

A Markov analysis of social learning and adaptation

A number of recent contributions to the literature have modelled social learning and adaptation in an economic context. Understanding the processes driving these models is important in order to explain and predict the behaviour of the economy. In this paper, we analyze the economic applications for a class of adaptive learning models with bounded rational agents. The dynamics of these economies can be thought of as arising from discrete-time Markov chains. In particular, conditions for uniqueness of equilibria, convergence and stability in the economic systems follow from the accessibility and communication structures of these Markov chains. We establish a correspondence between absorbing states of the Markov chains and economic equilibria, whether stable or unstable, and develop theorems giving conditions for absorption and recurrence. Furthermore, we develop practical applications of these theorems using a cobweb model. We use a genetic algorithm, operating under election, as an example of a well known adaptive learning process.     

ESTIMATION AND INFERENCE BY THE METHOD OF PROJECTION MINIMUM DISTANCE: AN APPLICATION TO THE NEW KEYNESIAN HYBRID PHILLIPS CURVE*

The stability of the solution path in a macroeconomic model implies that it admits a Wold representation. This Wold representation can be estimated semi‐parametrically by local projections and used to estimate the model's parameters by minimum distance techniques even when the stochastic process for the solution path is unknown or unconventional. We name this two‐step estimation procedure “projection minimum distance” and investigate its statistical properties for the broad class of models where the mapping between Wold coefficients and parameters is linear. This includes many situations with likelihood score functions nonlinear in the parameters that would otherwise require numerical optimization routines.     

Job market signaling and employer learning

We consider a signaling model where the sender?s continuation value after signaling depends on his type, for instance because the receiver is able to update his posterior belief. As a leading example, we introduce Bayesian learning in a variety of environments ranging from simple two-period to continuous-time models with stochastic production. Signaling equilibria present two major departures from those obtained in models without learning. First, new mixed-strategy equilibria involving multiple pooling are possible. Second, pooling equilibria can survive the Intuitive Criterion when learning is efficient enough.     

Structural stability and model design

The paper introduces the concept of structural stability and proposes that it should be considered a necessary property of scientifically valid models. Formalization of the concept is considered in both linear and non-linear models. A strong preference in favour of the wider use of non-linear models is supported by consideration of the dangers of linearization in dynamic models. The importance of structural stability is demonstrated with reference to dynamic rational expectations models which exhibit the saddle-point property. In such cases convergence to equilibrium is shown to be a structurally unstable property which can be forced by restrictive auxillary assumptions, which are highlighted.     

Expectational coordination in simple economic contexts

We consider an economic model that features (1) a continuum of agents and (2) an aggregate state of the world over which agents have an infinitesimal influence. We first review the connections between the ??eductive?? viewpoint on expectational stability and standard game-theoretical rationalizability concepts. The ??eductive?? reasoning selects different plausible beliefs that are a priori, and possibly a posteriori, ??diverse??. Such beliefs are associated with the sets of ??Cobweb tatonnement?? outcomes, ??Rationalizable States?? and ??Point-Rationalizable States?? (the latter two being shown to be convex). In the case where our model displays strategic complementarities, unsurprisingly, all our ??eductive?? criteria support similar conclusions, particularly when the equilibrium is unique. With strategic substitutabilities, the success of expectational coordination, in the case where a unique equilibrium does exists, relates with the absence of cycles of order 2 of the ??Cobweb?? mapping: in this case, full expectational coordination would be achieved. However, when cycles of order 2 do exist, our different criteria predict larger sets of outcomes, although all tied with cycles. Under differentiability assumptions, the Poincaré?CHopf method leads to other global stability results. At the local level, the different criteria under scrutiny can be adapted. They lead to the same expectational stability conclusions, only when there are local strategic complementarities or strategic substitutabilities. However, for the local stability analysis, it is demonstrated that the stochastic character of expectations can most often be forgotten.     

Modelling under ambiguity with dynamically consistent Choquet random walks and Choquet–Brownian motions

Ambiguity is pervasive in many environments and is increasingly being introduced into economic and financial models. This paper characterises ambiguity in the form of newly defined Choquet random walks: discrete-time binomial trees with capacities instead of exact probabilities on their branches. We describe the axiomatic basis of Choquet random walks, including dynamic consistency. We also discuss the convergence of Choquet random walks to Choquet–Brownian motion in continuous time. In contrast to previous literature, we derive tractable stochastic processes that allow for a wide range of ambiguity preferences to be represented in continuous time (including ambiguity-seeking preferences). Finally, we apply Choquet–Brownian ambiguity to a model of stationary inter-temporal portfolio choice. We find that both the mean and the variance of the underlying stochastic process are modified. This result opens the way for qualitative and quantitative results that differ from those of standard expected utility models and other models that feature ambiguity.     

Idiosyncratic and international transmission of shocks in the G7: Does EMU matter?

This paper analyzes the transmission of shocks across the Group of Seven industrialized countries (G7) before and after the introduction of the euro. We estimate global vector autoregressive (VAR) models for different periods to investigate changes in the domestic and international adjustment of macroeconomic variables following supply, demand, and nominal shocks. The shocks are identified with robust sign restrictions, which we derive from a small open economy dynamic stochastic general equilibrium (DSGE) model. Specifically, we analyze the adjustment of output, inflation, and the real effective exchange rate following those shocks. Our results indicate that changes in the adjustment are due to global convergence rather than to regional‐specific convergence.     

Self-tuning experience weighted attraction learning in games

Self-tuning experience weighted attraction (EWA) is a one-parameter theory of learning in games. It addresses a criticism that an earlier model (EWA) has too many parameters, by fixing some parameters at plausible values and replacing others with functions of experience so that they no longer need to be estimated. Consequently, it is econometrically simpler than the popular weighted fictitious play and reinforcement learning models. The functions of experience which replace free parameters “self-tune” over time, adjusting in a way that selects a sensible learning rule to capture subjects’ choice dynamics. For instance, the self-tuning EWA model can turn from a weighted fictitious play into an averaging reinforcement learning as subjects equilibrate and learn to ignore inferior foregone payoffs. The theory was tested on seven different games, and compared to the earlier parametric EWA model and a one-parameter stochastic equilibrium theory (QRE). Self-tuning EWA does as well as EWA in predicting behavior in new games, even though it has fewer parameters, and fits reliably better than the QRE equilibrium benchmark.     

Stochastic Learning and the Evolution of Conventions

This paper shows that aspiration driven, stochastic learning processes can enable the members of a population to reach more efficient conventions, which are modeled as equilibria of a coordination game. As opposed to adaptation processes based on either myopic optimization or imitating behaviour, which generally select the risk dominant equilibrium, the present paper provides conditions under which the efficient equilibrium is selected, even if it is risk dominated by another equilibrium. Further, the impact of players' aspiration levels on the existence and local stability of a convention is analyzed. In general, higher aspirations induce more efficient outcomes. In addition, in the case of a local interaction structure there exist an inverse relation between the size of the individual neighbourhoods and the aspiration level required in order to sustain efficient outcomes.     

Quality risk aversion, conjectures, and new product diffusion

In this paper, we provide a generalization of the standard models of the diffusion of a new product. Consumers are heterogeneous and risk averse, and the firm is uncertain about the demand curve: both learn from past observations. The attitude towards risk has important effects with regard to the diffusion pattern. In our model, downward-biased signals to consumers can prevent the success of the product, even if its objective quality is high: a “lock-in” result. We show, in addition, that the standard logistic pattern can be derived from the model. Finally, we discuss the steady states of the learning dynamics, with regard to the multiplicity and the local stability of equilibria, and to their welfare properties.