Sequential decisions under uncertainty and the maximum theorem

For sequential decision problems in which the decision-maker observes a process of state variables and chooses an adapted process of action variables, the paper defines a topology on the space of measures of processes of state variables which ensures the applicability of Berge's maximum theorem to the decision-maker's optimal behavior. The topology controls for the information available to the decision-maker at each decision date. The paper also discusses the implications of the analysis for the dynamic-programming approach to sequential decision-making under uncertainty, and for equilibrium existence proof strategies in sequential-market models and games.     

Markov Perfect equilibria in repeated asynchronous choice games

This paper examines the issue of multiplicity of Markov Perfect equilibria in alternating move repeated games. Such games are canonical models of environments with repeated, asynchronous choices due to inertia or replacement. Our main result is that the number of Markov Perfect equilibria is generically finite with respect to stage game payoffs. This holds despite the fact that the stochastic game representation of the alternating move repeated game is “non-generic” in the larger space of state dependent payoffs. We further obtain that the set of completely mixed Markov Perfect equilibria is generically empty with respect to stage game payoffs.     

Learning by imitation

This paper introduces a learning algorithm that allows for imitation in recursive dynamic games. The Kiyotaki–Wright model of money is a well-known example of such decision environments. In this context, learning by experience has been studied before. Here, we introduce imitation as an additional channel for learning. In numerical simulations, we observe that the presence of imitation either speeds up social convergence to the theoretical Markov–Nash equilibrium or leads every agent of the same type to the same mode of suboptimal behavior. We observe an increase in the probability of convergence to equilibrium, as the incentives for optimal play become more pronounced.     

Effects of decision interval on optimal intertemporal portfolios with serially correlated returns

This paper considers portfolio choice when decisions are made for several future time periods all at once. The risky asset share sequence must be precommitted for the entire decision interval, either constrained (as in Samuelson (1991)) or not constrained (as in Balvers and Mitchell (2000)) to be constant across time periods within the interval. For a broad, plausible class of dynamic returns processes, contrary to Samuelson, under log utility the decisions for the more distant future are more conservative. This class is exemplified by autocorrelated ARMA(p,q) processes and finite-state Markov processes. The source of Samuelson’s contrary result is elucidated.     

Multilateral negotiations and formation of coalitions

This paper analyses properties of games modeling multilateral negotiations leading to the formation of coalitions in an environment with widespread externalities. The payoff generated by each coalition is determined by an exogenous partition function (the parameter space). We show that in almost all games, except in a set of measure zero of the parameter space, the Markov perfect equilibrium value of coalitions and the state transition probability that describe the path of coalition formation is locally unique and stable. Therefore, comparative statics analysis are well-defined and can be performed using standard calculus tools. Global uniqueness does not hold in general, but the number of equilibria is finite and odd. In addition, a sufficient condition for global uniqueness is derived, and using this sufficient condition we show that there is a globally unique equilibrium in three-player superadditive games.     

Set-valued solution concepts using interval-type payoffs for interval games

Uncertainty is a daily presence in the real world. It affects our decision making and may have influence on cooperation. Often uncertainty is so severe that we can only predict some upper and lower bounds for the outcome of our actions, i.e., payoffs lie in some intervals. A suitable game theoretic model to support decision making in collaborative situations with interval data is that of cooperative interval games. Solution concepts that associate with each cooperative interval game sets of interval allocations with appealing properties provide a natural way to capture the uncertainty of coalition values into the players’ payoffs. In this paper, some set-valued solution concepts using interval payoffs, namely the interval core, the interval dominance core and the interval stable sets for cooperative interval games, are introduced and studied. The main results contained in the paper are a necessary and sufficient condition for the non-emptiness of the interval core of a cooperative interval game and the relations between the interval core, the interval dominance core and the interval stable sets of such a game.     

Discounted semi-Markov decision processes: linear programming and policy iteration

For semi-Markov decision processes with discounted rewards we derive the well known results regarding the structure of optimal strategies (nonrandomized, stationary Markov strategies) and the standard algorithms (linear programming, policy iteration). Our analysis is completely based on a primal linear programming formulation of the problem.     

A sufficient condition on ƒ for ƒμ to be in pNAD

In this paper we show, under quite weak assumptions on the cost function, that the Aumann-Shapley price mechanism is continuous with respect to the quantities consumed. The proof uses the theory of non-atomic games.     

Reason,Intuition, and Time

We study the influence of reason and intuition on decision‐making over time. Facing a sequence of similar problems, agents can either decide rationally according to expected utility theory or intuitively according to case‐based decision theory. Rational decisions are more precise but create higher costs, though these costs may decrease over time. We find that intuition will outperform reason in the long run if individuals are sufficiently ambitious. Moreover, intuitive decisions are prevalent in the early and late stages of a learning process, whereas reason governs decisions in intermediate stages. Examples range from playing behavior in games like chess to professional decisions during a manager's career. Copyright © 2015 John Wiley & Sons, Ltd.     

The core of the inductive limit of a direct system of economies with a communication structure

This paper studies direct systems of finite, that is with finitely many agents, communication structures, finite (balanced and superadditive) NTU games and finite economies. The inductive limit of such a system is again a communication structure, an NTU game or an economy, this time possibly with infinitely many agents. As a matter of fact, each infinite communication structure, each infinite NTU game and each infinite economy, is the inductive limit of a direct system of finite communication structures, finite NTU games and finite economies. A communication game is an NTU game with a communication structure on the set of players. To each economy, there corresponds a balanced and superadditive NTU game. To each economy with a communication structure on the set of agents, there corresponds a communication game. In the paper it is proved that the core of the inductive limit of a direct system of communication games is not empty and in fact the intersection of the cores of the finite communication games of the direct system. It follows that each infinite economy (with or without a communication structure on the set of agents) has a nonempty core. A direct system of economies is a generalisation of the Debreu and Scarf [Debreu, G., Scarf, H. A limit theorem on the core of an economy, International Economic Review 4, pp. 235–246.] example of `replica economies'. The proof of the nonemptiness of the core of the inductive limit of a direct system of economies is along the lines of the proof by Debreu and Scarf. As by-product it is shown that an NTU game is totally balanced if and only if all its finite subgames are balanced.     

Structure of teams—A cooperative game theory approach

One key task for HR departments and managers is to design the structure of teams in the firm. Two main dimensions when carrying out this decision are the size of the teams and the combination of specialized expertise within the team. In this article, we apply cooperative game theory to model this decision. First, we introduce production games. These games are an enhancement of team games and allow the modeling of teams. Together with the coalition structure approach of cooperative game theory, we analyze two standard situations of team theory.     

On strong representations of games by social choice functions

Games in characteristic function form are used to model an allocation of decision power among individuals involved in a voting situation. The problem of strong representation is to find a strategically acceptable social choice function that entails the allocation of power prescribed by a given game. Within the class of non-weak characteristic function games, we fully characterize the games that admit a strong representation. We apply this result to Peleg's problem of strong representation of simple games. Our results indicate that a strong representation requires significantly more than has been recognized in the literature.     

MARKOV DECISION PROCESSES

A review is presented of the development over the years of the theory and practical use of Markov decision processes. To this purpose three periods are considered: before 1966, from 1966 till 1972, and after 1973. In all 3 periods there has been some contribution from the Netherlands, but particularly in the last period the research in the Netherlands on the subject has become a major stream. In this review particular emphasis is given to the work which has been done in the Netherlands, but the main line of the paper is determined by the development of the applicability of the available theory.     

Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models

This paper analyzes the higher-order properties of the estimators based on the nested pseudo-likelihood (NPL) algorithm and the practical implementation of such estimators for parametric discrete Markov decision models. We derive the rate at which the NPL algorithm converges to the MLE and provide a theoretical explanation for the simulation results in Aguirregabiria and Mira [Aguirregabiria, V., Mira, P., 2002. Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica 70, 1519–1543], in which iterating the NPL algorithm improves the accuracy of the estimator. We then propose a new NPL algorithm that can achieve quadratic convergence without fully solving the fixed point problem in every iteration and apply our estimation procedure to a finite mixture model. We also develop one-step NPL bootstrap procedures for discrete Markov decision models. The Monte Carlo simulation evidence based on a machine replacement model of Rust [Rust, J., 1987. Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica 55, 999–1033] shows that the proposed one-step bootstrap test statistics and confidence intervals improve upon the first order asymptotics even with a relatively small number of iterations.     

On the relationship between robustness to incomplete information and noise-independent selection in global games

This note demonstrates that a symmetric 3×3 supermodular game may fail to have any equilibrium robust to incomplete information. Since the global game solution in symmetric 3×3 supermodular games is known to be independent of the noise structure, this result implies that a noise-independent selection in global games may not be a robust equilibrium. Our proof reveals that the assumption in global games that the noise errors are independent of the state imposes a non-trivial restriction on incomplete information perturbations.     

Investment behavior under Knightian uncertainty – An evolutionary approach

The ‘as if ’ view of economic rationality defends the profit maximization hypothesis by pointing out that only those firms who act as if they maximize profits can survive in the long run. Recently, the problem of arriving at a logically consistent definition of rational behavior in games has shown that one must sometimes study explicitly the evolutionary processes that form the basis of this view. The purpose of this paper is to investigate the usefulness of genetic programming as a tool for generating hypotheses about rational behavior in situations where explicit maximization is not well defined. We use an investment decision problem with Knightian uncertainty as a borderline test case, and show that when the artificial agents receive the same information about the unknown probability distributions, they develop behavior rules as if they were expected utility maximizers with Bayesian learning rules.     

Best response dynamics with level-n expectations in two-stage games

This paper analyzes behavior in repeatedly played two-stage games, where players choose actions in both stages according to best replies using level-n expectations about the opponent׳s actions in both stages. Level-n expectations are recursively defined in a way that a player holding level-n expectations correctly predicts the action of an opponent holding level-(n−1) expectations. A general conceptual framework to study such dynamics for two-stage games is developed and it is shown that, contrary to results for single-stage games, the fixed points of the dynamics depend on the level of the expectations. In particular, for level-0 expectation, fixed points correspond to a Nash equilibrium of a simultaneous move version of the game, whereas (under certain conditions) fixed points converge towards the subgame perfect equilibrium of the two-stage game if the level of expectations goes to infinity. The approach is illustrated using a two-stage duopoly game, where firms in the first stage invest in activities reducing their marginal costs and in the second stage engage in Cournot competition. Conditions for local stability of the fixed points are derived for different levels of expectations and it is shown that level-2 expectations are sufficient to move the fixed-point of the dynamics to a close neighborhood of the subgame perfect equilibrium. Furthermore, it is demonstrated that although firms benefit from unilateral increases in the level of expectations, an increase of n by all firms reduces all profits.     

Testing the Markov property with high frequency data

This paper develops a framework to nonparametrically test whether discrete-valued irregularly spaced financial transactions data follow a Markov process. For that purpose, we consider a specific optional sampling in which a continuous-time Markov process is observed only when it crosses some discrete level. This framework is convenient for it accommodates the irregular spacing that characterizes transactions data. Under such an observation rule, the current price duration is independent of a previous price duration given the previous price realization. A simple nonparametric test then follows by examining whether this conditional independence property holds. Monte Carlo simulations suggest that the asymptotic test has huge size distortions, though a bootstrap-based variant entails reasonable size and power properties in finite samples. As for an empirical illustration, we investigate whether bid–ask spreads follow Markov processes using transactions data from the New York Stock Exchange. The motivation lies on the fact that asymmetric information models of market microstructures predict that the Markov property does not hold for the bid–ask spread. We robustly reject the Markov assumption for two out of the five stocks under scrutiny. Finally, it is reassuring that our results are consistent with two alternative measures of asymmetric information.     

Vertical syndication-proof competitive prices in multilateral assignment markets

We consider a market comprising a number of perfectly complementary and homogeneous commodities. We concentrate on the incentives for firms producing these commodities to merge and form a vertical syndicate. The main result establishes that the nucleolus of the associated market game corresponds to the unique vector of prices with the following properties: (i) they are vertical syndication-proof, (ii) they are competitive, (iii) they yield the average of the buyers- and the sellers-optimal allocations in bilateral markets, and (iv) they depend on the traders’ bargaining power but not on their identity. The proof uses an isomorphism between our class of market games and the entire class of bankruptcy games.