Using dynamic programming with adaptive grid scheme for optimal control problems in economics

The study of the solutions of dynamic models with optimizing agents has often been limited by a lack of available analytical techniques to explicitly find the global solution paths. On the other hand, the application of numerical techniques such as dynamic programming to find the solution in interesting regions of the state was restricted by the use of fixed grid size techniques. Following Grüne (Numer. Math. 75 (3) (1997) 319; University of Bayreuth, submitted, 2003), in this paper an adaptive grid scheme is used for finding the global solutions of discrete time Hamilton–Jacobi–Bellman equations. Local error estimates are established and an adapting iteration for the discretization of the state space is developed. The advantage of the use of adaptive grid scheme is demonstrated by computing the solutions of one- and two-dimensional economic models which exhibit steep curvature, complicated dynamics due to multiple equilibria, thresholds (Skiba sets) separating domains of attraction and periodic solutions. We consider deterministic and stochastic model variants. The studied examples are from economic growth, investment theory, environmental and resource economics.     

Note on positive lower bound of capital in the stochastic growth model

In the context of the classical stochastic growth model, we provide a simple proof that the optimal capital sequence is strictly bounded away from zero whenever the initial capital is strictly positive. We assume that the utility function is bounded below and the shocks affecting output are bounded. However, the proof does not require an interval shock space, thus, admitting both discrete and continuous shocks. Further, we allow for finite marginal product at zero capital. Finally, we use our result to show that any optimal capital sequence converges globally to a unique invariant distribution, which is bounded away from zero.     

Solving dynamic public insurance games with endogenous agent distributions: Theory and computational approximation

We make two contributions in this paper. First, we extend the characterization of equilibrium payoff correspondences in history-dependent dynamic policy games to a class with endogenously heterogeneous private agents. In contrast to policy games involving representative agents, this extension has interesting consequences as it implies additional nonlinearity (i.e., bilinearity) between the game states (distributions) and continuation/promised values in the policymaker’s objective and incentive constraints. The second contribution of our paper is in addressing the computational challenges arising from this payoff-relevant nonlinearity. Exploiting the game’s structure, we propose implementable approximate bilinear programming formulations to construct estimates of the equilibrium value correspondence. Our approximation method respects the property of upper hemicontinuity in the target correspondence. We provide small-scale computational examples as proofs of concept.     

A methodology for the choice of the best fitting continuous-time stochastic models of crude oil price

The crude oil price is generally considered as the fundamental factor in the valuation of undeveloped reserves but it is not the unique one. Undeveloped field value also depends on the uncertainty relating to the convenience yield and the risk-free interest rate. The purpose of this paper is to decide on the best continuous-time stochastic models for these risk factors. The Generalized Method of Moments and the Maximum Likelihood Estimation are implemented to fit the parameters of continuous-time stochastic processes. The results of unit root tests without breaks reveal a mean reversion in convenience yield series. Multiple structural change tests show that the risk-free interest rate can be considered constant. The simulation of continuous-time stochastic processes and the mean error between the simulated prices and the market ones show that the Geometric Brownian Motion with jumps is the best model for the oil price compared to the other commonly used processes.     

Mitigation of the Lucas critique with stochastic control methods

Lucas (In: Brunner, K., Meltzer, A.H. (Eds.), The Phillips Curve and the Labor Markets, Supplementary Series to the Journal of Monetary Economics, 1976, pp. 19–46) pointed out, that when optimization is performed on a deterministic macro model, the resulting policy may not reflect the true optimal solution. Private agents may react to announced policies and consequently model parameters will start to drift. The aim of this paper is to develop a methodology for deriving an optimal policy in the presence of rational expectations and parameter drift. This drift is captured by a stochastic optimization framework with time-varying parameters. The resulting optimal policy is capable of tracking changes in the parameters due to policy changes. A numerical example illustrates how the methodology provides a way to mitigate the effects of the Lucas critique.     

A structural analysis of the correlated random coefficient wage regression model

We estimate a dynamic programming model of schooling decisions in which the log wage regression function is set within a correlated random coefficient model. We show that estimates of the dynamic programming model can be used to obtain a number of treatment effects, including the local average treatment effect (LATE). However, unlike LATE parameters obtained in a standard IV framework, our LATE estimates are obtained without imposing separability between individual specific heterogeneity and schooling choices and are therefore not subject to a “monotonicity” restriction. We find that returns to schooling are characterized by a high degree of dispersion across individuals.     

Inflationary equilibrium in a stochastic economy with independent agents

We prove the existence of stationary monetary equilibrium with inflation in a “Bewley” model with constant aggregate real variables but with idiosyncratic shocks to the endowments of a continuum of individual agents, when a central bank stands ready to borrow or lend fiat money at a fixed nominal rate of interest and the agents face borrowing constraints. We also find that, in the presence of real micro uncertainty about individual endowments, the rate of inflation is higher (equivalently, the real rate of interest is lower) than it would be in a “certainty-equivalent economy”; to wit, one in which every agent’s endowment is replaced by its expected value. Thus, underlying microeconomic uncertainty and borrowing constraints are shown to generate additional inflation.     

Thinning and harvesting in stochastic forest models

This paper analyzes a stochastic forest growth model in which the manager is able to first thin the forest to promote better growth before harvesting. Both Wicksell single thinning and harvesting cycle and Faustmann on-going rotation problems are considered. The Wicksell problem is analyzed by first restricting the class of decision times to (thinning, harvesting) pairs that bound the growth away from infinity and imbedding the problem in an infinite-dimensional linear program on a space of triplets of measures. These measures capture the thinning and harvesting decisions along with the behavior of the growth process prior to harvest. An auxiliary linear program then leads to a nonlinear optimization problem for which an optimal value and solution are determined. The values of all the problems are be related through a set of inequalities. The solution of the nonlinear problem determines (random) thinning and harvesting times for the single thinning and harvesting cycle which demonstrate the equality of the values of these various problems. Finally for the Wicksell problem, the unrestricted class of thinning and harvest times is shown to give the same value as the restricted class. The Faustmann on-going thinning and harvesting rotation problem is reduced to a Wicksell problem which then allows for the characterization of the value as the solution to a different nonlinear optimization problem. The effects of the opportunity to thin the forest are illustrated on a mean-reverting stochastic model.     

Dynamic programming for non-additive stochastic objectives

We derive the existence of an optimum and the techniques of dynamic programming for non-additive stochastic objectives. Our key assumption for non-negative objectives is that asymptotic impatience exceeds asymptotic ‘mean’ growth, where ‘mean’ growth is derived not only from intertemporal inelasticity and the random return on investment but also from the curvature of the non-additive stochastic aggregator (i.e. the ‘certainty equivalent’). We provide broad families of new, interesting, and tractable examples. They illustrate that ‘mean’ growth can exist even when the distribution of returns has unbounded support, that power discounting often implies infinite asymptotic impatience, and that non-positive objectives are easily handled with few restrictions on growth.     

Parametric continuity in dynamic programming problems

We examine conditions under which the solutions to parametric families of dynamic programming problems are continuous in the parameters. Our main results are that parametric continuity obtains whenever either (a) the family of dynamic programming problems satisfies strong (joint-) continuity properties in the parameter and state or (b) if it satisfies weaker (separate-) continuity requirements, provided these are supplemented by either stronger assumptions on the transition probabilities or by monotonicity restrictions such as are common in economic modelling. The usefulness of our results is illustrated by applying them to some commonly used dynamic economic models.     

Estimators and hypothesis tests for a stochastic frontier function: A Monte Carlo analysis

This paper uses Monte Carlo experimentation to investigate the finite sample properties of the maximum likelihood (ML) and corrected ordinary least squares (COLS) estimators of the half-normal stochastic frontier production function. Results indicate substantial bias in both ML and COLS when the percentage contribution of inefficiency in the composed error (denoted by *) is small, and also that ML should be used in preference to COLS because of large mean square error advantages when * is greater than 50%. The performance of a number of tests of the existence of technical inefficiency is also investigated. The Wald and likelihood ratio (LR) tests are shown to have incorrect size. A one-sided LR test and a test of the significance of the third moment of the OLS residuals are suggested as alternatives, and are shown to have correct size, with the one-sided LR test having the better power of the two.The author would like to thank Bill Griffiths, George Battese, Howard Doran, Bill Greene and two anonymous referees for valuable comments. Any errors which remain are those of the author.     

Tapping the supercomputer under your desk: Solving dynamic equilibrium models with graphics processors

This paper shows how to build algorithms that use graphics processing units (GPUs) installed in most modern computers to solve dynamic equilibrium models in economics. In particular, we rely on the compute unified device architecture (CUDA) of NVIDIA GPUs. We illustrate the power of the approach by solving a simple real business cycle model with value function iteration. We document improvements in speed of around 200 times and suggest that even further gains are likely.     

A multi-stage stochastic programming approach for pre-positioning of relief supplies considering returns

A dynamic pre-positioning problem is proposed to efficiently respond to victims’ need for relief supplies under uncertain and dynamic demand in humanitarian relief. The problem is formulated as a multi-stage stochastic programming model that considers pre-positioning with the dynamic procurement and return decisions about relief supplies over a time horizon. To validate the advantages of dynamic pre-positioning, three additional pre-positioning strategies are presented: pre-positioning with one-time procurement and without returns, pre-positioning with one-time procurement and returns, and pre-positioning with dynamic procurement and without returns. Using data from real-world disasters in the United States in the Emergency Events Database, we present a numerical analysis to study the applicability of the proposed models. We develop a sample average approximation approach to solving the proposed model in large-scale cases. Our main contribution is that we integrate dynamic procurement and return strategies into pre-positioning to decrease both costs and shortage risks in uncertain and dynamic contexts. The results illustrate that dynamic pre-positioning outperforms the other three strategies in cost savings. It also indicates that a higher return price is particularly helpful for decreasing unmet demand. The proposed models can help relief agencies evaluate and choose the solutions that will have the greatest overall effectiveness in the context of different relief practices.     

Endogeneity in quantile regression models: A control function approach

This paper considers a linear triangular simultaneous equations model with conditional quantile restrictions. The paper adjusts for endogeneity by adopting a control function approach and presents a simple two-step estimator that exploits the partially linear structure of the model. The first step consists of estimation of the residuals of the reduced-form equation for the endogenous explanatory variable. The second step is series estimation of the primary equation with the reduced-form residual included nonparametrically as an additional explanatory variable. This paper imposes no functional form restrictions on the stochastic relationship between the reduced-form residual and the disturbance term in the primary equation conditional on observable explanatory variables. The paper presents regularity conditions for consistency and asymptotic normality of the two-step estimator. In addition, the paper provides some discussions on related estimation methods in the literature.     

浅议WCDMA上行功率控制的作用

文章通过实际的数据结果,说明WCDMA的功率控制技术在一定范围内能较好地克服无线环境的远近效应、快衰落和阴影衰落对无线信号传播的影响,它对于提高无线系统性能和质量起到了非常重要的作用。     

Necessary conditions for infinite-horizon discounted two-stage optimal control problems

In many situations in Economics, one would like to analyse the optimal timing of switching between alternative and consecutive regimes. A natural framework for such an analysis seems to be one of multi-stage optimal control problems, where the switching instants between regimes are endogenously determined. The existing literature considers only the case of two-stage optimal control problems with finite horizon. However, in many cases, it seems more appropriate to consider a problem with an infinite horizon. By deriving the appropriate necessary conditions for such a problem we generalise the methodology in analysing endogenous regime switches.     

Analysis of electric distribution utilities efficiency levels by stochastic frontier in Brazilian power sector

The regulatory power sector model in several countries determines tariff review forms based, among other things, on sharing efficiency gains with consumers. As these reviews have an important impact on consumers and distribution utilities, it is necessary that the adopted methodologies always be improved. To this end, this article assessed a Bayesian inference application in order to estimate a stochastic cost frontier considering temporal efficiency dynamics. Taking this point into consideration is essential, since studies carried out to assess power sector efficiency have neglected the fact that part of efficiency increases originate from scale gain due to market expansion, which occurs over time. The sample assessed herein is composed of panel data from 61 electric power utilities between 2003 and 2016. The results demonstrate that the tariff review is positively affected by distributor efficiency.     

On the local stability of the solution to optimal control problems

New conditions for the local stability of optimal control problems are presented. The conditions are an extension of the results of a previous study focusing on the solution to problems solvable using calculus of variations. A comparison is made between the conditions introduced here and those presented in the literature. It is shown that our condition is roughly as powerful as the one presented by Sorger (J. Math. Anal. Appl. 148 (1990) 191) but it is easier to check.     

Using nonlinear model predictive control for dynamic decision problems in economics

This paper presents a new approach to solve dynamic decision models in economics. The proposed procedure, called Nonlinear Model Predictive Control (NMPC), relies on the iterative solution of optimal control problems on finite time horizons and is well established in engineering applications for stabilization and tracking problems. Only quite recently, extensions to more general optimal control problems including those appearing in economic applications have been investigated. Like Dynamic Programming (DP), NMPC does not rely on linearization techniques but uses the full nonlinear model and in this sense provides a global solution to the problem. However, unlike DP, NMPC only computes one optimal trajectory at a time, thus avoids to grid the state space and for this reason the computational demand grows much more moderately with the space dimension than for DP. In this paper we explain the basic idea of NMPC, give a proof concerning the accuracy of NMPC for discounted optimal control problems, present implementational details, and demonstrate the ability of NMPC to solve dynamic decision problems in economics by solving low and high dimensional examples, including models with multiple equilibria, tracking and stochastic problems.     

利用可编程逻辑控制器实现交通信号灯的控制

文章主要是采用西门子公司的可编程逻辑控制器PLC-200来实现交通信号灯的控制。