Bayesian identification,selection and estimation of semiparametric functions in high-dimensional additive models

In this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.     

Sieve estimation of panel data models with cross section dependence

In this paper we consider the problem of estimating semiparametric panel data models with cross section dependence, where the individual-specific regressors enter the model nonparametrically whereas the common factors enter the model linearly. We consider both heterogeneous and homogeneous regression relationships when both the time and cross-section dimensions are large. We propose sieve estimators for the nonparametric regression functions by extending Pesaran’s (2006) common correlated effect (CCE) estimator to our semiparametric framework. Asymptotic normal distributions for the proposed estimators are derived and asymptotic variance estimators are provided. Monte Carlo simulations indicate that our estimators perform well in finite samples.     

Nonparametric identification and estimation of sample selection models under symmetry

Under a conditional mean restriction Das et al. (2003) considered nonparametric estimation of sample selection models. However, their method can only identify the outcome regression function up to a constant. In this paper we strengthen the conditional mean restriction to a symmetry restriction under which selection biases due to selection on unobservables can be eliminated through proper matching of propensity scores; consequently we are able to identify and obtain consistent estimators for the average treatment effects and the structural regression functions. The results from a simulation study suggest that our estimators perform satisfactorily.     

Partial parametric estimation for nonstationary nonlinear regressions

This paper proposes an estimation method for a partial parametric model with multiple integrated time series. Our estimation procedure is based on the decomposition of the nonparametric part of the regression function into homogeneous and integrable components. It consists of two steps: In the first step we parameterize and fit the homogeneous component of the nonparametric part by the nonlinear least squares with other parametric terms in the model, and use in the second step the standard kernel method to nonparametrically estimate the integrable component of the nonparametric part from the residuals in the first step. We establish consistency and obtain the asymptotic distribution of our estimator. A simulation shows that our estimator performs well in finite samples. For the empirical illustration, we estimate the money demand functions for the US and Japan using our model and methodology.     

Semiparametric Bayesian inference in smooth coefficient models

We describe procedures for Bayesian estimation and testing in cross-sectional, panel data and nonlinear smooth coefficient models. The smooth coefficient model is a generalization of the partially linear or additive model wherein coefficients on linear explanatory variables are treated as unknown functions of an observable covariate. In the approach we describe, points on the regression lines are regarded as unknown parameters and priors are placed on differences between adjacent points to introduce the potential for smoothing the curves. The algorithms we describe are quite simple to implement—for example, estimation, testing and smoothing parameter selection can be carried out analytically in the cross-sectional smooth coefficient model.     

Detections of changes in return by a wavelet smoother with conditional heteroscedastic volatility

In this paper, we propose two estimators, an integral estimator and a discretized estimator, for the wavelet coefficient of regression functions in nonparametric regression models with heteroscedastic variance. These estimators can be used to test the jumps of the regression function. The model allows for lagged-dependent variables and other mixing regressors. The asymptotic distributions of the statistics are established, and the asymptotic critical values are analytically obtained from the asymptotic distribution. We also use the test to determine consistent estimators for the locations of change points. The jump sizes and locations of change points can be consistently estimated using wavelet coefficients, and the convergency rates of these estimators are derived. We perform some Monte Carlo simulations to check the powers and sizes of the test statistics. Finally, we give practical examples in finance and economics to detect changes in stock returns and short-term interest rates using the empirical wavelet method.     

An integrated maximum score estimator for a generalized censored quantile regression model

Quantile regression techniques have been widely used in empirical economics. In this paper, we consider the estimation of a generalized quantile regression model when data are subject to fixed or random censoring. Through a discretization technique, we transform the censored regression model into a sequence of binary choice models and further propose an integrated smoothed maximum score estimator by combining individual binary choice models, following the insights of Horowitz (1992) and Manski (1985). Unlike the estimators of Horowitz (1992) and Manski (1985), our estimators converge at the usual parametric rate through an integration process. In the case of fixed censoring, our approach overcomes a major drawback of existing approaches associated with the curse-of-dimensionality problem. Our approach for the fixed censored case can be extended readily to the case with random censoring for which other existing approaches are no longer applicable. Both of our estimators are consistent and asymptotically normal. A simulation study demonstrates that our estimators perform well in finite samples.     

Local rank tests in a multivariate nonparametric relationship

Consider a multivariate nonparametric model where the unknown vector of functions depends on two sets of explanatory variables. For a fixed level of one set of explanatory variables, we provide consistent statistical tests, called local rank tests, to determine whether the multivariate relationship can be explained by a smaller number of functions. We also provide estimators for the smallest number of functions, called local rank, explaining the relationship. The local rank tests and the estimators of local rank are defined in terms of the eigenvalues of a kernel-based estimator of some matrix. The asymptotics of the eigenvalues is established by using the so-called Fujikoshi expansion along with some techniques of the theory of U-statistics. We present a simulation study which examines the small sample properties of local rank tests. We also apply the local rank tests and the local rank estimators to a demand system given by a newly constructed data set. This work can be viewed as a “local” extension of the tests for a number of factors in a nonparametric relationship introduced by Stephen Donald.     

Testing for non-nested conditional moment restrictions using unconditional empirical likelihood

We propose non-nested hypothesis tests for conditional moment restriction models based on the method of generalized empirical likelihood (GEL). By utilizing the implied GEL probabilities from a sequence of unconditional moment restrictions that contains equivalent information of the conditional moment restrictions, we construct Kolmogorov–Smirnov and Cramér–von Mises type moment encompassing tests. Advantages of our tests over Otsu and Whang’s (2011) tests are: (i) they are free from smoothing parameters, (ii) they can be applied to weakly dependent data, and (iii) they allow non-smooth moment functions. We derive the null distributions, validity of a bootstrap procedure, and local and global power properties of our tests. The simulation results show that our tests have reasonable size and power performance in finite samples.     

Quantile regression for dynamic panel data with fixed effects

This paper studies a quantile regression dynamic panel model with fixed effects. Panel data fixed effects estimators are typically biased in the presence of lagged dependent variables as regressors. To reduce the dynamic bias, we suggest the use of the instrumental variables quantile regression method of Chernozhukov and Hansen (2006) along with lagged regressors as instruments. In addition, we describe how to employ the estimated models for prediction. Monte Carlo simulations show evidence that the instrumental variables approach sharply reduces the dynamic bias, and the empirical levels for prediction intervals are very close to nominal levels. Finally, we illustrate the procedures with an application to forecasting output growth rates for 18 OECD countries.     

Bayesian stochastic search for VAR model restrictions

We propose a Bayesian stochastic search approach to selecting restrictions for vector autoregressive (VAR) models. For this purpose, we develop a Markov chain Monte Carlo (MCMC) algorithm that visits high posterior probability restrictions on the elements of both the VAR regression coefficients and the error variance matrix. Numerical simulations show that stochastic search based on this algorithm can be effective at both selecting a satisfactory model and improving forecasting performance. To illustrate the potential of our approach, we apply our stochastic search to VAR modeling of inflation transmission from producer price index (PPI) components to the consumer price index (CPI).     

Nonparametric simultaneous testing for structural breaks

In this paper we consider a regression model with errors that are martingale differences. This modeling includes the regression of both independent and time series data. The aim is to study the appearance of structural breaks in both the mean and the variance functions, assuming that such breaks may occur simultaneously in both the functions. We develop nonparametric testing procedures that simultaneously test for structural breaks in the conditional mean and the conditional variance. The asymptotic distribution of an adaptive test statistic is established, as well as its asymptotic consistency and efficiency. Simulations illustrate the performance of the adaptive testing procedure. An application to the analysis of financial time series also demonstrates the usefulness of the proposed adaptive test in practice.     

Regression discontinuity inference with specification error

A regression discontinuity (RD) research design is appropriate for program evaluation problems in which treatment status (or the probability of treatment) depends on whether an observed covariate exceeds a fixed threshold. In many applications the treatment-determining covariate is discrete. This makes it impossible to compare outcomes for observations “just above” and “just below” the treatment threshold, and requires the researcher to choose a functional form for the relationship between the treatment variable and the outcomes of interest. We propose a simple econometric procedure to account for uncertainty in the choice of functional form for RD designs with discrete support. In particular, we model deviations of the true regression function from a given approximating function—the specification errors—as random. Conventional standard errors ignore the group structure induced by specification errors and tend to overstate the precision of the estimated program impacts. The proposed inference procedure that allows for specification error also has a natural interpretation within a Bayesian framework.     

Regression density estimation using smooth adaptive Gaussian mixtures

We model a regression density flexibly so that at each value of the covariates the density is a mixture of normals with the means, variances and mixture probabilities of the components changing smoothly as a function of the covariates. The model extends the existing models in two important ways. First, the components are allowed to be heteroscedastic regressions as the standard model with homoscedastic regressions can give a poor fit to heteroscedastic data, especially when the number of covariates is large. Furthermore, we typically need fewer components, which makes it easier to interpret the model and speeds up the computation. The second main extension is to introduce a novel variable selection prior into all the components of the model. The variable selection prior acts as a self-adjusting mechanism that prevents overfitting and makes it feasible to fit flexible high-dimensional surfaces. We use Bayesian inference and Markov Chain Monte Carlo methods to estimate the model. Simulated and real examples are used to show that the full generality of our model is required to fit a large class of densities, but also that special cases of the general model are interesting models for economic data.     

Control variate method for stationary processes

The sample mean is one of the most natural estimators of the population mean based on independent identically distributed sample. However, if some control variate is available, it is known that the control variate method reduces the variance of the sample mean. The control variate method often assumes that the variable of interest and the control variable are i.i.d. Here we assume that these variables are stationary processes with spectral density matrices, i.e. dependent. Then we propose an estimator of the mean of the stationary process of interest by using control variate method based on nonparametric spectral estimator. It is shown that this estimator improves the sample mean in the sense of mean square error. Also this analysis is extended to the case when the mean dynamics is of the form of regression. Then we propose a control variate estimator for the regression coefficients which improves the least squares estimator (LSE). Numerical studies will be given to see how our estimator improves the LSE.     

Estimation of semiparametric locally stationary diffusion models

This paper proposes a class of locally stationary diffusion processes. The model has a time varying but locally linear drift and a volatility coefficient that is allowed to vary over time and space. The model is semiparametric because we allow these functions to be unknown and the innovation process is parametrically specified, indeed completely known. We propose estimators of all the unknown quantities based on long span data. Our estimation method makes use of the property of local stationarity. We establish asymptotic theory for the proposed estimators as the time span increases, so we do not rely on infill asymptotics. We apply this method to interest rate data to illustrate the validity of our model. Finally, we present a simulation study to provide the finite-sample performance of the proposed estimators.     

Fitting dynamic factor models to non-stationary time series

Factor modelling of a large time series panel has widely proven useful to reduce its cross-sectional dimensionality. This is done by explaining common co-movements in the panel through the existence of a small number of common components, up to some idiosyncratic behaviour of each individual series. To capture serial correlation in the common components, a dynamic structure is used as in traditional (uni- or multivariate) time series analysis of second order structure, i.e. allowing for infinite-length filtering of the factors via dynamic loadings. In this paper, motivated from economic data observed over long time periods which show smooth transitions over time in their covariance structure, we allow the dynamic structure of the factor model to be non-stationary over time by proposing a deterministic time variation of its loadings. In this respect we generalize the existing recent work on static factor models with time-varying loadings as well as the classical, i.e. stationary, dynamic approximate factor model. Motivated from the stationary case, we estimate the common components of our dynamic factor model by the eigenvectors of a consistent estimator of the now time-varying spectral density matrix of the underlying data-generating process. This can be seen as a time-varying principal components approach in the frequency domain. We derive consistency of this estimator in a “double-asymptotic” framework of both cross-section and time dimension tending to infinity. The performance of the estimators is illustrated by a simulation study and an application to a macroeconomic data set.     

Tests for model specification in the presence of alternative hypotheses: Some further results

In Davidson and MacKinnon (1981), two of the present authors proposed a novel and very simple procedure for testing the specification of a nonlinear regression model against the evidence provided by a non-nested alternative. In this paper we extend their results in several directions. First, we relax a number of the assumptions of the previous paper; we admit the possibility that the nonlinear regression functions may depend on lagged dependent variables, and we do not require that the error terms be normally distributed. Second, we show how the earlier procedure may straightforwardly be generalized to the case where the two non-nested models involve different transformations of the dependent variable. Finally, we propose a simple procedure for testing non-nested linear regression models which have endogenous variables on the right-hand side, and have therefore been estimated by two-stage least squares.     

Inference on impulse response functions in structural VAR models

Skepticism toward traditional identifying assumptions based on exclusion restrictions has led to a surge in the use of structural VAR models in which structural shocks are identified by restricting the sign of the responses of selected macroeconomic aggregates to these shocks. Researchers commonly report the vector of pointwise posterior medians of the impulse responses as a measure of central tendency of the estimated response functions, along with pointwise 68% posterior error bands. It can be shown that this approach cannot be used to characterize the central tendency of the structural impulse response functions. We propose an alternative method of summarizing the evidence from sign-identified VAR models designed to enhance their practical usefulness. Our objective is to characterize the most likely admissible model(s) within the set of structural VAR models that satisfy the sign restrictions. We show how the set of most likely structural response functions can be computed from the posterior mode of the joint distribution of admissible models both in the fully identified and in the partially identified case, and we propose a highest-posterior density credible set that characterizes the joint uncertainty about this set. Our approach can also be used to resolve the long-standing problem of how to conduct joint inference on sets of structural impulse response functions in exactly identified VAR models. We illustrate the differences between our approach and the traditional approach for the analysis of the effects of monetary policy shocks and of the effects of oil demand and oil supply shocks.     

Modelling volatility by variance decomposition

In this paper, we propose two parametric alternatives to the standard GJR-GARCH model of Glosten et al. (1993), based on additive and multiplicative decompositions of the variance. They allow the variance of the model to have a smooth time-varying structure. The suggested parameterizations describe structural change in the conditional and unconditional variances where the transition between regimes over time is smooth. The main focus is on the multiplicative decomposition of the variance into an unconditional and conditional components. Estimation of the multiplicative model is discussed in detail. An empirical application to daily stock returns illustrates the functioning of the model. The results show that the ‘long memory type behaviour’ of the sample autocorrelation functions of the absolute returns can also be explained by deterministic changes in the unconditional variance.