Model averaging by jackknife criterion in models with dependent data

The past decade witnessed a literature on model averaging by frequentist methods. For the most part, the asymptotic optimality of various existing frequentist model averaging estimators has been established under i.i.d. errors. Recently, Hansen and Racine [Hansen, B.E., Racine, J., 2012. Jackknife model averaging. Journal of Econometrics 167, 38–46] developed a jackknife model averaging (JMA) estimator, which has an important advantage over its competitors in that it achieves the lowest possible asymptotic squared error under heteroscedastic errors. In this paper, we broaden Hansen and Racine’s scope of analysis to encompass models with (i) a non-diagonal error covariance structure, and (ii) lagged dependent variables, thus allowing for dependent data. We show that under these set-ups, the JMA estimator is asymptotically optimal by a criterion equivalent to that used by Hansen and Racine. A Monte Carlo study demonstrates the finite sample performance of the JMA estimator in a variety of model settings.     

Least squares model averaging by Mallows criterion

This paper is in response to a recent paper by Hansen (2007) who proposed an optimal model average estimator with weights selected by minimizing a Mallows criterion. The main contribution of Hansen’s paper is a demonstration that the Mallows criterion is asymptotically equivalent to the squared error, so the model average estimator that minimizes the Mallows criterion also minimizes the squared error in large samples. We are concerned with two assumptions that accompany Hansen’s approach. The first is the assumption that the approximating models are strictly nested in a way that depends on the ordering of regressors. Often there is no clear basis for the ordering and the approach does not permit non-nested models which are more realistic from a practical viewpoint. Second, for the optimality result to hold the model weights are required to lie within a special discrete set. In fact, Hansen noted both difficulties and called for extensions of the proof techniques. We provide an alternative proof which shows that the result on the optimality of the Mallows criterion in fact holds for continuous model weights and under a non-nested set-up that allows any linear combination of regressors in the approximating models that make up the model average estimator. These results provide a stronger theoretical basis for the use of the Mallows criterion in model averaging by strengthening existing findings.     

Averaging estimators for autoregressions with a near unit root

This paper uses local-to-unity theory to evaluate the asymptotic mean-squared error (AMSE) and forecast expected squared error from least-squares estimation of an autoregressive model with a root close to unity. We investigate unconstrained estimation, estimation imposing the unit root constraint, pre-test estimation, model selection estimation, and model average estimation. We find that the asymptotic risk depends only on the local-to-unity parameter, facilitating simple graphical comparisons. Our results strongly caution against pre-testing. Strong evidence supports averaging based on Mallows weights. In particular, our Mallows averaging method has uniformly and substantially smaller risk than the conventional unconstrained estimator, and this holds for autoregressive roots far from unity. Our averaging estimator is a new approach to forecast combination.     

Finite sample properties of maximum likelihood estimator in spatial models

We investigate the finite sample properties of the maximum likelihood estimator for the spatial autoregressive model. A stochastic expansion of the score function is used to develop the second-order bias and mean squared error of the maximum likelihood estimator. We show that the results can be expressed in terms of the expectations of cross products of quadratic forms, or ratios of quadratic forms in a normal vector which can be evaluated using the top order invariant polynomial. Our numerical calculations demonstrate that the second-order behaviors of the maximum likelihood estimator depend on the degree of sparseness of the weights matrix.     

Model averaging based on James–Stein estimators

Existing model averaging methods are generally based on ordinary least squares (OLS) estimators. However, it is well known that the James–Stein (JS) estimator dominates the OLS estimator under quadratic loss, provided that the dimension of coefficient is larger than two. Thus, we focus on model averaging based on JS estimators instead of OLS estimators. We develop a weight choice method and prove its asymptotic optimality. A simulation experiment shows promising results for the proposed model average estimator.     

On kernel nonparametric regression designed for complex survey data

In this article, we consider nonparametric regression analysis between two variables when data are sampled through a complex survey. While nonparametric regression analysis has been widely used with data that may be assumed to be generated from independently and identically distributed (iid) random variables, the methods and asymptotic analyses established for iid data need to be extended in the framework of complex survey designs. Local polynomial regression estimators are studied, which include as particular cases design-based versions of the Nadaraya–Watson estimator and of the local linear regression estimator. In this paper, special emphasis is given to the local linear regression estimator. Our estimators incorporate both the sampling weights and the kernel weights. We derive the asymptotic mean squared error (MSE) of the kernel estimators using a combined inference framework, and as a corollary consistency of the estimators is deduced. Selection of a bandwidth is necessary for the resulting estimators; an optimal bandwidth can be determined, according to the MSE criterion in the combined mode of inference. Simulation experiments are conducted to illustrate the proposed methodology and an application with the Canadian survey of labour and income dynamics is presented.     

VAR forecasting under misspecification

The paper considers multi-step forecasting of a stationary vector process under a quadratic loss function with a collection of finite-order vector autoregressions (VAR). Under severe misspecification it is preferable to use the multi-step loss function also for parameter estimation. We propose a modification to Shibata's (Ann. Statist. 8 (1980) 147) final prediction error criterion to jointly choose the VAR lag order and one of two predictors: the maximum likelihood estimator plug-in predictor or the loss function estimator plug-in predictor. A Monte Carlo experiment illustrates the theoretical results and documents the empirical performance of the selection criterion.     

Semiparametric deconvolution with unknown error variance

Deconvolution is a useful statistical technique for recovering an unknown density in the presence of measurement error. Typically, the method hinges on stringent assumptions about the nature of the measurement error, more specifically, that the distribution is entirely known. We relax this assumption in the context of a regression error component model and develop an estimator for the unknown density. We show semi-uniform consistency of the estimator and provide an application to the stochastic frontier model.     

Akaike's Information Criterion,Cp and Estimators of Loss for Elliptically Symmetric Distributions

In this article, we develop a modern perspective on Akaike's information criterion and Mallows's C_p for model selection, and propose generalisations to spherically and elliptically symmetric distributions. Despite the differences in their respective motivation, C_p and Akaike's information criterion are equivalent in the special case of Gaussian linear regression. In this case, they are also equivalent to a third criterion, an unbiased estimator of the quadratic prediction loss, derived from loss estimation theory. We then show that the form of the unbiased estimator of the quadratic prediction loss under a Gaussian assumption still holds under a more general distributional assumption, the family of spherically symmetric distributions. One of the features of our results is that our criterion does not rely on the specificity of the distribution, but only on its spherical symmetry. The same kind of criterion can be derived for a family of elliptically contoured distribution, which allows correlations, when considering the invariant loss. More specifically, the unbiasedness property is relative to a distribution associated to the original density.     

Pre-testing for linear restrictions in a regression model with spherically symmetric disturbances

In this paper we derive the exact risk (under quadratic loss) of pre-test estimators of the prediction vector and of the error variance of a linear regression model with spherically symmetric disturbances. The pre-test in question is one of the validity of a set of exact linear restrictions on the model's coefficient vector. We demonstrate how the known results for the model with normal disturbances can be extended to this broader case. We also show that the critical value of unity results in a minimum of the risk of the pre-test estimator of the error variance. To illustrate the results we assume multivariate Student-t regression disturbances and numerically evaluate the derived expressions.     

A simple estimator for nonlinear error in variable models

We propose a simple estimator for nonlinear method of moment models with measurement error of the classical type when no additional data, such as validation data or double measurements, are available. We assume that the marginal distributions of the measurement errors are Laplace (double exponential) with zero means and unknown variances and the measurement errors are independent of the latent variables and are independent of each other. Under these assumptions, we derive simple revised moment conditions in terms of the observed variables. They are used to make inference about the model parameters and the variance of the measurement error. The results of this paper show that the distributional assumption on the measurement errors can be used to point identify the parameters of interest. Our estimator is a parametric method of moments estimator that uses the revised moment conditions and hence is simple to compute. Our estimation method is particularly useful in situations where no additional data are available, which is the case in many economic data sets. Simulation study demonstrates good finite sample properties of our proposed estimator. We also examine the performance of the estimator in the case where the error distribution is misspecified.     

Subsampling realised kernels

In a recent paper we have introduced the class of realised kernel estimators of the increments of quadratic variation in the presence of noise. We showed that this estimator is consistent and derived its limit distribution under various assumptions on the kernel weights. In this paper we extend our analysis, looking at the class of subsampled realised kernels and we derive the limit theory for this class of estimators. We find that subsampling is highly advantageous for estimators based on discontinuous kernels, such as the truncated kernel. For kinked kernels, such as the Bartlett kernel, we show that subsampling is impotent, in the sense that subsampling has no effect on the asymptotic distribution. Perhaps surprisingly, for the efficient smooth kernels, such as the Parzen kernel, we show that subsampling is harmful as it increases the asymptotic variance. We also study the performance of subsampled realised kernels in simulations and in empirical work.     

Long difference instrumental variables estimation for dynamic panel models with fixed effects

This paper proposes a new instrumental variables estimator for a dynamic panel model with fixed effects with good bias and mean squared error properties even when identification of the model becomes weak near the unit circle. We adopt a weak instrument asymptotic approximation to study the behavior of various estimators near the unit circle. We show that an estimator based on long differencing the model is much less biased than conventional implementations of the GMM estimator for the dynamic panel model. We also show that under the weak instrument approximation conventional GMM estimators are dominated in terms of mean squared error by an estimator with far less moment conditions. The long difference (LD) estimator mimics the infeasible optimal procedure through its reliance on a small set of moment conditions.     

Spline regression for hazard rate estimation when data are censored and measured with error

In this paper, we study an estimation problem where the variables of interest are subject to both right censoring and measurement error. In this context, we propose a nonparametric estimation strategy of the hazard rate, based on a regression contrast minimized in a finite‐dimensional functional space generated by splines bases. We prove a risk bound of the estimator in terms of integrated mean square error and discuss the rate of convergence when the dimension of the projection space is adequately chosen. Then we define a data‐driven criterion of model selection and prove that the resulting estimator performs an adequate compromise. The method is illustrated via simulation experiments that show that the strategy is successful.     

Mean square error estimation in multi-stage sampling

Summary: Suppose for a homogeneous linear unbiased function of the sampled first stage unit (fsu)-values taken as an estimator of a survey population total, the sampling variance is expressed as a homogeneous quadratic function of the fsu-values. When the fsu-values are not ascertainable but unbiased estimators for them are separately available through sampling in later stages and substituted into the estimator, Raj (1968) gave a simple variance estimator formula for this multi-stage estimator of the population total. He requires that the variances of the estimated fsu-values in sampling at later stages and their unbiased estimators are available in certain `simple forms'. For the same set-up Rao (1975) derived an alternative variance estimator when the later stage sampling variances have more ‘complex forms’. Here we pursue with Raj's (1968) simple forms to derive a few alternative variance and mean square error estimators when the condition of homogeneity or unbiasedness in the original estimator of the total is relaxed and the variance of the original estimator is not expressed as a quadratic form.  We illustrate a particular three-stage sampling strategy and present a simulation-based numerical exercise showing the relative efficacies of two alternative variance estimators. Received: 19 February 1999     

Estimating quadratic variation consistently in the presence of endogenous and diurnal measurement error

We propose an econometric model that captures the effects of market microstructure on a latent price process. In particular, we allow for correlation between the measurement error and the return process and we allow the measurement error process to have a diurnal heteroskedasticity. We propose a modification of the TSRV estimator of quadratic variation. We show that this estimator is consistent, with a rate of convergence that depends on the size of the measurement error, but is no worse than n^−1/6$n^{-1/6}$. We investigate in simulation experiments the finite sample performance of various proposed implementations.     

Focused information criterion and model averaging in censored quantile regression

In this paper, we study model selection and model averaging for quantile regression with randomly right censored response. We consider a semi-parametric censored quantile regression model without distribution assumptions. Under general conditions, a focused information criterion and a frequentist model averaging estimator are proposed, and theoretical properties of the proposed methods are established. The performances of the procedures are illustrated by extensive simulations and the primary biliary cirrhosis data.     

Least-squares forecast averaging

This paper proposes forecast combination based on the method of Mallows Model Averaging (MMA). The method selects forecast weights by minimizing a Mallows criterion. This criterion is an asymptotically unbiased estimate of both the in-sample mean-squared error (MSE) and the out-of-sample one-step-ahead mean-squared forecast error (MSFE). Furthermore, the MMA weights are asymptotically mean-square optimal in the absence of time-series dependence. We show how to compute MMA weights in forecasting settings, and investigate the performance of the method in simple but illustrative simulation environments. We find that the MMA forecasts have low MSFE and have much lower maximum regret than other feasible forecasting methods, including equal weighting, BIC selection, weighted BIC, AIC selection, weighted AIC, Bates–Granger combination, predictive least squares, and Granger–Ramanathan combination.     

Estimation of bias and standard error of an improved estimator of normal mean

This paper considers an improved estimator of normal mean which is obtained by considering a feasible version of minimum mean squared error estimator. The exact expression for the bias and the mean squared error are fairly complicated and do not provide any guidelines as how to estimate the standard error of improved estimator. As is well known that any estimator without a formula for standard error has little practical utility. We therefore derive unbiased estimators for the bias and mean squared error of the improved estimator. Incidently, they turn out to be minimum variance unbiased estimators. Further, this exercise yields a simple formula for estimating the standard error. Based on the criterion of estimated standard error, the efficiency of the improved estimator with respect to the traditional unbiased estimator (i.e., sample mean) is examined numerically. The relationship with asymptotic standard error is also studied.     

Inference in semiparametric binary response models with interval data

This paper studies the semiparametric binary response model with interval data investigated by Manski and Tamer (2002). In this partially identified model, we propose a new estimator based on MT’s modified maximum score (MMS) method by introducing density weights to the objective function, which allows us to develop asymptotic properties of the proposed set estimator for inference. We show that the density-weighted MMS estimator converges at a nearly cube-root-n rate. We propose an asymptotically valid inference procedure for the identified region based on subsampling. Monte Carlo experiments provide supports to our inference procedure.