Practical Problems with Reduced‐rank ML Estimators for Cointegration Parameters and a Simple Alternative*

Johansen's reduced‐rank maximum likelihood (ML) estimator for cointegration parameters in vector error correction models is known to produce occasional extreme outliers. Using a small monetary system and German data we illustrate the practical importance of this problem. We also consider an alternative generalized least squares (GLS) system estimator which has better properties in this respect. The two estimators are compared in a small simulation study. It is found that the GLS estimator can indeed be an attractive alternative to ML estimation of cointegration parameters.     

Quasi-maximum likelihood estimation of volatility with high frequency data

This paper investigates the properties of the well-known maximum likelihood estimator in the presence of stochastic volatility and market microstructure noise, by extending the classic asymptotic results of quasi-maximum likelihood estimation. When trying to estimate the integrated volatility and the variance of noise, this parametric approach remains consistent, efficient and robust as a quasi-estimator under misspecified assumptions. Moreover, it shares the model-free feature with nonparametric alternatives, for instance realized kernels, while being advantageous over them in terms of finite sample performance. In light of quadratic representation, this estimator behaves like an iterative exponential realized kernel asymptotically. Comparisons with a variety of implementations of the Tukey–Hanning₂ kernel are provided using Monte Carlo simulations, and an empirical study with the Euro/US Dollar future illustrates its application in practice.     

Optimal estimation under nonstandard conditions

We analyze optimality properties of maximum likelihood (ML) and other estimators when the problem does not necessarily fall within the locally asymptotically normal (LAN) class, therefore covering cases that are excluded from conventional LAN theory such as unit root nonstationary time series. The classical Hájek–Le Cam optimality theory is adapted to cover this situation. We show that the expectation of certain monotone “bowl-shaped” functions of the squared estimation error are minimized by the ML estimator in locally asymptotically quadratic situations, which often occur in nonstationary time series analysis when the LAN property fails. Moreover, we demonstrate a direct connection between the (Bayesian property of) asymptotic normality of the posterior and the classical optimality properties of ML estimators.     

Sparse estimators and the oracle property,or the return of Hodges’ estimator

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360; 2002. Variable selection for Cox's proportional hazards model and frailty model. Annals of Statistics 30, 74–99; 2004. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of the American Statistical Association 99, 710–723] which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360]. We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.     

Likelihood estimation and inference in threshold regression

This paper studies likelihood-based estimation and inference in parametric discontinuous threshold regression models with i.i.d. data. The setup allows heteroskedasticity and threshold effects in both mean and variance. By interpreting the threshold point as a “middle” boundary of the threshold variable, we find that the Bayes estimator is asymptotically efficient among all estimators in the locally asymptotically minimax sense. In particular, the Bayes estimator of the threshold point is asymptotically strictly more efficient than the left-endpoint maximum likelihood estimator and the newly proposed middle-point maximum likelihood estimator. Algorithms are developed to calculate asymptotic distributions and risk for the estimators of the threshold point. The posterior interval is proved to be an asymptotically valid confidence interval and is attractive in both length and coverage in finite samples.     

Pseudo-maximum likelihood estimation in two classes of semiparametric diffusion models

Two classes of semiparametric diffusion models are considered, where either the drift or the diffusion term is parameterized, while the other term is left unspecified. We propose a pseudo-maximum likelihood estimator (PMLE) of the parametric component that maximizes the likelihood with a preliminary estimator of the unspecified term plugged in. It is demonstrated how models and estimators can be used in a two-step specification testing strategy of semiparametric and fully parametric models, and shown that approximate/simulated versions of the PMLE inherit the properties of the actual but infeasible estimator. A simulation study investigates the finite sample performance of the PMLE.     

Estimation of dynamic models with nonparametric simulated maximum likelihood

We propose an easy-to-implement simulated maximum likelihood estimator for dynamic models where no closed-form representation of the likelihood function is available. Our method can handle any simulable model without latent dynamics. Using simulated observations, we nonparametrically estimate the unknown density by kernel methods, and then construct a likelihood function that can be maximized. We prove that this nonparametric simulated maximum likelihood (NPSML) estimator is consistent and asymptotically efficient. The higher-order impact of simulations and kernel smoothing on the resulting estimator is also analyzed; in particular, it is shown that the NPSML does not suffer from the usual curse of dimensionality associated with kernel estimators. A simulation study shows good performance of the method when employed in the estimation of jump-diffusion models.     

MCMC maximum likelihood for latent state models

This paper develops a pure simulation-based approach for computing maximum likelihood estimates in latent state variable models using Markov Chain Monte Carlo methods (MCMC). Our MCMC algorithm simultaneously evaluates and optimizes the likelihood function without resorting to gradient methods. The approach relies on data augmentation, with insights similar to simulated annealing and evolutionary Monte Carlo algorithms. We prove a limit theorem in the degree of data augmentation and use this to provide standard errors and convergence diagnostics. The resulting estimator inherits the sampling asymptotic properties of maximum likelihood. We demonstrate the approach on two latent state models central to financial econometrics: a stochastic volatility and a multivariate jump-diffusion models. We find that convergence to the MLE is fast, requiring only a small degree of augmentation.     

Second-order properties of estimators of serial correlation from regression residuals

Second-order properties of estimators and tests offer a way of choosinf among aymptotically equivalent procedures. This paper studies the second-order terms of two estimators of serial correlation in the linear model. Using these second-order approximations, the maximum likelihood estimator is judge to be superior in terms of bias and variance. A small Monte Carlo experiment is done to assess the accuracy of the results.     

Estimating and testing linkage disequilibrium from sib data

We consider estimation and testing of linkage equilibrium from genotypic data on a random sample of sibs, such as monozygotic and dizygotic twins. We compute the maximum likelihood estimator with an EM‐algorithm and a likelihood ratio statistic that takes the family structure into account. As we are interested in applying this to twin data we also allow observations on single children, so that monozygotic twins can be included. We allow non‐zero recombination fraction between the loci of interest, so that linkage disequilibrium between both linked and unlinked loci can be tested. The EM‐algorithm for computing the maximum likelihood estimator of the haplotype frequencies and the likelihood ratio test‐statistic, are described in detail. It is shown that the usual estimators of haplotype frequencies based on ignoring that the sibs are related are inefficient, and the likelihood ratio test for testing that the loci are in linkage disequilibrium.     

GMM estimation of spatial autoregressive models with moving average disturbances

In this paper, we introduce the one-step generalized method of moments (GMM) estimation methods considered in Lee (2007a) and Liu, Lee, and Bollinger (2010) to spatial models that impose a spatial moving average process for the disturbance term. First, we determine the set of best linear and quadratic moment functions for GMM estimation. Second, we show that the optimal GMM estimator (GMME) formulated from this set is the most efficient estimator within the class of GMMEs formulated from the set of linear and quadratic moment functions. Our analytical results show that the one-step GMME can be more efficient than the quasi maximum likelihood (QMLE), when the disturbance term is simply i.i.d. With an extensive Monte Carlo study, we compare its finite sample properties against the MLE, the QMLE and the estimators suggested in Fingleton (2008a).     

Strong Consistency of the Maximum Likelihood Estimator in Generalized Linear and Nonlinear Mixed-Effects Models

Generalized linear and nonlinear mixed-effects models are used extensively in biomedical, social, and agricultural sciences. The statistical analysis of these models is based on the asymptotic properties of the maximum likelihood estimator. However, it is usually assumed that the maximum likelihood estimator is consistent, without providing a proof. A rigorous proof of the consistency by verifying conditions from existing results can be very difficult due to the integrated likelihood. In this paper, we present some easily verifiable conditions for the strong consistency of the maximum likelihood estimator in generalized linear and nonlinear mixed-effects models. Based on this result, we prove that the maximum likelihood estimator is consistent for some frequently used models such as mixed-effects logistic regression models and growth curve models.     

Efficiency of iterative estimators in the regression model with AR(1) disturbances

The mean square error approximation method of Nagar is applied to the iterated Prais-Winsten and (iterated) maximum likelihood estimators of regression coefficients in the model with AR(1) disturbances. Their mean square errors are found to equal that of the two-stage Prais-Winsten estimator at the second-order level of approximation.     

Estimation of the trace of the scaled covariance matrix of a multivariate t-model using a known information

The trace of the scaled covariance matrix of the multivariate t-distribution is considered for estimation using a power transformation. The proposed estimator always dominates the usual maximum likelihood estimator in the sense of having smaller risk under a quadratic loss function. The dominance behaviour is proved analytically as well as computationally by using Monte-Carlo simulation.     

Time and causality: A Monte Carlo assessment of the timing-of-events approach

We present new Monte Carlo evidence regarding the feasibility of separating causality from selection within non-experimental duration data, by means of the non-parametric maximum likelihood estimator (NPMLE). Key findings are: (i) the NPMLE is extremely reliable, and it accurately separates the causal effects of treatment and duration dependence from sorting effects, almost regardless of the true unobserved heterogeneity distribution; (ii) the NPMLE is normally distributed, and standard errors can be computed directly from the optimally selected model; and (iii) unjustified restrictions on the heterogeneity distribution, e.g., in terms of a pre-specified number of support points, may cause substantial bias.     

On the asymptotic optimality of the LIML estimator with possibly many instruments

We consider the estimation of the coefficients of a linear structural equation in a simultaneous equation system when there are many instrumental variables. We derive some asymptotic properties of the limited information maximum likelihood (LIML) estimator when the number of instruments is large; some of these results are new as well as old, and we relate them to results in some recent studies. We have found that the variance of the limiting distribution of the LIML estimator and its modifications often attain the asymptotic lower bound when the number of instruments is large and the disturbance terms are not necessarily normally distributed, that is, for the micro-econometric models of some cases recently called many instruments and many weak instruments.     

Bayesian point estimation of the cointegration space

A neglected aspect of the otherwise fairly well developed Bayesian analysis of cointegration is point estimation of the cointegration space. It is pointed out here that, due to the well known non-identification of the cointegration vectors, the parameter space is not Euclidean and the loss functions underlying the conventional Bayes estimators are therefore questionable. We present a Bayes estimator of the cointegration space which takes the curved geometry of the parameter space into account. This estimate has the interpretation of being the posterior mean cointegration space and is invariant to the order of the time series, a property not shared with many of the Bayes estimators in the cointegration literature. An overall measure of cointegration space uncertainty is also proposed. Australian interest rate data are used for illustration. A small simulation study shows that the new Bayes estimator compares favorably to the maximum likelihood estimator.     

Semiparametric maximum likelihood estimation in Cox proportional hazards model with covariate measurement errors

This paper studies semiparametric maximum likelihood estimators in the Cox proportional hazards model with covariate error, assuming that the conditional distribution of the true covariate given the surrogate is known. We show that the estimator of the regression coefficient is asymptotically normal and efficient, its covariance matrix can be estimated consistently by differentiation of the profile likelihood, and the likelihood ratio test is asymptotically chi-squared. We also provide efficient algorithms for the computations of the semiparametric maximum likelihood estimate and the profile likelihood. The performance of this method is successfully demonstrated in simulation studies.     

Estimating a generalized correlation coefficient for a generalized bivariate probit model

In this paper we consider semiparametric estimation of a generalized correlation coefficient in a generalized bivariate probit model. The generalized correlation coefficient provides a simple summary statistic measuring the relationship between the two binary decision processes in a general framework. Our semiparametric estimation procedure consists of two steps, combining semiparametric estimators for univariate binary choice models with the method of maximum likelihood for the bivariate probit model with nonparametrically generated regressors. The estimator is shown to be consistent and asymptotically normal. The estimator performs well in our simulation study.     

VAR forecasting under misspecification

The paper considers multi-step forecasting of a stationary vector process under a quadratic loss function with a collection of finite-order vector autoregressions (VAR). Under severe misspecification it is preferable to use the multi-step loss function also for parameter estimation. We propose a modification to Shibata's (Ann. Statist. 8 (1980) 147) final prediction error criterion to jointly choose the VAR lag order and one of two predictors: the maximum likelihood estimator plug-in predictor or the loss function estimator plug-in predictor. A Monte Carlo experiment illustrates the theoretical results and documents the empirical performance of the selection criterion.