A simple approach for the analysis of generalizea linear mixed models

A broad class of generalized linear mixed models, e.g. variance components models for binary data, percentages or count data, will be introduced by incorporating additional random effects into the linear predictor of a generalized linear model structure. Parameters are estimated by a combination of quasi-likelihood and iterated MINQUE (minimum norm quadratic unbiased estimation), the latter being numerically equivalent to REML (restricted, or residual, maximum likelihood). First, conditional upon the additional random effects, observations on a working variable and weights are derived by quasi-likelihood, using iteratively re-weighted least squares. Second, a linear mixed model is fitted to the working variable, employing the weights for the residual error terms, by iterated MINQUE. The latter may be regarded as a least squares procedure applied to squared and product terms of error contrasts derived from the working variable. No full distributional assumptions are needed for estimation. The model may be fitted with standardly available software for weighted regression and REML.     

Restricted maximum likelihood estimation under Eisenhart model Ill

For a balanced two-way mixed model, the maximum likelihood (ML) and restricted ML (REML) estimators of the variance components were obtained and compared under the non-negativity requirements of the variance components by L ee and K apadia (1984). In this note, for a mixed (random blocks) incomplete block model, explicit forms for the REML estimators of variance components are obtained. They are always non-negative and have smaller mean squared error (MSE) than the analysis of variance (AOV) estimators. The asymptotic sampling variances of the maximum likelihood (ML) estimators and the REML estimators are compared and the balanced incomplete block design (BIBD) is considered as a special case. The ML estimators are shown to have smaller asymptotic variances than the REML estimators, but a numerical result in the randomized complete block design (RCBD) demonstrated that the performances of the REML and ML estimators are not much different in the MSE sense.     

Improved forecasting of autoregressive series by weighted least squares approximate REML estimation

Restricted maximum likelihood (REML) estimation has recently been shown to provide less biased estimates in autoregressive series. A simple weighted least squares approximate REML procedure has been developed that is particularly useful for vector autoregressive processes. Here, we compare the forecasts of such processes using both the standard ordinary least squares (OLS) estimates and the new approximate REML estimates. Forecasts based on the approximate REML estimates are found to provide a significant improvement over those obtained using the standard OLS estimates.     

Maximum likelihood and restricted maximum likelihood estimation for a class of Gaussian Markov random fields

This work describes a Gaussian Markov random field model that includes several previously proposed models, and studies properties of its maximum likelihood (ML) and restricted maximum likelihood (REML) estimators in a special case. Specifically, for models where a particular relation holds between the regression and precision matrices of the model, we provide sufficient conditions for existence and uniqueness of ML and REML estimators of the covariance parameters, and provide a straightforward way to compute them. It is found that the ML estimator always exists while the REML estimator may not exist with positive probability. A numerical comparison suggests that for this model ML estimators of covariance parameters have, overall, better frequentist properties than REML estimators.     

On the inefficiency of the restricted maximum likelihood

The restricted maximum likelihood is preferred by many to the full maximum likelihood for estimation with variance component and other random coefficient models, because the variance estimator is unbiased. It is shown that this unbiasedness is accompanied in some balanced designs by an inflation of the mean squared error. An estimator of the cluster‐level variance that is uniformly more efficient than the full maximum likelihood is derived. Estimators of the variance ratio are also studied.     

On the analysis of multivariate growth curves

Growth curve data arise when repeated measurements are observed on a number of individuals with an ordered dimension for occasions. Such data appear frequently in almost all fields in which statistical models are used, for instance in medicine, agriculture and engineering. In medicine, for example, more than one variable is often measured on each occasion. However, analyses are usually based on exploration of repeated measurements of only one variable. The consequence is that the information contained in the between-variables correlation structure will be discarded.  In this study we propose a multivariate model based on the random coefficient regression model for the analysis of growth curve data. Closed-form expressions for the model parameters are derived under the maximum likelihood (ML) and the restricted maximum likelihood (REML) framework. It is shown that in certain situations estimated variances of growth curve parameters are greater for REML. Also a method is proposed for testing general linear hypotheses. One numerical example is provided to illustrate the methods discussed. Received: 22 February 1999     

On Maximum Likelihood Estimation of Dynamic Panel Data Models

We analyse the finite sample properties of maximum likelihood estimators for dynamic panel data models. In particular, we consider transformed maximum likelihood (TML) and random effects maximum likelihood (RML) estimation. We show that TML and RML estimators are solutions to a cubic first‐order condition in the autoregressive parameter. Furthermore, in finite samples both likelihood estimators might lead to a negative estimate of the variance of the individual‐specific effects. We consider different approaches taking into account the non‐negativity restriction for the variance. We show that these approaches may lead to a solution different from the unique global unconstrained maximum. In an extensive Monte Carlo study we find that this issue is non‐negligible for small values of T and that different approaches might lead to different finite sample properties. Furthermore, we find that the Likelihood Ratio statistic provides size control in small samples, albeit with low power due to the flatness of the log‐likelihood function. We illustrate these issues modelling US state level unemployment dynamics.     

Hedonic Housing Prices in Paris: An Unbalanced Spatial Lag Pseudo‐Panel Model with Nested Random Effects

This paper estimates a hedonic housing model based on flats sold in the city of Paris over the period 1990–2003. This is done using maximum likelihood estimation, taking into account the nested structure of the data. Paris is historically divided into 20 arrondissements, each divided into four quartiers (quarters), which in turn contain between 15 and 169 blocks (îlot, in French) per quartier. This is an unbalanced pseudo?panel data containing 156,896 transactions. Despite the richness of the data, many neighborhood characteristics are not observed, and we attempt to capture these neighborhood spillover effects using a spatial lag model. Using likelihood ratio tests, we find significant spatial lag effects as well as significant nested random error effects. The empirical results show that the hedonic housing estimates and the corresponding marginal effects are affected by taking into account the nested aspects of the Paris housing data as well as the spatial neighborhood effects.Copyright © 2014 John Wiley & Sons, Ltd.     

Forecasting With the Standardized Self‐Perturbed Kalman Filter

We propose and study the finite‐sample properties of a modified version of the self‐perturbed Kalman filter of Park and Jun (Electronics Letters 1992; 28 : 558–559) for the online estimation of models subject to parameter instability. The perturbation term in the updating equation of the state covariance matrix is weighted by the estimate of the measurement error variance. This avoids the calibration of a design parameter as the perturbation term is scaled by the amount of uncertainty in the data. It is shown by Monte Carlo simulations that this perturbation method is associated with a good tracking of the dynamics of the parameters compared to other online algorithms and to classical and Bayesian methods. The standardized self‐perturbed Kalman filter is adopted to forecast the equity premium on the S&P 500 index under several model specifications, and determines the extent to which realized variance can be used to predict excess returns. Copyright © 2016 John Wiley & Sons, Ltd.     

Estimates of relative risk

Summary For the two-sample problem with proportional hazard functions, we consider estimation of the constant of proportionality, known as relative risk, using complete uncensored data. For this very special case of Cox’s (1972) regression model for survival data, we find a two-step estimate which is asymptotically equivalent to Cox’s partial likelihood estimate, and we show that both estimates are asymptotically optimal (in the sense of minimum asymptotic variance) among all regular rank estimates of relative risk. Supported by NSF grant 81-00748.     

Asymptotics of SIMEX-based variance estimation

In this paper, we study the asymptotic properties of simulation extrapolation (SIMEX) based variance estimation that was proposed by Wang et al. (J R Stat Soc Series B 71:425–445, 2009). We first investigate the asymptotic normality of the parameter estimator in general parametric variance function and the local linear estimator for nonparametric variance function when permutation SIMEX (PSIMEX) is used. The asymptotic optimal bandwidth selection with respect to approximate mean integrated squared error (AMISE) for nonparametric estimator is also studied. We finally discuss constructing confidence intervals/bands of the parameter/function of interest. Other than applying the asymptotic results so that normal approximation can be used, we recommend a nonparametric Monte Carlo algorithm to avoid estimating the asymptotic variance of estimator. Simulation studies are carried out for illustration.     

Practical Problems with Reduced‐rank ML Estimators for Cointegration Parameters and a Simple Alternative*

Johansen's reduced‐rank maximum likelihood (ML) estimator for cointegration parameters in vector error correction models is known to produce occasional extreme outliers. Using a small monetary system and German data we illustrate the practical importance of this problem. We also consider an alternative generalized least squares (GLS) system estimator which has better properties in this respect. The two estimators are compared in a small simulation study. It is found that the GLS estimator can indeed be an attractive alternative to ML estimation of cointegration parameters.     

Analysis of a mixed model for ordinal data by iterative re-weighted REML

An estimation procedure will be presented for a class of threshold models for ordinal data. These models may include both fixed and random effects with associated components of variance on an underlying scale. The residual error distribution on the underlying scale may be rendered greater flexibility by introducing additional shape parameters, e.g. a kurtosis parameter or parameters to model heterogeneous residual variances as a function of factors and covariates. The estimation procedure is an extension of an iterative re-weighted restricted maximum likelihood procedure, originally developed for generalized linear mixed models. This procedure will be illustrated with a practical problem involving damage to potato tubers and with data from animal breeding and medical research from the literature.     

Poisson Regression for Clustered Data

We compare five methods for parameter estimation of a Poisson regression model for clustered data: (1) ordinary (naive) Poisson regression (OP), which ignores intracluster correlation, (2) Poisson regression with fixed cluster‐specific intercepts (FI), (3) a generalized estimating equations (GEE) approach with an equi‐correlation matrix, (4) an exact generalized estimating equations (EGEE) approach with an exact covariance matrix, and (5) maximum likelihood (ML). Special attention is given to the simplest case of the Poisson regression with a cluster‐specific intercept random when the asymptotic covariance matrix is obtained in closed form. We prove that methods 1–5, except GEE, produce the same estimates of slope coefficients for balanced data (an equal number of observations in each cluster and the same vectors of covariates). All five methods lead to consistent estimates of slopes but have different efficiency for unbalanced data design. It is shown that the FI approach can be derived as a limiting case of maximum likelihood when the cluster variance increases to infinity. Exact asymptotic covariance matrices are derived for each method. In terms of asymptotic efficiency, the methods split into two groups: OP & GEE and EGEE & FI & ML. Thus, contrary to the existing practice, there is no advantage in using GEE because it is substantially outperformed by EGEE and FI. In particular, EGEE does not require integration and is easy to compute with the asymptotic variances of the slope estimates close to those of the ML.     

A simple estimator for nonlinear error in variable models

We propose a simple estimator for nonlinear method of moment models with measurement error of the classical type when no additional data, such as validation data or double measurements, are available. We assume that the marginal distributions of the measurement errors are Laplace (double exponential) with zero means and unknown variances and the measurement errors are independent of the latent variables and are independent of each other. Under these assumptions, we derive simple revised moment conditions in terms of the observed variables. They are used to make inference about the model parameters and the variance of the measurement error. The results of this paper show that the distributional assumption on the measurement errors can be used to point identify the parameters of interest. Our estimator is a parametric method of moments estimator that uses the revised moment conditions and hence is simple to compute. Our estimation method is particularly useful in situations where no additional data are available, which is the case in many economic data sets. Simulation study demonstrates good finite sample properties of our proposed estimator. We also examine the performance of the estimator in the case where the error distribution is misspecified.     

Forecasting the volatility of asset returns: The informational gains from option prices

A new class of forecasting models is proposed that extends the realized GARCH class of models through the inclusion of option prices to forecast the variance of asset returns. The VIX is used to approximate option prices, resulting in a set of cross-equation restrictions on the model’s parameters. The full model is characterized by a nonlinear system of three equations containing asset returns, the realized variance, and the VIX, with estimation of the parameters based on maximum likelihood methods. The forecasting properties of the new class of forecasting models, as well as a number of special cases, are investigated and applied to forecasting the daily S&P500 index realized variance using intra-day and daily data from September 2001 to November 2017. The forecasting results provide strong support for including the realized variance and the VIX to improve variance forecasts, with linear conditional variance models performing well for short-term one-day-ahead forecasts, whereas log-linear conditional variance models tend to perform better for intermediate five-day-ahead forecasts.     

A minimum chi-squared method for indirect parameter estimation from Poisson data

In this paper, we introduce a new algorithm for estimating non-negative parameters from Poisson observations of a linear transformation of the parameters. The proposed objective function fits both a weighted least squares (WLS) and a minimum χ² estimation framework, and results in a convex optimization problem. Unlike conventional WLS methods, the weights do not need to be estimated from the datas, but are incorporated in the objective function. The iterative algorithm is derived from an alternating projection procedure in which "distance" is determined by the chi-squared test statistic, which is interpreted as a measure of the discrepancy between two distributions. This may be viewed as an alternative to the Kullback-Leibler divergence which corresponds to the maximum likelihood (ML) estimation. The algorithm is similar in form to, and shares many properties with, the expectation maximization algorithm for ML estimation. In particular, we show that every limit point of the algorithm is an estimator, and the sequence of projected (by the linear transformation into the data space) means converge. Despite the similarities, we show that the new estimators are quite distinct from ML estimators, and obtain conditions under which they are identical.     

Modeling microstructure price dynamics with symmetric Hawkes and diffusion model using ultra-high-frequency stock data

This study examines the theoretical and empirical perspectives of the symmetric Hawkes model of the price tick structure. Combined with the maximum likelihood estimation, the model provides a proper method of volatility estimation specialized in ultra-high-frequency analysis. Empirical studies based on the model using the ultra-high-frequency data of stocks in the S&P 500 are performed. The performance of the volatility measure, intraday estimation, and the dynamics of the parameters are discussed. A new approach of diffusion analogy to the symmetric Hawkes model is proposed with the distributional properties very close to the Hawkes model. As a diffusion process, the model provides more analytical simplicity when computing the variance formula, incorporating skewness and examining the probabilistic property. An estimation of the diffusion model is performed using the simulated maximum likelihood method and shows similar patterns to the Hawkes model.     

Preliminary test estimators of the parameters of simple linear model with measurement error

In this paper, we discuss the properties of preliminary test estimators (PTE) of the parameters of simple linear model with measurement error (ME model) when the slope of the linear model is suspected to be zero. Expressions of the bias, MSE and efficiencies are obtained under conditional as well as unconditional situations with known reliability coefficient. Conditional model results are compared to the standard model without measurement error. We also provide the unconditional model analysis in finite samples. Asymptotic theory under local alternatives is developed when the variance of measurement error or the ratio of the variance of the model error relative to the variance of the measurement error is known. Asymptotic expressions of bias and MSE of the estimators along with their efficiencies are obtained. In every case, it is shown that the measurement error tend to increase the variability of the estimators compared to the estimators without measurement error. Graphs and tables are provided to see these results and to determine optimum level of significance for minimum guaranteed efficiency. Received October 2001 RID="*" ID="*"  A. K. Md. E. Saleh is a Distinguished Research Professor and H. M. Kim is a Ph.D. candidate in the School of Mathematics and Statistics, Carleton University, Ottawa. Acknowledgment. The authors gratefully acknowledge the constructive suggestion of the referees to improve the paper. The research is supported by NSERC grant A3088.     

Normal mixture GARCH(1,1): applications to exchange rate modelling

Some recent specifications for GARCH error processes explicitly assume a conditional variance that is generated by a mixture of normal components, albeit with some parameter restrictions. This paper analyses the general normal mixture GARCH(1,1) model which can capture time variation in both conditional skewness and kurtosis. A main focus of the paper is to provide evidence that, for modelling exchange rates, generalized two‐component normal mixture GARCH(1,1) models perform better than those with three or more components, and better than symmetric and skewed Student's t‐GARCH models. In addition to the extensive empirical results based on simulation and on historical data on three US dollar foreign exchange rates (British pound, euro and Japanese yen), we derive: expressions for the conditional and unconditional moments of all models; parameter conditions to ensure that the second and fourth conditional and unconditional moments are positive and finite; and analytic derivatives for the maximum likelihood estimation of the model parameters and standard errors of the estimates. Copyright © 2006 John Wiley & Sons, Ltd.