Mean squared error of ridge estimators in logistic regression

It is well known that the maximum likelihood estimator (MLE) is inadmissible when estimating the multidimensional Gaussian location parameter. We show that the verdict is much more subtle for the binary location parameter. We consider this problem in a regression framework by considering a ridge logistic regression (RR) with three alternative ways of shrinking the estimates of the event probabilities. While it is shown that all three variants reduce the mean squared error (MSE) of the MLE, there is at the same time, for every amount of shrinkage, a true value of the location parameter for which we are overshrinking, thus implying the minimaxity of the MLE in this family of estimators. Little shrinkage also always reduces the MSE of individual predictions for all three RR estimators; however, only the naive estimator that shrinks toward 1/2 retains this property for any generalized MSE (GMSE). In contrast, for the two RR estimators that shrink toward the common mean probability, there is always a GMSE for which even a minute amount of shrinkage increases the error. These theoretical results are illustrated on a numerical example. The estimators are also applied to a real data set, and practical implications of our results are discussed.     

Shrinkage Estimation Strategies in Generalised Ridge Regression Models: Low/High-Dimension Regime

In this study, we suggest pretest and shrinkage methods based on the generalised ridge regression estimation that is suitable for both multicollinear and high-dimensional problems. We review and develop theoretical results for some of the shrinkage estimators. The relative performance of the shrinkage estimators to some penalty methods is compared and assessed by both simulation and real-data analysis. We show that the suggested methods can be accounted as good competitors to regularisation techniques, by means of a mean squared error of estimation and prediction error. A thorough comparison of pretest and shrinkage estimators based on the maximum likelihood method to the penalty methods. In this paper, we extend the comparison outlined in his work using the least squares method for the generalised ridge regression.     

Shrinkage estimation of the exponentiated Weibull regression model for time-to-event data

The exponentiated Weibull distribution is a convenient alternative to the generalized gamma distribution to model time-to-event data. It accommodates both monotone and nonmonotone hazard shapes, and flexible enough to describe data with wide ranging characteristics. It can also be used for regression analysis of time-to-event data. The maximum likelihood method is thus far the most widely used technique for inference, though there is a considerable body of research of improving the maximum likelihood estimators in terms of asymptotic efficiency. For example, there has recently been considerable attention on applying James–Stein shrinkage ideas to parameter estimation in regression models. We propose nonpenalty shrinkage estimation for the exponentiated Weibull regression model for time-to-event data. Comparative studies suggest that the shrinkage estimators outperform the maximum likelihood estimators in terms of statistical efficiency. Overall, the shrinkage method leads to more accurate statistical inference, a fundamental and desirable component of statistical theory.     

Penalty and related estimation strategies in the spatial error model

Spatial autoregressive models are powerful tools in the analysis of data sets from diverse scientific areas of research such as econometrics, plant species richness, cancer mortality rates, image processing, analysis of the functional Magnetic Resonance Imaging (fMRI) data, and many more. An important class in the host of spatial autoregressive models is the class of spatial error models in which spatially lagged error terms are assumed. In this paper, we propose efficient shrinkage and penalty estimators for the regression coefficients of the spatial error model. We carry out asymptotic as well as simulation analyses to illustrate the gain in efficiency achieved by these new estimators. Furthermore, we apply the new methodology to housing prices data and provide a bootstrap approach to compute prediction errors of the new estimators.     

Linear and nonlinear regression with stable errors

In this paper we describe methods and evaluate programs for linear regression by maximum likelihood when the errors have a heavy tailed stable distribution. The asymptotic Fisher information matrix for both the regression coefficients and the error distribution parameters are derived, giving large sample confidence intervals for all parameters. Simulated examples are shown where the errors are stably distributed and also where the errors are heavy tailed but are not stable, as well as a real example using financial data. The results are then extended to nonlinear models and to non-homogeneous error terms.     

Some identification and estimation results for regression models with stochastically varying coefficients

Although various theoretical and applied papers have appeared in recent years concerned with the estimation and use of regression models with stochastically varying coefficients, little is available in the literature on the properties of the proposed estimators or the identifiability of the parameters of such models. The present paper derives sufficient conditions under which the maximum likelihood estimator is consistent and asymptotically normal and also provides sufficient conditions for the estimation of regression models with stationary stochastically varying coefficients. In many instances these requirements are found to have simple, intuitively appealing interpretations. Consistency and asymptotic normality is also proven for a two-step estimator and a method suggested by Rosenberg for generating initial estimates.     

Estimation strategies for the regression coefficient parameter matrix in multivariate multiple regression

We consider improved estimation strategies for the parameter matrix in multivariate multiple regression under a general and natural linear constraint. In the context of two competing models where one model includes all predictors and the other restricts variable coefficients to a candidate linear subspace based on prior information, there is a need of combining two estimation techniques in an optimal way. In this scenario, we suggest some shrinkage estimators for the targeted parameter matrix. Also, we examine the relative performances of the suggested estimators in the direction of the subspace and candidate subspace restricted type estimators. We develop a large sample theory for the estimators including derivation of asymptotic bias and asymptotic distributional risk of the suggested estimators. Furthermore, we conduct Monte Carlo simulation studies to appraise the relative performance of the suggested estimators with the classical estimators. The methods are also applied on a real data set for illustrative purposes.     

Nowcasting and forecasting GDP in emerging markets using global financial and macroeconomic diffusion indexes

This paper contributes to the nascent literature on nowcasting and forecasting GDP in emerging market economies using big data methods. This is done by analyzing the usefulness of various dimension-reduction, machine learning and shrinkage methods, including sparse principal component analysis (SPCA), the elastic net, the least absolute shrinkage operator, and least angle regression when constructing predictions using latent global macroeconomic and financial factors (diffusion indexes) in a dynamic factor model (DFM). We also utilize a judgmental dimension-reduction method called the Bloomberg Relevance Index (BRI), which is an index that assigns a measure of importance to each variable in a dataset depending on the variable’s usage by market participants. Our empirical analysis shows that, when specified using dimension-reduction methods (particularly BRI and SPCA), DFMs yield superior predictions relative to both benchmark linear econometric models and simple DFMs. Moreover, global financial and macroeconomic (business cycle) diffusion indexes constructed using targeted predictors are found to be important in four of the five emerging market economies that we study (Brazil, Mexico, South Africa, and Turkey). These findings point to the importance of spillover effects across emerging market economies, and underscore the significance of characterizing such linkages parsimoniously when utilizing high-dimensional global datasets.     

Shrinkage estimation of semiparametric multiplicative error models

Within models for nonnegative time series, it is common to encounter deterministic components (trends, seasonalities) which can be specified in a flexible form. This work proposes the use of shrinkage type estimation for the parameters of such components. The amount of smoothing to be imposed on the estimates can be chosen using different methodologies: Cross-Validation for dependent data or the recently proposed Focused Information Criterion. We illustrate such a methodology using a semiparametric autoregressive conditional duration model that decomposes the conditional expectations of durations into their dynamic (parametric) and diurnal (flexible) components. We use a shrinkage estimator that jointly estimates the parameters of the two components and controls the smoothness of the estimated flexible component. The results show that, from the forecasting perspective, an appropriate shrinkage strategy can significantly improve on the baseline maximum likelihood estimation.     

Strong Consistency of the Maximum Likelihood Estimator in Generalized Linear and Nonlinear Mixed-Effects Models

Generalized linear and nonlinear mixed-effects models are used extensively in biomedical, social, and agricultural sciences. The statistical analysis of these models is based on the asymptotic properties of the maximum likelihood estimator. However, it is usually assumed that the maximum likelihood estimator is consistent, without providing a proof. A rigorous proof of the consistency by verifying conditions from existing results can be very difficult due to the integrated likelihood. In this paper, we present some easily verifiable conditions for the strong consistency of the maximum likelihood estimator in generalized linear and nonlinear mixed-effects models. Based on this result, we prove that the maximum likelihood estimator is consistent for some frequently used models such as mixed-effects logistic regression models and growth curve models.     

Decomposing the effects of crowd-wisdom aggregators: The bias–information–noise (BIN) model

Aggregating predictions from multiple judges often yields more accurate predictions than relying on a single judge, which is known as the wisdom-of-the-crowd effect. However, a wide range of aggregation methods are available, which range from one-size-fits-all techniques, such as simple averaging, prediction markets, and Bayesian aggregators, to customized (supervised) techniques that require past performance data, such as weighted averaging. In this study, we applied a wide range of aggregation methods to subjective probability estimates from geopolitical forecasting tournaments. We used the bias–information–noise (BIN) model to disentangle three mechanisms that allow aggregators to improve the accuracy of predictions: reducing bias and noise, and extracting valid information across forecasters. Simple averaging operates almost entirely by reducing noise, whereas more complex techniques such as prediction markets and Bayesian aggregators exploit all three pathways to allow better signal extraction as well as greater noise and bias reduction. Finally, we explored the utility of a BIN approach for the modular construction of aggregators.     

Bayesian model comparison for time‐varying parameter VARs with stochastic volatility

We develop importance sampling methods for computing two popular Bayesian model comparison criteria, namely, the marginal likelihood and the deviance information criterion (DIC) for time‐varying parameter vector autoregressions (TVP‐VARs), where both the regression coefficients and volatilities are drifting over time. The proposed estimators are based on the integrated likelihood, which are substantially more reliable than alternatives. Using US data, we find overwhelming support for the TVP‐VAR with stochastic volatility compared to a conventional constant coefficients VAR with homoskedastic innovations. Most of the gains, however, appear to have come from allowing for stochastic volatility rather than time variation in the VAR coefficients or contemporaneous relationships. Indeed, according to both criteria, a constant coefficients VAR with stochastic volatility outperforms the more general model with time‐varying parameters.     

Selection of influential variables in ordinal data with preponderance of zeros

Presence of excess zero in ordinal data is pervasive in areas like medical and social sciences. Unfortunately, analysis of such kind of data has so far hardly been looked into, perhaps for the reason that the underlying model that fits such data, is not a generalized linear model. Obviously some methodological developments and intensive computations are required. The current investigation is concerned with the selection of variables in such models. In many occasions where the number of predictors is quite large and some of them are not useful, the maximum likelihood approach is not the automatic choice. As, apart from the messy calculations involved, this approach fails to provide efficient estimates of the underlying parameters. The proposed penalized approach includes ?₁ penalty (LASSO) and the mixture of ?₁ and ?₂ penalties (elastic net). We propose a coordinate descent algorithm to fit a wide class of ordinal regression models and select useful variables appearing in both the ordinal regression and the logistic regression based mixing component. A rigorous discussion on the selection of predictors has been made through a simulation study. The proposed method is illustrated by analyzing the severity of driver injury from Michigan upper peninsula road accidents.     

Posterior Inference in Bayesian Quantile Regression with Asymmetric Laplace Likelihood

The paper discusses the asymptotic validity of posterior inference of pseudo‐Bayesian quantile regression methods with complete or censored data when an asymmetric Laplace likelihood is used. The asymmetric Laplace likelihood has a special place in the Bayesian quantile regression framework because the usual quantile regression estimator can be derived as the maximum likelihood estimator under such a model, and this working likelihood enables highly efficient Markov chain Monte Carlo algorithms for posterior sampling. However, it seems to be under‐recognised that the stationary distribution for the resulting posterior does not provide valid posterior inference directly. We demonstrate that a simple adjustment to the covariance matrix of the posterior chain leads to asymptotically valid posterior inference. Our simulation results confirm that the posterior inference, when appropriately adjusted, is an attractive alternative to other asymptotic approximations in quantile regression, especially in the presence of censored data.     

Investigating the impact of excess zeros on hurdle‐generalized Poisson regression model with right censored count data

Typically, a Poisson model is assumed for count data. In many cases, there are many zeros in the dependent variable, thus the mean is not equal to the variance value of the dependent variable. Therefore, Poisson model is not suitable anymore for this kind of data because of too many zeros. Thus, we suggest using a hurdle‐generalized Poisson regression model. Furthermore, the response variable in such cases is censored for some values because of some big values. A censored hurdle‐generalized Poisson regression model is introduced on count data with many zeros in this paper. The estimation of regression parameters using the maximum likelihood method is discussed and the goodness‐of‐fit for the regression model is examined. An example and a simulation will be used to illustrate the effects of right censoring on the parameter estimation and their standard errors.     

Predicting equity premium using dynamic model averaging. Does the state–space representation matter?

Dynamic model averaging (DMA) has become a very useful tool with regards to dealing with two important aspects of time-series analysis, namely, parameter instability and model uncertainty. An important component of DMA is the Kalman filter. It is used to filter out the latent time-varying regression coefficients of the predictive regression of interest, and produce the model predictive likelihood, which is needed to construct the probability of each model in the model set. To apply the Kalman filter, one must write the model of interest in linear state–space form. In this study, we demonstrate that the state–space representation has implications on out-of-sample prediction performance, and the degree of shrinkage. Using Monte Carlo simulations as well as financial data at different sampling frequencies, we document that the way in which the current literature tends to formulate the candidate time-varying parameter predictive regression in linear state–space form ignores empirical features that are often present in the data at hand, namely, predictor persistence and predictor endogeneity. We suggest a straightforward way to account for these features in the DMA setting. Results using the widely applied Goyal and Welch (2008) dataset document that modifying the DMA framework as we suggest has a bearing on equity premium point prediction performance from a statistical as well as an economic viewpoint.     

Over Factoranalyse

Summary This paper is an exposition about the model and techniques in factor analysis, a method of studying the covariance matrix of several properties on the basis of a sample co-variance matrix of independent observations on n individuals. The indeterminacy of the basis of the so called factor space and several possibilities of interpretation are discussed. The scale invariant maximum likelihood estimation of the parameters of the assumed normal distribution which also provides a test on the dimension of the factor space is compared with the customary but unjustified attack of the estimation problem by means of component analysis or modifications of it. The prohibitive slowness of convergence of iterative procedures recommended till now can be removed by steepest ascent methods together with Aitken's acceleration method. An estimate of the original observations according to the model assumed, as to be compared with the data, is given.     

Forecasting European carbon returns using dimension reduction techniques: Commodity versus financial fundamentals

Using a broad selection of 53 carbon (EUA) related, commodity and financial predictors, we provide a comprehensive assessment of the out-of-sample (OOS) predictability of weekly European carbon futures return. We assess forecast performance using both statistical and economic value metrics over an OOS period spanning from January 2013 to May 2018. Two main types of dimension reduction techniques are employed: (i) shrinkage of coefficient estimates and (ii) factor models. We find that: (1) these dimension reduction techniques can beat the benchmark significantly with positive gains in forecast accuracy, despite very few individual predictors being able to; (2) forecast accuracy is sensitive to the sample period, and only Group-average models and Commodity-predictors tend to beat the benchmark consistently; the Group-average models can improve both the prediction accuracy and stability significantly by averaging the predictions of All-predictors model and the benchmark. Further, to demonstrate the usefulness of forecasts to the end-user, we estimate the certainty equivalent gains (economic value) generated. Almost all dimension reduction techniques do well especially those which apply shrinkage alone. We find including All-predictors and Group-average variable sets achieve the highest economic gains and portfolio performance. Our main results are robust to alternative specifications.     

Poisson Regression for Clustered Data

We compare five methods for parameter estimation of a Poisson regression model for clustered data: (1) ordinary (naive) Poisson regression (OP), which ignores intracluster correlation, (2) Poisson regression with fixed cluster‐specific intercepts (FI), (3) a generalized estimating equations (GEE) approach with an equi‐correlation matrix, (4) an exact generalized estimating equations (EGEE) approach with an exact covariance matrix, and (5) maximum likelihood (ML). Special attention is given to the simplest case of the Poisson regression with a cluster‐specific intercept random when the asymptotic covariance matrix is obtained in closed form. We prove that methods 1–5, except GEE, produce the same estimates of slope coefficients for balanced data (an equal number of observations in each cluster and the same vectors of covariates). All five methods lead to consistent estimates of slopes but have different efficiency for unbalanced data design. It is shown that the FI approach can be derived as a limiting case of maximum likelihood when the cluster variance increases to infinity. Exact asymptotic covariance matrices are derived for each method. In terms of asymptotic efficiency, the methods split into two groups: OP & GEE and EGEE & FI & ML. Thus, contrary to the existing practice, there is no advantage in using GEE because it is substantially outperformed by EGEE and FI. In particular, EGEE does not require integration and is easy to compute with the asymptotic variances of the slope estimates close to those of the ML.     

Bayesian estimation and model selection for spatial Durbin error model with finite distributed lags

In this paper we investigate a spatial Durbin error model with finite distributed lags and consider the Bayesian MCMC estimation of the model with a smoothness prior. We study also the corresponding Bayesian model selection procedure for the spatial Durbin error model, the spatial autoregressive model and the matrix exponential spatial specification model. We derive expressions of the marginal likelihood of the three models, which greatly simplify the model selection procedure. Simulation results suggest that the Bayesian estimates of high order spatial distributed lag coefficients are more precise than the maximum likelihood estimates. When the data is generated with a general declining pattern or a unimodal pattern for lag coefficients, the spatial Durbin error model can better capture the pattern than the SAR and the MESS models in most cases. We apply the procedure to study the effect of right to work (RTW) laws on manufacturing employment.