A Selective Overview and Comparison of Robust Mixture Regression Estimators

Mixture regression models have been widely used in business, marketing and social sciences to model mixed regression relationships arising from a clustered and thus heterogeneous population. The unknown mixture regression parameters are usually estimated by maximum likelihood estimators using the expectation–maximisation algorithm based on the normality assumption of component error density. However, it is well known that the normality-based maximum likelihood estimation is very sensitive to outliers or heavy-tailed error distributions. This paper aims to give a selective overview of the recently proposed robust mixture regression methods and compare their performance using simulation studies.     

Asymptotics and smoothing parameter selection for penalized spline regression with various loss functions

Penalized splines are used in various types of regression analyses, including non‐parametric quantile, robust and the usual mean regression. In this paper, we focus on the penalized spline estimator with general convex loss functions. By specifying the loss function, we can obtain the mean estimator, quantile estimator and robust estimator. We will first study the asymptotic properties of penalized splines. Specifically, we will show the asymptotic bias and variance as well as the asymptotic normality of the estimator. Next, we will discuss smoothing parameter selection for the minimization of the mean integrated squares error. The new smoothing parameter can be expressed uniquely using the asymptotic bias and variance of the penalized spline estimator. To validate the new smoothing parameter selection method, we will provide a simulation. The simulation results show that the consistency of the estimator with the proposed smoothing parameter selection method can be confirmed and that the proposed estimator has better behavior than the estimator with generalized approximate cross‐validation. A real data example is also addressed.     

Forecasting in GARCH models with polynomially modified innovations

Orthogonal polynomials can be used to modify the moments of the distribution of a random variable. In this paper, polynomially adjusted distributions are employed to model the skewness and kurtosis of the conditional distributions of GARCH models. To flexibly capture the skewness and kurtosis of data, the distributions of the innovations that are polynomially reshaped include, besides the Gaussian, also leptokurtic laws such as the logistic and the hyperbolic secant. Modeling GARCH innovations with polynomially adjusted distributions can effectively improve the precision of the forecasts. This strategy is analyzed in GARCH models with different specifications for the conditional variance, such as the APARCH, the EGARCH, the Realized GARCH, and APARCH with time-varying skewness and kurtosis. An empirical application on different types of asset returns shows the good performance of these models in providing accurate forecasts according to several criteria based on density forecasting, downside risk, and volatility prediction.     

International Differences in Conditional Conservatism – The Role of Unconditional Conservatism and Income Smoothing

Abstract

Prior research documents that conditional conservatism, measured as the asymmetric timeliness of earnings reflecting bad vs. good news, varies with cross-country differences in institutional regimes. In this paper, we examine the determinants of conditional conservatism and related earnings attributes internationally. First, using panel data, we investigate whether competing earnings attributes such as unconditional conservatism and income smoothing affect conditional conservatism and its international differences. We find that these attributes are predictably correlated with conditional conservatism. Second, we address the question whether income smoothing and conditional conservatism are two fundamentally different earnings attributes. We show theoretically that both attributes yield different earnings distributions and that the motivations for producing earnings which possess these attributes differ. To test these predictions empirically, we calculate firm-specific time-series measures of asymmetric timeliness, using a novel trigonometric measure based on the standard Basu (1997)-type regression. Using this cross-sectional data, we test whether conditional conservatism and income smoothing are different and find them to be only weakly correlated for a broad international sample. Also, we demonstrate that income smoothing explains international differences in conditional conservatism. Finally, we estimate simple determinant models of conditional conservatism and income smoothing, showing that both earnings attributes are driven by different explanatory firm-level factors: Conditional conservatism increases with the importance of debt financing, while income smoothing increases with the importance of dividends. Despite some important limitations, we believe our results to be meaningful because they show that cross-country differences in conditional conservatism are influenced by the effects of other accounting properties, predominantly income smoothing. Especially, legal regime appears to drive income smoothing while losing its explanatory power for conditional conservatism when firm-specific factors are controlled for.     

Bayesian modeling of joint and conditional distributions

In this paper, we study a Bayesian approach to flexible modeling of conditional distributions. The approach uses a flexible model for the joint distribution of the dependent and independent variables and then extracts the conditional distributions of interest from the estimated joint distribution. We use a finite mixture of multivariate normals (FMMN) to estimate the joint distribution. The conditional distributions can then be assessed analytically or through simulations. The discrete variables are handled through the use of latent variables. The estimation procedure employs an MCMC algorithm. We provide a characterization of the Kullback–Leibler closure of FMMN and show that the joint and conditional predictive densities implied by the FMMN model are consistent estimators for a large class of data generating processes with continuous and discrete observables. The method can be used as a robust regression model with discrete and continuous dependent and independent variables and as a Bayesian alternative to semi- and non-parametric models such as quantile and kernel regression. In experiments, the method compares favorably with classical nonparametric and alternative Bayesian methods.     

Missing observations in ARIMA models: Skipping approach versus additive outlier approach

Optimal estimation of missing values in ARMA models is typically performed by using the Kalman filter for likelihood evaluation, ‘skipping’ in the computations the missing observations, obtaining the maximum likelihood (ML) estimators of the model parameters, and using some smoothing algorithm. The same type of procedure has been extended to nonstationary ARIMA models in Gómez and Maravall (1994). An alternative procedure suggests filling in the holes in the series with arbitrary values and then performing ML estimation of the ARIMA model with additive outliers (AO). When the model parameters are not known the two methods differ, since the AO likelihood is affected by the arbitrary values. We develop the proper likelihood for the AO approach in the general non-stationary case and show the equivalence of this and the skipping method. Finally, the two methods are compared through simulation, and their relative advantages assessed; the comparison also includes the AO method with the uncorrected likelihood.     

Can the random walk model be beaten in out-of-sample density forecasts? Evidence from intraday foreign exchange rates

It has been documented that random walk outperforms most economic structural and time series models in out-of-sample forecasts of the conditional mean dynamics of exchange rates. In this paper, we study whether random walk has similar dominance in out-of-sample forecasts of the conditional probability density of exchange rates given that the probability density forecasts are often needed in many applications in economics and finance. We first develop a nonparametric portmanteau test for optimal density forecasts of univariate time series models in an out-of-sample setting and provide simulation evidence on its finite sample performance. Then we conduct a comprehensive empirical analysis on the out-of-sample performances of a wide variety of nonlinear time series models in forecasting the intraday probability densities of two major exchange rates—Euro/Dollar and Yen/Dollar. It is found that some sophisticated time series models that capture time-varying higher order conditional moments, such as Markov regime-switching models, have better density forecasts for exchange rates than random walk or modified random walk with GARCH and Student-t innovations. This finding dramatically differs from that on mean forecasts and suggests that sophisticated time series models could be useful in out-of-sample applications involving the probability density.     

Robust Hierarchical Bayes Small Area Estimation for the Nested Error Linear Regression Model

Standard model‐based small area estimates perform poorly in presence of outliers. Sinha & Rao ( 2009 ) developed robust frequentist predictors of small area means. In this article, we present a robust Bayesian method to handle outliers in unit‐level data by extending the nested error regression model. We consider a finite mixture of normal distributions for the unit‐level error to model outliers and produce noninformative Bayes predictors of small area means. Our modelling approach generalises that of Datta & Ghosh ( 1991 ) under the normality assumption. Application of our method to a data set which is suspected to contain an outlier confirms this suspicion, correctly identifies the suspected outlier and produces robust predictors and posterior standard deviations of the small area means. Evaluation of several procedures including the M‐quantile method of Chambers & Tzavidis ( 2006 ) via simulations shows that our proposed method is as good as other procedures in terms of bias, variability and coverage probability of confidence and credible intervals when there are no outliers. In the presence of outliers, while our method and Sinha–Rao method perform similarly, they improve over the other methods. This superior performance of our procedure shows its dual (Bayes and frequentist) dominance, which should make it attractive to all practitioners, Bayesians and frequentists, of small area estimation.     

Nonparametric estimation of conditional distribution functions with longitudinal data and time-varying parametric models

Nonparametric estimation and inferences of conditional distribution functions with longitudinal data have important applications in biomedical studies. We propose in this paper an estimation approach based on time-varying parametric models. Our model assumes that the conditional distribution of the outcome variable at each given time point can be approximated by a parametric model, but the parameters are smooth functions of time. Our estimation is based on a two-step smoothing method, in which we first obtain the raw estimators of the conditional distribution functions at a set of disjoint time points, and then compute the final estimators at any time by smoothing the raw estimators. Asymptotic properties, including the asymptotic biases, variances and mean squared errors, are derived for the local polynomial smoothed estimators. Applicability of our two-step estimation method is demonstrated through a large epidemiological study of childhood growth and blood pressure. Finite sample properties of our procedures are investigated through simulation study.     

Testing for non-nested conditional moment restrictions using unconditional empirical likelihood

We propose non-nested hypothesis tests for conditional moment restriction models based on the method of generalized empirical likelihood (GEL). By utilizing the implied GEL probabilities from a sequence of unconditional moment restrictions that contains equivalent information of the conditional moment restrictions, we construct Kolmogorov–Smirnov and Cramér–von Mises type moment encompassing tests. Advantages of our tests over Otsu and Whang’s (2011) tests are: (i) they are free from smoothing parameters, (ii) they can be applied to weakly dependent data, and (iii) they allow non-smooth moment functions. We derive the null distributions, validity of a bootstrap procedure, and local and global power properties of our tests. The simulation results show that our tests have reasonable size and power performance in finite samples.     

Tuning parameter selection in penalized generalized linear models for discrete data

In recent years, we have seen an increased interest in the penalized likelihood methodology, which can be efficiently used for shrinkage and selection purposes. This strategy can also result in unbiased, sparse, and continuous estimators. However, the performance of the penalized likelihood approach depends on the proper choice of the regularization parameter. Therefore, it is important to select it appropriately. To this end, the generalized cross‐validation method is commonly used. In this article, we firstly propose new estimates of the norm of the error in the generalized linear models framework, through the use of Kantorovich inequalities. Then these estimates are used in order to derive a tuning parameter selector in penalized generalized linear models. The proposed method does not depend on resampling as the standard methods and therefore results in a considerable gain in computational time while producing improved results. A thorough simulation study is conducted to support theoretical findings; and a comparison of the penalized methods with the L₁, the hard thresholding, and the smoothly clipped absolute deviation penalty functions is performed, for the cases of penalized Logistic regression and penalized Poisson regression. A real data example is being analyzed, and a discussion follows. © 2014 The Authors. Statistica Neerlandica © 2014 VVS.     

Robust feature screening for varying coefficient models via quantile partial correlation

This article is concerned with feature screening for varying coefficient models with ultrahigh-dimensional predictors. We propose a new sure independence screening method based on quantile partial correlation (QPC-SIS), which is quite robust against outliers and heavy-tailed distributions. Then we establish the sure screening property for the QPC-SIS, and conduct simulations to examine its finite sample performance. The results of simulation study indicate that the QPC-SIS performs better than other methods like sure independent screening (SIS), sure independent ranking and screening, distance correlation-sure independent screening, conditional correlation sure independence screening and nonparametric independent screening, which shows the validity and rationality of QPC-SIS.     

On Business Cycle Asymmetries in G7 Countries

We investigate whether business cycle dynamics in seven industrialized countries (the G7) are characterized by asymmetries in conditional mean. We provide evidence on this issue using a variety of time series models. Our approach is fully parametric. Our testing strategy is robust to any conditional heteroskedasticity, outliers, and/or long memory that may be present. Our results indicate fairly strong evidence of nonlinearities in the conditional mean dynamics of the GDP growth rates for Canada, Germany, Italy, Japan, and the US. For France and the UK, the conditional mean dynamics appear to be largely linear. Our study shows that while the existence of conditional heteroskedasticity and long memory does not have much effect on testing for linearity in the conditional mean, accounting for outliers does reduce the evidence against linearity.     

A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t$t$ densities that approximates accurately the target distribution–typically a posterior distribution, of which we only require a kernel–in the sense that the Kullback–Leibler divergence between target and mixture is minimized. We label this approach Mixture of  t$t$by Importance Sampling weighted Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis–Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach. This is useful for importance or Metropolis–Hastings sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model’s mixture regimes’ parameters. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.     

The Three Basic Types of Residuals for a Linear Model

We consider residuals for the linear model with a general covariance structure. In contrast to the situation where observations are independent there are several alternative definitions. We draw attention to three quite distinct types of residuals: the marginal residuals, the model‐specified residuals and the full‐conditional residuals. We adopt a very broad perspective including linear mixed models, time series and smoothers as well as models for spatial and multivariate data. We concentrate on defining these different residual types and discussing their interrelationships. The full‐conditional residuals are seen to play several important roles.     

Asymptotic distribution of the cointegrating vector estimator in error correction models with conditional heteroskedasticity

This paper explores the asymptotic distribution of the cointegrating vector estimator in error correction models with conditionally heteroskedastic errors. Asymptotic properties of the maximum likelihood estimator (MLE) of the cointegrating vector, which estimates the cointegrating vector and the multivariate GARCH process jointly, are provided. The MLE of the cointegrating vector follows mixture normal, and its asymptotic distribution depends on the conditional heteroskedasticity and the kurtosis of standardized innovations. The reduced rank regression (RRR) estimator and the regression-based cointegrating vector estimators do not consider conditional heteroskedasticity, and thus the efficiency gain of the MLE emerges as the magnitude of conditional heteroskedasticity increases. The simulation results indicate that the relative power of the t-statistics based on the MLE improves significantly as the GARCH effect increases.     

Interval estimation of value-at-risk based on GARCH models with heavy-tailed innovations

ARCH and GARCH models are widely used to model financial market volatilities in risk management applications. Considering a GARCH model with heavy-tailed innovations, we characterize the limiting distribution of an estimator of the conditional value-at-risk (VaR), which corresponds to the extremal quantile of the conditional distribution of the GARCH process. We propose two methods, the normal approximation method and the data tilting method, for constructing confidence intervals for the conditional VaR estimator and assess their accuracies by simulation studies. Finally, we apply the proposed approach to an energy market data set.     

Nonparametric/semiparametric estimation and testing of econometric models with data dependent smoothing parameters

We consider nonparametric/semiparametric estimation and testing of econometric models with data dependent smoothing parameters. Most of the existing works on asymptotic distributions of a nonparametric/semiparametric estimator or a test statistic are based on some deterministic smoothing parameters, while in practice it is important to use data-driven methods to select the smoothing parameters. In this paper we give a simple sufficient condition that can be used to establish the first order asymptotic equivalence of a nonparametric estimator or a test statistic with stochastic smoothing parameters to those using deterministic smoothing parameters. We also allow for general weakly dependent data.     

Unraveling the Relationship Between Presidential Approval and the Economy: A Multidimensional Semiparametric Approach

Empirical studies analyzing the determinants of US presidential popularity have delivered quite inconclusive results concerning the role of economic variables by assuming linear relationships. We employ penalized spline smoothing in the context of semiparametric additive mixed models and allow for flexible functional forms and thus possible nonlinear effects for the economic determinants. By controlling for the well‐known politically motivated covariables, we find strong evidence for nonlinear and negative effects of unemployment, inflation and government consumption on presidential approval. Additionally, we present new results in favor of nonparametric trivariate interaction effects between the macroeconomic covariables. Copyright © 2014 John Wiley & Sons, Ltd.     

Bayesian robustness modelling using the O‐regularly varying distributions

The theory of robustness modelling is essentially based on heavy‐tailed distributions, because longer tails are more prepared to deal with diverse information (such as outliers) because of the higher probabilities on the tails. There are many classes of distributions that can be regarded as heavy tails; some of them have interesting properties and are not explored in statistics. In the present work, we propose a robustness modelling approach based on the O‐regularly varying class (ORV), which is a generalization of the regular variation family; however, the ORV class allows more flexible tails behaviour, which can improve the way in which the outlying information is discarded by the model. We establish sufficient conditions in the location and in the scale parameter structures, which allow to resolve automatically the conflicts of information. We also provide a procedure for generating new distributions within the ORV class.