Three Sides of Smoothing: Categorical Data Smoothing, Nonparametric Regression, and Density Estimation

The past forty years have seen a great deal of research into the construction and properties of nonparametric estimates of smooth functions. This research has focused primarily on two sides of the smoothing problem: nonparametric regression and density estimation. Theoretical results for these two situations are similar, and multivariate density estimation was an early justification for the Nadaraya-Watson kernel regression estimator.
A third, less well-explored, strand of applications of smoothing is to the estimation of probabilities in categorical data. In this paper the position of categorical data smoothing as a bridge between nonparametric regression and density estimation is explored. Nonparametric regression provides a paradigm for the construction of effective categorical smoothing estimates, and use of an appropriate likelihood function yields cell probability estimates with many desirable properties. Such estimates can be used to construct regression estimates when one or more of the categorical variables are viewed as response variables. They also lead naturally to the construction of well-behaved density estimates using local or penalized likelihood estimation, which can then be used in a regression context. Several real data sets are used to illustrate these points.     

Threshold estimation of Markov models with jumps and interest rate modeling

We reconstruct the level-dependent diffusion coefficient of a univariate semimartingale with jumps which is observed discretely. The consistency and asymptotic normality of our estimator are provided in the presence of both finite and infinite activity (finite variation) jumps. Our results rely on kernel estimation, using the properties of the local time of the data generating process, and the fact that it is possible to disentangle the discontinuous part of the state variable through those squared increments between observations not exceeding a suitable threshold function. We also reconstruct the drift and the jump intensity coefficients when they are level-dependent and jumps have finite activity, through consistent and asymptotically normal estimators. Simulated experiments show that the newly proposed estimators perform better in finite samples than alternative estimators, and this allows us to reexamine the estimation of a univariate model for the short term interest rate, for which we find fewer jumps and more variance due to the diffusion part than previous studies.     

Optimal estimates and optimal predictors of finite population characteristics in the presence of nonresponse

Summary When samples from a finite population are studied, for instance by interview, there will usually be some units from which no response is obtained. In this paper optimal predictors of finite population characteristics, when nonresponse is present, are studied. The predictors are studied under simple regression superpopulation models. The optimal predictors are connected to the classical weighted sample estimates which are shown to be maximum likelihood estimates, provided the probability function is fully described by the sampling design. The predictors are compared with respects to their efficiencies for some simple models and a possible explanation to the fact that the poststratification estimate which compensate for nonresponse does no better than the simple estimate, is pointed out.     

Market microstructure noise,integrated variance estimators,and the accuracy of asymptotic approximations

A growing literature has been advocating consistent kernel estimation of integrated variance in the presence of financial market microstructure noise. We find that, for realistic sample sizes encountered in practice, the asymptotic results derived for the proposed estimators may provide unsatisfactory representations of their finite sample properties. In addition, the existing asymptotic results might not offer sufficient guidance for practical implementations. We show how to optimize the finite sample properties of kernel-based integrated variance estimators. Empirically, we find that their suboptimal implementation can, in some cases, lead to little or no finite sample gains when compared to the classical realized variance estimator. Significant statistical and economic gains can, however, be recovered by using our proposed finite sample methods.     

Additive cubic spline regression with Dirichlet process mixture errors

The goal of this article is to develop a flexible Bayesian analysis of regression models for continuous and categorical outcomes. In the models we study, covariate (or regression) effects are modeled additively by cubic splines, and the error distribution (that of the latent outcomes in the case of categorical data) is modeled as a Dirichlet process mixture. We employ a relatively unexplored but attractive basis in which the spline coefficients are the unknown function ordinates at the knots. We exploit this feature to develop a proper prior distribution on the coefficients that involves the first and second differences of the ordinates, quantities about which one may have prior knowledge. We also discuss the problem of comparing models with different numbers of knots or different error distributions through marginal likelihoods and Bayes factors which are computed within the framework of Chib (1995) as extended to DPM models by Basu and Chib (2003). The techniques are illustrated with simulated and real data.     

A non-local perspective on the power properties of the CUSUM and CUSUM of squares tests for structural change

We consider the power properties of the CUSUM and CUSUM of squares (CUSQ) tests in the presence of a one-time change in the parameters of a linear regression model. A result due to Ploberger and Krämer [1990. The local power of the cusum and cusum of squares tests. Econometric Theory 6, 335–347.] is that the CUSQ test has only trivial asymptotic local power in this case, while the CUSUM test has non-trivial local asymptotic power unless the change is orthogonal to the mean regressor. The main theme of the paper is that such conclusions obtained from a local asymptotic framework are not reliable guides to what happens in finite samples. The approach we take is to derive expansions of the test statistics that retain terms related to the magnitude of the change under the alternative hypothesis. This enables us to analyze what happens for non-local to zero breaks. Our theoretical results are able to explain how the power function of the tests can be drastically different depending on whether one deals with a static regression with uncorrelated errors, a static regression with correlated errors, a dynamic regression with lagged dependent variables, or whether a correction for non-normality is applied in the case of the CUSQ. We discuss in which cases the tests are subject to a non-monotonic power function that goes to zero as the magnitude of the change increases, and uncover some curious properties. All theoretical results are verified to yield good guides to the finite sample power through simulation experiments. We finally highlight the practical importance of our results.     

Functional coefficient regression models with time trend

We consider the problem of estimating a varying coefficient regression model when regressors include a time trend. We show that the commonly used local constant kernel estimation method leads to an inconsistent estimation result, while a local polynomial estimator yields a consistent estimation result. We establish the asymptotic normality result for the proposed estimator. We also provide asymptotic analysis of the data-driven (least squares cross validation) method of selecting the smoothing parameters. In addition, we consider a partially linear time trend model and establish the asymptotic distribution of our proposed estimator. Two test statistics are proposed to test the null hypotheses of a linear and of a partially linear time trend models. Simulations are reported to examine the finite sample performances of the proposed estimators and the test statistics.     

Testing for a break in trend when the order of integration is unknown

Harvey, Leybourne and Taylor [Harvey, D.I., Leybourne, S.J., Taylor, A.M.R. 2009. Simple, robust and powerful tests of the breaking trend hypothesis. Econometric Theory 25, 995–1029] develop a test for the presence of a broken linear trend at an unknown point in the sample whose size is asymptotically robust as to whether the (unknown) order of integration of the data is either zero or one. This test is not size controlled, however, when this order assumes fractional values; its asymptotic size can be either zero or one in such cases. In this paper we suggest a new test, based on a sup-Wald statistic, which is asymptotically size-robust across fractional values of the order of integration (including zero or one). We examine the asymptotic power of the test under a local trend break alternative. The finite sample properties of the test are also investigated.     

Nonparametric estimation of conditional VaR and expected shortfall

This paper considers a new nonparametric estimation of conditional value-at-risk and expected shortfall functions. Conditional value-at-risk is estimated by inverting the weighted double kernel local linear estimate of the conditional distribution function. The nonparametric estimator of conditional expected shortfall is constructed by a plugging-in method. Both the asymptotic normality and consistency of the proposed nonparametric estimators are established at both boundary and interior points for time series data. We show that the weighted double kernel local linear conditional distribution estimator has the advantages of always being a distribution, continuous, and differentiable, besides the good properties from both the double kernel local linear and weighted Nadaraya–Watson estimators. Moreover, an ad hoc data-driven fashion bandwidth selection method is proposed, based on the nonparametric version of the Akaike information criterion. Finally, an empirical study is carried out to illustrate the finite sample performance of the proposed estimators.     

Robust and Powerful Tests for Nonlinear Deterministic Components

We develop a test for the presence of nonlinear deterministic components in a univariate time series, approximated using a Fourier series expansion, designed to be asymptotically robust to the order of integration of the process and to any weak dependence present. We show that our proposed test has uniformly greater local asymptotic power than the existing tests of Harvey, Leybourne and Xiao (2010) when the shocks are I(1), identical local asymptotic power when the shocks are I(0), and also improved finite sample properties. We also consider the issue of determining the number of Fourier frequencies used to specify any nonlinear deterministic components.     

The performance of estimators based on the propensity score

We investigate the finite sample properties of a large number of estimators for the average treatment effect on the treated that are suitable when adjustment for observed covariates is required, like inverse probability weighting, kernel and other variants of matching, as well as different parametric models. The simulation design used is based on real data usually employed for the evaluation of labour market programmes in Germany. We vary several dimensions of the design that are of practical importance, like sample size, the type of the outcome variable, and aspects of the selection process. We find that trimming individual observations with too much weight as well as the choice of tuning parameters are important for all estimators. A conclusion from our simulations is that a particular radius matching estimator combined with regression performs best overall, in particular when robustness to misspecifications of the propensity score and different types of outcome variables is considered an important property.     

Spline regression for hazard rate estimation when data are censored and measured with error

In this paper, we study an estimation problem where the variables of interest are subject to both right censoring and measurement error. In this context, we propose a nonparametric estimation strategy of the hazard rate, based on a regression contrast minimized in a finite‐dimensional functional space generated by splines bases. We prove a risk bound of the estimator in terms of integrated mean square error and discuss the rate of convergence when the dimension of the projection space is adequately chosen. Then we define a data‐driven criterion of model selection and prove that the resulting estimator performs an adequate compromise. The method is illustrated via simulation experiments that show that the strategy is successful.     

crs: A PACKAGE FOR NONPARAMETRIC SPLINE ESTIMATION IN R

crs is a library for R written by Jeffrey S. Racine (Maintainer) and Zhenghua Nie. This add‐on package provides a collection of functions for spline‐based nonparametric estimation of regression functions with both continuous and categorical regressors. Currently, the crs package integrates data‐driven methods for selecting the spline degree, the number of knots and the necessary bandwidths for nonparametric conditional mean, IV and quantile regression. A function for multivariate density spline estimation with mixed data is also currently in the works. As a bonus, the authors have also provided the first simple R interface to the NOMAD (‘nonsmooth mesh adaptive direct search’) optimization solver which can be applied to solve other mixed integer optimization problems that future users might find useful in other settings. Although the crs package shares some of the same functionalities as its kernel‐based counterpart—the np package by the same author—it currently lacks some of the features the np package provides, such as hypothesis testing and semiparametric estimation. However, what it lacks in breadth, crs makes up in speed. A Monte Carlo experiment in this review uncovers sizable speed gains compared to its np counterpart, with a marginal loss in terms of goodness of fit. Therefore, the package will be extremely useful for applied econometricians interested in employing nonparametric techniques using large amounts of data with a small number of discrete covariates. Copyright © 2014 John Wiley & Sons, Ltd.     

Small‐sample confidence intervals for multivariate impulse response functions at long horizons

Existing methods for constructing confidence bands for multivariate impulse response functions may have poor coverage at long lead times when variables are highly persistent. The goal of this paper is to propose a simple method that is not pointwise and that is robust to the presence of highly persistent processes. We use approximations based on local‐to‐unity asymptotic theory, and allow the horizon to be a fixed fraction of the sample size. We show that our method has better coverage properties at long horizons than existing methods, and may provide different economic conclusions in empirical applications. We also propose a modification of this method which has good coverage properties at both short and long horizons. Copyright © 2006 John Wiley & Sons, Ltd.     

REVERSE REGRESSIONS AND LONG‐HORIZON FORECASTING

Long‐horizon predictive regressions in finance pose formidable econometric problems when estimated using available sample sizes. Hodrick in 1992 proposed a remedy that is based on running a reverse regression of short‐horizon returns on the long‐run mean of the predictor. Unfortunately, this only allows the null of no predictability to be tested, and assumes stationary regressors. In this paper, we revisit long‐horizon forecasting from reverse regressions, and argue that reverse regression methods avoid serious size distortions in long‐horizon predictive regressions, even when there is some predictability and/or near unit roots. Meanwhile, the reverse regression methodology has the practical advantage of being easily applicable when there are many predictors. We apply these methods to forecasting excess bond returns using the term structure of forward rates, and find that there is indeed some return forecastability. However, confidence intervals for the coefficients of the predictive regressions are about twice as wide as those obtained with the conventional approach to inference. We also include an application to forecasting excess stock returns. Copyright © 2011 John Wiley & Sons, Ltd.     

Robust Hierarchical Bayes Small Area Estimation for the Nested Error Linear Regression Model

Standard model‐based small area estimates perform poorly in presence of outliers. Sinha & Rao ( 2009 ) developed robust frequentist predictors of small area means. In this article, we present a robust Bayesian method to handle outliers in unit‐level data by extending the nested error regression model. We consider a finite mixture of normal distributions for the unit‐level error to model outliers and produce noninformative Bayes predictors of small area means. Our modelling approach generalises that of Datta & Ghosh ( 1991 ) under the normality assumption. Application of our method to a data set which is suspected to contain an outlier confirms this suspicion, correctly identifies the suspected outlier and produces robust predictors and posterior standard deviations of the small area means. Evaluation of several procedures including the M‐quantile method of Chambers & Tzavidis ( 2006 ) via simulations shows that our proposed method is as good as other procedures in terms of bias, variability and coverage probability of confidence and credible intervals when there are no outliers. In the presence of outliers, while our method and Sinha–Rao method perform similarly, they improve over the other methods. This superior performance of our procedure shows its dual (Bayes and frequentist) dominance, which should make it attractive to all practitioners, Bayesians and frequentists, of small area estimation.     

Bayesian regression with B‐splines under combinations of shape constraints and smoothness properties

In this paper, we approach the problem of shape constrained regression from a Bayesian perspective. A B‐splines basis is used to model the regression function. The smoothness of the regression function is controlled by the order of the B‐splines, and the shape is controlled by the shape of an associated control polygon. Controlling the shape of the control polygon reduces to some inequality constraints on the spline coefficients. Our approach enables us to take into account combinations of shape constraints and to localize each shape constraint on a given interval. The performance of our method is investigated through a simulation study. Applications to a real data sets in food industry and Global Warming are provided.     

Break Date Estimation for Models with Deterministic Structural Change

In this article, we consider estimating the timing of a break in level and/or trend when the order of integration and autocorrelation properties of the data are unknown. For stationary innovations, break point estimation is commonly performed by minimizing the sum of squared residuals across all candidate break points, using a regression of the levels of the series on the assumed deterministic components. For unit root processes, the obvious modification is to use a first differenced version of the regression, while a further alternative in a stationary autoregressive setting is to consider a GLS‐type quasi‐differenced regression. Given uncertainty over which of these approaches to adopt in practice, we develop a hybrid break fraction estimator that selects from the levels‐based estimator, the first‐difference‐based estimator, and a range of quasi‐difference‐based estimators, according to which achieves the global minimum sum of squared residuals. We establish the asymptotic properties of the estimators considered, and compare their performance in practically relevant sample sizes using simulation. We find that the new hybrid estimator has desirable asymptotic properties and performs very well in finite samples, providing a reliable approach to break date estimation without requiring decisions to be made regarding the autocorrelation properties of the data.     

Quasi-maximum likelihood estimation of volatility with high frequency data

This paper investigates the properties of the well-known maximum likelihood estimator in the presence of stochastic volatility and market microstructure noise, by extending the classic asymptotic results of quasi-maximum likelihood estimation. When trying to estimate the integrated volatility and the variance of noise, this parametric approach remains consistent, efficient and robust as a quasi-estimator under misspecified assumptions. Moreover, it shares the model-free feature with nonparametric alternatives, for instance realized kernels, while being advantageous over them in terms of finite sample performance. In light of quadratic representation, this estimator behaves like an iterative exponential realized kernel asymptotically. Comparisons with a variety of implementations of the Tukey–Hanning₂ kernel are provided using Monte Carlo simulations, and an empirical study with the Euro/US Dollar future illustrates its application in practice.     

Textual Analysis in Real Estate

This paper incorporates text data from MLS listings into a hedonic pricing model. We show that the comments section of the MLS, which is populated by real estate agents who arguably have the most local market knowledge and know what homebuyers value, provides information that improves the performance of both in‐sample and out‐of‐sample pricing estimates. Text is found to decrease pricing error by more than 25%. Information from text is incorporated into a linear model using a tokenization approach. By doing so, the implicit prices for various words and phrases are estimated. The estimation focuses on simultaneous variable selection and estimation for linear models in the presence of a large number of variables using a penalized regression. The LASSO procedure and variants are shown to outperform least‐squares in out‐of‐sample testing. Copyright © 2016 John Wiley & Sons, Ltd.