On the backfitting algorithm for additive regression models

We analyse additive regression model fitting via the backfitting algorithm. We show that in the case of a large class of curve estimators, which includes regressograms, simple step-by-step formulae can be given for the back-fitting algorithm. The result of each cycle of the algorithm may be represented succinctly in terms of a sequence of d projections in n-dimensional space, where d is the number of design coordinates and n is sample size. It follows from our formulae that the limit of the algorithm is simply the projection of the data onto that vector space which is orthogonal to the space of all n-vectors fixed by each of the projections. The formulae also provide the convergence rate of the algorithm, the variance of the backfitting estimator, consistency of the estimator, and the relationship of the estimator to that obtained by directly minimizing mean squared distance.     

On the comprehensive method of testing non-nested regression models

The paper derives the specific form of the exponentially combined likelihood function of two competing multivariate non-linear regression models and shows that the application of the comprehensive approach to testing non-nested regression models will, in general, be indeterminate. It establishes that in the univariate case there exists a large number of tests of non-nested regression models which are consistent in addition to having the same asymptotic distribution under the null hypothesis. The paper then derives a set of conditions under which all these consistent tests are asymptotically equivalent not only under the null hypothesis but also under local alternatives. As an application of this latter result the paper establishes the asymptotic equivalence of the tests recently proposed by Davidson and MacKinnon, and Fisher and McAleer under local alternatives, and shows that within the class of tests considered in the paper these proposed tests possess maximum local power. The latter test has this property only when the number of explanatory variables of the ‘true’ model is not more than that of the ‘false’ model.     

On the properties of the coefficient of determination in regression models with infinite variance variables

We examine the asymptotic properties of the coefficient of determination, R²$R^2$, in models with α-stable$\alpha \text{-stable}$   random variables. If the regressor and error term share the same index of stability α<2$\alpha <2$, we show that the R²$R^2$  statistic does not converge to a constant but has a nondegenerate distribution on the entire [0,1]$[0,1]$ interval. We provide closed-form expressions for the cumulative distribution function and probability density function of this limit random variable, and we show that the density function is unbounded at 0 and 1. If the indices of stability of the regressor and error term are unequal, we show that the coefficient of determination converges in probability to either 0 or 1, depending on which variable has the smaller index of stability, irrespective of the value of the slope coefficient. In an empirical application, we revisit the Fama and MacBeth (1973) two-stage regression and demonstrate that in the infinite-variance case the R²$R^2$  statistic of the second-stage regression converges to 0 in probability even if the slope coefficient is nonzero. We deduce that a small value of the R²$R^2$  statistic should not, in itself, be used to reject the usefulness of a regression model.     

Testing the constancy in varying-coefficient regression models

Two hypothesis testing problems are considered in this paper to check the constancy of the coefficients in the varying-coefficient regression model. Tests for the two corresponding hypothesis testing problems are derived by two p-values. The proposed p-values can be thought as the generalized p-values, which are given by linear interpolation based on fiducial method. When all of the coefficients are constants, the p-value is uniformly distributed on interval (0, 1). Furthermore, the bound of the difference between the cumulative distribution function of the p-value and the uniform distribution on (0, 1) is given, which tends to 0 under some conditions. Meanwhile, the proposed tests are proved to be consistent under mild conditions. In addition, the proposed new method could be extended to include a broader range of hypotheses. Some good finite sample performances of the tests are investigated by simulations, in which a comparison with other test is given. Finally, a simple example based on real data is given to illustrate the application of our test, different result was obtained based on the proposed test.     

Multimodality in GARCH regression models

It is shown empirically that mixed autoregressive moving average regression models with generalized autoregressive conditional heteroskedasticity (Reg-ARMA-GARCH models) can have multimodality in the likelihood that is caused by a dummy variable in the conditional mean. Maximum likelihood estimates at the local and global modes are investigated and turn out to be qualitatively different, leading to different model-based forecast intervals. In the simpler GARCH(p,q) regression model, we derive analytical conditions for bimodality of the corresponding likelihood. In that case, the likelihood is symmetrical around a local minimum. We propose a solution to avoid this bimodality.     

Time-varying sparsity in dynamic regression models

A novel Bayesian method for inference in dynamic regression models is proposed where both the values of the regression coefficients and the importance of the variables are allowed to change over time. We focus on forecasting and so the parsimony of the model is important for good performance. A prior is developed which allows the shrinkage of the regression coefficients to suitably change over time and an efficient Markov chain Monte Carlo method for posterior inference is described. The new method is applied to two forecasting problems in econometrics: equity premium prediction and inflation forecasting. The results show that this method outperforms current competing Bayesian methods.     

Reduced rank regression in cointegrated models

The coefficient matrix of a cointegrated first-order autoregression is estimated by reduced rank regression (RRR), depending on the larger canonical correlations and vectors of the first difference of the observed series and the lagged variables. In a suitable coordinate system the components of the least-squares (LS) estimator associated with the lagged nonstationary variables are of order 1/T, where T is the sample size, and are asymptotically functionals of a Brownian motion process; the components associated with the lagged stationary variables are of the order T^−1/2 and are asymptotically normal. The components of the RRR estimator associated with the stationary part are asymptotically the same as for the LS estimator. Some components of the RRR estimator associated with nonstationary regressors have zero error to order 1/T and the other components have a more concentrated distribution than the corresponding components of the LS estimator.     

On tests concerning coefficient matrices of linear restriction with normal regression models

Summary As an application of tests of general linear hypotheses methods are presented for testing the equality of coefficient matrices of linear restrictions with normal univariate and multivariate regression models. Geometrical interpretations of the results are given. The present paper generalizes some of the earlier results obtained byTocher, andBennett.     

Inference in regression models with many regressors

We investigate the behavior of various standard and modified F$F$, likelihood ratio (LR$LR$), and Lagrange multiplier (LM$LM$) tests in linear homoskedastic regressions, adapting an alternative asymptotic framework in which the number of regressors and possibly restrictions grows proportionately to the sample size. When the restrictions are not numerous, the rescaled classical test statistics are asymptotically chi-squared, irrespective of whether there are many or few regressors. However, when the restrictions are numerous, standard asymptotic versions of classical tests are invalid. We propose and analyze asymptotically valid versions of the classical tests, including those that are robust to the numerosity of regressors and restrictions. The local power of all asymptotically valid tests under consideration turns out to be equal. The “exact” F$F$ test that appeals to critical values of the F$F$ distribution is also asymptotically valid and robust to the numerosity of regressors and restrictions.     

Testing fuzzy linear hypotheses in linear regression models

In this paper statistical tests with fuzzily formulated hypotheses are discussed, i.e., hypotheses H₀ and H₁ are fuzzy sets. The classical criteria of the errors of type I and type II are generalized, and this approach is applied to the linear hypothesis in the linear regression model. A sufficient condition to control both generalized criteria simultaneously is presented even in case of testing H₀ against the omnibus alternative H₁:¬ H₀. This is completely different from the classical case of testing crisp complementary hypotheses.     

On the flexible functional forms

This note proposes a general structure of the so-called flexible functional forms able to describe direct utility functions. It is obtained by solving the functional equation:

Selecting local models in multiple regression by maximizing power

This paper considers multiple regression procedures for analyzing the relationship between a response variable and a vector of d covariates in a nonparametric setting where tuning parameters need to be selected. We introduce an approach which handles the dilemma that with high dimensional data the sparsity of data in regions of the sample space makes estimation of nonparametric curves and surfaces virtually impossible. This is accomplished by abandoning the goal of trying to estimate true underlying curves and instead estimating measures of dependence that can determine important relationships between variables. These dependence measures are based on local parametric fits on subsets of the covariate space that vary in both dimension and size within each dimension. The subset which maximizes a signal to noise ratio is chosen, where the signal is a local estimate of a dependence parameter which depends on the subset dimension and size, and the noise is an estimate of the standard error (SE) of the estimated signal. This approach of choosing the window size to maximize a signal to noise ratio lifts the curse of dimensionality because for regions with sparsity of data the SE is very large. It corresponds to asymptotically maximizing the probability of correctly finding nonspurious relationships between covariates and a response or, more precisely, maximizing asymptotic power among a class of asymptotic level αt-tests indexed by subsets of the covariate space. Subsets that achieve this goal are called features. We investigate the properties of specific procedures based on the preceding ideas using asymptotic theory and Monte Carlo simulations and find that within a selected dimension, the volume of the optimally selected subset does not tend to zero as n → ∞ unless the volume of the subset of the covariate space where the response depends on the covariate vector tends to zero.     

Estimating regression models of finite but unknown order

This paper considers some problems associated with estimation and inference in the normal linear regression model

$\text{y}{}_{\mathrm{t}}\text{=}\text{∑}\text{j=1}\text{m0}\text{β}{}_{\mathrm{j}}\text{x}{}_{\mathrm{tj}}\text{+ε}{}_{\mathrm{t}}\text{,}\text{var}\text{(ε}{}_{\mathrm{t}}\text{)=σ}{}^2$

, when $\text{m}{}_0$ is unknown. The regressors are taken to be stochastic and assumed to satisfy V. Grenander's (1954) conditions almost surely. It is further supposed that estimation and inference are undertaken in the usual way, conditional on a value of $\text{m}{}_0$ chosen to minimize the estimation criterion function

$\text{EC(m, T)=}\text{σ}\text{?}{}^2{}_{\mathrm{m}}\text{+ mg(T)}$

, with respect to m, where σ&#x0302;²_m is the maximum likelihood estimate of $\text{σ}{}^2$. It is shown that, subject to weak side conditions, if $\text{g(T)}\text{→}\text{a.s.}\text{0}$ and $\text{Tg(T)}\text{→}\text{a.s.}\text{∞}$ then this estimate is weakly consistent. It follows that estimates conditional on the chosen value of $\text{m}{}_0$ are asymptotically efficient, and inference undertaken in the usual way is justified in large samples. When $\text{g(T)}$ converges to a positive constant with probability one, then in large samples $\text{m}{}_0$ will never be chosen too small, but the probability of choosing $\text{m}{}_0$ too large remains positive.The results of the paper are stronger than similar ones [R. Shibata (1976), R.J. Bhansali and D.Y. Downham (1977)] in that a known upper bound on $\text{m}{}_0$ is not assumed. The strengthening is made possible by the assumptions of strictly exogenous regressors and normally distributed disturbances. The main results are used to show that if the model selection criteria of H. Akaike (1974), T. Amemiya (1980), C.L. Mallows (1973) or E. Parzen (1979) are used to choose $\text{m}{}_0$ in (1), then in the limit the probability of choosing $\text{m}{}_0$ too large is at least 0.2883. The approach taken by G. Schwarz (1978) leads to a consistent estimator of $\text{m}{}_0$, however. These results are illustrated in a small sampling experiment.     

Non-parametric maximum likelihood estimation of censored regression models

This paper presents a consistent estimator of a censored linear regression model which does not require knowledge of the distribution of the error term. The estimator considered here applies Duncan's (1982) suggestion that the likelihood function for the censored regression model be treated as a functional of both the unknown regression vector and the unknown error distribution. Our estimator is the majorizing regression vector for this non-parametric likelihood functional. We find conditions which ensure the consistency of the NPMLE. The paper concludes with the results of Monte Carlo experiments which show the NPMLE to be more efficient than Powell's Least Absolute Deviations (LAD) estimator, particularly when the fraction of censored observations is large and the sample size is small.     

Bootstrap J tests of nonnested linear regression models

The J test for nonnested regression models often overrejects very severely as an asymptotic test. We provide a theoretical analysis which explains why and when it performs badly. This analysis implies that, except in certain extreme cases, the J test will perform very well when bootstrapped. Using several methods to speed up the simulations, we obtain extremely accurate Monte Carlo results on the finite-sample performance of the bootstrapped J test. These results fully support the predictions of our theoretical analysis, even in contexts where the analysis is not strictly applicable.