Stick-breaking autoregressive processes

This paper considers the problem of defining a time-dependent nonparametric prior for use in Bayesian nonparametric modelling of time series. A recursive construction allows the definition of priors whose marginals have a general stick-breaking form. The processes with Poisson-Dirichlet and Dirichlet process marginals are investigated in some detail. We develop a general conditional Markov Chain Monte Carlo (MCMC) method for inference in the wide subclass of these models where the parameters of the marginal stick-breaking process are nondecreasing sequences. We derive a generalised Pólya urn scheme type representation of the Dirichlet process construction, which allows us to develop a marginal MCMC method for this case. We apply the proposed methods to financial data to develop a semi-parametric stochastic volatility model with a time-varying nonparametric returns distribution. Finally, we present two examples concerning the analysis of regional GDP and its growth.     

Bayesian semiparametric multivariate GARCH modeling

This paper proposes a Bayesian nonparametric modeling approach for the return distribution in multivariate GARCH models. In contrast to the parametric literature the return distribution can display general forms of asymmetry and thick tails. An infinite mixture of multivariate normals is given a flexible Dirichlet process prior. The GARCH functional form enters into each of the components of this mixture. We discuss conjugate methods that allow for scale mixtures and nonconjugate methods which provide mixing over both the location and scale of the normal components. MCMC methods are introduced for posterior simulation and computation of the predictive density. Bayes factors and density forecasts with comparisons to GARCH models with Student-t$t$ innovations demonstrate the gains from our flexible modeling approach.     

Beta-product dependent Pitman–Yor processes for Bayesian inference

Multiple time series data may exhibit clustering over time and the clustering effect may change across different series. This paper is motivated by the Bayesian non-parametric modelling of the dependence between clustering effects in multiple time series analysis. We follow a Dirichlet process mixture approach and define a new class of multivariate dependent Pitman–Yor processes (DPY). The proposed DPY are represented in terms of vectors of stick-breaking processes which determine dependent clustering structures in the time series. We follow a hierarchical specification of the DPY base measure to account for various degrees of information pooling across the series. We discuss some theoretical properties of the DPY and use them to define Bayesian non-parametric repeated measurement and vector autoregressive models. We provide efficient Monte Carlo Markov Chain algorithms for posterior computation of the proposed models and illustrate the effectiveness of the method with a simulation study and an application to the United States and the European Union business cycle.     

Empirical Bayesball Remixed: Empirical Bayes Methods for Longitudinal Data

Empirical Bayes methods for Gaussian and binomial compound decision problems involving longitudinal data are considered. A recent convex optimization reformulation of the nonparametric maximum likelihood estimator of Kiefer and Wolfowitz (Annals of Mathematical Statistics 1956; 27 : 887–906) is employed to construct nonparametric Bayes rules for compound decisions. The methods are illustrated with an application to predict baseball batting averages, and the age profile of batting performance. An important aspect of the empirical application is the general bivariate specification of the distribution of heterogeneous location and scale effects for players that exhibits a weak positive association between location and scale attributes. Prediction of players' batting averages for 2012 based on performance in the prior decade using the proposed methods shows substantially improved performance over more naive methods with more restrictive treatment of unobserved heterogeneity. Comparisons are also made with nonparametric Bayesian methods based on Dirichlet process priors, which can be viewed as a regularized, or smoothed, version of the Kiefer–Wolfowitz method. Copyright © 2016 John Wiley & Sons, Ltd.     

A Bayesian mixed logit–probit model for multinomial choice

In this paper, we introduce a new flexible mixed model for multinomial discrete choice where the key individual- and alternative-specific parameters of interest are allowed to follow an assumption-free nonparametric density specification, while other alternative-specific coefficients are assumed to be drawn from a multivariate Normal distribution, which eliminates the independence of irrelevant alternatives assumption at the individual level. A hierarchical specification of our model allows us to break down a complex data structure into a set of submodels with the desired features that are naturally assembled in the original system. We estimate the model, using a Bayesian Markov Chain Monte Carlo technique with a multivariate Dirichlet Process (DP) prior on the coefficients with nonparametrically estimated density. We employ a “latent class” sampling algorithm, which is applicable to a general class of models, including non-conjugate DP base priors. The model is applied to supermarket choices of a panel of Houston households whose shopping behavior was observed over a 24-month period in years 2004–2005. We estimate the nonparametric density of two key variables of interest: the price of a basket of goods based on scanner data, and driving distance to the supermarket based on their respective locations. Our semi-parametric approach allows us to identify a complex multi-modal preference distribution, which distinguishes between inframarginal consumers and consumers who strongly value either lower prices or shopping convenience.     

A Bayesian non-parametric estimate for multivariate regression

The predictive approach to inference in multivariate regression problems is considered in a Bayesian non-parametric framework. A simple estimate of the regression coefficient matrix is derived under the Dirichlet process prior and then generalized by using a mixture of Dirichlet processes prior. These estimates are shown to belong to the class of linear Bayes estimates and their relation to ridge regression estimates is also exhibited. The paper ends with an illustrative application.     

Nonparametric Bayesian modelling of monotone preferences for discrete choice experiments

Discrete choice experiments are widely used to learn about the distribution of individual preferences for product attributes. Such experiments are often designed and conducted deliberately for the purpose of designing new products. There is a long-standing literature on nonparametric and Bayesian modelling of preferences for the study of consumer choice when there is a market for each product, but this work does not apply when such markets fail to exist as is the case with most product attributes. This paper takes up the common case in which attributes can be quantified and preferences over these attributes are monotone. It shows that monotonicity is the only shape constraint appropriate for a utility function in these circumstances. The paper models components of utility using a Dirichlet prior distribution and demonstrates that all monotone nondecreasing utility functions are supported by the prior. It develops a Markov chain Monte Carlo algorithm for posterior simulation that is reliable and practical given the number of attributes, choices and sample sizes characteristic of discrete choice experiments. The paper uses the algorithm to demonstrate the flexibility of the model in capturing heterogeneous preferences and applies it to a discrete choice experiment that elicits preferences for different auto insurance policies.     

Semiparametric Bayesian inference in multiple equation models

This paper outlines an approach to Bayesian semiparametric regression in multiple equation models which can be used to carry out inference in seemingly unrelated regressions or simultaneous equations models with nonparametric components. The approach treats the points on each nonparametric regression line as unknown parameters and uses a prior on the degree of smoothness of each line to ensure valid posterior inference despite the fact that the number of parameters is greater than the number of observations. We develop an empirical Bayesian approach that allows us to estimate the prior smoothing hyperparameters from the data. An advantage of our semiparametric model is that it is written as a seemingly unrelated regressions model with independent normal–Wishart prior. Since this model is a common one, textbook results for posterior inference, model comparison, prediction and posterior computation are immediately available. We use this model in an application involving a two‐equation structural model drawn from the labour and returns to schooling literatures. Copyright © 2005 John Wiley & Sons, Ltd.     

On model selection in Bayesian regression

We discuss the problem of constructing a suitable regression model from a nonparametric Bayesian viewpoint. For this purpose, we consider the case when the error terms have symmetric and unimodal densities. By the Khintchine and Shepp theorem, the density of response variable can be written as a scale mixture of uniform densities. The mixing distribution is assumed to have a Dirichlet process prior. We further consider appropriate prior distributions for other parameters as the components of the predictive device. Among the possible submodels, we select the one which has the highest posterior probability. An example is given to illustrate the approach.     

Nonparametric Bayes inference for concave distribution functions

Bayesian inference for concave distribution functions is investigated. This is made by transforming a mixture of Dirichlet processes on the space of distribution functions to the space of concave distribution functions. We give a method for sampling from the posterior distribution using a Pólya urn scheme in combination with a Markov chain Monte Carlo algorithm. The methods are extended to estimation of concave distribution functions for incompletely observed data.     

Bayesian parametric and semiparametric factor models for large realized covariance matrices

This paper introduces a new factor structure suitable for modeling large realized covariance matrices with full likelihood‐based estimation. Parametric and nonparametric versions are introduced. Because of the computational advantages of our approach, we can model the factor nonparametrically as a Dirichlet process mixture or as an infinite hidden Markov mixture, which leads to an infinite mixture of inverse‐Wishart distributions. Applications to 10 assets and 60 assets show that the models perform well. By exploiting parallel computing the models can be estimated in a matter of a few minutes.     

Alternative econometric implementations of multi-factor models of the U.S. financial markets

This paper analyzes the empirical performance of two alternative ways in which multi-factor models with time-varying risk exposures and premia may be estimated. The first method echoes the seminal two-pass approach introduced by Fama and MacBeth (1973). The second approach is based on a Bayesian latent mixture model with breaks in risk exposures and idiosyncratic volatility. Our application to monthly, 1980–2010 U.S. data on stock, bond, and publicly traded real estate returns shows that the classical, two-stage approach that relies on a nonparametric, rolling window estimation of time-varying betas yields results that are unreasonable. There is evidence that most portfolios of stocks, bonds, and REITs have been grossly over-priced. On the contrary, the Bayesian approach yields sensible results and a few factor risk premia are precisely estimated with a plausible sign. Predictive log-likelihood scores indicate that discrete breaks in both risk exposures and variances are required to fit the data.     

Statistical Decision Problems and Bayesian Nonparametric Methods

This paper considers parametric statistical decision problems conducted within a Bayesian nonparametric context. Our work was motivated by the realisation that typical parametric model selection procedures are essentially incoherent. We argue that one solution to this problem is to use a flexible enough model in the first place, a model that will not be checked no matter what data arrive. Ideally, one would use a nonparametric model to describe all the uncertainty about the density function generating the data. However, parametric models are the preferred choice for many statisticians, despite the incoherence involved in model checking, incoherence that is quite often ignored for pragmatic reasons. In this paper we show how coherent parametric inference can be carried out via decision theory and Bayesian nonparametrics. None of the ingredients discussed here are new, but our main point only becomes evident when one sees all priors—even parametric ones—as measures on sets of densities as opposed to measures on finite-dimensional parameter spaces.     

Bayesian semiparametric stochastic volatility modeling

This paper extends the existing fully parametric Bayesian literature on stochastic volatility to allow for more general return distributions. Instead of specifying a particular distribution for the return innovation, nonparametric Bayesian methods are used to flexibly model the skewness and kurtosis of the distribution while the dynamics of volatility continue to be modeled with a parametric structure. Our semiparametric Bayesian approach provides a full characterization of parametric and distributional uncertainty. A Markov chain Monte Carlo sampling approach to estimation is presented with theoretical and computational issues for simulation from the posterior predictive distributions. An empirical example compares the new model to standard parametric stochastic volatility models.     

Bayesian approach to LR assessment in case of rare type match

The likelihood ratio (LR) is largely used to evaluate the relative weight of forensic data regarding two hypotheses, and for its assessment, Bayesian methods are widespread in the forensic field. However, the Bayesian ‘recipe’ for the LR presented in most of the literature consists of plugging‐in Bayesian estimates of the involved nuisance parameters into a frequentist‐defined LR: frequentist and Bayesian methods are thus mixed, giving rise to solutions obtained by hybrid reasoning. This paper provides the derivation of a proper Bayesian approach to assess LRs for the ‘rare type match problem’, the situation in which the expert wants to evaluate a match between the DNA profile of a suspect and that of a trace from the crime scene, and this profile has never been observed before in the database of reference. LR assessment using the two most popular Bayesian models (beta‐binomial and Dirichlet‐multinomial) is discussed and compared with corresponding plug‐in versions.     

A semi-parametric Bayesian approach to the instrumental variable problem

We develop a Bayesian semi-parametric approach to the instrumental variable problem. We assume linear structural and reduced form equations, but model the error distributions non-parametrically. A Dirichlet process prior is used for the joint distribution of structural and instrumental variable equations errors. Our implementation of the Dirichlet process prior uses a normal distribution as a base model. It can therefore be interpreted as modeling the unknown joint distribution with a mixture of normal distributions with a variable number of mixture components. We demonstrate that this procedure is both feasible and sensible using actual and simulated data. Sampling experiments compare inferences from the non-parametric Bayesian procedure with those based on procedures from the recent literature on weak instrument asymptotics. When errors are non-normal, our procedure is more efficient than standard Bayesian or classical methods.     

Additive cubic spline regression with Dirichlet process mixture errors

The goal of this article is to develop a flexible Bayesian analysis of regression models for continuous and categorical outcomes. In the models we study, covariate (or regression) effects are modeled additively by cubic splines, and the error distribution (that of the latent outcomes in the case of categorical data) is modeled as a Dirichlet process mixture. We employ a relatively unexplored but attractive basis in which the spline coefficients are the unknown function ordinates at the knots. We exploit this feature to develop a proper prior distribution on the coefficients that involves the first and second differences of the ordinates, quantities about which one may have prior knowledge. We also discuss the problem of comparing models with different numbers of knots or different error distributions through marginal likelihoods and Bayes factors which are computed within the framework of Chib (1995) as extended to DPM models by Basu and Chib (2003). The techniques are illustrated with simulated and real data.     

Bayesian clustering of distributions in stochastic frontier analysis

In stochastic frontier analysis, firm-specific efficiencies and their distribution are often main variables of interest. If firms fall into several groups, it is natural to allow each group to have its own distribution. This paper considers a method for nonparametrically modelling these distributions using Dirichlet processes. A common problem when applying nonparametric methods to grouped data is small sample sizes for some groups which can lead to poor inference. Methods that allow dependence between each group’s distribution are one set of solutions. The proposed model clusters the groups and assumes that the unknown distribution for each group in a cluster are the same. These clusters are inferred from the data. Markov chain Monte Carlo methods are necessary for model-fitting and efficient methods are described. The model is illustrated on a cost frontier application to US hospitals.     

Nonparametric function estimation subject to monotonicity, convexity and other shape constraints

This paper uses free-knot and fixed-knot regression splines in a Bayesian context to develop methods for the nonparametric estimation of functions subject to shape constraints in models with log-concave likelihood functions. The shape constraints we consider include monotonicity, convexity and functions with a single minimum. A computationally efficient MCMC sampling algorithm is developed that converges faster than previous methods for non-Gaussian models. Simulation results indicate the monotonically constrained function estimates have good small sample properties relative to (i) unconstrained function estimates, and (ii) function estimates obtained from other constrained estimation methods when such methods exist. Also, asymptotic results show the methodology provides consistent estimates for a large class of smooth functions. Two detailed illustrations exemplify the ideas.     

Some Problems with the Ferrier/Hirschberg Bootstrap Idea

This paper demonstrates that the bootstrap procedure suggested by Ferrier and Hirschberg (1997) gives inconsistent estimates. A very simple example is given to illustrate the statistical issues underlying nonparametric efficiency measurement and the problems with the Ferrier/Hirschberg approach, and may serve as a primer on bootstrapping in nonparametric models of production processes.