Reflections on Fourteen Cryptic Issues Concerning the Nature of Statistical Inference*

The present paper provides the original formulation and a joint response of a group of statistically trained scientists to fourteen cryptic issues for discussion, which were handed out to the public by Professor Dr. D.R. Cox after his Bernoulli Lecture 1997 at Groningen University.     

Bayesian and Frequentist Inference for Ecological Inference: The R×C Case

In this paper we propose Bayesian and frequentist approaches to ecological inference, based on R × C contingency tables, including a covariate. The proposed Bayesian model extends the binomial-beta hierarchical model developed by K ing , R osen and T anner (1999) from the 2×2 case to the R × C case. As in the 2×2 case, the inferential procedure employs Markov chain Monte Carlo (MCMC) methods. As such, the resulting MCMC analysis is rich but computationally intensive. The frequentist approach, based on first moments rather than on the entire likelihood, provides quick inference via nonlinear least-squares, while retaining good frequentist properties. The two approaches are illustrated with simulated data, as well as with real data on voting patterns in Weimar Germany. In the final section of the paper we provide an overview of a range of alternative inferential approaches which trade-off computational intensity for statistical efficiency.     

On randomization, modeling and experimental design; a new example in an old discussion

Fisher and "Student" quarreled in the early days of statistics about the design of experiments, meant to measure the difference in yield between to breeds of corn. This discussion comes down to randomization versus model building. More than half a century has passed since, but the different views remain. In this paper the discussion is put in terms of artificial randomization and natural randomization, the latter being what remains after appropriate modeling. Also the Bayesian position is discussed. An example in terms of the old corn-breeding discussion is given, showing that a simple robust model may lead to inference and experimental design that outperforms the inference from randomized experiments by far. Finally similar possibilities are suggested in statistical auditing.     

Bayesian prior elicitation in DSGE models: Macro- vs micropriors

Bayesian approaches to the estimation of DSGE models are becoming increasingly popular. Prior knowledge is normally formalized either directly on deep parameters' values (‘microprior’) or indirectly, on macroeconomic indicators, e.g. moments of observable variables (‘macroprior’). We introduce a non-parametric macroprior which is elicited from impulse response functions and assess its performance in shaping posterior estimates. We find that using a macroprior can lead to substantially different posterior estimates. We probe into the details of our result, showing that model misspecification is likely to be responsible of that. In addition, we assess to what extent the use of macropriors is impaired by the need of calibrating some hyperparameters.     

On Regression Sampling in Statistical Auditing: Bad Answers to the Wrong Questions?

This paper criticizes the use of regression in audit samples to obtain confidence intervals for the error rate. Also the methodology to evaluate methods by simulation studies using real–life populations is criticized. This is done from a Bayesian viewpoint, which goes as far as stating that in this type of research the wrong questions are answered. A fundamental discussion on the role of model building, illustrated by the use of models in auditing, forms the centre of the paper.     

What Does Objective Mean in a Dirichlet‐multinomial Process?

The Dirichlet‐multinomial process can be seen as the generalisation of the binomial model with beta prior distribution when the number of categories is larger than two. In such a scenario, setting informative prior distributions when the number of categories is great becomes difficult, so the need for an objective approach arises. However, what does objective mean in the Dirichlet‐multinomial process? To deal with this question, we study the sensitivity of the posterior distribution to the choice of an objective Dirichlet prior from those presented in the available literature. We illustrate the impact of the selection of the prior distribution in several scenarios and discuss the most sensible ones.     

Forecasting an accumulated series based on partial accumulation II: A new Bayesian method for short series with stable seasonal patterns

The problem of forecasting a time series with only a small amount of data is addressed within a Bayesian framework. The quantity to be predicted is the accumulated value of a positive and continuous variable for which partially accumulated data are available. These conditions appear in a natural way in many situations. A simple model is proposed to describe the relationship between the partial and total values of the variable to be forecasted assuming stable seasonality, which is specified in stochastic terms. Analytical results are obtained for both the point forecast and the entire posterior predictive distribution. The proposed technique does not involve approximations. It allows the use of non-informative priors so that implementation may be automatic. The procedure works well when standard methods cannot be applied due to the reduced number of observations. It also improves on previous results published by the authors. Some real examples are included.     

Forecasting intermittent demand by hyperbolic-exponential smoothing

Croston’s method is generally viewed as being superior to exponential smoothing when the demand is intermittent, but it has the drawbacks of bias and an inability to deal with obsolescence, where the demand for an item ceases altogether. Several variants have been reported, some of which are unbiased on certain types of demand, but only one recent variant addresses the problem of obsolescence. We describe a new hybrid of Croston’s method and Bayesian inference called Hyperbolic-Exponential Smoothing, which is unbiased on non-intermittent and stochastic intermittent demand, decays hyperbolically when obsolescence occurs, and performs well in experiments.     

Bayesian inference and prediction in the single server Markovian queue

The use of stochastic models and performance measures for the analysis of real life queuing scenarios are based on the fundamental premise that parameters values are known. This is a rarity since more often than not, parameters are usually unknown and require to be estimated. This paper presents techniques for the same from Bayesian perspective. The queue we intend to deal with is the M/M/1 queuing model. Several closed form expressions on posterior inference and prediction are presented which can be readily implemented using standard spreadsheet tools. Previous work in this direction resulted in non-existence of posterior moments. A way out is suggested. Interval estimates and tests of hypothesis on performance measures are also presented.     

On some properties of Markov chain Monte Carlo simulation methods based on the particle filter

Andrieu et al. (2010) prove that Markov chain Monte Carlo samplers still converge to the correct posterior distribution of the model parameters when the likelihood estimated by the particle filter (with a finite number of particles) is used instead of the likelihood. A critical issue for performance is the choice of the number of particles. We add the following contributions. First, we provide analytically derived, practical guidelines on the optimal number of particles to use. Second, we show that a fully adapted auxiliary particle filter is unbiased and can drastically decrease computing time compared to a standard particle filter. Third, we introduce a new estimator of the likelihood based on the output of the auxiliary particle filter and use the framework of Del Moral (2004) to provide a direct proof of the unbiasedness of the estimator. Fourth, we show that the results in the article apply more generally to Markov chain Monte Carlo sampling schemes with the likelihood estimated in an unbiased manner.     

Empirical factor demands and flexible functional forms: a bayesian approach

In this paper we compare classical econometrics, calibration and Bayesian inference in the context of the empirical analysis of factor demands. Our application is based on a popular flexible functional form for the firm's cost function, namely Diewert's Generalized Leontief function, and uses the well-known Berndt and Wood 1947–1971 KLEM data on the US manufacturing sector. We illustrate how the Gibbs sampling methodology can be easily used to calibrate parameter values and elasticities on the basis of previous knowledge from alternative studies on the same data, but with different functional forms. We rely on a system of mixed non-informative diffuse priors for some key parameters and informative tight priors for others. Within the Gibbs sampler, we employ rejection sampling to incorporate parameter restrictions, which are suggested by economic theory but in general rejected by economic data. Our results show that values of those parameters that relate to non-informative priors are almost equal to the standard SUR estimates, whereas differences come out for those parameters to which we have assigned informative priors. Moreover, discrepancies can be appreciated in some crucial parameter estimates obtained with or without rejection sampling.     

Forecasting television ratings

Despite the state of flux in media today, television remains the dominant player globally for advertising spending. Since television advertising time is purchased on the basis of projected future ratings, and ad costs have skyrocketed, there is increasingly pressure to forecast television ratings accurately. The forecasting methods that have been used in the past are not generally very reliable, and many have not been validated; also, even more distressingly, none have been tested in today’s multichannel environment. In this study we compare eight different forecasting models, ranging from a naïve empirical method to a state-of-the-art Bayesian model-averaging method. Our data come from a recent time period, namely 2004-2008, in a market with over 70 channels, making the data more typical of today’s viewing environment. The simple models that are commonly used in industry do not forecast as well as any econometric models. Furthermore, time series methods are not applicable, as many programs are broadcast only once. However, we find that a relatively straightforward random effects regression model often performs as well as more sophisticated Bayesian models in out-of-sample forecasting. Finally, we demonstrate that making improvements in ratings forecasts could save the television industry between $250 and $586 million per year.     

On the forecasting performance of a small-scale DSGE model

Dynamic stochastic general equilibrium (DSGE) models have recently become standard tools for policy analysis. Nevertheless, their forecasting properties have still barely been explored. In this article, we address this problem by examining the quality of forecasts of the key U.S. economic variables: the three-month Treasury bill yield, the GDP growth rate and GDP price index inflation, from a small-size DSGE model, trivariate vector autoregression (VAR) models and the Philadelphia Fed Survey of Professional Forecasters (SPF). The ex post forecast errors are evaluated on the basis of the data from the period 1994–2006. We apply the Philadelphia Fed “Real-Time Data Set for Macroeconomists” to ensure that the data used in estimating the DSGE and VAR models was comparable to the information available to the SPF.Overall, the results are mixed. When comparing the root mean squared errors for some forecast horizons, it appears that the DSGE model outperforms the other methods in forecasting the GDP growth rate. However, this characteristic turned out to be statistically insignificant. Most of the SPF's forecasts of GDP price index inflation and the short-term interest rate are better than those from the DSGE and VAR models.