Asymmetry in unemployment rate forecast errors

Asymmetries in unemployment dynamics have been observed in the time series of a number of countries, including the United States. This paper studies asymmetries in unemployment rate forecast errors. We consider conditions under which optimal forecasts will display asymmetrically-distributed errors and how the degree of asymmetry might vary with the forecast horizon. Using data from the U.S. Survey of Professional Forecasters and the Federal Reserve Greenbook, we find substantial evidence of forecast error asymmetry, which tends to increase with the forecast horizon; we also find noteworthy differences in forecasts from these two sources. The results give insight into the abilities of professional forecasters to adapt their forecasts to asymmetry in underlying processes.     

Quantiles as optimal point forecasts

Loss functions play a central role in the theory and practice of forecasting. If the loss function is quadratic, the mean of the predictive distribution is the unique optimal point predictor. If the loss is symmetric piecewise linear, any median is an optimal point forecast. Quantiles arise as optimal point forecasts under a general class of economically relevant loss functions, which nests the asymmetric piecewise linear loss, and which we refer to as generalized piecewise linear (GPL). The level of the quantile depends on a generic asymmetry parameter which reflects the possibly distinct costs of underprediction and overprediction. Conversely, a loss function for which quantiles are optimal point forecasts is necessarily GPL. We review characterizations of this type in the work of Thomson, Saerens and Komunjer, and relate to proper scoring rules, incentive-compatible compensation schemes and quantile regression. In the empirical part of the paper, the relevance of decision theoretic guidance in the transition from a predictive distribution to a point forecast is illustrated using the Bank of England’s density forecasts of United Kingdom inflation rates, and probabilistic predictions of wind energy resources in the Pacific Northwest.     

Optimal prediction under LINLIN loss: Empirical evidence

I compare the forecasts of returns from the mean predictor (optimal under MSE), with the pseudo-optimal and optimal predictor for an asymmetric loss function under the assumption that agents have an asymmetric LINLIN loss function. The results strongly suggest not using the conditional mean predictor under conditions of asymmetry. In general, forecasts can be improved by the use of optimal predictor rather than the pseudo-optimal predictor, suggesting that the loss reduction from using the optimal predictor can actually be important for practitioners as well.     

Quantile forecast optimal combination to enhance safety stock estimation

The safety stock calculation requires a measure of the forecast error uncertainty. Such errors are usually assumed to be Gaussian iid (independently and identically distributed). However, deviations from iid lead to a deterioration in the performance of the supply chain. Recent research has shown that, contrary to theoretical approaches, empirical techniques that do not rely on the aforementioned assumptions can enhance the calculation of safety stocks. In particular, GARCH models cope with time-varying heterocedastic forecast error, and kernel density estimation does not need to rely on a determined distribution. However, if the forecast errors are time-varying heterocedastic and do not follow a determined distribution, the previous approaches are inadequate. We overcome this by proposing an optimal combination of the empirical methods that minimizes the asymmetric piecewise linear loss function, also known as the tick loss. The results show that combining quantile forecasts yields safety stocks with a lower cost. The methodology is illustrated with simulations and real data experiments for different lead times.     

Asymmetric loss in the Greenbook and the Survey of Professional Forecasters

This paper examines the forecast rationality of the Greenbook and the Survey of Professional Forecasters (SPF) under asymmetric loss functions, using the method proposed by Elliott, Komunjer, and Timmermann (2005) with a rolling window strategy. Over rolling periods, the degree and direction of the asymmetry in forecast loss functions are time-varying. While rationality under symmetric loss is often rejected, forecast rationality under asymmetric loss fails to be rejected over nearly all rolling periods. Besides, real output growth is consistently under-predicted in the 1990s, and the inflation rate is consistently over-predicted in the 1980s and 1990s. In general, inflation forecasts, especially for long horizons, exhibit greater levels of loss asymmetry in magnitude and frequency. The loss asymmetry of real output growth forecasts is more pronounced when the last revised vintage data are used than when the real-time vintage is used. All of these results hold for both the Greenbook and SPF forecasts. The results are also similar with the use of different sets of instrumental variables for estimating the asymmetric loss and testing for forecast rationality.     

Are GDP forecasts optimal? Evidence on European countries

We assess the accuracy of real GDP growth forecasts released by governments and international organizations for European countries in the years 1999–2017. We implement three testing procedures characterized by different assumptions on the forecasters’ loss functions. First, we test forecast rationality within the traditional approach based on a quadratic loss function (Mincer and Zarnowitz, 1969). Second, following Elliott, Timmermann and Komunjer (2005), we test rationality by allowing for a flexible loss function where the shape parameter driving the extent of asymmetry is unknown and estimated from the empirical distribution of forecast errors. Lastly, we implement the tests proposed by Patton and Timmermann (2007a) that hold regardless of the functional form of the loss function. We conclude that governmental forecasts are biased and not rational under a symmetric and quadratic loss function, but they are optimal under more general assumptions on the loss function. We also find that the preferences of forecasters change with the forecasting horizon: when moving from one- to two-year-ahead forecasts, the optimistic bias increases and the parameter of asymmetry in the loss function significantly increases.     

CONSTRUCTING OPTIMAL DENSITY FORECASTS FROM POINT FORECAST COMBINATIONS

Decision makers often observe point forecasts of the same variable computed, for instance, by commercial banks, IMF and the World Bank, but the econometric models used by such institutions are frequently unknown. This paper shows how to use the information available on point forecasts to compute optimal density forecasts. Our idea builds upon the combination of point forecasts under general loss functions and unknown forecast error distributions. We use real‐time data to forecast the density of US inflation. The results indicate that the proposed method materially improves the real‐time accuracy of density forecasts vis‐à‐vis those from the (unknown) individual econometric models. Copyright © 2013 John Wiley & Sons, Ltd.     

Combining forecasts: Performance and coherence

There is general agreement in many forecasting contexts that combining individual predictions leads to better final forecasts. However, the relative error reduction in a combined forecast depends upon the extent to which the component forecasts contain unique/independent information. Unfortunately, obtaining independent predictions is difficult in many situations, as these forecasts may be based on similar statistical models and/or overlapping information. The current study addresses this problem by incorporating a measure of coherence into an analytic evaluation framework so that the degree of independence between sets of forecasts can be identified easily. The framework also decomposes the performance and coherence measures in order to illustrate the underlying aspects that are responsible for error reduction. The framework is demonstrated using UK retail prices index inflation forecasts for the period 1998–2014, and implications for forecast users are discussed.     

The performance of alternative forecasting methods for SETAR models

We compare a number of methods that have been proposed in the literature for obtaining h-step ahead minimum mean square error forecasts for self-exciting threshold autoregressive (SETAR) models. These forecasts are compared to those from an AR model. The comparison of forecasting methods is made using Monte Carlo simulation. The Monte-Carlo method of calculating SETAR forecasts is generally at least as good as that of the other methods we consider. An exception is when the disturbances in the SETAR model come from a highly asymmetric distribution, when a Bootstrap method is to be preferred.An empirical application calculates multi-period forecasts from a SETAR model of US gross national product using a number of the forecasting methods. We find that whether there are improvements in forecast performance relative to a linear AR model depends on the historical epoch we select, and whether forecasts are evaluated conditional on the regime the process was in at the time the forecast was made.     

Efficient estimation of forecast uncertainty based on recent forecast errors

Multi-step-ahead forecasts of the forecast uncertainty of an individual forecaster are often based on the horizon-specific sample means of his recent squared forecast errors, where the number of past forecast errors available decreases one-to-one with the forecast horizon. In this paper, the efficiency gains from the joint estimation of forecast uncertainty for all horizons in such samples are investigated. If the forecast uncertainty is estimated by seemingly unrelated regressions, it turns out that the covariance matrix of the squared forecast errors does not have to be estimated, but simply needs to have a certain structure, which is a very useful property in small samples. Considering optimal and non-optimal forecasts, it is found that the efficiency gains can be substantial for longer horizons in small samples. The superior performance of the seemingly-unrelated-regressions approach is confirmed in several empirical applications.     

Forecasting value at risk and expected shortfall with mixed data sampling

I propose applying the Mixed Data Sampling (MIDAS) framework to forecast Value at Risk (VaR) and Expected shortfall (ES). The new methods exploit the serial dependence on short-horizon returns to directly forecast the tail dynamics of the desired horizon. I perform a comprehensive comparison of out-of-sample VaR and ES forecasts with established models for a wide range of financial assets and backtests. The MIDAS-based models significantly outperform traditional GARCH-based forecasts and alternative conditional quantile specifications, especially in terms of multi-day forecast horizons. My analysis advocates models that feature asymmetric conditional quantiles and the use of the Asymmetric Laplace density to jointly estimate VaR and ES.     

A hybrid system-identification method for forecasting telecommunications product demands

A crucial challenge for telecommunications companies is how to forecast changes in demand for specific products over the next 6 to 18 months—the length of a typical short-range capacity-planning and capital-budgeting planning horizon. The problem is especially acute when only short histories of product sales are available. This paper presents a new two-level approach to forecasting demand from short-term data. The lower of the two levels consists of adaptive system-identification algorithms borrowed from signal processing, especially, Hidden Markov Model (HMM) methods [Hidden Markov Models: Estimation and Control (1995) Springer Verlag]. Although they have primarily been used in engineering applications such as automated speech recognition and seismic data processing, HMM techniques also appear to be very promising for predicting probabilities of individual customer behaviors from relatively short samples of recent product-purchasing histories. The upper level of our approach applies a classification tree algorithm to combine information from the lower-level forecasting algorithms. In contrast to other forecast-combination algorithms, such as weighted averaging or Bayesian aggregation formulas, the classification tree approach exploits high-order interactions among error patterns from different predictive systems. It creates a hybrid, forecasting algorithm that out-performs any of the individual algorithms on which it is based. This tree-based approach to hybridizing forecasts provides a new, general way to combine and improve individual forecasts, whether or not they are based on HMM algorithms. The paper concludes with the results of validation tests. These show the power of HMM methods to forecast what individual customers are likely to do next. They also show the gain from classification tree post-processing of the predictions from lower-level forecasts. In essence, these techniques enhance the limited techniques available for new product forecasting.     

Content horizons for univariate time-series forecasts

This paper investigates the maximum horizon at which conditioning information can be shown to have value for univariate time series forecasts. In particular, we consider the problem of determining the horizon beyond which forecasts from univariate time series models of stationary processes add nothing to the forecast implicit in the unconditional mean. We refer to this as the content horizon for forecasts, and provide a formal definition of the corresponding forecast content function at horizons s=1,… S. This function depends upon parameter estimation uncertainty as well as on autocorrelation structure of the process. We show that for autoregressive processes it is possible to give an asymptotic expression for the forecast content function, and show by simulation that the expression gives a good approximation even at modest sample sizes. The results are applied to the growth rate of GDP and to inflation, using US and Canadian data.     

A note on forecasting the prices of gold and silver: Asymmetric loss and forecast rationality

Given that the prices of gold and silver have witnessed large and substantial swings in recent years, policymakers and investors need readily available and reliable forecasts of the prices of these two precious metals. Survey data of forecasts of the prices of gold and silver provide a particularly rich data environment for policymakers and investors to study developments in the markets for gold and silver. Our research helps to develop a deeper understanding of the properties of survey data of the prices of gold and silver. We study the shape of forecasters’ loss function and the rationality of their forecasts. Assuming an asymmetric loss function weakens evidence against forecast rationality, but results depend on the empirical model being studied.     

On economic evaluation of directional forecasts

It is commonly accepted that information is helpful if it can be exploited to improve a decision making process. In economics, decisions are often based on forecasts of the upward or downward movements of the variable of interest. We point out that directional forecasts can provide a useful framework for assessing the economic forecast value when loss functions (or success measures) are properly formulated to account for the realized signs and realized magnitudes of directional movements. We discuss a general approach to (directional) forecast evaluation which is based on the loss function proposed by Granger, Pesaran and Skouras. It is simple to implement and provides an economically interpretable loss/success functional framework. We show that, in addition, this loss function is more robust to outlying forecasts than traditional loss functions. As such, the measure of the directional forecast value is a readily available complement to the commonly used squared error loss criterion.     

Efficient forecast tests for conditional policy forecasts

Central Banks regularly make forecasts, such as the Fed’s Greenbook forecast, that are conditioned on hypothetical paths for the policy interest rate. While there are good public policy reasons to evaluate the quality of such forecasts, up until now, the most common approach has been to ignore their conditional nature and apply standard forecast efficiency tests. In this paper we derive tests for the efficiency of conditional forecasts. Intuitively, these tests involve implicit estimates of the degree to which the conditioning path is counterfactual and the magnitude of the policy feedback over the forecast horizon. We apply the tests to the Greenbook forecast and the Bank of England’s inflation report forecast, finding some evidence of forecast inefficiency. Nonetheless, we argue that the conditional nature of the forecasts made by central banks represents a substantial impediment to the analysis of their quality—stronger assumptions are needed and forecast inefficiency may go undetected for longer than would be the case if central banks were instead to report unconditional forecasts.     

Election forecasting: Too far out?

We consider two criteria for evaluating election forecasts: accuracy (precision) and lead (distance from the event), specifically the trade-off between the two in poll-based forecasts. We evaluate how much “lead” still allows prediction of the election outcome. How much further back can we go, supposing we tolerate a little more error? Our analysis offers estimates of the “optimal” lead time for election forecasts, based on a dataset of over 26,000 vote intention polls from 338 elections in 44 countries between 1942 and 2014. We find that optimization of a forecast is possible, and typically occurs two to three months before the election, but can be influenced by the arrangement of political institutions. To demonstrate how our optimization guidelines perform in practice, we consider recent elections in the UK, the US, and France.     

A comparison of the forecasting performance of WEFA and ARIMA time series methods

This paper examines the forecasting performance of the Wharton model (MARK III) over the period 1973 through 1975 and compares it with that of ARIMA models' performance over the same period. Despite strong intimation in the literature to the contrary, we find that this econometric model, at least, exhibits greater accuracy in every respect relative to ARIMA methods, in terms of its forecasts cum constant adjustments. When constant adjustments are disallowed then its forecasts are still more accurate than ARIMA forecasts over a 4- and 8-quarter forecasting horizon, but less accurate over a 1-quarter horizon. The comparison was carried out over twenty three macrovariables, under a slight handicap for the Wharton Model, in that the latter's parameters were estimated over a sample ending in 1969.3 while the ARIMA models were reidentified and reestimated as of the quarter immediately preceding the forecast.     

Asymmetric loss functions and the rationality of expected stock returns

We combine the innovative approaches of Elliott, Komunjer, and Timmermann (2005) and Patton and Timmermann (2007) with a block bootstrap to analyze whether asymmetric loss functions can rationalize the S&P 500 return expectations of individual forecasters from the Livingston Surveys. Although the rationality of these forecasts has often been rejected, earlier studies have relied on the assumption that positive and negative forecast errors of identical magnitudes are equally important to forecasters. Allowing for homogenous asymmetric loss, our evidence still strongly rejects forecast rationality. However, if we allow for variation in asymmetric loss functions across forecasters, not only do we find significant differences in preferences, but also we can often no longer reject forecast rationality. Our conclusions raise serious doubts about the homogeneous expectations assumption often made in asset pricing, portfolio construction and corporate finance models.     

UNCERTAINTY AND DENSITY FORECASTS OF ARMA MODELS: COMPARISON OF ASYMPTOTIC,BAYESIAN, AND BOOTSTRAP PROCEDURES

The objective of this paper is to analyze the effects of uncertainty on density forecasts of stationary linear univariate ARMA models. We consider three specific sources of uncertainty: parameter estimation, error distribution, and lag order. Depending on the estimation sample size and the forecast horizon, each of these sources may have different effects. We consider asymptotic, Bayesian, and bootstrap procedures proposed to deal with uncertainty and compare their finite sample properties. The results are illustrated constructing fan charts for UK inflation.